Certificates expire – a tale of DirectAccess woe

When certificates expire, DirectAccess breaks.

Such a simple sentence, yet so powerful. While this is certainly no ground-breaking, earth-shattering piece of news, it is something that I need to remind people of far too often. Since I have been implementing and supporting DirectAccess for many years now, it is no surprise that I receive numerous calls like “My DirectAccess is down, help me now!” every year. What is the cause of these outages? It almost always boils down to an expired certificate.

There are three different places that certificates are used in a DirectAccess environment. Let’s review those three, so that you can stay on top of keeping those certificates fresh and renewed!

SSL certificate for NLS

The first certificate at play in the DirectAccess world is not even on your DA server, or at least it shouldn’t be. If you are co-hosting your NLS website on the DirectAccess server itself, you have other problems (or you will). Move that guy off to its own server! Assuming you have setup the NLS website properly and it is running on its own web server that is NOT the DirectAccess server, that web server contains an SSL certificate which is used to validate traffic coming into the NLS website. This SSL certificate might be one that you purchased from a public CA, in which case you would probably get a reminder from them annually about renewing it. However, many times the NLS certificate is one issued from your internal CA server to save costs, and if that is the case then it may not be on anybody’s radar to renew it when it expires. An expired NLS certificate will wreak havoc with your DA client computers, don’t risk it. Make sure that certificate gets updated regularly!

SSL certificate for IP-HTTPS

The second certificate in DA is also an SSL certificate, but this one does reside on the DirectAccess server. This SSL certificate should be one that you purchased from a public authority, or is maybe even a wildcard that your company holds. In either case, hopefully the CA will prompt you with some emails when the expiration date is coming. To be safe though, check your certificate store on the DirectAccess server to make sure your own calendar is marked with enough time to renew this one as well. An invalid IP-HTTPS certificate means that many of your DA clients will suddenly fail to connect.

Machine certificates for IPsec

The final place that certificates are used in a DirectAccess environment is the machine certificates issued from your internal CA server to both the DA server, as well as every single one of your DA client machines (the laptops, tablets, etc). While it is possible to get DirectAccess working without these certificates, they really enhance the security of the tunnel authentication process and so I absolutely recommend that you use these certificates on all of your DA machines. These certificates take about 3 seconds to put in place on the machines, and you can easily establish a GPO with Autoenrollment rules so that these certificates get rolled around automatically, without any administrative input from your end. Try to get an automated system in place, or this one will turn into an administrative nightmare. Many companies have used the built-in “Computer” template in Windows CA for issuing this certificate, which is completely fine, except that the default validity date is only one year. If you have 1000 DirectAccess client computers and do not have an autoenrollment policy established, this means you have to renew 3 certificates every single day year round in order to keep up. What happens when one of these expires? If the machine certificate on a DA client expires, that particular machine will not be able to connect until it gets a new copy of the certificate. This might mean that the user needs to come into the office in order to receive the new cert. But if the machine certificate that is on the DirectAccess server expires, watch out. You will have a situation where everyone will immediately stop connecting. As soon as you replace that certificate with a new one, they will immediately start working again, but in the meantime your ear will be bright red from all the phone calls.

Bottom line: Mark your calendars for certificate expiration dates, and give yourself enough time to get them done before the actual expiration happens!

Jordan Krause
Microsoft MVP | Cloud and Datacenter Management
jordan.krause@ivonetworks.com

Security policies can block DirectAccess Manage-Out traffic

Anyone who has installed DirectAccess a number of times probably knows by now that existing corporate GPO settings, particularly security settings, can wreak havoc on a DirectAccess server. If you join your DirectAccess server to the domain and let all of your existing policies apply to it, you may have effectively broken DirectAccess before you even get it configured, because DA has many moving parts and any lockdown settings can negatively affect it. The answer to this is pretty simple, when we join a DirectAccess server to the domain we pre-stage it inside an OU where GPO inheritance is blocked, making sure that the existing security policies do not get applied to the server.

I ran across an issue recently that was completely new to me, but is something Microsoft has tackled at least once before and is also related to existing security policy settings that might exist inside a network. In this case we got DirectAccess itself up and running, that process went well and we had inbound connections working all day long. The customer wanted to add manage-out capabilities to this environment, and to meet the needs of their network that was not ready for full IPv6 routing, we employed ISATAP. This is something we have done numerous times, and I configured the Selective ISATAP environment in exactly the same way that I always do. Everything setup fine, the management servers received ISATAP IPv6 addresses, the DA client computers got their firewall rules telling them what kind of traffic to allow on the inbound, and we were able to successfully ping from the management servers, outbound to the DirectAccess laptops. Success! Right?

Wrong.

Only pings would work. ICMP was the only kind of traffic that would make it through the manage-out connections. The DA clients could reach inbound properly, and the DirectAccess server could reach outbound to the clients properly as well. We could RDP successfully from the console of the DirectAccess server outbound to a DirectAccess client. But we could NOT RDP from a management server to a DA client. Only ping.

Very much testing ensued, including a scrap of ISATAP and going with a real, native IPv6 configuration inside the network which still gave us the exact same symptoms?! We then took some trace files on the DirectAccess server, and noticed a number of IPsec Security Association authentication failure messages. IPsec authentication in DirectAccess itself was working fine, because we had both Infrastructure and Intranet tunnels live on the inbound. However, it appeared that the DirectAccess server was attempting to build new Security Associations whenever we tried to perform a manage-out functions from a management server to a DA client, and these associations were failing. The DA server builds these manage-out tunnels itself, but it does so “on behalf” of the management server. This turns out to be the core of the issue. In researching this, we stumbled on an old post:

http://blogs.technet.com/b/edgeaccessblog/archive/2012/05/15/kb-directaccess-manage-out-fails-for-any-non-icmp-traffic-in-forefront-unified-access-gateway-2010.aspx

These were the exact symptoms we were experiencing, where ICMP was the only type of traffic that would make it through the manage-out traffic flow. When we checked into this “Access this computer from the network” security policy setting, it was set normally on the management servers and the DirectAccess server itself, but checking this same setting on the client computers showed us that an existing security policy inside the network was modifying these permissions on all of their client computers! We created a new policy that set the permissions for “Access this computer from the network” back to the default settings, shown in the screenshot below, and immediately our manage-out traffic from the management servers to the DirectAccess client computers started working as it should.

Here is the particular setting to watch out for, make sure these are set to the defaults!!

Computer Configuration | Windows Settings | Security Settings | Local Policies | User Rights Assignment

Access this computer from the network” must include these four: Administrators, Backup Operators, Everyone, and Users.

permissionsJordan Krause
Microsoft MVP | Enterprise Security

 

BigHand now works over DirectAccess

Implementing DirectAccess for legal industry customers has become a commonplace activity around here over the last couple of years, for good reason. DirectAccess is essentially a completely automatic VPN connection, and so using DirectAccess instead of a traditional VPN saves lawyers time, billable time, every single time that they open their computer to use it.

Following many DirectAccess installations, all of the applications your users need on a day to day basis immediately start working, whenever the user is connect to the internet. This is the exact purpose of DirectAccess and is what we strive for in every install. However, the heart of DirectAccess is IPv6 packets, and applications which do not know how to “talk IPv6” cannot connect successfully over a DirectAccess tunnel.

NOTE: IPv6 is not required inside your network for DirectAccess to work. This is a very common misconception. Almost 100% of the DirectAccess installs I have worked with or seen in the field are on networks that are completely IPv4. But even in these IPv4-only networks, DirectAccess itself still utilizes IPv6 packets between the DA client and the DA server – the portion of the communication that is happening over the internet. This is core behavior of how DirectAccess works and cannot be changed.

Since DirectAccess packets are always IPv6 when they are moving over the internet, that means your client-side applications must be capable of generating IPv6 packets in order to make use of that IPv6 DA tunnel. In our work with DirectAccess over the past number of years, we have discovered a number of mainstream applications which are not capable of creating these IPv6 packets, and therefore fail to connect when a laptop is using DirectAccess.

What we are here to talk about today is BigHand Digital Dictation.

BigHand is a very common application that we run across when working with law firms, and it is always frustrating to install a brand new DirectAccess environment, only to discover that BigHand cannot connect. The same is true for WorkSite, and often these two applications go hand-in-hand. Neither connect natively over DirectAccess, and these two pieces of software are typically very important to the way that law firms do business.

This is why we created App46.

The App46 tool created by IVO Networks is an agent that we install on the DirectAccess client computers. This agent intercepts packet streams from these non-IPv6 applications, translates those packets into IPv6, and then sends them on their way successfully across the DirectAccess tunnels. We have utilized App46 in many customer locations now in order to make both WorkSite as well as BigHand connect and work just like they do when inside the office.

If you have any questions or would like additional information about App46, as always feel free to reach out to me directly.

Jordan Krause
Microsoft MVP | Enterprise Security
jordan.krause@ivonetworks.com

For IVO Networks – Software Sales, please contact:
Eric Bettermann – eric.bettermann@ivonetworks.com
651.503.5632

Does Windows 10 support DirectAccess?

A few customers have asked me recently about something troubling that they had been told. DirectAccess isn’t supported in Windows 10?

I’m happy to report that this is entirely false!

Windows 10 absolutely contains DirectAccess and all of the components to utilize it, just like Windows 8 and 8.1. Also just like Windows 8.x, the only SKU in Windows 10 that will contain the DirectAccess components is Enterprise, so you must be running that version on your client machines in order to utilize DirectAccess.

Not much has changed in DA during the swing to Windows 10, it works seamlessly and automatically, with all of the special magic that it has always contained. I mean really, could they make it better?

I think that some of the “unsupportability” confusion may stem from two things. First, when the Windows 10 preview came out last year, there was trouble getting it to work with DirectAccess. This happened for folks who were relying on the WMI filtering in their DA environment to determine which machines received the DA connectivity settings, and which ones did not. Now, I’m not a fan of the WMI filtering feature in the first place, so I never set it up that way. It’s much better to use a group for filtering! But regardless of that, for those folks who use the WMI filtering, tweaks had to be made in that filtering in order for it to recognize Windows 10 machines so that it would give them the DirectAccess settings in the first place.

A second point of confusion is UAG. DirectAccess works in Windows 10, and your server-side infrastructure of DirectAccess doesn’t make much difference, it can be Server 2012, Server 2012 R2, or even a Server 2008 R2 running UAG – for DirectAccess. Not for the UAG web portals. If you are running UAG for DirectAccess, you can continue doing that and it’ll continue to work very well. I know many people who are continuing to use UAG instead of Server 2012 for DirectAccess, because it works well and contains a pretty stellar firewalling system to protect itself.

But for those who are using UAG for web portals, those SSLVPN style functions, that is where support is ending as Windows 10 comes out. Up until this point, whenever new operating systems or browsers have been released there is always some kind of update or service pack or patch to UAG in order to make it correctly recognize and work with the UAG web portals. This releasing of updates for UAG is now going to stop, and so Windows 10 is officially unsupported with regards to UAG web portals. This is true.

But thankfully…this has nothing to do with DirectAccess.

Jordan Krause
Microsoft MVP | Enterprise Security
jordan.krause@ivonetworks.com

DirectAccess and FRS are not friends

File Replication Services (FRS) is something that Microsoft has been trying to get customers to stop using for many years. The newer DFS-R is superior in every way, and the migration from FRS to DFS-R is now quite a simple process, everyone should be making the switch. But of course as we always see with every outdated technology, some of us still love what we know, or for whatever the reason there are still enough customers using FRS right now that Microsoft has decided to update the DirectAccess unsupported configurations article in TechNet to address problems they have seen with this coexistence:


 

File Replication Service (FRS) distribution of Group Policy objects (SYSVOL replications)

Do not deploy DirectAccess in environments where your domain controllers are running the File Replication Service (FRS) for distribution of Group Policy objects (SYSVOL replications). Deployment of DirectAccess is not supported when you use FRS.

You are using FRS if you have domain controllers that are running Windows Server 2003 or Windows Server 2003 R2. In addition, you might be using FRS if you previously used Windows 2000 Server or Windows Server 2003 domain controllers and you never migrated SYSVOL replication from FRS to Distributed File System Replication (DFS-R).

If you deploy DirectAccess with FRS SYSVOL replication, you risk the unintentional deletion of DirectAccess Group Policy objects that contain the DirectAccess server and client configuration information. If these objects are deleted, your DirectAccess deployment will suffer an outage, and client computers that use DirectAccess will not be able to connect to your network.

If you are planning to deploy DirectAccess, you must use domain controllers that are running operating systems later than Windows Server 2003 R2, and you must use DFS-R.


 

This isn’t to say that it won’t work, for a while at least, but it is now officially unsupported. I have personally installed DirectAccess into Server 2003 domain environments numerous times and have never seen this problem firsthand, but it’s possible those orgs were simply not using FRS.

Jordan Krause
Microsoft MVP | Enterprise Security
Jordan.Krause@ivonetworks.com

Microsoft Ignite 2015!

So this post is a few days later than intended, but it seems like I hit the ground running and haven’t stopped since setting foot in Chicago! The first-ever Ignite conference is going very well, with an astounding 23,000 of us in attendance! “Mobile First, Cloud First” is our motto here and while I don’t work as regularly today with Azure as I’m sure I will over the coming years, I couldn’t agree more with the “mobile first” message. Enabling users to work from wherever they are has been the focus of IVO Networks for many years, and I am super excited to soak up as much knowledge here at Ignite as possible.

I also want to put out a public congrats and recognition to my friends Idan and Ohad whose company Aorato was recently acquired by Microsoft and was mentioned in the Keynote address on Monday! If you haven’t heard of it, be sure to check out Microsoft Advanced Threat Analytics. This promises to be an amazing technology that is going to save plenty of business’s bacon.

Well, that’s it for now. Off to another session and then some back to back after-hours events to fill in the night. Never a dull moment at these conferences.

Jordan Krause
Microsoft MVP | Enterprise Security

DirectAccess advanced features disable IP-HTTPS null encryption

If you’ve ever attended one of our “DirectAccess: Best Practices” webinars, or read anything that I have written regarding the DirectAccess transition technologies, you know I am an absolute advocate of keeping Teredo enabled whenever possible. The premise for this recommendation is simple: improved performance. There are a number of different possible client/server combinations in a DirectAccess connection, and depending on what combination is used, you may or may not be running as efficiently as possible. I come across many DirectAccess implementations in the wild that were configured in a way where Teredo is disabled, and so all client connections are being made via the IP-HTTPS transition tunnel. Technically this is fine, as all clients are able to make IP-HTTPS connections, but depending on what operating systems you are running, you may be double-encrypting every packet that flows in or out of every DirectAccess client computer. This makes for slower connections on the client side, and heavier resource utilization on the server side, resulting in slower speeds and fewer users that you can squeeze onto that DA server.

To give a little background, the reason your traffic might be doubly encrypted is because all DirectAccess traffic is encrypted with IPsec all the time. In addition to that encryption, when certain operating system combinations are in effect in a DA connection, your IP-HTTPS tunnel might also be encrypting that traffic stream a second time with TLS. Think of it this way, your transition tunnel (Teredo or IP-HTTPS) is like a conduit flowing through a wall. Inside that conduit run your individual networking cables, which are akin to your DirectAccess IPsec tunnels. Your IPsec tunnels that flow inside the conduit are already encrypted, so there is no need to encrypt the conduit itself. However, that is exactly what IP-HTTPS does in many circumstances. This is the core reason why we keep Teredo enabled when possible, so that any users which can connect using Teredo do so, and lessen the encryption processing load on both the clients and the DirectAccess servers. Here are some examples of IP-HTTPS connections that will help to show in what cases you are experiencing double encryption:

Client/Server operating system combinations when connecting via IP-HTTPS:
Windows 7 / UAG DirectAccess: Yes, double encryption.
Windows 7 / 2012 DirectAccess: Yes, double encryption.
Windows 8 / UAG DirectAccess: Yes, double encryption.
Windows 8 / 2012 DirectAccess: No, only the IPsec encryption is happening here, at least in most cases.

In the event that your client computer is running Windows 8 or higher, and the DirectAccess server is running Server 2012 or higher, then we are finally in a position where the negotiations of the tunnel are smart enough to negotiate null encryption on the IP-HTTPS tunnel itself. Because all DirectAccess traffic is already secured inside IPsec, we can safely negotiate NULL, and take some of the processing load for the IP-HTTPS tunnel off the client and the server. This is great news, and as the newer operating systems roll out further and further, this will help to speed up DA connections and reduce the load on DA servers everywhere!

BUT…there’s a catch, which is the purpose of this article. If you enable some of the advanced features on your Server 2012 DirectAccess server, namely VPN or OTP, your NULL encryption negotiation option will be gone. There is nothing in the wizards or GUI which tells you this is going to happen, but as soon as you enable OTP or add the VPN role to a DirectAccess server, the ability for your IP-HTTPS tunnels to connect using NULL vanishes, and all of your IP-HTTPS connections, even those between Windows 8 and Server 2012, are always double encrypted. This happens namely because you don’t want OTP credentials which are used as part of the IPsec tunnel authorization to be sent clear-text across the internet, and if you have users connecting via SSTP VPN, same is true. You certainly wouldn’t want their VPN tunnels to be able to authenticate and pass credentials over that null stream, sending critical information via clear text across the internet.

So while NULL encryption for IP-HTTPS tunnels gets disabled for very good reasons, the fact that this happens is not advertised anywhere for the administrator to see, and so you may be caught off guard to find that you have enabled VPN only to experience higher CPU load on your DA server, or complaints from users about having slightly slower connections than they used to have.

Jordan Krause
Microsoft MVP | Enterprise Security
jordan.krause@ivonetworks.com

Windows Server 2012 R2 Administrator Cookbook – New Book!

We are happy to announce that IVO Network’s own Microsoft MVP, Jordan Krause, has had an opportunity to put together and publish a second book!

This publication is designed to help both new and experienced server administrators to better understand and utilize Microsoft Windows Server 2012 R2 in their corporate environments.

https://www.packtpub.com/networking-and-servers/windows-server-2012-r2-administrator-cookbook

Cover

Enabling new columns in Remote Access Management Console | Remote Client Status

I recently helped a customer roll out DirectAccess on a couple of load balanced Server 2012 boxes. It is pretty common to get some questions after we finish installing, but I was asked something that (surprisingly) I haven’t ever been asked before. Specifically, his question was “When I am viewing the currently connected DirectAccess computers inside the Remote Access Management Console, how can I see what DirectAccess server they are connecting through without having to go into the details of each connection?”

“There’s a column right in there that shows you” was my response. But indeed, there was not. I checked some of my own servers, and sure enough there was a Server column right there where I could identify and even sort the connections by which node they were connecting through. But this column was not showing up in his console. Then we checked his other server, and the Server field was visible on that one! But still not on the first DirectAccess server.

It seems like we poked and prodded around every inch of the interface, but couldn’t find a menu that let us choose which columns were visible and which were not. Then just as I hung up the phone with him (isn’t that how it always happens?), I accidentally clicked on it. All you have to do is right-click on one of the Titles of the existing columns, and you see a list of available columns to select or de-select. The important part is that you must right-click on the name (right on the text) of one of the columns that is already being shown. Right-clicking anywhere else in the Console will not get you this selection.

Server

So there was our answer. We simply checked the Server box in this list, and the new column was immediately available for viewing and helping us to manipulate the data as needed. Sometimes the simplest options are the hardest to find!

Jordan Krause
Microsoft MVP | Enterprise Security
jordan.krause@ivonetworks.com

Forefront TMG SP2 Rollup 5 available for download

Hotfix Rollup 5 for Microsoft Forefront TMG SP2 was recently released, and can be downloaded here:

https://support.microsoft.com/kb/2954173

This rollup includes the following hotfixes:

2963805 – Account lockout alerts are not logged after you install Rollup 4 for TMG 2010 SP2

2963811 – The TMG Firewall service (wspsrv.exe) may crash when the DiffServ filter is enabled

2963823 – “1413 Invalid Index” after you enable cookie sharing across array members

3963834 – HTTPS traffic may not be inspected when a user accesses a site

2967726 – New connections are not accepted on a specific web proxy or web listener in Threat Management Gateway 2010

2965004 – EnableSharedCookie option doesn’t work if the Forefront TMG service runs under a specific account

2932469 – An incorrect value is used for IPsec Main Mode key lifetime in Threat Management Gateway 2010

2966284 – A zero value is always returned when an average counter of the “Forefront TMG Web Proxy” object is queried from the .NET Framework

2967763 – The “Const SE_VPS_VALUE = 2″ setting does not work for users if the UPN is not associated with a real domain

2973749 – HTTP Connectivity verifiers return unexpected failures in TMG 2010