Kerberos and load balancers

Kerberos is a ticket-based authentication protocol, which requires deep understanding to make it work properly. Information about setting it up is somehow scattered over the web, so it took me some time to find relevant reads. In my previous post I wrote about my initial experience with Kerberos and I want to share some more in this one.

Working in a load-balanced environment introduces (administrative) complexity to your system and Kerberos is no exception to this. In this post I will focus on setting your IIS (>= 7.0) correctly up to work with Kerberos in such scenarios.

Application pool account

By default when you create a new application pool it will run under ApplicationPoolIdentity – a “virtual account”, unique for the application pool.

Application pool identities have same rights as the Network Service account and will access network resources using the computer domain account, e.g., DOMAIN\machine$. Kerberos requires SPNs for the resources you want to access and in the case of multiple machines providing the same resource, you have to register a SPN for each machine domain account. Although it is perfectly possibly, it will only work in some of the cases (depending on how you have set the distribution load on the load balancer). A SPN is just a mapping of what account will be able to decrypt the Kerberos ticket for a desired resource. When you have multiple machines behind a load balancer, though, you don’t have control over which machine will receive the request. The solution is to use a common service domain account for the application pool on all machines.

Optimizing the authentication process

Kerberos is a request-based authentication protocol, which means every client request must include a Kerberos ticket and IIS have to validate it against the Active Directory. This increases the network traffic and could lead to performance issues with your application. IIS provides a configuration property authPersistNonNTLM, which can be set to change the default behavior. This property can only be set using the Configuration Editor.

Enabling authPersistNonNTLM will change the authentication from request-based to session-based, i.e. IIS will cache your credentials for all subsequent HTTP Keep-Alive requests in the same TCP session.

We enabled this property on all machines behind the load balancer, but then suddenly we began experiencing sporadic account mismatch issues in our application: the user making the request at the client was not the same as the user receiving the request in the wen application (we log every request with a unique ID, a sort of correlation ID). This caused serious security troubles, as users with valid permissions were forbidden to execute one or another function in our web application. The key here is the TCP session IIS defines – a combination of IP and port. You have to remember that all your machines work behind a load balancer, so requests have same IP and port. The result was, a user makes a request, IIS caches user’s credentials, then another user makes another request, and IIS thinks it is the same user. Unfortunately, in a load-balanced environment you cannot make use of authPersistNonNTLM.

Conclusion

The more I work with Kerberos, the more I get persuaded how much details mean. One may say there are better authentication mechanisms, and maybe he will be right, but I still believe Kerberos provides simplicity when working in an Active Directory, having one knows how to configure it properly. But hey, how many systems run without configuration? 🙂