The page cannot be displayed because your server’s current configuration does not support it. To perform this task, use the command line operations in Stsadm.exe.

How odd, given the emphasis on PowerShell in SharePoint 2010! After a bit of head scratching and examining application and ULS logs, I navigated to the Central Admin home page and everything appeared to be fine, but then when I got around to creating a new Site Collection a bit later, I got the same error, even though I was able to create web/service applications. I had the same error when logged on as farm admin, farm admin + local admin rights, farm admin + SQL SysAdmin and farm admin + domain admin rights, so I was pretty sure it wasn’t a permission issue (and I should note my temporary fiddlery here is only really suitable for non-production environments). This error also occurred on some other Site Collection-specific pages.

After searching for a solution I found a number of suggestions that this was related to insufficient rights for Active Directory Account Creation Mode. So I played around with SQL permissions/accounts a bit more and was eventually able to loosen things to the point where I could create a new site using PowerShell (still no luck using Central Administration). I also (strangely), had to specify an outbound e-mail server first!?!?! ULS Viewer unveiled that mystery, as well as an error attempting to create an account for my logged on user (which obviously already exists) in Active Directory. This error didn’t prevent me from creating the site, but this behaviour confirmed that the site was definitely running in Active Directory Account Creation Mode.

What is Active Directory Account Creation Mode?

Unfortunately, current Microsoft documentation is non-existent as far as I can tell, so I’ll start with the TechNet description from WSS2:

A new feature of Microsoft Windows SharePoint Services is account creation mode for Active Directory directory service. This feature replaces the local account creation feature in SharePoint Team Services 1.0 from Microsoft. Use Active Directory account creation mode when it is necessary to create new user accounts rather than using existing domain accounts. For example, an Internet service provider (ISP) might need the ability to allow SharePoint site owners the capability to create user accounts or invite users to collaborate on a Web site where existing domain accounts for those users do not already exist.

Basically it’s a hosting mode in which SharePoint creates new users in Active Directory and these are the only accounts that can be used in this mode. I can attribute my lack of experience with this mode to my lack of experience with the free versions of SharePoint. Nearly all of my work has been focused on SPS2003, MOSS 2007 and SPS2010. I can’t pin down for certain whether this mode ever existed in the full versions of the product, but according to this article it is now SharePoint Foundation-only and greyed out for SharePoint Server 2010. This TechNet forum post from published SharePoint author and MCC John Ferringer seems to back up that assertion. This post describes the mode well and also appears to answer my next question (my italics, John’s bold):

SharePoint Foundation 2010 (the “free” version of SharePoint that is the successor to Windows SharePoint Services, or WSS) does have something called “Active Directory Account Creation” mode available, which functions much like what you saw in WSS v2. Accounts are first created in SharePoint, and then added to an Organizational Unit in Active Directory. The problem is that this mode is only available at the time you install SharePoint, (its an option off the Advanced Settings button) and you can’t change that configuration setting after the fact. Additionally, you can’t use existing AD accounts in that SharePoint farm, you’ll only be able to use accounts that you create through the tool and you can’t give an account an email address that’s already used by another account in AD. So you need to be mindful of those limitations if you chose to use that mode.

I wish I’d seen this post sooner. It would have saved me some time. For what it’s worth, my investigation backs up this description as follows:

• There is no facility to disable this mode through Central Administration.
• The STSADM commands only support identifying whether WSS is in Active Directory Account Creation Mode using stsadm.exe -o getproperty -pn createadaccounts. There is no corresponding setproperty command.
• The PSCONFIG commands only support creating the farm in this mode – there does not appear to be a means of reverting from it. I believe the configdb’s addomain and adorgunit parameters are responsible for enabling this mode (I could be wrong – the documentation is a bit scant), but I can’t find a facility for reverting it.
• PowerShell is now the preferred means of creating the farm and would be the preferred means of enabling this mode using the New-SPConfigurationDatabase command. As far as I can tell the DirectoryDomain and DirectoryOrganizationUnit parameters are responsible for enabling this mode now, although again, the documentation is unclear to me.
• I even tried to make the change through the API with the help of a friendly neighbourhood developer. Things have changed a bit from WSS2 to WSS3/SharePoint Foundation. At any rate, we found the attribute and set it to “false”, but unfortunately this did not rescue my nascent farm.

In short, it looks like there’s no way back.

How Could This Have Happened?

Given that this mode is only supposed to be present in SharePoint Foundation, I’m really at a loss to explain how I activated it on SharePoint Server 2010. I may have fat-fingered one of the New-SPConfigurationDatabase DirectoryDomain or DirectoryOrganizationUnit parameters I suppose, but I would be really surprised if I did that in a way that allowed me to successfully run the command. In the end, I reverted to my pre-installation snapshots and rebuilt my farm. I don’t feel that was such a waste now that I’ve found John Ferringer’s description and realise there never would have been a way back, but if you find this post through similar folly, hopefully you won’t waste any time trying to revert the farm like I did.

Interestingly, while researching this topic, I stumbled across a TechNet forum post from Andrew Milsark of FPWeb stating that they’ve, “moved away from using AD Account creation mode all together”.

• The acceptable placement of server roles in those topologies.
• Supported domain trust directions.
• Alternate Access Mappings requirements.
• Picking people from other domains.

This is an account of the relevant issues and the steps that we took to reach our conclusions.

The proposed topology

In this scenario we had two web front ends and an index server inside the corporate network and two additional web front end servers in the DMZ. The DMZ servers were members of an external domain and an ISA firewall separated the networks. The DMZ web front end servers would respond to external requests while the two internal servers would respond to internal requests. The DMZ servers were configured with the same service and application pool configuration as the servers on the internal domain. The SharePoint web applications ran under internal domain accounts.

Before diving in, you might wonder why someone would want to use this topology. The primary benefits are that some physical farm resources are consolidated and that the external users cannot authenticate against the internal domain; there is a 1-way domain trust, where the external domain trusts the internal domain. Internal users would hit load balanced internal domain web front end servers when they are logged on to the internal network for improved performance (removing ISA as a bottleneck).

This topology was proposed based on the planning guidance linked above. While the topology diagrams only detail web front end servers in the DMZ, this paragraph suggests that additional web front end servers can reside in the internal domain:

You can place one or more Web servers inside the corporate network to serve internal requests. This results in splitting the Web servers between the perimeter network and the corporate network. If you do this, ensure that traffic from the Internet is load-balanced to the Web servers in the perimeter network and that traffic from inside the corporate network is independently load-balanced to the Web servers inside the corporate network. You must also set up different alternate access mapping zones and firewall publishing rules for each network segment.

This guidance is not very clear when it comes to the alternate access mapping zones requirement. In conversations with Microsoft technical support I was unable to get clarification on that guidance, although the problem that we encountered may hint at its meaning.

Picking people across domains

This deployment was escalated to me when we encountered some unexplained people picker behaviour. When we would browse to a web application on the web front end servers in the DMZ we were successfully able to search for users in both domains. When we would browse to a web application on the internal network’s web front end servers we were only able to pick people from the internal domain. Searching for known users in the external domain (where there were no internal matches) we would get a, “No exact match was found” or “No results were found to match your search item. Please enter a new term or less specific term.” In short, the internal domain servers couldn’t pick from the external domain on any web application.

Our troubleshooting with Microsoft started with a methodical review of the stsadm peoplepicker configuration, ports, permissions, etc. To abbreviate a very long second chapter of this story, the problem persisted throughout a number of different reconfigurations at the domain and forest level, with and without explicitly declared credentials for the external domain. We settled on a 1-way domain trust and configured the people picker with explicit credentials for the external domain.

As a brief aside, I should note that the Full Metal Architect blog is an excellent resource on this topic. Per the guidance in those articles and this Microsoft support blog, I started to troubleshoot with PSGetSID, at which point I got the first indication that this may not be a SharePoint issue per se. We noticed the same behaviour with PSGetSID as in SharePoint. We were unable to resolve the SIDs of external users from the internal domain servers. We also noticed that both SharePoint and PSGetSID started to work as soon as we briefly switched to a two-way domain trust (as a test).

However, I couldn’t reconcile these results with my network monitor captures, in which SharePoint was successfully retrieving precisely the same information in its LDAP query of the external domain from any server. On every server in the farm I could see that the people picker was successfully returning five of six partial attributes for the user that I hoped to pick. It could find cn, distinguishedName, displayName, userAccountControl and sAMAccountName. The objectSid was the one value that was blank, but we saw this same result on both internal domain servers and in the DMZ.

PartialAttribute: objectSid=( )

So we knew that:

• SharePoint was issuing successful LDAP queries of both domains from any server in the farm, but the results of the query from the untrusted domain never made it in to the people picker’s results on the internal domain’s servers.
• People picking worked on all servers as soon as we changed to a two-way trust.

At this point I was speaking with a technical lead on the 1st-line support team who suggested that this topology was unsupported. He supported this claim with two TechNet posts from Joel Oleson (in which he explicitly says all servers need to be in one domain) and Neil Hodgkinson (in which he also says “you cannot split a farm across multiple AD domains”). Typically I would take their guidance at face value but since this was directly contradicting the Microsoft guidance at the top of this article and since their posts were so old, I persisted for a more official response from Microsoft. The next morning I was told that those posts were wrong and that a farm can have servers in multiple domains.

Microsoft pushed the issue to a Tier 2 support technician, who quickly escalated to his escalation point. Finally getting somewhere… He quickly explained that we absolutely won’t be able to pick external users from the web applications on internal domain servers for the same reason that you can’t assign permissions to users from an untrusted domain. Although internal domain users can authenticate to the external domain resources they cannot grant users from the untrusted domain access to the internal domain’s resources. In the same way (since SharePoint uses SIDs to track all user activity), if a SharePoint web application is looking for a user in an untrusted domain, it won’t be able to get the SID in order to assign tasks or do anything else with a user, even though the initial LDAP query succeeds and even if the task has nothing to do with securing a resource. This was my fundamental stumbling block. Since we weren’t granting access to anything I couldn’t see why a trust would be required, but since SharePoint uses SIDs to track all user activity it makes sense.

Topology implications

So… our options were to put a two-way trust in place or to create an alternate access mapping zone that would point exclusively at the DMZ servers (although the support contact agreed that using a different zone for picking would result in pretty poor usability). In the end, we wound up simplifying the topology and used ISA publishing, since the two-way trust entailed similar security implications to the inversion of the 1-way trust for an ISA reverse proxy. I also asked if enabling Selective Authentication might help to lock down the trust of the external domain but the Microsoft support team were unable to get the people picker to work with Selective Authentication enabled. I believe that every user that might be picked would need to be enabled for Selective Authentication in this scenario, which would completely defeat the purpose of selectively authenticating.

Since then, in further discussions with Tier 2 support, we’ve been told that servers in different domains are not supported unless all web front end servers are in the external domain. They added that you can only have a web front end on the internal domain if it is an index server, e.g., not serving requests to users. I must emphasise that this was the response to a support case and not official Microsoft guidance, although I have commitments from Microsoft support that the TechNet documentation will be updated in the near future. We’ll need to wait to see how updates to that guidance pan out when they are released (this might take some time, as it involves multiple teams). For now, I would proceed with extreme caution if considering any topologies with servers in multiple domains.

A few months ago I reminded myself of a major gotcha when planning a virtual infrastructure. Assume that you run more than one domain in more than one forest and that trusts are in place to authenticate users across those forests. This could be a development/test/staging environment, or as will no doubt be more common in the coming years, it could be a virtualised infrastructure.

Assume that Kerberos needs to operate across these domains for any number of purposes, from application authentication to Active Directory migration. If you want Kerberos to work, you’re going to need to synchronise time across these domains, as the clock synchronisation is used to time stamp tickets in order to prevent replay attacks. Time synchronisation is of course built in to the Windows domain infrastructure, and should support this nicely. Configuring the Windows Time service on a PDC emulator is a bit fiddly, but should be achievable for anyone who runs a multiple domain infrastructure. But what if it goes wrong?

I spent a few hours bashing my head against this utterly confounding problem until the obvious whacked me in the face. The problem:

• I would run w32tm /resync /rediscover and all would synchronise successfully, then about five seconds later the clock would revert to its former time, about 2 minutes and 45 seconds ahead of the recently received absolute value
• There were no error logs, just successful resynchronisation messages and then nothing to indicate why the clock reverted
• I set the registry keys to enable debug logging and this revealed nothing more than that the time of the events was shifting as you would expect

Then it hit me. The PDC emulator was a virtual machine and the host virtual server was in another domain/forest. That host server was failing time synchronisation with its DC, so I manually reset the host clock and voila! The PDC emulator was synchronised within seconds. Obviously there’s another task to find out why the host was failing synchronisation with its time server, but that’s totally beside the point.

The lesson: Obey the best practice guidance and turn off Host Time Synchronisation for virtual domain controllers. In Virtual Server 2005 R2 the setting is in the Virtual Machine Addition Properties.

And a follow-up note: once Host Time Synchronisation is disabled you will need to find a new source of reliable clock for the the VMs, as they don’t have a CMOS clock to rely on. You could use something as simple and free as Atomic Clock Sync. If you fail to do this the VMs will lose time when they are shut down.