Corey Roth [MVP]

When deploying a multiple server FAST Search for SharePoint farm, sometimes you can run into a Configuration Failed error.  The SDK has some guidance on this but I wanted to walk you through the troubleshooting steps I used to determine what the issue was and then how to fix it.  It all starts with the deployment.xml file that you use to configure your FAST Search for SharePoint farm.  The contents of this file have to be absolutely perfect in order for everything to work.  On top of that, the file must be UTF-8 encoded and not UTF-16.  We’ll talk about that in a bit.  Although you specify a deployment.xml file when you configure the Admin server, the configuration wizard will not fail here.  This leads you to believe there is nothing wrong with your deployment.xml file.

When the Configuration Wizard failed on my non-admin server, I took a look at the logs.  There were numerous errors there, but the one I locked onto and actually found some search results on the Internet for was the following:

"C:\FASTSearch\bin\MonitoringServiceConfig.exe" Output - Error: The file 'C:\FASTSearch\etc\middleware.cfg' was not found.

This led me to a forum post that recommended I run the following PowerShell command on the non-admin server:

Set-FASTSearchIPSec –create

This resulted in the following error.

Set-FASTSearchIPSec : XML Validation error: Data at the root level is invalid. Line 1, position 1.

Now we’re getting somewhere.  I had seen this error mentioned in the SDK article on troubleshooting Configuration Wizard failures.  I had ran across this article before now, but I didn’t really think it applied since I never got an error when I installed my first (admin) server.  I was wrong.  The issue goes back to when I was editing the deployment.xml file.  I opened it up in Visual Studio 2010 so it would be easier to edit.  Don’t do that.  It saves it in a UTF-16 format which FAST does not like.  Edit it with Notepad or any other editor that isn’t going to mess with the encoding.

Now here is what the SDK article doesn’t tell you.  How to resolve it.  What we need to do is go back to the admin server and reload a deployment.xml file that doesn’t have the offending UTF-16 encoding.  Unfortunately, you can’t just run the Configuration Wizard again.  Instead you need to log on to your admin server and go to the following folder C:\FASTSearch\etc\config_data\deployment.  At this point, I make the standard disclaimer to back up your files and be careful.  I then deleted the deployment.xml file in this folder and created a new one with Notepad.  Be sure that your file has a .xml extension and not a .txt extension when you do this.  Now cut and paste the text of your previous deployment.xml file into Notepad and save it.  I’m assuming you have a copy of it somewhere.

When you are done editing deployment.xml, you will need to run the following PowerShell command on the admin server.

Set-FASTSearchConfiguration

This will reload the new deployment.xml file and set this as the configuration for your FAST Search for SharePoint farm.  When it completes, take the deployment.xml from this folder and copy it to your non-admin server(s).  Run the Configuration Wizard again and specify the path to this new deployment.xml.  If everything is correct, you should be able to complete the Configuration Wizard successfully.

In the FAST Search for SharePoint configuration process, getting your certificates configured correctly is probably the most important part of the process.  Without the certificates configured correctly, FAST simply will not work.  When running the script SecureFASTSearchConnector.ps1 on each SharePoint application server, you might encounter the following error.

Could not install certificate. Script can be rerun to only set access rights when reason for error is detected. Access is denied.

The error message might seem a bit confusing at first, but it simply is an access denied type error message.  How do you resolve it?  First, to the best of my knowledge, you can only run this script as the farm account.  I know running as the farm account is bad, but it’s the only way I have seen it work.  You might be able to get it to run with your setup account by granting enough permissions, but it’s hard to say.  Typically, I’ll just Shift+Right Click on my SharePoint Management Shell link and choose the Run as Different User menu option to specify the credentials of the farm account.  If you don’t have User Account Control (UAC) enabled, the script should run just fine. 

However, if your server has User Account Control enabled, this can also cause problems.  To my knowledge, you can’t do a run as different user + run as administrator at the same time.  This means you actually have you log into your box with the farm account via remote desktop.  I know, it’s not an ideal situation at all.  However, hopefully you haven’t removed your farm account’s administrator access yet when you were setting up the User Profile Service.  Once you log in as the farm account, run the SharePoint Management Shell as an administrator and then you will be able to execute this script. 

Be sure to run this script on each SharePoint server that will be hosting the Search Service applications.  This would also be a good time to remind you that the username parameter expects the name of the account running the SharePoint Server Search 14 service.  Be sure this is running as a new dedicated account and not the farm account.

In a multiple server SharePoint farm, you often find yourself modifying the Search Service Application topology at some point in the initial configuration process.  The scenario is that you typically go into your Search Service Application, add query components or indexers, and then save your topology changes.  Sometimes, when the process doesn’t work, you might find yourself looking at a timeout error when you go to commit your changes.  There are probably a number of reasons that this can occur.  I can going to provide you with a few things that you can check should this error occur for you.

First, go to Services on Server in Central Administration.  Check each server involved with the topology change and verify that the SharePoint Server Search service shows that it is Started.  If it is stuck on Starting, there is a potential issue.  At this point, go to the Windows Services control panel and see if the SharePoint Server Search 14 service is started there.  Likely it isn’t.  If the service isn’t started, you can attempt to fix this in a number of ways.  You can try to start the service manually from the Windows Services control panel.  You can stop the service with PowerShell and attempt to restart it (more involved).  Lastly, you can just try rebooting the server in question.  Rebooting usually will work.

Once SharePoint shows the service as started in Services on Server, you can go try to make your topology changes again in the Search Service Application.  If all goes well, the changes should complete after several minutes.

Everyone has their own take on what service accounts should exist on a given SharePoint farm.  From the official documentation, you pretty much have to patch together bits of pieces of things to come up with a concrete list.  The purpose of today’s post is to give you a starting point for your own account list.  It also includes the permissions that need to be granted in order to make things work properly.  In a lot of organizations, companies lock down a lot of the privileges accounts have via group policy.  This guide will hopefully give you a list that you can start with on your next install to make the process go smoothly.  Did I get something wrong?  Perhaps.  Leave a comment and I’ll continue to update this post so that it has the best information possible.

Let’s start with the general naming of accounts.  If you read my article on naming conventions, then you know consistency is key.  You also know I can’t stand abbreviations when it comes to naming anything.  However, if you know Active Directory, you also know that account names (SAM Account Names) are limited to 20 characters (for backwards compatibility reasons).  For this reason, you really have to abbreviate some things and you are best if you keep your account name prefix short.  Ideally you want all of the SharePoint accounts to have the same prefix so that they are easily identified and found.  For my accounts, I typically go with the prefix sp_.  SharePoint is pretty recognizable with the sp abbreviation.  I typically don’t like underscores but I tend to include them in account names.  You can leave it out just as well.  I also go with lowercase letters for the prefix as I find it just looks better.  Of course this is just one possible prefix, you can always come up with something you prefer better.  Just keep in mind the account name length limitations.

Let’s take a look at the account list.

No accounts should ever have domain administrator privileges.  Log on a service, Log on as a batch job, and Allow log on locally are all privileges inside Local Security Policy.  SharePoint usually does a pretty good job of assigning these privileges when you set up the accounts.  However, if your servers are running under a tight group policy, then it is more than possible that the GPO will remove these privileges or your accounts might be in Deny groups.  If your servers do run under a GPO which strips privileges, everything will more than likely work up until the point you reboot.  Once you log back in, you will find that none of your services have started.  At this point, you will need to work with your Active Directory administrator to get the required permissions.

This is quite a few accounts.  Can you get by with less?  Maybe, but I recommend against it.  Some domain administrators are more concerned about the clicks required to create a new account than the actual security of your SharePoint farm.  You might be able to consolidate your application pool accounts.  However, some accounts have conflicting security requirements.  For example, the sp_Farm account has full permissions on your SharePoint farm but the sp_SearchCrawl account must only have read permissions or bad things happen.  The lesson here is don’t take shortcuts when it comes to SharePoint accounts.

What about environment specific accounts (i.e.: development, test, production)?  A lot of organizations use separate accounts for test versus production.  This is a pretty good practice but you have to have a naming convention to accommodate these different environments.  You might be tempted to append something like dev, test, or prod to the account names.  However the issue with the names I provided is that many of them use up most of the available characters.  Instead, what I recommend is that you leave the production accounts as is.  By default if you see an account name, you assume it’s production.  For test, your best bet it to probably append or prefix a letter such as T.  For example, the farm account would be sp_FarmT or tsp_Farm.  Neither might be an elegant solution, but keep in mind you can always include the full word in the display name of the account (i.e.: SharePoint Farm Test).  Have a better way to organize these accounts?  Leave a comment.

I hope this serves as a good starting point for those of you setting up new farms.  Do you have accounts that you think should be on the list?  Let me know.  One area in particular, I didn’t cover was the BI features such as PerformancePoint and PowerPivot.  I’ll add these areas in at a later date.

Follow me on twitter @coreyroth.

My team and I have come to rely on SharePoint Workspace for our current project on a daily basis.  Most of us never even open a web browser even and browse project documents solely by using SharePoint Workspace.  It’s a great tool and allows us to grab a copy of the latest documents and view them offline on the airplane or wherever.  Leave it up to us to also find a way to break it.  Back in April our world came crashing to a halt when all of the sudden, SharePoint Workspace would no longer sync our document library.  When trying to sync we were met with the following error.  Here is what you saw when you click on Resolve in the ribbon.

Data in this list references content type "", which is no longer in the list schema.

Doing a quick search on the Internet quickly yielded nothing.  In our case, no changes to the document library occurred.  The schema was the same and the only thing new was that documents were uploaded on a daily basis.  I tried various things but could not come up with a solution.  I finally relented and burned one of my MSDN subscription support incidents.  The support engineer, Patrick Gan, immediately suspected OneNote.  We have used OneNote pretty heavily on this project.  He did some testing on his end and eventually he was able to reproduce the problem.  As a result, I moved all of my OneNote files out of the library and tried to synchronize again and sure enough the issue went away.

Now, it turns out, not all OneNote files cause the issue.  Individual .one files are fine.  There are also OneNote files with an extension of .onetoc2, .onebin and .onepkg.  I believe it to be one of these two file extensions that causes the problem but never narrowed down which one.  It appears that when you make use of the File –> Share menu in OneNote when it published your notes to a document library these files get created and the synchronization errors occur.  The support engineer also mentioned that OneNote creates folders when you share this way that may be the cause of the issue.  In those folders is where the .onebin file is stored so that may be the case.

Another workaround that I have received is to open the offending notebook in OneNote, click the File tab and then click View Sync Status.  From there, click on Work Offline.  This will prevent OneNote from resyncing files to that particular library.  I created a new document library for now that just has my OneNote files in it.  I then used the Change Location button to choose the new document library.  At this point, you may need to go in and clean up the unwanted .one* files in the previous document library.  It should sync at this point, but if it doesn’t, disconnect the existing library from the workspace and recreate it and it should work.

If you receive the above issue, hopefully one of the stated workarounds will resolve your issue.  On the bright side, this is classified as an official bug and there will be a fix “in the very near future” (whatever that means).  Until a fix is available, try these workarounds.  Thanks to Patrick once again for assisting us in resolving the issue.

Sorry, I’ve been slacking.  I’m a full week behind on posting my slides from SharePoint Saturday Houston 2011.  This SharePoint Saturday was certainly unforgettable and I had a great time both attending and speaking.  I presented my Beginning SharePoint 2010 Development talk to a room packed full of people.  The turnout for the event was great.  Many thanks again to the master mind @Victor_Chat and all of the great people H-SPUG for putting the event on.  I had a great time.  I’ve uploaded the slide deck to SlideShare for your reference.  I’ve also provided links to my relevant blog posts which will help get you started with SharePoint 2010 development.  If you have additional questions, feel free to ask.

Beginning SharePoint 2010 Development for ASP.NET Developers Slide Deck

Intro to SharePoint 2010 Development: How to Build and Deploy a Web Part

Intro to SharePoint Development: How to Build and Deploy a Web Part

Office 365 How to: Build and Deploy a Web Part with SharePoint Online

Follow me on twitter @coreyroth.