Corey Roth [MVP]

I am excited to be returning to Houston next week to speak at Houston TechFest.  This looks like it will be a great event and I’ve been curious to see what Tech Fests look like in other cities.  I’ll be speaking on an advanced SharePoint development topic about how extend the BCS using BDC Models and Custom Connectors.  If you like to see a lot of code in a session, this one is for you.  I hope to see you all there!

Getting FAST Search for SharePoint 2010 installed is a pretty involved process.  It’s definitely easier in the RTM since there is a wizard now and you don’t have to run 20 different PowerShell scripts.  However, one issue I encountered myself a few months ago was an error I received in the Configuration Wizard.  It occurs on the screen prompting you for the name of your FAST User account.  When you specify the username and password, you might get the following error.

User Validation Failed.  Username or domain is not valid.

Immediately, you assume that you typed the password in wrong, but you soon realize that is not the case.  I honestly don’t remember where I found this information otherwise I would give him or her credit.  I’ve seen a few people run into the issue in the forums lately, so I thought I would post something on it.  To resolve the issue, you need to grant the Allow log on locally privilege to your FAST user account in your Local Security Policy.  Add it and run the configuration wizard again and you should be up and running.

I’m excited to announce that my guest post came out today on the Scripting Guys (@ScriptingGuys) PowerShell Blog.  In this post, I talked about how to manage search crawls using simple PowerShell commands.  Check it out if you want to learn more.  I really enjoyed contributing to this great community blog and I’m thankful for being included. 

How to Use SharePoint 2010 Windows PowerShell Cmdlets to Manage Search Crawls on Scripting Guys Blog

When I’m collaborating with other users on an issue, I’m always surprised to see how many people don’t really know some of the basic shortcuts to make working with command prompts easier.  I guess I have forgotten that not everyone has been working with command prompts since MS-DOS or csh so they don’t have any exposure to any of the tricks.  I once was a noob myself, but now I’m showing my age with this post. :) 

Often, I’ll see a user type out the entire path to the 12 (or 14) hive or I’ll see him or her retype the entire command that they just entered before.  You don’t have to do that! :)   No wonder why some people say they loathe the command prompt.  They are doing everything the hard way. Today’s tips apply to both the Windows Command Prompt and PowerShell.  If you have been using command prompts for years, this post probably contains nothing new for you.  You never know though, you might learn something new or see a feature that you forgot about.

Tab Key

The tab key is your friend.  Think of it just like you do in Visual Studio.  It’s the IntelliSense of the command prompt world.  By pressing it, the command prompt fills in file and folder names for you so that you don’t have to type it all.  This is useful for completing long folder names such as Program Files or for executing that new download you got with the really long filename.  For example to change to the program files folder, you could type the following.

cd pro<tab>

It also works with the change directory command when you use a slash but be sure and put a space after your CD command otherwise it won’t work.

cd \progr<tab>

If you didn’t get the folder or filename you were looking for the first time, just press tab again.  If nothing matches, the tab key will do nothing or it might beep at you.

In PowerShell, it provides you the name of cmdlets that match what you typed.

Get-SPWe<tab>

This would allow you to cycle through any cmdlets that match such as Get-SPWeb, Get-SPWebApplication, Get-SPWebPartPack.  It would be easier to demonstrate this ability with a video, but hopefully you get the idea.

Command History

Don’t type that last command you entered just to correct a syntax error!  There are a number of ways to execute previous commands.  Start with the Up and Down arrow keys.  This lets you go see and execute any of the previous commands you issued.  The F5 and F8 keys also have the same effect.  F5 shows you the oldest command you issued and F8 shows you the last command you entered.  Pressing F7 is really slick and shows you a list of all the commands you entered.

F9 allow you to pick a command from the above list by number.  I’m not really sure when that would be useful, but its there if you need it.

Copy and Paste

You can copy and paste to and from a command prompt or PowerShell window, but you have to use a mouse to do it!  Ack! At least, I’ve never found a way to do it with a keyboard.  Unfortunately, Ctrl+C, Ctrl+V and things like that simply don’t work.  To paste something in to a command prompt or PowerShell, you have to use the edit menu, which you can find by clicking on the icon in the top left corner of the window.  For example, if I wanted to paste a path for use with a change directory command, I could use it as you see below.

There is actually a better way to work with paths though that you will see in a bit.  If you want to copy something from a command prompt PowerShell, you can do that too.  Go to the edit menu again and choose mark.

Then you just use your mouse to mark the text and go back to the edit menu and choose Copy.

Drag and Drop

I mentioned earlier that there was a better way to get paths into PowerShell.  If you have a folder open in Windows Explorer, you can actually drag files and folders directly into a command prompt or PowerShell and it will put the path there.  This is great for executing files with long names or for switching quickly to folders with long paths such as the 14 hive.

Path Variable

The Path variable has been around for ages and can still prove to be quite handy.  Probably the place I use it the most is to allow me to execute stsadm from anywhere.  This was more useful with SharePoint 2007 and is not as relevant any more, but it’s still worth mentioning.  What you want to do is add common paths of folders that contain executables that you run often.  You can set the path directly from the command prompt, but it you want to make it permanent, the easiest way is to go to your Computer Properties –> Advanced –> Environment Variables.  Scroll through the list on the bottom, click on Path and then Edit.  In the Variable Value textbox, go to the end add a semicolon (;) delimiter and then add your path such as c:\program files\common files\microsoft shared\web server extensions\12\bin.  Here is what it looks like.

When you save the changes, open a new command prompt, and you will be able to execute files from that new path from anywhere.  For example, I can run stsadm from anywhere now as shown in the screenshot below.

More

Sometime, you execute a command and get a lot of results back.  Whether it’s a directory listing or a list of processes, you might want a way to make the results more readable.  The quickest way to get yourself some breathing room is by hitting the Pause key.  You have to be fast though. :)  Another way is to use a pipe (|) with the more command.  This has been around a while and works with commands prompts or PowerShell.  Here is an example.

Get-Process | more

The results look like the screenshot below.

This gives you the results conveniently one page at a time and you can press the Spacebar to view the next page or the return key to view one additional line at a time.  If you decide you don’t need to page through the rest of the results just press Ctrl+C.  You can use this keystroke to break you out of most commands.

Output Redirection

The last thing I will cover is output direction.  This has been around at least since the UNIX days and we will see it in modern command prompts.  It gives you the ability to redirect all output to a file using the greater than (>) symbol.  To redirect the results of the above command into a file we would issue the following.

Get-Process > processes.txt

When you execute it, you will not see any results on the screen.

Opening the file we see the results from the command.  It’s a useful technique when you are dealing with lots of results from a command prompt or PowerShell.

If you’re new to command prompts, I hope this post proves to be useful for you.  I hope to save people some time and frustration the next time they have to issue some commands.  Have some other useful tips to add?  Leave a comment!

Follow me on twitter.

When I got my new laptop, I had a ton of issues with high guest CPU utilization in recent versions of Orcale VirtualBox.  It turns out I wasn’t alone with this situation seeing as how many comments I had on my previous post.  We all tried a lot of different things with varying degrees of success.  Personally, my work around was to go all the way back to version 3.0.12.  It worked alright, but things could be better.  Luckily, someone by the name of EdwardW came through for all of us and discovered that disabling Nested Paging seems to correct the issue.  I tried it myself and have been running for over a day with various virtual machines and they are performing great.  I only know Edward by his handle EdwardW, but we are all thankful for his diligence in finding a solution to our issue.  He even claimed to have it working well with multiple virtual CPUs.

If you’re not familiar with this setting.  Shut down your virtual machine so that you can edit the settings.  Then go to the system tab, click on Acceleration and then uncheck the Enable Nested Paging checkbox.  Click OK and start the virtual machine up and you should quickly notice some performance improvements.

It seems to work great for me so far.  I was quick to pronounce a victory last time, but things appear to be running faster than ever now.  Give it a try for yourself.  The CPU utilization was even decent when I had a search crawl running.

The strange thing about all of this is that my previous laptop did not have this issue.  I now suspect it is likely to have something to do with newer processors such as the i5 and i7.  I could be wrong, but it certainly seems like the case for me.  Is performance so good now that I can run FAST Search?  Well, I wouldn’t say that, but it is better.  Give it a try for yourself and let us know how it works.

Anyone who likes SharePoint Enterprise Search has to love FAST.  As a developer, sooner or later you are going to want to take advantage of the FAST Query Language also known as FQL.  This lets you truly leverage the power of FAST and perform very advanced queries.  I’ve shown you how to use the KeywordQuery class in SharePoint 2010.  Now we’re going to expand on that to execute FQL queries.  It’s actually easier than you might suspect.  It all starts with setting the EnableFQL on the KeywordQuery class to true.  Of course, you still have to remember how to put the rest of it together first. :)

Note, there is also another FQL out there known as the Facebook Query Language.  If you’ve come here looking for that, you’ve come to the wrong place as I don’t know anything about that.  Although maybe I should. :)

When dealing with search, we always start by figuring out what our service application proxy is called.  On a typical FAST installation, you are likely to have three different Search Service Applications: FAST Search Connector, FAST Search Query, and a regular SharePoint Search Service Application to handle people search.  This means you need to find the right one (FAST Query) in order to actually get some meaningful results.  Head to your Service Applications page in Central Administration and take a look.

In my particular case I happened to have named my service application FAST Query SSA.  I got this name straight out of the installation guide for FAST, but I have found that people don’t have much consistency on what they name this.  If you are not sure which application to use, select one of them and click the Properties button.  If it looks something like this, you have found the right one.  I also think it’s the only FAST service application that has a proxy, but I could be wrong.

Excellent!  So now you know the name of your service application so in turn you know the name of the proxy.  Now we leverage the SeachQueryAndSiteSettingsServiceProxy again to get a reference to the SearchServiceApplicationProxy that we need.

These lines are similar to my previous KeywordQuery post except that we specify the name of the FAST Query Search Service Application Proxy.  Now it’s just a matter of creating an instance of the KeywordQuery class and enabling FQL.

The rest of the code looks pretty much the same.  Let’s start out with an FQL query that looks for an author that end withs the last name of Murphy.  To do this, we use the ends-with FQL operator.  Our query will look like the following:

author:ends-with(murphy)

This is what the rest of the code looks like.  You can see that I used the above query here in the QueryText property.

I use the Execute() method to perform the query and then the results are loaded into a data table.  If you are curious about the details on the rest of those lines of code, see the KeywordQuery post.  Using the dataset visualizer, I can see the results of my query.

Pretty simple, right?  Now, if I wanted to see all documents that are about termination or beer (obviously the two could be related :) ), we could use the or operator.  I’ll just put the new query in the QueryText property and execute the code.

Here are the results:

What I really like about this is that you can see what the rank of each result is right there.  If you want to start affecting relevance, you should be able to start using the XRANK operator like the example below.

xrank(accounting, budget, boost=500)

This should rank documents with the word budget in them higher.  Unfortunately, it has never worked for me though.  I’m still looking into it.  Perhaps there is a configuration setting I need.  As you can see, if you know how to query SharePoint Enterprise Search, it’s not hard to query FAST using FQL either.  Remember, that all of your regular KeywordQuery syntax operations work in FAST too.  You don’t have to learn all of FQL to query FAST.  It’s just there for when you want to perform more advanced queries.

In sticking with our theme of debugging things from yesterday, I thought I would write up a quick post on how to debug a cmdlet.  This may be obvious to you experienced developers out there, but someone new to writing cmdlets may not be familiar with the process.  Luckily, cmdlets are easier to debug than the Custom Index Connectors I talked about yesterday.  I hope to educate people as much as I can on how to build cmdlets, so that we can get more contributions to the SharePoint PowerShell Community Toolkit.  If you haven’t checked it out yet, please do. :-)

Once you have built your cmdlet, set a breakpoint in your code in Visual Studio.  Once you have done that, open a new PowerShell window (or a SharePoint Management Shell) and install your module or snapin.  The technique for debugging is the same whether you load your cmdlet with Add-PSSnapin or Import-Module.  In Visual Studio, go to Debug –> Attach to Process.  Scroll through the list and look for powershell.exe.  If you launched PowerShell using the SharePoint 2010 Management Shell icon you will notice the title in the list as shown below.

When you find the right process, select it and click the attach button.  Now, go to your PowerShell window and run your command.  If all goes according to plan, your breakpoint will be hit and you can step through the code of your cmdlet.

That’s all there is to it.  Isn’t developing for PowerShell easy? The snippet of code you see above is actually from a new cmdlet I am releasing soon in the SharePoint PowerShell Community Toolkit that lets you test search queries from the command line.  I’ve already found it to be a handy cmdlet when I want to test a query using the API quickly.

I have been researching Custom Index Connectors in SharePoint 2010 over the past few weeks.  I have been working with the MyFileConnector sample provided in the SDK, but after I made some changes, I realized it wasn’t immediately obvious how to debug my code.  Luckily, @mwiselka helped me out and pointed me in the right direction.  When it comes to debugging an index connector, timing is critical as well as picking the right process.  In this case it’s not a matter of picking the right w3wp.exe process. It’s a matter of picking the right search process.  If you don’t do the process right, your symbols will never be loaded and your break point will never be hit.

To get started, compile your connector and copy the assembly to the GAC.  Now you need to restart the search service.  It is critical that you restart the service any time you make a change or try to debug again.  I would also do an iisreset as this seems to be required as well most of the time.  If this is your first time installing the custom index connector be sure and follow all of the steps in the SDK which includes making registry changes and configuring the content source.  I usually restart the service from PowerShell or a command prompt.

net stop osearch14

net start osearch14

The process we need to debug is called mssdmn.exe.  When you restart the service you will only see one of these listed in your processes list.

You may be tempted to attach to that process right away and kick off a full crawl of your new content source.  Don’t do that.  It will not work.  When you kick off your first crawl, a second instance of mssdmn.exe is launched.  That is the process you attach to.  I remind you that timing is critical.  After you start the crawl, you need to refresh the process list quickly and attach it if you are trying to catch something early in the process.  It really just depends on the speed of your server, sometimes it takes quite a while for the process to show up.

Notice the new process listed.  Once you attach to the correct process, you should have no issue with your breakpoint being hit.

Now, you might be thinking, I’ll just wait and attach to all mssdmn.exe processes that show up.  I thought that might work but it doesn’t.  What I have found is that any time you attach to the original mssdmn.exe process, it causes your breakpoint not to be hit.  I have no idea why it behaves this way, but from what I have seen so far, you only attach to the newly spawned mssdmn.exe process.  If you are having trouble getting the timing right, one thing that @mwiselka recommended was to start a crawl of another content source to spawn the mssdmn.exe process and attach to it.  Then when you do your crawl, it should use the same spawned process.

You would think that you could use something like System.Diagnostics.Debugger.Launch() like I have done with features, but I have had no luck getting that to work.  You can see from my code snippet above, that adding a Thread.Sleep() can be a useful way to buy yourself time to attach to the process as well.  It’s kind of hack, but it does seem to work.  If the above technique doesn’t work, try it again.  Recompile, restart osearch14, and iisreset.  This technique can also be used with regular BCS Custom Connectors as well if you want to debug them while indexing.

Hopefully, you were able to impress the boss with my last post on the Chart Web Part.  I explained how to use the BCS with the Chart Web Part to display great looking charts using external data.  Another option is to use Excel Services.  Excel Services is pretty easy to setup now and you usually don’t have to do much configuration out of the box.  Let’s see what we can do.  This assumes you have Excel Services installed and configured to trust spreadsheets from your SharePoint server.

First, let’s start with my super awesome spreadsheet.  It's a simple example, but hopefully you get the idea.  If you have Excel Services working, you should have the View in Browser option when you are looking at a file in a document library.  Viewing Office documents in the browser might even be your default if you have installed Office Web Apps.

Clicking on that link, shows us the Excel document we are working with. 

SharePoint 2010 has much better support for a wide variety of Excel documents.  For the most part it will render most documents even if they have unsupported features in them.  This post has some details on what is supported and what is not.

To use Excel Services with the Chart Web Part, we need the URL to the Excel document.  You can get this in a number of ways.  Just be aware if you try to copy a link from the document library it might give you a link to the xlviewer.aspx which will not work in the Chart Web Part.  Make sure you get a link that references the .xslx file directly from your SharePoint server.

For example, this link will not work in the Chart Web Part:

http://sp2010/ECM/_layouts/xlviewer.aspx?id=/ECM/Company%20Documents/2010%20Budget.xlsx&Source=http%3A%2F%2Fsp2010%2FECM%2FCompany%2520Documents%2FForms%2FAllItems%2Easpx&DefaultItemOpen=1

This link will work in the Chart Web Part:

http://sp2010/ECM/Company%20Documents/2010%20Budget.xlsx

Once you have the link to your spreadsheet, edit a page and add a Chart Web Part to it.  If you don’t remember how, you can find the details on it on my previous Chart Web Part post.  Now, click the Data & Appearance link and then Connect Chart to Data.  Now choose Excel Services.  You will then be presented with a screen prompting you for the Excel Web Service URL, Excel Workbook Path, and Range Name.  The Excel Web Service URL should already be filled out for you.  It will look something like this.

http://sp2010/_vti_bin/excelservice.asmx

Now you need to provide a path to the workbook.  Use a complete URL like the one I used above.  The last thing you need to provide is the range of the spreadsheet you want to use.  Here you need to know some Excel basics.  You start by specifying the sheet name followed by the first cell using the !, $, and : delimiters.  In my case I want A1 through B5 on Sheet1, it looks like this:

Sheet1!$A$1:$B$5

If  the first row of your spreadsheet has column names, check the box.  Here is what it looks like completed.

Click Next and if Excel Services like your spreadsheet, it should give you a preview of the data.  If it doesn’t like it, it might give you a user friendly error or it might give you a Exception has been thrown by the target of an invocation error.  If that is the case, there is likely an unsupported feature such as a chart that you need to remove for the spreadsheet to work.  Here is what the preview looks like.

Click next, and you will choose the Axis and other settings for the chart.  Change any settings you want and then you are done.

Once you click finish you should be able to see your data using the default chart.

Of course if you want another type of chart, you can click Data & Appearance and pick another type from the Customize your Chart link.

One thing to note is that Excel Services does cache data for a while so if you change data in the spreadsheet, it might take a few minutes for the graph to reflect your changes.  This was a simple example, but as you can see it’s not very complicated to get a nice looking chart up and running quickly.  If you do run into issues, I recommend tearing your spreadsheet apart and look for unsupported features.