Corey Roth [MVP]

I’ve made quite a few Enterprise Search posts and since I have a few talks coming up, I thought it might be a good idea to group up some of the posts for people that may be new learning it.  I hope you may find some of these useful.

Customization

• Customizing Enterprise Search Results
• More on the XSLT behind the CoreResultsWebPart
• How to: Build a custom advanced search control for Enterprise Search
• How to: Create a scope for a custom List Template Type
• Document Link Handler for Enterprise Search

Administration

• Sharing an Index between SharePoint Farms
• Changing the Maximum File Size in the Enterprise Search Index

Querying Results

• How to: Use the MOSS Enterprise Search FullTextSqlQuery class
• How to: Use the MOSS Enterprise Search KeywordQuery class
• How to: Query Search using the Web Service
• Calling the Enterprise Search web service from Silverlight
• Exposing Enterprise Search Results with RSS
• String[] in columns returned from KeywordQuery.Execute
• How to: Specify the Properties Returned in a Keyword Query when using the Search Web Service
• How to search your site columns with Enterprise Search
• Using LINQ to make Enterprise Search Results more usable
• How to: Specify a Content Source using the Keyword Syntax
• Specifying Scope with the KeywordQuery Class
• Working with Managed Properties in Enterprise Search
• Excluding Folders from Search Results

Other Development

• How to: Create a Scope Display Group with the API
• How to: Delete Crawled Properties
• How to: Add a File Type to Enterprise Search programmatically

Wildcard Search

• Custom Sort Orders with Wildcard Search
• Disabling Full Text Search with Wildcard Search Release 5
• Wildcard Search Release 3
• What you give up with Full Text SQL Queries using Wildcard Search
• Using Wildcard Search Web Part to search for recently modified files

BDC

• Setting permissions on your BDC LobSystemInstance for Enterprise Search

Errors

• Object not found. Crawler was denied access.
• Crawl has been paused
• Search Service is currently offline.  An indexer move may be in progress
• The Parameter is incorrect
• HRESULT E_FAIL has been returned from a call to a COM component
• ArgumentNullException when calling Enterprise Search web service
• SearchBoxWebPart won’t submit

The above list is some of the more useful posts that you may be interested in with Enterprise Search.  I didn’t include every post, so if you are interested in more, check out the Enterprise Search tag.

I’ve seen a number of posts by people asking about how to customize the search results screen in MOSS Enterprise Search, so I thought I would take a little bit of time to describe how to do it.  Since this goes well with the two new talks I am building on Enterprise Search for SPSOzarks, I figured this would be a good time.  To customize search results, you will need to start by going to the results.aspx page of your Search Center.  If you don’t want to experiment on your existing Search Center, you can always create a new one.  Edit the page, and then modify the CoreResultsWebPart.  Some of the things we can change can be done via properties. However, most customizations will involve changing XSL.

In the Results Display / Views section, you can customize the number of results per page as well as how much text is included in the summary of each result.  In the Results Query Option section, you can enable search term stemming and turn off duplicate removal.  Although I typically leave these options alone.  The Selected Columns field we will talk about in a bit when we discuss adding more data to the search results page.  You can also choose to sort by relevance or modified date here.  Sorting by other managed properties is not an option with the CoreResultsWebPart.  Although if you use the Wildcard Search web part you can sort by other managed properties because it uses a Full Text SQL Query.

The Cross-Web Part query ID is used to group all of the other search web parts on the page with the same set of results.  The default value, User Query, uses the value the user typed into the search box for the query.  You can use the other values Query2, Query3, etc.. to group a different set of results together when using a fixed keyword query.  The Fixed Keyword Query can be used to hard code a pre-determined query on a results page.  Useful if you always want to display the same thing on a given page.  If you use this option, be sure and change the Cross-Web Part Query ID to something besides User Query.

Those are some of the basic customizations you can do, but the most common thing people want is to change the formatting or add additional data to the search results page.  The way the CoreResultsWebPart (or CRWP for short) displays results is with an XSL transform.  I won’t go much into changing the formatting since all it really involves is changing the HTML inside the XSL document, but I will cover how to display more data in your results.  The first thing you need to know is that any thing you want to display has to be a managed property.  That means anything you have mapped to a managed property can be displayed in the search results.  Before adding your new managed properties to the XSL document, you have to tell the CRWP to include your custom managed properties.  You do this by modifying the Selected Columns field.  Clicking on it gives you a fancy editor that looks like this:

The editor doesn’t format things very well, but the syntax is pretty simple.  To add your own Managed Property, find a place at the end before the Columns element closes and add a Column element.  For example, to add a managed property called DocumentType, it would like this:

I believe the case is sensitive.  If you get the managed property name wrong, you will get an error when you try to execute a search query.  The next thing you need to do is edit the XSL document to include your new managed property.  If you click the XSL Editor button, you will get a similar screen.

You can edit the XSL directly in here, but I would recommend, you copy and paste the whole thing into a file somewhere and store it in source control.  It’s quite easy to forget to close a tag or something else and get an error displaying your search results.  Saving the content of the XSL somewhere gives you something to fall back on in case you mess anything up.

Now there is a lot in this XSL file,but there really is not that much you are concerned with.  If you want to customize result, the place to start is with the Result template, look for the following line.

This template gets executed for each search result.  The srch-Icon span tag displays the icon next to the search result.  The srch-Title span tag displays the title and link to the result.  The $url variable has the path to the search result.  It makes a call to a special template to highlight keyword hits in the title.  After this point, is a good place to add your own managed properties.  For example, I want to display the DocumentType managed property mentioned above.  The best thing to use is a span with a class of srch-Description.  Then it is just simple XSL with the value-of element to display the data.  The one thing to note is that in this XSL document, all Managed Properties must be specified in all lower case.  I have no idea why it was implemented this way, but if you try to match the case you used in the Selected Columns XML document, it simply will not work.  Here is what it looks like.

After this, the description, document size, author, and modified date are displayed.  All of these can easily be customized, moved around, wrapped in other elements, or formatted differently.  Here is what my new search results screen looks like.

As you can see, it’s not difficult at all to add your own managed properties.  I’ve customized this page in many different ways.  It doesn’t have to look like a search results page at all.  You can make it a grid, a list or whatever.  This makes it very easy to provide a rich search experience with all of the additional data that your users need. 

Follow me on twitter.

I am working on a talk about how to get data out of Enterprise Search for #SPSOzarks.  One of my clients particularly has asked me a lot on this subject.  Sure you can get to the data by using the API with KeywordQuery, FullTextSqlQuery, or the Web Service.  However, sometimes the simplest solution is best.  If you don’t need any custom managed properties, just need a keyword query, and just want some XML with search results in it, why not use the built in RSS feed.  It in fact is quite easy to use.  Consider the following search results.

The RSS button you see on the Search Results page works must like the CoreResultsWebPart does in that it uses the query string syntax to execute Keyword queries.  If you look at the link that the RSS button provides, it makes a call to the application page at _layouts/srchrss.aspx and it makes use of the same k and s parameters that the CoreResultsWebPart does plus a few extra ones.  The k parameter is for the user’s keyword query (in this case, we searched for the word Company).  The s parameter specifies the scope.  You can also specify a start parameter to start the results on a specific record.  There is also a u parameter (which I don’t know what it does) and a source parameter to specify a referring URL.  So to construct an RSS feed for the above results, it would look something like the following:

//_layouts/srchrss.aspx?k=company&s=All%20Sites">http://<servername>/<SearchCenterUrl>/_layouts/srchrss.aspx?k=company&s=All%20Sites

It would yield XML that looks like this.

I cut out most of the item elements from the XML for brevity but you get the point.  You get a link the results along with a title and description.  At this point, you can easily consume these results with an RSS client or maybe use LINQ to XML to parse them.  Interested in what the document looks like when there are no matching search results? 

You just get an RSS document back with no items and the description tells you that no results were found.   As you can see executing queries with RSS is pretty simple, so if you need a quick solution, this may be the way to go.

I’ve been working on putting some stuff together for my talks at SharePoint Saturday Ozarks (#SPSOzarks) and ran into this interesting issue.  Now this may just be an isolated incident, but I have discovered a few errors that occur when trying to query a managed property of type decimal.  To start out, I created a custom content type with a site column called DocumentPriority (name doesn’t really matter obviously).  I configured the site column as a number with 0 decimal places.  Search picks this up as a crawled property but it determines it is of type decimal instead of number.  I go ahead and map this to a managed property of the same type and then attempt to query it in the usual manner:

DocumentPriority:5

I expected to see some results or at least the message saying no results were available but in fact I get the following error message.

Your search cannot be completed because of a service error. Try your search again or contact your administrator for more information.

That of course confuses me, so I try the query again and I get:

The search request was unable to connect to the Search Service.

This confuses me even more.  I start to wonder what is going on.  I try some different searches and discover other queries work fine, but when I try this query again, I get the same error messages.  I decide to do some more research and check the Event Log.  I find the following events

The last query machine has been taken out of rotation.  Check previous event logs to determine cause.  Propagation may be necessary. Component: 68105977-f711-43b5-a25e-4f2d210625e1

Query machine 'MOSS-SERVER' has been taken out of rotation due to this error: Values of this type are not turned into strings necessary to generate scope information. The item will not be locatable in all of the appropriate scopes but this is not a serious error.   0x80044100.  It will be retried in 15 seconds. Component: 68105977-f711-43b5-a25e-4f2d210625e1

This means that the query server is dying anytime it receives a query like this.  That is extremely bad.  I haven’t resolved the issue yet, but I think updating will help.  This particular server does not have SP2 on it which I think will resolve it.  Well technically I think it was the August Cumulative  update from last year that fixes it, but I thought it was worth noting.  I thought it was worth noting none the less in case you haven’t updated yet.  I am going to update this server and see what happens.  I’ll post an update if anything changes.

As people that know me well know, I am all about installing anything beta.  I am more than willing to try out some buggy software and risk having to reinstall what’s on my computer because that’s how I roll.  When I joined the iPhone developer program, the situation was no different.  I proudly installed Beta 3 of 3.0 and things were working great.  Now at this point I will stop you and tell you that it was not the Beta that caused an issue with my phone.  It was the fact that I experienced a hardware failure on my phone while having the beta installed. 

Here is what happened.  I used Beta 3 for a couple weeks and everything worked great.  I later installed Beta 4 and it worked for a while.  One day, I pull my phone out of my pocket, only to find that the device is turned off.  I turn it back on and I get the pink Connect to iTunes screen.  No big deal, I plug it in and I restore Beta 4 again and it works fine for about another week.  The issue then occurs again, and this time I am unable to restore the OS onto the device.  Beta 5 comes out so I try it, it doesn’t work.  I always end up getting an Error Code 28 (sometimes error 9).  I try multiple ways of flashing using iTunes and Xcode but nothing works.

At this point, I am at a loss and try to go to the Tulsa Woodland Hills Apple store for help.  I made several attempts to go to the Apple store and beg for a new phone, all but the last one were unsuccessful.  The first time I went in the tech immediately noticed that I had the pink Connect to iTunes screen.  This is code to the Genius Bar that says we don’t have to support this device.  The first guy turned me away; basically accusing me that my device ID was not registered with the Apple Developer program.  I claimed that in fact I am a registered developer and I am more than willing to log in and show him, but he wouldn’t have any of that.  So I left the store with my brick.

You see Apple’s policy is that you get absolutely zero (0) support for your iPhone while you have a beta installed.  So even though you are doing them a favor and helping them test out their software, they will leave you high and dry.  Now they have disclaimers when you download the beta that this should only be conducted on devices used for testing, etc.  That is fine, but no where does it say that if you have an issue with your phone whilst having the beta that they will not support you.

Sometime, after Beta 5 came out, I decided to try the Apple store again.  At this point I had tried doing the restore with a different computer, cable, operating system, etc.  The results was always the same.  I go in there and explain my situation to another tech and I basically get the same story.  However, he said come back after 3.0 is released officially to the public and we’ll see what we can do.  Frustrated, I accepted that and I scheduled myself an appointment for June 17th when 3.0 was announced at the WWDC.  I knew that the release version of the OS wouldn’t install either, but I accepted it and impatiently waited.

So on June 17th, I enter the Apple Store at 2:45 and it is slammed.  Amazingly though, the Genius Bar is right on time and I get a tech right away.  He sees the pink screen on my device and then disappears for a minute.  He goes and talks to his lead and he comes back with the answer that I have to go through the developer site for support since it has the beta on it.  I respond by saying that 3.0 has been released to the public, I need some support.  He responds several times by saying that our policies haven’t been changed though to allow us to work with the beta device.

This time I refuse to take that as answer.  I ask to talk to the lead.  The current lead, Sean Love, comes over and we talk for a bit.  After a while, it finally becomes clear to me.  He tries to restore the phone and he gets the same error message.  If they see the pink screen on the device, they assume your pir8ed the 3.0 beta.  I state I am in the developer program, I can show you.  He seems almost surprised that I said that, but he allows me to login and I show him where my ID is registered.  At this point, he takes some info from me and tells me he can help me out.  I am relieved.  A few minutes later he comes out with a new iPhone for me and I finally am able to leave the store with a working phone.  I want to take the time to express my gratitude to Sean for helping me out with my issue.  I really do appreciate it.

I would like to reiterate that I do not feel at all that the beta OS destroyed my phone.  It seemed to suffer from hardware failure as it could not write to a portion of the flash memory (or whatever it is) during the restore.  The reason, I will never install a beta on a phone again is because if you have any other kind of issue, Apple will not support it.  I write this today to make this point clear to more people and maybe urge Apple to change their policy.  I was without an iPhone for almost two months and it was completely unacceptable.  Developers need some line of hardware support.  I read the forums, other people had similar issues.  Some tried calling the developer number and they got turned down as well.  Hardware issues can happen while the beta is installed and we shouldn’t be out of luck because of it.  There will have to be some amazing stuff in the next beta for me to consider installing it on my device.  Most likely I will wait next time.

I decided to download the trial of BDC Meta Man Web Edition from Lightning Tools today since I have used BDC Meta Man so much in the past.  I found the tool to be quite functional and I easily able to connect to a database table and import an application definition.  There are several steps to the install process involving the installation and deployment of several packages along with several features scoped at the web application and site collection level.  One thing to note is that .NET Framework 3.5 SP1 is required.  The servers I tried it on I thought had it, but in fact only had 3.5.  @nickswan quickly helped me figure out that was my issue after I had everything installed.  A batch file allows you to quickly add the 6 needed solution packages to the solution store.  Then you just have to deploy each one to your web application and activate the relevant features.  The included documentation makes it pretty clear what you need to do.  There are quite a few steps to the install, but it’s not bad at all.

Once installed, a link is added to the Site Collection Administration links labeled BDC Meta Man Web Edition.  The cool thing about this is you don’t have to have direct access to your SSP site for it to work.  When you open the page, you can start on a new BDC application definition (SQL or Oracle) or connect to an existing definition.  Unfortunately, the trial doesn’t allow you to connect to existing definitions so I couldn’t try that out, but I was able to quickly make a new application definition for a new table.  You start by providing a connection to the server, then pick the database and tables you want.  You can configure all of the usual things in the BDC such as the methods and actions.  One you are complete, there is a button to add the LOB system to MOSS.  Clicking this uploads the new BDC application definition directly into the BDC.  There is no need to save off the XML schema file and then manually upload it later.

All in all, its a pretty useful tool I would say.  It has a nice AJAX interface that makes building application definitions easy.  The only minor issue I really saw is that with tables with a large number of columns, they all did not fit nicely into the window.  It doesn’t look like you can build an application definition for a web service with this tool (but someone correct me if I am wrong).  I would also really like to see an export option or something of that nature to export the current or existing application definition.  If you work with the BDC a lot, give the tool a try the next time you need to do an application definition.

I am about to do start giving some training again, so I thought I would work on some more content for the blog that I could refer to people to when they are trying things out.  Today, I want to start out with the basic task of deploying a page to SharePoint.  There are actually lots of ways to get pages into SharePoint.  You can use the whole page publishing model in MOSS, but sometimes you just need a simple page with a simple web part deployed and a feature makes this task pretty easy.

For today’s example, I want to deploy a simple page with a web part or two on it.  To start, you will need to create a new folder for your feature.  You can name this whatever you want.  For the purpose of this example, we’ll call it PageTest.  Before we begin though we need a .aspx page which will host these web parts.  You can create one on your own, but I find it’s easiest to just “borrow” one from SharePoint that already exists.  A good one to borrow I have found is in the STS site template (12\TEMPLATE\SiteTemplate\STS) called default.aspx.  This .aspx page has two WebPartZones named Left and Right.  Put a copy of this file into your new feature folder.  You can customize this file however you desire, such as changing the layout or adding or subtracting WebPartZones.

By now, I am sure you probably know how to make a basic Feature.xml file, but here is one just in case.  In this case, we have a feature with a scope of Web so we’ll be deploying to an individual site.

The Elements.xml file deploys files using the Module and File elements.  The Module element tells SharePoint to deploy files from a physical folder, specified in the Path attribute, to a virtual folder inside SharePoint, specified in the Url attribute.  The Path attribute is relative to the feature folder while the Url attribute is relative to the site it was deployed on.  No paths are specified in our case.

The Module element can contain one or more File elements to deploy pages.  You need one File element for each page you want to deploy inside of SharePoint.  Note: that it is possible to use the same default.aspx page in your feature folder and deploy it multiple times with different web parts (i.e.: page1.aspx, page2.aspx, etc.).

The Name attribute specifies the name the file will be called inside SharePoint.  The Url specifies the name of the file in your feature folder.  In our case, default.aspx.  The confusing part her of course being that Url in the Module element refers to the path inside SharePoint and on the File element it refers to the name of the file in your feature folder.

For each page that you can deploy, you can use CAML to automatically add web parts to the page.  To do this, you use the AllUsersWebPart element.  Remember in the page we had WebPartZones named Left and Right?  We use those same Ids here, to specify which zone to add the web part too.  The WebPartZoneOrder element specifies the order in which the web parts are added to a given zone (you can have multiple web parts in a zone and they will stack).  The definition of the web part goes inside a CDATA element inside the AllUsersWebPart element.  If you built the web part yourself, you can just paste in the XML from your .webpart file.  If you are using an out-of-the-box SharePoint web part, you can add it to an existing page and export it or go to the web part gallery and export it.  Here is what a completed file looks like with multiple web parts.

When SharePoint is deploying the page, it combines your .aspx page with what is in the elements.xml file to output the completed page.  At this point you are ready to install and activate your feature.  You can put this in a solution package at this point to ease deployment.  When you activate the feature, you should get something that looks like this.

Of course, deactivating the feature won’t remove this page or its web parts.  However, you can always easily implement a feature receiver to cleanup the files you deployed.  This is how you deploy simple pages using CAML.  This is effectively the same way that Site Templates deploy pages as well (except the web parts and views are defined in onet.xml instead).