Corey Roth [MVP] : Business Connectivity Services

Slides from my talks at SPTechCon Boston 2011

CoreyRoth — Mon, 20 Jun 2011 21:54:46 GMT

This post is long overdue, but better late than never. I had a great time at SPTechCon 2011 in Boston. I had the chance to meet a lot of people in person for the first time that I had only previously known through twitter. I gave two talks this year. The first being my Instant ECM talk showing you how to leverage out-of-the-box SharePoint ECM features. My second talk was an advanced developer talk discussing how to build Business Data Connectivity models and custom connectors. My slides can be found on SlideShare. It was definitely great to get out to the east coast and see how people were using SharePoint.

Instant ECM with SharePoint 2010 - SPTechCon Boston 2011 Slides

Advanced BCS: Business Data Connectivity Models and Custom Connectors – SPTechCon 2011 Slides

Follow me on twitter @coreyroth.

Slides from my BCS talk at Houston TechFest posted

CoreyRoth — Tue, 12 Oct 2010 14:39:00 GMT

I had a great time at Houston TechFest this weekend. I met a lot of new people and I got to see a lot of old friends. At TechFest, I spoke about how to code advanced SharePoint BCS solutions using BCS entity models and custom connectors. The slide deck is posted on slide share. I’ve also attached the code for the custom connector I used at the bottom of this post. If you have any additional questions, feel free to ask.

How to: Debug a Custom Index Connector

CoreyRoth — Wed, 08 Sep 2010 16:17:51 GMT

I have been researching Custom Index Connectors in SharePoint 2010 over the past few weeks. I have been working with the MyFileConnector sample provided in the SDK, but after I made some changes, I realized it wasn’t immediately obvious how to debug my code. Luckily, @mwiselka helped me out and pointed me in the right direction. When it comes to debugging an index connector, timing is critical as well as picking the right process. In this case it’s not a matter of picking the right w3wp.exe process. It’s a matter of picking the right search process. If you don’t do the process right, your symbols will never be loaded and your break point will never be hit.

To get started, compile your connector and copy the assembly to the GAC. Now you need to restart the search service. It is critical that you restart the service any time you make a change or try to debug again. I would also do an iisreset as this seems to be required as well most of the time. If this is your first time installing the custom index connector be sure and follow all of the steps in the SDK which includes making registry changes and configuring the content source. I usually restart the service from PowerShell or a command prompt.

net stop osearch14

net start osearch14

The process we need to debug is called mssdmn.exe. When you restart the service you will only see one of these listed in your processes list.

You may be tempted to attach to that process right away and kick off a full crawl of your new content source. Don’t do that. It will not work. When you kick off your first crawl, a second instance of mssdmn.exe is launched. That is the process you attach to. I remind you that timing is critical. After you start the crawl, you need to refresh the process list quickly and attach it if you are trying to catch something early in the process. It really just depends on the speed of your server, sometimes it takes quite a while for the process to show up.

Notice the new process listed. Once you attach to the correct process, you should have no issue with your breakpoint being hit.

Now, you might be thinking, I’ll just wait and attach to all mssdmn.exe processes that show up. I thought that might work but it doesn’t. What I have found is that any time you attach to the original mssdmn.exe process, it causes your breakpoint not to be hit. I have no idea why it behaves this way, but from what I have seen so far, you only attach to the newly spawned mssdmn.exe process. If you are having trouble getting the timing right, one thing that @mwiselka recommended was to start a crawl of another content source to spawn the mssdmn.exe process and attach to it. Then when you do your crawl, it should use the same spawned process.

You would think that you could use something like System.Diagnostics.Debugger.Launch() like I have done with features, but I have had no luck getting that to work. You can see from my code snippet above, that adding a Thread.Sleep() can be a useful way to buy yourself time to attach to the process as well. It’s kind of hack, but it does seem to work. If the above technique doesn’t work, try it again. Recompile, restart osearch14, and iisreset. This technique can also be used with regular BCS Custom Connectors as well if you want to debug them while indexing.

Slides from my talks at NWA TechFest

CoreyRoth — Fri, 09 Jul 2010 19:32:00 GMT

I had the opportunity to give two talks at NWA TechFest yesterday in Rogers, AR. It was a pretty good event organized by the one and only @DavidWalker. I got to meet a lot of new people and talk about all sorts of things related to SharePoint. There were plenty of non-SharePoint people to talk to there too. As promised, I am making my slides available and they are already up on SlideShare.net.

How the SharePoint 2010 Business Connectivity Services will change your life

Introduction to SharePoint 2010 Enterprise Search

Thanks for coming and I’m looking forward to seeing a lot of you at Tulsa TechFest this fall. Also, I’m speaking at the Tulsa SharePoint Interest Group this Monday about PowerShell, so come on by.

How to: Create a BCS Profile Page in SharePoint 2010

CoreyRoth — Thu, 15 Apr 2010 20:36:42 GMT

I was working on updating my Search demo for #spshou and my BCS search results were really ugly. I didn’t have a default action specified on my search results, nor did I have a profile page, so instead the search results would give me the BDC protocol URL which is not usable by an end user.

In MOSS 2007, if you did not create a default action, it would automatically create a profile page for you. In SharePoint 2010, this is not the case. It’s pretty easy but there are a few steps involved.

Start by opening SharePoint Designer and open your external content type in question. In the top right, you will find the Create Profile Page button.

Give it a click. However, if you haven’t configured a profile host URL, you will get the following error. Chances are you haven’t configured one of these yet.

Unable to provision profile page because the profile host URL was not specified.

After doing a search, I found only one usable result on the Internet on MSDN forums. It had the answer, but it wasn’t clear to me at first. It said to use the Configure button in the ribbon. However, I was looking in SPD, and in fact its in the Service Application instead as you can see below.

Select the application you want to edit and then you can click Configure. This brings up the popup shown below which prompts you for a URL of the site that to keep you profile pages in. I like this since you can configure these at a per application definition level now instead of one location for every entity on the SSP.

I decided that I wanted my profile pages in a specific subsite, so I went ahead and created a new one.

After I finished configuring my site and setting up the Profile Page Host, I was able to go back to SPD and click the button. This pops up a progress bar, but it doesn’t really give you any visual indication that anything is done. I then went back to my content sources and recrawled my BCS content source. When it finished, I executed a new search, and now I get actual usable URLs in my search results.

As you can see it generated kind of a long URL with _bdc in it. I believe the http and sp2010 come from the namespace specified in SharePoint Designer. Finally it names a page after the entity itself. So far, I’m not sure if any of the page name stuff is configurable or not, so I need to look into that. Once you click on the page, you get a simple profile page to display the results.

It’s simple, but it has all the pertinent information from the underlying database table. It beats what I had before. I think a custom default action can be specified just like in MOSS 2007, but I get the feeling you have to edit the application definition XML manually to make it happen. I’m still looking into it. Anyhow, I hope this helps, the next time you try to use the BCS.

Mapping Enterprise Search managed properties from the BCS in SharePoint 2010

CoreyRoth — Thu, 15 Apr 2010 18:59:28 GMT

When I was mapping some managed properties from one of my Content Sources using the BCS, I noticed that the crawled properties were named slightly different in SharePoint 2010. I thought this was worth a quick note, so here we are. In MOSS 2007, each crawled property was prefixed by the application definition name, followed by the entity name. Now (as of Beta 2) everything seems to be prefixed by Read ListElement. Here is what some of my properties look like.

Notice it actually shows a space now instead of x0020. The naming convention appears to be static here so unfortunately you don’t really have a way of telling what entity each item is coming from. This might be a good thing though. Since before, if you had a field with the same name in 10 different entities, you had to map each one to the managed property. The prefix appears to always be the same too (even if you change the name of the operation). Read List is the name of the operation used to iterate a BCS list now as you can see in SharePoint Designer.

The name can be changed in SPD, however it appears that the prefix never changes as far as crawled properties go. Anyhow, I just wanted to quickly share this change I noticed today. I’ll be posting more on the BCS and Search soon.

Slides from my talk on the SharePoint 2010 BCS at Tulsa SharePoint Interest Group

CoreyRoth — Tue, 13 Apr 2010 14:45:04 GMT

We had a great turnout last night for my talk on How the SharePoint 2010 Business Connectivity Services will change your life. The talk went quite well despite the long wait times in SharePoint Designer 2010. :-) I really enjoyed giving this talk because it truly wows a lot of people. When they see how you can bring data in from a backend system, edit it in SharePoint and then sync it with SharePoint Workspace or edit it directly from Microsoft Outlook, they are truly impressed. I can’t wait to give this talk again on a release build. If you want to see my awesome slide deck, I’ve posted it on SlideShare. Thanks for coming.

Follow me on twitter.

Tulsa SharePoint Interest Group – How the SharePoint 2010 BCS will change your life

CoreyRoth — Fri, 09 Apr 2010 14:51:49 GMT

I’m excited to bring a SharePoint 2010 talk to the Tulsa SharePoint Interest Group this Monday (4/12). Here is a brief summary.

How many hours have your wasted building simple ASP.NET applications to do nothing more than simple CRUD operations against a database? Many tools have made this easier, but now it's so easy, you'll be up and running in minutes. This session will show you hot easy it is to get started integrating external data from your line of business systems in SharePoint 2010. You will learn how to register an external content type using SharePoint Designer based upon a database table or web service and then build an external list. With external lists, you will see how you can perform CRUD operations on your line of business directly from SharePoint without ever having to do manual configuration in XML files. Finally, we will walk through how to create custom edit forms for your list using InfoPath 2010.

For more information or to register, click here.

Also, Tulsa SharePoint Interest Group is now on twitter @TulsaSPIG.

Speaking about How Business Connectivity Services (BCS) may change your life in SharePoint 2010 at Tulsa SharePoint Interest Group on 4/12

CoreyRoth — Tue, 09 Mar 2010 23:00:09 GMT

Ok, so the title may be overly dramatic, but the BCS is really freaking cool now. With SharePoint 2010 just around the corner, now is the time to start learning how you can leverage it. I’ll be talking about the Business Connectivity Services at the Tulsa SharePoint Interest Group on 4/12. If you’re not familiar with everything yet, here is what has changed. The Business Data Catalog from MOSS 2007 had morphed into the new Business Connectivity Services also known as the BCS. To add to the confusion, there are places where you will still see term BDC lying around such as the case of the BDC entity model (although maybe the name has changed by now). With 2010, we get the ability to easily and quickly create new application definitions using SharePoint Designer 2010. On top of that the data we have abstracted through the BCS can be manipulated in something new called the external list. It’s so cool that in fact you can edit and do all CRUD operations on that external list and the changes get persisted to the backend database (or web service, etc). If you’re not familiar with the BCS yet, this is a must-see session.

If you are going to make it to the event, please RSVP. Thanks.

Impress the Boss with the SharePoint 2010 Chart Web Part

CoreyRoth — Wed, 24 Feb 2010 17:09:12 GMT

Let’s face it. Nowadays, management absolutely loves BI. Especially when there are lots of pretty charts and graphs. The thing is I’m not a BI guy. Cubes scare me to death. Luckily, there is the new snazzy Chart Web Part in SharePoint 2010 that gives you some BI like capabilities. It doesn’t allow you to drill down and pivot and do all that fancy stuff, but it does let you make some nice graphs and charts that any ordinary developer can do.

To get started, first you need an Enterprise version of SharePoint 2010. Unfortunately, I used the key that was later determined to be the wrong one which led me to reinstall SharePoint with a new key. If you don’t have any Enterprise options available, then you will be reinstalling. Assuming, you do have an Enterprise key, you then need to activate the SharePoint Server Enterprise Site Collection Features.

This adds the chart web part (among other things). Then edit any page and add a Chart Web Part. You can find it under Miscellaneous as of Beta 2 (they may have found it a new home in later versions).

When you add it to the page, it uses some dummy data and displays a simple bar chart.

At this point, you might be asking yourself “What kind of data can I bind this to?”. By clicking Data & Appearance, you will see the following screen which leads you to links to customize the appearance or bind to data.

We’ll start by going to Connect Chat To Data. Look at these great options we have to choose from.

We can connect to another web part, a list, an external content type (looks like they need to update the BDC wording they have there), and to Excel Services. I tried going straight to an external content type, but I got a yellow screen. Apparently that is broken on my version, but I’m sure it will work in RTM. There is a work around though. You can pick an external list using the Connect to a List option. This is exactly what I am going to do.

Remember that external list I created on my BCS blog post? I’m going to use that list and extend it some. What I did is I created a new custom list to contain Monthly Sales information. I used an External Data field to allow the user to pick a product from the external content type and enter in some sales data in a field.

This is the list we are going to use on our chart. On the next step of the wizard, we are allowed to pick a list. Note that it also allows you to choose other sites in the collection as well.

The next step allows you to filter your data first if you are so inclined.

The Chart Web Part has a ton of configurable options. On this last step is where you will start seeing some of them. The main thing to set here is your X and Y fields. You can also specify something to group by as well.

Once you finish this last step, you’ll get something that looks like this.

So, what we have here is a chart bound to data coming from a database (via external content type) and a SharePoint list. Pretty cool, right? I think so. It gets better though. What if your boss doesn’t like bar charts? No problem. There are tons of chart types to choose from. Just click on Data & Appearance again and then Customize Your Chart. Look at all of these built in chart types you have.

On the next step, you can further customize the chart you choose. It has some nice pre-built color themes and you can customize the size and what not. It also gives you a live preview as you change settings.

Here is what my new chart looks like.

You can customize things even more by going to Advanced Properties. It would probably take me a week to show you everything that this web part does, so I recommend you go try it out for yourself. The charts and graphs this thing produces are so cool, you should have no issue getting your boss to sign off on the Enterprise license. :)

It’s easy to configure an External List with Business Connectivity Services (BCS) in SharePoint Foundation 2010

CoreyRoth — Tue, 02 Feb 2010 23:02:01 GMT

One of my favorite features in SharePoint 2010 is External Lists. If you haven’t heard by now, the MOSS 2007 Business Data Catalog (BDC) has become Business Connectivity Services (BCS) in SharePoint 2010. The ever better part is that it is included in SharePoint Foundation and does not require SharePoint Server. One new concept that we get from this is the External List. To the end user, these look like regular lists in SharePoint, however they really are reaching out to external systems such as databases and web services. Whereas the BDC only “officially” supported read operations, the BCS supports full CRUD operations on your external data source.

The title of the article mentions how easy it is to set up. As you will see shortly, it really is quite easy using SharePoint Designer. In fact, if you have a database table, you can have it exposed in SharePoint in under 10 minutes. If you worked with the BDC before, you know that we had to manipulate huge XML files that represented the application definition. We relied on third party tools to make this easier, but it was never a clean experience until now. Before we get started with SharePoint Designer though, first make sure that you have the Business Data Connectivity (likely to be renamed) Service Application started. To verify this go to Central Administration –> Manage Service Applications.

You can tell this is SharePoint Foundation since there are only a few Service Applications (SA). Ok, admittedly it could be server with very few installed, but you get the point. If you click the Manage button, you can manage any application definitions you have created already. Although, we’ll skip this since we’ll be using SharePoint Designer. If the SA is not started for some reason, go to Central Administration –> Services on Server and click Start next to the appropriate service. Also remember if you are using Windows Server 2008 R2, you will need to have the WCF hotfix installed before any SA works.

Now let’s take a look at our table. In my case I have a table of products with a few columns. I want to expose this table to SharePoint and make it editable.

Now open SharePoint Designer 2010 and connect to your SharePoint site. My server is called sp2010, so I would click open site and then enter http://sp2010. SharePoint Designer is redesigned and has a new item on the left-hand tab called External Content Types. This is where we want to go. Click the External Content Type button in the New section of the ribbon.

It will take a minute, but then you will get some details about your new External Content Type. Here is what mine looks like after I gave it a name of Products.

We’re then going to click the link Click here to discover external data sources and define operations. This is where we pick that we want to pull data from my database table. We can also use this to connect to a web service or talk to a custom .NET object.

I’m just going to go with the defaults on the next screen for my data connection. I gave it my database name of bcs_test as well as my SQL Server name. You can also configure out which identity is used here to talk to the database. Remember that you may need to grant permissions on the SQL Server itself for the appropriate user. It will then iterate your data source and display it to you. Pick out the table you want, and then right click on it. This gives you a list of operations you can add as you can see below.

As you can see, you can add individual operations such as Read Item, Read List, Create, Update, and Delete. You can also create an association if you have multiple tables related to each other. However, I want everything, so I am going to choose Create All Operations. This starts a wizard. The most important part for you will be the Parameters configuration. Here you set what field to show in a picker control as well as what the id is. Usually, you don’t need to configure much. Here I am setting that the name field should show up in the picker.

It automatically detects my primary key and maps it to the identifier for me.

You can also specify an Office Property to allow you to map things into existing types such as an Outlook Contact or Task. This lets you use Office to edit things directly in the BCS. I won’t cover that today though since that is a little bit more involved. The next step in the wizard allows you to do some filtering. In my case, I am skipping it.

When that is finished, go back to the main tab for your External Content Type and you should see something like this. It has the operations you have chosen as well as what fields are in the type.

Now, we want to save out External Content Type, by pressing the Save (Disk Icon) in the top left corner. On to the fun part. Let’s use SharePoint Designer to create an External List by using the Create Lists & Form button in the ribbon.

Clicking it you will get the following screen asking about your list. I’m calling my lists Products and I just use default values for the rest. If you are using server, you can click the Create InfoPath Form checkbox and you will be able to customize the InfoPath form for the list right there. Since we are using Foundation today, I’ll leave that unchecked.

You can also create the list directly through SharePoint as well. Once the list is created, go to your SharePoint site, navigate to the list, and you should see something that looks like this.

Comparing it to my table from the SQL Server, you can see that it matches. You can see that it looks a lot like a regular list in SharePoint. Now, what if I want to change the price on the Plush Bear? Not a problem. Click on the item and select edit, change the value in the form and save it.

Now what does the data in SQL look like?

It has the new price of course. You can also add and delete rows and perform bulk operations, but screenshots really don’t do it justice. It’s this simple to set up, you should go out and try it for yourself. One reminder, I will give you is that even though this looks like a list, it doesn’t always quite act like a list. You won’t find it in SPWeb.Lists for example. There are a also a number of other things that do and don’t work. Be on the lookout for a follow-up post on what you can and can’t do with an external list. This is a great new feature in SharePoint 2010 and I hope you will like it as much as I do.

Features I am looking forward to the most in SharePoint 2010

CoreyRoth — Mon, 13 Jul 2009 15:47:36 GMT

I’m not privy to the NDA, so I got to look at some of the new SharePoint 2010 (#sp2010) information for the first time with the Sneak Peak videos on the Microsoft site. I’ll try not to just repeat information in the videos, but tell you what I am looking forward to and make comments. I am sure everything they have stated so far is subject to change, but if half of it even gets implemented we’ll be in good shape. As I was watching the admin video, the first thing that I noted is that there will be a logging database. This appears to replace the need to find errors in the 12 hive’s LOGS folder. This is very exciting and should make it much easier to track down problems. Also of interest is that there is much improved export support and you will be able to backup specific sites and lists. This one is a no-brainer and should have always been included to begin with.

Another thing I saw was various things to help support large lists. The first thing being an admin configurable threshold on allowing how many items can be displayed in the default view at a time. The default was 5000 in the demo, so I am wondering if this will be the new suggested limit as opposed to the existing 3000 item limit today. What is cool is that the interface will display all of the items for an administrator but it will notify a regular user that too many items have been returned and that they need to use a filter.

Visual Studio 2010 looks like it will be a great experience for developing SharePoint solutions. Out-of-the-box there is built in support for editing all types of things in SharePoint including importing workflows created by SharePoint Designer and existing .wsp packages. Building web parts no longer requires generating HTML via code due to the new Visual Web Part Designer. It appears they created a new typed called VisualWebPartUserControl which inherits from UserControl. You can drag and drop controls right onto the design surface and then easily deploy the web part with minimal effort. There are lots of designers and tools for working with the features and solution package itself, but it appears it takes care of most everything for you while still allowing you to customize things when you need to.

The changes to the Business Data Catalog really excite me. The BDC is now known as Business Connectivity Services (BCS). Application definition files can be created easily with Visual Studio 2010 or SPD. A visual design surface is available and allows you to easily create an application definition for an entity which comes from a database, web service, or .NET object. It also now has true insert/update/delete support and will create the methods in your application definition so that you can use that functionality later in a list. The new External List feature allows you to associate this application definition with a list and perform all of the CRUD operations on it just like it was a regular SharePoint list. This will make it very easy to integrate external data into SharePoint. My only question with this that comes to mind is there a way to customize the edit form or will that result in an unsupported scenario.

Some other cool things about the BCS is that it integrates with Office. Document templates can be created and data can be retrieved directly from the BCS to fill in values in a document. Microsoft Groove has been renamed SharePoint Workspace and provides a graphical interface for working with SharePoint. On top of that it provides the capability of syncing entire sites offline including BCS data. Documents can be updated and LOB data can be changed and then it can be synced back to SharePoint. I think this will provide great functionality for any type of field or remote users who are occasionally connected.

As a developer, another feature I was really excited about was LINQ support for SharePoint lists. You will be able to point the spmetal tool at a SharePoint list to generate a strongly typed data context class. You can then query the list as you would anything else using LINQ. I took a look at the CTP of the Developer Documentation today as well. It looks like all of the collections now have a method called GetTypeEnumerator(T). This returns an IEnumerator<T> which means any collection that implements this can also be queried with LINQ. I am really hoping this eliminates CAML queries, however I did see in the Client Object Model demo, that they still used a CAML query there. Since I brought up the Client OM, I’ll mention that there is a client object model available now for SharePoint which will make things like integrating with Silverlight very easy.

Unfortunately, they haven’t really produced any information about the changes in Search yet. I am looking forward to hearing most about those. Hopefully, they completely scrapped the Search Center and started over. I am wondering if there are any changes in the Records Center too. I also wonder how this will affect partners that have built tools around MOSS 2007. I can see Lightning Tools BDC Meta Man and AvePoint being directly affected. However, I am sure there will still be new areas for these companies to explore with the new product. This is an exciting time and I can’t wait for the public beta and to hear more about the product. Unfortunately, it’s not looking like I will make it to the SharePoint Conference this year, so I will have to be getting a lot of information second hand.

On a related note, don’t forget about the Tulsa SharePoint Interest Group tonight.

Follow me on twitter.