Using features to create Choice site columns is pretty straightforward (check this post for additional information). See below a sample feature element for that purpose.
<Field
ID="{538c71e4-8650-4ce7-b021-920effa66346}"
Type="Choice"
Name="Publishing_x0020_Status"
StaticName="Publishing_x0020_Status"
DisplayName="Publishing Status"
Required="FALSE"
Format="Dropdown"
FillInChoice="FALSE"
Group="Custom Columns">
<Default>Not Published</Default>
<CHOICES>
<CHOICE>
Not Published
</CHOICE>
<CHOICE>
Pending
</CHOICE>
<CHOICE>
Published
</CHOICE>
<CHOICE>
Error
</CHOICE>
</CHOICES>
</Field>
Although the XML is well formed and all the attributes and values are correct, this field definition has a problem. If you create a site column using this definition, here is what happens:
- If you check the site column configuration in SharePoint, everything will look fine
- If you add this site column to a list (either through a content type, or directly) and you edit an item using the default list forms, everything will work as expected.
- If you try to edit the list items using a DataSheet View, you won’t be able to select any value from this choice field and SharePoint will always throw a validation error.
The problem here are the invisible characters (spaces, newlines and tabs) between the <CHOICE> and </CHOICE> tags and their inner values. Apparently, when using list forms these characters are trimmed from the valid choices, but when using the datasheet view they are not, causing a strange behavior when editing an item in that view.
The correct definition for this field would be:
<Field
ID="{538c71e4-8650-4ce7-b021-920effa66346}"
Type="Choice"
Name="Publishing_x0020_Status"
StaticName="Publishing_x0020_Status"
DisplayName="Publishing Status"
Required="FALSE"
Format="Dropdown"
FillInChoice="FALSE"
Group="Custom Columns">
<Default>Not Published</Default>
<CHOICES>
<CHOICE>Not Published</CHOICE>
<CHOICE>Pending</CHOICE>
<CHOICE>Published</CHOICE>
<CHOICE>Error</CHOICE>
</CHOICES>
</Field>
There is an issue with SharePoint 2010’s CAML parser that causes SharePoint to ignore fields in a Content Type definition. When defining a Content Type feature element, avoid placing comments inside the <fieldrefs> element as that will cause SharePoint to create the Content Type disregarding all the fields, even though the XML is well formed.
<?xml version="1.0" encoding="utf-8" ?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
<ContentType
ID="0x0100C5647A362F236548B218C15302286758"
Name="MyCT"
Description="Simple Custom Content Type"
Inherits="TRUE"
Overwrite="TRUE"
Version="0">
<Folder TargetName=”_cts/MyCT” />
<FieldRefs>
<FieldRef ID="{fa564e0f-0c70-4ab9-b863-0177e6ddd247}" Name="Title" />
<!-- This is a comment –->
<FieldRef ID="{b402db15-ee44-4ec4-89e3-23e10a8fc64c}" Name="My_x0200_Field" />
<FieldRef ID="{538c71e4-8650-4ce7-b021-920effa66346}" Name="Status" />
</FieldRefs>
</ContentType>
</Elements>
The Content Type feature element above will cause the Content Type to be created with only the Title field, which is inherited from the parent Content Type. All the fields referenced in this Content Type definition will be ignored.
The European SharePoint Conference organization committee has been setting up a series of free SharePoint-related webinars in preparation for the conference. I’ll be presenting one of these webinars, next Tuesday, June 14th 2011, at 12PM CET, about LINQ to SharePoint.
The abstract for this webinar is as follows:
Regarding data access, one of the new features introduced with SharePoint 2010 is the possibility to use LINQ to manipulate list items. However, despite being immensely useful, this technology has its own limitations. During this webinar André will talk about what you can and can't do with LINQ to SharePoint, as well as the scenarios where this technology should or should not be applied.
Although it is a free webinar, you must register to be able to attend.
A few weeks ago I had an article of mine published on PROGRAMAR magazine, issue number 28, a Portuguese developer-focused magazine. Here is the translated version of the article.
SharePoint 2010 is a complex platform with an impressive set of out-of-the-box features which allow it to fit a great variety of situations. One of the new and most powerful features is called Business Connectivity Services and this article is an introduction to this technology and its potential.
What are Business Connectivity Services and what are they for?
Business Connectivity Services (BCS) is the name of the technology that allows SharePoint 2010 and Office 2010 to read from and write information to external systems. It’s an evolution of the Business Data Catalog (BDC) technology introduced in SharePoint 2007, and on which several improvements were made, namely:
- Possibility not only to read, but also to write information to the external data source;
- Support for more complex authentication scenarios;
- Support for multiple data sources;
- Integration with Office client applications;
- New and better ways to present the data;
- Tools specifically focused on creating and manipulating models;
- Extensibility through .Net assemblies.
The main goal of this technology is to allow the integration of information from external systems and present it on SharePoint and Office client applications with the minimal effort possible and, ideally, without writing a single line of code. There are, in fact, some scenarios where you can use BCS technology only through configuration, but the true flexibility lies in its extensibility which allows it to fulfill almost any integration need.
Architecture
BCS technology is not limited to a service or API inside SharePoint 2010. In fact, it’s a set of components, services and tools as presented in the schema below.
Fig.1: Business Connectivity Services Architecture
Business Data Connectivity (BDC) Service
The Business Data Connectivity Service is one of the core components of BCS. It’s the component which stores, in its central metadata repository, the descriptions of the information which we want to access as well as the external systems that holds it.
Metadata Store
The metadata repository is the database used by the Business Data Connectivity Service to store the descriptions of the information and the external systems that hold it. This repository does not store any of the information from the external systems, only the metadata required to retrieve it.
Connectors
The connectors are the pieces the allow the Business Data Connectivity Service to connect to the external data sources described in the models stored in its Metadata Store. Three connectors are included in the product:
- Database Connector – allows the connection to SQL Server databases.
- WCF/Web Services Connector – allows the connection to WCF services or web services.
- .Net Assembly Connector – allows connections using a custom developed .Net assembly, allowing access to virtually any external data source, including simultaneous connections to multiple data sources.
The connector mechanism is extensible and allows custom connectors to be developed, for cases when the out-of-the-box connectors are not enough.
BDC Client Runtime
Office 2010 applications are also able to expose information from external systems through BCS. That is possible because Office 2010 includes the BDC Client Runtime, a component that, in the client application context, performs the same tasks that the BDC Service performs in the server context. That is, it accesses the metadata repository and, through the definitions stored in it, it accesses the external information itself.
Client Data Cache
To improve performance when accessing information, as well as to support offline access to information, the BCS use a cache to store the retrieved external data. This cache is based on a SQL Server 2005 Compact Edition database and has an automatic synchronization mechanism that allows all changes performed on the data in offline mode to be replicated as soon as the external system becomes available once again.
Core Concepts
Now that you know the architecture for Business Connectivity Services, it’s important to understand what kind of metadata is stored in the Metadata Store by the Business Data Connectivity Service.
Model
The metadata used by the BDC Service and stored in the Metadata Store consists of XML files which describe Models, usually named BDC Metadata Models. In SharePoint 2007, these metadata files were called application definition files.
A model contains, in declarative form, all the necessary information so that the BCS are able to connect to an external system and retrieve the data stored in it.
Lob System
Regarding BCS, the Lob System (or Line-of-Business System) refers to the external system which stores the data we want to access. This system can be a relational database, or any other system that exposes that data through web services or WCF services.
External Content Type
The External Content Type (ECT) is the core and most important concept of the BCS, since it describes a business entity, that is, it describes the structure and behavior of the data we want to access. Examples of ECTs are Client, Invoice or Employee.
The ECT definition specifies the structure and behavior of the entity, that is:
- The fields that compose an instance of the entity and their respective data types. Example: Name, Address or Country.
- The mapping of these fields to objects used by Office client applications. Example: the field Name of the entity maps to the FullName field in Outlook.
- The methods that must be invoked by the BCS to read, create, update and delete instances of the entity. These methods might correspond, for instance, to stored procedures, SQL queries or web services.
In the context of a Model, we can define several entities, that is, several External Content Types.
Methods
The methods are abstractions of the external system’s API and they allow BDC Service to know which stored procedures or web services to call when manipulating the data. The definition of a method is created in the context of an entity and is always based on one of the stereotypes supplied by the BCS. There are about 20 stereotypes, such as Updater (to update an item), Finder (to list items), SpecificFinder (to retrieve a specific item), Deleter (to delete an item) among others.
Filters
Filters describe the parameters that can be passed to the methods in the definition of each entity. There are 18 types of filters that can be used in methods and which let the BDC Service know which information should be passed to them. Examples of filters are the UserName, which passes the username of the current user, or the Limit, which defines the maximum number of items that are returned in a single method call.
Associations
An association is a relationship between two entities (External Content Types). However, since there is no guarantee that the data source is a relational database, the association requires that a special method exists. This method allows the BCS to retrieve elements of the related entity from elements of the source entity.
Model Sample
Fortunately, in most cases, we don’t need to manually edit the model since the tools supplied by Microsoft allow us to visually perform most of the configurations. However, just as an example, a model file looks like the one shown below.
Fig.2: Partial sample of a BDC Model
Presenting External Data
But all this complexity has a purpose – to allow the presentation and manipulation of the data stored in external systems – so there are several alternatives to do so.
External List
The External List is a new type of list in SharePoint 2010 which, being bound to an External Content Type, allows the visualization and manipulation of the data exposed through that ECT as if it was stored in a regular SharePoint list. In reality, the data still resides in the external system and it’s read and manipulated in real time.
Fig.3: External list
The greatest advantage of External Lists is the fact that they look and feel exactly like regular lists and, additionally, SharePoint’s object model treats them as such, allowing developers to read and write list items as if they were stored in SharePoint.
On the other hand, not all works exactly as in traditional lists. In particular:
- Workflows
- Alerts
- Folders
- Attachments
- RSS Feeds
- Export to Excel
Before an ECT can be used in an External List, it has to define, at least, a Finder method (to list items) and a SpecificFinder method (to retrieve a specific item). This allows the External List to present the list of items and the detail for each one. Additionally, if the ECT has an Updater method (updates an item), a Deleter method (deletes an item) and a Creator method (creates a new item), the External List will make the corresponding actions available.
External Data Column
The External Data Column was already around in SharePoint 2007 and, although it has been slightly improved in SharePoint 2010, its objective remains the same – allow the use of external data as a list column. It works similarly to a lookup column, allowing the user to select one of the items return by the ECT.
Fig.4: External Data Column
One of the advantages of External Data Columns is the possibility to used them in Word 2010, allowing the user to select one item exposed through the BCS and use that data inside the documents.
Just like for External Lists, to use an ECT in an External Data Column, the ECT must define, at least, the Finder and SpecificFinder methods.
Business Data Web Parts
Business Data Web Parts are a set of web parts that are able to connect to external data sources through an ECT and present that information in SharePoint. These web parts were also present in SharePoint 2007 but were improved in SharePoint 2010, now being able to cache the external information for better performance.
Fig.5: Business Data Web Parts
The Business Data Web Parts use XSLT to present the information, which grants them enormous flexibility regarding its graphical appearance as well as the possibility to be edited in SharePoint Designer 2010.
The web parts included in this set are:
- Business Data List – lists instances (items) of an entity (ECT).
- Business Data Item – presents the details of an instance (item) of an entity (ECT).
- Business Data Item Builder – uses query string parameters to create an instance (item) of an entity (ECT) which can then be passed on to other web parts, such as the Business Data Item web part.
- Business Data Related List – lists instances (items) of a related entity (ECT). It’s especially useful to present data in Master/Detail scenarios.
- Business Data Connectivity Filter – filters the data retrieved from an ECT before it is consumed by another web part, such as the Business Data List web part.
- Business Data Actions – presents the available actions for an instance (item) of an entity (ECT).
Search
One of the largest benefits offered by the BCS is the possibility to index and search external data exposed through the ECTs as if it is stored in SharePoint lists.
For an ECT to be indexed it has to define, at least, the IDEnumerator method and the SpecificFinder method. The first will allow SharePoint to retrieve the IDs of all the items and the second to retrieve each item’s detail. Additionally, the model must have the ShowInSearchUI property so that SharePoint can use it for indexing.
But this is for the data to be indexed. To allow the users to see detailed information about a specific item when they click on a search result, we also need to configure the Profile Page for each indexed ECT.
A Profile Page is just a page in SharePoint with a few predefined web parts which receives the item identifier through a query string parameter and presents detailed information about that item, including items of related ECTs (through Associations).
Profile Pages are configured in Central Administration, accessing the Business Data Connectivity Service management page. The only information we need to supply is the URL address of the website where these pages will automatically be created, and SharePoint will do the rest for us.
At this point, it suffices to tell the Search Service that it must index a new Content Source of type Line of Business Data and then start a Full Crawl. After the crawl finishes, the external data is available to be searched and each item’s information will be presented in its respective Profile Page.
User Profiles
Using the BCS, SharePoint 2010 can use external data sources to complement User Profile information. To do that, you just need to able to map User Profiles to items of an ECT, using a field on each side (such as the username).
It is not possible to configure an ECT as a main data source for the profile synchronization, but you can complement an Active Directory synchronization with data retrieved from an ECT.
Office Client Integration
The integration of external data in Office 2010 applications is also one of the new SharePoint 2010 features in what regards the Business Connectivity Services. Until now, this kind of functionality was only possible through considerably complex custom developments.
With the BCS it’s possible to present external data in Office applications, use it in offline scenarios and, in specific cases, update the data directly in the external data source. However, not all the Office 2010 applications have native support for such integration. At the moment, only Outlook 2010, Word 2010, Access 2010, InfoPath 2010 and SharePoint Workspace 2010 are able to do it, being that each one uses this technology in a different way.
Outlook 2010
Outlook 2010 is one of the Office applications that profits the most from the client features of the BCS. To be able to view the data exposed through an ECT in Outlook 2010, two steps are required when configuring that ECT:
- Defining what is the type of data exposed by the ECT among the types of data supported by Outlook: Contacts, Tasks, Appointments or Posts. This configuration can be performed in SharePoint Designer or directly in the Model’s XML.
- Mapping the ECT fields with the Outlook fields used in that type of data. For instance, specify which fields of the ECT map to the fields Last Name, First Name, E-mail Address and others, in Outlook.
 | If there is an External List which exposes the data from the ECT, it will have a Connect to Outlook button in its ribbon. By pressing that button SharePoint will analyze the ECT specification and include it in a Click Once installation package. That package will be immediately deployed to the user’s Outlook 2010 as an Office Add-In. |
Once the package is installed, the list will appear in Outlook’s interface allowing the user to interact with the external data as if it was a list of regular contacts, tasks, appointments or posts. In case the ECT defines the required methods, you can even use Outlook to update the data from the external data source. All the fields exposed by the ECT which are not mapped to fields of the Outlook object are shown in a separate section of that object’s detail and can also be updated.
Fig.6: Additional information (unmapped) from the ECT in Outlook’s interface
Like all the other Office applications, Outlook takes advantage of a data caching and synchronization mechanism allowing the user to work offline and automatically synchronizing the data as soon as the external system becomes available.
Word 2010
Word 2010 is another of the Office applications which has native support for the BCS. However, the use cases for this technology in Word are different from the ones available in Outlook. Using BCS in Word 2010 is limited to inserting external data in documents through Quick Parts.
For those who never heard of them, Quick Parts are a Word feature which allows the creation of dynamic fields in the middle of a document’s text. These fields can be automatically filled with data from the document’s content type, in SharePoint. This includes data from an External Data Column that exists in the Document Library where the document is stored.
Here’s how it works:
- In a Document Library, create an External Data Column configuring it to expose the data of a specific ECT and defining the fields of the ECT which are exposed.
- Create a new document in that library, using the New button from the ribbon.
- In Word, through the Insert ribbon, insert one (ore more) Quick Part, selecting the Document Property(s) that match the external data you wish to include in the document.
Fig.7: Quick parts with external data
- Word will allow the user to select an item from the ECT, using the Entity Data Picker, and will automatically populate all the related fields.
Fig.8: External Data Picker
Access 2010
Access 2010 is able to import a Business Data Connectivity (BDC) model an present the external data as regular database tables. However, the created tables are read-only, which means you cannot write back to the external data source.
InfoPath 2010
When an External List is created, the corresponding forms for viewing, adding and editing the external data are also generated. By default, these forms are generated as regular ASP.NET pages but, using SharePoint Designer or the External List ribbon, it’s possible to create smarter forms using InfoPath. The forms are also automatically generated but they can later be modified using InfoPath.
It is also possible to drag an External Data Picker to an InfoPath form and define an External List as a data source, thus allowing reading from and writing to external data sources.
SharePoint Workspace 2010
 | SharePoint Workspace 2010 is the evolution of Groove 2007 and is positioned as the tool for offline access to data stored in SharePoint 2010, including External Lists. Just like for the other types of lists, you just need to click the Sync to SharePoint Workspace button so that their content is downloaded to the user’s machine becoming available when she is disconnected from the server. |
Regarding BCS, what SharePoint Workspace does is download the ECT definition associated with the External List and store it locally, as well as the list forms for inserting, editing and viewing the data which were generated for that External List. Just like the other described Office applications, SharePoint Workspace uses the local cache to ensure the availability of the external data even when the external system is not available.
Solutions and Tools
One of the most frequent complaints of who used Business Data Catalog in SharePoint 2007 was the lack of proper tools that allowed for a good experience when creating and manipulating metadata (models). Microsoft took notice of those complaints and supplied us with two fantastic tools to use with Business Connectivity Services.
- SharePoint Designer 2010
- Visual Studio 2010
SharePoint Designer 2010
SharePoint Designer 2010 is a free and mandatory tool for any SharePoint power user or developer. It includes a large set of features mainly focused on creating no code solutions, that is, SharePoint customizations that do not require any custom code.
Fig.9: Using SharePoint Designer to manage ECTs
Regarding Business Connectivity Services, SharePoint Designer 2010 allows:
- Creating and manipulating External Content Types, including changing configurations, creating new methods and mapping ECTs with Office objects. When creating ECTs you are only able to connect to SQL Server databases, web services whose schemas are supported by BCS or existing .NET assemblies.
- Creating and configuring External Lists based on existing ECTs.
- Generating and editing InfoPath forms to support External Lists.
- Using external data in workflows.
- Creating web part pages and profile pages.
These features allow the use of BCS without any custom coding and fulfill the most common and simple requirements.
Visual Studio 2010
With Visual Studio 2010 we can create more complex solutions for cases in which SharePoint Designer 2010’s features are not enough. Additionally, with Visual Studio 2010 we can create reusable components which can then be incorporated in solutions through SharePoint Designer.
Fig.10: Visual editor for BDC Models in Visual Studio 2010
Some of the use cases for Visual Studio 2010 are:
- Creating and manipulating External Content Types using the new Business Data Connectivity Model project template. This template includes a set of designers that allow for the visual editing of the Model and respective ECTs, and also the development of .NET code solutions to access virtually any type of external data source.
- Creating reusable components for BCS using the multiple API extensibility points, such as Code Actions which can be used inside Outlook, External Data Parts which can be used in declarative task panes in Outlook, custom workflow activities and others.
- Creating Office application add-ins with BCS support, using the BCS object model.
- Creating custom workflows that leverage data from external lists or use the BCS object model.
Types of Solution for each Tool
The table below will help you select the best tool for each need.
| Capability | SharePoint Designer 2010 | Visual Studio 2010 |
| Connect to… | - WCF/WS, SQL Server and existing .NET assemblies
| - Any data source through .NET assemblies
|
| Development model | | |
| Suited for… | - Simple models with native interfaces (External Lists, Outlook, SharePoint Workspace, InfoPath, Search) and simple associations based on foreign keys
| - Complex models with custom connectivity logic for aggregation, transformation, security, etc.
- Custom interfaces via Office customizations
- Reusable client and server controls that connect to external data
|
| Limitations | - The data source must expose interfaces with supported format
- No support for advanced stereotypes (bulk operations)
- No support for generic or polymorphic services
- Only foreign key associations
| - Visual designer only works for models based on .NET objects
- Separate development and packaging for client and server components
|
Use Cases
One of the most common questions regarding the use of Business Connectivity Services is what are the use cases of this technology or, when should I use BCS.
Some of the most common uses cases for BCS are:
- Presenting information stored in a SQL Server database. Using BCS it’s possible to present and, if necessary, modify the information using External Lists without developing a single line of custom code. It’s like an instantaneous back-office.
- Enriching the domain user’s Profiles with information from the HR management system or the ERP system. As mentioned before, BCS allows you to fulfill this requirement by configuring an ECT as an additional data source for SharePoint’s profile synchronization.
- Synchronizing contacts stored in a line-of-business system or ERP. Using BCS it’s possible to define an ECT which exposes these contacts through an External List and connect it to Outlook where they can be managed as regular contacts. This scenario allows the users to access to the contacts even when they’re out of the office, in offline mode.
- Presenting information from distinct data sources. Using the .NET assemblies connector and developing an ECT with Visual Studio 2010, we can build scenarios with aggregation of data from multiple data sources exposed as a single entity.
- Indexing and searching information stored in a line-of-business or ERP system. BCS allows SharePoint’s search service to index contents exposes through ECTs and searching them as if the information is stored in SharePoint.
There are several other scenarios where BCS can be useful, sometimes just as one of the components of the solution.
Features per SharePoint Version
The infrastructure used by the Business Connectivity Services is available in all SharePoint versions, including SharePoint Foundation 2010. However, not all is included in the free version. The table below helps to clarify which features are included in each version of SharePoint 2010.
| Feature | SharePoint Foundation 2010 | SharePoint Server 2010 Standard | SharePoint Server 2010 Enterprise |
| BDC Service | X | X | X |
| Connector Framework | X | X | X |
| External List | X | X | X |
| External Data Column | X | X | X |
| Secure Store Service | | X | X |
| External Data Search | | X | X |
| Profile Pages | | X | X |
| Business Data Web Parts | | | X |
| Office Client integration | | | X |
The support for Business Connectivity Services in Office applications requires Microsoft Office 2010 Professional Plus, or higher.
Useful Links
Here are a few useful links for who is now starting and wants to know more about Business Connectivity Services.
Microsoft Business Connectivity Services Team Blog
Official blog of the team that developed BCS, with several articles with multiple levels of complexity. Mandatory to all the interested in this technology.
http://blogs.msdn.com/b/bcs/
BCS Team Channel
The YouTube channel where the product team published a few videos.
http://www.youtube.com/user/MOSSBCSTeam
Connecting to a .NET Framework Source Using Business Connectivity Services in Office 2010
Visual How To about developing an ECT using Visual Studio 2010 to retrieve data from an external source.
http://msdn.microsoft.com/en-us/library/ff394331(office.14).aspx
Microsoft Business Connectivity Services
SharePoint 2010 SDK’s section dedicated to Business Connectivity Services.
http://msdn.microsoft.com/en-us/library/ee556826.aspx
Business Connectivity Services Resource Center
TechNet Resource Center dedicated to Business Connectivity Services.
http://technet.microsoft.com/en-us/sharepoint/ee518675.aspx
Business Connectivity Services: Technical Articles
MSDN technical articles related to Business Connectivity Services.
http://msdn.microsoft.com/en-us/library/gg481768.aspx
I’m pleased to announce I have been chosen as a speaker in the European SharePoint Conference 2011, which will take place in Berlin, 17th – 20th October 2011.
I’ll be delivering 3 sessions, two on Wednesday and one on Thursday, as described below.
W18 - Using BCS to Integrate Azure Services with SharePoint 2010
The new Business Connectivity Services (BCS) offer a simple and elegant way of integration SharePoint 2010 and Office 2010 with virtually any business data source. All the business integration features introduced in SharePoint 2007 with Business Data Catalog (BDC) were significantly improved in this new version. In this session we'll talk about the main concepts and the type of solutions that can be developed using BCS, and we'll show how to integrate Azure Services with SharePoint 2010 and Office 2010 using BCS, Visual Studio 2010 and SharePoint Designer 2010.
W26 - Federated Authentication in SharePoint 2010
SharePoint 2010 introduces a new claims-based authentication model which leverages the power and flexibility of the new Windows Identity Foundation. In this session we will show how to use Windows Azure Access Control Services to allow a user to authenticate in SharePoint using a web identity such as Facebook or Google account.
Th28 - SharePoint + Azure = Better Together
Microsoft has a large range of technologies that, when combined, can yield fantastic results with surprisingly little effort. What happens when you mix SharePoint 2010, BizTalk 2010, Windows Azure and Microsoft Pivot? You get a fantastic interface to analyze large amounts of visual data which can be store in SharePoint 2010 and manipulated in the cloud. In this session we'll show the concepts behind this kind of integration and how you can build such a scenario.
Note, however, that the conference program is still subject to change. Check the full program at http://www.sharepointeurope.com/conferences/2011/conference-at-a-glance.aspx.
I look forward to seeing you there!
If you’re into SharePoint and you happen to be in Lisbon next Saturday, April 16th, then you may be interested in the 10th Meeting of the Portuguese SharePoint Community (SPUG PT). The meeting is free and it will take place between 9.30AM and 1.00PM at Microsoft Portugal’s auditorium, in Porto Salvo (Tagus Park).
Program
09.30AM – Reception
10.00AM – Introductions
10.30AM – Session: “LINQ to SharePoint” by André Vala
11.30AM – Coffee break
11.45AM – Discussion panel
01.00PM – End of meeting and lunch (optional)
If you wish to attend, please register here: spugmeeting10.eventbrite.com
I hope to see you there :)
If you’re developing a SharePoint-based public Internet portal, you’re probably using the BLOB (Binary Large Object) cache. This caching mechanism stores a copy of the binary files (images, documents and such) in the file system of each front-end web server. This can save a lot of database accesses, thus improving the performance of the web site.
As for any caching mechanism, it is possible that the BLOB cache becomes out-of-sync with the content database, storing an old version of a file that has been updated in the content database.
The quickest way to re-sync the BLOB cache is to flush it and let SharePoint repopulate it with the current version of the binary files. Flushing the BLOB cache can only be done through a PowerShell script (since there is no place for it on the SharePoint UI):
$webApp = Get-SPWebApplication "<WebApplicationURL>"
[Microsoft.SharePoint.Publishing.PublishingCache]::FlushBlobCache($webApp)
Write-Host "Flushed the BLOB cache for:" $webApp
In which <WebApplicationURL> is the URL address of the web application for which you want to flush the BLOB cache.
You can find additional information in: http://technet.microsoft.com/en-us/library/gg277249.aspx
Using SharePoint Tools for Visual Studio 2010 allows you to build SharePoint solution packages very easily. When you create a SharePoint project, Visual Studio allows you to specify if the assembly for that project should be included in the package and, in case it does, where you want it to be deployed to.
By default, the assembly is included in the package and is deployed to the Global Assembly Cache. You can change both configurations by clicking on the project in Solution Explorer and editing the properties on the Properties window.
However, there is no project property that allows you to specify you want to add the assembly to the Safe Controls section on the web application configuration file. To do this, follow the steps below:
1. On Solution Explorer, click on the project
2. On the Properties window, specify you don’t want the assembly to be included in the package (set Include Assembly in Package as False)
3. On Solution Explorer, double-click the Package folder to open the Package editor
4. On the Package Editor, click the Advanced button (bottom of the screen)
5. Click the Add button
6. Select Add Assembly from Project Output…
7. Select the current project (which contains the package)
8. Add the safe controls information
If you’ve created SharePoint projects using SharePoint Tools for Visual Studio 2010, you probably know that, when you create a SharePoint Feature and generate the SharePoint package (WSP file), the final name given to that feature is not what you expected.
If you create a new SharePoint project named MyProject, and then create a feature inside it named MyFeature, the final name and folder for the feature will be MyProject_MyFeature.
I don’t know about you, but I like the features to have the names I give them, without the project’s name added to them. The good news is that you can change it :)
Here’s how:
- On Solution Explorer, double-click the Feature name to open the Feature Editor
- On Packaging Explorer, click on the Feature name
- On the Properties window, change the value in the Deployment Path property
By default, the value for this property is:
$SharePoint.Project.FileNameWithoutExtension$_$SharePoint.Feature.FileNameWithoutExtension$
To remove the project’s name from the Feature name, just remove the first token (and the trailing underscore), leaving it as:
$SharePoint.Feature.FileNameWithoutExtension$
Creating an optimized SharePoint development environment was not an easy task with Sharepoint 2007, but it’s even harder with the new SharePoint 2010. The new version has a lot more features but, because of that, it’s also much more resource demanding, and that will make you sweat when building a new virtual machine to perform well on a laptop.
Following Raul’s post on SharePoint 2010 Tips for Performance, and because there seems to be a lot of people in the community facing the same issues I have, I'll be describing in this article some recommendations and best practices that I’ve been collecting here and there.
Some of these recommendations apply not only to development virtual machines, but also to non virtualized production servers.
Creating the Virtual Machine
When creating the virtual machine, you should take the following hardware requirements into account:
- Allow your virtual machine to use all CPU cores available in the host
- Allow your virtual machine, at least, 4GB of RAM
- Place your virtual machine in the fastest hard drive of the computer
- Use e-Sata external drives because USB 2.0 and FireWire are too slow
- Use 7200 RPM drives or, ideally, SSD drives
To avoid waste of resources, configure your VM and turn off everything you don’t need:
- CD Drive
- USB Support
- Printer Support
Additionally, access the BIOS for the VM and disable the serial and parallel ports.
Optimizing the Operating System
The operating system to use is Windows Server 2008 R2 64-bits. The standard edition is enough for a development environment, but be sure to check out the other versions in case you need some feature that is only present in the enterprise version.
In a development environment, there are several optimizations that can help you free some resources:
- In the network configuration options, uninstall the QoS Packet Scheduler and the Link Layer Topology Discovery components, and uncheck IPv6.
- Turn off the Windows Firewall.
- In the Computer properties > Advanced > Performance Options > Visual Effects, select the option Adjust for best performance.
- In the Control Panel > Hardware > Sound, don’t turn on the Windows Audio Service and select No Sounds as the Sound Scheme.
- Turn off IE ESC for both administrators and users
The pagefile is also a possible optimization target:
- If possible, place it in a disk different from the OS disk
- Define the pagefile initial dimension to be the amount of RAM of the machine, and the final dimension to be the triple of the amount of RAM of the machine (if you have enough disk space)
Whenever possible, it is a good idea to defragment the disks before shutting down the machine (this also applies to the host machine).
Optimizing SQL Server
SQL Server is where SharePoint stores all the data and, for that reason, is one of the main causes of the high resource consumption. You should use SQL Server 2008 64-bits SP1 with CU2 (or higher) or, ideally, SQL Server 2008 R2 64-bits. The important thing here is that it’s Enterprise or Developer edition.
Storage
One of the main optimization points for SQL Server is storage, that is, disk performance. To assure the best possible performance, you should create additional virtual disks so that you can separate in different disks:
- SQL Server’s TempDB
- Database data files (.mdf e .ldf)
- Database log files (.ldf)
These disks should have the following characteristics:
- Single file
- Pre-allocated size
- NTFS Formatted using 64K blocks
Ideally, each of these virtual disks should be stored in a distinct physical disk, but when you intend to use the virtual machine in a laptop, it’s not easy to carry 3 or more external hard drives in your backpack.
Anyway, even if you don’t have enough physical hard drives, the virtual disks should be stored in physical disks according to their speed, using the following order:
- TempDB disk (fastest)
- Log file disk
- Data file disk
- System disk (slowest)
Data Files
By default, SQL Server specifies small sizes for the data files and uses small auto-growth sizes to save disk space. However, the auto-growth operation as well as the fact that they grow in small blocks, causes performance degradation as well as fragmentation of the files in the disk.
To increase the performance you should:
- Use more than one data file in the primary file group for the TempDB, search DBs and content DBs of SharPoint.
- Always use, at most, one data file for each CPU core of the system. Ideally, you should use one data file for each 2 CPU cores.
- If possible, store the data files in different physical disks (very hard for a development environment).
- Pre-allocate all data files, including the data files for TempDB.
- The data files for each DB should be equally sized.
- Specify a size for the data files that is enough to avoid auto-growth (for the TempDB, the total size should be 10% of the total size of the largest DB of the system).
- Specify data file growth size to be 256MB or 512MB
Lock Pages in Memory
Another important configuration, is to authorize SQL Server to use the Lock Pages in Memory feature to keep Windows from unloading the memory pages that might still be needed, leading to a performance degradation (more information in: http://support.microsoft.com/kb/918483).
Optimizing SharePoint
Use SharePoint’s complete installation, instead of the stand alone installation, so that it doesn’t install a new SQL Server instance or automatically configures the Farm.
After the installation, configure SharePoint turning on only the features that you absolutely need. Each unnecessary service you turn on will mean less memory and CPU time available for the services that really matter.
I’m proud to be a member of the new Portuguese SharePoint User Group (SPUG PT) and I’ll be cross-posting and translating all my future posts to the User Group’s blog (http://www.sharepointpt.org) which will also feature posts from other community members. Be sure to check it out, if you’re a Portuguese speaker.
The User Group is just starting and already has a very strong presence in the first Microsoft events of the year: Road To SharePoint 2010 and TechDays 2010.
Starting with Road To SharePoint 2010, an event dedicated to SharePoint 2010 composed of 19 sessions over 5 different days and aimed at preparing everyone to the arrival of the new SharePoint, we had sessions presented by Rodrigo Pinto (Rumos Professional Services), Luís Calado (Microsoft), Denis Heliszkowski (Microsoft) and Ricardo Calejo (Devscope), on the first two days of the event.
But we still have 3 days to go, on which you’ll be able to attend the session I’ve prepared with Raúl Ribeiro (Create IT) about the use of Business Connectivity Services to create Business Solutions based on Office 2010 and SharePoint 2010, as well as the session on Business Intelligence in SharePoint presented by Rui Romano and Rui Quintino (Devscope), and many others.
Don’t miss it, it’s a great event for anyone who intends to start working with SharePoint 2010 as fast as possible.
Regarding TechDays 2010, it’s the largest technology event of the year in the Microsoft world. A 3-day event with many sessions about all the novelties on Microsoft products including, of course, SharePoint 2010. We still don’t have all the information regarding sessions and speakers, but we already know we can count on sessions by André Lage (Sybase), Raúl Ribeiro (Create IT), David Rosa (Rumos Professional Services), Denis Heliszkowski (Microsoft), Luís Calado (Microsoft), Rodrigo Pinto (Rumos Professional Services) and me (Create IT) :)
I hope I can meet you there.
Disclaimer: This post is based on notes taken while watching a conference session. For that reason, it may contain incorrect information or data that I might have misunderstood. Also, the product it refers to was not available yet at the time of the writing, thus, not allowing me to validate the present information.
This was the fourth session I attended on the second day of the SharePoint Conference 2009, and it was given by Venky Veeraraghavan from Microsoft.
Venky started by pointing us to the twitter account of the SharePoint Social team (@SPSocial). Follow it to get updated information on this, or mention it to get their attention.
My Site Personal Portal
For SharePoint 2010, Microsoft has made a few changes to the components of the My Site, namely to the page structure of the portal. Now, a newly created My Site has three pages:
- My Network (the homepage, shows the Activity Feed for all users in your network)
- My Profile (shows the profile data)
- My Content (the entry page for the personal site). Additionally, you can create any pages you want on your personal site.
The My Profile page has a new layout and includes areas like:
- Overview (shows the activity feed in Published mode)
- Organization (shows the new Silverlight-based Organizational Chart which allows you to navigate in the organizational structure)
- Content (shows the latest content created by the user)
- Tags and Notes (shows tags used and notes written by the user)
- Colleagues (shows and manages the list of colleagues the user follows)
- Memberships (shows and manages the list of site memberships)
As in the previous version, these pages are located in two different places:
- My Network and My Profile pages are shared by all users and, for that reason, are located in the My Site Host site collection.
- My Content and any custom pages you create will be placed on your personal site collection.
Also, as seen in SharePoint 2007, both the My Site Host site collection and the users’ personal site collections belong to the same Web Application. This Web Application is associated with three Service Applications:
- Enterprise Managed Metadata Service Application
- User Profile Application
- Search Service Application
The New Activity Feed
One of the greatest additions to the My Site is the new Activity Feed. The Activity Feed is an extensible component which exposes tracked data through a Web Part and an Atom 2.0 feed. The Activity Feed has two modes:
- Consolidated – Lists the activities of everything a user tracks. This mode is used on the My Network page.
- Published – Lists my activities. This mode is used on the My Profile Page.
The architecture that supports the Activity Feed can be briefly described as:
- Everything starts with a Gatherer Timer Job which periodically analyzes the Social Database (which stores all the tagging, rating and comment data) and the change log of the User Profile Database.
- Using a new API called Injection Object Model, the gatherer injects the relevant information into the Activity Feed structure in the User Profile Database.
- This data is then exposed via a Web Part, the Object Model or the Atom feed.
The main extensibility point is the possibility to create Custom Gatherers, which collect information from other data sources and inject them in the feed using the Injection Object Model.
Regarding the consumption of the Atom 2.0 feed, the URLs for:
- Consolidated Feed:
http://<mysitehost>/_layouts/activityfeed.aspx?consolidated=true
- Published Feed:
http://<mysitehost>/_layouts/activityfeed.aspx?publisher=<accountname>
Since consuming a feed is a very common and easy way to retrieve live information, it’s expected that SharePoint will have to handle a large number of requests. Currently, at it’s beta version, SharePoint 2010 can handle 500-700 requests per second (RPS) with full activity detail, or 2100 RPS without it. However, the RTM is aiming at 2000 RPS.
Profile Pictures
The way profile pictures are stored has also been changed. Instead of storing the profile picture inside each user’s personal site, SharePoint 2010 will store all profile pictures in a central picture library which will be accessed through the URL:
- http://<mysitehost>/UserPhotos
When a picture is uploaded, SharePoint will automatically create the resized versions of it:
- 32x32 (for general use in SharePoint)
- 48x48 (for AD and client applications)
- 96x96 (for the Profile Page)
The Picture Picker control used by SharePoint can be customized and even replaced to support any policies and picture stores.
The profile pictures will be synchronized with Active Directory, allowing the profiles to be bootstrapped if the pictures are already in Active Directory when the profiles are created, or writing back to Active Directory, if SharePoint is configured to do so. Client applications such as Outlook and OCS can use pictures stored in AD.
Microsoft also thought about the upgrading process from SharePoint 2007 to SharePoint 2010, and there is a PowerShell cmdlet (Upgrade-SPProfilePhotoStore) that migrates all profile pictures from the previous structure to the new store.
Social Feedback
Another of the biggest additions is the Social Feedback mechanism which is everywhere in SharePoint 2010. It’s a new mechanism that allows users to tag, rate and comment on any URL, either inside or outside of SharePoint (through bookmarklets). Feedback is the primary mechanism for promoting documents and web pages to the newsfeed and it also helps categorize, annotate, promote and in the retrieval of relevant links.
The user doesn’t need write permissions on a specific item to be able to give feedback on it(read permissions are obviously required), since this data is stored on a separate database. At a very basic level, each feedback item is has only three fields: Person, URI and Feedback.
Tags
- Also known as keyword bookmarking
- There is no web part for tagging
- Can be applied to web pages, list items and documents
- Can be used to sort and filter lists and document libraries, but only by document authors and requires the enterprise keywords field
- Tags are indexed by search
- Can be applied to external or non-SharePoint pages via bookmarklets
- Can be used with Enterprise taxonomy management
- Can be used in Office 2010 Client (but only document authors and requires enterprise keywords field) and Office Web Apps
Notes
- Short comments (less than 3000 characters)
- There is a specific web part for comments
- Can be applied to web pages, list items and documents
- Cannot be used for sorting and filtering in lists
- Notes are not indexed by search
- Can be applied to external or non-SharePoint pages via bookmarklets
- Can be used in Office 2010 Client and Office Web Apps
Ratings
- 5-star ratings
- There is a specific web part for ratings
- Can be applied to web pages, list items and documents
- Can be used for sorting and filtering in lists and document libraries
- Ratings are indexed by search
- Cannot be applied to external or non-SharePoint pages
- Cannot be used in Office 2010 Client nor Office Web Apps
The concept of My Links from SharePoint 2007 will be replaced by social tags, and there will be a mechanism to migrate each user’s links to tags.
The social feedback mechanism is supported on three services:
- Search Service, which indexes tags and ratings and performs all the security trimming
- User Profiles and Synchronization
- Enterprise Metadata Management, for tagging and profile properties
The use of social feedback mechanisms will bring a new set of challenges to the organizations. Planning will be a key step to make it work, and it has to occur at various levels:
- Planning for scale – social feedback generates very large amounts of data.
- Planning for privacy – organizations have to make decisions about who can give feedback and how security trimming is applied to this feedback.
- Planning for adoption – organizations have to start with a small number of committed users and gather support from all departments for this to succeed.
Expertise Search
The last of the additions I’ll be pointing out is the Expertise Search mechanism which allows:
- Users to search for other users by their expertise tags
- SharePoint 2010 to suggest expertise tags for each user
While the first is pretty straightforward, the second is much more interesting. So, how does SharePoint 2010 know what expertise tags to suggest?
Well, Outlook 2010 can analyze your e-mail (your sent items, in particular) looking for the people to whom you send e-mails the most and also extracting the most common keywords. This information is then sent to SharePoint to be included in the user’s profile, if he agrees to it, where it can be indexed and searched normally.
Don’t worry, the analysis can be configured to be:
- Turned off. No analysis will be done on your e-mail.
- Analyze and upload automatically to the profile (user consent at the client).
- Analyze but only suggest when editing the profile (user consent on the server).
Disclaimer: This post is based on notes taken while watching a conference session. For that reason, it may contain incorrect information or data that I might have misunderstood. Also, the product it refers to was not available yet at the time of the writing, thus, not allowing me to validate the present information.
This was the third session I attended on the second day of the SharePoint Conference 2009, and it was given by Pej Javaheri and Steve Tullis, both from Microsoft.
The first cool thing about this session was that the slide deck was presented using PowerPoint Web App. Talk about eating your own dog food :)
Most of the session was one big demo showing off an overview of some of the top new features on Excel 2010 and Excel Services 2010, which I will describe below.
Power Pivot
Power Pivot is the new name of what was previously known as project “Gemini”. It’s a new capability that the SQL Server team has worked on and brought to Excel, which allows deeper analytics on very large data sets. One other nice thing is that it can be made available through SharePoint 2010.
You can find more information about this on: http://www.powerpivot.com.
Business Intelligence Center
SharePoint 2010 includes an new site template called Business Intelligence Center which is an improvement over the previous Reports Center site template. This new template contains extensive guidance to help you start using the new BI components offered by SharePoint 2010.
Excel 2010: Pivot Charts and Pivot Tables
Excel 2010 introduces Pivot Charts. Pivot Charts are like normal charts connected to the same source data as the Pivot Table, however now you can drill down on hierarchies and select which members you want to see, right on the chart object, instead of having to do that on the associated Pivot Table, and your selections will affect both the Pivot Chart and the Pivot Table.
Both on Pivot Charts and Pivot tables, on the field selection popup menu, when drilling down a hierarchy to select a member, you will now find a search box to help you search an item instead of having to browse through all the hierarchy to find it.
Another new thing is the ability to define named field sets and use them in Pivot Tables and Pivot Charts. These named sets are stored inside the workbook and can be reused by other users.
In a Pivot Table, you can perform a series of calculations by using a new feature called Show Value As. This new feature allows you change the way you view a field by selecting from a list of predefined calculations, such as:
- % of Grand Total
- % of Row Total
- % of Column Total
- % of Parent Row Total
Before you either had to perform the calculations outside the Pivot Table or put these as cube formulas.
Excel 2010: Conditional Formatting
A few improvements were made to the conditional formatting capabilities of Excel, and now Data Bars support positive and negative values. Also, new Icon Sets have been introduced.
Excel 2010: Slicers
Visual Slicers are a new Excel object that allows users to filter the data on Pivot Charts and Pivot Tables with a single click. Slicers show you a set of members of a hierarchy as buttons which you can press to filter data. Additionally, members that have no available data will be grayed out, giving you a quick visual understanding of which members have data and which have not.
Excel 2010: Sparkline
Also an addition to Excel are the Sparklines. A Sparkline is like a mini chart that illustrates a trend in a set of data. Excel comes with three types of Sparklines:
The Sparkline object is treated like any other cell content (such as a formula) which means it can be copied, pasted and expanded.
Excel 2010: What-If Analysis
When accessing external data (a SSAS cube) to build a Pivot Table, you can now use What-If Analysis to write back to the data source.
Excel Services 2010: More Workbooks and Client Fidelity
Excel Services (Excel Web App) will now try to open all workbooks in View Mode even if they contain unsupported features. In the previous version, Excel Services wouldn’t even open the workbook showing an error message which wasn’t much help to solve the problem.
In Edit Mode, however, it’s a different story but the list of unsupported features was not shown. I can tell you what I saw from the demos, but this is not an extensive list:
- Images are supported
- Charts, Sparklines and Slicers are supported
- Pivot tables are supported
- Grouping is supported
Don’t misunderstand me. I’m not saying you can add sparklines and slicers while editing the workbook. I’m saying you can edit a workbook that contains these objects and even interact with them, but that’s it.
As a final remark, all of this works very well on Safari and FireFox (besides Internet Explorer, of course).
Excel Services 2010: Edit and Collaboration
The big news here is that you can edit a workbook using the Excel Web App. You have a limited set of operations you can perform but I can tell you it’s still quite impressive. The Edit Mode support co-editing, which means more than one user can edit the document simultaneously and Excel Web App will warn you of who is also collaborating with you.
Edit Mode has a reduced version of the Excel Ribbon but has a fairly impressive set of features (for a web app):
- Supports undo and redo
- Supports live preview (when selecting colors or formats)
- Supports auto-complete when entering a formula
- Supports grouping of rows and columns
It probably supports a lot more cool features, but this is what I saw in the session.
The new Excel Web Access Web Part allows you to turn interactivity on and off, which means the users can now type in the cells. In the previous version, if you wanted the users to interact with your Excel workbook through the Excel Web Access Web Part, you had to define parameters which would be shown on the side of the web part.
Just one note: interactivity is not the same as editing. You can interact, and see changes in charts and calculations, but the original workbook remains unchanged.
Excel Services 2010: JSOM (JavaScript Object Model)
Another addition to Excel Services 2010, is the JavaScript Object Model (JSOM) which allows developers to handle events in Excel Web Access Web Parts (like mouse clicks) and react to them.
Steve demonstrated the JSOM in a SharePoint Dashboard with two different Excel Web Access Web Parts. The first one had an Excel grid and the second one had two charts. Whenever he clicked a row in the first one, the charts showed data that was related to the selected row.
Excel Services 2010: REST API
One of the coolest new features (in my perspective) is the REST API exposed by Excel Services 2010 which can be used to retrieve data on:
- Ranges
- Charts
- Tables
- PivotTables
The data is retrieved as an Atom feed (which you can syndicate to) or in HTML format allowing it to be embedded wherever you wish. When retrieving a Chart, Excel Services will return a PNG image of the chart.
One of the demos regarding the REST API was creating a quick part in Word, of type field, and setting the source of that field to be the REST URL for a Chart. By doing this, Word inserts the chart image in a document, but allows you to update it like any other field.
Disclaimer: This post is based on notes taken while watching a conference session. For that reason, it may contain incorrect information or data that I might have misunderstood. Also, the product it refers to was not available yet at the time of the writing, thus, not allowing me to validate the present information.
This was the second session I attended on the second day of the SharePoint Conference 2009 and it was given by Rolando Jimenez from Microsoft.
Business Connectivity Services
According to Rolando, Business Connectivity Services (BCS) are “a set of out-of-the-box features, services and tools that enhance SharePoint to streamline solutions of deep integration of external data and services.”
In the SharePoint Pie, Business Connectivity Services belongs to the Composite slice. It’s the new and much improved version of the old Business Data Catalog, introduced in SharePoint 2007.
Regarding BCS, Microsoft has invested in three main areas:
- Presentation: extend Office and SharePoint’s user experience and capabilities to external data and processes.
- Connectivity: read-write capable connectivity from client and server to databases, WCF/Web Services and .NET sources.
- Tooling: integrated tooling experience scales from simple solutions to advanced applications with rich client packaging and deployment.
Core Concepts
The main building block for BCS is the new concept of External Content Type (ECT), which is the new name for what was previously known as BDC Entity. An ECT describes:
- How to connect to the data
- Schema of the data
- Behavior within Office and SharePoint
This definition will then be used by the BCS Runtime, both in SharePoint (thin client) and in Office Client applications (rich client) to access the external data.
The other important concept is the External List. This is a new type of list in SharePoint 2010 that allows you to surface data retrieved by BCS as defined by the External Content Type associated to it.
BCS Tools Overview
There are three types of solutions that use BCS:
- Simple No Code solutions, which can be built by Power Users using SharePoint Designer
- Surface data in SharePoint using External Lists
- Connect those lists to Outlook and SharePoint Workspace and take them offline
- Intermediate No Code solutions, which can be built by developers using SharePoint SDK (and eventually also SharePoint Designer)
- Customized InfoPath forms
- Customized Office Ribbon and TaskPanes
- Customized QuickParts in Word
- Customized Workflows
- Customized Web Part Pages
- Advanced Solutions, built by Advanced Developers using Visual Studio (and SharePoint SDK and SharePoint Designer).
- Custom connectivity for data aggregation, transformation, security, …
- Using custom code to integrate data into any Office application (even the ones that don’t support BCS directly)
- Business logic in forms
- Create reusable components (UI parts, ECTs, actions) to be included in simpler solutions
Regarding solution creation, there are mainly two approaches:
- Client-Server Environment - PowerUsers use a live connection to the server and directly build simple no code solutions on the server or rich client.
- Single Machine Development Environment - Developers use a combination of SharePoint Designer, SharePoint SDK and Visual Studio to produce advanced solutions, packaged in WSP files that can then be deployed to any SharePoint farm.
Demos
After the overviews, Rolando demonstrated a few Advanced Solution scenarios. Here are the notes I gathered from the demos.
Demo 1: BCS Hello World
- There is a new project type in Visual Studio 2010 called Business Data Connectivity Model Project which can contain a collection of ECTs
- The graphical designer makes it very easy to build ECTs
- An ECT is a class with at least:
- One property called ID
- A method called ReadList
- A method called ReadItem
- Visual Studio 2010 includes a BCS Explorer tool which graphically shows the structure of the XML-behind generated when designing the ECTs
- Visual Studio 2010 can package and deploy the ECT directly to SharePoint
Demo 2: Author Custom Connectivity in Visual Studio
- If you want to retrieve data from multiple tables and aggregate it in a single ECTs you have to develop the code yourself (for each ECT class method).
- If you want to allow writes on ECTs, you must add an Update method to the ECT class.
- SharePoint automatically generates and Edit Form for External Lists that support writes.
Demo 3: Discover and Configure ECT for Outlook Offlining in SharePoint Designer
- Using SharePoint Designer you can define a mapping between ECT properties and Outlook objects, allowing you to connect an external list to Outlook (really neat stuff).
- When packaging the solution, Visual Studio will generate a WSP for SharePoint deployment and a Click-Once package for Office Client deployment (similar to Office Add-Ins).
- The same code (ECT assembly) is executed by the BCS Runtime on the Office Client and on SharePoint.
Demo 4: Custom Rich Client Integration (Excel) with VSTO
- Using the BCS API, a developer can built an Excel Add-In, using VSTO, that loads the data using the ECT and inserts into a spreadsheet.
- The developer can specify if it allows BCS to cache the data or if the data always has to be live.
Disclaimer: This post is based on notes taken while watching a conference session. For that reason, it may contain incorrect information or data that I might have misunderstood. Also, the product it refers to was not available yet at the time of the writing, thus, not allowing me to validate the present information.
This was the first session I attended on the second day of the SharePoint Conference 2009 and it was lectured by Umesh Unnikrishnan from Microsoft.
Core Concepts
- Service is a middle tier feature that performs the useful function of providing data or processing resources to SharePoint features. For instance, Search is a Service.
- Service Applications is a configured logical instance of a Service, which provides data or computer resources and exposes administrative interfaces.
- Service Instance is a running physical instance of a Service (in the sense of a process).
Web Applications are associated with Service Applications by means of a Service Application Proxy, which connects the two allowing the Web Application to use the Service Application. The associations between a Web Application and a Service Application can be changed at any time by administrators, and can be managed in groups (Service Application Proxy Groups).
Evolution of Shared Services Provider
In SharePoint 2010, the Shared Services Provider is replaced by separate Service Applications:
- User Profiles Service Application
- Search Service Application
- Excel Service Application
- Business Connectivity Application
- And all the new Service Applications in SharePoint 2010
This means that:
- Services are now independent, and don’t exist under the umbrella of the Shared Services Provider
- There is no more Shared Services Provider administration site, all the administration is integrated in the Central Administration site
- You can develop Service Applications and integrate them in SharePoint 2010
- When SharePoint 2007 is upgraded to SharePoint 2010, the services provided by the SSP are directly mapped to their respective Service Applications
Service Framework
SharePoint 2010 ships with around 20 out-of-the-box Services, and other products like Office Web Apps, Project Server and SQL Power Pivot also include Services. All these services are built using the same framework that is provided to extend this model, which is incorporated into SharePoint Foundation:
- Built-in support for scaling via multi-server support and load balancer
- Includes mechanism to host and deploy WCF-based Service Applications
- Includes Administration UI and PowerShell integration
- Includes Timer Job support
- Can be multi-tenant aware
Service Administration
Regarding Service Administration in SharePoint 2010:
- All the administration is performed in Central Administration or through PowerShell scripting. Each service plugs its management UI into the Service Management Page.
- The new association model is much more flexible than before, and allows each Web Application to use only the Services it really needs.
- The Service Application model incorporates software-based fault tolerant round-robin load balancing. There is, however, support for hardware-based load balancing and extensibility points for 3rd party components.
- Service Applications support cross-farm federation adding even more flexibility to its configuration. This allows a farm to provide services to other farms.
Farm Administrators can configure Delegated Administrators, which are users with Central Administration access that manage one or more Service Applications. Central Administration UI is security trimmed allowing Delegated Administrator to interact only with their managed Service Applications.
There are four pages dedicated to Service Application administration:
- Manage Service Applications page
- Create/Delete Services Applications
- Manage Service Application metadata
- Connect to remote Service Applications
- Publish and Secure Service Applications
- Service-specific management UI page
- Service Application specific settings
- Dashboard showing Service Application status (search crawl status, profile import status, etc)
- Manage Service Associations page
- Add/remove associations between Web Applications and Service Applications
- Manage Services on Server page
- Start/stop Service Instances on specific servers
Service Security
The security model was improved:
- Inside the farm, services use claims-based authorization
- Between farms, services communicate via WCF-based web services
- There is support for SSL/Transport Security
The Service Applications can be isolated:
- Each Service Application uses a separate database and, optionally, a separate application pool
- Different Service Applications for a single Service can run under different accounts and use different databases
Security is managed per Service Application. There are two types of security:
- Admin Security
- Specifies who has admin rights over a Service Application
- Used for security trimming
- By default, all Farm Administrators are included
- Access Security
- Specifies claims that have access to the service
- By default, the farm claim has access
- Some services might require more granular access rights
Service Application Deployment
The configuration of the Service Applications can be performed:
- Automatically, via Farm Configuration Wizard, which creates all Service Applications with their default settings.
- Manually, using the Central Administration UI (New Button in the Manage Service Applications page). This allows you to:
- Specify custom application pool, database location, service account
- Create Service Applications and their Service Application Proxies
Service Application Associations
By default, all Service Applications in a farm are associated with all Web Applications (through Service Application Proxies). This default association scheme can be changed so that associations are managed separately for each Web Application and Service Application.
Service Application Publishing
Publishing a Service Application makes it available outside the farm, where it can be discovered and consumed by remote farms. However, you still have to perform associations to give Web Applications access to the Services Application and cross-farm trust is required via certificate exchange.
Sample Topologies
Single Farm
- Simplest one
- All Web Applications are associated with all Service Applications
Isolated Hosting
- Single farm
- Each Web Application is associated with only the needed Service Applications
- Some Service Applications are shared between different Web Applications
- There can be multiple Service Applications for a single Service, associated with different Web Applications, to ensure isolation
Shared Resource Farm
- Multiple farms
- One farm contains all the Service Applications and publishes them to be consumed by Web Applications in other farms
- The other farms contain only the Web Applications which consume the published Service Applications of the first farm
