Welcome to Comunidade Bloggers |create|it| Sign in | Join | Help

André Vala

FAST Search Server


  • SP24 Conference
    European SharePoint Community Awards 2012 Winner
    Locations of visitors to this page

    Comunidade Portuguesa de SharePoint

    Member of the SharePoint Community

    View André Vala's profile on LinkedIn

    © André Vala and Create IT, 2006-2013. Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to André Vala and Create IT with appropriate and specific direction to the original content.

Microsoft WebCamp 2014: Working with AngularJS

Last week I delivered a session at Microsoft WebCamp 2014 event. A full-day event, with 3 simultaneous session tracks, focused on web technologies, not only from Microsoft but also open source, and design/communication topics. A lot of good stuff was shown, from node.js to service stack.

My session was about Working with AngularJS, a fantastic Single Page Application framework in JavaScript. It’s an introductory session, meant to give attendees a basic understanding of the framework and its components. I had a packed room, with around 200 people, and the feedback was great so I felt it might be worth sharing the slides and demos I wrote for the session:

You’re probably wondering why would I, a SharePoint guy, be talking about AngularJS. SharePoint guys are not exactly known for embracing the latest and greatest web technologies, mostly because when a new SharePoint version is released, it’s already one .Net version behind, not to mention all the new JavaScript frameworks. With SharePoint 2013, all that is changed. The new SharePoint App Model allows us to build our apps in any technology we want, and that means we can use all the big boys toys now such as ASP.NET MVC and, of course, AngularJS.

Also presenting a session on WebCamp about KnockoutJS was my good friend and colleague João “Jota” Martins, which is a great JavaScript data-binding library. Although they share some concepts, Knockout and Angular are very different technologies, on very different abstraction levels, so it’s good to know when to use one or the other. Checkout his slides and code samples here.

European SharePoint Conference 2014

The European SharePoint Conference 2014, Europe’s largest SharePoint-focused conference, took place last week (May 5th to May 8th) in Barcelona, and it was a great! I was fortunate to be a part of the Programme Team and contribute, although just a little bit, to the success of this huge event. A few statistics about the conference:

  • 6 pre-conference full-day tutorials by SharePoint all stars like Dan Holme, Rafal Lukawiecki and Joel Oleson.
  • 4 keynotes by Simon Skaria, Dan Holme, Christian Buckley, Mark Kashman and Rafal Lukawiecki
  • 110 breakout sessions over 8 simultaneous tracks
  • Over 80 speakers
  • More than 1200 attendees from all around the world
  • 55 sponsors

Each day was focused on a specific theme based on the main trends in the industry:

  • The first day was focused on Cloud and Office 365
  • The second day’s theme was Enterprise Social, Enterprise Content Management and Best Practices
  • The third day was about Solutions and Devices

Without getting into too much detail, here are the general trends around SharePoint and Office 365, some of which were already shared in Microsoft’s SharePoint Conference, last March:

  • The future is the cloud but SharePoint on-premise is not going away anytime soon. Next year a new version of SharePoint Server will be released and, after that, there are plans for an additional one. However, all the new features will first appear on SharePoint Online (Office 365) and some might never be released on-premises.
  • When developing for SharePoint on-premises, always try to use the cloud-proof App Model to allow your solution to be used in SharePoint Online. Only go for Full Trust Code (old school solutions) if there is no other way.
  • SharePoint Social will not be developed further because Yammer will end up replacing it completely.
  • Yammer will be integrated into all Office collaborative experiences, from Outlook to Word, both Online and On Premises. Outlook will feature Yammer groups, Office documents will have an associated Yammer thread, and Yammer groups will have an associated calendar.
  • The “Codename Oslo” Experience will bring us a new way of working, where the information you need is brought too you automatically through the power of the Office Graph, FAST-based search technology and the use of machine learning. It will be available through Office 365 and my guess is that it will become the new Office 365 homepage – an personalized information hub tailored to each user’s needs.
  • Microsoft’s Business Intelligence stack continues to evolve. PowerBI, PowerPivot, PowerQuery, PowerMap and PowerView are all very impressive pieces of technology and I hear there will be announcements soon around these topics…
  • SharePoint development is now achieving a maturity level that allows real Application Lifecycle Management. Source control, continuous integration, automated testing, automated code analysis and continuous deployment, are concepts that can currently be applied to SharePoint projects.

The Expo Hall featured the sponsors’ booths where everyone could meet their products and services, and meet some familiar faces such as Mike Fitzmaurice (Nintex), Edin Kapic (Beezy), Chris McNulty (HiSoftware) and Christian Buckley (Metalogix). The Community Area was also located in the Expo Hall, and a lot was happening here, especially during breaks:

  • Soapbox sessions were being delivered by some of the SharePoint all stars (Dan Holme, Michael Noel, Mike Fitzmaurice, Joel Oleson, Jeremy Thake, Geoff Evelyn, Paolo Pialorsi, Agnes Molnar, Edin Kapic and Eric Overfield)
  • The ShareMentor program was kicked off by Agnes Molnar
  • Book signings
  • Prize draws

Just like in previous years, one of the highlights of the ESPC is always the Gala Dinner. A social event that mixes attendees, speakers and organizers, and where the European SharePoint Community Awards winners are announced, as well as the Top 25 SharePoint Influencers. As expected, it was a great occasion to meet new people and the fact that |create|it| won an ESPC award for the third consecutive year only made it better.

Overall, it was a great experience and I just hope I get to attend it next year.

SharePoint Conference 2014: Keynote Takeaways

Microsoft’s SharePoint Conference 2014 started yesterday in Las Vegas with a keynote by Bill Clinton, followed by the usual suspects, Jared Spataro, Jeff Teper and Arpan Shah, presenting all the new stuff. This year I was unable to attend the Conference in person, but I got to watch the keynote live on Channel 9, and here’s what I’ve learned from it (and from a bunch of additional sources such as tweets, blog posts, and the like).

As the conference started, a few posts were published on Office Blogs:

Cloud vs On-Prem

Make no mistake – feature-wise, SharePoint On-Premise will start to lag behind SharePoint Online / Office 365. The only way is up. Here are some facts that support this claim:

  • The next SharePoint Server version will be out in 2015. Meanwhile, Office 365 is getting updates almost on a weekly basis, and maintaining a very impressive 99,98% uptime.
  • SharePoint “2015” will still include Social features but these will remain unchanged from SharePoint 2013. No investment will be made to improve it (more on this below).
  • SharePoint SP1 (released in February 2014) adds support for hybrid scenarios where an on-premise SharePoint farm can have personal sites stored on OneDrive for Business (formerly SkyDrive Pro) in the cloud, and better Yammer integration with Yammer feeds instead of SharePoint Social feeds (more information here).
  • Office 365 will have a new site template for Video Portal which will allow you to create a YouTube-like experience for your enterprise based on Azure Media Services.
  • Elastic scale will come to Office 365, which will support site collections of up to 1TB (additional fees apply) and infinite scale per tenancy.
  • OneDrive for Business will be released as a standalone service, competing directly with Dropbox and similar services (more information here).


Yammer will eventually replace SharePoint’s native social features so, according to Jared Spataro, we should all “Go Yammer”.

  • Yammer is now integrated into Dynamics CRM
  • Yammer is now accessible directly from the suite bar on Office 365, without requiring additional authentication
  • Yammer conversations will be completely integrated with Outlook.com showing up as you click on a document, email or contact.
  • Yammer will be a part of Office 365 Academic and Midsize Business Plans. It was already part of all Enterprise Plans.
  • Microsoft expanded Yammer’s Enterprise Graph and built the Office Graph, which maps relationships between people and information by recording posts, likes, emails, replies, shares and uploads.
  • A new app named Oslo will unleash the Office Graph potential and allow you to navigate through these relationships and discover new information.
  • Yammer groups will become a cross-Office 365 concept. Creating a group anywhere on Office 365 will automatically provision a corresponding inbox, social feed, calendar and document library that group members can use. This new feature effectively replaces SharePoint’s workspaces.
  • Yammer will be integrated with Lync and Skype.

Office Apps

With Office 2013 Service Pack 1, Office Apps gain additional features:

  • PowerPoint and Access now support the new App Model.
  • Outlook now supports a new type of App – Compose App – which allow you to interact will email messages or appointments when you are creating them.
  • New Office Web Widgets, ListView and PeoplePicker, available as a NuGet package (more information here).

Tooling and Development

  • New APIs for Office 365 focused on files/folders, calendar and contacts (more information here). It will also be possible to provision site collections on Office 365 programmatically.
  • New open source Android SDK for Office 365 to help developers access data in Office 365 from Android Apps (more information here).
  • New Cloud Business App project template on Visual Studio 2013 helps you create SharePoint Apps that leverage Windows Azure.
  • Office Developer Tools for Visual Studio 2013 – March 2014 Update (get it here).
  • Napa Development Tools were updated and improved

Forms Solutions

At the Conference, Joel Oleson organized a funeral procession for InfoPath, and that was about all I heard about Microsoft’s new approach to form solutions. I read somewhere that Access will have an important role in InfoPath’s replacement but, so far, nothing specific. Since this is one of the topics that SharePoint professionals and clients are most worried about, I was expecting it to be addressed during the keynote. I believe there will be, at least, one breakout session on this topic so let’s wait and see.

User License Enforcement in SharePoint 2013

Note: this post was first published in the European SharePoint Community blog.

Amongst the several new features included in SharePoint 2013, there is one that I have unfairly overlooked until just recently and that is User License Enforcement. I say unfairly because it’s a feature that has been requested for a long time and I believe it deserves more attention than it has been receiving.

Up until now, any user that accessed a SharePoint farm would have access to all the features included in the installed SharePoint SKU, regardless of the CAL (Client Access License) level assigned to her. This means that, if a company had a SharePoint farm with an Enterprise license, any user would be able to access all Enterprise features, even if it was assigned a Standard CAL.

On most large companies, for cost saving purposes, it’s quite common to find scenarios where only some of the users are assigned an Enterprise CAL, while every other user is assigned a Standard CAL. One way to ensure that each user can only access the features included in his license, is to have two separate SharePoint farms, one with a Standard SKU and the other with an Enterprise SKU. However, this is hardly an acceptable solution.

SharePoint 2013 solves this with the new User License Enforcement capability, which enables the mapping of licenses with specific users or Active Directory security groups. When User License Enforcement is enabled, users can only access the SharePoint features included with their license, otherwise they are blocked. When it’s disabled, which is the default, SharePoint behaves the same as in previous versions.

User License Enforcement management is performed exclusively through PowerShell cmdlets in SharePoint 2013 Management Shell. There are eight cmdlets which are explained in detail in the following sections:

  • Get-SPUserLicensing
  • Enable-SPUserLicensing
  • Disable-SPUserLicensing
  • Get-SPUserLicense
  • Get-SPUserLicenseMapping
  • New-SPUserLicenseMapping
  • Add-SPUserLicenseMapping
  • Remove-SPUserLicenseMapping

Enabling and Disabling User License Enforcement

To check if this feature is enabled, use the Get-SPUserLicensing cmdlet in SharePoint 2013 Management Shell. It will return true if the feature is enabled, and false otherwise (see Figure 1). As stated before, User License Enforcement is disabled by default.

Figure 1 - Usage of Get-SPUserLicensing cmdlet

To enable it, execute the Enable-SPUserLicensing cmdlet. To disable it, execute the Disable-SPUserLicensing cmdlet. Both cmdlets require no parameters and return no values (see Figure 2).

Figure 2 - Using the Enable-SPUserLicensing and Disable-SPUserLicensing cmdlets

Now that you know how to enable and disable it, I should advise you to keep it disabled until you have configured it properly. Otherwise you might involuntarily block access to Enterprise features for users that should have access to them or allow access to features that users should not be allowed to use.

Checking Available User Licenses

To check what licenses are available in your SharePoint farm, you can use the Get-SPUserLicense cmdlet (see figure 3).

Figure 3 - Using the Get-SPUserLicense cmdlet

There are five license types (although on my farm there are only four):

  • Enterprise
  • Standard
  • Project
  • OfficeWebAppsEdit
  • Duet

The Enterprise and Standard licenses are the two SharePoint Server editions, which have been around for several versions already. You can check the differences between the two in TechNet: http://technet.microsoft.com/en-us/library/sharepoint-online-service-description.aspx#bkmk_FeaturesOnPremise.

As for the others: the Project license is used to allow access to Project Server features, the OfficeWebAppsEdit license is used to allow users to edit documents in Office Web Apps, and the Duet license is used to allow access to Duet features (integration between SharePoint and SAP).

Mapping User Licenses

Mapping users to licenses is, in fact, mapping claims to rights. To create a new mapping you must execute the New-SPUserLicenseMapping cmdlet which can be used in a few different ways, depending on the type of claim you are mapping.

You can map an Active Directory security group to a license, executing the following command:

$mapping = New-SPUserLicenseMapping
–SecurityGroup “Enterprise Users”
–License “Enterprise”

You can also map a Forms-based Role to a license by using the –RoleProvider and –Role parameters instead of the –SecurityGroup parameter:

$mapping = New-SPUserLicenseMapping
–RoleProvider “MyRoleProvider”
–Role “Enterprise Users”
–License “Enterprise”

Finally, you can map a claim to a license by using the –ClaimType, –OriginalProvider and –Value parameters, or just the –Claim parameter with a reference to a SPClaim object.

Figure 4 - Using the New-SPUserLicenseMapping cmdlet

When creating the mapping you can also specify which Web Application the mapping applies to, allowing you to configure user licenses on a per-web application basis. If you don’t specify a web application, the mapping will be applied to the whole farm.

After creating the mapping, you must call the Add-SPUserLicenseMapping to add it to the farm:

Add-SPUserLicenseMapping –Mapping $mapping

Checking User License Mappings

To check which mappings are configured in the farm, you can use the Get-SPUserLicenseMapping cmdlet without any additional parameters. Note that any mappings defined on a web application level will not be returned when retrieving the farm level mappings.

Figure 5 - Using Get-SPUserLicenseMapping cmdlet

To list the mappings configured for a specific web application, add the –Web Application parameter with the respective URL value.

Removing User License Mappings

To remove a mapping, you can use the Remove-SPUserLicenseMapping cmdlet. The only required parameter is the identity of the mapping (a GUID) which you can easily get by executing the Get-SPUserLicenseMapping cmdlet.

Enforcement Effects

Two common questions regarding User License Enforcement are:

  • Where is the user license enforced?
  • What happens when a user tries to access a feature that is not available to his license type?

Regarding the first question, the user license enforcement will be used in the following situations:

  • When accessing a page with web parts that require a specific license (e.g. Excel Viewer Web Part or InfoPath Form Viewer);
  • Accessing the web part gallery to add a new web part to a page;
  • Selecting a site template when creating a new site;
  • Trying to edit a document in Office Web Apps.

As for what happens in these situations, two things can happen:

  • A user without the required license won’t see the components he has no access to (web parts and site templates); or
  • SharePoint will deny access to a component the user has no license for, presenting a message explaining why the user cannot access the component.

Additional Notes

There are a few additional notes I believe are important to keep in mind:

  • User License Enforcement is available only for On Premise SharePoint 2013 deployments. SharePoint Online uses a per-user licensing model hence it does not require this capability to control access to specific features.
  • User License Enforcement works only with Web Applications that use Claims-based Authentication.
  • Once enabled, every user has to be assigned a license, even service accounts. If you forget to assign a license to a user, it will be logged as Unlicensed and will be denied access to most SharePoint features.


You can find additional information on the PowerShell cmdlets on TechNet: http://technet.microsoft.com/en-us/library/jj219609.aspx.

I’m Coming Back!

I’ve just noticed that I haven’t posted anything for more than a year now and I have no good excuses for this absence. It’s actually a bit unfair to this blog, since I’ve been publishing some new content elsewhere. Anyway, I intend to make it right by means of a comeback so here’s what I’ve been up to recently.

I’ve published an article on What’s New in Search in SharePoint 2013 on two Portuguese magazines, Revista Programar (February 2013) and PC Guia (April 2013). You can get the first one here, if you can read Portuguese. If not, I will be posting that same article in English here, in the coming weeks.

Last September I’ve recorded an How-To Video for the European SharePoint Community on How to Use Host-Named Site Collections. If you’re interested you can check it out here.

Also, for the European SharePoint Community, I’ve recently published an article on User License Enforcement in SharePoint 2013. I’ll post it here in the coming weeks, but you can also read it on the SharePoint Community Blog.

Finally, I’ve been invited to be a part of the European SharePoint Conference 2014 Programme Team so, if you’re planning on attending the conference in May 2014, I’ll meet you there!

Happy SharePointing!

SharePoint Conference 2012

The SharePoint Conference is always a great event, but when it matches the launch of a new SharePoint version it rises to a whole other level. According to Jeff Spataro, this is the largest SharePoint Conference to date with:

  • 10,000 attendees
  • From 85 countries
  • Over 200 sponsors and exhibitors
  • Over 300 breakout sessions
  • 45 hands-on labs

The Mandalay Bay Resort is the perfect venue for such a massive event, and Microsoft really knows how to take advantage of all the space in the Convention Centre. I’ve been here for the 2009 SharePoint Conference, and I can tell you it gets better every year.

Here are some of the new stuff they came up with:

  • Your badge can be customized using up to 3 stickers that describe what are your main areas of interest (Search, Websites, Collaboration, …), what kind of attendee you are (IT Pro, Developer, Business) or some other SharePoint related expression.
  • Your backpack can also be customized using a sticker with your attendee profile.
  • There is an enormous Community Wall where you can leave your thoughts on what community means to you. But get this: you don’t to write it down on the wall. Instead, one of the professional designers will include your phrase in the wall and will draw something related to your home country. This ensures the whole wall will have a consistent and artistic look & feel. I can tell you it’s looking really great!
  • There is also a SharePoint Timeline, where each attendee writes his signature on his SharePoint freshman year. One guy wrote he was a SharePoint newbie in 1979 because “he was born to work with SharePoint” :)
  • A Photo Booth where you can use a few props to take silly pictures and e-mail them to your friends.
  • Possibility to complete, on the spot, the two SharePoint 2013 beta certification exams for free, and other certification exams with a discount price.

The Keynote was delivered with plenty of music and videos, and a well known ensemble of presenters, for the usual wow effect in this kind of events. It started with Jared Spataro saying that “SharePoint is not just a product, it is a way of life”. He went on thanking the huge SharePoint community (both developers, administrators and customers) for all the support and enthusiasm. Throughout the keynote’s two hours of length, a great number of features were announced or presented, from the new version of SharePoint Online in Office 365 to Yammer, and through all the new stuff in Search, Web Content Management, Enterprise Content Management, Social and, finally, the new Office and SharePoint App model.

The Breakout Sessions were split into three groups, according to the intended audience: IT Professionals, Developers and Business Decision Makers. Each of these groups of sessions took place in a different floor of the convention center, which made it quite simple to go from one to the next, provided they belonged to the same track (i.e. audience).

Besides learning about the new features of the product, one of the main goals of this is event is, of course, Networking. The organization also thought about that and gave the attendees a great number of opportunities to do so:

  • The Conference Reception – Sunday before the start of the conference.
  • Club SPC / Lounge SPC – Two simultaneous parties on Monday evening in two of the bars of the resort.
  • Attendee Party – Mega party in the Mandalay Beach with dinner and live music by Jon Bon Jovi and the Kings of Suburbia, on Tuesday evening.
  • Ask the Experts – Specific period, on Wednesday after the breakout sessions are over, where you can find experts on all subjects related to SharePoint. Maybe it’s not a networking event, but you always end up meeting new people.
  • The Community Lounge – a specific area inside the Exhibition Hall with a bunch of fun stuff and where you could always find people.
  • Avepoint’s Red Party – a partner hosted party, by invitation only and with limited entries, on Monday evening.
  • Axceler’s Party – also a partner hosted party, by invitation only and with limited entries, on Wednesday evening.
  • MySPC website/app allowed you to find attendees that shared the same interests as you, as well as attendees which are connected to your Facebook, Twitter or LinkedIn networks, and setup meetings.
  • Finally, the expo floor was always packed sponsors and exhibitors which would actively engage you in conversation, even if only to scan your badge and tell you about their products or services.

Overall it was a fantastic conference, well worth the time and money. Here are my key takeaways:

  • SharePoint 2013 was designed for the cloud first. This means that, although SharePoint Server 2013 (on premise) and SharePoint Online (in Office 365) are currently equivalent feature-wise, this will probably not be the case in a near future. This is because SharePoint Online will have much shorter update cycles (around 90 days, I was told) and, for that reason, will always get the new features first. Keep this in mind when deciding if you go for SharePoint Online or On Premise.
  • SharePoint Apps are the new and preferred development model for SharePoint. And also in this matter, everyone directs you to hosting your apps in the cloud (although that is not an absolute requirement).
  • JavaScript is starting to emerge as a first class citizen in SharePoint and Office (not to mention Windows 8). So, stop fighting it and start embracing it. You can always use TypeScript to make your life easier as well as a bunch of JavaScript frameworks that can really help you out.
  • Search is everywhere and in the middle of everything.

Over the next few week I’ll be posting more specific articles about the new features.

Office 2013 and SharePoint 2013 reached RTM

Microsoft has just announced (okay, it was three days ago…) that the new Office 2013 family of applications reached the RTM (Release to Manufacturing) milestone. If you’re not familiar with the term, this means the development and testing phases of the product are finished.

According to Microsoft, this is their most ambitious Office release to date since it includes not only the Office client applications, but also the servers and cloud services, which means:

  • Office 2013
  • SharePoint 2013
  • Lync 2013
  • Exchange 2013
  • Office 365

General availability is planned for the first quarter of 2013 but, starting in mid-November, volume licensing customers, developers and IT professionals with TechNet and MSDN subscriptions, will also be able to download the latest bits.

Add that to the Windows 8 launch later this month, and this is shaping up to be a busy last quarter for 2012.

Office 2013 and SharePoint 2013 Preview

Just in case you’ve been disconnected from the outside world for the past 24 hours (and for some reason you can still read this blog), Microsoft has just released the Preview Version of Office 2013 and SharePoint 2013 so that everyone can try it.

The past few months have been filled with rumors and guesses about the new features of Office and SharePoint. Well, no need for rumors anymore. You can actually try all the shinny new bells and whistles. Start here:

Microsoft did it the right way this time, and also released a ton of documents to help us get up to speed with the new features. I’m just starting to browse through all the documentation, blogs, samples and downloads, but I’ll be sure to post my findings here throughout the next few months.

FAST Search Tip #1: Enabling Stemming in PowerShell

Recently, I’ve been working with FAST Search for SharePoint 2010 and, although it’s a great search platform with pretty impressive features, I believe there are still too many manual installation and configurations steps, and too many of those features are only accessible through PowerShell. Don’t get me wrong, PowerShell is great and, by far, the best way to deploy your solution to the client’s environment, but a little more graphical UI wouldn’t hurt.

According to FAST Search’s TechNet documentation, you can use the Set-FASTSearchMetadataManagedProperty cmdlet to configure the main settings of a managed property. Among other things, you can enable stemming for a specific managed property. Here’s how it’s done:

$title = Get-FASTSearchMetadataManagedProperty –Name title
Set-FASTSearchMetadataManagedProperty –ManagedProperty $title –StemmingEnabled $true

The problem is that this doesn’t work. The StemmingEnabled argument is actually ignored by the cmdlet so, we need to use a different approach. This is the right way:

$title = Get-FASTSearchMetadataManagedProperty –Name title
$title.StemmingEnabled = 1

Although Microsoft is yet to update TechNet’s documentation, it has published a support article in its knowledge base (KB2468430) which describes this very solution.

SharePoint Tip #27: Choice Columns and DataSheet Views

Using features to create Choice site columns is pretty straightforward (check this post for additional information). See below a sample feature element for that purpose.






    DisplayName="Publishing Status"




    Group="Custom Columns">

    <Default>Not Published</Default>



          Not Published













Although the XML is well formed and all the attributes and values are correct, this field definition has a problem. If you create a site column using this definition, here is what happens:

  • If you check the site column configuration in SharePoint, everything will look fine
  • If you add this site column to a list (either through a content type, or directly) and you edit an item using the default list forms, everything will work as expected.
  • If you try to edit the list items using a DataSheet View, you won’t be able to select any value from this choice field and SharePoint will always throw a validation error.

The problem here are the invisible characters (spaces, newlines and tabs) between the <CHOICE> and </CHOICE> tags and their inner values. Apparently, when using list forms these characters are trimmed from the valid choices, but when using the datasheet view they are not, causing a strange behavior when editing an item in that view.

The correct definition for this field would be:






    DisplayName="Publishing Status"




    Group="Custom Columns">

    <Default>Not Published</Default>


       <CHOICE>Not Published</CHOICE>






SharePoint Tip #26: Comments in Content Type Definition

There is an issue with SharePoint 2010’s CAML parser that causes SharePoint to ignore fields in a Content Type definition. When defining a Content Type feature element, avoid placing comments inside the <fieldrefs> element as that will cause SharePoint to create the Content Type disregarding all the fields, even though the XML is well formed.

<?xml version="1.0" encoding="utf-8" ?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
    Description="Simple Custom Content Type" 
    <Folder TargetName=”_cts/MyCT” />
      <FieldRef ID="{fa564e0f-0c70-4ab9-b863-0177e6ddd247}" Name="Title" />

      <!-- This is a comment –->   
      <FieldRef ID="{b402db15-ee44-4ec4-89e3-23e10a8fc64c}" Name="My_x0200_Field" />
      <FieldRef ID="{538c71e4-8650-4ce7-b021-920effa66346}" Name="Status" />

The Content Type feature element above will cause the Content Type to be created with only the Title field, which is inherited from the parent Content Type. All the fields referenced in this Content Type definition will be ignored.

Free Webinar: LINQ to SharePoint

The European SharePoint Conference organization committee has been setting up a series of free SharePoint-related webinars in preparation for the conference. I’ll be presenting one of these webinars, next Tuesday, June 14th 2011, at 12PM CET, about LINQ to SharePoint.

The abstract for this webinar is as follows:

Regarding data access, one of the new features introduced with SharePoint 2010 is the possibility to use LINQ to manipulate list items. However, despite being immensely useful, this technology has its own limitations. During this webinar André will talk about what you can and can't do with LINQ to SharePoint, as well as the scenarios where this technology should or should not be applied.

Although it is a free webinar, you must register to be able to attend.

Business Connectivity Services

A few weeks ago I had an article of mine published on PROGRAMAR magazine, issue number 28, a Portuguese developer-focused magazine. Here is the translated version of the article.

SharePoint 2010 is a complex platform with an impressive set of out-of-the-box features which allow it to fit a great variety of situations. One of the new and most powerful features is called Business Connectivity Services and this article is an introduction to this technology and its potential.

What are Business Connectivity Services and what are they for?

Business Connectivity Services (BCS) is the name of the technology that allows SharePoint 2010 and Office 2010 to read from and write information to external systems. It’s an evolution of the Business Data Catalog (BDC) technology introduced in SharePoint 2007, and on which several improvements were made, namely:

  • Possibility not only to read, but also to write information to the external data source;
  • Support for more complex authentication scenarios;
  • Support for multiple data sources;
  • Integration with Office client applications;
  • New and better ways to present the data;
  • Tools specifically focused on creating and manipulating models;
  • Extensibility through .Net assemblies.

The main goal of this technology is to allow the integration of information from external systems and present it on SharePoint and Office client applications with the minimal effort possible and, ideally, without writing a single line of code. There are, in fact, some scenarios where you can use BCS technology only through configuration, but the true flexibility lies in its extensibility which allows it to fulfill almost any integration need.


BCS technology is not limited to a service or API inside SharePoint 2010. In fact, it’s a set of components, services and tools as presented in the schema below.

Fig.1: Business Connectivity Services Architecture

Business Data Connectivity (BDC) Service

The Business Data Connectivity Service is one of the core components of BCS. It’s the component which stores, in its central metadata repository, the descriptions of the information which we want to access as well as the external systems that holds it.

Metadata Store

The metadata repository is the database used by the Business Data Connectivity Service to store the descriptions of the information and the external systems that hold it. This repository does not store any of the information from the external systems, only the metadata required to retrieve it.


The connectors are the pieces the allow the Business Data Connectivity Service to connect to the external data sources described in the models stored in its Metadata Store. Three connectors are included in the product:

  • Database Connector – allows the connection to SQL Server databases.
  • WCF/Web Services Connector – allows the connection to WCF services or web services.
  • .Net Assembly Connector – allows connections using a custom developed .Net assembly, allowing access to virtually any external data source, including simultaneous connections to multiple data sources.

The connector mechanism is extensible and allows custom connectors to be developed, for cases when the out-of-the-box connectors are not enough.

BDC Client Runtime

Office 2010 applications are also able to expose information from external systems through BCS. That is possible because Office 2010 includes the BDC Client Runtime, a component that, in the client application context, performs the same tasks that the BDC Service performs in the server context. That is, it accesses the metadata repository and, through the definitions stored in it, it accesses the external information itself.

Client Data Cache

To improve performance when accessing information, as well as to support offline access to information, the BCS use a cache to store the retrieved external data. This cache is based on a SQL Server 2005 Compact Edition database and has an automatic synchronization mechanism that allows all changes performed on the data in offline mode to be replicated as soon as the external system becomes available once again.

Core Concepts

Now that you know the architecture for Business Connectivity Services, it’s important to understand what kind of metadata is stored in the Metadata Store by the Business Data Connectivity Service.


The metadata used by the BDC Service and stored in the Metadata Store consists of XML files which describe Models, usually named BDC Metadata Models. In SharePoint 2007, these metadata files were called application definition files.

A model contains, in declarative form, all the necessary information so that the BCS are able to connect to an external system and retrieve the data stored in it.

Lob System

Regarding BCS, the Lob System (or Line-of-Business System) refers to the external system which stores the data we want to access. This system can be a relational database, or any other system that exposes that data through web services or WCF services.

External Content Type

The External Content Type (ECT) is the core and most important concept of the BCS, since it describes a business entity, that is, it describes the structure and behavior of the data we want to access. Examples of ECTs are Client, Invoice or Employee.

The ECT definition specifies the structure and behavior of the entity, that is:

  • The fields that compose an instance of the entity and their respective data types. Example: Name, Address or Country.
  • The mapping of these fields to objects used by Office client applications. Example: the field Name of the entity maps to the FullName field in Outlook.
  • The methods that must be invoked by the BCS to read, create, update and delete instances of the entity. These methods might correspond, for instance, to stored procedures, SQL queries or web services.

In the context of a Model, we can define several entities, that is, several External Content Types.


The methods are abstractions of the external system’s API and they allow BDC Service to know which stored procedures or web services to call when manipulating the data. The definition of a method is created in the context of an entity and is always based on one of the stereotypes supplied by the BCS. There are about 20 stereotypes, such as Updater (to update an item), Finder (to list items), SpecificFinder (to retrieve a specific item), Deleter (to delete an item) among others.


Filters describe the parameters that can be passed to the methods in the definition of each entity. There are 18 types of filters that can be used in methods and which let the BDC Service know which information should be passed to them. Examples of filters are the UserName, which passes the username of the current user, or the Limit, which defines the maximum number of items that are returned in a single method call.


An association is a relationship between two entities (External Content Types). However, since there is no guarantee that the data source is a relational database, the association requires that a special method exists. This method allows the BCS to retrieve elements of the related entity from elements of the source entity.

Model Sample

Fortunately, in most cases, we don’t need to manually edit the model since the tools supplied by Microsoft allow us to visually perform most of the configurations. However, just as an example, a model file looks like the one shown below.

Fig.2: Partial sample of a BDC Model

Presenting External Data

But all this complexity has a purpose – to allow the presentation and manipulation of the data stored in external systems – so there are several alternatives to do so.

External List

The External List is a new type of list in SharePoint 2010 which, being bound to an External Content Type, allows the visualization and manipulation of the data exposed through that ECT as if it was stored in a regular SharePoint list. In reality, the data still resides in the external system and it’s read and manipulated in real time.

Fig.3: External list

The greatest advantage of External Lists is the fact that they look and feel exactly like regular lists and, additionally, SharePoint’s object model treats them as such, allowing developers to read and write list items as if they were stored in SharePoint.

On the other hand, not all works exactly as in traditional lists. In particular:

  • Workflows
  • Alerts
  • Folders
  • Attachments
  • RSS Feeds
  • Export to Excel

Before an ECT can be used in an External List, it has to define, at least, a Finder method (to list items) and a SpecificFinder method (to retrieve a specific item). This allows the External List to present the list of items and the detail for each one. Additionally, if the ECT has an Updater method (updates an item), a Deleter method (deletes an item) and a Creator method (creates a new item), the External List will make the corresponding actions available.

External Data Column

The External Data Column was already around in SharePoint 2007 and, although it has been slightly improved in SharePoint 2010, its objective remains the same – allow the use of external data as a list column. It works similarly to a lookup column, allowing the user to select one of the items return by the ECT.

Fig.4: External Data Column

One of the advantages of External Data Columns is the possibility to used them in Word 2010, allowing the user to select one item exposed through the BCS and use that data inside the documents.

Just like for External Lists, to use an ECT in an External Data Column, the ECT must define, at least, the Finder and SpecificFinder methods.

Business Data Web Parts

Business Data Web Parts are a set of web parts that are able to connect to external data sources through an ECT and present that information in SharePoint. These web parts were also present in SharePoint 2007 but were improved in SharePoint 2010, now being able to cache the external information for better performance.

Fig.5: Business Data Web Parts

The Business Data Web Parts use XSLT to present the information, which grants them enormous flexibility regarding its graphical appearance as well as the possibility to be edited in SharePoint Designer 2010.

The web parts included in this set are:

  • Business Data List – lists instances (items) of an entity (ECT).
  • Business Data Item – presents the details of an instance (item) of an entity (ECT).
  • Business Data Item Builder – uses query string parameters to create an instance (item) of an entity (ECT) which can then be passed on to other web parts, such as the Business Data Item web part.
  • Business Data Related List – lists instances (items) of a related entity (ECT). It’s especially useful to present data in Master/Detail scenarios.
  • Business Data Connectivity Filter – filters the data retrieved from an ECT before it is consumed by another web part, such as the Business Data List web part.
  • Business Data Actions – presents the available actions for an instance (item) of an entity (ECT).


One of the largest benefits offered by the BCS is the possibility to index and search external data exposed through the ECTs as if it is stored in SharePoint lists.

For an ECT to be indexed it has to define, at least, the IDEnumerator method and the SpecificFinder method. The first will allow SharePoint to retrieve the IDs of all the items and the second to retrieve each item’s detail. Additionally, the model must have the ShowInSearchUI property so that SharePoint can use it for indexing.

But this is for the data to be indexed. To allow the users to see detailed information about a specific item when they click on a search result, we also need to configure the Profile Page for each indexed ECT.

A Profile Page is just a page in SharePoint with a few predefined web parts which receives the item identifier through a query string parameter and presents detailed information about that item, including items of related ECTs (through Associations).

Profile Pages are configured in Central Administration, accessing the Business Data Connectivity Service management page. The only information we need to supply is the URL address of the website where these pages will automatically be created, and SharePoint will do the rest for us.

At this point, it suffices to tell the Search Service that it must index a new Content Source of type Line of Business Data and then start a Full Crawl. After the crawl finishes, the external data is available to be searched and each item’s information will be presented in its respective Profile Page.

User Profiles

Using the BCS, SharePoint 2010 can use external data sources to complement User Profile information. To do that, you just need to able to map User Profiles to items of an ECT, using a field on each side (such as the username).

It is not possible to configure an ECT as a main data source for the profile synchronization, but you can complement an Active Directory synchronization with data retrieved from an ECT.

Office Client Integration

The integration of external data in Office 2010 applications is also one of the new SharePoint 2010 features in what regards the Business Connectivity Services. Until now, this kind of functionality was only possible through considerably complex custom developments.

With the BCS it’s possible to present external data in Office applications, use it in offline scenarios and, in specific cases, update the data directly in the external data source. However, not all the Office 2010 applications have native support for such integration. At the moment, only Outlook 2010, Word 2010, Access 2010, InfoPath 2010 and SharePoint Workspace 2010 are able to do it, being that each one uses this technology in a different way.

Outlook 2010

Outlook 2010 is one of the Office applications that profits the most from the client features of the BCS. To be able to view the data exposed through an ECT in Outlook 2010, two steps are required when configuring that ECT:

  1. Defining what is the type of data exposed by the ECT among the types of data supported by Outlook: Contacts, Tasks, Appointments or Posts. This configuration can be performed in SharePoint Designer or directly in the Model’s XML.
  2. Mapping the ECT fields with the Outlook fields used in that type of data. For instance, specify which fields of the ECT map to the fields Last Name, First Name, E-mail Address and others, in Outlook.
If there is an External List which exposes the data from the ECT, it will have a Connect to Outlook button in its ribbon. By pressing that button SharePoint will analyze the ECT specification and include it in a Click Once installation package. That package will be immediately deployed to the user’s Outlook 2010 as an Office Add-In.

Once the package is installed, the list will appear in Outlook’s interface allowing the user to interact with the external data as if it was a list of regular contacts, tasks, appointments or posts. In case the ECT defines the required methods, you can even use Outlook to update the data from the external data source. All the fields exposed by the ECT which are not mapped to fields of the Outlook object are shown in a separate section of that object’s detail and can also be updated.

Fig.6: Additional information (unmapped) from the ECT in Outlook’s interface

Like all the other Office applications, Outlook takes advantage of a data caching and synchronization mechanism allowing the user to work offline and automatically synchronizing the data as soon as the external system becomes available.

Word 2010

Word 2010 is another of the Office applications which has native support for the BCS. However, the use cases for this technology in Word are different from the ones available in Outlook. Using BCS in Word 2010 is limited to inserting external data in documents through Quick Parts.

For those who never heard of them, Quick Parts are a Word feature which allows the creation of dynamic fields in the middle of a document’s text. These fields can be automatically filled with data from the document’s content type, in SharePoint. This includes data from an External Data Column that exists in the Document Library where the document is stored.

Here’s how it works:

  1. In a Document Library, create an External Data Column configuring it to expose the data of a specific ECT and defining the fields of the ECT which are exposed.
  2. Create a new document in that library, using the New button from the ribbon.
  3. In Word, through the Insert ribbon, insert one (ore more) Quick Part, selecting the Document Property(s) that match the external data you wish to include in the document.

    Fig.7: Quick parts with external data

  4. Word will allow the user to select an item from the ECT, using the Entity Data Picker, and will automatically populate all the related fields.

Fig.8: External Data Picker

Access 2010

Access 2010 is able to import a Business Data Connectivity (BDC) model an present the external data as regular database tables. However, the created tables are read-only, which means you cannot write back to the external data source.

InfoPath 2010

When an External List is created, the corresponding forms for viewing, adding and editing the external data are also generated. By default, these forms are generated as regular ASP.NET pages but, using SharePoint Designer or the External List ribbon, it’s possible to create smarter forms using InfoPath. The forms are also automatically generated but they can later be modified using InfoPath.

It is also possible to drag an External Data Picker to an InfoPath form and define an External List as a data source, thus allowing reading from and writing to external data sources.

SharePoint Workspace 2010
SharePoint Workspace 2010 is the evolution of Groove 2007 and is positioned as the tool for offline access to data stored in SharePoint 2010, including External Lists. Just like for the other types of lists, you just need to click the Sync to SharePoint Workspace button so that their content is downloaded to the user’s machine becoming available when she is disconnected from the server.

Regarding BCS, what SharePoint Workspace does is download the ECT definition associated with the External List and store it locally, as well as the list forms for inserting, editing and viewing the data which were generated for that External List. Just like the other described Office applications, SharePoint Workspace uses the local cache to ensure the availability of the external data even when the external system is not available.

Solutions and Tools

One of the most frequent complaints of who used Business Data Catalog in SharePoint 2007 was the lack of proper tools that allowed for a good experience when creating and manipulating metadata (models). Microsoft took notice of those complaints and supplied us with two fantastic tools to use with Business Connectivity Services.

  • SharePoint Designer 2010
  • Visual Studio 2010

SharePoint Designer 2010

SharePoint Designer 2010 is a free and mandatory tool for any SharePoint power user or developer. It includes a large set of features mainly focused on creating no code solutions, that is, SharePoint customizations that do not require any custom code.

Fig.9: Using SharePoint Designer to manage ECTs

Regarding Business Connectivity Services, SharePoint Designer 2010 allows:

  • Creating and manipulating External Content Types, including changing configurations, creating new methods and mapping ECTs with Office objects. When creating ECTs you are only able to connect to SQL Server databases, web services whose schemas are supported by BCS or existing .NET assemblies.
  • Creating and configuring External Lists based on existing ECTs.
  • Generating and editing InfoPath forms to support External Lists.
  • Using external data in workflows.
  • Creating web part pages and profile pages.

These features allow the use of BCS without any custom coding and fulfill the most common and simple requirements.

Visual Studio 2010

With Visual Studio 2010 we can create more complex solutions for cases in which SharePoint Designer 2010’s features are not enough. Additionally, with Visual Studio 2010 we can create reusable components which can then be incorporated in solutions through SharePoint Designer.

Fig.10: Visual editor for BDC Models in Visual Studio 2010

Some of the use cases for Visual Studio 2010 are:

  • Creating and manipulating External Content Types using the new Business Data Connectivity Model project template. This template includes a set of designers that allow for the visual editing of the Model and respective ECTs, and also the development of .NET code solutions to access virtually any type of external data source.
  • Creating reusable components for BCS using the multiple API extensibility points, such as Code Actions which can be used inside Outlook, External Data Parts which can be used in declarative task panes in Outlook, custom workflow activities and others.
  • Creating Office application add-ins with BCS support, using the BCS object model.
  • Creating custom workflows that leverage data from external lists or use the BCS object model.

Types of Solution for each Tool

The table below will help you select the best tool for each need.

Capability SharePoint Designer 2010 Visual Studio 2010
Connect to…
  • WCF/WS, SQL Server and existing .NET assemblies
  • Any data source through .NET assemblies
Development model
  • Discover and configure
  • Create and publish
Suited for…
  • Simple models with native interfaces (External Lists, Outlook, SharePoint Workspace, InfoPath, Search) and simple associations based on foreign keys
  • Complex models with custom connectivity logic for aggregation, transformation, security, etc.
  • Custom interfaces via Office customizations
  • Reusable client and server controls that connect to external data
  • The data source must expose interfaces with supported format
  • No support for advanced stereotypes (bulk operations)
  • No support for generic or polymorphic services
  • Only foreign key associations
  • Visual designer only works for models based on .NET objects
  • Separate development and packaging for client and server components

Use Cases

One of the most common questions regarding the use of Business Connectivity Services is what are the use cases of this technology or, when should I use BCS.

Some of the most common uses cases for BCS are:

  • Presenting information stored in a SQL Server database. Using BCS it’s possible to present and, if necessary, modify the information using External Lists without developing a single line of custom code. It’s like an instantaneous back-office.
  • Enriching the domain user’s Profiles with information from the HR management system or the ERP system. As mentioned before, BCS allows you to fulfill this requirement by configuring an ECT as an additional data source for SharePoint’s profile synchronization.
  • Synchronizing contacts stored in a line-of-business system or ERP. Using BCS it’s possible to define an ECT which exposes these contacts through an External List and connect it to Outlook where they can be managed as regular contacts. This scenario allows the users to access to the contacts even when they’re out of the office, in offline mode.
  • Presenting information from distinct data sources. Using the .NET assemblies connector and developing an ECT with Visual Studio 2010, we can build scenarios with aggregation of data from multiple data sources exposed as a single entity.
  • Indexing and searching information stored in a line-of-business or ERP system. BCS allows SharePoint’s search service to index contents exposes through ECTs and searching them as if the information is stored in SharePoint.

There are several other scenarios where BCS can be useful, sometimes just as one of the components of the solution.

Features per SharePoint Version

The infrastructure used by the Business Connectivity Services is available in all SharePoint versions, including SharePoint Foundation 2010. However, not all is included in the free version. The table below helps to clarify which features are included in each version of SharePoint 2010.

Feature SharePoint Foundation 2010 SharePoint Server 2010 Standard SharePoint Server 2010 Enterprise
BDC Service




Connector Framework




External List




External Data Column




Secure Store Service  



External Data Search  



Profile Pages  



Business Data Web Parts    


Office Client integration    


The support for Business Connectivity Services in Office applications requires Microsoft Office 2010 Professional Plus, or higher.

Useful Links

Here are a few useful links for who is now starting and wants to know more about Business Connectivity Services.

Microsoft Business Connectivity Services Team Blog
Official blog of the team that developed BCS, with several articles with multiple levels of complexity. Mandatory to all the interested in this technology.

BCS Team Channel
The YouTube channel where the product team published a few videos.

Connecting to a .NET Framework Source Using Business Connectivity Services in Office 2010
Visual How To about developing an ECT using Visual Studio 2010 to retrieve data from an external source.

Microsoft Business Connectivity Services
SharePoint 2010 SDK’s section dedicated to Business Connectivity Services.

Business Connectivity Services Resource Center
TechNet Resource Center dedicated to Business Connectivity Services.

Business Connectivity Services: Technical Articles
MSDN technical articles related to Business Connectivity Services.

European SharePoint Conference 2011

I’m pleased to announce I have been chosen as a speaker in the European SharePoint Conference 2011, which will take place in Berlin, 17th – 20th October 2011.

I’ll be delivering 3 sessions, two on Wednesday and one on Thursday, as described below.

W18 - Using BCS to Integrate Azure Services with SharePoint 2010

The new Business Connectivity Services (BCS) offer a simple and elegant way of integration SharePoint 2010 and Office 2010 with virtually any business data source. All the business integration features introduced in SharePoint 2007 with Business Data Catalog (BDC) were significantly improved in this new version. In this session we'll talk about the main concepts and the type of solutions that can be developed using BCS, and we'll show how to integrate Azure Services with SharePoint 2010 and Office 2010 using BCS, Visual Studio 2010 and SharePoint Designer 2010.

W26 - Federated Authentication in SharePoint 2010

SharePoint 2010 introduces a new claims-based authentication model which leverages the power and flexibility of the new Windows Identity Foundation. In this session we will show how to use Windows Azure Access Control Services to allow a user to authenticate in SharePoint using a web identity such as Facebook or Google account.

Th28 - SharePoint + Azure = Better Together

Microsoft has a large range of technologies that, when combined, can yield fantastic results with surprisingly little effort. What happens when you mix SharePoint 2010, BizTalk 2010, Windows Azure and Microsoft Pivot? You get a fantastic interface to analyze large amounts of visual data which can be store in SharePoint 2010 and manipulated in the cloud. In this session we'll show the concepts behind this kind of integration and how you can build such a scenario.

Note, however, that the conference program is still subject to change. Check the full program at http://www.sharepointeurope.com/conferences/2011/conference-at-a-glance.aspx.

I look forward to seeing you there!

SharePoint PT: 10th Meeting

If you’re into SharePoint and you happen to be in Lisbon next Saturday, April 16th, then you may be interested in the 10th Meeting of the Portuguese SharePoint Community (SPUG PT). The meeting is free and it will take place between 9.30AM and 1.00PM at Microsoft Portugal’s auditorium, in Porto Salvo (Tagus Park).

09.30AM – Reception
10.00AM – Introductions
10.30AM – Session: “LINQ to SharePoint” by André Vala
11.30AM – Coffee break
11.45AM – Discussion panel
01.00PM – End of meeting and lunch (optional)

If you wish to attend, please register here: spugmeeting10.eventbrite.com

I hope to see you there :)

More Posts Next page »