The first step here is to ensure that your consumer code is idempotent, but that’s something that you should always ensure – just imagine that you have to re-consume a couple of messages, in that case lacking idempotency may result in your system getting strange data and bugs.

Setting up Kafka on your machine

(Feel free to skip this section if you have everything set up.)

Kafka releases aren’t included in most package managers, except for Homebrew in Mac systems. They should be available, however, in the official download page. For Unix systems (Mac, Linux, WSL, *BSD), simply get one of the binary downloads and extract it wherever you prefer. Then, make sure to add the bin folder to your $PATH, so that you can use these scripts in any folder. In my case, I put the extracted archive in $HOME/.local/share/kafka, so I edit my .zshrc (or .bashrc in a plainer Linux/WSL system) and add:

export PATH=$HOME/.local/share/kafka/bin:$PATH

Now we can use the official kafka scripts in any directory. That is, unless we have some form of authentication. We can add additional configs to a file somewhere, and include that file in our commands. Let’s put a file called dev.cfg in $HOME/.config/kafka/ (the exact directory doesn’t matter, but .config is the appropriate directory for configs), and write the following on it:

security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
    username="me" \
    password="noneofyourbusiness";

This is just an example, your broker’s security config may vary. After doing this, just add to any of the Kafka commands you use:

--command-config $HOME/.config/kafka/dev.cfg

Checking consumer offsets

Before migrating topics, it’s best if we make sure that the old topic consumer is totally up-to-date. We can use this command to check the consumption lag:

kafka-consumer-groups.sh \
	--bootstrap-server='<your broker endpoints, separated by commas>' \
	--describe \
	--group='<your consumer group>'

This should yield something like the output below:

TOPIC      PARTITION   NEWEST_OFFSET   OLDEST_OFFSET     CONSUMER_OFFSET     LEAD         LAG
your-topic 0           30000000        0                 30000000            30000000     0
your-topic 1           23900000        0                 23900000            23900000     0

CLIENT_HOST      CLIENT_ID         TOPIC          ASSIGNED_PARTITIONS
/128.0.0.0       your-consumer     your-topic     0
/128.0.0.1     	 your-consumer     your-topic     1

From this we can see that there is zero lag, since the newest offset is equal to the consumer offset on both partitions. We can also see that our consumers are still running (they’re the clients with ID “your-consumer”).

Performing the Kafka consumer migration

Before switching topics or brokers, we should first create a consumer group for the new configuration. This will allow us to change the offsets before beginning the message consumption proper. The easiest way to do this is to use kcat, a simple command-line tool that allows us to consume and produce Kafka messages. To create a consumer group, you must consume messages, so it’s a matter of setting up kcat (check the install guide on the kcat repo) and consuming a few messages from the new topic:

kcat \
	-C \
	-b '<your brokers>' \
	-t new-topic \
	-G new-consumer-group

# If you have authentication, just add: 

	-X security.protocol=SASL_SSL \
	-X sasl.mechanisms=SCRAM-SHA-512 \
	-X sasl.username="me" \
	-X sasl.password="noneofyourbusiness"

Just run this for a second, check that messages come out, and then run the describe command for the new group:

kafka-consumer-groups.sh \
	--bootstrap-server='new-broker' \
	--describe \
	--group='new-group'


TOPIC     PARTITION   NEWEST_OFFSET   OLDEST_OFFSET     CONSUMER_OFFSET     LEAD         LAG
new-topic 0           12387000        0                 12                  12           12386988
new-topic 1           12360000        0                 38                  38           12359962

CLIENT_HOST      CLIENT_ID         TOPIC          ASSIGNED_PARTITIONS

We have huge lag here, but that’s fine, as we’re going to reset the offsets anyway. Stop the workers consuming from the older topic by whatever means you can and note down the time at which they stopped. We will now reset the consumer group offset for this new group. Just a note: in this case, we don’t have any consumers connected to this group, but if we did we’d also have to turn them off. You cannot reset a consumer group offset if there are clients connected to it.

To reset the offsets, just run the following command (here we’ll assume that we stopped the previous consumers at 10:00 of March 15th, 2023, and subtract ten minutes just to be safe):

kafka-consumer-groups.sh \
	--bootstrap-server='new-brokers' \
	--group='new-group' \
	--topic='new-topic' \
	--reset-offsets \
	--to-datetime 2023-03-15T09:50:00.000 \
	--dry-run

Running this dry run will not make any changes, It will only show you the new offsets for the consumer group, e.g.:

TOPIC     PARTITION   NEWEST_OFFSET   OLDEST_OFFSET     CONSUMER_OFFSET     LEAD         LAG
new-topic 0           12387000        0                 12386998            12386988     2
new-topic 1           12360000        0                 12369951            12369951     49

CLIENT_HOST      CLIENT_ID         TOPIC          ASSIGNED_PARTITIONS

If nothing looks strange to you, go ahead, substitute --dry-run with --execute and run the command. Afterwards, deploy the new consumer configuration, turn on the worker, and let it run.

The post Performing broker migrations on your Kafka consumers appeared first on Blog IT.

The pipelines bring to us the opportunity to create automatic tasks to execute development operations, like deploys, migrations, tests, etc. Focused in the deploy routine, some applications need other operations to ensure the stability between the application and other resources, like the database.

During the development process, the developers usually customize and improve the application database in the development environment scope, and it can be made easily by using frameworks that contains routines to create source files that have the responsability to keep the history of changes and are used to update the database for determinated version. These files are known as migrations. 

In this post, we will see how to apply migrations for the databases that are located in different environments (staging, production, etc.), using the Code-First entity concept, and the Entity Framework Core.

Migrations

To generate database migrations, we can execute the “Add-Migraton <name>” command from the Entity Framework. This command will generate some source files that will contain the code to apply the migration to the database. To execute these migration files, you just need to execute the “Update-Database” command. But, Microsoft recommends to create a SQL file individually, and suggest to execute this file direct to the database, without need to execute an intermediary software (C# + EF) to access the database. To do this, we will have to follow the steps below:

• 1. Create a Pipeline in Azure Devops;
• 2. Install dotnet-ef-tool;
• 3. Generate SQL script;
• 4. Execute SQL script to the database;
• 5. Script File as a Pipeline Artifact.

NOTE: We don’t need to worry about which migration need to be applyed to the database. The dotnet-ef-tool command will generate a script cantaining the needed logic. Check the third step for more details.

1. Create a Pipeline in Azure Devops

We will create a common deployment pipeline in Azure Devops for ASP.NET Core backend applications. Check the end of the post to see the full result of the pipeline.

2. Install dotnet-ef-tool

To execute tool commands of Entity Framework, we have to install a package named dotnet-ef-tool. This needs to be available into the pipeline’s context to be used for the SQL script generation. To install this tool by a pipeline task, we can use the task below:

- task: DotNetCoreCLI@2
  displayName: 'Install EF tool'
  inputs:
    command: 'custom'
    custom: 'tool'
    arguments: 'install --global dotnet-ef'

3. Generate SQL script

To generate the SQL script containing the logic to apply migrations, we will use the dotnet-ef-tool. By default, the command does not generate the logic to check which migration should be applied, so we have to enable this behavior using the parameter: –idempotent. By using this behavior, we don’t need to worry about the migrations that will be applyed to the database since the generated script already has it. Below, you can see the task that could be used to generate the script file:

- task: DotNetCoreCLI@2
  displayName: 'Generate SQL Script'
  inputs:
    command: 'custom'
    custom: 'ef'
    arguments: 'migrations script --idempotent --project $(Build.SourcesDirectory)\Data.csproj --output $(System.ArtifactsDirectory)/script.sql'

NOTE: We don’t need to stablish any database connection for the script generation, since we are following the Code First concept for entity creation, so, the command will just look at the entities source code of the project to generate the SQL file. If we don’t use the –idempotent argument, the script will generate the SQL without the conditional logic to determinate which migration should be applied, causing an error if the database already exists.

4. Execute SQL script to the database

Now, with the SQL script available, we just need to execute this file to the database. In this post, we will use a common Azure Database as an example, so for this case, we can use the following task to accomplish this requirement:

- task: SqlAzureDacpacDeployment@1
  displayName: 'Update Database'
  inputs:
    azureSubscription: '<Service Connection Identifier>'
    AuthenticationType: 'connectionString' # You can use other method to authenticate to the database if you want
    ConnectionString: '<Database Connection String>'
    deployType: 'SqlTask'
    SqlFile: '$(System.ArtifactsDirectory)/script.sql'

5. Script File as a Pipeline Artifact

Well done. The last task will only be used to make the file available to be accessed by the pipeline runner user. To do this, you can use the task below to add the generated script.sql file to the file bundle (artifacts). To check the artifacts, just access the artifacts option from the pipeline details after the job’s execution.

- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifacts'
  inputs:
    PathtoPublish: '$(System.ArtifactsDirectory)/script.sql'
    ArtifactName: '$(artifactName)'
    publishLocation: 'Container'

Conclusion

In this post, we could learn an easy way to apply the database migrations to the remote environment by using automation (pipelines) to accomplish it. This method is simple, and does not have any database validation before script execution. So, if you have an active and sensitive environment, be sure that the needed validations are considered before the database update, to ensure that the database will not be update wrongly.

Pipeline Result

trigger:
- none

pool:
  vmImage: windows-latest

variables:
  solution: '**/*.sln'
  buildPlatform: 'AnyCPU'
  buildConfiguration: 'Release'
  artifactName: 'artifacts'
  environment: 'Development'

- task: NuGetToolInstaller@1
  displayName: 'NuGet Installation'

- task: NuGetCommand@2
  displayName: 'NuGet Restore'
  inputs:
    restoreSolution: '$(solution)'

- task: VSBuild@1
  displayName: 'Build Project'
  inputs:
    solution: 'backend/WebAPI'
    msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)"'
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

- task: PublishSymbols@2
  displayName: 'Publish Symbols'
  inputs:
    SearchPattern: '**/bin/**/*.pdb'
    SymbolServerType: 'FileShare'
    CompressSymbols: false

- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifacts'
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)'
    ArtifactName: 'artifacts'
    publishLocation: 'Container'

- task: DownloadBuildArtifacts@0
  displayName: 'Build Artifacts'
  inputs:
    buildType: 'current'
    downloadType: 'single'
    artifactName: '$(artifactName)'
    downloadPath: '$(System.ArtifactsDirectory)'

- task: DotNetCoreCLI@2
  displayName: 'Install EF tool'
  inputs:
    command: 'custom'
    custom: 'tool'
    arguments: 'install --global dotnet-ef'

- task: DotNetCoreCLI@2
  displayName: 'Generate SQL Script'
  inputs:
    command: 'custom'
    custom: 'ef'
    arguments: 'migrations script --idempotent --project $(Build.SourcesDirectory)\Data.csproj --output $(System.ArtifactsDirectory)/script.sql'

- task: SqlAzureDacpacDeployment@1
  displayName: 'Update Database'
  inputs:
    azureSubscription: '<Service Connection Identifier>'
    AuthenticationType: 'connectionString'
    ConnectionString: '<Connection String>'
    deployType: 'SqlTask'
    SqlFile: '$(System.ArtifactsDirectory)/script.sql'

- task: AzureRmWebAppDeployment@4
  displayName: 'Deploy Application'
  inputs:
    ConnectionType: 'AzureRM'
    azureSubscription: '<Service Connection Identifier>'
    appType: 'webApp'
    WebAppName: 'armazemlegalapi'
    deployToSlotOrASE: true
    ResourceGroupName: '<Resource Group Identifier>'
    SlotName: 'production'
    packageForLinux: '$(System.ArtifactsDirectory)/Project.zip'

- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifacts'
  inputs:
    PathtoPublish: '$(System.ArtifactsDirectory)/script.sql'
    ArtifactName: '$(artifactName)'
    publishLocation: 'Container'

References

• Microsoft Managing Schemas Article

The post Entity Framework – Update Database using Azure Pipelines appeared first on Blog IT.

Recently, I migrated a SharePoint 2010 farm to SharePoint 2016 and used an automated installation process using PowerShell and AutoSPInstaller to install the new SharePoint 2016 farm that ensured that the latest updates where applied, leaving the SharePoint 2016 farm updated and working properly (if you want to learn more about how to install a SharePoint farm in an automated way using PowerShell, I invite you to click here and here).

After installing the farm and migrating the web applications, new updates were installed in the farm (if you want to keep up with the latest SharePoint updates, click here) but when I ran the SharePoint Products and Configuration Wizard to ensure the new updates were properly installed (to know more about the need to run SharePoint Products and Configuration Wizard after a SharePoint farm update, click here), I got an error that was preventing me to finish the update process.

Important Note: Both source SharePoint 2010 and target SharePoint 2016 farms have Standard SharePoint licenses.

The Problem

After installing the updates and after running the SharePoint Products and Configuration, I was getting the following error:

The error details:

An exception of type Microsoft.SharePoint.PostSetupConfiguration.PostSetupConfigurationTaskException was thrown.  Additional exception information:

Feature (Name = [SharePoint Portal Server Status Indicator List template], Id = [065c78be-5231-477e-a972-14177cc5b3c7], Description = [SharePoint Portal Server Status Indicator List template], Install Location = [BizAppsListTemplates]) is referenced in database [Content_DB], but isn’t installed on the current farm. The missing feature might cause upgrade to fail. If necessary, please install any solution that contains the feature and restart upgrade.        (EventID:ajxkh)

Feature (Name = [SharePoint Server Enterprise Site Features], Id = [0806d127-06e6-447a-980e-2e90b03101b8], Description = [Features such as the Visio Services, Access Services, and Excel Services applications included in the SharePoint Server Enterprise License.], Install Location = [PremiumWeb]) is referenced in database Content_DB] but isn’t installed on the current farm. The missing feature might cause upgrade to fail. If necessary, please install any solution that contains the feature and restart upgrade.        (EventID:ajxkh)

Feature (Name = [SharePoint Portal Server Report Library], Id = [2510d73f-7109-4ccc-8a1c-314894deeb3a], Description = [SharePoint Portal Server Report Library], Install Location = [ReportListTemplate]) is referenced in database [Content_DB], but isn’t installed on the current farm. The missing feature might cause upgrade to fail. If necessary, please install any solution that contains the feature and restart upgrade.        (EventID:ajxkh)

Feature (Name = [InfoPath Forms Services Support], Id = [c88c4ff1-dbf5-4649-ad9f-c6c426ebcbf5], Description = [InfoPath Forms Services Lists and Related Pages to Enable Presentation of Forms on the Server.], Install Location = [IPFSSiteFeatures]) is referenced in database Content_DB], but isn’t installed on the current farm. The missing feature might cause upgrade to fail. If necessary, please install any solution that contains the feature and restart upgrade.        (EventID:ajxkh)

Feature (Name = [Data Connections Feature], Id = [00bfea71-dbd7-4f72-b8cb-da7ac0440130], Description = [], Install Location = [DataConnectionLibrary]) is referenced in database Content_DB], but isn’t installed on the current farm. The missing feature might cause upgrade to fail. If necessary, please install any solution that contains the feature and restart upgrade.        (EventID:ajxkh)

Upgrade [SPContentDatabase Name=Content_DB] failed.        (EventID:an59t)

Exception: The upgraded database schema doesn’t match the TargetSchema        (EventID:an59t)

Upgrade Timer job is exiting due to exception: Microsoft.SharePoint.Upgrade.SPUpgradeException: The upgraded database schema doesn’t match the TargetSchema

There are two main problems here:

• There are references in the content database to SharePoint Enterprise Features (InfoPath Form Services, Visio Services, Excel Services, etc) – my guess here is that sometime along the way, the source SharePoint 2010 farm had an Enterprise SharePoint license and was downgraded to a Standard license, leaving behind however references to Enterprise features in the content database
• There is a mismatch between the content database schema version and the version of SharePoint updates installed in the farm

One additional symptom that confirms there is an upgrade issue is the Upgrade Status page in Central Administration that showed the upgrade status as Failed:

The Solution

The solution to this problem was a two part solution, both applied in the target SharePoint 2016 farm:

• Fixing the missing Enterprise features references in the content database
• Fixing the content database schema version mismatch

Fixing the missing Enterprise features references in the content database

To fix the problem with references to Enterprise SharePoint features, I used the SharePoint Feature Administration and Clean Up Tool, which basically enables us to delete from the Farm, Web Application, Site Collection and sub sites any references to features that are not installed in the farm. The version I used was the version 2.4.8 that you can download here, that supports SharePoint 2016. The version 3.0, for some reason, didn’t detect the farm, which made it useless (maybe a beta version, not sure).

In this case, to remove the missing feature references, I used the SharePoint Feature Administration and Clean Up Tool to:

• Remove the missing features from the farm
• Remove the missing features from the sub sites

First, I removed the missing features from the farm, by removing the features marked as “Faulty” in the “Farm Feature Administration” tab, using the “Uninstall” button for each missing feature.

After that, I removed the features marked as ERROR in the “Remove / deactivate features in the selected sites” tab, by selecting all the site collection faulty features and using the “Remove from selected SiteCollection” button.

Finally, for each site, I selected the faulty site features and removed them by using the “Remove from selected Site (SPWeb)” button.

Fixing the content database schema version mismatch

To fix the version mismatch between the content database schema version and the version of installed SharePoint updates, I executed the following PowerShell command (executed with “run as administrator”):

Get-SPContentDatabase | Upgrade-SPContentDatabase

After this, I ran the SharePoint Products and Configuration Wizard again, and this time, it ran successfully:

To confirm if everything was OK, I checked the following pages in Central Administration in the target SharePoint 2016 farm:

• The Upgrade Status page
• The Database Upgrade Status page

In the Upgrade Status page, there is now a Succeeded upgrade status:

In the Database Upgrade Status page, there is an indication of “No action required” for all content databases in the farm, meaning that all databases were properly upgraded and the respective schema version is the same as the installed SharePoint updates:

Hope this helps!

Related Articles

To learn why your business should migrate to SharePoint Online and Office 365, click here and here.

If you want to convert your tenant’s root classic site into a modern SharePoint site, click here.

In the meantime, SharePoint 2019 RTM is already out there! If you want to know all about the new features available in the new SharePoint Server release, click here.

If you are a SharePoint administrator or a SharePoint developer who wants to learn more about how to install a SharePoint 2019 farm in an automated way using PowerShell, I invite you to click here and here.

If you want to learn how to upgrade a SharePoint 2013 farm to SharePoint 2019, click here and here.

If you want to learn all the steps and precautions necessary to successfully keep your SharePoint farm updated and be ready to start your move to the cloud, click here.

If you learn how to greatly speed up your SharePoint farm update process to ensure your SharePoint farm keeps updated and you stay one step closer to start your move to the cloud, click here.

If SharePoint 2019 is still not an option, you can learn more about how to install a SharePoint 2016 farm in an automated way using PowerShell, click here and here.

If you are involved in a SharePoint upgrade and want to learn more about the upgrade process, click here and here.

If you are new to SharePoint and Office 365 and want to learn all about it, take a look at these learning resources.

If you are work in a large organization who is using Office 365 or thinking to move to Office 365 and is considering between a single or multiple Office 365 tenants, I invite you to read this article.

If you want to know all about the latest SharePoint and Office 365 announcements from SharePoint Conference 2019, click here and here.

Happy SharePointing!

The post SharePoint 2016 Upgrade: How to solve the “Feature is referenced in database but isn’t installed on the current farm” error appeared first on Blog IT.

<Field

    ID="{538c71e4-8650-4ce7-b021-920effa66346}"

    Type="Choice"

    Name="Publishing_x0020_Status"

    StaticName="Publishing_x0020_Status"

    DisplayName="Publishing Status"

    Required="FALSE" 

    Format="Dropdown"

    FillInChoice="FALSE"

    Group="Custom Columns">

    <Default>Not Published</Default>

    <CHOICES>

       <CHOICE>

          Not Published

       </CHOICE>

       <CHOICE>

          Pending

       </CHOICE>

       <CHOICE>

          Published

       </CHOICE>

       <CHOICE>

          Error

       </CHOICE>

    </CHOICES>

</Field>

Although the XML is well formed and all the attributes and values are correct, this field definition has a problem. If you create a site column using this definition, here is what happens:

• If you check the site column configuration in SharePoint, everything will look fine
• If you add this site column to a list (either through a content type, or directly) and you edit an item using the default list forms, everything will work as expected.
• If you try to edit the list items using a DataSheet View, you won’t be able to select any value from this choice field and SharePoint will always throw a validation error.

The problem here are the invisible characters (spaces, newlines and tabs) between the <CHOICE> and </CHOICE> tags and their inner values. Apparently, when using list forms these characters are trimmed from the valid choices, but when using the datasheet view they are not, causing a strange behavior when editing an item in that view.

The correct definition for this field would be:

<Field

    ID="{538c71e4-8650-4ce7-b021-920effa66346}"

    Type="Choice"

    Name="Publishing_x0020_Status"

    StaticName="Publishing_x0020_Status"

    DisplayName="Publishing Status"

    Required="FALSE" 

    Format="Dropdown"

    FillInChoice="FALSE"

    Group="Custom Columns">

    <Default>Not Published</Default>

    <CHOICES>

       <CHOICE>Not Published</CHOICE>

       <CHOICE>Pending</CHOICE>

       <CHOICE>Published</CHOICE>

       <CHOICE>Error</CHOICE>

    </CHOICES>

</Field>

The post SharePoint Tip #27: Choice Columns and DataSheet Views appeared first on Blog IT.

<?xml version="1.0" encoding="utf-8" ?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <ContentType
    ID="0x0100C5647A362F236548B218C15302286758"
    Name="MyCT"
    Description="Simple Custom Content Type" 
    Inherits="TRUE"
    Overwrite="TRUE"
    Version="0">
    <Folder TargetName=”_cts/MyCT” />
    <FieldRefs>
      <FieldRef ID="{fa564e0f-0c70-4ab9-b863-0177e6ddd247}" Name="Title" />

      <!-- This is a comment –->   
      <FieldRef ID="{b402db15-ee44-4ec4-89e3-23e10a8fc64c}" Name="My_x0200_Field" />
      <FieldRef ID="{538c71e4-8650-4ce7-b021-920effa66346}" Name="Status" />
    </FieldRefs>
  </ContentType>
</Elements>

The Content Type feature element above will cause the Content Type to be created with only the Title field, which is inherited from the parent Content Type. All the fields referenced in this Content Type definition will be ignored.

The post SharePoint Tip #26: Comments in Content Type Definition appeared first on Blog IT.

By default, the assembly is included in the package and is deployed to the Global Assembly Cache. You can change both configurations by clicking on the project in Solution Explorer and editing the properties on the Properties window.

However, there is no project property that allows you to specify you want to add the assembly to the Safe Controls section on the web application configuration file. To do this, follow the steps below:

1. On Solution Explorer, click on the project
2. On the Properties window, specify you don’t want the assembly to be included in the package (set Include Assembly in Package as False)

3. On Solution Explorer, double-click the Package folder to open the Package editor
4. On the Package Editor, click the Advanced button (bottom of the screen)

5. Click the Add button
6. Select Add Assembly from Project Output…

7. Select the current project (which contains the package)
8. Add the safe controls information

The post Adding an Assembly to Safe Controls in SharePoint Tools for Visual Studio 2010 appeared first on Blog IT.

If you create a new SharePoint project named MyProject, and then create a feature inside it named MyFeature, the final name and folder for the feature will be MyProject_MyFeature.

I don’t know about you, but I like the features to have the names I give them, without the project’s name added to them. The good news is that you can change it 🙂

Here’s how:

1. On Solution Explorer, double-click the Feature name to open the Feature Editor
2. On Packaging Explorer, click on the Feature name
3. On the Properties window, change the value in the Deployment Path property

By default, the value for this property is:

$SharePoint.Project.FileNameWithoutExtension$_$SharePoint.Feature.FileNameWithoutExtension$

To remove the project’s name from the Feature name, just remove the first token (and the trailing underscore), leaving it as:

$SharePoint.Feature.FileNameWithoutExtension$

The post Renaming a Feature in SharePoint Tools for Visual Studio 2010 appeared first on Blog IT.

Why should I use Resources?

In my opinion, you should always use resource files for your features. Why?

• First, because it’s easy to do even if you only have one language.
• Second, because if you do, all the objects you are provisioning in SharePoint will be localized and will be displayed in the correct language of the website (provided you have deployed the resource file for that language).

Examples of things that should be placed in resource files:

• Title and Description of you feature, so that they will be displayed in the language of the website;
• Site column display name and description;
• Content type name and description;
• List template name and description;
• List instance title and description;
• Custom action title and description.

How do I use it?

There are three steps to use resources in a feature:

1. Build the resource (.resx) files;
2. Place the resource files in the correct folder;
3. Use the resource strings in the feature manifest and element manifests.

Note: The next sections show how to use feature-specific resources. I’ll briefly discuss shared resources in the end of the post.

Building the resource (.resx) files

The best way to build the resource files is using Visual Studio 2008, but you can do it manually on any text editor since they’re just XML text files.

<?xml version="1.0" encoding="utf-8"?>
<root>
  <resheader name="resmimetype">
    <value>text/microsoft-resx</value>
  </resheader>
  <resheader name="version">
    <value>2.0</value>
  </resheader>
  <resheader name="reader">
    <value>
      System.Resources.ResXResourceReader, 
      System.Windows.Forms, 
      Version=2.0.0.0, 
      Culture=neutral, 
      PublicKeyToken=b77a5c561934e089
    </value>
  </resheader>
  <resheader name="writer">
    <value>
      System.Resources.ResXResourceWriter, 
      System.Windows.Forms, 
      Version=2.0.0.0, 
      Culture=neutral, 
      PublicKeyToken=b77a5c561934e089
    </value>
  </resheader>
  <data name="MyFeatureName" xml:space="preserve">
    <value>My Feature Name</value>
  </data>
  <data name="MyFieldDisplayName" xml:space="preserve">
    <value>My Field Display Name</value>
  </data>
  <data name="MyFieldChoice1" xml:space="preserve">
    <value>My Choice 1</value>
  </data>
  <data name="MyFieldChoice2" xml:space="preserve">
    <value>My Choice 2</value>
  </data>
</root>

See above an example of a resource (.resx) file. The <data> elements are the ones that hold the localized strings:

• The name attribute is the key used to retrieve the localized string;
• The <value> child element is the localized string itself.

Each language must have its own resource file which follows a specific naming convention:

• The main resource file must be called Resources.resx. This is the culture neutral resource file (or fall back resource file) which will be used by SharePoint whenever there is no resource file for a specific culture.
• All culture specific resource files must be named Resources.[culture].resx. Some examples:
  • Resources.en-US.resx (english – United States)
  • Resources.pt-PT.resx (portuguese – Portugal)
  • Resources.fr-FR.resx (french – France)
  • Resources.pt-BR.resx (portuguese – Brazil)

Placing the resource files in the correct folder

These files must be placed in a Resources folder inside your feature’s folder. So, if your feature’s folder is C:\…\12\TEMPLATE\FEATURES\MyFeature, the resource files need to be placed in the folder C:\…\12\TEMPLATE\FEATURES\MyFeature\Resources.

Resources and the Feature Manifest

After you have created the resource files and placed the in the correct folder, you can now used them in your feature manifest (feature.xml).

<?xml version="1.0" encoding="utf-8" ?>
<Feature xmlns="http://schemas.microsoft.com/sharepoint/"
         Id="5DFD12AF-D0AA-4c63-8FB8-C49DB1191083"
         Title="$Resources:MyFeatureName"
         Scope="Site"
         Version="1.0.0.0">
  <ElementManifests>
    <ElementManifest Location="SiteColumns.xml"/>
    <ElementFile Location="Resources\Resources.resx" />
    <ElementFile Location="Resources\Resources.pt-PT.resx" />
    <ElementFile Location="Resources\Resources.es-ES.resx" />
  </ElementManifests>
</Feature>

The sample above, shows a feature manifest that uses resources to specify the feature title. As you can see, the value of the Title attribute is $Resources:MyFeatureName. This tells SharePoint that it should check the resource file for the current website’s culture and retrieve the string that has the key MyFeatureName.

Notice the use of <ElementFile> elements to reference the resource files in the feature. This is required if you are deploying this feature through a solution package (.wsp).

Important: because we are using feature-specific resource files (resource files that are only used for this specific feature), you cannot use the DefaultResourceFile attribute on the <Feature> element. If you do, SharePoint will not look for resource files in the local Resources folder for this feature.

Resources and the Element Manifest

Besides using localized strings to specify the feature title and description, you can (and should) use localized strings for most feature elements.

<?xml version="1.0" encoding="utf-8" ?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
    <Field Type="Choice"
         DisplayName="$Resources:MyFieldDisplayName"
         Required="FALSE"
         Format="Dropdown"
         FillInChoice="FALSE"
         ID="{485b2176-4cfc-4923-8085-c003b85dab36}"
         StaticName="MyField"
         Name="MyField">
    <Default>$Resources:MyFieldChoice1</Default>
    <CHOICES>
      <CHOICE>$Resources:MyFieldChoice1</CHOICE>
      <CHOICE>$Resources:MyFieldChoice2</CHOICE>
    </CHOICES>
  </Field>
 </Elements>

The sample above specifies a choice field, and uses localized strings for:

• Field display name
• Each of the field’s choices
• The field’s default choice

The theory is exactly the same as it was used for the feature manifest: you use the expression $Resources:[key] whenever you want SharePoint to retrieve a localized string from the resource file of the current website’s culture.

Shared Resource Files

The method shown in the previous sections assumed that each feature has its own resource files. However, that is not always the case and SharePoint itself uses shared resource files. That is, resource files that can be shared by multiple features.

There are only two differences when using shared resource files:

• The folder where the resource files are placed;
• The way you reference the localized string in you feature manifest and element manifests.

Other than that, the file format and contents can be exactly the same.

Folder for Shared Resources

Shared resource files must be placed in the folder C:\…\12\Resources instead of the local Resources folder inside the feature folder.

Referencing Shared Resources

Because the shared resources files can have any name you want, you must reference the localized strings in a more specific way, so as to tell SharePoint which resource file holds a particular string.

<?xml version="1.0" encoding="utf-8" ?>
<Feature xmlns="http://schemas.microsoft.com/sharepoint/"
         Id="5DFD12AF-D0AA-4c63-8FB8-C49DB1191083"
         Title="$Resources:MyResources,MyFeatureName"
         Scope="Site"
         Version="1.0.0.0">
  <ElementManifests>
    <ElementManifest Location="SiteColumns.xml"/>
  </ElementManifests>
</Feature>

As you can see in the above sample, instead of $Resources:MyFeatureName I’m referencing the localized string with $Resources:MyResources,MyFeatureName. This tells SharePoint to look for:

• A localized string whose key is MyFeatureName
• In a resource file named MyResources.[Culture].resx
• In the C:\…\12\Resources folder

If all your strings are in the same shared resource file, then you can specify that file as the default resource file and reference the strings as shown in the first sample.

<?xml version="1.0" encoding="utf-8" ?>
<Feature xmlns="http://schemas.microsoft.com/sharepoint/"
         Id="5DFD12AF-D0AA-4c63-8FB8-C49DB1191083"
         Title="$Resources:MyFeatureName"
         DefaultResourceFile="MyResources"
         Scope="Site"
         Version="1.0.0.0">
  <ElementManifests>
    <ElementManifest Location="SiteColumns.xml"/>
  </ElementManifests>
</Feature>

The sample above does exactly the same as the previous one. The only difference is that it specifies MyResources.[culture].resx as the default resource file and, because of that, you don’t need to specify it on all references to the localized strings.

Additional Notes

So, how does SharePoint choose which resource file to use when loading the localized strings?

1. Looks for the resource file of the exact culture of the website. If you have a website based on a portuguese (pt-PT) site template, it will look for the Resources.pt-PT.resx. If it’s there, it loads all the strings from it.
2. If it’s not there, it looks for any resource files of the same culture, even if it has a different location. So if your website’s language is pt-PT (Portuguese – Portugal) and you don’t have a Resources.pt-PT.resx file, but you have a Resources.pt-BR.resx (Portuguese – Brazil) file, then SharePoint will use it.
3. If there is no resource file for the website’s culture, SharePoint will use the fall back resource file (Resources.resx).

Samples

You can download samples for:

• Feature with Local Resources
• Feature with Shared Resources

These samples include:

• The solution manifest file (manifest.xml).
• The solution cab structure file (solution.ddf).
• The feature manifest file (Feature.xml).
• The element manifest file (SiteColumns.xml).
• The resource files (.resx)
• A batch file (build.bat) that creates the solution package

Warning: Do not install both solutions in the same farm, since some of the IDs are the same and the feature has the same name on both solutions. If you want to use it as is, test each solution separately, removing one before installing the other.

Notice: These samples are given for illustrative purposes only. Feel free to modify and use them as templates for your solutions and features.

The post SharePoint 2007 Deployment: Using Resources in Features appeared first on Blog IT.

This post is about developing features that associate other features to an existing site definition. This is one of the most powerful types of feature since it allows you to add new functionality to existing site definitions. Check the first post SharePoint 2007 Deployment: Overview for an introduction and the series index.

Package Structure

As I mentioned previously in the post SharePoint 2007 Deployment: Creating and Using Features, to build a feature you need to create the following files:

• The feature manifest file (which must be named feature.xml)
• One or more element manifest files

The feature manifest file contains the generic information about the feature package, and the element manifest files contain the information about each specific type of element that makes up the feature. Since I already explained all the possible contents of the feature manifest file in the above mentioned post, I will focus this one the element manifest that allows the feature stapling (feature site template association).

You can then place these two files inside a Solution following the instructions in the post SharePoint 2007 Deployment: Creating Solutions, to provide an easy way to deploy the feature (or upgrade it).

Feature Stapling

Feature stapling is the process of associating features to existing site definitions so that, when a new site is provisioned from that definition the associated features are automatically activated.

This means that you need, at least, two features to do this:

• The feature (or features) you wish to associate (that is, to staple) to a site definition;
• The feature that performs the association (the stapler).

The first one can be any feature with scope Site or Web. The second is the one I’m presenting in this post.

Allowed Scopes

The scopes to which a feature can be deployed, are dictated by the types of elements included in it. A feature with feature site template association elements can be deployed to Site Collection, Web Application or Farm scopes.

Feature Manifest

I will only present a simple feature manifest, since the additional options were presented in the above mentioned post.

<?xml version="1.0" encoding="utf-8" ?>
<Feature
    xmlns="http://schemas.microsoft.com/sharepoint/"
    Id="{8213A053-46B0-43f9-B00C-B2A8CF7A3355}"
    Title="My Feature Stapling"
    Description="This feature staples other features to a
    site definition."
    Scope="Farm"
    Creator="Create IT"
    Version="1.0.0.0">
  <ElementManifests>
    <ElementManifest Location="Elements.xml"/>
  </ElementManifests>
</Feature>

Notes about this feature manifest:

• The title of the feature is My Feature Stapling.
• It will be deployed as a Farm feature, since it’s Scope value is Farm. By default, SharePoint automatically activates features of this scope. You can, however, override this behaviour by setting the ActivateOnDefault attribute to false on the Feature element.
• It references a single element manifest file: Elements.xml.

Element Manifest

The element manifest file can have any name you wish (in this example it’s called Elements.xml), but it’s root element must be <Elements>. Inside this root element, you can place any number of feature element descriptions. In this example I will present the use of the <FeatureSiteTemplateAssociation> element which is used to associate features to existing site definitions.

<?xml version="1.0" encoding="utf-8" ?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <!-- My Site Columns -->
  <FeatureSiteTemplateAssociation 
    Id="{CC3144A5-E055-4474-928E-5D21CDE53D38}" 
    TemplateName="STS#0" />
  <!-- My Content Types -->
  <FeatureSiteTemplateAssociation 
    Id="{E739683D-ACB8-4187-A764-1323BE76D12D}" 
    TemplateName="STS#0" />
</Elements>

This example associates two features with the Team Site definition. The <FeatureSiteTemplateAssociation> element has no child elements and only two attributes:

• Id – (required) The GUID of the feature that is to be stapled to the site definition.
• TemplateName – (required) The name of the site definition including the configuration Id. For SharePoint’s out-of-the-box site definitions, this attribute can be:
  • STS#0 (Team Site)
  • STS#1 (Blank Site)
  • STS#2 (Document Workspace)
  • MPS#0 (Basic Meeting Workspace)
  • MPS#1 (Blank Meeting Workspace)
  • MPS#2 (Decision Meeting Workspace)
  • MPS#3 (Social Meeting Workspace)
  • MPS#4 (Multipage Meeting Workspace)
  • CENTRALADMIN#0 (Central Administration)
  • WIKI#0 (Wiki Site)
  • BLOG#0 (Blog Site)
  • BDR#0 (Document Center)
  • OFFILE#1 (Records Center)
  • SPSMSITEHOST#0 (My Site Host)
  • SPSMSITE#0 (Personalization Site)
  • CMSPUBLISHING#0 (Publishing Site)
  • BLANKINTERNET#2 (Publishing Site with Workflow)
  • BLANKINTERNETCONTAINER#0 (Publishing Portal)
  • SPSPORTAL#0 (Collaboration Portal)
  • SPSNHOME#0 (News Site)
  • SPSITES#0 (Site Directory)
  • SPSREPORTCENTER#0 (Report Center)
  • SRCHCEN#0 (Search Center with Tabs)
  • SRCHCENTERLITE#0 (Search Center)

There can be other site definitions depending on which site templates you have installed on your farm. The best way to check which ones you can use is to go to the folder C:\…\12\TEMPLATE\[LCID]\XML where [LCID] represent the Language ID of the site definition you are looking for, and open each webtemp*.xml file.

Each of these files will have the following structure:

<?xml version="1.0" encoding="utf-8"?>
<!-- _lcid="1033" _version="12.0.4518" _dal="1" -->
<!-- _LocalBinding -->
<Templates xmlns:ows="Microsoft SharePoint">
  <Template Name="STS" ID="1">
    <Configuration ID="0" 
                   Title="Team Site" 
                   Hidden="FALSE" 
                   ImageUrl="/_layouts/images/stsprev.png" 
                   Description="A site for teams to quickly […] 
                   DisplayCategory="Collaboration" >          
    </Configuration>
    <Configuration ID="1" 
                   Title="Blank Site" 
                   Hidden="FALSE" 
                   ImageUrl="/_layouts/images/blankprev.png" 
                   Description="A blank site for you to […] 
                   DisplayCategory="Collaboration" 
                   AllowGlobalFeatureAssociations="False" >      
    </Configuration>
    <Configuration ID="2" 
                   Title="Document Workspace" 
                   Hidden="FALSE" 
                   ImageUrl="/_layouts/images/dwsprev.png" 
                   Description="A site for colleagues to work[…] 
                   DisplayCategory="Collaboration" >          
    </Configuration>
    [...]
  </Template>
</Templates>

In the sample above you can see the description of the three configurations of the STS site template. To build the TemplateName required for the feature site template association, you get the template name (attribute Name of the element Template), which in this case is STS, add a hash sign (#) and complete the string with the configuration ID (attribute ID of the element Configuration), which in this case can be 0, 1 or 2, depending on the chosen configuration.

Sample

You can download this sample here. This sample includes:

• The solution manifest file (manifest.xml).
• The solution cab structure file (solution.ddf).
• The feature manifest file (Feature.xml).
• The feature site template association element manifest file (Elements.xml).
• A batch file (build.bat) that creates the solution package (MyFeatureStapling.wsp).

Notice: This sample is given for illustrative purposes only. Feel free to modify and use it as a template for your solutions and features.

The post SharePoint 2007 Deployment: Feature Stapling appeared first on Blog IT.

This post is about developing features to bind content types to lists that were provisioned through an onet.xml file. Check the first post SharePoint 2007 Deployment: Overview for an introduction and the series index.

Package Structure

As I mentioned previously in the post SharePoint 2007 Deployment: Creating and Using Features, to build a feature you need to create the following files:

• The feature manifest file (which must be named feature.xml)
• One or more element manifest files

The feature manifest file contains the generic information about the feature package, and the element manifest files contain the information about each specific type of element that makes up the feature. Since I already explained all the possible contents of the feature manifest file in the above mentioned post, I will focus this one the element manifest that allows the binding of content types.

You can then place these two files inside a Solution following the instructions in the post SharePoint 2007 Deployment: Creating Solutions, to provide an easy way to deploy the feature (or upgrade it).

Content Type Binding

When you provision a list through a feature, using the method explained in the post SharePoint 2007 Deployment: List Template Features, you can bind a content type to it only by editing the schema.xml file of the list template.

However, if the list was provisioned through a site definition (onet.xml file) and you had no access to its definition, you must do it using a Content Type Binding feature element. This can only be used to attach a content type to a list provisioned through a site definition.

Allowed Scopes

The scopes to which a feature can be deployed, are dictated by the types of elements included in it. A feature with content type binding elements can only be deployed to a Site Collection scope.

Attempting to install a feature of another scope that contains content type binding elements, will result in an error.

Feature Manifest

I will only present a simple feature manifest, since the additional options were presented in the above mentioned post.

<?xml version="1.0" encoding="utf-8" ?>
<Feature
    xmlns="http://schemas.microsoft.com/sharepoint/"
    Id="{ACDECF85-BDF7-446c-AC1B-C5F133C83F15}"
    Title="Content Type Binding"
    Description="This feature binds a content type to the masterpage
gallery."
    Scope="Site"
    Creator="Create IT"
    Version="1.0.0.0">
  
  <ElementManifests>
    <ElementManifest Location="ContentTypeBinding.xml"/>
  </ElementManifests>
</Feature>

Notes about this feature manifest:

• The title of the feature is Content Type Binding.
• It will be deployed as a Site Collection feature, since it's Scope value is Site.
• It references a single element manifest file: ContentTypeBinding.xml.

Element Manifest

The element manifest file can have any name you wish (in this example it's called ContentTypeBinding.xml), but it's root element must be <Elements>. Inside this root element, you can place any number of feature element descriptions. In this example I will present the use of the <ContentTypeBinding> element which is used to binding Content Types to list definitions.

<?xml version="1.0" encoding="utf-8" ?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <ContentTypeBinding
    ContentTypeId="0x010101"
    ListUrl="_catalogs/masterpage" />
</Elements>

This example binds the content type Form (which has the ID 0x010101) to the master page gallery (which has the URL _catalogs/masterpage). There <ContentTypeBinding> element has no child elements, and only two attributes:

• ContentTypeId – (required) The ID of the content type that will be bound to a list.
• ListUrl – (required) The URL of the list to which the content type will be bound.

Additional Notes

The content type will be bound to the list definition as soon as the administrator activates this feature on a site collection. However, deactivating the feature will not remove the connection between the content type and the list. If you wish to remove it, you must do so manually.

Sample

You can download this sample here. This sample includes:

• The solution manifest file (manifest.xml).
• The solution cab structure file (solution.ddf).
• The feature manifest file (feature.xml).
• The content type binding element manifest file (ContentTypeBinding.xml).
• A batch file (build.bat) that creates the solution package (ContentTypeBinding.wsp).

Notice: This sample is given for illustrative purposes only. Feel free to modify and use it as a template for your solutions and features.

The post SharePoint 2007 Deployment: Content Type Binding Features appeared first on Blog IT.