Recently I had the need to index documents stored in Azure Storage Blobs. Additionally I wanted to use the blob storage metadata also to add some information to those documents. Because I needed rich text information on the metadata I could use blob storage metadata directly. Se here why.
So I had to use 2 different data sources. One for the documents and another to the metadata. So I chose Azure Blob Storage and Azure Table Storage. This is the full diagram of the final solution:
The indexers are responsible for updating the index with the contents of the 2 different data sources. There is a very important field that in my case it’s called the UniqueIdentifier field because this field is marked with the key property. This is the field that uniquely identifies each document on the Azure Search Index.
And it’s this field that is responsible for correlating the items that come from one data source (documents from blob storage) and items that come from the other data source (records from table storage).
Every document inserted in blob storage has a custom metadata property named also UniqueIdentifier that will have a table storage record associated with the corresponding metadata.
I was getting a 400 Bad Request when inserting blobs in Azure Blob Storage because I was setting metadata with non-ASCII characters.
// Retrieve storage account from connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
// Create the blob client.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve a reference to a container.
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
// Create the container if it doesn't already exist.
// Add some metadata to the container.
According to Microsfot documentation:
“You will receive a 400 Bad Request if any name/value pairs contain non-ASCII characters. Metadata name/value pairs are valid HTTP headers, and so must adhere to all restrictions governing HTTP headers. It is therefore recommended that you use URL encoding or Base64 encoding for names and values containing non-ASCII characters.”
Registration for Cloud Pro PT next meeting is open.
Check out the awesome agenda:
18:30 – Welcome – Snacks
19:00 – Service Fabric – The Magic Happens Here – Luís Rato
20:00 – Break
20:15 – Networking & Virtual Machines – Vitor Tomaz
21:15 – Acknowledgment
You are all invited to join us after the meeting and have dinner.
I’m proud to announce that I was selected to present a session at Collab 365 Global Conference 2016. This is a global online conference delivering sessions about SharePoint, Office 365 and Azure on the 19th-20th October. And it’s free. Register here.
In my session I will talk about event driven applications with Azure Functions, it will cover triggers and webhooks that Azure Functions provide. It will also have a look at various integrations and at last, it will check the scaling and costs.
See you there
Logging in with Google, Microsoft and Facebook SDKs to Azure Mobile Services
“One of the values that Azure Mobile Services provides is an easy way to implement authentication for mobile applications, via a very simple API – call a login function (or equivalent) on the client object in any of the supported platforms, and your user gets presented with a simple web-based interface that allows them to log in to your mobile service. This is what we call a server-side authentication flow, where the service guides the client to the provider (via redirections in the web page) and then back to itself.”
Azure Service Bus – ‘NamespaceType’ default value change
“Recently we announced that we are splitting the user experience between Service Bus and Notification Hubs to enable a better experience for Notification Hubs.”
Rich Push with Notification Hubs in iOS 8
“In order to engage users with instant rich content, applications often push notifications beyond just plain text. These notifications can contain urls, images, sounds, and more. “
Better support for paging with Table Storage in Azure Mobile Services .NET backend
“When we released the .NET backend for Azure Mobile Services, we provided support to store data in Azure Table Storage and MongoDB databases (in addition to the SQL Azure storage which has been used since the first version of the service).”