Add scheduled adaptive live tiles to your UWP app

Today I’ll help you add some bling to your UWP apps (and maybe something¬†useful) with scheduled live tiles using the built-in features on the W10 SDK and some Azure help – although any cloud storage provider should do fine. I’ll divide this into the “backend stuff”, where we’ll generate our tile templates, and the “client stuff”, where your app will consume the tile notifications.

It goes without saying that you need Visual Studio (VS) and the W10 SDK, which I assume you have installed.

I’ll provide some examples from my pet project Pocketnow WP, so don’t worry if you see some references to it ūüôā

1. The backend stuff

Here I’ll make use of some Azure features, so it’s best that you have a working subscription. What we want to achieve is a set of XML¬†files on blob storage with the tile templates. You can use any other means to achieve this, as long as the end result remains the same.

This particular setup is free in Azure given that the tile generation is simple and doesn’t usually involve much data IO or CPU.

First off, please a¬†create a mobile service on the management portal, or use an existing one if your app already makes use of it. I won’t go over the¬†details, so I assume you already have some experience on the subject. In the end, I want you to be on the “Get started” page, ready to download your starter project (in C#). If you already have a mobile services project in your solution, use that one and skip this step.

dashboard

After opening the project in VS, you’ll have the default mobile service project structure with lots of unnecessary things (for us). We just want to focus on the ScheduledJobs folder where we will put our brilliant new job to generate the templates:

mobile5

You could use the default “SampleJob.cs” as the starting point, but let’s add a new one to keep it clean:

mobile6

Give it a fancy name ending with “Job” (this is important!)

mobile7

You’ll end up with an empty class like this:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;

namespace pocketnoww10Service.ScheduledJobs
{
    public class LovelyLiveTileGeneratorJob
    {
    
    }
}

Next, we need to make it runnable by Azure, so make it inherit from ScheduledJob and add the ExecuteAsync method:

public class LovelyLiveTileGeneratorJob : ScheduledJob
{
    public async override Task ExecuteAsync()
    {
        Services.Log.Info("Hello from scheduled job! The time is precisely " + DateTime.Now.ToUniversalTime());
    }
}

You should now be able to publish the mobile service to Azure and test it, but let’s add the rest of the code before we try it.

The next step is to generate the templates as described in the¬†SDK reference. I’ll produce a string version of the XML step-by-step, but you can use the code provided in the MSDN reference. I’m adding a new UpdateLiveTileFile method which will generate the tile template code and write it to a container in Azure blob storage. In this next bit of code I¬†assume that I have a model for my stories which I want to appear in my users’ live tiles, entering the method as a list. Inside I’ll iterate them and build template strings:

 

private async Task UpdateLiveTileFile(IEnumerable<StoryModel> stories)
{
    foreach (StoryModel story in stories)
    {
		string xmlContent = 
			"<tile>" +
			"  <visual branding=\"nameAndLogo\">" +
			"    <binding template=\"TileMedium\" branding=\"name\">" +
			"      <text hint-style=\"body\" hint-wrap=\"true\">" + System.Security.SecurityElement.Escape(story.Title) + "</text>" +
			"     <image placement=\"background\" src=\"" + System.Security.SecurityElement.Escape(story.PictureSource) + "\" hint-overlay=\"40\" />" +
			"    </binding>" +
			"    <binding template=\"TileWide\">" +
			"      <group>" +
			"        <subgroup hint-weight=\"33\">" +
			"          <image placement=\"inline\" src=\"" + System.Security.SecurityElement.Escape(story.AuthorPictureSource) + "\" hint-crop=\"circle\" />" +
			"        </subgroup>" +
			"        <subgroup>" +
			"      <text hint-style=\"body\" hint-wrap=\"true\">" + System.Security.SecurityElement.Escape(story.Title) + "</text>    " +
			"      </subgroup>" +
			"      </group>" +
			"     <image placement=\"background\" src=\"" + System.Security.SecurityElement.Escape(story.PictureSource) + "\" hint-overlay=\"40\" />" +
			"    </binding>" +
			"    <binding template=\"TileLarge\">" +
			"      <group>" +
			"        <subgroup hint-weight=\"33\">" +
			"          <image placement=\"inline\" src=\"" + System.Security.SecurityElement.Escape(story.AuthorPictureSource) + "\" hint-crop=\"circle\" />" +
			"        </subgroup>" +
			"        <subgroup>" +
			"        <text hint-style=\"caption\" hint-wrap=\"true\">" + System.Security.SecurityElement.Escape(story.Title) + "</text>    " +
			"        </subgroup>" +
			"      </group>" +
			"      <image placement=\"inline\" src=\"" + System.Security.SecurityElement.Escape(story.PictureSource) + "\" />      " +
			"    </binding>" +
			"  </visual>" +
			"</tile>";
    }
}

To protect your eyes, here’s a generated template using this code. You can see 3 binding sections, each for a different tile size (in this case Medium, Wide and Large). Again, check the reference for a description of the options:

<tile>
  <visual branding="nameAndLogo">
    <binding template="TileMedium" branding="name">
      <text hint-style="body" hint-wrap="true">Samsung Galaxy Tab S2 9.7 treated to Marshmallow makeover in Europe</text>
     <image placement="background" src="http://pocketnow.com/images/300/190/1454330272/2016/04/Galaxy-Tab-S2-9.7-1.jpg" hint-overlay="40" />
    </binding>
    <binding template="TileWide">
      <group>
        <subgroup hint-weight="33">
          <image placement="inline" src="http://pocketnow.com/images/50/50/1454330272/userphoto/72.jpg" hint-crop="circle" />
        </subgroup>
        <subgroup>
      <text hint-style="body" hint-wrap="true">Samsung Galaxy Tab S2 9.7 treated to Marshmallow makeover in Europe</text>    
      </subgroup>
      </group>
     <image placement="background" src="http://pocketnow.com/images/300/190/1454330272/2016/04/Galaxy-Tab-S2-9.7-1.jpg" hint-overlay="40" />
    </binding>
    <binding template="TileLarge">
      <group>
        <subgroup hint-weight="33">
          <image placement="inline" src="http://pocketnow.com/images/50/50/1454330272/userphoto/72.jpg" hint-crop="circle" />
        </subgroup>
        <subgroup>
        <text hint-style="caption" hint-wrap="true">Samsung Galaxy Tab S2 9.7 treated to Marshmallow makeover in Europe</text>    
        </subgroup>
      </group>
      <image placement="inline" src="http://pocketnow.com/images/300/190/1454330272/2016/04/Galaxy-Tab-S2-9.7-1.jpg" />      
    </binding>
  </visual>
</tile>

Now that we have our generated templates, we need to store them on Azure (or your equivalent). If you already have a storage account you can use it and skip the first step, but please create a new container:

First, create a storage account:
storage1

Then create a container:

storage2

Now we need to get a connection string to connect the mobile service to this storage account. To do this go to the dashboard and check the access keys in the bottom app bar, then take note of your storage account name (if you already forgot it) and copy the primary access key:

storage3

storage4

We can now add a new connection string to¬†our web.config to connect to this storage account, so open it (should be on the project root) and add a new connection called “StorageConnectionString” inside the connectionStrings tag using the access key you copied. It should look like this:

<connectionStrings>
    <add name="MS_TableConnectionString" connectionString="XXXX" providerName="System.Data.SqlClient" />
    <add name="MS_NotificationHubconnectionString" connectionString="XXXX" />
    <add name="StorageConnectionString" connectionString="DefaultEndpointsProtocol=https;AccountName=<YOUR_STORAGE_ACCOUNT>;AccountKey=<YOUR_PRIMARY_ACCESS_KEY>" />
</connectionStrings>

Now we need to connect to the container using this connection string. In the beginning of UpdateLiveTileFile method add some initialization code (check the inline comments):

// Get the connection string
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(ConfigurationManager.ConnectionStrings["StorageConnectionString"].ConnectionString);

// Get a blob client
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();

// Create or get an existing container (use a nice name for this, AND NOT HARDCODED)
CloudBlobContainer container = blobClient.GetContainerReference("LovelyLiveTileGeneratorContainerName");
bool result = await container.CreateIfNotExistsAsync();

// Set the permissions on the container to public so that client apps can access it
// This means you shouldn't use this container for anything else
await container.SetPermissionsAsync(new BlobContainerPermissions
{
    PublicAccess = BlobContainerPublicAccessType.Blob
});

Finally, prepare our generation loop to write to the blob for each tile with a different number each time. In the end the loop should look like this.

// Start an iterator to be appended to the file name
int i = stories.Count() - 1;
foreach (StoryModel story in stories)
{
    #region Template generation code
    ...
    #endregion

    // Get a file reference -> CHANGE THE NAME OF THE FILE!
    CloudBlockBlob fileBlob = container.GetBlockBlobReference(string.Format("LovelyLiveTileFile-{0}.xml", i));

    // And write to the file
    await fileBlob.UploadTextAsync(xmlContent);
    i--;
}

Our next step is to publish the mobile service to Azure. To ease the process you can download a publish profile from your mobile service dashboard:

mobile8

Then go to your project and hit Publish from the context menu:

mobile9

Now browse to the publish profile you just downloaded:

mobile10

mobile11

Then press OK till you reach the last page, where you press Publish:

mobile12

You should now run the job once to ensure the files are being generated. To achieve this, go to the mobile service page on the azure management portal and access the “Scheduler” tab. Here you should open the job you just published:

schedulerThen click “Run once”

run_once

If all went well (no reason it shouldn’t have), if you access your blob container URL directly from a browser you should be presented with the generated tile templates:

blob

That’s it for our first part! Now comes the easier bit – consume the files we’ve just generated.


2. The client stuff

It’s considerably easy to consume the generated tile template files from a UWP app. I usually put my code on the App.xaml.cs, but it can go anywhere. Here’s a summary of the steps you need to do:

  1. Instantiate a TileUpdater
  2. Set a list of URIs to be polled
  3. Enable queued notifications
  4. Set the polling frequency
  5. Start polling
  6. Profit

This translates to very simple code that can run at startup or on demand (if you tie it to your settings page).

First off, add a new method on your App.xaml.cs (or wherever you want a method to subscribe your app to tile updates):

public static void SubscribeToTileNotifications()
{
	TileUpdater tileUpdater = TileUpdateManager.CreateTileUpdaterForApplication();
}

Now then, we need a way to turn tile updates on and off. I usually do this by writing a bool on my roaming settings (usually on a settings page which I won’t include), but you can use whichever method you prefer. So, let’s assume that somewhere in your app there’s a process that writes to the tag “LIVE_TILE_ONOFF” on the roaming settings and add some logic to the subscribe method you just created:

ApplicationDataContainer appSettings = ApplicationData.Current.RoamingSettings;
if (appSettings.Values.Keys.Contains("LIVE_TILE_ONOFF")
    && (bool)appSettings.Values["LIVE_TILE_ONOFF"])
{
   // subscription code
}
else
{
   // unsubscription code
}

Next, we need to construct a list with URLs pointing to the templates we are creating on the blob container. A simple loop should do (don’t forget to set your blob container URL on TILE_UPDATE_URL), so add it under the subscription code comment:

List<Uri> tileUpdateUris = new List<Uri>();
string TILE_UPDATE_URL = "https://<YOUR_STORAGE_ACCOUNT>.blob.core.windows.net/LovelyLiveTileGeneratorContainerName/LovelyLiveTileFile-{0}.xml";
int TILE_UPDATE_SIZE = 5;
for (int i = 0; i < TILE_UPDATE_SIZE; i++)
{
	tileUpdateUris.Add(new Uri(string.Format(TILE_UPDATE_URL, i)));
}

We can now ask the tile updater to subscribe to notifications. There are several options for the update recurrence, but I’ll stick with an hardcoded hourly update.

tileUpdater.EnableNotificationQueue(true);
tileUpdater.StartPeriodicUpdateBatch(tileUpdateUris, PeriodicUpdateRecurrence.Hour);

Finally, you should also add some code to unsubscribe from notifications (by user command), so add this under the unsubscription code comment on the else block:

tileUpdater.Clear();
tileUpdater.EnableNotificationQueue(false);
tileUpdater.StopPeriodicUpdate();

In the end you should end up with something like this:

public static void SubscribeToTileNotifications()
{
	TileUpdater tileUpdater = TileUpdateManager.CreateTileUpdaterForApplication();

	ApplicationDataContainer appSettings = ApplicationData.Current.RoamingSettings;
	if (appSettings.Values.Keys.Contains("LIVE_TILE_ONOFF")
        && (bool)appSettings.Values["LIVE_TILE_ONOFF"])
	{
		List<Uri> tileUpdateUris = new List<Uri>();
		string TILE_UPDATE_URL = "https://<YOUR_STORAGE_ACCOUNT>.blob.core.windows.net/LovelyLiveTileGeneratorContainerName/LovelyLiveTileFile-{0}.xml";
		int TILE_UPDATE_SIZE = 5;
		for (int i = 0; i < TILE_UPDATE_SIZE; i++)
		{
			tileUpdateUris.Add(new Uri(string.Format(TILE_UPDATE_URL, i)));
		}

		tileUpdater.EnableNotificationQueue(true);
		tileUpdater.StartPeriodicUpdateBatch(tileUpdateUris, PeriodicUpdateRecurrence.Hour);
	}
	else
	{
		tileUpdater.Clear();
		tileUpdater.EnableNotificationQueue(false);
		tileUpdater.StopPeriodicUpdate();
	}
}

All that’s left to do is call your new method SubscribeToTileNotifications from somewhere in your app and you’re done, your tiles will come to life!

Quartz.NET: A simple single producer/multi consumer backoff strategy

When handling a producer/consumer problem where there’s a single source of data to be consumed by multiple entities, you might arrive at a state where your consumers will start kicking and punching each other just to get a seat at the table – and that’s clearly not what you want. A black-eyed consumer will obviously lack the will to carry on much longer working for you.
So, how to add some sense to the impending blood-bath the moment our producer starts pouring out its golden nuggets? Academics will of course advise you a myriad of ultra-valid approaches that will evidently solve all your problems. Yet, at the expense of an extra week on your sprint and maybe a few grey-hairs (which you should already have if you’re working on multi-threaded systems).
Ranging from locks to lock-free, non-blocking, synchronized and all that jargon, it sure looks cool until you come up with crazy solutions like “Maybe I’ll use exceptions to communicate”. But do you really need the cannon to kill the fly? You should always measure your cost vs usefulness level when making these type of decisions. If your system is simple enough and your problem is making sure you don’t have consumers starving out and craving for some work, maybe one of your best options is to use some flavor of exponential backoff. Here’s some semi-credible Wikipedia page, give it a read and think about if your system would benefit from this. In that case, continue reading for some spectacular Quart.NET simple implementation!


Now that you’ve decided to use exponential backoff on your system, make sure you have a reference for Quartz.NET on your Visual Studio solution (you can grab it here).

1. Coding the jobs

The first step on this endeavor is to add your Producer/Consumer entry-points, meaning, two classes with an Execute method to be called by Quartz that will in turn execute your existing producer/consumer code. Go ahead and create two new classes (I called them Producer and Consumer, but let the creativity go wild here). Remember, Ctrl+. is your friend:

namespace CreateIt.Examples
{
   public class Producer
   {
      // The class that knows how to produce stuff
      ProducerWorker _producerWorker;
	  
      public Producer() 
	  {
         // initialize your worker -> dependency injection highly recommended, but I will keep it simple
         _producerWorker = new ProducerWorker();
      }
	  
      public void Execute(Quartz.IJobExecutionContext context)
	  {
         // start the worker
         _producerWorker.Produce();
      }
   }
   
   public class Consumer 
   {
      // The class that knows how to consumer stuff
      ConsumerWorker _consumer;
	  
      public Consumer() 
	  {
         // initialize your worker (note: use dependency injection)
         _consumerWorker = new ConsumerWorker();
      }
	  
      public void Execute(Quartz.IJobExecutionContext context) 
	  {
         // start the worker
         bool hasConsumed = _consumerWorker.Consume();
      }
   }
}

In the example above, ProducerWorker and ConsumerWorker should be your own classes, make sure you replace them with the correct name as this is just a guideline.

2. Scheduling

The next step is to configure the jobs to run in a given schedule. I’ll use the configuration file approach for this, but it should be doable using the code API as well (see the documentation for more on this topic).
Scheduling by configuration in Quartz is done with an XML file describing the jobs and triggers. It comprises a list of <schedule> tags containing a <job> and a list of <trigger>. You can add as many <schedule> tags as you want, even of the same job type, but you do have to give different names to the jobs. To start with, add three new schedules, one Producer and two Consumers:

<schedule>	 	 
  <job>	 	 
    <name>Producer</name>	 	 
    <job-type>CreateIt.Examples.Producer, CreateIt.Examples</job-type>	 	 
  </job>	 	 
  <trigger>	 	 
    <simple>	 	 
      <name>ProducerTrigger</name>	 	 
      <job-name>Producer</job-name>	 	 
      <repeat-count>0</repeat-count>
      <repeat-interval>10000</repeat-interval>
    </simple>	 	 
  </trigger>    	 	 
</schedule>	 	 
<schedule>	 	
  <job>	 	 
    <name>Consumer_1</name>	 	 
    <job-type>CreateIt.Examples.Consumer, CreateIt.Examples</job-type>	 	 
  </job>	 	 
  <trigger>	 	 
    <simple>	 	 
      <name>Consumer_1_Trigger</name>	 	 
      <job-name>Consumer_1</job-name>	 	 
      <repeat-count>0</repeat-count>
      <repeat-interval>1000</repeat-interval> 
    </simple>	 	 
  </trigger>    	 	 
</schedule>	 	 
<schedule>	 	 
  <job>	 	 
    <name>Consumer_2</name>	 	 
    <job-type>CreateIt.Examples.Consumer, CreateIt.Examples</job-type>	 	 
  </job>	 	 
  <trigger>	 	 
    <simple>	 	 
      <name>Consumer_2_Trigger</name>	 	 
      <job-name>Consumer_2</job-name>	 	 
      <repeat-count>0</repeat-count>
      <repeat-interval>1000</repeat-interval>
    </simple>	 	 
  </trigger>    	 	 
</schedule>

Notice the connection between the trigger and the job through the job name. You could in theory have several different jobs and triggers inside the same schedule if they are part of the same workflow, but to show the separation of responsibilities the example clearly divides the Producer and Consumer schedules. You can set triggers to fire at any given time with either unlimited or a limited set of repetitions (check the documentation for more info). I’ve also set the repeat-count parameter to 0 so that the jobs repeat forever, and the repeat-interval to 10s for the producer and 1s for the consumers.

You should now add the following to your App.config file to configure the schedule properties. Make sure your job XML file is in an accessible location, mine is named “quartz_jobs.xml” and is in the same folder as the App.config:

<configuration>
  <configSections>
    <section name="quartz" type="System.Configuration.NameValueSectionHandler, System, Version=1.0.5000.0,Culture=neutral, PublicKeyToken=b77a5c561934e089"/>
  </configSections>
  <quartz>
    <add key="quartz.scheduler.instanceName" value="ProducerConsumerScheduler"/>
    <add key="quartz.threadPool.type" value="Quartz.Simpl.SimpleThreadPool, Quartz"/>
    <add key="quartz.threadPool.threadCount" value="3"/>
    <add key="quartz.plugin.xml.type" value="Quartz.Plugin.Xml.XMLSchedulingDataProcessorPlugin, Quartz"/>
    <add key="quartz.plugin.xml.fileNames" value="~/quartz_jobs.xml"/>
  </quartz>
</configuration>

Looking inside the quartz section, only a handful of properties are actually configured. It should be pretty straightforward to grasp the meaning of each one by the key, so I won’t enter into details. You can meddle with a lot of options to make sure your scheduler is fine tuned for your needs, again, check the documentation.

For this example I’m asking for a simple thread pool with 3 threads (one producer, two consumers). I also set the XML plugin to XMLSchedulingDataProcessorPlugin so that my job definitions are understandable by quartz. You can add your own plugins, but the provided ones are usually more than enough.

3. Booting the scheduler – unleashing the beast

We’ve reached the fun part after all this configuration stuff, we will launch our scheduler!

With that in mind, I’m using an ordinary console application with no other logic – you can add this to your initialization workflow. Just make sure that the scheduler’s threads are kept alive in your application regardless of where and when you launch it:

namespace CreateIt.Examples
{
	/// <summary>
	/// Console application class for our Quartz scheduler
	/// </summary>
	public class QuartzSchedulerSample
	{
		/// <summary>
		/// The Quartz scheduler
		/// </summary>
		private Quartz.IScheduler _scheduler;
		
		/// <summary>
		/// The Quartz scheduler
		/// </summary>
		protected Quartz.IScheduler Scheduler
		{
			get
			{
				return _scheduler; 
			}
			
			set 
			{ 
				_scheduler = value; 
			}
		}
			
		/// <summary>
		/// The entry point where we will start scheduler
		/// </summary>
		public static void Main()
		{
			QuartzSchedulerSample quartz = new QuartzSchedulerSample();
			quartz.StartScheduler();

			Console.Write("Press any key to continue . . .");
			Console.ReadKey(true);

			quartz.ShutdownScheduler();
		}
		
		/// <summary>
		/// Starts the scheduler.
		/// </summary>
		public void StartScheduler()
		{
			try
			{
				Scheduler = Quartz.Impl.StdSchedulerFactory.GetDefaultScheduler();
				Scheduler.Start();
			}
			catch (Exception ex)
			{
				// Handle exception...
			}
		}

		/// <summary>
		/// Stops the scheduler.
		/// </summary>
		public void ShutdownScheduler()
		{
			Scheduler.Shutdown();
		}
	}
}

Easy, right? If you start your application you should now see your producer and consumer classes working concurrently, with the producer handing off work every 10 seconds and the consumers waking up every second to check on that.

4. Handling starvation with a backoff strategy

We’ve now reached the core of this post, that is, how to setup a simple backoff pattern on top of Quartz. So far we’ve implemented a simple schedule with one producer and two consumers that blindly wake up every 10s/1s to do stuff. You will hardly find a concurrency problem in these conditions, but if you up the consumer count a bit you will start seeing some workers waiting for longer and longer times before they can grab a piece of work. The plan is to make each consumer wait an increasing amount of time each time it fails to acquire or complete a unit of work. To achieve this effect, 3 steps are required by the consumers: update our backoff time, calculate the next execution date based on that time, and ask Quartz to reschedule the consumer (a self-reschedule, actually). An extra step is also needed to guarantee that state is keeped between executions.

NOTE: we will be working on the Consumer class we defined up on 1., so I will probably not mention it in the rest of the section.

4.1 Update the backoff time

First of all, we need some data structures to hold information regarding the current backoff state. There are several ways to achieve this, and it should be defined according to your system’s needs, but for the sake of our simple example I will set this up with hard-coded values. Use this as a guideline, but please use all the configuration utilities you should already have in place (if you don’t have any, make sure you do).

To start of, we need a way to store or produce an increasing list of times to wait. This can either be random, calculated or hard-coded values. I’m going for the last option, so I’ll add a constant array with the default backoff times in seconds:

/// <summary>
/// The default backoff times (in seconds)
/// </summary>
private const int[] _backoffTimes = new int[] { 1, 5, 60, 300, 3600, 86400 };

Next, we need some way of tracking where we stand, that is, which of the backoff times are we currently pointing to if we need to wait. A simple short will do:

/// <summary>
/// Current backoff index
/// </summary>
private short _backoffIndex = 0;

Now for the core logic, we add a method to update the backoff time given the execution result:

/// <summary>
/// Updates the backoff information given the result of an execution
/// </summary>
/// <param name="executionResult">The result of an execution</param>
private void UpdateBackoffIndex(bool executionResult)
{
     // If the execution was successfull, reset the backoff escalation
     if (executionResult)
     {
	     _backoffIndex = 0;
     }
     else
     { 
	     // If not, advance the index if we're not at the last position yet
	     if (_backoffIndex < (_backoffTimes.Length - 1))
	     {
	          _backoffIndex++;
	     }
     }
}

Finally, update the backoff index in the Consumer’s Execute method passing the execution result:

public void Execute(Quartz.IJobExecutionContext context) 
{
     // start the worker
     bool hasConsumed = _consumerWorker.Consume();
		 
     UpdateBackoffIndex(hasConsumed);
}

4.2 Calculate the next execution date

To find the next execution date is simple, just get the backoff seconds from the backoff array using the current index. This simple method does just that:

/// <summary>
/// Calculates the next execution date based on the current backoff index
/// </summary>
/// <returns>The next execution date</returns>
private DateTime GetNextExecutionDate()
{
     int backoffTime = _backoffTimes[_backoffIndex];
     return DateTime.Now.AddSeconds(backoffTime);
}

Do note that we haven’t yet added logic to manage the current index, we are just plumbing around.

4.3 Self-rescheduling

If you recall our Consumer class, the Execution method is given a context parameter to help the job access the scheduling infrastructure:

public void Execute(Quartz.IJobExecutionContext context);

You can access lots of interesting stuff from here (namely all the job details like the name and group for logging), but what we need to focus on is getting the current trigger so that we can “recycle” it with a new execution date. To reschedule the current job add the following method:

/// <summary>
/// Re-schedules the current job
/// </summary>
/// <param name="context">Job execution context</param>
private void Reschedule(IJobExecutionContext context)
{
     // Get the next execution date
     DateTime nextExecutionDate = GetNextExecutionDate();

     // Get the current trigger
     ITrigger currentTrigger = context.Trigger;
	
     // Get a new builder instance from the current trigger
     TriggerBuilder builder = currentTrigger.GetTriggerBuilder();

     // Create a new trigger instance using the builder from the current trigger
     // and set its start time to the next executed date obtained before.
     // This will use the same configuration parameters
     ITrigger newTrigger = builder
          .StartAt(nextExecutionDate)
          .ForJob(context.JobDetail)
          .Build();

     // Re-schedule the job using the current trigger key and the new trigger configuration
     context.Scheduler.RescheduleJob(currentTrigger.Key, newTrigger);
}

The method starts by getting the new execution date and the current trigger from the job context. It then asks the trigger for its own builder so that we can build a new one from that, which will give us the same configuration, so all we need to do is set the start date and the target job and build the new trigger. Finally, we ask the scheduler to re-schedule the current job using the new trigger (but recycling the current trigger key). This is the most confusing bit, but the comments should help you decipher the code.

4.4 Keeping state between executions

Now that we have our re-scheduling mechanism in place, there’s only one thing missing to make this work. One quirk of Quartz (semi-pun intended) is that jobs don’t maintain their states across executions unless you specifically instruct them to do so. This means that if we want to maintain our current backoff index, we need to instruct Quartz to save it in the JobDataMap. Thankfully, this is farily easy to achieve, with only two small adds to the current code.

First off, store the current index in the JobDataMap at the beginning of the Reschedule method:

context.JobDetail.JobDataMap["backoffIndex"] = _backoffIndex;

Then, read it at the beginning of the Execute method:

if (context.JobDetail.JobDataMap.ContainsKey("backoffIndex"))
{ 
	_backoffIndex = (short)context.JobDetail.JobDataMap["backoffIndex"];
}

In the first execution the data map won’t have the index key, so it will keep the default value of 0.

5. Wrapping up

That’s it, your consumers should not longer starve each other out! Also, if you add a bit of log you can follow through their decisions to backoff in case of difficulties and start to fine tune and adapt this solution for your needs.

Bear in mind that this is clearly not full proof and won’t handle your data-center sized farm, but it can help in those simple cases where all you want is something that works and is quick to implement.

Hope it helps!
Ricardo

Adapting .netTiers for Enterprise Library v6.0 and .NET v4.5

Code generation is now more than ever a core activity in software development. With the growing complexity of systems and the need to meet deadlines, getting your database access layer generated automatically is crucial, but one can’t forget to groom your tools¬†and update them¬†whenever possible.

CodeSmith has been providing a code generation tool based on .netTiers for a while now, supporting several versions of the .NET framework up to v4.0, but newer versions are still unnatended and that leaves “legacy” code out of all the new features and improvements. With the release of .NET 4.5 and new library updates based on it, namely Enterprise Library 6.0 (EL6.0) and all its sub-libraries, it is important to keep the code generation templates up to date. This guide will attempt to point out all the specific parts of the templates that must be adapted in order for .netTiers to make use of EL6.0 in a file-by-file basis. To follow it you must get EL6.0 from Microsoft (download site)¬†and have a directory with your existing .netTiers templates to work from.

The directory tree must be similar to the following image:

.netTiers Directory Tree

My target environment:

  • Visual Studio 2012 (VS)
  • .NET 4.5
  • Enterprise Library 6.0 (EL)
  • Unity Application Block 2.0 (you can change this to 3.0 without¬†effort)
  • CodeSmith 7.0.2 (CS)
  • .netTiers 2.3.1 (NT)

A few words of notice:

  1. Migrating between framework or library versions is not easy, and it¬†must be done slowly and with a critical mind, especially on code bound¬†to production systems. If you don’t think that an instruction in this¬†guide applies to your specific case, please do make any changes you deem¬†suitable. As a suggestion, I advise you to read the migration guide to¬†Enterprise Library 6.0 available here.
  2. Microsoft removed some features available in EL v5.0, namely Caching and Security which were mostly added to .NET itself. This implies that to fully migrate .netTiers to work with v6.0, caching and security features must be re-written from scratch. I consider this out of the scope of this post, so I will just point out places where this affects .netTiers, leaving the implementation open.
  3. I try to point out the lines you need to change to ease the finding process, but they might be off by a slight margin due to various reasons, so try to look at what is being changed after searching for the line.

From now on, numbered items will represent folders or files in the directory tree (folders will be bold), while dots represent actions that must be followed. Good luck!

  1. References
    • Add a new folder named ‘EntLibv6_0’ and extract¬†your¬†Enterprise Library 6.0 DLLs into it
  2. VisualStudio
    • Copy file ‘entlib.v5_0.config.cst’ and change its name to¬†‘entlib.v6_0.config.cst’. Edit the renamed file:
      • Line 5 – change it to
          <%@ Property Name="EntLibVersion" Type="MoM.Templates.EntLibVersion" Default="v6_0"
             Category="02. Framework Generation - Optional"
             Description="Indicates the Enterprise Library version to use.
             Options include v3.1, v5.0 and v6.0" %>
      • Comment lines 89 to 99 (related to caching)
          <cachingConfiguration defaultCacheManager="<%=BLLNameSpace %>.EntityCache">
             <cacheManagers>      
                <add expirationPollFrequencyInSeconds="60" maximumElementsInCacheBeforeScavenging="1000"
                   numberToRemoveWhenScavenging="10" backingStoreName="Null Storage"
                   name="<%= BLLNameSpace %>.EntityCache"
                   type="Microsoft.Practices.EnterpriseLibrary.Caching.CacheManager, Microsoft.Practices.EnterpriseLibrary.Caching, Version=5.0.414.0, Culture=neutral,                 PublicKeyToken=31bf3856ad364e35" />
             </cacheManagers>
             <backingStores>
                <add encryptionProviderName=""                     type="Microsoft.Practices.EnterpriseLibrary.Caching.BackingStoreImplementations.NullBackingStore, Microsoft.Practices.EnterpriseLibrary._entLibVersion %>"
                   name="Null Storage" />
             </backingStores>
          </cachingConfiguration>
        
    • Edit file ‘vsnet2005.project.cst’:
      • Line 88 – change a) to b)
        a) bool includeObjectBuilder = EntLibVersion != MoM.Templates.EntLibVersion.v5_0;
        b) bool includeObjectBuilder = EntLibVersion != MoM.Templates.EntLibVersion.v5_0 && EntLibVersion != MoM.Templates.EntLibVersion.v6_0;
      • Line 181 – add another condition to the if statement:
        VisualStudioVersion == MoM.Templates.VSNetVersion.v2012
      • Comment lines 320 to 327:
        <Reference Include="Microsoft.Practices.EnterpriseLibrary.Caching, <%= entlibVersionText %>">
           <SpecificVersion>False</SpecificVersion>
           <HintPath><%= LibraryPath %>\Microsoft.Practices.EnterpriseLibrary.Caching.dll</HintPath>
        </Reference>
        <Reference Include="Microsoft.Practices.EnterpriseLibrary.Caching.Cryptography, <%= entlibVersionText %>">
           <SpecificVersion>False</SpecificVersion>
           <HintPath><%= LibraryPath %>\Microsoft.Practices.EnterpriseLibrary.Caching.Cryptography.dll
        </HintPath>
        </Reference>
      • Comment lines 360 to 371:
        <Reference Include="Microsoft.Practices.EnterpriseLibrary.Security, <%= entlibVersionText %>">
           <SpecificVersion>False</SpecificVersion>
           <HintPath><%= LibraryPath %>\Microsoft.Practices.EnterpriseLibrary.Security.dll</HintPath>
        </Reference>
        <Reference Include="Microsoft.Practices.EnterpriseLibrary.Security.Cache.CachingStore.dll,
        <%= entlibVersionText %>">
           <SpecificVersion>False</SpecificVersion>
           <HintPath><%= LibraryPath %>\Microsoft.Practices.EnterpriseLibrary.Security.Cache.CachingStore.dll</HintPath>
        </Reference>
        <Reference Include="Microsoft.Practices.EnterpriseLibrary.Security.Cryptography, <%= entlibVersionText %>">
           <SpecificVersion>False</SpecificVersion>
           <HintPath><%= LibraryPath %>\Microsoft.Practices.EnterpriseLibrary.Security.Cryptography.dll</HintPath>
        </Reference>
      • Comment lines 883 to 887:
        <Compile Include="<%=BLLSubFolder%>IEntityCacheItem.cs">
           <SubType>Code</SubType>
        </Compile>
        <Compile Include="<%=BLLSubFolder%>EntityCache.cs">
           <SubType>Code</SubType>
        </Compile>
      • Line 1094 – change a) for b):
        a) <% if(EntLibVersion == MoM.Templates.EntLibVersion.v5_0) { %>
        b) <% if(EntLibVersion == MoM.Templates.EntLibVersion.v5_0 || EntLibVersion == MoM.Templates.EntLibVersion.v6_0) { %>
  3. NetTiers.cst
    • Line 30 – change 'Default="v5_0"' to¬†'Default="v6_0"'
    • Line¬†31 – change 'Default="v2010"' to¬†'Default="v2012"'
    • Line¬†32¬†– change¬†'Default="v4"' to¬†'Default="v4_5"'
    • To¬†assure that all EL5.0 features are maintained, it is necessary to change all references of a) with b) (lines 1296, 1372, 1449, 1469, 1992)
      a) 'if (EntLibVersion == MoM.Templates.EntLibVersion.v5_0)'
      b) 'if (EntLibVersion == MoM.Templates.EntLibVersion.v5_0 || EntLibVersion == MoM.Templates.EntLibVersion.v6_0)'
    • At¬†line 1308, change a) to b)
      a)
      SafeCopyFile(this.CodeTemplateInfo.DirectoryName + "\\References\\" + entLibSubDir + "\\Microsoft.Practices.EnterpriseLibrary.Caching.dll", libPath + "\\Microsoft.Practices.EnterpriseLibrary.Caching.dll");
      SafeCopyFile(this.CodeTemplateInfo.DirectoryName + "\\References\\" + entLibSubDir + "\\Microsoft.Practices.EnterpriseLibrary.Caching.Cryptography.dll", libPath + "\\Microsoft.Practices.EnterpriseLibrary.Caching.Cryptography.dll");
      SafeCopyFile(this.CodeTemplateInfo.DirectoryName + "\\References\\" + entLibSubDir + "\\Microsoft.Practices.EnterpriseLibrary.Logging.dll", libPath + "\\Microsoft.Practices.EnterpriseLibrary.Logging.dll");
      SafeCopyFile(this.CodeTemplateInfo.DirectoryName + "\\References\\" + entLibSubDir + "\\Microsoft.Practices.EnterpriseLibrary.Logging.Database.dll", libPath + "\\Microsoft.Practices.EnterpriseLibrary.Logging.Database.dll");
      SafeCopyFile(this.CodeTemplateInfo.DirectoryName + "\\References\\" + entLibSubDir + "\\Microsoft.Practices.EnterpriseLibrary.Security.dll", libPath + "\\Microsoft.Practices.EnterpriseLibrary.Security.dll");
      SafeCopyFile(this.CodeTemplateInfo.DirectoryName + "\\References\\" + entLibSubDir + "\\Microsoft.Practices.EnterpriseLibrary.Security.Cache.CachingStore.dll", libPath +  "\\Microsoft.Practices.EnterpriseLibrary.Security.Cache.CachingStore.dll");
      SafeCopyFile(this.CodeTemplateInfo.DirectoryName + "\\References\\" + entLibSubDir + "\\Microsoft.Practices.EnterpriseLibrary.Security.Cryptography.dll", libPath + "\\Microsoft.Practices.EnterpriseLibrary.Security.Cryptography.dll");
      SafeCopyFile(this.CodeTemplateInfo.DirectoryName + "\\References\\" + entLibSubDir + "\\Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.dll", libPath + "\\Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.dll");
      SafeCopyFile(this.CodeTemplateInfo.DirectoryName + "\\References\\" + entLibSubDir + "\\Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.Logging.dll", libPath + \\Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.Logging.dll);
      
      b) if(EntLibVersion != MoM.Templates.EntLibVersion.v6_0)
      {
         SafeCopyFile(this.CodeTemplateInfo.DirectoryName + "\\References\\" + entLibSubDir + "\\Microsoft.Practices.EnterpriseLibrary.Caching.dll", libPath + "\\Microsoft.Practices.EnterpriseLibrary.Caching.dll");
         SafeCopyFile(this.CodeTemplateInfo.DirectoryName + "\\References\\" + entLibSubDir + "\\Microsoft.Practices.EnterpriseLibrary.Caching.Cryptography.dll", libPath + "\\Microsoft.Practices.EnterpriseLibrary.Caching.Cryptography.dll");
         SafeCopyFile(this.CodeTemplateInfo.DirectoryName + "\\References\\" + entLibSubDir + "\\Microsoft.Practices.EnterpriseLibrary.Security.dll", libPath + "\\Microsoft.Practices.EnterpriseLibrary.Security.dll");
         SafeCopyFile(this.CodeTemplateInfo.DirectoryName + "\\References\\" + entLibSubDir +
      "\\Microsoft.Practices.EnterpriseLibrary.Security.Cache.CachingStore.dll", libPath +
      "\\Microsoft.Practices.EnterpriseLibrary.Security.Cache.CachingStore.dll");
         SafeCopyFile(this.CodeTemplateInfo.DirectoryName + "\\References\\" + entLibSubDir +
      "\\Microsoft.Practices.EnterpriseLibrary.Security.Cryptography.dll", libPath + "\\Microsoft.Practices.EnterpriseLibrary.Security.Cryptography.dll");
      }
      
      SafeCopyFile(this.CodeTemplateInfo.DirectoryName + "\\References\\" + entLibSubDir + "\\Microsoft.Practices.EnterpriseLibrary.Logging.dll", libPath + "\\Microsoft.Practices.EnterpriseLibrary.Logging.dll");
      SafeCopyFile(this.CodeTemplateInfo.DirectoryName + "\\References\\" + entLibSubDir + "\\Microsoft.Practices.EnterpriseLibrary.Logging.Database.dll", libPath + "\\Microsoft.Practices.EnterpriseLibrary.Logging.Database.dll");
      SafeCopyFile(this.CodeTemplateInfo.DirectoryName + "\\References\\" + entLibSubDir + "\\Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.dll", libPath + "\\Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.dll");
      SafeCopyFile(this.CodeTemplateInfo.DirectoryName + "\\References\\" + entLibSubDir + "\\Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.Logging.dll", libPath + "\\Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.Logging.dll");
    • Comment¬†out the following lines (1432 to 1444):
        //----------------------------------------------------------------------------------------------------------------------------------------------
        //-- Entity IEntityCacheItem file
        //----------------------------------------------------------------------------------------------------------------------------------------------
        XmlElement iEntityCacheItemNode = AddFileNode(commonNode, "IEntityCacheItem.cs");
        this.GetTemplate("IEntityCacheItem.cst").SetProperty("NameSpace", BLLNameSpace);
        this.RenderToFile("IEntityCacheItem.cst", rootPathBLL + "\\IEntityCacheItem.cs", true);
        
        //----------------------------------------------------------------------------------------------------------------------------------------------
        //-- Entity EntityCache file
        //----------------------------------------------------------------------------------------------------------------------------------------------
        XmlElement iEntityCacheNode = AddFileNode(commonNode, "EntityCache.cs");
        this.GetTemplate("EntityCache.cst").SetProperty("NameSpace", BLLNameSpace);
        this.RenderToFile("EntityCache.cst", rootPathBLL + "\\EntityCache.cs", true);
  4. TemplateLib
    1. FrameworkTemplace.cst
      • Add¬†entry
        <%@ Register Name="entlibv6_0config" Template="../VisualStudio/entlib.v6_0.config.cst" MergeProperties="False" ExcludeProperties="" %>
    2. CreateTemplate.cs
      • Add¬†entry
        CodeTemplates.Add("entlib.v6_0.config.cst", base.CreateTemplate<entlibv6_0config>());
        this.PerformStep();
    3. CommonSqlCode.cs
      • Add¬†to the switch inside method¬†GetEntLibVersionSignature(EntLibVersion version):
        case MoM.Templates.EntLibVersion.v6_0 :
           entlibVersionText = "Version=6.0.1304.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"; break;
      • Add¬†to the switch inside method¬†GetUnityVersionSignature(EntLibVersion version):
        case MoM.Templates.EntLibVersion.v6_0 :
           unityVersion = "Version=2.0.414.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"; break;
      • Add¬†to the switch inside method¬†GetServiceLocationVersionSignature(EntLibVersion version):
        case MoM.Templates.EntLibVersion.v6_0 :
           unityVersion = "Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"; break;
      • Add¬†to the switch inside method¬†GetEntLibOBVersionSignature(EntLibVersion version):
        case MoM.Templates.EntLibVersion.v6_0 :
           entlibOBVersionText = string.Empty; break;
      • Add¬†to the switch inside¬†method¬†GetEntLibOBClassName(EntLibVersion version):
        case MoM.Templates.EntLibVersion.v6_0:
           entlibOBClassName = string.Empty; break;
      • Add¬†to the switch inside method¬†GetMSBuildExtensionsVersionString(VSNetVersion version):
        case ( VSNetVersion.v2012 ) :
           versionNumber = "11.0"; break;
      • Add¬†to the switch inside method¬†GetVisualStudioSolutionFileVersionString(VSNetVersion version):
        case ( VSNetVersion.v2012 ) :
           versionNumber = "12.0"; break;
      • Add¬†to the switch inside method¬†GetVisualStudioProductVersionString(VSNetVersion version):
        case ( VSNetVersion.v2012 ) : 
           toolsVersion = "4.5"; break;
      • Add¬†to the switch inside method¬†GetVisualStudioToolVersionString(VSNetVersion version):
        case ( VSNetVersion.v2012 ) :
           versionNumber = "11.0.50727"; break;
      • Add¬†to the switch inside method¬†GetVisualStudioNameString(VSNetVersion version):
        case ( VSNetVersion.v2012 ) : 
           versionNumber = "2012"; break;
      • Add¬†to the switch inside method¬†GetVisualStudioGeneralVersionString(DotNetFrameworkVersion
        version):

        case ( VSNetVersion.v2012 ) :
           versionNumber = "11.0.0.0"; break;
      • Add¬†to the switch inside method¬†GetDotNetFrameworkString(VSNetVersion version):
        case ( DotNetFrameworkVersion.v4_5 ) :
           versionNumber = "4.5"; break;
      • Add¬†to EntLibVersion enum:
        /// <summary>Use Enterprise Library version 6.0</summary>
        v6_0 = 16
      • Add¬†to VSNetVersion enum:
        v2012
      • Add¬†to DotNetFrameworkVersion enum:
        /// <summary> version 4.5 </summary>
        v4_5
  5. Entities
    1. EntityManager.cst
      • Comment¬†lines 100 to 106
        //bool isCacheable = defaultType.GetInterface("IEntityCacheItem") != null;
        //see if entity is cachable, if IEntityCacheItem
        //retrieve from cache.
        //if (isCacheable)
        //   entity = EntityCache.GetItem<Entity>(key.ToString());
      • Comment¬†lines 141 to 142
        //if (entity.GetType().GetInterface("IEntityCacheItem") != null) 
        //   EntityCache.AddCache(key, entity);
    2. EntityManager_EntLib5.cst
      • Comment¬†lines 98 to 103
        //bool isCacheable = defaultType.GetInterface("IEntityCacheItem") != null;
        //see if entity is cachable, if IEntityCacheItem
        //retrieve from cache.
        //if (isCacheable) 
        //   entity = EntityCache.GetItem<TEntity>(key.ToString());
      • Comment¬†lines 139 to 140
        //if (entity.GetType().GetInterface("IEntityCacheItem") != null) 
        //   EntityCache.AddCache(key, entity);
    3. Validation
      1. PropertyValidator.cst
        • Lines¬†190, 198 and 212, change a) for b)
          a) <%if(EntLibVersion != MoM.Templates.EntLibVersion.v5_0) { %>
          b) <% if(EntLibVersion != MoM.Templates.EntLibVersion.v5_0 && EntLibVersion != MoM.Templates.EntLibVersion.v6_0) { %>
  6. DataAccessLayer
    1. Bases
      1. EntityProviderBaseCoreClass.generated.cst
        • Line¬†33, change a) for b)
          a) <% if(EntLibVersion != MoM.Templates.EntLibVersion.v5_0) { %>
          
          b) <% if(EntLibVersion != MoM.Templates.EntLibVersion.v5_0 && EntLibVersion != MoM.Templates.EntLibVersion.v6_0) { %>
  7. Components
    1. ServiceBaseCore.cst
      • Line¬†38, change a) for b)
        a) <% if(EntLibVersion == MoM.Templates.EntLibVersion.v5_0) { %>
        b) <% if(EntLibVersion == MoM.Templates.EntLibVersion.v5_0 || EntLibVersion == MoM.Templates.EntLibVersion.v6_0) { %>
    2. SecurityContextBase.cst.
      NOTE: This file generates code that relies on Security features that are no longer available in Enterprise Library 6.0, to it must be re-implemented from scratch

      • Line¬†20 – comment 'using¬†Microsoft.Practices.EnterpriseLibrary.Security;'
      • Line¬†35 – comment 'private IAuthorizationProvider ruleProvider¬†= null;'
      • Line¬†36 – comment 'private ISecurityCacheProvider¬†securityCacheProvider = null;'
      • Line¬†70 – comment property RuleProvider¬†Line 109 – comment the contents of the method 'IsAuthorized',¬†leaving only 'return true' to maintain¬†compatibility with callers
    3. SecurityContext.cst
      • Line 20 – comment 'using¬†Microsoft.Practices.EnterpriseLibrary.Security;'
    4. ContextView.cst
      • Line 20 – comment 'using¬†Microsoft.Practices.EnterpriseLibrary.Security;'

Any comments, suggestions and problems found please share them in the comments below.

That’s it! You should now be able to generate your data¬†access layer targeting .NET 4.5 and Enterprise Library 6.0 as you would do¬†before with .NET 4.0 and Enterprise Library 5.0 (don’t forget to re-check¬†your template settings before starting generating away).