IoT: Raspberry Pi2 and Azure Event Hubs and Mono and SQL Database–experiences

A couple of months ago I bought a Pi2 , to complement the Pi1 I use mostly as a media center. I also bought modmypi’s Raspberry Pi YouTube Workshop Kit, a pack that includes a breadboard, cables, a set of sensors, and that pairs with a set of Tutorial videos on how to set it up. The tutorials are all done using Python, but my goal was (obviously) to do the same using .Net/Mono on raspian.

Using an approach and code that initially was similar for example to Jan Tielens’ in his “Raspberry Pi + GPIOs with DS18B20 + Azure + C# = Internet Thermometer” article, and which I’ll describe in a later post, I now have my Pi2 sending temperature readings to an Azure Event Hub using REST, from where it is read by Azure Stream Analytics and then dropped into an Azure SQL Database. I still hope to wire this up to PowerBI, but there doesn’t seem to be a way at the moment to connect my MSDN Azure account with my corporate account where we have 100 PowerBI licenses, so that will have to wait.

What I wanted to share for now are some tips regarding the process, which are not described elsewhere in other articles I read on the net, and which I guess are very specific to the IoT/sensor world (to which I am new). Keep in mind that my simple goal was to have the Pi2 send temperature readings to Azure every minute.

Service Bus Queue vs Event Hub

My initial code was posting readings to an SB Queue. I didn’t antecipate using Event Hubs, since an event every minute doesn’t justify the platforms’ capabilities. Turns out that Azure Stream Analytics doesn’t support SB Queues as the source, so I had to change the connection code that posts temperature readings using REST. The changes were very few, but included:

• Dropping the support for custom headers, which I was injecting in the message sent to the Service Bus (example: sensor id). I had to move this information into the message payload itself;
• Changing the URL to which the message is posted, including the API version. To write to SB I was using https://myservice.servicebus.windows.net/queuename/messages?timeout=60&api-version=2015-01 and had to change this to: https://myservice.servicebus.windows.net/eventhubname/messages?timeout=60&api-version=2014-05 .

Reading the Temperature – closing the Stream

The code that reads from the sensor, in Jan Tielens’ code (and other similar code found on the net), doesn’t allow for repeat readings in a loop. This line of code:

var w1slavetext = deviceDir.GetFiles(“w1_slave”).FirstOrDefault().OpenText().ReadToEnd();

… actually leaves a text stream open (StreamReader class), that has to be closed for repeat readings to work. So that was another fix.

The main loop – time between readings

My application is simple console application implementing a while(true) loop, that does this:

• Read a temperature value from the sensor
• Send a message to an Event Hub, by doing an HTTP post of a JSON-serialized message
• Wait for 60 seconds with Thread.Sleep

One thing I noticed was that the readings were spaced, not 60 seconds, but 60+something. This “something”, usually 2-3 seconds, were obviously caused by the time the first two steps took. So to fix this I created a System.Diagnostics.Stopwatch at the start of the main loop, and at its end waited for 60 seconds minus the time it took for the first 2 operations to execute.

Now the readings were close enough (to a few milliseconds) to one every minute. Simple fix, simple mistake to make.

The main loop – long operations

The previous solution has a problem, which I quickly found out about. After running for a few hours, I had some posts to the Event Hubs that took a long time. More than 60 seconds. Maybe the cause was some Wifi problem, or network issue, don’t know. But what this meant was that I was calling Thread.Sleep with a negative value, which crashed the app. So another fix: if the operations took more than 60 seconds, don’t sleep and do another temperature reading immediately.

The main loop – SHA tokens’ lifetime

At the top of the app, before the main loop, the first thing I do is to create a SHA token used to connect to the event hub. This token has a lifetime, which I think is one hour by default. So, as you can expect: after one hour of reading temperature (60 readings), the SHA token expired, and sending the message failed with a 401 (permission denied), and I had an exception that stopped the app. Back to the code, another simple fix: wrap the sending of the message to the eventhub (which uses the WebClient class) with a try/catch, and when I find a WebException with 401 as the error code, recreate the SHA token and send the message again.

The main loop – the all encompassing try-catch

The last fix I did after I started having unhandled exceptions which I am not sure are due to the Pi2, Mono, network, whatever: I just wrapped the code inside the main loop inside a general try-catch, logged any error to the console output, and continue the loop execution. A “just in case” solution.

Finally, getting information about the device

This is not specific to the handing of the readings themselves, but I think is relevant. In the payload of my messages I wanted to include some information specific to the device, and found out I could find this information by reading from some devices/streams provided by the Raspian OS. I dug into some samples in the net, and ended up doing code that gets both the serial and the model name. The OS calls I do, using the Process/ProcessStartInfo classes as a way to get into bash are:

cat /proc/cpuinfo | grep Serial | awk ‘{print $3}’   — this gets you the device’s serial, for example “00000000f5b55a06”

cat /proc/cpuinfo | grep ‘model name’ | head -n 1 – this gets you a string from where you can extract the model name, for example “ARMv7 Processor rev 5 (v7l)”

I’m still cleaning up the code and making sure it’s stable, but I’ll post it to github pretty soon. Contact me if want to see it sooner. Anyway, what I did already realize is that the colder time of the day, in my place at least, is between 22:00 and 02:00, which surprised me, and the temperature variation is about 4 Celcius on average.  Interesting info!