IoT: Raspberry Pi2 and Azure Event Hubs and Mono and SQL Database–experiences

A couple of months ago I bought a Pi2 , to complement the Pi1 I use mostly as a media center. I also bought modmypi’s Raspberry Pi YouTube Workshop Kit, a pack that includes a breadboard, cables, a set of sensors, and that pairs with a set of Tutorial videos on how to set it up. The tutorials are all done using Python, but my goal was (obviously) to do the same using .Net/Mono on raspian.

Using an approach and code that initially was similar for example to Jan Tielens’ in his “Raspberry Pi + GPIOs with DS18B20 + Azure + C# = Internet Thermometer” article, and which I’ll describe in a later post, I now have my Pi2 sending temperature readings to an Azure Event Hub using REST, from where it is read by Azure Stream Analytics and then dropped into an Azure SQL Database. I still hope to wire this up to PowerBI, but there doesn’t seem to be a way at the moment to connect my MSDN Azure account with my corporate account where we have 100 PowerBI licenses, so that will have to wait.

What I wanted to share for now are some tips regarding the process, which are not described elsewhere in other articles I read on the net, and which I guess are very specific to the IoT/sensor world (to which I am new). Keep in mind that my simple goal was to have the Pi2 send temperature readings to Azure every minute.

Service Bus Queue vs Event Hub

My initial code was posting readings to an SB Queue. I didn’t antecipate using Event Hubs, since an event every minute doesn’t justify the platforms’ capabilities. Turns out that Azure Stream Analytics doesn’t support SB Queues as the source, so I had to change the connection code that posts temperature readings using REST. The changes were very few, but included:

  • Dropping the support for custom headers, which I was injecting in the message sent to the Service Bus (example: sensor id). I had to move this information into the message payload itself;
  • Changing the URL to which the message is posted, including the API version. To write to SB I was using and had to change this to: .

Reading the Temperature – closing the Stream

The code that reads from the sensor, in Jan Tielens’ code (and other similar code found on the net), doesn’t allow for repeat readings in a loop. This line of code:

var w1slavetext = deviceDir.GetFiles(“w1_slave”).FirstOrDefault().OpenText().ReadToEnd();

… actually leaves a text stream open (StreamReader class), that has to be closed for repeat readings to work. So that was another fix.

The main loop – time between readings

My application is simple console application implementing a while(true) loop, that does this:

  • Read a temperature value from the sensor
  • Send a message to an Event Hub, by doing an HTTP post of a JSON-serialized message
  • Wait for 60 seconds with Thread.Sleep

One thing I noticed was that the readings were spaced, not 60 seconds, but 60+something. This “something”, usually 2-3 seconds, were obviously caused by the time the first two steps took. So to fix this I created a System.Diagnostics.Stopwatch at the start of the main loop, and at its end waited for 60 seconds minus the time it took for the first 2 operations to execute.

Now the readings were close enough (to a few milliseconds) to one every minute. Simple fix, simple mistake to make.

The main loop – long operations

The previous solution has a problem, which I quickly found out about. After running for a few hours, I had some posts to the Event Hubs that took a long time. More than 60 seconds. Maybe the cause was some Wifi problem, or network issue, don’t know. But what this meant was that I was calling Thread.Sleep with a negative value, which crashed the app. So another fix: if the operations took more than 60 seconds, don’t sleep and do another temperature reading immediately.

The main loop – SHA tokens’ lifetime

At the top of the app, before the main loop, the first thing I do is to create a SHA token used to connect to the event hub. This token has a lifetime, which I think is one hour by default. So, as you can expect: after one hour of reading temperature (60 readings), the SHA token expired, and sending the message failed with a 401 (permission denied), and I had an exception that stopped the app. Back to the code, another simple fix: wrap the sending of the message to the eventhub (which uses the WebClient class) with a try/catch, and when I find a WebException with 401 as the error code, recreate the SHA token and send the message again.

The main loop – the all encompassing try-catch

The last fix I did after I started having unhandled exceptions which I am not sure are due to the Pi2, Mono, network, whatever: I just wrapped the code inside the main loop inside a general try-catch, logged any error to the console output, and continue the loop execution. A “just in case” solution.

Finally, getting information about the device

This is not specific to the handing of the readings themselves, but I think is relevant. In the payload of my messages I wanted to include some information specific to the device, and found out I could find this information by reading from some devices/streams provided by the Raspian OS. I dug into some samples in the net, and ended up doing code that gets both the serial and the model name. The OS calls I do, using the Process/ProcessStartInfo classes as a way to get into bash are:

cat /proc/cpuinfo | grep Serial | awk ‘{print $3}’   — this gets you the device’s serial, for example “00000000f5b55a06”

cat /proc/cpuinfo | grep ‘model name’ | head -n 1 – this gets you a string from where you can extract the model name, for example “ARMv7 Processor rev 5 (v7l)”


I’m still cleaning up the code and making sure it’s stable, but I’ll post it to github pretty soon. Contact me if want to see it sooner. Anyway, what I did already realize is that the colder time of the day, in my place at least, is between 22:00 and 02:00, which surprised me, and the temperature variation is about 4 Celcius on average.  Interesting info!

«The Azure App Service Architecture» @ Microsoft Developer TechRefresh 2015–Lisboa

Yesterday we had another Developer TechRefresh at Microsoft Lisboa, where I and my colleague André Vala both presented sessions. My session was a repeat of the Build 2015 session of the same name, presenting the architecture and demoing the two new main components of the new Azure App Service: API Apps and Logic Apps. The first especially I would say are almost ready for prime-time, and both of them are a very good play by Microsoft in the micro-services/mashup space. Very interesting technology, although obviously not everything is finished yet and there are some issues. 

Session slides are available on slideshare. The video of the session was recorded, will link to it when available.

PS: I just wished Microsoft fixed/replaced the “new” Azure portal, I still have frequent errors using it.

ITARC15 Architecting a Large Software Project – Lessons Learned

This morning I presented my “Lessons Learned” workshop at ITARC 2015 in Stockholm, Sweden. This session had previously been presented at Netponto, and was improved with more content targeted at software architects and also updated with more current information. The goal of the workshop (3,5 hours!) is to share experiences and discuss approaches in developing complex software projects. I had great feedback from the participants, and provocative and relevant questions.

One issue that did come up is the definition of “Large”: this was a large project for Portugal’s standards, plus it was complex and took a long time until release. But for local Swedish standards, it wasn’t that “large” Smile. Even so, the contents are general enough to be interesting, or so I was told. Also it was interesting to learn about some cultural differences between Portugal and Sweden – and those were much less than would be expected.

Great session, loved doing it. Here’s the slidedeck, for those interested.

«Architecting a Large Software Project – Lessons Learned» @ Netponto 50th Meeting – Lisboa 22/Nov

Two weeks ago I presented a session at the 50th meeting of the Lisbon Netponto Group, the largest community of .Net development in Lisboa. This two-hour session, which was filled with real examples, described the lessons learned in a 3-year project I was involved in as a Software Architect, and which had its first release this last summer and has seen early success in the customer organization. The compilation of lessons include the feedback of the developers in the team, and was a huge learning experience.

The slides include architecture aspects, technical aspects, as well as negotiation and functional hints. It’s not meant to be an absolute best-practices architecture guide, it’s only the result of this very specific project.

You can check/download the deck here.

Special thanks to NetPonto for the invitation, and to the audience for the participation and great feeedback in this LONG session :).

Microsoft Developer TechRefresh 2014 – Lisboa

Last week (June 18) I presented two sessions at the 2nd edition of the TechRefresh event in Lisbon. The first session (“Azure WebSites: 9 new things”) was an update of the Azure Bootcamp session I did last march, now focusing on the new[est] features on Azure WebSites. A demo-heavy session to a full room, it included:

  1. Backup and Restore (and a bit of Visual Studio Online “Monaco”)
  2. AlwaysOn
    Technorati Tags: ,,

  3. WebJobs (the SDk was updated the day before, had no time to update the demos, but the demo still worked)
  4. Traffic Manager – a new demo, it included creating sites in two data centers and then checking which gets hit from different world regions (using;
  5. Diagnostics, Monitoring and Alerts
  6. Remote Debugging
  7. Staged Publishing – showed the new and amazingly flexible “deployment slots”
  8. Realtime with SignalR+WebSockets
  9. Realtime and auto-scaling with SignalR and a ServiceBus backend

Right after this session, I was on the “Architecture Snacks” panel with Luis Calado, José António Silva and Virgílio Esteves. Each of us presented real-life architecture challenges with Windows Azure solutions (mine were both related to hybrid scenarios), and discussed possible solutions and approaches with the other panelists and audience. Good session with interesting feedback

Microsoft WebCamp 20.05.2014–Databinding with KnockoutJS

Earlier this week I did a presentation at Microsoft WebCamp 2014, a 3-track event focused on Microsoft web technologies, open source web libraries, and design/communication trends and tendencies. The session as a level ~250-300 session on KnockoutJS, Steven Sanderson’s et al. data-binding Javascript framework.

Earlier, my colleague and friend André “SharePoint God” Vala did a session on AngularJS, so we got to share information on two libraries with very different levels of abstraction for web development. Had about 70-80 people in the room, in what was session with more code samples I’ve done for some time now.

You can download the slides here or check them on Slideshare, and the code samples are here.

Global Windows Azure Bootcamp 2014 – Lisboa

The Global Windows Azure Bootcamp this year will include Lisbon. Next saturday, at Microsoft in Lisbon, starting at 09:00, a bunch of great speakers will present on Azure topics. My session will be an evolved version of the last session I did at last year’s Microsoft Developer TechRefresh, on Windows Azure Web Sites. I expect to do a demo-filled session with (hopefully) something new to show you.

Registration here.


ps: a few months back I participated with “Equipa 23” on BPI’s Appyday hackaton. 24 hours of coding of a mobile app at a great venue. We went with a PhoneGap+Telerik Kendo UI Mobile powered solution, with Azure Mobile Services as a backend, which proved itself to be almost perfect for the app. All the regular features were great (database, custom api, git), but the social integration (besides authentication, that is) and features such as image upload + transformation were more lacking. A topic for another post.

Microsoft Developer TechRefresh (Lisboa 2013.09.24)–“Power your Website with Windows Azure”

Last tuesday I presented a session at another local Microsoft event, this time focused on Azure Websites and the new features announced at Build 2013 (which I attended) and since then.

The session was mostly demo-based, and included the “traditional” WebMatrix/WordPress demo, but with the added value of demoing a content migration from another blog engine. After this I did a continuous integration, test and automated deployment demo. Didn’t have the time to show custom deployment scripts, but it was the star of the show, in my opinion. It’s easy to get it working and covers a large percentage of Webdev use cases. The following demo was simply showing how to associate a custom domain to a web site, and what is possible in terms of SSL (both kinds, SNI&IP).

The next demo was related to a request and the need to clarify that you can host wcf web services in an Azure WebSite, and for this demo I also showed , especially helpful for diagnostics and operations.

I ended the session talking about architecture, scaling, service tiers and pricing, and a Q&A. There were several questions, perhaps the most relevant being about Azure Web Sites vs Azure Web Roles. The stronger scaling possibilities, and running setup scripts, are strong arguments for Web Roles, the simplicity and more focused web offering, for Web Sites. This would be my choice, in general.

Another interesting event, with good session feedback.

AzureConf2013 Today!

AzureConf, and online free event on Windows Azure, will be held today, a few hours from now. More information at the events website here. I am especially looking forward to the IaaS, Mobile and Media Services sessions, where we have some customer scenarios ongoing.

Also interesting and related to something I have in hands at the moment is Nuno Godinho’s post “Lessons Learned: Taking the best out of Windows Azure Virtual Machines”. Interesting feedback from using IaaS. SharePoint Server has been implemented everywhere in the last few years. Azure IaaS is especially interesting for these scenarios.

Software Estimation –A Step Closer To The Silver Bullet – Microsoft Lisboa 2013.04.13 (Updated)

Tomorrow I’ll be redoing my Software Estimation presentation at SharePoint’s Community. This presentation started with an invitation to present on this topic from one of our customers, and evolved into a general presentation on Software Estimation and the techniques Create It uses. I delivered it last year at Netponto, this year is SharePoint’s Community turn with an updated version :).

There’s more information about the event here.

Update: you can find the slides here. The video of the previous delivery of the session at Netponto is here (note: contents in Portuguese). As to the session on Saturday, the topic always raises lots of interest and conversations, and this was no exception. I n my view, this topic should be studied in CompSci courses.

Microsoft Azure, Software Architecture, Systems Integration, Azure WebSites