Introduction

An Azure SQL Server failover group is a group of databases that can be automatically or manually failed over from a primary server to a secondary server in a different Azure region in case of a disaster in the primary server. Failover groups provide high availability and disaster recovery for Azure SQL Server databases.

The process described in this post is composed by two main steps:

• Provisioning the database
• Adding the database to the failover group

Provisioning the database

The first step is to create the database. This step is composed by the following actions:

• Get the tenant information
• Create the database

Tenant information class:

Code to create database programatically:

Adding the database to the failover group

The second step adds the newly created database to the failover group. This step is composed by the following actions:

• Get the failover group where we want to add the database
• Re-add the existing databases to the failover group – necessary since when we get the failover group the list of databases of the group is empty and, without this step, the failover group would only have the new database
• Add the new database to the failover group

Other Articles

To learn why your business should migrate to SharePoint Online and Office 365, click here and here.

If you want to learn how to develop SPFx solutions, click here.

If you want to learn how you can rename a modern SharePoint site, click here.

If you want to learn how to save time time scheduling your meetings, click here.

If you want to learn how to enable Microsoft Teams Attendance List Download, click here.

If you want to learn how to create a dynamic org-wide team in Microsoft Teams with all active employees, click here.

If you want to modernize your SharePoint classic root site to a modern SharePoint site, click here.

If you are a SharePoint administrator or a SharePoint developer who wants to learn more about how to install a SharePoint 2019 farm in an automated way using PowerShell, I invite you to click here and here.

If you learn how to greatly speed up your SharePoint farm update process to ensure your SharePoint farm keeps updated and you stay one step closer to start your move to the cloud, click here.

If you prefer to use the traditional method to update your farm and want to learn all the steps and precautions necessary to successfully keep your SharePoint farm updated, click here.

If you want to learn how to upgrade a SharePoint 2013 farm to SharePoint 2019, click here and here.

If SharePoint 2019 is still not an option, you can learn more about how to install a SharePoint 2016 farm in an automated way using PowerShell, click here and here.

If you want to learn how to upgrade a SharePoint 2010 farm to SharePoint 2016, click here and here.

If you are new to SharePoint and Office 365 and want to learn all about it, take a look at these learning resources.

If you are work in a large organization who is using Office 365 or thinking to move to Office 365 and is considering between a single or multiple Office 365 tenants, I invite you to read this article.

If you want to know all about the latest SharePoint and Office 365 announcements from Ignite and some more recent announcements, including Microsoft Search, What’s New to Build a Modern Intranet with SharePoint in Office 365, Deeper Integration between Microsoft Teams and SharePoint and the latest news on SharePoint development, click here.

If your organization is still not ready to go all in to SharePoint Online and Office 365, a hybrid scenario may be the best choice. SharePoint 2019 RTM was recently announced and if you to learn all about SharePoint 2019 and all its features, click here.

Happy Coding!

The post Provision a database programmatically in Azure SQL database with a failover group appeared first on Blog IT.

The pipelines bring to us the opportunity to create automatic tasks to execute development operations, like deploys, migrations, tests, etc. Focused in the deploy routine, some applications need other operations to ensure the stability between the application and other resources, like the database.

During the development process, the developers usually customize and improve the application database in the development environment scope, and it can be made easily by using frameworks that contains routines to create source files that have the responsability to keep the history of changes and are used to update the database for determinated version. These files are known as migrations. 

In this post, we will see how to apply migrations for the databases that are located in different environments (staging, production, etc.), using the Code-First entity concept, and the Entity Framework Core.

Migrations

To generate database migrations, we can execute the “Add-Migraton <name>” command from the Entity Framework. This command will generate some source files that will contain the code to apply the migration to the database. To execute these migration files, you just need to execute the “Update-Database” command. But, Microsoft recommends to create a SQL file individually, and suggest to execute this file direct to the database, without need to execute an intermediary software (C# + EF) to access the database. To do this, we will have to follow the steps below:

• 1. Create a Pipeline in Azure Devops;
• 2. Install dotnet-ef-tool;
• 3. Generate SQL script;
• 4. Execute SQL script to the database;
• 5. Script File as a Pipeline Artifact.

NOTE: We don’t need to worry about which migration need to be applyed to the database. The dotnet-ef-tool command will generate a script cantaining the needed logic. Check the third step for more details.

1. Create a Pipeline in Azure Devops

We will create a common deployment pipeline in Azure Devops for ASP.NET Core backend applications. Check the end of the post to see the full result of the pipeline.

2. Install dotnet-ef-tool

To execute tool commands of Entity Framework, we have to install a package named dotnet-ef-tool. This needs to be available into the pipeline’s context to be used for the SQL script generation. To install this tool by a pipeline task, we can use the task below:

- task: DotNetCoreCLI@2
  displayName: 'Install EF tool'
  inputs:
    command: 'custom'
    custom: 'tool'
    arguments: 'install --global dotnet-ef'

3. Generate SQL script

To generate the SQL script containing the logic to apply migrations, we will use the dotnet-ef-tool. By default, the command does not generate the logic to check which migration should be applied, so we have to enable this behavior using the parameter: –idempotent. By using this behavior, we don’t need to worry about the migrations that will be applyed to the database since the generated script already has it. Below, you can see the task that could be used to generate the script file:

- task: DotNetCoreCLI@2
  displayName: 'Generate SQL Script'
  inputs:
    command: 'custom'
    custom: 'ef'
    arguments: 'migrations script --idempotent --project $(Build.SourcesDirectory)\Data.csproj --output $(System.ArtifactsDirectory)/script.sql'

NOTE: We don’t need to stablish any database connection for the script generation, since we are following the Code First concept for entity creation, so, the command will just look at the entities source code of the project to generate the SQL file. If we don’t use the –idempotent argument, the script will generate the SQL without the conditional logic to determinate which migration should be applied, causing an error if the database already exists.

4. Execute SQL script to the database

Now, with the SQL script available, we just need to execute this file to the database. In this post, we will use a common Azure Database as an example, so for this case, we can use the following task to accomplish this requirement:

- task: SqlAzureDacpacDeployment@1
  displayName: 'Update Database'
  inputs:
    azureSubscription: '<Service Connection Identifier>'
    AuthenticationType: 'connectionString' # You can use other method to authenticate to the database if you want
    ConnectionString: '<Database Connection String>'
    deployType: 'SqlTask'
    SqlFile: '$(System.ArtifactsDirectory)/script.sql'

5. Script File as a Pipeline Artifact

Well done. The last task will only be used to make the file available to be accessed by the pipeline runner user. To do this, you can use the task below to add the generated script.sql file to the file bundle (artifacts). To check the artifacts, just access the artifacts option from the pipeline details after the job’s execution.

- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifacts'
  inputs:
    PathtoPublish: '$(System.ArtifactsDirectory)/script.sql'
    ArtifactName: '$(artifactName)'
    publishLocation: 'Container'

Conclusion

In this post, we could learn an easy way to apply the database migrations to the remote environment by using automation (pipelines) to accomplish it. This method is simple, and does not have any database validation before script execution. So, if you have an active and sensitive environment, be sure that the needed validations are considered before the database update, to ensure that the database will not be update wrongly.

Pipeline Result

trigger:
- none

pool:
  vmImage: windows-latest

variables:
  solution: '**/*.sln'
  buildPlatform: 'AnyCPU'
  buildConfiguration: 'Release'
  artifactName: 'artifacts'
  environment: 'Development'

- task: NuGetToolInstaller@1
  displayName: 'NuGet Installation'

- task: NuGetCommand@2
  displayName: 'NuGet Restore'
  inputs:
    restoreSolution: '$(solution)'

- task: VSBuild@1
  displayName: 'Build Project'
  inputs:
    solution: 'backend/WebAPI'
    msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)"'
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

- task: PublishSymbols@2
  displayName: 'Publish Symbols'
  inputs:
    SearchPattern: '**/bin/**/*.pdb'
    SymbolServerType: 'FileShare'
    CompressSymbols: false

- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifacts'
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)'
    ArtifactName: 'artifacts'
    publishLocation: 'Container'

- task: DownloadBuildArtifacts@0
  displayName: 'Build Artifacts'
  inputs:
    buildType: 'current'
    downloadType: 'single'
    artifactName: '$(artifactName)'
    downloadPath: '$(System.ArtifactsDirectory)'

- task: DotNetCoreCLI@2
  displayName: 'Install EF tool'
  inputs:
    command: 'custom'
    custom: 'tool'
    arguments: 'install --global dotnet-ef'

- task: DotNetCoreCLI@2
  displayName: 'Generate SQL Script'
  inputs:
    command: 'custom'
    custom: 'ef'
    arguments: 'migrations script --idempotent --project $(Build.SourcesDirectory)\Data.csproj --output $(System.ArtifactsDirectory)/script.sql'

- task: SqlAzureDacpacDeployment@1
  displayName: 'Update Database'
  inputs:
    azureSubscription: '<Service Connection Identifier>'
    AuthenticationType: 'connectionString'
    ConnectionString: '<Connection String>'
    deployType: 'SqlTask'
    SqlFile: '$(System.ArtifactsDirectory)/script.sql'

- task: AzureRmWebAppDeployment@4
  displayName: 'Deploy Application'
  inputs:
    ConnectionType: 'AzureRM'
    azureSubscription: '<Service Connection Identifier>'
    appType: 'webApp'
    WebAppName: 'armazemlegalapi'
    deployToSlotOrASE: true
    ResourceGroupName: '<Resource Group Identifier>'
    SlotName: 'production'
    packageForLinux: '$(System.ArtifactsDirectory)/Project.zip'

- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifacts'
  inputs:
    PathtoPublish: '$(System.ArtifactsDirectory)/script.sql'
    ArtifactName: '$(artifactName)'
    publishLocation: 'Container'

References

• Microsoft Managing Schemas Article

The post Entity Framework – Update Database using Azure Pipelines appeared first on Blog IT.

In this article we will talk about Event-Driven Architectures. We choose to use the Azure Cloud Infrastructure.
Service Bus provides reliable, secure asynchronous messaging at scale. This article is written by the engineering team at CreateIT and it is intended to show you a case study in one of our projects for a client.

We’ll take a deeper dive into the Service Bus technology, architecture, and design choices. The post will cover both conceptual material as well as implementation details. Most importantly, we will discuss design and implementation of some of the features that provide secure and reliable messaging at scale, while minimizing operational cost.

Service Bus Entities

When we are working with Azure Service Bus, we can choose two Entities: Topics or Queues. You can have multiple Topics or Queues per Service Bus Namespace, but firstly you need to differ one from another. If you want a FIFO queue and only have one Consumer, then Queues are the way to go. If you need multiple Consumers, then the Topic is the better option. In this specific case we will create a Subscription per Consumer (Topics are only available from the Standard Pricing Tier).

Event-driven architectures

Benefits with event-driven architectures

What are the benefits of using a queue in the middle of these systems?

• We can decide to load balance the input from Customer Services. Let’s say there are a lot of updates being made to a customer, meaning a lot of events being published. We scale the number of consumers and user the competing pattern
• We can throttle the input. If on black Friday there are a ton of events and in case our Audit Log system is down. We simply store these events on the queue and consume them when the service is back online again. Of course we’d need to implement some logic for this behavior, but adding this “middleware” buys us options.

In our use case, we wanted to move to an implementation where the Web API doesn’t get affected by any changes on these external systems. But in order to change the implementation, we must first figure out what are the challenges associated with this change.

Challenges with event-driven architectures

Message/Event order

Azure Service Bus has a feature called sessions. A session provides a context to send and retrieve messages that will preserve ordered delivery. However, in our use case we chose not to use it.

Message Lock Duration

When we are using Queues, every message has a lock Duration. During this time the consumer needs to process it. But if this consumer needs to contact multiple external systems this time may rise and our messages could stop in the Dead-Letter. So the best practice is to change it accordingly to your needs. We recommend you to set this time extremely high in the beginning, and then do some tests to calculate the average.

After that add 30% more of its value, in case of some lengthy  requests (in this case if we have outliers, they might stop in the Queue). If you are using a Topic, you will have a Lock Duration per Subscription. So make sure to adjust this time accordingly to its functions.

Implementation

In the Figure 1 you can see the initial architecture for the Customer Management system. It was responsible to make the requests to the other systems.

With the new implementation, a message broker was introduced and we used the event-carried state transfer pattern, meaning our events had all the information the consumer needed in order to do their job. We took in consideration the event notifications pattern, where the consumer would have to make a request to the API that originated the event, in order to get more information. But this brings new problems to the table. What if when the consumer code runs, the information for that customer ID changed? What if the event was CUSTOMER_CREATED but in the meanwhile the customer was deleted?

Retries with Polly

In a distributed system, many things can go wrong. The network can fail or have additional latency, systems may be temporarily down, etc. We use the Azure.ServiceBus.Messaging NuGet package so we are able to check if the exception is a transient fault or not (more information on these docs), then use Polly to setup retry logic and fallbacks. There are other options to implement retry policies, for example we took in consideration the Retry guidance for Azure Services documentation from Microsoft. Since we use the latest Azure SDK,  the appropriate class would be ServiceBusRetryPolicy.
We configured Polly to retry to publish a message three times (this configuration is on appsettings.json), with exponential times between each attempt.
If after the third retry we can’t publish the message we need to save it, because it has crucial information. So to solve this issue we created a Fallback Gateway, to write these messages to a Container inside an Azure Storage Account.

Filters for message routing

This section only applies for Topics Entities on the Azure Service Bus.
We can add Filters on our Subscriptions to help us with routing each message to its specific Consumer. We considered two filter types:

• SQL Filter
• Correlation Filter

Using the Correlation Filter you can configure Custom Properties and create Filters for your needs. You just need to make sure the producer of the messages, includes the header, that you are currently using to filter, on the message.

With SQL Filters you can create conditional expression to evaluate the current message. Just make sure that all the system properties are prefixed with sys. in the expression. Either way, both filters work just choose one that suits you the most!

Dead-Letter Queue

In case the consumer application can’t process the message after the Max Delivery Count attempts, instead of returning it to the queue it will be sent automatically to the Dead-Letter queue. If you are using the Topic, each subscriber has its own Dead-Letter queue. You can also, configure different Max Delivery Counts Values for each Subscriber.

All messages that are published to the Service Bus have a TTL (Time-To-Live). After this time ends the message will be transferred automatically to the Dead-Letter. So make sure you adjust this time accordingly to your needs.

With this we are able to save messages that weren’t processed by the consumer application, but we should always strive to have an empty dead-letter queue.

Conclusion

Our first steps into an Event-Driven Architecture was a truly success!
We were able to expand our previous solution to be compatible with multiple external systems and instead of having the API sending a HTTPS request for each one we had this Application sending one message to a Topic in the Service Bus.
One of our goals was to have load balance in the Publisher Application. We went from a 1->3 dependency to a 1-> 1, as you can see in the Figure 2.

Which is great and keeps the system scalable and future proof. Our solution became more decoupled in order to keep the Application agnostic to these changes.
If you have a similar situation with an Event-Driven Architecture then we totally recommend you to check more about this technology and it’s features.

We would like to share a link to Microsoft Azure Service Bus GitHub. Most of the implementations of either the publisher or the subscriber were inspired by this documentation, so make sure you check it out!
If you have any questions, please write them down below.

Additional Links

Service Bus Exceptions
Service Bus Basic Steps
Tutorial with DotNet

The post Case Study: Azure Service Bus and Event-Driven Architectures appeared first on Blog IT.

WebPagetest is a tool that was originally developed by AOL for use internally and was open-sourced in 2008 under a BSD license. The online version at www.webpagetest.org is run for the benefit of the performance community with several companies and individuals providing the testing infrastructure around the globe.

This tool tests any website against 6 major performance affecting factors, and provides a myriad of graphs and logs that make abundantly clear what might be slowing down your site.

In this post I’ll provide ways to make your site get straight A’s in WebPagetest.

Setup

To start this off, let’s setup our environment. We’ll just need the following:

• Visual Studio 2017
• Microsoft Azure account

In Visual Studio, let’s create a new empty ASP.NET Web Application project. Then, we’ll need the latest and greatest Umbraco NuGet package (I used version 7.12.4). Once it finishes installing, just launch the website and install Umbraco with all defaults. This will bootstrap Umbraco with the Starter Website, which we’ll use as our “guinea pig” for WebPagetest.

Next: publish it! We can use the publish wizard to automatically create our new WebApp and SQL Database in Azure. Before installing Umbraco in Azure, we’ll need to change the Web.config so that the install wizard is run again (I use Filezilla to change it in Azure):

Clear the Umbraco version number:

<add key="umbracoConfigurationStatus" value="">

Clear the Umbraco connection string:

<add name="umbracoDbDSN" connectionstring="" providername="System.Data.SqlClient">

With these changes in place, we’re good to go. This time, we’ll not use the defaults in the Umbraco install wizard, since we’ll want to use the SQL Database we’ve just created in Azure.

First test

For our tests, we’ll use the people page of the Starter Website. This is the score I got with a standard (S0) database and a basic WebApp:

Since this is a very small site, only used for demonstration purposes, half of the metrics are already cleared! However, this is not usually the case for bigger websites. For this reason, I’ll still present some solutions to improve the grade for these metrics.

First Byte Time

These test measures the time it takes for the first byte to reach the client’s browser after the initial http request.

There are two main factors that influence this result:

• Server power
• The webpage complexity (integrations with external services, complex logic involved, etc)

How to get an A

The easiest way to mitigate this problem is by caching. You can see how to do output caching in Umbraco by reading this old post of mine: https://blogit.create.pt////andresantos/2016/06/30/umbraco-and-donut-output-cache/.

Keep-alive Enabled

Keep alive is a method to allow the same tcp connection for HTTP conversation instead of opening a new one with each new request.

How to get an A

This setting is active by default in IIS, so, it’s also active by default in Azure WebApps, so it’s easy to get an A!

Compress Transfer

Gzip compresses your webpages, style sheets and javascripts, before sending them over to the browser. This drastically reduces transfer time since the files are much smaller.

How to get an A

Just add this setting to your Web.config file:

<httpcompression dynamiccompressionenablecpuusage="0" dynamiccompressiondisablecpuusage="90" nocompressionforhttp10="false" nocompressionforproxies="false">
    <statictypes>
        <add mimetype="text/*" enabled="true">
        <add mimetype="message/*" enabled="true">
        <add mimetype="application/javascript" enabled="true">
        <add mimetype="application/font-woff" enabled="true">
        <add mimetype="application/font-woff2" enabled="true">
        <add mimetype="application/vnd.ms-fontobject" enabled="true">
        <add mimetype="application/octet-stream" enabled="true">
        <add mimetype="*/*" enabled="false">
    </add></add></add></add></add></add></add></add></statictypes>
    <dynamictypes>
        <add mimetype="text/*" enabled="true">
        <add mimetype="message/*" enabled="true">
        <add mimetype="application/javascript" enabled="true">
        <add mimetype="*/*" enabled="false">
    </add></add></add></add></dynamictypes>
</httpcompression>

Compress Images

Image compression is minimizing the size in bytes of a graphics file without degrading the quality of the image to an unacceptable level.

How to get an A

In Umbraco, to get an A in this grade, you need to do two things:

1. Crop every image and use srcsets where you can
2. Use the PostProcessor plugin for Image Processor

Cropping an image is easy in Umbraco:

<div class="employee-grid__item__image" style="background-image: url('@person.Photo.GetCropUrl(width: 323, height: 300, quality: 85)')"></div>

In order to use the PostProcessor plugin, you just need to install it via nuget: https://www.nuget.org/packages/ImageProcessor.Web.PostProcessor/1.3.1.25.

Cache Static Content

Static content is content that changes rarely. For this reason it can be cached in the user’s browser to avoid downloading the same file over and over again.

How to get an A

Just set the time it takes for the content to expire in the user’s browser and add extra mime types if you want:

<staticcontent>
    <clientcache cachecontrolmode="UseMaxAge" cachecontrolmaxage="7.24:00:00">
    <remove fileextension=".air">
    <mimemap fileextension=".air" mimetype="application/vnd.adobe.air-application-installer-package+zip">
    <remove fileextension=".svg">
    <mimemap fileextension=".svg" mimetype="image/svg+xml">
    <remove fileextension=".woff">
    <mimemap fileextension=".woff" mimetype="application/x-font-woff">
    <remove fileextension=".woff2">
    <mimemap fileextension=".woff2" mimetype="application/x-font-woff2">
    <remove fileextension=".less">
    <mimemap fileextension=".less" mimetype="text/css">
    <remove fileextension=".mp4">
    <mimemap fileextension=".mp4" mimetype="video/mp4">
    <remove fileextension=".json">
    <mimemap fileextension=".json" mimetype="application/json">
</mimemap></remove></mimemap></remove></mimemap></remove></mimemap></remove></mimemap></remove></mimemap></remove></mimemap></remove></clientcache></staticcontent>

Effective use of CDN

A content delivery network (CDN) refers to a geographically distributed group of servers which work together to provide fast delivery of Internet content. A CDN allows for the quick transfer of assets needed for loading Internet content including HTML pages, javascript files, stylesheets, images, and videos.

How to get an A

In Umbraco, you can achieve this last grade by doing two things:

1. Use an Azure Blob Storage for media storage by installing this nuget package: https://github.com/JimBobSquarePants/UmbracoFileSystemProviders.Azure.
2. Create an Azure CDN for serving these blobs through a content delivery network.

After creating an Azure CDN service and waiting about an hour for it to be available, the following presents my configuration for the media assets to be provided by it:

<!--?xml version="1.0" encoding="utf-8"?-->
<security>
  <services>
    <service name="LocalFileImageService" type="ImageProcessor.Web.Services.LocalFileImageService, ImageProcessor.Web">
    <!--Disable the LocalFileImageService and enable this one when using virtual paths. -->
    <service prefix="media/" name="CloudImageService" type="ImageProcessor.Web.Services.CloudImageService, ImageProcessor.Web">
      <settings>
        <setting key="Container" value="media">
        <setting key="MaxBytes" value="8194304">
        <setting key="Timeout" value="30000">
        <setting key="Host" value="https://<umbracositename>.blob.core.windows.net/media">
      </setting></setting></setting></setting></settings>
    </service>
  </service></services>
</security>

<!--?xml version="1.0" encoding="utf-8"?-->
<caching currentcache="AzureBlobCache">
  <caches>
    <cache name="AzureBlobCache" type="ImageProcessor.Web.Plugins.AzureBlobCache.AzureBlobCache, ImageProcessor.Web.Plugins.AzureBlobCache" maxdays="365">
      <settings>
        <setting key="CachedStorageAccount" value="DefaultEndpointsProtocol=https;AccountName=<accountname>;AccountKey=<accountkey>;EndpointSuffix=core.windows.net">
        <setting key="CachedBlobContainer" value="cache">
        <setting key="UseCachedContainerInUrl" value="false">
        <setting key="SourceStorageAccount" value="DefaultEndpointsProtocol=https;AccountName=<accountname>;AccountKey=<accountkey>;EndpointSuffix=core.windows.net">
        <setting key="SourceBlobContainer" value="media">
        <setting key="StreamCachedImage" value="false">
        <setting key="CachedCDNRoot" value="https://<cdnrootname>.azureedge.net">
        <setting key="CachedCDNTimeout" value="1000">
      </setting></setting></setting></setting></setting></setting></setting></setting></settings>
    </cache>
  </caches>
</caching>

Conclusion

So, if you followed these tips correctly, you’ll be able to run WebPagetest and get the same result as I did:

You can find the complete report here: https://www.webpagetest.org/result/181127_2A_bea6941dcd20d38ab54c29409fca9363/.

The post Straight A’s in WebPagetest with Umbraco appeared first on Blog IT.

Create IT as an Internet connection of 100mbps Down/20mbps Up. Azure was capping at a 150mbps symmetrical connection.

TeamViewer VPN, Azure Site-to-Site and Point-to-Site connections were capped at 10mpbs.

Azure plans

In Azure, the most economical plans were chosen, considering our requirements. Some plans were free, some were cheaper than the others, but having VPN capabilities and all configurations supported, we chose the cheaper plans that were available with all these needed features supported.

Browsers

We used Chrome, Firefox and Edge. This was to eliminate browser differences. Not showing any differences in total execution times, we kept using Chrome as a default testing browser.

LAN Connection

Lan wise, our internal network is based on a 1gbps Local Area Network.

On-Premises Service Host

The On-Premises Service was hosted on a machine running Windows 10 Retail with latest updates installed and IIS Express. All code was made with Visual Studio 2017 Enterprise. The relevant host hardware specifications are:

1. Intel® Core™ i7-6700HQ CPU @ 2.6GHz
2. 32GB of RAM DDR4 @ 3400MHz
3. NVMe M.2 PCI-e 240GB SSD

Testing Hours

All tests were executed on working hours, 9am to 18pm, GMT.

The Writer

I’m a consultant @ Create It, a Portuguese company. If you read this, make sure I know about it! ? Hugs and kisses! This was an absolute pleasure to make ?

The post Latency test between Azure and On-Premises – Specifications appeared first on Blog IT.

Note: 5MB results must be multiplied by 10 (value x 10)

Here are the results. All of them. Don’t forget to multiply 5MB results by 10!

With the graph below, we can check how message sizes influence latency. After doing some math, these are the results:

As we can confirm, latency follows an exponential growing rate, which means, the bigger the message, even greater will be latency. We can confirm that, for each scenario, this applies.

But what does it tell me? This tells you that you need to be careful if you’re planning on sending big messages between two points. This exponential growing rate applies to all scenarios!

All values listed and noted, so we gave tests a score. This score is calculated by adding all three results (10kb, 100kb and 5mb) and dividing all by three. This will give us average execution time. For your best understanding, below is an exponential graph. This climbs faster and faster to infinity, being the horizontal axle message sizes, and the vertical one latency:

Now for the results!

As we can check, Test #2 (Local On-Premises LAN) was the winner here, latency wise. This was expected. Although this is the fastest, it’s the riskiest and NOT RECOMMENDED! It’s true that latency is reduced, but with only ~53ms difference between Azure Site-to-Site on a 100Kb message, is it worth to have a full infrastructure depending on your maintenance? And the networking gear it requires? What about having someone responsible for the room 24/7? How about costs? Think about it! Think about savings when migrating your business to Cloud!

Second place goes to HTTP without VPN (Exposed services). Not a Cloud solution, and a risky one too! If you like hackers messing with your vital business services and trying to break in from all over the world 24/7, go right ahead!

Regarding fast and safe Cloud solutions, which is the main reason of reading this document after all, the winner was Test #4 (Azure Site-to-Site VPN), getting third place on the board, by a difference of 8 points! This managed to be the safest, more efficient and with 99.9% uptime method of expanding your office or network, and have a low-latency data exchange. This is the optimal and safer solution. No hackers, no maintenance, no worries!

Consult us to migrate your business! You’re at the doorstep to your future!

For full test specifications, read here!

The post Latency test between Azure and On-Premises – Conclusions appeared first on Blog IT.

First step is to configure an Hybrid Connection in Function Apps network configuration. Then, you can run Hybrid Connection Manager (assuming you’ve downloaded it from Azure), to manage your connection.

After saving which connection you want to use, you can always confirm connection status in this window:

But wait, I see Service Bus referenced in that print screen! What’s that?

Service Bus is a technology of asynchronously sending and receiving messages from multiple publishers to multiple subscribers. This is, as you’ve probably guessed, a publish/subscribe mechanism.

This is an example of a publish/subscribe architecture. The image should be self-explanatory. A sender sends a message, and all subscribers have filters to only process the messages that they subscribe to.

Why didn’t you test this method?

Well, before you call me anything, you got to understand why.

Architecture!

This concept relies on checking the bus for new messages from time to time, with a 10 second interval for instance. Each 10 second, your program/service, or whatever you want to call it, will check the bus for new messages and retrieves them. This generates a “waiting game”, that’s why it’s called an asynchronous process. It’s not designed to be fast, but to be reliable and to connect multiple services into one giant message box.

Example:

A highway. A highway in USA connects multiple states together, and you can choose where to leave it. You can choose a city in a certain state, or a totally different city in another state, it just depends where you’re going to. This is the same. The service bus it’s like that highway. A publisher publishes a car. Each message (car) has a topic (destination), and every subscriber (city/state) retrieves a message (car) depending on their topic (destination).

This is the main reason that latency doesn’t apply here. It’s a very robust and clean way of integrating services with applications, but latency isn’t a concern here. It’s just not made to be fast. It’s made to be simple.

Conclusions and summary next post!

The post Latency test between Azure and On-Premises – Part Seven appeared first on Blog IT.

This function calls our  REST Web Service (On-Premises) via Point-to-Site VPN connection to Azure. This can be called on-demand via URL, via Run button on the image, or via Logic Apps.

Important note:
In this scenario, the client makes one single HTTP request, instead of 10 requests as previous scenarios.

This is a Logic App (Designer View). We start by a Recurrence Action, which is a timer set to run each 30 second. When recurrence action is fired, this logic app starts by calling “HttpTriggerCSharp1”, which is the function app showed previously. When this function app is finished, the logic app sends an e-mail to my Create account reporting total execution time.

Finally, this is the e-mail that is sent by our logic app. We can see that total execution time reported by the function app is 63ms. Read on for full results!

Test #6 10kb message

Called by a Logic App trigger, the client makes a HTTP request to our On-Prem Service, receiving a 10kb message, the same as before.

We can see that total execution time was 59ms. This is measured by calling the function app via Run button. We’ll be doing this due to being easier to obtain results. This same result is reported by mail if the function is called by Logic Apps. Execution time oscillated between 47ms and 70ms, being the median value ~59ms. Not bad, and slightly faster than TeamViewer VPN, but slower than direct HTTP request without VPN.

Regarding this same result and comparing it to Site-to-Site VPN, we can see that this method took more ~15ms to execute. This can have a very important influence if you are choosing between PTS and Site-to-Site for your business. Also, Site-to-Site has a more constant flow, while this test was more irregular (had more peaks).

Rest assured that this method works, if you can trade a slight latency increase over cost.

Test #6 100kb message

Now for 100kb message, we’ll repeat the same steps to obtain the results.

As we can verify, the test took 78ms to execute. It peaked 97ms and drops to 68ms. Overall, it took more ~8ms to execute than Site-to-Site but was ~22ms faster than TeamViewer. Regarding direct HTTP request without any VPN, this was slower ~17ms. That’s quite a punch there. Let’s try 5Mb.

Test#6 5mb message

Now to the ultimate 5MB test. No more words needed here.

We can see that this is a deal breaker here. The latency here is way over any other test we did. 3.8s! This may happen due to several reasons, the main ones being:

1. Function App performance not meeting our requirements,
2. The way that Azure connects PTS VPN to Function Apps,
3. Consuming a large data might be doing some memory swaps server-side.

Overall test discussion

Well, this is very surprising. Function Apps is clearly not the way for a large message. The processing is inefficient, therefore takes way too much time. PTS connection to a VM in Azure was faster by more than one second, and surprisingly cheaper!

This solution may still be in Preview, or may have bugs, or maybe it just wasn’t made for this type of usage. Regarding complexity, is clearly simpler to use than a VM, it’s serverless and scalable. You can get rid of VM and make your functions directly accessible via any web browser on any platform.

For exposing your logic in a cleaner, faster way, this is what you need. For data crunching or analytics, it’s also the cleanest way, but if you’re planning to use it to make HTTP requests to your on-prem services, you should consider other options, unless execution time isn’t on the equation.

Read the next post here about relays and service bus!

The post Latency test between Azure and On-Premises – Part Six appeared first on Blog IT.

Will this be faster that Azure Site-to-Site?

Test #5 10kb message

10Kb is our base message. PTS didn’t show any issues regarding this one.

Execution time is practically the same as before (on Site-to-Site scenario), with the same message size. 1% is not noticeable on a large batch execution. Let’s see for 100KB. If results change more drastically, then we can analyze and find a reason for this.

Test #5 100kb message

We didn’t see a big difference before. Will 100kb be the game changer?

Actually, it is! It’s ~10ms faster than before! Site-to-Site is beginning to lose to PTS.

Probable causes are:

1. Router delays due to packet queues (Site-to-Site)
2. Router processing power limitations of encrypting packets (and routing them)
3. General hardware limitations

 We start to see some differences now. ~10ms is a noticable improvement. How fast will 5MB be?

Test #5 5mb message

The ultimate test on PTS.

Well, PTS it’s not for large messages. It’s inefficient and takes a lot of time! You should only use this for a one-time-only scenario, like accessing a VM or downloading some files from Azure. Processing large messages with PTS is not recommended, as execution times increased ~1 second in average.

 Overall test discussion

This is an awkward situation. 100KB is faster than Site-to-Site, but 5MB it’s not. While processing small to medium messages you should be fine. Not recommended for production as it showed to have some unusual peaks, so latency may be compromised.

Go to next test!

The post Latency test between Azure and On-Premises – Part Five appeared first on Blog IT.

Consider it as an extended office, with VMs and Azure Functions running outside your premises, as if they were there right next to you! It’s the future.

We’ll be using the same messages, as well the same service-client logic (Via HTTP GET). Instead of using TeamViewer VPN or exposing our service, we’ll use a secure VPN connection to a gateway, that has one VM that’s running the client app associated to it.

We’ll start by doing 10kb, following 100kb and 5MB. Again, this is the most common scenario nowadays!

Test #4 10kb message

Running 10kb of data through Azure VPN Site-to-Site is surely a piece of cake.

It took ~44ms to execute! It’s roughly the same as 263bits through TeamViewer VPN! Using the same 10kb, this test scenario has a reduced execution time of~20ms! Now that’s an improvement. A secure, tight VPN to Azure, site-to-site, with reduced latency! That’s quite a hit in P2P (Peer-to-Peer) VPN connections!

Test #4 100kb message

Well, after seeing a ~20ms drop with a 10kb message, will execution times drop with a message that’s 10 times bigger?

Remember how TeamViewer VPN managed ~100ms on this test? If that’s quick, this is blazing fast! Less ~30ms than P2P VPN solution. Azure Site-to-Site connection is proving to be better than TeamViewer, and only ~9ms slower than HTTP without VPN. Let’s go for 5MB.

Test #4 5MB message

Let’s go through the ultimate test. Can we saturate the Azure VPN connection with this?

Well, that’s astonishing. For a message this size, execution time it’s greater by ~150ms only! Guess that’s the price to pay for a secure tunnel. Direct HTTP execution times are smaller, but considering risk over latency, this is a great and optimal candidate.

Overall test discussion

Well, this did prove me wrong. Initially, I had the idea that TeamViewers’ P2P VPN connection was born to be faster, but it’s not! Azure Site-to-Site is faster, safer, scalable and available to multiple devices, both on Azure and on-premises. Personally, I think this solution is optimal for enterprise integration, as is your business logic needs. For a fast and secure connection to Cloud computing, this is an excellent candidate. It has 99.9% uptime, which is great for 24/7 intensive data crunching or message exchanging.

Proceed to test #5 here!

The post Latency test between Azure and On-Premises – Part Four appeared first on Blog IT.