Introduction

An Azure SQL Server failover group is a group of databases that can be automatically or manually failed over from a primary server to a secondary server in a different Azure region in case of a disaster in the primary server. Failover groups provide high availability and disaster recovery for Azure SQL Server databases.

The process described in this post is composed by two main steps:

• Provisioning the database
• Adding the database to the failover group

Provisioning the database

The first step is to create the database. This step is composed by the following actions:

• Get the tenant information
• Create the database

Tenant information class:

Code to create database programatically:

Adding the database to the failover group

The second step adds the newly created database to the failover group. This step is composed by the following actions:

• Get the failover group where we want to add the database
• Re-add the existing databases to the failover group – necessary since when we get the failover group the list of databases of the group is empty and, without this step, the failover group would only have the new database
• Add the new database to the failover group

Other Articles

To learn why your business should migrate to SharePoint Online and Office 365, click here and here.

If you want to learn how to develop SPFx solutions, click here.

If you want to learn how you can rename a modern SharePoint site, click here.

If you want to learn how to save time time scheduling your meetings, click here.

If you want to learn how to enable Microsoft Teams Attendance List Download, click here.

If you want to learn how to create a dynamic org-wide team in Microsoft Teams with all active employees, click here.

If you want to modernize your SharePoint classic root site to a modern SharePoint site, click here.

If you are a SharePoint administrator or a SharePoint developer who wants to learn more about how to install a SharePoint 2019 farm in an automated way using PowerShell, I invite you to click here and here.

If you learn how to greatly speed up your SharePoint farm update process to ensure your SharePoint farm keeps updated and you stay one step closer to start your move to the cloud, click here.

If you prefer to use the traditional method to update your farm and want to learn all the steps and precautions necessary to successfully keep your SharePoint farm updated, click here.

If you want to learn how to upgrade a SharePoint 2013 farm to SharePoint 2019, click here and here.

If SharePoint 2019 is still not an option, you can learn more about how to install a SharePoint 2016 farm in an automated way using PowerShell, click here and here.

If you want to learn how to upgrade a SharePoint 2010 farm to SharePoint 2016, click here and here.

If you are new to SharePoint and Office 365 and want to learn all about it, take a look at these learning resources.

If you are work in a large organization who is using Office 365 or thinking to move to Office 365 and is considering between a single or multiple Office 365 tenants, I invite you to read this article.

If you want to know all about the latest SharePoint and Office 365 announcements from Ignite and some more recent announcements, including Microsoft Search, What’s New to Build a Modern Intranet with SharePoint in Office 365, Deeper Integration between Microsoft Teams and SharePoint and the latest news on SharePoint development, click here.

If your organization is still not ready to go all in to SharePoint Online and Office 365, a hybrid scenario may be the best choice. SharePoint 2019 RTM was recently announced and if you to learn all about SharePoint 2019 and all its features, click here.

Happy Coding!

The post Provision a database programmatically in Azure SQL database with a failover group appeared first on Blog IT.

The first step here is to ensure that your consumer code is idempotent, but that’s something that you should always ensure – just imagine that you have to re-consume a couple of messages, in that case lacking idempotency may result in your system getting strange data and bugs.

Setting up Kafka on your machine

(Feel free to skip this section if you have everything set up.)

Kafka releases aren’t included in most package managers, except for Homebrew in Mac systems. They should be available, however, in the official download page. For Unix systems (Mac, Linux, WSL, *BSD), simply get one of the binary downloads and extract it wherever you prefer. Then, make sure to add the bin folder to your $PATH, so that you can use these scripts in any folder. In my case, I put the extracted archive in $HOME/.local/share/kafka, so I edit my .zshrc (or .bashrc in a plainer Linux/WSL system) and add:

export PATH=$HOME/.local/share/kafka/bin:$PATH

Now we can use the official kafka scripts in any directory. That is, unless we have some form of authentication. We can add additional configs to a file somewhere, and include that file in our commands. Let’s put a file called dev.cfg in $HOME/.config/kafka/ (the exact directory doesn’t matter, but .config is the appropriate directory for configs), and write the following on it:

security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
    username="me" \
    password="noneofyourbusiness";

This is just an example, your broker’s security config may vary. After doing this, just add to any of the Kafka commands you use:

--command-config $HOME/.config/kafka/dev.cfg

Checking consumer offsets

Before migrating topics, it’s best if we make sure that the old topic consumer is totally up-to-date. We can use this command to check the consumption lag:

kafka-consumer-groups.sh \
	--bootstrap-server='<your broker endpoints, separated by commas>' \
	--describe \
	--group='<your consumer group>'

This should yield something like the output below:

TOPIC      PARTITION   NEWEST_OFFSET   OLDEST_OFFSET     CONSUMER_OFFSET     LEAD         LAG
your-topic 0           30000000        0                 30000000            30000000     0
your-topic 1           23900000        0                 23900000            23900000     0

CLIENT_HOST      CLIENT_ID         TOPIC          ASSIGNED_PARTITIONS
/128.0.0.0       your-consumer     your-topic     0
/128.0.0.1     	 your-consumer     your-topic     1

From this we can see that there is zero lag, since the newest offset is equal to the consumer offset on both partitions. We can also see that our consumers are still running (they’re the clients with ID “your-consumer”).

Performing the Kafka consumer migration

Before switching topics or brokers, we should first create a consumer group for the new configuration. This will allow us to change the offsets before beginning the message consumption proper. The easiest way to do this is to use kcat, a simple command-line tool that allows us to consume and produce Kafka messages. To create a consumer group, you must consume messages, so it’s a matter of setting up kcat (check the install guide on the kcat repo) and consuming a few messages from the new topic:

kcat \
	-C \
	-b '<your brokers>' \
	-t new-topic \
	-G new-consumer-group

# If you have authentication, just add: 

	-X security.protocol=SASL_SSL \
	-X sasl.mechanisms=SCRAM-SHA-512 \
	-X sasl.username="me" \
	-X sasl.password="noneofyourbusiness"

Just run this for a second, check that messages come out, and then run the describe command for the new group:

kafka-consumer-groups.sh \
	--bootstrap-server='new-broker' \
	--describe \
	--group='new-group'


TOPIC     PARTITION   NEWEST_OFFSET   OLDEST_OFFSET     CONSUMER_OFFSET     LEAD         LAG
new-topic 0           12387000        0                 12                  12           12386988
new-topic 1           12360000        0                 38                  38           12359962

CLIENT_HOST      CLIENT_ID         TOPIC          ASSIGNED_PARTITIONS

We have huge lag here, but that’s fine, as we’re going to reset the offsets anyway. Stop the workers consuming from the older topic by whatever means you can and note down the time at which they stopped. We will now reset the consumer group offset for this new group. Just a note: in this case, we don’t have any consumers connected to this group, but if we did we’d also have to turn them off. You cannot reset a consumer group offset if there are clients connected to it.

To reset the offsets, just run the following command (here we’ll assume that we stopped the previous consumers at 10:00 of March 15th, 2023, and subtract ten minutes just to be safe):

kafka-consumer-groups.sh \
	--bootstrap-server='new-brokers' \
	--group='new-group' \
	--topic='new-topic' \
	--reset-offsets \
	--to-datetime 2023-03-15T09:50:00.000 \
	--dry-run

Running this dry run will not make any changes, It will only show you the new offsets for the consumer group, e.g.:

TOPIC     PARTITION   NEWEST_OFFSET   OLDEST_OFFSET     CONSUMER_OFFSET     LEAD         LAG
new-topic 0           12387000        0                 12386998            12386988     2
new-topic 1           12360000        0                 12369951            12369951     49

CLIENT_HOST      CLIENT_ID         TOPIC          ASSIGNED_PARTITIONS

If nothing looks strange to you, go ahead, substitute --dry-run with --execute and run the command. Afterwards, deploy the new consumer configuration, turn on the worker, and let it run.

The post Performing broker migrations on your Kafka consumers appeared first on Blog IT.

The pipelines bring to us the opportunity to create automatic tasks to execute development operations, like deploys, migrations, tests, etc. Focused in the deploy routine, some applications need other operations to ensure the stability between the application and other resources, like the database.

During the development process, the developers usually customize and improve the application database in the development environment scope, and it can be made easily by using frameworks that contains routines to create source files that have the responsability to keep the history of changes and are used to update the database for determinated version. These files are known as migrations. 

In this post, we will see how to apply migrations for the databases that are located in different environments (staging, production, etc.), using the Code-First entity concept, and the Entity Framework Core.

Migrations

To generate database migrations, we can execute the “Add-Migraton <name>” command from the Entity Framework. This command will generate some source files that will contain the code to apply the migration to the database. To execute these migration files, you just need to execute the “Update-Database” command. But, Microsoft recommends to create a SQL file individually, and suggest to execute this file direct to the database, without need to execute an intermediary software (C# + EF) to access the database. To do this, we will have to follow the steps below:

• 1. Create a Pipeline in Azure Devops;
• 2. Install dotnet-ef-tool;
• 3. Generate SQL script;
• 4. Execute SQL script to the database;
• 5. Script File as a Pipeline Artifact.

NOTE: We don’t need to worry about which migration need to be applyed to the database. The dotnet-ef-tool command will generate a script cantaining the needed logic. Check the third step for more details.

1. Create a Pipeline in Azure Devops

We will create a common deployment pipeline in Azure Devops for ASP.NET Core backend applications. Check the end of the post to see the full result of the pipeline.

2. Install dotnet-ef-tool

To execute tool commands of Entity Framework, we have to install a package named dotnet-ef-tool. This needs to be available into the pipeline’s context to be used for the SQL script generation. To install this tool by a pipeline task, we can use the task below:

- task: DotNetCoreCLI@2
  displayName: 'Install EF tool'
  inputs:
    command: 'custom'
    custom: 'tool'
    arguments: 'install --global dotnet-ef'

3. Generate SQL script

To generate the SQL script containing the logic to apply migrations, we will use the dotnet-ef-tool. By default, the command does not generate the logic to check which migration should be applied, so we have to enable this behavior using the parameter: –idempotent. By using this behavior, we don’t need to worry about the migrations that will be applyed to the database since the generated script already has it. Below, you can see the task that could be used to generate the script file:

- task: DotNetCoreCLI@2
  displayName: 'Generate SQL Script'
  inputs:
    command: 'custom'
    custom: 'ef'
    arguments: 'migrations script --idempotent --project $(Build.SourcesDirectory)\Data.csproj --output $(System.ArtifactsDirectory)/script.sql'

NOTE: We don’t need to stablish any database connection for the script generation, since we are following the Code First concept for entity creation, so, the command will just look at the entities source code of the project to generate the SQL file. If we don’t use the –idempotent argument, the script will generate the SQL without the conditional logic to determinate which migration should be applied, causing an error if the database already exists.

4. Execute SQL script to the database

Now, with the SQL script available, we just need to execute this file to the database. In this post, we will use a common Azure Database as an example, so for this case, we can use the following task to accomplish this requirement:

- task: SqlAzureDacpacDeployment@1
  displayName: 'Update Database'
  inputs:
    azureSubscription: '<Service Connection Identifier>'
    AuthenticationType: 'connectionString' # You can use other method to authenticate to the database if you want
    ConnectionString: '<Database Connection String>'
    deployType: 'SqlTask'
    SqlFile: '$(System.ArtifactsDirectory)/script.sql'

5. Script File as a Pipeline Artifact

Well done. The last task will only be used to make the file available to be accessed by the pipeline runner user. To do this, you can use the task below to add the generated script.sql file to the file bundle (artifacts). To check the artifacts, just access the artifacts option from the pipeline details after the job’s execution.

- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifacts'
  inputs:
    PathtoPublish: '$(System.ArtifactsDirectory)/script.sql'
    ArtifactName: '$(artifactName)'
    publishLocation: 'Container'

Conclusion

In this post, we could learn an easy way to apply the database migrations to the remote environment by using automation (pipelines) to accomplish it. This method is simple, and does not have any database validation before script execution. So, if you have an active and sensitive environment, be sure that the needed validations are considered before the database update, to ensure that the database will not be update wrongly.

Pipeline Result

trigger:
- none

pool:
  vmImage: windows-latest

variables:
  solution: '**/*.sln'
  buildPlatform: 'AnyCPU'
  buildConfiguration: 'Release'
  artifactName: 'artifacts'
  environment: 'Development'

- task: NuGetToolInstaller@1
  displayName: 'NuGet Installation'

- task: NuGetCommand@2
  displayName: 'NuGet Restore'
  inputs:
    restoreSolution: '$(solution)'

- task: VSBuild@1
  displayName: 'Build Project'
  inputs:
    solution: 'backend/WebAPI'
    msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)"'
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

- task: PublishSymbols@2
  displayName: 'Publish Symbols'
  inputs:
    SearchPattern: '**/bin/**/*.pdb'
    SymbolServerType: 'FileShare'
    CompressSymbols: false

- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifacts'
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)'
    ArtifactName: 'artifacts'
    publishLocation: 'Container'

- task: DownloadBuildArtifacts@0
  displayName: 'Build Artifacts'
  inputs:
    buildType: 'current'
    downloadType: 'single'
    artifactName: '$(artifactName)'
    downloadPath: '$(System.ArtifactsDirectory)'

- task: DotNetCoreCLI@2
  displayName: 'Install EF tool'
  inputs:
    command: 'custom'
    custom: 'tool'
    arguments: 'install --global dotnet-ef'

- task: DotNetCoreCLI@2
  displayName: 'Generate SQL Script'
  inputs:
    command: 'custom'
    custom: 'ef'
    arguments: 'migrations script --idempotent --project $(Build.SourcesDirectory)\Data.csproj --output $(System.ArtifactsDirectory)/script.sql'

- task: SqlAzureDacpacDeployment@1
  displayName: 'Update Database'
  inputs:
    azureSubscription: '<Service Connection Identifier>'
    AuthenticationType: 'connectionString'
    ConnectionString: '<Connection String>'
    deployType: 'SqlTask'
    SqlFile: '$(System.ArtifactsDirectory)/script.sql'

- task: AzureRmWebAppDeployment@4
  displayName: 'Deploy Application'
  inputs:
    ConnectionType: 'AzureRM'
    azureSubscription: '<Service Connection Identifier>'
    appType: 'webApp'
    WebAppName: 'armazemlegalapi'
    deployToSlotOrASE: true
    ResourceGroupName: '<Resource Group Identifier>'
    SlotName: 'production'
    packageForLinux: '$(System.ArtifactsDirectory)/Project.zip'

- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifacts'
  inputs:
    PathtoPublish: '$(System.ArtifactsDirectory)/script.sql'
    ArtifactName: '$(artifactName)'
    publishLocation: 'Container'

References

• Microsoft Managing Schemas Article

The post Entity Framework – Update Database using Azure Pipelines appeared first on Blog IT.

This post compares different strategies to dockerize a .NET 6 application and how to create a < 100mb docker image to host a .NET 6 asp.net web application. Using the docker multi stage builds feature and a self-contained .NET with some build options the final docker image can be reduced from 760mb to 83mb, almost 10x smaller!

All the images used were taken from the official Microsoft .NET docker repository.

1. .NET 6 SDK image

Let’s start with a simple Dockerfile based on the official .NET 6 SDK build image.

FROM mcr.microsoft.com/dotnet/sdk:6.0
WORKDIR /app

COPY . ./

RUN dotnet publish "WebApi.csproj" -c Release -o /app/publish

ENTRYPOINT ["dotnet", "/app/publish/WebApi.dll"]

$ docker build -t api-aspnet .
$ docker images api-aspnet
REPOSITORY   TAG       IMAGE ID       CREATED       SIZE
api-aspnet   latest    700870327c08   1 weeks ago   761MB

We are using a docker single stage build using the base .net sdk image. After building the image we can see that the final image size is 761mb. Let’s try to check where the space is being used. For that we can use the docker history to see the layers created for this image.

$ docker history api-aspnet
IMAGE          CREATED        CREATED BY                                      SIZE      COMMENT
700870327c08   1 weeks ago    ENTRYPOINT ["dotnet" "/app/publish/WebApi.dl…   0B        buildkit.dockerfile.v0
<missing>      1 weeks ago    RUN /bin/sh -c dotnet publish "WebApi.csproj…   4.18MB    buildkit.dockerfile.v0
<missing>      1 weeks ago    RUN /bin/sh -c dotnet build -c Release # bui…   12.2MB    buildkit.dockerfile.v0
<missing>      1 weeks ago    COPY . ./ # buildkit                            721kB     buildkit.dockerfile.v0
<missing>      1 weeks ago    RUN /bin/sh -c dotnet restore # buildkit        28.2MB    buildkit.dockerfile.v0
<missing>      1 weeks ago    COPY WebApi.csproj ./ # buildkit                327B      buildkit.dockerfile.v0
<missing>      1 weeks ago    WORKDIR /app                                    0B        buildkit.dockerfile.v0
<missing>      2 months ago   /bin/sh -c powershell_version=7.2.1     && c…   40.6MB
<missing>      2 months ago   /bin/sh -c curl -fSL --output dotnet.tar.gz …   392MB
<missing>      2 months ago   /bin/sh -c apt-get update     && apt-get ins…   74.8MB
<missing>      2 months ago   /bin/sh -c #(nop)  ENV ASPNETCORE_URLS= DOTN…   0B
<missing>      2 months ago   /bin/sh -c #(nop) COPY dir:a54b266469a09b122…   20.3MB
<missing>      2 months ago   /bin/sh -c #(nop)  ENV ASPNET_VERSION=6.0.1 …   0B
<missing>      2 months ago   /bin/sh -c ln -s /usr/share/dotnet/dotnet /u…   24B
<missing>      2 months ago   /bin/sh -c #(nop) COPY dir:6c537cc098876a5f6…   70.6MB
<missing>      2 months ago   /bin/sh -c #(nop)  ENV DOTNET_VERSION=6.0.1     0B
<missing>      2 months ago   /bin/sh -c #(nop)  ENV ASPNETCORE_URLS=http:…   0B
<missing>      2 months ago   /bin/sh -c apt-get update     && apt-get ins…   37MB
<missing>      2 months ago   /bin/sh -c #(nop)  CMD ["bash"]                 0B
<missing>      2 months ago   /bin/sh -c #(nop) ADD file:09675d11695f65c55…   80.4MB

The .NET SDK + runtime takes most of the space, around 60%. So let’s try to shrink it.

2. .NET 6 SDK image + docker multi stage build

The previous section showed us that the .NET SDK + runtime takes most of the space of the final image. This is because we are using the same docker base image to build and to run the app so the final image has all the things needed to compile and run the app. Using a docker multi stage build we can cut the .NET SDK from the final image and use it only to build our app and include only the .NET runtime in the final image to run the application.

FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /app

COPY . ./

RUN dotnet publish "WebApi.csproj" -c Release -o /app/publish

FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS runtime

WORKDIR /app
COPY --from=build /app/publish .

ENTRYPOINT ["dotnet", "WebApi.dll"]

Here we define 2 images: the build image used to compile the app which includes all the things needed to build our .NET 6 application and the final one, the runtime  which only includes the .NET runtime to run our app. The first one to use for development (which contained everything needed to build your application), and a slimmed-down one to use for production, which only contained your compiled code and what is needed to run it. With multi-stage builds, you use multiple FROM statements in your Dockerfile. Each FROM instruction can use a different base, and each of them begins a new stage of the build. You can selectively copy artifacts from one stage to another, leaving behind everything you don’t want in the final image.

$ docker build -t api-aspnet-multistage .
$ docker images api-aspnet-multistage
REPOSITORY              TAG       IMAGE ID       CREATED       SIZE
api-aspnet-multistage   latest    1021a94feca4   1 weeks ago   212MB

Bang, we reduced our image to 212mb, this is because the complete SDK was removed from the final image.
Let’s see again the layers and the space being used for each one:

USER@DESKTOP-BJUBV5T MINGW64 ~/Documents/sonae/identitymanager-credenciaiscontinente (fix/invalid-account-association-check)
$ docker history api-aspnet-multistage
IMAGE          CREATED        CREATED BY                                      SIZE      COMMENT
1021a94feca4   1 weeks ago    ENTRYPOINT ["dotnet" "WebApi.dll"]              0B        buildkit.dockerfile.v0
<missing>      1 weeks ago    COPY /app/publish . # buildkit                  4.18MB    buildkit.dockerfile.v0
<missing>      1 weeks ago    WORKDIR /app                                    0B        buildkit.dockerfile.v0
<missing>      2 months ago   /bin/sh -c #(nop) COPY dir:a54b266469a09b122…   20.3MB
<missing>      2 months ago   /bin/sh -c #(nop)  ENV ASPNET_VERSION=6.0.1 …   0B
<missing>      2 months ago   /bin/sh -c ln -s /usr/share/dotnet/dotnet /u…   24B
<missing>      2 months ago   /bin/sh -c #(nop) COPY dir:6c537cc098876a5f6…   70.6MB
<missing>      2 months ago   /bin/sh -c #(nop)  ENV DOTNET_VERSION=6.0.1     0B
<missing>      2 months ago   /bin/sh -c #(nop)  ENV ASPNETCORE_URLS=http:…   0B
<missing>      2 months ago   /bin/sh -c apt-get update     && apt-get ins…   37MB
<missing>      2 months ago   /bin/sh -c #(nop)  CMD ["bash"]                 0B
<missing>      2 months ago   /bin/sh -c #(nop) ADD file:09675d11695f65c55…   80.4MB

Let´s try to reduce the bigger part, the base image.

3. Multi stage build with Alpine linux Microsoft official image

We can change our base image to use the Alpine Linux instead of the default one Debian bullsyeye which is the base image of the most of the official Microsoft .NET docker images.

FROM mcr.microsoft.com/dotnet/sdk:6.0-alpine AS build
WORKDIR /app

COPY . ./
RUN dotnet publish "WebApi.csproj" -c Release -o /app/publish

FROM mcr.microsoft.com/dotnet/aspnet:6.0-alpine

WORKDIR /app
COPY --from=build /app/publish .
ENTRYPOINT ["dotnet", "WebApi.dll"]

docker build -t api-multi-alpine .
docker images api-multi-alpine
REPOSITORY         TAG       IMAGE ID       CREATED       SIZE
api-multi-alpine   latest    0c7f1c3b0bfa   1 weeks ago   104MB

$ docker history api-multi-alpine
IMAGE          CREATED        CREATED BY                                      SIZE      COMMENT
0c7f1c3b0bfa   1 weeks ago    ENTRYPOINT ["dotnet" "WebApi.dll"]              0B        buildkit.dockerfile.v0
<missing>      1 weeks ago    COPY /app/publish . # buildkit                  4.12MB    buildkit.dockerfile.v0
<missing>      1 weeks ago    WORKDIR /app                                    0B        buildkit.dockerfile.v0
<missing>      2 months ago   /bin/sh -c wget -O aspnetcore.tar.gz https:/…   20.3MB
<missing>      2 months ago   /bin/sh -c #(nop)  ENV ASPNET_VERSION=6.0.1 …   0B
<missing>      2 months ago   /bin/sh -c wget -O dotnet.tar.gz https://dot…   69.8MB
<missing>      2 months ago   /bin/sh -c #(nop)  ENV DOTNET_VERSION=6.0.1     0B
<missing>      3 months ago   /bin/sh -c #(nop)  ENV ASPNETCORE_URLS=http:…   0B
<missing>      3 months ago   /bin/sh -c apk add --no-cache         ca-cer…   4.35MB
<missing>      3 months ago   /bin/sh -c #(nop)  CMD ["/bin/sh"]              0B
<missing>      3 months ago   /bin/sh -c #(nop) ADD file:762c899ec0505d1a3…   5.61MB

Using the alpine instead of the debian base image we were able to reduce our base image size to 10% of our final image.
Now the .NET runtime is the part that is using most of the space of our image so let’s try to reduce this part.

4. Raw alpine with a trim-self-contained and trimmed .NET build

Instead of using the official Microsoft Alpine image which comes with the full .NET runtime we can use the vanilla Alpine base image and build our App using the .NET self-contained build PublishSingleFile option.
Publishing our app as self-contained produces an application that includes the .NET runtime and libraries in a single binary, so we don’t have to use the .NET runtime installed on our base system.

We can also use the PublishTrimmed option while building and publishing our app. With this options the final binary includes only a subset of the .NET framework assemblies, the assemblies that are referenced by our app. The other ones are removed from the final procuded binary file. However, there is a risk that the build-time analysis of the application can cause failures at run time, due to not being able to reliably analyze various problematic code patterns (largely centered on reflection use). To mitigate these problems, warnings are produced whenever the trimmer cannot fully analyze a code pattern. For information on what the trim warnings mean and how to resolve them, see Introduction to trim warnings.

Also, with a self-contained build we also need to specify the target platform which the final binary will be compiled. In this case we are using the alpine-x64 because our base image is the Alpine Linux.

FROM mcr.microsoft.com/dotnet/sdk:6.0-alpine AS build
WORKDIR /app

COPY . ./

RUN dotnet publish "WebApi.csproj" -c Release -o /app/publish \
    --runtime alpine-x64 \
    --self-contained true \
    /p:PublishTrimmed=true \
    /p:TrimMode=Link \
    /p:PublishSingleFile=true

FROM mcr.microsoft.com/dotnet/aspnet:6.0-alpine

WORKDIR /app
COPY --from=build /app/publish .
ENTRYPOINT ["./WebApi"]

docker build -t api-multi-alpine-raw .
docker images api-multi-alpine-raw

$ docker history api-multi-alpine-raw
IMAGE          CREATED        CREATED BY                                      SIZE      COMMENT
ac78719fc09a   1 weeks ago    ENTRYPOINT ["./WebApi"]                         0B        buildkit.dockerfile.v0
<missing>      1 weeks ago    COPY /app/publish . # buildkit                  42.2MB    buildkit.dockerfile.v0
<missing>      1 weeks ago    WORKDIR /app                                    0B        buildkit.dockerfile.v0
<missing>      1 weeks ago    RUN /bin/sh -c apk add --no-cache libstdc++ …   35.5MB    buildkit.dockerfile.v0
<missing>      3 months ago   /bin/sh -c #(nop)  CMD ["/bin/sh"]              0B
<missing>      3 months ago   /bin/sh -c #(nop) ADD file:8f5bc5ce64ef781ad…   5.59MB

The final image was reduced to 83mb because it only includes the Alpine base image and a single binary that includes our WebApp compiled code and the assemblies that are referenced by our app!

References

https://docs.docker.com/develop/develop-images/multistage-build/
https://docs.microsoft.com/en-us/dotnet/core/deploying/
https://docs.microsoft.com/en-us/dotnet/core/deploying/trimming/trim-self-contained

https://hub.docker.com/_/microsoft-dotnet

The post Smaller .NET 6 docker images appeared first on Blog IT.

In this article we will talk about Event-Driven Architectures. We choose to use the Azure Cloud Infrastructure.
Service Bus provides reliable, secure asynchronous messaging at scale. This article is written by the engineering team at CreateIT and it is intended to show you a case study in one of our projects for a client.

We’ll take a deeper dive into the Service Bus technology, architecture, and design choices. The post will cover both conceptual material as well as implementation details. Most importantly, we will discuss design and implementation of some of the features that provide secure and reliable messaging at scale, while minimizing operational cost.

Service Bus Entities

When we are working with Azure Service Bus, we can choose two Entities: Topics or Queues. You can have multiple Topics or Queues per Service Bus Namespace, but firstly you need to differ one from another. If you want a FIFO queue and only have one Consumer, then Queues are the way to go. If you need multiple Consumers, then the Topic is the better option. In this specific case we will create a Subscription per Consumer (Topics are only available from the Standard Pricing Tier).

Event-driven architectures

Benefits with event-driven architectures

What are the benefits of using a queue in the middle of these systems?

• We can decide to load balance the input from Customer Services. Let’s say there are a lot of updates being made to a customer, meaning a lot of events being published. We scale the number of consumers and user the competing pattern
• We can throttle the input. If on black Friday there are a ton of events and in case our Audit Log system is down. We simply store these events on the queue and consume them when the service is back online again. Of course we’d need to implement some logic for this behavior, but adding this “middleware” buys us options.

In our use case, we wanted to move to an implementation where the Web API doesn’t get affected by any changes on these external systems. But in order to change the implementation, we must first figure out what are the challenges associated with this change.

Challenges with event-driven architectures

Message/Event order

Azure Service Bus has a feature called sessions. A session provides a context to send and retrieve messages that will preserve ordered delivery. However, in our use case we chose not to use it.

Message Lock Duration

When we are using Queues, every message has a lock Duration. During this time the consumer needs to process it. But if this consumer needs to contact multiple external systems this time may rise and our messages could stop in the Dead-Letter. So the best practice is to change it accordingly to your needs. We recommend you to set this time extremely high in the beginning, and then do some tests to calculate the average.

After that add 30% more of its value, in case of some lengthy  requests (in this case if we have outliers, they might stop in the Queue). If you are using a Topic, you will have a Lock Duration per Subscription. So make sure to adjust this time accordingly to its functions.

Implementation

In the Figure 1 you can see the initial architecture for the Customer Management system. It was responsible to make the requests to the other systems.

With the new implementation, a message broker was introduced and we used the event-carried state transfer pattern, meaning our events had all the information the consumer needed in order to do their job. We took in consideration the event notifications pattern, where the consumer would have to make a request to the API that originated the event, in order to get more information. But this brings new problems to the table. What if when the consumer code runs, the information for that customer ID changed? What if the event was CUSTOMER_CREATED but in the meanwhile the customer was deleted?

Retries with Polly

In a distributed system, many things can go wrong. The network can fail or have additional latency, systems may be temporarily down, etc. We use the Azure.ServiceBus.Messaging NuGet package so we are able to check if the exception is a transient fault or not (more information on these docs), then use Polly to setup retry logic and fallbacks. There are other options to implement retry policies, for example we took in consideration the Retry guidance for Azure Services documentation from Microsoft. Since we use the latest Azure SDK,  the appropriate class would be ServiceBusRetryPolicy.
We configured Polly to retry to publish a message three times (this configuration is on appsettings.json), with exponential times between each attempt.
If after the third retry we can’t publish the message we need to save it, because it has crucial information. So to solve this issue we created a Fallback Gateway, to write these messages to a Container inside an Azure Storage Account.

Filters for message routing

This section only applies for Topics Entities on the Azure Service Bus.
We can add Filters on our Subscriptions to help us with routing each message to its specific Consumer. We considered two filter types:

• SQL Filter
• Correlation Filter

Using the Correlation Filter you can configure Custom Properties and create Filters for your needs. You just need to make sure the producer of the messages, includes the header, that you are currently using to filter, on the message.

With SQL Filters you can create conditional expression to evaluate the current message. Just make sure that all the system properties are prefixed with sys. in the expression. Either way, both filters work just choose one that suits you the most!

Dead-Letter Queue

In case the consumer application can’t process the message after the Max Delivery Count attempts, instead of returning it to the queue it will be sent automatically to the Dead-Letter queue. If you are using the Topic, each subscriber has its own Dead-Letter queue. You can also, configure different Max Delivery Counts Values for each Subscriber.

All messages that are published to the Service Bus have a TTL (Time-To-Live). After this time ends the message will be transferred automatically to the Dead-Letter. So make sure you adjust this time accordingly to your needs.

With this we are able to save messages that weren’t processed by the consumer application, but we should always strive to have an empty dead-letter queue.

Conclusion

Our first steps into an Event-Driven Architecture was a truly success!
We were able to expand our previous solution to be compatible with multiple external systems and instead of having the API sending a HTTPS request for each one we had this Application sending one message to a Topic in the Service Bus.
One of our goals was to have load balance in the Publisher Application. We went from a 1->3 dependency to a 1-> 1, as you can see in the Figure 2.

Which is great and keeps the system scalable and future proof. Our solution became more decoupled in order to keep the Application agnostic to these changes.
If you have a similar situation with an Event-Driven Architecture then we totally recommend you to check more about this technology and it’s features.

We would like to share a link to Microsoft Azure Service Bus GitHub. Most of the implementations of either the publisher or the subscriber were inspired by this documentation, so make sure you check it out!
If you have any questions, please write them down below.

Additional Links

Service Bus Exceptions
Service Bus Basic Steps
Tutorial with DotNet

The post Case Study: Azure Service Bus and Event-Driven Architectures appeared first on Blog IT.

Introduction

In AWS the main ways that we have to send an email are:

• Using Amazon Simple Email Service (SES)
• Using Amazon Simple Notification Service (SNS)

For this example we are going to use Amazon Simple Notification Service.

You can check the main differences between SES and SNS here

The components

Now i’m going to explain the components that we are going to use.

Amazon Simple Queue Service (SQS)

Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications. SQS eliminates the complexity and overhead associated with managing and operating message oriented middleware, and empowers developers to focus on differentiating work. Using SQS, you can send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be available.

More Info: SQS

Amazon Simple Notification Service (SNS)

Amazon Simple Notification Service (SNS) is a highly available, durable, secure, fully managed pub/sub messaging service that enables you to decouple microservices, distributed systems, and serverless applications. Amazon SNS provides topics for high-throughput, push-based, many-to-many messaging. Using Amazon SNS topics, your publisher systems can fan out messages to a large number of subscriber endpoints for parallel processing, including Amazon SQS queues, AWS Lambda functions, and HTTP/S webhooks. Additionally, SNS can be used to fan out notifications to end users using mobile push, SMS, and email.

More Info: SNS

AWS Lambda

AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume – there is no charge when your code is not running.

With Lambda, you can run code for virtually any type of application or backend service – all with zero administration. Just upload your code and Lambda takes care of everything required to run and scale your code with high availability. You can set up your code to automatically trigger from other AWS services or call it directly from any web or mobile app.

More Info: Lambda

Objective

We are going to build a system using a SQS queue where we are going to send the body of the email. The lambda that long pools the queue will be triggered when the message is sent to the queue and will publish the message to the SNS topic so we can send an email with the message body.

More Info: Long Pooling

Lets Go

Create a SQS Queue where we are going to send the email body message. This queue will be a Standard Queue. This example will not have a deadletter queue configurated, but is always good to have one.

We need to create a SNS Topic. An Amazon SNS topic is a logical access point which acts as a communication channel.

Create a subscription to that topic. The subscription resource subscribes an endpoint to an Amazon Simple Notification Service (Amazon SNS) topic. For a subscription to be created, the owner of the endpoint must confirm the subscription. The protocol of the subscription will be Email. The endpoint will be the email address that will receive the email. For this example i used a temporary mail so that i didn’t fill my personal inbox with emails.

More Info: Temp-Mail

After you configure the subscription you will receive an email with a link to confirm the subscription.

When you click on the link you will be redirected to the subscription confirmation page

Create the lambda that will use long pooling to check if there are new messages in the queue and publish the messages to the SNS Topic.

In this example i used Python to develop the lambda and called it “EmailLambda”. Note: you need to set the role of the lambda so that the lambda has permissions to long pool the sqs queue and publish a message to the SNS topic.

More Info: SNS IAM Roles SQS IAM Roles

Here is the lambda code that i used.

#!/usr/bin/python3
import json
import boto3
import os

def lambda_handler(event, context):
    batch_processes=[]
    for record in event['Records']:
        send_request(record["body"])
		

def send_request(body):
    # Create an SNS client
    sns = boto3.client('sns')

    # Publish a simple message to the specified SNS topic
    response = sns.publish(
        TopicArn=os.environ['email_topic'],    
        Message=body,    
    )

    # Print out the response
    print(response)
 

Set the lambda enviroment variable email_topic to the ARN of the SNS Topic we created earlier.

Add the lambda trigger and set it with the sqs queue that will receive the body of the email.

Lets Test it !!!

Now the only thing that we need to do is send a new message to the SQS queue with the email body that we want.

Now we will receive an email from no-reply@sns.amazonaws.com with the body of the queue message.

And there you have it. An email notification in AWS using SQS, SNS and Lambda.

Thanks for reading this post and have an…

The post Send Email In AWS Using SQS, SNS and Lambda Function appeared first on Blog IT.

NoSQL is like a new definition of something that is a database but is different than the usual relational model. Likewise, making a parallel by going back 30 years ago. Back then probably no one knew what a relational database was also. Let’s use this and start with a small historical and motivational point of view for NoSQL. PS: I was not a developer back then.

This article will be a first part of a series of articles. The goal will be to unite some knowledge about this topic.

Starting in the 1980’s

Relational databases emerged bringing ACID properties – Atomicity, Consistency, Isolation and Persistency. Properties taken for granted nowadays. Also, they also brought the SQL language. SQL is common enough across different systems for one to use. Although there are different flavors, can almost be considered a standard.

Relation databases also allowed for a simple and very common integration mechanism between 2 or more systems. Data can be easily shared by reading or writing data in a table on a shared relational database. This is today still a very common integration pattern.

In the 1990’s

Object database’s started to appear with more strength. They had been around for some time. Most important, they implement a different database paradigm model. This new paradigm tried to solve the impedance mismatch problem. The impedance mismatch was caused by relational databases. It emerges from the need to map objects used in memory in our application model to tables in a relational database. This page https://en.wikipedia.org/wiki/Object-relational_impedance_mismatch is a good reference for this issue.

Saving an application object into these database model should be a direct operation. Conceptually no mapping would be necessary when an object database type was used. Albeit, these gains, object databases were not able to substitute relational databases.

In the 2000’s

As internet availability grows 2 things started to have a big impact:

• Generation of enormous amount of data that had to be stored and processed;
• Accesses from anywhere in the globe became more and more frequent. Therefore latency was an issue to consider. And even speed of light limitation can contribute significantly to this. Data can now be stored far from where we are. For example, a distance of 10.000 km will add about 100 ms on each round trip. Therefore, data needs to spread around the globe to provide a quick access.

In this environment, relational databases do not always solve customer needs, either because of flexibility, price or performance issues:

• Relational databases enforces a ridged set of rules in the relational model. These will impact the flexibility of the application development;
• Price because of the need for more powerful machines and also the software licenses that go with this;
• Performance, because relational databases do not naturally meet the increasing workload. Huge CPU, memory, storage and throughput is needed to be perform in this environment. And for relational model, vertical scaling can only get you so far.

Scaling Vertically versus Horizontally

Relational databases usual scaling method is a vertical one. As a result whenever the workload incresses we will use a bigger machine with more hardware resources.

On the other hand, the big internet companies adopted an horizontal scaling paradigm. This is a cluster type environment with lots and lots of machines. In other words, this means that more machines will be used to handle bigger workloads.

Relational databases do not thrive very well in this paradigm. They hardly take any benefit of using more machines. Hence, NoSQL like databases arises to take advantage of the horizontal scale paradigm. Also companies like Google and Amazon started researching in this area. As a result, Google created BigTable and Amazon DynamoDB.

Relational databases dominated the market, why are NoSQL databases being used now?

I think this is important. Dominance and usage factors are usually a combination of several aspects. What has been changing:

• Developers hide databases behind integration layers. This makes it simpler to use. And in addition, easier to replace one database or persistence method per other;
• Cloud growth in some cases means we can try different approaches with less effort regarding infrastructure;
• Handling big quantities of data introduced new needs. NoSQL provides a good development flexibility. And in addition can also take advantage of cluster solutions.

Finally, what is NoSQL?

Common NoSQL databases characteristics

NoSQL doesn’t have a clear direct meaning. The term alludes to something like “Not Only SQL”. A more exact explanation should be “Non Relational database”. This would reinforce the new model paradigm and not the SQL language itself. The fact that some NoSQL databases actually supports some form of SQL language adds even more to the confusion.

But, the name is catchy and so common that will probably stay. The point is: defining NoSQL is hard. Therefore, we will do the next best thing. Introduce the dominant traits of these database systems:

• Non relational;
• Usually (not all) cluster-friendly;
• Most of them are open source;
• No-schema / schema-less;
• Have a big internet drive (cluster and bigdata friendly).

By these common traits, almost any non relational database can be a NoSQL database! Also, there are probably published studies in all these areas dating back 30 or 40 year ago. Out of curiosity, the name NoSQL itself is referred to being born by the end of 2009. The term emerged as a twitter hashtag for a meetup were people talked about these subjects. Even though some of these traits were not new at the time.

Next steps

NoSQL has a lot to explore. It will provide a more direct paradigm that will make simpler specific needs like this ones to query Json objects inside Sql Server and performance considerations.

My plan for the next article(s) will be to:

• Present some of the NoSQL data models;
• Talk about the no schema or the schema less feature;
• Discuss the aggregate concept. And how this relates to the CAP theorem.

Further reads

There is a lot of information online. For me, I like Martin Fowler approach. Some of these information’s are from his content. You can check his website here: https://martinfowler.com/nosql.html. He also has a book: https://martinfowler.com/books/nosql.html.

The post NoSQL First Act – a historical introduction appeared first on Blog IT.

WebPagetest is a tool that was originally developed by AOL for use internally and was open-sourced in 2008 under a BSD license. The online version at www.webpagetest.org is run for the benefit of the performance community with several companies and individuals providing the testing infrastructure around the globe.

This tool tests any website against 6 major performance affecting factors, and provides a myriad of graphs and logs that make abundantly clear what might be slowing down your site.

In this post I’ll provide ways to make your site get straight A’s in WebPagetest.

Setup

To start this off, let’s setup our environment. We’ll just need the following:

• Visual Studio 2017
• Microsoft Azure account

In Visual Studio, let’s create a new empty ASP.NET Web Application project. Then, we’ll need the latest and greatest Umbraco NuGet package (I used version 7.12.4). Once it finishes installing, just launch the website and install Umbraco with all defaults. This will bootstrap Umbraco with the Starter Website, which we’ll use as our “guinea pig” for WebPagetest.

Next: publish it! We can use the publish wizard to automatically create our new WebApp and SQL Database in Azure. Before installing Umbraco in Azure, we’ll need to change the Web.config so that the install wizard is run again (I use Filezilla to change it in Azure):

Clear the Umbraco version number:

<add key="umbracoConfigurationStatus" value="">

Clear the Umbraco connection string:

<add name="umbracoDbDSN" connectionstring="" providername="System.Data.SqlClient">

With these changes in place, we’re good to go. This time, we’ll not use the defaults in the Umbraco install wizard, since we’ll want to use the SQL Database we’ve just created in Azure.

First test

For our tests, we’ll use the people page of the Starter Website. This is the score I got with a standard (S0) database and a basic WebApp:

Since this is a very small site, only used for demonstration purposes, half of the metrics are already cleared! However, this is not usually the case for bigger websites. For this reason, I’ll still present some solutions to improve the grade for these metrics.

First Byte Time

These test measures the time it takes for the first byte to reach the client’s browser after the initial http request.

There are two main factors that influence this result:

• Server power
• The webpage complexity (integrations with external services, complex logic involved, etc)

How to get an A

The easiest way to mitigate this problem is by caching. You can see how to do output caching in Umbraco by reading this old post of mine: https://blogit.create.pt////andresantos/2016/06/30/umbraco-and-donut-output-cache/.

Keep-alive Enabled

Keep alive is a method to allow the same tcp connection for HTTP conversation instead of opening a new one with each new request.

How to get an A

This setting is active by default in IIS, so, it’s also active by default in Azure WebApps, so it’s easy to get an A!

Compress Transfer

Gzip compresses your webpages, style sheets and javascripts, before sending them over to the browser. This drastically reduces transfer time since the files are much smaller.

How to get an A

Just add this setting to your Web.config file:

<httpcompression dynamiccompressionenablecpuusage="0" dynamiccompressiondisablecpuusage="90" nocompressionforhttp10="false" nocompressionforproxies="false">
    <statictypes>
        <add mimetype="text/*" enabled="true">
        <add mimetype="message/*" enabled="true">
        <add mimetype="application/javascript" enabled="true">
        <add mimetype="application/font-woff" enabled="true">
        <add mimetype="application/font-woff2" enabled="true">
        <add mimetype="application/vnd.ms-fontobject" enabled="true">
        <add mimetype="application/octet-stream" enabled="true">
        <add mimetype="*/*" enabled="false">
    </add></add></add></add></add></add></add></add></statictypes>
    <dynamictypes>
        <add mimetype="text/*" enabled="true">
        <add mimetype="message/*" enabled="true">
        <add mimetype="application/javascript" enabled="true">
        <add mimetype="*/*" enabled="false">
    </add></add></add></add></dynamictypes>
</httpcompression>

Compress Images

Image compression is minimizing the size in bytes of a graphics file without degrading the quality of the image to an unacceptable level.

How to get an A

In Umbraco, to get an A in this grade, you need to do two things:

1. Crop every image and use srcsets where you can
2. Use the PostProcessor plugin for Image Processor

Cropping an image is easy in Umbraco:

<div class="employee-grid__item__image" style="background-image: url('@person.Photo.GetCropUrl(width: 323, height: 300, quality: 85)')"></div>

In order to use the PostProcessor plugin, you just need to install it via nuget: https://www.nuget.org/packages/ImageProcessor.Web.PostProcessor/1.3.1.25.

Cache Static Content

Static content is content that changes rarely. For this reason it can be cached in the user’s browser to avoid downloading the same file over and over again.

How to get an A

Just set the time it takes for the content to expire in the user’s browser and add extra mime types if you want:

<staticcontent>
    <clientcache cachecontrolmode="UseMaxAge" cachecontrolmaxage="7.24:00:00">
    <remove fileextension=".air">
    <mimemap fileextension=".air" mimetype="application/vnd.adobe.air-application-installer-package+zip">
    <remove fileextension=".svg">
    <mimemap fileextension=".svg" mimetype="image/svg+xml">
    <remove fileextension=".woff">
    <mimemap fileextension=".woff" mimetype="application/x-font-woff">
    <remove fileextension=".woff2">
    <mimemap fileextension=".woff2" mimetype="application/x-font-woff2">
    <remove fileextension=".less">
    <mimemap fileextension=".less" mimetype="text/css">
    <remove fileextension=".mp4">
    <mimemap fileextension=".mp4" mimetype="video/mp4">
    <remove fileextension=".json">
    <mimemap fileextension=".json" mimetype="application/json">
</mimemap></remove></mimemap></remove></mimemap></remove></mimemap></remove></mimemap></remove></mimemap></remove></mimemap></remove></clientcache></staticcontent>

Effective use of CDN

A content delivery network (CDN) refers to a geographically distributed group of servers which work together to provide fast delivery of Internet content. A CDN allows for the quick transfer of assets needed for loading Internet content including HTML pages, javascript files, stylesheets, images, and videos.

How to get an A

In Umbraco, you can achieve this last grade by doing two things:

1. Use an Azure Blob Storage for media storage by installing this nuget package: https://github.com/JimBobSquarePants/UmbracoFileSystemProviders.Azure.
2. Create an Azure CDN for serving these blobs through a content delivery network.

After creating an Azure CDN service and waiting about an hour for it to be available, the following presents my configuration for the media assets to be provided by it:

<!--?xml version="1.0" encoding="utf-8"?-->
<security>
  <services>
    <service name="LocalFileImageService" type="ImageProcessor.Web.Services.LocalFileImageService, ImageProcessor.Web">
    <!--Disable the LocalFileImageService and enable this one when using virtual paths. -->
    <service prefix="media/" name="CloudImageService" type="ImageProcessor.Web.Services.CloudImageService, ImageProcessor.Web">
      <settings>
        <setting key="Container" value="media">
        <setting key="MaxBytes" value="8194304">
        <setting key="Timeout" value="30000">
        <setting key="Host" value="https://<umbracositename>.blob.core.windows.net/media">
      </setting></setting></setting></setting></settings>
    </service>
  </service></services>
</security>

<!--?xml version="1.0" encoding="utf-8"?-->
<caching currentcache="AzureBlobCache">
  <caches>
    <cache name="AzureBlobCache" type="ImageProcessor.Web.Plugins.AzureBlobCache.AzureBlobCache, ImageProcessor.Web.Plugins.AzureBlobCache" maxdays="365">
      <settings>
        <setting key="CachedStorageAccount" value="DefaultEndpointsProtocol=https;AccountName=<accountname>;AccountKey=<accountkey>;EndpointSuffix=core.windows.net">
        <setting key="CachedBlobContainer" value="cache">
        <setting key="UseCachedContainerInUrl" value="false">
        <setting key="SourceStorageAccount" value="DefaultEndpointsProtocol=https;AccountName=<accountname>;AccountKey=<accountkey>;EndpointSuffix=core.windows.net">
        <setting key="SourceBlobContainer" value="media">
        <setting key="StreamCachedImage" value="false">
        <setting key="CachedCDNRoot" value="https://<cdnrootname>.azureedge.net">
        <setting key="CachedCDNTimeout" value="1000">
      </setting></setting></setting></setting></setting></setting></setting></setting></settings>
    </cache>
  </caches>
</caching>

Conclusion

So, if you followed these tips correctly, you’ll be able to run WebPagetest and get the same result as I did:

You can find the complete report here: https://www.webpagetest.org/result/181127_2A_bea6941dcd20d38ab54c29409fca9363/.

The post Straight A’s in WebPagetest with Umbraco appeared first on Blog IT.

Create IT as an Internet connection of 100mbps Down/20mbps Up. Azure was capping at a 150mbps symmetrical connection.

TeamViewer VPN, Azure Site-to-Site and Point-to-Site connections were capped at 10mpbs.

Azure plans

In Azure, the most economical plans were chosen, considering our requirements. Some plans were free, some were cheaper than the others, but having VPN capabilities and all configurations supported, we chose the cheaper plans that were available with all these needed features supported.

Browsers

We used Chrome, Firefox and Edge. This was to eliminate browser differences. Not showing any differences in total execution times, we kept using Chrome as a default testing browser.

LAN Connection

Lan wise, our internal network is based on a 1gbps Local Area Network.

On-Premises Service Host

The On-Premises Service was hosted on a machine running Windows 10 Retail with latest updates installed and IIS Express. All code was made with Visual Studio 2017 Enterprise. The relevant host hardware specifications are:

1. Intel® Core™ i7-6700HQ CPU @ 2.6GHz
2. 32GB of RAM DDR4 @ 3400MHz
3. NVMe M.2 PCI-e 240GB SSD

Testing Hours

All tests were executed on working hours, 9am to 18pm, GMT.

The Writer

I’m a consultant @ Create It, a Portuguese company. If you read this, make sure I know about it! ? Hugs and kisses! This was an absolute pleasure to make ?

The post Latency test between Azure and On-Premises – Specifications appeared first on Blog IT.

Note: 5MB results must be multiplied by 10 (value x 10)

Here are the results. All of them. Don’t forget to multiply 5MB results by 10!

With the graph below, we can check how message sizes influence latency. After doing some math, these are the results:

As we can confirm, latency follows an exponential growing rate, which means, the bigger the message, even greater will be latency. We can confirm that, for each scenario, this applies.

But what does it tell me? This tells you that you need to be careful if you’re planning on sending big messages between two points. This exponential growing rate applies to all scenarios!

All values listed and noted, so we gave tests a score. This score is calculated by adding all three results (10kb, 100kb and 5mb) and dividing all by three. This will give us average execution time. For your best understanding, below is an exponential graph. This climbs faster and faster to infinity, being the horizontal axle message sizes, and the vertical one latency:

Now for the results!

As we can check, Test #2 (Local On-Premises LAN) was the winner here, latency wise. This was expected. Although this is the fastest, it’s the riskiest and NOT RECOMMENDED! It’s true that latency is reduced, but with only ~53ms difference between Azure Site-to-Site on a 100Kb message, is it worth to have a full infrastructure depending on your maintenance? And the networking gear it requires? What about having someone responsible for the room 24/7? How about costs? Think about it! Think about savings when migrating your business to Cloud!

Second place goes to HTTP without VPN (Exposed services). Not a Cloud solution, and a risky one too! If you like hackers messing with your vital business services and trying to break in from all over the world 24/7, go right ahead!

Regarding fast and safe Cloud solutions, which is the main reason of reading this document after all, the winner was Test #4 (Azure Site-to-Site VPN), getting third place on the board, by a difference of 8 points! This managed to be the safest, more efficient and with 99.9% uptime method of expanding your office or network, and have a low-latency data exchange. This is the optimal and safer solution. No hackers, no maintenance, no worries!

Consult us to migrate your business! You’re at the doorstep to your future!

For full test specifications, read here!

The post Latency test between Azure and On-Premises – Conclusions appeared first on Blog IT.