• Introduction
• Get Started
• Logs
  • Canonical logs
• Traces
• Metrics
• General Best Practices
• Resources
  • GitHub demo repo
• Conclusion

Introduction

In this small post, I’ll share some resources, notes I’ve taken while learning, and best practices for making our systems observable. I’ve always had a knowledge gap regarding observability, and recently I’ve truly enjoyed learning more about this area in our software industry.

Quick note: In this post I’ll only share about 3 telemetry signals. Profile is another signal that I will research in the future.

Get Started

Follow these steps to get started with auto-instrumentation in your application using OpenTelemetry: https://opentelemetry.io/docs/languages/net/getting-started/#instrumentation

For OpenTelemetry in a front-end app you can check these useful resources:

• Grafana faro
• Next.js
• Guide for OpenTelemetry in Next.js
• Browser OpenTelemetry getting started

Client-side instrumentation in OpenTelemetry is part of their roadmap which is great to see, since I’ve only seen vendor-specific solutions and products for front-end apps (e.g. New Relic, Datadog). For browser instrumentation otel doesn’t seem to be super mature yet, but a lot of effort is being put into this area by the OpenTelemetry team.

Logs

We all know about logs 😄. It’s data that we all need in order to troubleshoot and know what is happening in our applications. We shouldn’t overdo it, creating tons and tons of logs since that will probably create noise and make it harder to troubleshoot problems.

For logs, we can use these best practices. From this list, these are an absolute must to follow:

• Avoid string interpolation
• Use structured logging
• Log redaction for sensitive information

In addition to the list above, we should also include the TraceId and SpanId in our log records, to correlate logs with traces. If you are using the Serilog console sink, by default the message template won’t have those fields so if you want them, consider using JsonFormatter or CompactJsonFormatter. Here is an example Serilog configuration in appsettings.json (setup to remove unnecessary/noisy logs):

"Serilog": {
    "Using": [
      "Serilog.Sinks.Console"
    ],
    "MinimumLevel": {
      "Default": "Information",
      "Override": {
        "Microsoft.AspNetCore": "Warning",
        "Microsoft.Extensions.Diagnostics.HealthChecks": "Warning"
      }
    },
    "WriteTo": [
      {
        "Name": "Console",
        "Args": {
          "formatter": {
            "type": "Serilog.Formatting.Json.JsonFormatter, Serilog",
            "renderMessage": true
          }
        }
      }
    ],
    "Enrich": [
      "FromLogContext",
      "WithMachineName",
      "WithThreadId",
      "WithProcessId",
      "WithProcessName",
      "WithExceptionDetails",
      "WithExceptionStackTraceHash",
      "WithEnvironmentName"
    ],
    "Properties": {
      "Application": "GrafanaDemoOtelApp"
    }
  }

Below are some documentation links for logging in .NET. The ILogger extension methods are not always the best choice (e.g. logger.LogInformation), especially in high-performance scenarios or if your logs are in a hot path:

• High-performance logging in .NET
• Compile-time logging source generation

Canonical logs

There is also a different way of logging, based on having more attributes in one single log line. I’ve seen this in Stripe where they call it canonical log lines. Charity Majors also references this canonical logs term in her blog post about Observability 2.0 (that I reference in the Resources section).

This idea is very interesting, but might lack awareness. At least in .NET land, I didn’t find many references to this style of logging or example code that we could follow when there are many ILogger instances involved.

Traces

For traces in .NET we have these best practices. So far I’ve seen four common solutions for adding correlation ids in traces (not all are standards):

• W3C trace context – current standard in the HTTP protocol for tracing
• X-Correlation-Id – a non-standard HTTP header for RESTful APIs (also known as X-Request-Id). I thought this was a standard since it’s widely used, but I didn’t find a RFC from IETF or any other organization.
• Request-Id – this is a known header in the .NET ecosystem
• B3 Zipkin propagation – Zipkin format standard
• AWS X-Ray Trace Id – proprietary solution for AWS that adds headers for tracing

Not every company/project uses W3C trace context, you have some options above to pick from. I prefer the standard W3C trace context 😄 (maybe the industry will widely adopt this in the future) and using OpenTelemetry to manage these headers (HTTP, AMQP, etc) and correlation with logs automatically. The code you don’t write can’t have bugs 😆.

With that said, in some situations, you might have integrations with 3rd party software and need to use their custom headers or project limitations and need to use a particular format. At the end of the day what’s important is that you have distributed tracing working E2E.

There is also a relevant spec for distributed tracing called Baggage which OpenTelemetry implements and we can use in our apps. The most important part here is trace propagation to get the full trace from the publisher to the consumer.

Metrics

For metrics, it’s important to follow naming conventions for custom metrics. Especially if your organization has a platform team, setting conventions helps everyone. I do know some otel semantic conventions aren’t stable, and that also leads to some nuget packages being pre-release.

But anyhow, set conventions for your team or read and follow OpenTelemetry semantic conventions.
An important resource I found is the comments on Prometheus best practices related to high cardinality metrics.

When I started trying out custom metrics instrumentation I discovered that OpenTelemetry is not always used (the SDK + OTLP). We have the Prometheus SDK which is mature and widely used. Then for Java there are other solutions like Micrometer and others that integrate very well with Spring. In regards to the Java ecosystem, I read this otel Java benchmarks and this Spring post just because I was interested in knowing what the industry is adopting and why.

General Best Practices

There is a ton to be learned with SRE principles and practices. But one in particular was very useful for me and my team: always categorize our custom metrics according to the 4 Golden Signals. Any metric we can’t categorize is probably not useful for us.

Source of Denise Yu’s art.

Google’s SRE book is amazing to learn more about the 4 Golden signals and creating SLO-based alerts. All our alerts should be actionable (or the support team will not be happy), so it helps if they are based on SLOs that are defined as a team.

They also have some best practices for production services.

Resources

• Glossary of many observability terms in case you’re not familiar with them: https://github.com/prathamesh-sonpatki/o11y-wiki
• Awesome Observability GitHub repo
• If dashboards make you happy check the Grafana observability report dashboard
• AWS observability best practices guide
• About RED and USE method
• Traces Instrumentation best practices in .NET
• What are the Limitations of Prometheus Labels?
• CNCF OpenTelemetry certification
• TAG Observability whitepaper – this is an amazing resource with tons of information! I also recommend checking out the other resources they have in the tag-observability repo and community
• Resources specifically about Observability 2.0:
  • Observability 2.0 by Charity Majors
  • Re-Redefining Observability
  • Is It Time To Version Observability? (Signs Point To Yes) – Charity Majors
• Talks
  • How Prometheus Revolutionized Monitoring at SoundCloud – Björn Rabenstein
  • How to Include Latency in SLO-based Alerting – Björn Rabenstein, Grafana Labs
  • Myths and Historical Accidents: OpenTelemetry and the Future of Observability Part 1
  • Modern Platform Engineering: 9 Secrets of Generative Teams – Liz Fong-Jones
  • Context Propagation makes OpenTelemetry awesome

GitHub demo repo

I’ve been developing a demo app (it has fewer features than the otel demo) to demonstrate how to build an app with OpenTelemetry, Grafana and Prometheus. It’s primarily focused on a small app I can showcase in my talks.

If you’re interested take a look: https://github.com/BOLT04/grafana-observability-demo

Conclusion

Hopefully, some of these resources I’ve shared are useful to you 😄. I still have a ton to learn and explore, but I’m happy with the knowledge I’ve acquired so far.

There are some specific standards + projects that I’ll dive in and explore more, like: eBPF; OpenMetrics. OpenMetrics is something I’d like to spend some quality time reading about, but I know it’s archived and reddit says the same. Just want to read and watch some talks about it to feed my curiosity 😃.

Last but not least, I want to follow the work that some industry leaders are doing like Charity Majors, specifically about Observability 2.0 😄. I discovered this term in the Thouthworks tech radar, and the part “high-cardinality event data in a single data store” caught my interest.
I’m still learning, researching, and listening to the opinions of industry leaders about this term to then develop my own opinions. Maybe I’ll make a blog post about this in the future 😁.

If you’re interested, check out my other blog posts:

• Dreamforce 2023 Highlights
• Getting Started with CloudEvents and AsyncAPI

The post Learning: Observability appeared first on Blog IT.

Unit Tests, Integration Tests, and End-to-End Tests: What’s the Difference?

• Unit tests focus on testing individual units or components of your application in isolation. This type of test verifies whether the unit/component works correctly and adheres to its business logic.
• Integration tests aim to test multiple units or components together, ensuring that they interact properly. Integration tests can uncover issues related to data flow, communication between components, and external dependencies like databases.
• End-to-end (E2E) tests simulate a complete user scenario by testing the entire application from start to finish. E2E tests can help identify issues related to multiple components, external APIs, and user interfaces.

public class InMemoryDbAppFactory : WebApplicationFactory<Program>
{
    private readonly string _environment;

    public InMemoryDbAppFactory()
    {
        _environment = "IntegrationTests";
    }

    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        Environment.SetEnvironmentVariable("ASPNETCORE_ENVIRONMENT", _environment);
        builder.UseEnvironment(_environment);
        builder.UseSetting("https_port", "8080");

        builder.ConfigureTestServices(services =>
        {
            services.RemoveAll<ITestRepository>();
            services.TryAddTransient<ITestRepository, TestRepositoryTest>();

            // Anonymous authentication
            services
                .AddAuthentication("Test")
                .AddScheme<AuthenticationSchemeOptions, TestAuthenticationHandler>("Test", options => { });
        });
    }
}

public static partial class ServiceCollectionExtensions
{
    public static IServiceCollection AddDbContexts(
        this IServiceCollection services,
        IConfiguration configuration,
        IWebHostEnvironment environment
    )
    {
        if (environment.IsEnvironment("IntegrationTests"))
        {
            var connDb = new SqliteConnection("DataSource=db;mode=memory;cache=shared");
            connDb.Open();
            services.AddDbContext<dbContext>(options => options.UseSqlite(connDb));
        }
        else
        {
            services.AddDbContext<dbContext>(
                options => options.UseSqlServer(configuration.GetConnectionString("ConnectionString")).UseExceptionProcessor(),
                ServiceLifetime.Scoped
            );
        }

        return services;
    }
}

[Collection("MemoryDbIntegrationTests")]
public class ApiTests : IClassFixture<InMemoryDbAppFactory>
{
    private readonly InMemoryDbAppFactory _factory;
    private readonly HttpClient _client;

    public ApiTests(InMemoryDbAppFactory factory)
    {
        _factory = factory;
        _client = factory.CreateClient();

        // Ensures a clean database before each test
        factory.ResetDb();
    }

    [Fact]
    public async Task GetList_Should_Return_List()
    {
        // Arrange

        // Act
        var response = await _client.GetAsync("/api/list");
        var content = await response.Content.ReadFromJsonAsync<IEnumerable<Dto>>();

        // Assert
        response.EnsureSuccessStatusCode(); // Status Code 200-299
        content.Should().NotBeNull();
        content.Should().HaveCount(2);
    }
}

Using an in-memory database for integration tests has both advantages and disadvantages:

Advantages:

• Faster test execution: In-memory databases provide quicker test execution times since the data is stored in memory instead of on disk. This can significantly improve your overall testing performance and help you find issues faster.
• Easy to set up and tear down: In-memory databases are easy to create, modify, and delete without the need for external dependencies or configuration changes. This makes it easier to write, run, and maintain your tests.
• Consistent test data: In-memory databases allow you to create a known set of data for your tests, ensuring that each test starts with the same initial conditions. This can help reduce the likelihood of inconsistent test results and make it easier to identify issues related to data flow or dependencies between tests.

Disadvantages:

• Limited scalability: In-memory databases have limited capacity and may not be suitable for testing large datasets or complex scenarios that require high levels of concurrency.
• Lack of realism: In-memory databases may not accurately represent the behavior or performance of a production database, especially when dealing with complex queries or data modification operations. This can make it difficult to identify issues related to database schema, indexing, and query optimization.
• Limited support for advanced features: using SQLite as our in memory database we don’t have support for multiple schemas and some advanced query features.
• Seed data: the need to create seed data can be tedious and time consuming.

When deciding whether to use an in-memory database, mock data, or a real web server in memory for your integration tests, consider the specific requirements of your project and weigh the advantages and disadvantages of each approach. In many cases, using a combination of testing types and approaches can help you ensure the stability, reliability, and performance of your .NET WebAPI application.

The post Integration Tests in .NET WebApi appeared first on Blog IT.

The first step here is to ensure that your consumer code is idempotent, but that’s something that you should always ensure – just imagine that you have to re-consume a couple of messages, in that case lacking idempotency may result in your system getting strange data and bugs.

Setting up Kafka on your machine

(Feel free to skip this section if you have everything set up.)

Kafka releases aren’t included in most package managers, except for Homebrew in Mac systems. They should be available, however, in the official download page. For Unix systems (Mac, Linux, WSL, *BSD), simply get one of the binary downloads and extract it wherever you prefer. Then, make sure to add the bin folder to your $PATH, so that you can use these scripts in any folder. In my case, I put the extracted archive in $HOME/.local/share/kafka, so I edit my .zshrc (or .bashrc in a plainer Linux/WSL system) and add:

export PATH=$HOME/.local/share/kafka/bin:$PATH

Now we can use the official kafka scripts in any directory. That is, unless we have some form of authentication. We can add additional configs to a file somewhere, and include that file in our commands. Let’s put a file called dev.cfg in $HOME/.config/kafka/ (the exact directory doesn’t matter, but .config is the appropriate directory for configs), and write the following on it:

security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
    username="me" \
    password="noneofyourbusiness";

This is just an example, your broker’s security config may vary. After doing this, just add to any of the Kafka commands you use:

--command-config $HOME/.config/kafka/dev.cfg

Checking consumer offsets

Before migrating topics, it’s best if we make sure that the old topic consumer is totally up-to-date. We can use this command to check the consumption lag:

kafka-consumer-groups.sh \
	--bootstrap-server='<your broker endpoints, separated by commas>' \
	--describe \
	--group='<your consumer group>'

This should yield something like the output below:

TOPIC      PARTITION   NEWEST_OFFSET   OLDEST_OFFSET     CONSUMER_OFFSET     LEAD         LAG
your-topic 0           30000000        0                 30000000            30000000     0
your-topic 1           23900000        0                 23900000            23900000     0

CLIENT_HOST      CLIENT_ID         TOPIC          ASSIGNED_PARTITIONS
/128.0.0.0       your-consumer     your-topic     0
/128.0.0.1     	 your-consumer     your-topic     1

From this we can see that there is zero lag, since the newest offset is equal to the consumer offset on both partitions. We can also see that our consumers are still running (they’re the clients with ID “your-consumer”).

Performing the Kafka consumer migration

Before switching topics or brokers, we should first create a consumer group for the new configuration. This will allow us to change the offsets before beginning the message consumption proper. The easiest way to do this is to use kcat, a simple command-line tool that allows us to consume and produce Kafka messages. To create a consumer group, you must consume messages, so it’s a matter of setting up kcat (check the install guide on the kcat repo) and consuming a few messages from the new topic:

kcat \
	-C \
	-b '<your brokers>' \
	-t new-topic \
	-G new-consumer-group

# If you have authentication, just add: 

	-X security.protocol=SASL_SSL \
	-X sasl.mechanisms=SCRAM-SHA-512 \
	-X sasl.username="me" \
	-X sasl.password="noneofyourbusiness"

Just run this for a second, check that messages come out, and then run the describe command for the new group:

kafka-consumer-groups.sh \
	--bootstrap-server='new-broker' \
	--describe \
	--group='new-group'


TOPIC     PARTITION   NEWEST_OFFSET   OLDEST_OFFSET     CONSUMER_OFFSET     LEAD         LAG
new-topic 0           12387000        0                 12                  12           12386988
new-topic 1           12360000        0                 38                  38           12359962

CLIENT_HOST      CLIENT_ID         TOPIC          ASSIGNED_PARTITIONS

We have huge lag here, but that’s fine, as we’re going to reset the offsets anyway. Stop the workers consuming from the older topic by whatever means you can and note down the time at which they stopped. We will now reset the consumer group offset for this new group. Just a note: in this case, we don’t have any consumers connected to this group, but if we did we’d also have to turn them off. You cannot reset a consumer group offset if there are clients connected to it.

To reset the offsets, just run the following command (here we’ll assume that we stopped the previous consumers at 10:00 of March 15th, 2023, and subtract ten minutes just to be safe):

kafka-consumer-groups.sh \
	--bootstrap-server='new-brokers' \
	--group='new-group' \
	--topic='new-topic' \
	--reset-offsets \
	--to-datetime 2023-03-15T09:50:00.000 \
	--dry-run

Running this dry run will not make any changes, It will only show you the new offsets for the consumer group, e.g.:

TOPIC     PARTITION   NEWEST_OFFSET   OLDEST_OFFSET     CONSUMER_OFFSET     LEAD         LAG
new-topic 0           12387000        0                 12386998            12386988     2
new-topic 1           12360000        0                 12369951            12369951     49

CLIENT_HOST      CLIENT_ID         TOPIC          ASSIGNED_PARTITIONS

If nothing looks strange to you, go ahead, substitute --dry-run with --execute and run the command. Afterwards, deploy the new consumer configuration, turn on the worker, and let it run.

The post Performing broker migrations on your Kafka consumers appeared first on Blog IT.

There are currently some convoluted ways of doing this, using CSS filters to change brightness or contrast, or using CSS variables to separate each of the color’s values and use the calc function to manipulate them.

With the CSS Color Module Level 5, there are some great new features that will make this manipulation much easier to achieve.

Keep in mind, these features are not yet widely supported by browsers, being either experimental or hidden behind flags. Hopefully, these will start to roll out during this year.

Color spaces

Before getting into color manipulation, one other feature that will improve the results of dynamic color palettes are color spaces.

Color spaces essentially determine how colors are defined and how they can be represented. The CSS Color 4 introduces a bunch of ways to define colors, one of them being Lightness Chroma Hue (LCH). LCH introduces many improvements over RGB, such as more colors and more uniformity. For more detail on this, I recommend Lea Verou’s blog post “LCH colors in CSS: what, why, and how?”.

Color mix

The color-mix function lets us mix two colors with a given weight to each. You can mix the theme color with white or black to make it lighter or darker, or mix it with the secondary color to get interpolated values between them.

color: color-mix(in srgb, var(--primary-color) 70%, white);
color: color-mix(in srgb, var(--primary-color) 70%, black);
color: color-mix(in lch, var(--primary-color), var(--secondary-color));

Resources:

• CSS Color 5 – color-mix
• Can I Use – color-mix
• MDN – color-mix

Relative colors

With relative colors, you can define color values based on a given color, and change the individual values that make up that given color. Remember the approach I mentioned above that separates each of the color’s components into a CSS variable? This blog post by Jim Nielsen compares the two approaches.

// Reduce the primary color transparency
color: rgb(from var(--primary-color) r g b / 80%);

// Reduce the lightness using the LCH color space
color: lch(from var(--primary-color) calc(l / 2) c h);

Resources:

• CSS Color 5 – relative colors

Color contrast

With the color-contrast function, we can let the browser automatically determine a color that contrasts with a given color and matches accessibility requirements.

At its simplest, the function receives a color value (typically the background color) and returns either white or black, depending on which has better contrast.

If you want custom contrast colors instead of black or white, you can provide a list of colors to pick from (using the vs keyword), and the color with the highest contrast will be chosen. As a backup, you can define a contrast target (using the to keyword) and if no color matches that contrast, white or black will be picked. The contrast target can be given as a number or as a level from WCAG (AA, AA-large, AAA, and AAA-large).

// Chooses black or white based on which has better contrast
color: color-contrast(var(--bg-color));

// Chooses a contrasting color from a given list (after the "vs"), that result in  a contrast level of at least AA. If none match, picks white or black
color: color-contrast(
	var(--bg-color)
	vs
	var(--text-color),
	var(--text-color--light),
	var(--text-color--lightest),
	to AA
);

Resources:

• CSS Color 5 – color-contrast
• Can I Use – color-contrast
• MDN – color-contrast

Conclusion

The future is looking bright for dynamic color themes in CSS. We will soon be able to move away from convoluted hacks and start using native functions.

Do you find these color features useful? How do you get around this currently? Do you know of any other features that you think help with this topic? Leave a comment or reach out to me on Twitter @pedronavelopes.

The post The future of dynamic color palettes in CSS appeared first on Blog IT.

There are two types of 404 routes that we want to handle.

• Static 404 – the URL the user navigates to doesn’t match any route
• Dynamic 404 – the URL matches a route, but the data associated with that route does not exist. For example, you navigate to a detail page with an ID, but no item exists with that ID.

For this post, I assume that there is a component named NotFoundComponent that is the not found page to show the user. Replace this with whatever component

Static 404 routes

To set up a static 404 page, you need to add two routes to your RoutingModule configuration.

const routes: Routes = [
    // Other routes
    { path: '404', component: NotFoundComponent },
    { path: '**', component: NotFoundComponent }
]

We used a catchall route '**' that matches whatever route didn’t match above, and shows our not found page.

Make sure to keep the catchall route at the end of your routes, otherwise the routes below it will not be accessible.

Another option here would’ve been to redirect the user to the '404' route, but that way you lose the context of the URL. If the user navigates back, he would go to the non-existing route and be redirected to the '404' path again. If this is the behavior you want, switch the last route with { path: '**', redirectTo: '404' }.

Dynamic 404 routes

Now that we have the static 404 route configured, we can use the router to navigate to it whenever we want to show the not found page to the user.

this.router.navigate(['/404'], { skipLocationChange: true });

By using the option skipLocationChange: true, we have the same behavior as mentioned above. Even though the router navigates to the route, the browser URL stays the same, allowing the user to easily go back.

Personalizing the 404 message

You may want to personalize the message shown on the 404 page. One way to do this is using the navigation state.

When navigating to the 404 page, you can include a state in the navigation options.

this.router.navigate(['/404'], { 
    skipLocationChange: true,
    state: {
        // Whatever data you need on the 404 page
    	source: 'article',
        id: 123456
    }
});

On your NotFoundComponent, you can access this object in the constructor using the router’s getCurrentNavigation method.

export class NotFoundComponent {
  constructor(private router: Router) {
    console.log(this.router.getCurrentNavigation().extras.state.source);
  }
}

You must access the state in the constructor, as it is only available while the router is navigating.

Conclusion

In this post, we saw how to implement a 404 route in Angular. We looked at how we can redirect to it dynamically and even how to personalize the 404 page with route dependent data!

Have questions, corrections, or an alternative way of handling this? Leave a comment or reach out to me on Twitter @pedronavelopes and let me know! Thanks for reading!

The post Handling not found routes in Angular appeared first on Blog IT.

In this post, we’ll look at a way to make scrollable components more intuitive on mobile devices using CSS scroll shadows.

Let’s think of a table. It’s a collection of rows with columned data. One way to make a table responsive is by collapsing each row into a card. This will, however, remove the link between rows for any given column.

An alternative is to make the table scroll horizontally. This gives your data space to breathe and maintains the link given by columns. All done and pretty easy, right? Well, you might run into this problem:

If your column width happens to match the width of the table element, your users will have no idea that there is extra content to the side.

Adding Shadows

One way to fix this is by using scroll shadows. A scroll shadow is a shadow applied at the border of the element when there is the possibility of scrolling. That same example with a scroll shadow might look like this:

As you can see, the shadow is only visible when there is space to scroll. There are many ways to do this, I’m going to show you a way that only uses CSS.

First, we start by adding the shadows with two radial gradients.

.scroll-shadows-h {
  background:
    radial-gradient(farthest-side at 0 50%, rgba(0,0,0,0.4), rgba(0,0,0,0)),
    radial-gradient(farthest-side at 100% 50%, rgba(0,0,0,0.4), rgba(0,0,0,0)) 100% 0;
  background-repeat: no-repeat;
  background-size: 14px 100%, 14px 100%;
}

Hiding the shadows

This results in the shadows being visible even when there isn’t space to scroll. To fix this, we add shadow covers that hide the shadows at the beginning and end of the element, instead of its viewport.

We can implement this using the background-attachment property. To quote the MDN, “The background-attachment CSS property sets whether a background image’s position is fixed within the viewport, or scrolls with its containing block”.

Setting the background-attachment of the shadow covers to local, means that they are fixed relative to the content of the element. When we scroll inside the element, the background scrolls as well.

.scroll-shadows-h {
  background:
    /* Shadow covers */
    linear-gradient(to right, #fff 30%, rgba(255,255,255,0)),
    linear-gradient(to right, rgba(255,255,255,0), #fff 70%) 100% 0,
   
    /* Shadows */
    radial-gradient(farthest-side at 0 50%, rgba(0,0,0,0.4), rgba(0,0,0,0)),
    radial-gradient(farthest-side at 100% 50%, rgba(0,0,0,0.4), rgba(0,0,0,0)) 100% 0;
  background-repeat: no-repeat;
  background-size: 40px 100%, 40px 100%, 14px 100%, 14px 100%;
  background-attachment: local, local, scroll, scroll;
}

This class is the class used in the example in the image above. The cool thing about this implementation is that it’s very versatile. We can apply it vertically or use any background we want to. You could replace the shadows with an image (e.g. an arrow) and this effect would still work, as the covers will continue to overlap it. Check the CodePen below for some examples!

Thanks for reading! Do you find this useful? Do you think there are better ways to achieve this effect? Let me know in the comments below, or reach out to me on Twitter @pedronavelopes!

This blog post was inspired by Lea Verou’s post “Pure CSS scrolling shadows with background-attachment: local“. Go check it out!

The post CSS Scroll Shadows appeared first on Blog IT.

Let’s say we have the following table Documents:

CREATE TABLE [dbo].[Documents](
    [Id] [integer] PRIMARY KEY,
    [UserId] [integer] NOT NULL,
    [Title] [nvarchar](50) NOT NULL,
    [Body] [nvarchar](250) NOT NULL,
    [CreatedOn] [datetime] NOT NULL
)

and that we want to get the two most recent documents for each user. For instance, if we have the following records:

the query should return

To do this query using EF Core and LINQ we can try to group the Documents by UserId and then sort each group by the CreatedOn column to pick the first two documents for each user.

We can start by trying to group all documents for each user

 var usersDocs = await ctx
        .Documents
        .GroupBy(doc => doc.UserId)

and then for each group we try to sort its elements by the CreatedOn column

    var usersDocs = await ctx
        .Documents
        .GroupBy(doc => doc.UserId)
        .SelectMany(userDocs => userDocs.OrderByDescending(doc => doc.CreatedOn).Take(2))
        .ToArrayAsync();

If we try to execute this query using a previous version of EF Core 6 we get the following error:

    .OrderByDescending(doc => doc.CreatedOn)' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.'

This is because the previous versions of EF don’t know how to translate the GroupBy inner expressions to sql. After following their suggestion I ended up with this query

    var usersDocs = await ctx
        .Documents
        .Select(doc => doc.UserId)
        .Distinct()
        .SelectMany(userId => 
            ctx
            .Documents
            .Where(doc => doc.UserId == userId)
            .OrderByDescending(doc => doc.CreatedOn)
            .Take(2)
        )
        .ToArrayAsync();

However this query has two main issues. It does a distinct over the UserId column, loads all the user ids to application memory and then it does a query for each user resulting in a n+1 query problem.

One way to solve these issues is to forget LINQ and rewrite the query using raw sql with a partition by UserId and the ROW_NUMBER() window funtion, doing a CTE .

Now, with the EF Core 6 we can use the first version of the query, since the EF Core team has added the support for translating some GroupBy inner expressions and it can translate this LINQ query to a single sql query.

    var usersDocs = await ctx
        .Documents
        .GroupBy(doc => doc.UserId)
        .SelectMany(userDocs => userDocs.OrderByDescending(doc => doc.CreatedOn).Take(2))
        .ToArrayAsync();

You can read more about these features and other improvements added to GroupBy queries here and check the related github issues 12088 13805 

Happy coding!

The post .NET EF Core 6 support for groupby top(n) queries appeared first on Blog IT.

I would like you to keep in mind that this will not make your website fully accessible, it’s just an easy way to vastly improve accessibility.

1 – Follow WCAG and WebAIM checklist

The WCAG (Web Content Accessibility Guidelines) is a set of guidelines on how the web can be made more accessible. It takes 4 principles into consideration:

1. Perceivable – Just because something is perceivable with one sense (usually sight), not all users can perceive it
2. Operable – Can users use UI components and navigate the content. For example, hover interactions can’t be operated by someone without a mouse
3. Understandable – Can all users understand the content
4. Robust – Is it robust enough for the content to be consumed by a wide variety of user agents, and does it work with accessibility tools.

The WebAIM Checklist was created from these guidelines and simplified the implementation of these principles. This checklist should be seen as a companion to start implementing accessible interfaces, not as the final resource on what make a website accessible.

2 – Provide alternatives to non-text content

Non-sighted users may have a hard time taking in non-text content, as such, it’s important to provide alternatives to these.

• Provide alt text for images The alt text should be an alternative to the image content
  • If an image is not relevant for the context, you can hide it from screen readers by providing an empty alt text.
• Use aria-label for other elements
  • The aria-label lets you label elements for screen readers

// 👎 No descriptive alt text
<img src="puppy.jpg">

// 👍 Descriptive alt text
<img src="puppy.jpg" alt="Labrador puppy running in the grass">

// 👍 Empty alt text excludes the image that isn't relevant
<button>
	<img src="home-icon.jpg" alt="">
	Home
</button>

3 – Use native and semantic elements

When implementing an HTML page, try to use native and semantic elements.

If you need a button, you should use a <button> element, not a div with an onclick event or an anchor link.

This is the case because different elements mean different things, and have different behaviors. By using the native HTML elements, we know it will have the expected behavior. And by using a semantic element, we ensure that the user understands what the element will do, as in, a button will trigger an action, and a link will trigger a navigation to somewhere else.

// 👎 Non-semantic element
<div class="button" click="onclick">
	Click me!
</div>

// 👍 Semantic element
<button click="onclick">
	Click me!
</button>

4 – Use ARIA attributes in elements

When there is no native element which matches your need, you may have to implement it yourself.

To make sure the element you are implementing is accessible, you should use ARIA (Accessible Rich Internet Applications) attributes.

These will allow you to add semantics to elements which have no meaning, modify existing semantics, add extra labels and descriptions, express relationships between elements, etc.

The ARIA practices document specifies the patterns that each element should have. When implementing an element, you give it a role and commit to following all the patterns and interactions defined in the ARIA practices.

5 – Understand focus and tab order

Tab order refers to the order in which the items on a page will be focused using the keyboard.

Some elements are implicitly focusable (such as inputs, buttons and links), and are added to the tab order automatically.

The tab order is determined by the DOM order, but this can be different from the visual order which can be changed by CSS. Therefore, you should make sure that the reading and navigation order are logical and intuitive.

To change the tab order, you can use the tab index attribute:

• -1
  • Removes the element from the natural tab order, but it is focusable through code.
  • Useful when we have hidden elements (such as modals) which should not be focusable when hidden
• 0
  • Adds the element to the natural tab order.
  • Allows elements to be focused and receive keyboard input.
• 0
  • Jumps element to the front of the tab order
  • If there are multiple elements, order goes from lowest to highest
  • This is discouraged because it can make it confusing for screen readers, as they navigate the page linearly

Make sure you have a focus ring. A focus ring lets users know which element is focused. This is essential for keyboard users.

6 – Bypass Blocks

If your page has a lot of navigation links, a keyboard user will have to tab through all these links before being able to reach the main content. To make your website more convenient to use, you can provide users with a skip link.

The skip link will be the first focusable element on the page and links the user to the main content, skipping the focus to it directly.

<body>
  <a href="#maincontent">Skip to main content</a>
  <!-- Navigation links --->
  <main id="maincontent">
    <h1>Heading</h1>
    <p>This is the first paragraph</p>
  </main>
</body>

To make sure the skip link doesn’t affect your page, you can hide it until it is focused, meaning only keyboard users will see it.

Alternatively to skip links, if you provide a proper heading structure, screen readers will also be able to skip around the page and navigate to each section much more easily. This will, however, not provide a skip mechanism for users who just use the keyboard to navigate.

7 – Use ARIA attributes in CSS selectors

You can use ARIA attributes in your CSS selectors. This makes sure that ARIA is being used correctly, and removes the need for additional selectors.

// 👎 Aditional selector
.toggle.pressed

// 👍 ARIA attributes
.toggle[aria-pressed="true"]

8 – Responsive design

Even if you don’t need to implement mobile designs, a responsive design ensures that the user can zoom in the page and the content adjusts properly. Your page should be zoomable up to 200% without losing content or functionality.

• Use relative font-size, allowing the browser to rescale the text
• Use appropriate touch targets (>48px) so users have enough space to touch a single button
• Space touch targets about 32px from each other

9 – Color and Contrast

There are tools which allow you to easily check for contrast. Some browsers even let you check this directly in the dev tools.

You also have to keep in mind that some users will not be able to distinguish some colors. As such, information should not be conveyed by color alone. Make sure you add other elements (such as captions or icons) to indicate the state of the elements.

10 – How to apply these tips

Good accessibility ensures good UX. As such, accessibility should be focused from the beginning of development.

When this doesn’t happen, we inevitably have to fix accessibility issues later. You can identify some issues using the following techniques:

• Use the accessibility inspector included in the browser
• Navigate through your website using only a keyboard
• Simulate impaired vision through browser extensions (e.g. NoCoffee)

After identifying some issues, how should we tackle them? As with bugs, we should focus on high impact, low effort issues.

• How frequently is this piece of UI used?
• How badly does this affect our users?
• How expensive is it to fix?

Conclusion

Accessibility is starting to make it to the front of developer concerns. With more popularity come more resources to help us develop better interfaces and experiences for all users.

Hopefully, this article provided you with an initial idea on how to approach web accessibility.

The post 10 easy ways to improve web accessibility appeared first on Blog IT.

How do we know collaboration is happening?

My reflection results from one recent team experience, in a software development context. We worked collaboratively to implement a new feature and after a work session two things about the final result called my attention:

1. It was not possible to identify the “owner” of the idea. Everyone said something that contributed to the work produced. It truly belonged to everyone.

2. The result we found was better than something I was aware of being able to find.

It is something I read, heard, and even talked about often. But for me it was good to be able to experience and learn in practice. There are few things that replace practice.

This is basic, it is. I knew that. But for me it was a special moment to be able to feel this energy. At the same time, I look at this and think “how come we don’t always do this?”.

How collaboration happens?

Some simple and practical actions to improve collaborative work.

Accept all contributions

The brain is very quick to identify reasons for not doing anything. Perhaps it is a hidden survival mechanism. As a result, idle is safer than boiling. This is a common place, on the other hand it is differentiating. The process of building on each other’s ideas is one of the critical factors to generate energy that will keep the work session active.

Feed the ideas that come up

Initially denying an idea because we see a point we disagree with inhibits collaboration. In other words, the team must be comfortable sharing. A flow of new ideas when each team member builds and contributes on top of each other’s ideas is fantastic. At the end it is not even possible to identify whose final idea is. There is only “us” in the result.

Desire to discover something new

When we restrict ourselves only to what we know, what we can discover will be limited. Dismantle pre-conceived ideas, especially our own. Being open minded to different points of view allows more contributions. It is easy and safe to say no. Take a chance and say why not? Or even better how can we achieve this?

Try It Out

Offer someone the opportunity to share their ideas. These are small but extremely important things, and they are also actionable. They are within everyone’s reach.

The post Collaborative work, why and how? appeared first on Blog IT.

How to install JMeter, what are the components of it and how to use it. It will, also, show a short introduction of what is performance testing.

First of all, let’s talk about Performance Testing. What is Performance Testing?

Performance Testing is a software testing process used for testing the speed, response time, stability, reliability, scalability and resource usage of a software application under a particular workload.  
The focus of Performance Testing is checking a software program’s

• Speed – Determines whether the application responds quickly
• Scalability – Determines the maximum user load the software application can handle.
• Stability – Determines if the application is stable under varying loads

What is the Purpose of doing it?

The main purpose of performance testing is to identify and eliminate the performance bottlenecks in the software application.

Now let’s introduce JMeter

How to install it?

JMeter is an application base in java. This means, that every computer with java installed can run this application. So let’s see how to install it.

1- Download Java.

 First of all, you will need to install Java on your computer. Here you can download and Install the latest version of Java SE Development Kit https://www.oracle.com/java/technologies/javase-downloads.html.

2- Download JMeter.

At the point of writing, the latest version is Apache JMeter 5.4.1 (Requires Java 8+) and you can download it here http://jmeter.apache.org/download_jmeter.cgi. Choose the Binaries file (either apache-jmeter-5.4.1.zip or apache-jmeter-5.4.1.tgz).

3- Installation.

Installing Jmeter is easy! You only need to unzip the file to a directory of your choice and that’s it!

4- Start JMeter

JMeter can be used in 3 modes. These 3 modes are:

1. GUI Mode
2. Server Mode
3. Command Line Mode

In this article, we will focus on GUI Mode. So, to start the GUI Mode, you will need to execute the following file /bin/ApacheJMeter.jar. This will open the application as the following example.

How to use JMeter?

Now that we have JMeter installed and running, we need to understand the components that are used to create a Test Plan.

When we open JMeter, we can see that there is already a “Test Plan” component. This component is where we add components for our JMeter Test. Here is an image of how these components are organized.

The components that we are gonna talk here will be:

• Thread Group
• Samplers
• Listeners
• Config Elements

Thread Group

Thread group is a collection of threads. This is the component where you define how many threads will run. Essentially, each thread represents a user using the application under test.

The thread group let’s you set the value of threads that you want to run(simulating the number of users, that you to test, using your app).For example, if you set threads number to 100. JMeter will create and make 100 request to the application under test.

Samplers

JMeter supports testing HTTP, FTP, JDBC and many other protocols.

We already know that Thread Groups simulate user request to the server, but how does a Thread Group know which type of requests (HTTP, FTP etc.) it needs to make? The answer is Samplers.

The user request could be FTP Request, HTTP Request, JDBC Request, etc. In this example, we will concentrate in HTTP Request.

Listeners

Listeners shows the results of the test execution. They can show results in a different format such as a tree, table, graph or log file.

Graph result listeners display the server response times on a Graph.

View Result Tree show results of the user request in basic HTML format.

Table Result show summary of a test result in table format.

Log show summary of a test result in the text file.

Config Elements

Set up defaults and variables for later use by samplers.

The figure below shows some commonly used configuration elements in JMeter.

Let’s create a test

As was shown before, JMeter starts with a Test Plan Component already created (you can change the name to anything more suitable to your case). 

Now let’s start creating our test. In this case we will use https://my-json-server.typicode.com/ to simulate an API.

Initially, let’s add a Thread Group. Right click on the test plan, then Add -> Threads(Users) -> Thread Group.

You can see that there is some settings that we need to adjust. At the top, there is a setting that says Action to be taken after a Sampler Error. Normally, I always use this in Continue but you can use the one that suits your case.

The next 3 settings will be the ones that we gonna focus for now.

• Number of Threads (users) – is the number of threads(users) that will be running your test plan.
• Ramp-up period (seconds) – is the value in seconds that defines how much time until all threads are working. Example: 100 Threads and 10 seconds of ramp-up period, this means that the threads will start, gradually, until at the point of 10 seconds all 100 threads will be working.
• Loop Count – You can set this to the Infinite and will only stop threads when you press the button to stop. Also, you can set a loop count, that is the times that each thread will execute the test plan. Example, if you put the value 3, the threads will execute the plan 3 times and then will stop.

So, in our example lets try 10 threads, with 5 seconds of Ramp-up Time and the value 3 in the loop count. I will also change the name of the thread group to know how many threads I am using. It Should look like this.

Let’s continue adding more components. This time, let’s add a way to define variables. Right-click on the Thread Group, then Add -> Config-Element -> User Defined Variables. 

This component will let you define some variable to use afterwards. Let’s define some.

• url: my-json-server.typicode.com
• postId: 1

 Now, to use these variables in other components you will need to do it in the following way.

• ${url} for the url Value 
• ${postId} for the postId Value

Finally, let’s add some real requests. Here as example we will make 3 different requests. Http Request to get all posts, Http Request to get one Post, and Http Request to Post a new Post. Remember that the site used in this example has paths to request posts and to add posts.

How to add an HTTP Request ? Right-click on the Thread Group, then Add -> Smapler -> Http Request.

Now that we have an Http Request component, let’s configure to make a Http Request to get all posts. The path to this request is https://jsonplaceholder.typicode.com/posts. The configurations of this component are:

• Protocol -> “https”
• Server Name or IP -> ${url} (because we have this value defined in a variable)
• Port Number -> in this case, nothing but if you need to your specific case, here is where you put the port number. Example: to make a request to https://localhost:80, you would need to put the value 80 in here.
• HTTP Request -> GET
• Path -> “/typicode/demo/posts”

Now that we have everything, let’s run ? Click on the start Button above and see what happens. Nothing ? Stop the process and let’s see what is missing.

The piece that is missing is listeners. We need listeners to understand what is happening. In this case, the View Results Tree and Summary Report. Let’s add these two listeners by Right-clicking in the Thread Group, then Add -> Listener -> View Results Tree and Add -> Listener -> Summary Report.

Now, if you start the process, you will see that these listeners represent information about the request. 

The View Results Tree show every request information. In here, you can see the request body and headers, and the response body and headers.

In this example, when we choose the Response Body of one request we will see that there will be a JSON with data relative to what they call as posts.

The Summary Report Aggregates the information of different types of request. In this case, we only have one request, so, it will show informations about that request.What are the informations that this will show ? 

• #Samples – are the number of requests made 
• Average – shows  the average time needed to make these requests 
• Min – shows the value of the request that take less time to finish
• Max – shows the value of the request that take more time to finish
• Std. Dev. – shows the standard deviation of the request times to the average
• Error % – shows the percentage of requests that failed
• Throughput – shows the number of requests processed by time 
• Received KB/sec –  shows the amount of data downloaded from the server  time
• Sent KB/sec – shows the amount of data uploaded to the server time 
• Avg. Bytes – shows the average size of the response sample

We have all we need, let’s add more Requests. Let’s create a request that creates a post, and create a request that obtains only one post. Also, change the name of the Http Request components to now what is the correspondent request. Remember to put the Http Request before the listeners. The Listeners will only listen to what are before them. Remember this! 

The create a post request configurations are:

1. Protocol -> “https”
2. Server Name or IP -> ${url} (because we have this value defined in a variable)
3. HTTP Request -> Post
4. Path -> “/typicode/demo/posts”
5. Body Data – we will put a JSON like this {“title”: “Car Crash”}

And, The get one post request configurations are:

1. Protocol -> “https”
2. Server Name or IP -> ${url} (because we have this value defined in a variable)
3. HTTP Request -> Get
4. Path -> “/typicode/demo/posts/${postId}”

Run the test plan and see the results. Try to do it for another API. 

These are the basic examples that we have for you right now. More advanced tips will come. 

See you next time.

References

• https://www.guru99.com/jmeter-element-reference.html
• https://www.guru99.com/hands-on-with-jmeter-gui.html

The post Performance Testing An API, How To Start Doing It With JMeter? appeared first on Blog IT.