Friday, September 13, 2019

Principled GraphQL

In case you didn’t notice it yet, I’m a big fan of GraphQL. It solves so many problems when dealing with an ever growing list of devices and (micro)services.

The people who created Apollo distilled their experiences into a set of best practices for creating, maintaining, and operating a data graph.

INTEGRITY PRINCIPLES
AGILITY PRINCIPLES
OPERATIONS PRINCIPLES

A must read for every GraphQL enthousiast!

    Thursday, September 12, 2019

    ElasticSearch - Exposing your ElasticSearch instance

    ElasticSearch is safe out-of-the-box. If you do a new installation, your ElasticSearch instance is only listening to internal traffic. If you want to make the ElasticSearch API’s accessible outside the VM where you installed it, you have to take some extra steps:

    • Go to the ElasticSearch folder that you configured to store your index and configuration data.
    • Open the ElasticSearch.yml file inside the config folder.
    • Set the network.host value to a non-loopback address. I’ll use 0.0.0.0 but a specific IP address is of course better:

    network.host: 0.0.0.0

    If you know restart your ElasticSearch instance, you’ll notice that we are not there yet. The node fails to start with the following error message:

    [2019-07-10T14:34:59,782][INFO ][o.e.d.DiscoveryModule    ] [ESSRV1] using discovery type [zen] and seed hosts providers [settings]

    [2019-07-10T14:35:00,436][INFO ][o.e.n.Node               ] [ESSRV1] initialized

    [2019-07-10T14:35:00,436][INFO ][o.e.n.Node               ] [ESSRV1] starting ...

    [2019-07-10T14:35:00,634][INFO ][o.e.t.TransportService   ] [ESSRV1] publish_address {10.1.100.236:9300}, bound_addresses {[::]:9300}

    [2019-07-10T14:35:00,643][INFO ][o.e.b.BootstrapChecks    ] [ESSRV1] bound or publishing to a non-loopback address, enforcing bootstrap checks

    [2019-07-10T14:35:00,647][ERROR][o.e.b.Bootstrap          ] [ESSRV1] node validation exception

    [1] bootstrap checks failed

    [1]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured

    [2019-07-10T14:35:00,650][INFO ][o.e.n.Node               ] [ESSRV1] stopping ...

    [2019-07-10T14:35:00,666][INFO ][o.e.n.Node               ] [ESSRV1] stopped

    [2019-07-10T14:35:00,666][INFO ][o.e.n.Node               ] [ESSRV1] closing ...

    [2019-07-10T14:35:00,678][INFO ][o.e.n.Node               ] [ESSRV1] closed

    The moment you provided a custom setting for network.host, Elasticsearch assumes that you are moving from development mode to production mode, and upgrades a number of system startup checks from warnings to exceptions. So we have to configure one extra setting; the initial_master_nodes:

    cluster.initial_master_nodes: node-1

    Azure DevOps Server–Error publishing nuget packages

    As part of our build process we package and push a set of nuget packages to our artifacts repository. Recently we made the switch to Azure Artifacts after which uploading packages started to fail with the following error message:

    The nuget command failed with exit code(1) and error(Response status code does not indicate success: 409 (Conflict - The feed already contains ‘ExceptionHandling.Database 6.4.1'. (DevOps Activity ID: CAEC4462-2E20-4C69-9AFE-3BBC3C961E20)).)

    Packages failed to publish

    Ok, it seems that Azure Artifacts doesn’t like it when you try to upload the same packages multiple times. (We bumped the package version manually and didn’t care about overwriting an existing package so far). Can we fix this?

    We are using the NuGet task. This task has an interesting option: ‘Allow duplicates to be skipped’. Let’s try this…

    Unfortunately we ended up with the same error message as above. Let’s take a look at the documentation:

    If you continually publish a set of packages and only change the version number of the subset of packages that changed, use this option. It allows the task to report success even if some of your packages are rejected with 409 Conflict errors.

    This option is currently only available on Azure Pipelines and using Windows agents. If NuGet.exe encounters a conflict, the task will fail.

    Aha, the last line explains everything. We are using Azure DevOps Server and it seems that this option only works in the cloud(no idea why?).

    As a workaround I switched to a commandline task and invoked nuget.exe directly. If you are using a recent nuget.exe version you can add the –skipduplicate option to skip existing packages.

    Wednesday, September 11, 2019

    ElasticSearch - Cache aggregations and suggestions

    A quick tip for everyone using ElasticSearch(aren’t you all?);

    It can be a good idea to separate the search request that fetches the results from the search request that returns the aggregations or suggestions. Why would you do that?

    Caching of course! On each shard the request cache module caches the local search results. But(!) by default the request cache module only caches the results of search requests where size=0.  This means that it only works when you don’t want any hits returned.

    We tested it on our development cluster and the difference in performance is significant.

    More information: https://www.elastic.co/guide/en/elasticsearch/reference/current/shard-request-cache.html

    Tuesday, September 10, 2019

    XUnit - Async lifetime

    The setup of your test context in XUnit is typically done through the constructor. For context cleanup, you can add the IDisposable interface to your test class, and put the cleanup code in the Dispose() method.

    But what if your setup/teardown logic contains some async methods? It would certainly be an anti-pattern to add this code inside your synchronous constructor or Dispose.

    The correct way to do this in XUnit is through the IAsyncLifetime interface:

    This interface provides you an async alternative to manage the lifetime of your test context.  It can be used inside your test classes directly but also works in Class and Collection fixtures.

    Monday, September 9, 2019

    Team Foundation Server–402 (Payment Required)

    As part of our build process we push some of our artifacts as NuGet packages to Team Foundation Server.

    After adding a new build agent, executing the push failed with the following log output:

    2019-09-02T13:30:07.1191540Z ##[section]Starting: dotnet push

    2019-09-02T13:30:07.1191540Z ==============================================================================

    2019-09-02T13:30:07.1191540Z Task         : .NET Core

    2019-09-02T13:30:07.1191540Z Description  : Build, test, package, or publish a dotnet application, or run a custom dotnet command. For package commands, supports NuGet.org and authenticated feeds like Package Management and MyGet.

    2019-09-02T13:30:07.1191540Z Version      : 2.131.0

    2019-09-02T13:30:07.1191540Z Author       : Microsoft Corporation

    2019-09-02T13:30:07.1191540Z Help         : [More Information](https://go.microsoft.com/fwlink/?linkid=832194)

    2019-09-02T13:30:07.1191540Z ==============================================================================

    2019-09-02T13:30:07.5722786Z SYSTEMVSSCONNECTION exists true

    2019-09-02T13:30:07.6035200Z Saving NuGet.config to a temporary config file.

    2019-09-02T13:30:07.6035200Z Saving NuGet.config to a temporary config file.

    2019-09-02T13:30:07.6035200Z [command]"C:\Program Files\dotnet\dotnet.exe" nuget push F:\Build\_work\12\s\Hub\Hub.Interfaces\bin\Development\Hub.Interfaces.1.0.0.nupkg --source http://tfs:8080/tfs/DefaultCollection/_packaging/892779dc-d854-4c9f-8b26-833d52585ae6/nuget/v3/index.json --api-key VSTS

    2019-09-02T13:30:07.9941927Z error: Unable to load the service index for source http://tfs:8080/tfs/DefaultCollection/_packaging/892779dc-d854-4c9f-8b26-833d52585ae6/nuget/v3/index.json.

    2019-09-02T13:30:07.9941927Z error:   Response status code does not indicate success: 402 (Payment Required - The user does not have a license for the extension ms.feed. (TFS Activity ID: ACE122BD-49E4-4927-B927-1C2A37F8D18B)).

    2019-09-02T13:30:08.0253954Z ##[error]Error: C:\Program Files\dotnet\dotnet.exe failed with return code: 1

    2019-09-02T13:30:08.0253954Z ##[error]Packages failed to publish

    2019-09-02T13:30:08.0253954Z ##[section]Finishing: dotnet push

    The problem is that this new agent is running using a service account that isn’t know inside Team Foundation Server. Let’s fix this…

    • Browse to your Team Foundation Server collection
    • On the Start page click on the Gear icon and select Users from the dropdown

    • On the Users tab, select Package Management on the left.

    • Afterwards click on Assign to add extra users.

    Remark: The steps above apply to Team Foundation Server not to Azure DevOps(the latest incarnation of TFS)

    Thursday, September 5, 2019

    FluentValidation–Validating collections

    On one of my projects we are using FluentValidation. After upgrading to FluentValidation 8 the way we had to validate our collections changed.

    Before we were using this:

    Now we had to use the RuleForEach method:

    But what if you also want to validate if the collection itself is empty or not. This is still possible through RuleFor:

    Wednesday, September 4, 2019

    Entity Framework Core Query Objects

    Although the name suggests otherwise you are not limited to using entity types (=objects with an id) in EF Core. EF Core 2.1 introduces support for query types, which can be used to query data that isn’t mapped to entity types.

    This allows you to directly map your query results to DTO’s, something especially useful on the query side when you are having a CQRS style architecture.

    Let’s try this!

    First we have to create our DTO:

    Next step is to make EF Core aware of the existence of this query type. We’ll use the fluent configuration api but instead of calling the Entity<T>() method on the modelBuilder we’ll use the Query<T>() method:

    Next, we configure the DbContext to include the DbQuery<T>:

    You can then define queries directly in OnModelBuilding using a LINQ query through ToQuery():

    Of course you can also map a query type to a view using ToView():

    A last option is to directly map it to a raw SQL query using the FromSql method on your DbContext:

    Remark: The query type must be registered with the DbContext as a DbQuery and all columns in the SQL need to match all of the properties in the query type.

    More information: https://docs.microsoft.com/en-us/ef/core/modeling/query-types

    Tuesday, September 3, 2019

    NSubstitute Roslyn Analyzers

    Quick tip if you are using NSubstitute (like me) as your mocking library. Certainly don’t forget to include the NSubstitute Roslyn analyzers as well.

    This allows you to detect some cases where NSubstitute will fail at runtime (e.g. when you attempt to mock non-virtual members).

    For the full list of rules that are validated, take a look at the documentation:

    https://github.com/nsubstitute/NSubstitute.Analyzers/tree/master/documentation

    Monday, September 2, 2019

    .NET Memory management posters

    If you finally want to be able to answer the dreaded “explain memory management in .NET” interview question, I have a solution for you…

    Take a look at the .NET Memory Management posters created by Konrad Kokosa: