Monday, September 30, 2019

ASP.NET Core 3.0–Swashbuckle error

As probably most people in the .NET ecosystem I’m using Swashbuckle to generate my OpenAPI documentation. (Anyone using NSwag instead?)

After upgrading to ASP.NET Core 3.0 (and switching to the 5.0-rc4 prerelease version of Swashbuckle), the following code no longer compiled:

I had to replace the Info class which couldn’t be found by the OpenApiInfo class:

This OpenApiInfo class is now part of the Microsoft.OpenApi.Models namespace.

Friday, September 27, 2019

C# 8.0 Nullable reference types for framework and library authors

C# 8.0 introduces the concept of nullable reference types. If you are a framework or library author, introducing this functionality in your code is not without its challenges.

A good introduction can be found here:

Thursday, September 26, 2019

Setup the Kubernetes dashboard on Docker for Windows

A useful tool when you are new to Kubernetes is the Kubernetes Dashboard.

Unfortunately the Kubernetes Dashboard is not included out-of-the-box with Docker for Windows however it can be easily setup for your local cluster.

To setup the dashboard use the following command:

kubectl apply -f 
https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml 
The output should look like this:

secret/kubernetes-dashboard-certs created
serviceaccount/kubernetes-dashboard created
role.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created
deployment.apps/kubernetes-dashboard created
service/kubernetes-dashboard created

To view the Dashboard in your web browser run:

kubectl proxy

And nagivate to your Kubernetes Dashboard at:

http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/.

The first time you open the dashboard you have to enter an authentication token.

Creating an authorization token

To create an authorization token, we first need a Service Account:

kubectl create serviceaccount admin-user

Now we need a clusterrolebinding to assign the ClusterRole to the created admin-user:

kubectl create clusterrolebinding cluster-admin-binding --clusterrole cluster-admin --user admin-user

Check an associated secret:

kubectl get serviceaccounts admin-user -o yaml

As a last step we need to request the secret data:

kubectl describe secret admin-user-token-5s5rv

Now you can copy and past the generated token in the Token field.

Wednesday, September 25, 2019

Switch between Kubernetes contexts

Lost some time yesterday figuring out how to switch between different Kubernetes environments. So a quick post, just as a reminder for myself:

You can view contexts using the kubectl config command:

kubectl config get-contexts

CURRENT  NAME                            CLUSTER                    NAMESPACE           
*         docker-desktop                   docker-desktop             docker-desktop                                   
          docker-for-desktop               docker-desktop             docker-desktop

You can set the context by specifying the context name:

kubectl config use-context docker-for-desktop

Tuesday, September 24, 2019

ElasticSearch–Performance testing

When trying to load test our ElasticSearch cluster, we noticed big variations in results that we couldn’t explain based on the changes we made.

Turned out that our tests were not executed in comparable situations as we didn’t clear the ElasticSearch cache.

So before running our tests, we cleared the cache using following command:

POST /<myindexname>/_cache/clear?request=true

If you want to view what’s inside the Elastic node cache, you can use the following command::

GET /_cat/nodes?v&h=id,name,queryCacheMemory,queryCacheEvictions,requestCacheMemory,requestCacheHitCount,requestCacheMissCount,flushTotal,flushTotalTime

Monday, September 23, 2019

GraphQL Rules

As with every technology you give to your team everyone has different opinions and conventions. A style guide becomes an indispensible part of your development organisation. Otherwise the ‘tabs vs spaces’ discussion can go on forever.

This also applies to GraphQL. So to help you get started take a look at https://graphql-rules.com/.

Rules and recommendations mentioned here were the results of 3 years' experience of using GraphQL both on the frontend and backend sides. We also include the recommendations and experience of Caleb Meredith (PostGraphQL author, Facebook ex-employee) and Shopify engineers.

This guide is intended to be open source and could change in the future, - the rules may be improved on, changed, or even become outdated. What is written here is a culmination of time and pain suffered from the use of horrible GraphQL-schemas.

Friday, September 20, 2019

Cannot create or delete the Performance Category 'C:\Windows\TEMP\tmp3DA0.tmp' because access is denied.

After migrating some .NET applications from an old server to a brand new Windows Server 2019 instance, we stumbled over a range of errors.

Yesterday we got one step closer to a solution but we are not there yet. The application still doesn’t work but now we get the following error message:

Server Error in '/AppServices' Application.


Cannot create or delete the Performance Category 'C:\Windows\TEMP\tmp3DA0.tmp' because access is denied.

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.UnauthorizedAccessException: Cannot create or delete the Performance Category 'C:\Windows\TEMP\tmp3DA0.tmp' because access is denied.

ASP.NET is not authorized to access the requested resource. Consider granting access rights to the resource to the ASP.NET request identity. ASP.NET has a base process identity (typically {MACHINE}\ASPNET on IIS 5 or Network Service on IIS 6 and IIS 7, and the configured application pool identity on IIS 7.5) that is used if the application is not impersonating. If the application is impersonating via <identity impersonate="true"/>, the identity will be the anonymous user (typically IUSR_MACHINENAME) or the authenticated request user.

To grant ASP.NET access to a file, right-click the file in File Explorer, choose "Properties" and select the Security tab. Click "Add" to add the appropriate user or group. Highlight the ASP.NET account, and check the boxes for the desired access.

Source Error:

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.


Stack Trace:

 

[UnauthorizedAccessException: Cannot create or delete the Performance Category 'C:\Windows\TEMP\tmp3DA0.tmp' because access is denied.]

   System.Diagnostics.PerformanceCounterLib.RegisterFiles(String arg0, Boolean unregister) +482

   System.Diagnostics.PerformanceCounterLib.RegisterCategory(String categoryName, PerformanceCounterCategoryType categoryType, String categoryHelp, CounterCreationDataCollection creationData) +105

   System.Diagnostics.PerformanceCounterCategory.Create(String categoryName, String categoryHelp, PerformanceCounterCategoryType categoryType, CounterCreationDataCollection counterData) +275

   Akka.Monitoring.PerformanceCounters.ActorPerformanceCountersMonitor.Init(IEnumerable`1 akkaMetrics) in D:\olympus\akka-monitoring\src\Akka.Monitoring.PerformanceCounters\ActorPerformanceCountersMonitor.cs:116

 

[TargetInvocationException: Exception has been thrown by the target of an invocation.]

   System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) +0

   System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments) +128

   System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) +142

   Owin.Loader.<>c__DisplayClass12.<MakeDelegate>b__b(IAppBuilder builder) +93

   Owin.Loader.<>c__DisplayClass1.<LoadImplementation>b__0(IAppBuilder builder) +212

   Microsoft.Owin.Host.SystemWeb.OwinAppContext.Initialize(Action`1 startup) +873

   Microsoft.Owin.Host.SystemWeb.OwinBuilder.Build(Action`1 startup) +51

   Microsoft.Owin.Host.SystemWeb.OwinHttpModule.InitializeBlueprint() +101

   System.Threading.LazyInitializer.EnsureInitializedCore(T& target, Boolean& initialized, Object& syncLock, Func`1 valueFactory) +135

   Microsoft.Owin.Host.SystemWeb.OwinHttpModule.Init(HttpApplication context) +160

   System.Web.HttpApplication.RegisterEventSubscriptionsWithIIS(IntPtr appContext, HttpContext context, MethodInfo[] handlers) +581

   System.Web.HttpApplication.InitSpecial(HttpApplicationState state, MethodInfo[] handlers, IntPtr appContext, HttpContext context) +168

   System.Web.HttpApplicationFactory.GetSpecialApplicationInstance(IntPtr appContext, HttpContext context) +277

   System.Web.Hosting.PipelineRuntime.InitializeApplication(IntPtr appContext) +369

 

[HttpException (0x80004005): Exception has been thrown by the target of an invocation.]

   System.Web.HttpRuntime.FirstRequestInit(HttpContext context) +532

   System.Web.HttpRuntime.EnsureFirstRequestInit(HttpContext context) +111

   System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context) +714

I couldn’t find a ‘good’ solution to get rid of the error message above. In the end I did 2 things to get rid of the error:

  1. I temporarily gave the application pool accountAdministrator rights on the server.
  2. I gave the application pool account ‘Write’ permissions on the following registry path: Computer\HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\

After creating the performance counters on startup, I revoked these permissions.

Anyone with a better solution?

Thursday, September 19, 2019

Access to the registry key 'Global' is denied.

After migrating some .NET applications from an old server to a brand new Windows Server 2019 instance, we stumbled over a range of errors.

Here is the first error we got:

Server Error in '/AppServices' Application.


Access to the registry key 'Global' is denied.

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.UnauthorizedAccessException: Access to the registry key 'Global' is denied.

ASP.NET is not authorized to access the requested resource. Consider granting access rights to the resource to the ASP.NET request identity. ASP.NET has a base process identity (typically {MACHINE}\ASPNET on IIS 5 or Network Service on IIS 6 and IIS 7, and the configured application pool identity on IIS 7.5) that is used if the application is not impersonating. If the application is impersonating via < identity impersonate="true"/>, the identity will be the anonymous user (typically IUSR_MACHINENAME) or the authenticated request user.

To grant ASP.NET access to a file, right-click the file in File Explorer, choose "Properties" and select the Security tab. Click "Add" to add the appropriate user or group. Highlight the ASP.NET account, and check the boxes for the desired access.

Source Error:

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.


Stack Trace:

 

[UnauthorizedAccessException: Access to the registry key 'Global' is denied.]

   Microsoft.Win32.RegistryKey.Win32Error(Int32 errorCode, String str) +144

   Microsoft.Win32.RegistryKey.InternalGetValue(String name, Object defaultValue, Boolean doNotExpand, Boolean checkSecurity) +640

   Microsoft.Win32.RegistryKey.GetValue(String name) +62

   System.Diagnostics.PerformanceMonitor.GetData(String item) +121

   System.Diagnostics.PerformanceCounterLib.GetPerformanceData(String item) +177

   System.Diagnostics.PerformanceCounterLib.get_CategoryTable() +138

   System.Diagnostics.PerformanceCounterLib.CategoryExists(String machine, String category) +72

   System.Diagnostics.PerformanceCounterCategory.Exists(String categoryName, String machineName) +155

   Akka.Monitoring.PerformanceCounters.ActorPerformanceCountersMonitor.Init(IEnumerable`1 akkaMetrics) in D:\olympus\akka-monitoring\src\Akka.Monitoring.PerformanceCounters\ActorPerformanceCountersMonitor.cs:102

 

[HttpException (0x80004005): Exception has been thrown by the target of an invocation.]

   System.Web.HttpRuntime.FirstRequestInit(HttpContext context) +532

   System.Web.HttpRuntime.EnsureFirstRequestInit(HttpContext context) +111

   System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context) +714

 


Version Information: Microsoft .NET Framework Version:4.0.30319; ASP.NET Version:4.7.3429.0

 

What’s happening here?

As part of our application initialization, we try to create some performance counters for Akka.Net. As the application is running under the AppPoolIdentity it has no rights to register the performance counters in the registry, explaining the ‘Access to the registry key 'Global' is denied. ‘ error message.

First step to fix this is to add the AppPool account to the Performance Monitor Users group on the server:

Unfortunately we are not there yet, but that is for the next post…

Wednesday, September 18, 2019

Microsoft Orleans - Configure .NET garbage collection

To increase the performance of your Orleans application it is important that you tweak the .NET garbage collection settings. According to the documentation the best settings is a combination of gcServer=true and gcConcurrent=true.

This can be done inside your csproj file by adding the following lines:

<PropertyGroup>
  <ServerGarbageCollection>true</ServerGarbageCollection>
  <ConcurrentGarbageCollection>true</ConcurrentGarbageCollection>
</PropertyGroup>

Remark: If the changed settings are not picked up immediately, deleting the bin/ and obj/ folders can help. (As mentioned here)

Tuesday, September 17, 2019

Missing compiler required member 'Microsoft.CSharp.RuntimeBinder.CSharpArgumentInfo.Create'

After introducing the dynamic keyword in my .NET core codebase, the project failed to compile with the following error message:

Error      CS0656 Missing compiler required member 'Microsoft.CSharp.RuntimeBinder.CSharpArgumentInfo.Create' SOFACore.Messaging              C:\Projects\SOFACore\SOFACore.Messaging\MessagingModule.cs   

To fix it I had to add the Microsoft.CSharp nuget package. This package is necessary to use the C# dynamic data type in a .NET Standard library.

Friday, September 13, 2019

Principled GraphQL

In case you didn’t notice it yet, I’m a big fan of GraphQL. It solves so many problems when dealing with an ever growing list of devices and (micro)services.

The people who created Apollo distilled their experiences into a set of best practices for creating, maintaining, and operating a data graph.

INTEGRITY PRINCIPLES
AGILITY PRINCIPLES
OPERATIONS PRINCIPLES

A must read for every GraphQL enthousiast!

    Thursday, September 12, 2019

    ElasticSearch - Exposing your ElasticSearch instance

    ElasticSearch is safe out-of-the-box. If you do a new installation, your ElasticSearch instance is only listening to internal traffic. If you want to make the ElasticSearch API’s accessible outside the VM where you installed it, you have to take some extra steps:

    • Go to the ElasticSearch folder that you configured to store your index and configuration data.
    • Open the ElasticSearch.yml file inside the config folder.
    • Set the network.host value to a non-loopback address. I’ll use 0.0.0.0 but a specific IP address is of course better:

    network.host: 0.0.0.0

    If you know restart your ElasticSearch instance, you’ll notice that we are not there yet. The node fails to start with the following error message:

    [2019-07-10T14:34:59,782][INFO ][o.e.d.DiscoveryModule    ] [ESSRV1] using discovery type [zen] and seed hosts providers [settings]

    [2019-07-10T14:35:00,436][INFO ][o.e.n.Node               ] [ESSRV1] initialized

    [2019-07-10T14:35:00,436][INFO ][o.e.n.Node               ] [ESSRV1] starting ...

    [2019-07-10T14:35:00,634][INFO ][o.e.t.TransportService   ] [ESSRV1] publish_address {10.1.100.236:9300}, bound_addresses {[::]:9300}

    [2019-07-10T14:35:00,643][INFO ][o.e.b.BootstrapChecks    ] [ESSRV1] bound or publishing to a non-loopback address, enforcing bootstrap checks

    [2019-07-10T14:35:00,647][ERROR][o.e.b.Bootstrap          ] [ESSRV1] node validation exception

    [1] bootstrap checks failed

    [1]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured

    [2019-07-10T14:35:00,650][INFO ][o.e.n.Node               ] [ESSRV1] stopping ...

    [2019-07-10T14:35:00,666][INFO ][o.e.n.Node               ] [ESSRV1] stopped

    [2019-07-10T14:35:00,666][INFO ][o.e.n.Node               ] [ESSRV1] closing ...

    [2019-07-10T14:35:00,678][INFO ][o.e.n.Node               ] [ESSRV1] closed

    The moment you provided a custom setting for network.host, Elasticsearch assumes that you are moving from development mode to production mode, and upgrades a number of system startup checks from warnings to exceptions. So we have to configure one extra setting; the initial_master_nodes:

    cluster.initial_master_nodes: node-1

    Azure DevOps Server–Error publishing nuget packages

    As part of our build process we package and push a set of nuget packages to our artifacts repository. Recently we made the switch to Azure Artifacts after which uploading packages started to fail with the following error message:

    The nuget command failed with exit code(1) and error(Response status code does not indicate success: 409 (Conflict - The feed already contains ‘ExceptionHandling.Database 6.4.1'. (DevOps Activity ID: CAEC4462-2E20-4C69-9AFE-3BBC3C961E20)).)

    Packages failed to publish

    Ok, it seems that Azure Artifacts doesn’t like it when you try to upload the same packages multiple times. (We bumped the package version manually and didn’t care about overwriting an existing package so far). Can we fix this?

    We are using the NuGet task. This task has an interesting option: ‘Allow duplicates to be skipped’. Let’s try this…

    Unfortunately we ended up with the same error message as above. Let’s take a look at the documentation:

    If you continually publish a set of packages and only change the version number of the subset of packages that changed, use this option. It allows the task to report success even if some of your packages are rejected with 409 Conflict errors.

    This option is currently only available on Azure Pipelines and using Windows agents. If NuGet.exe encounters a conflict, the task will fail.

    Aha, the last line explains everything. We are using Azure DevOps Server and it seems that this option only works in the cloud(no idea why?).

    As a workaround I switched to a commandline task and invoked nuget.exe directly. If you are using a recent nuget.exe version you can add the –skipduplicate option to skip existing packages.

    Wednesday, September 11, 2019

    ElasticSearch - Cache aggregations and suggestions

    A quick tip for everyone using ElasticSearch(aren’t you all?);

    It can be a good idea to separate the search request that fetches the results from the search request that returns the aggregations or suggestions. Why would you do that?

    Caching of course! On each shard the request cache module caches the local search results. But(!) by default the request cache module only caches the results of search requests where size=0.  This means that it only works when you don’t want any hits returned.

    We tested it on our development cluster and the difference in performance is significant.

    More information: https://www.elastic.co/guide/en/elasticsearch/reference/current/shard-request-cache.html

    Tuesday, September 10, 2019

    XUnit - Async lifetime

    The setup of your test context in XUnit is typically done through the constructor. For context cleanup, you can add the IDisposable interface to your test class, and put the cleanup code in the Dispose() method.

    But what if your setup/teardown logic contains some async methods? It would certainly be an anti-pattern to add this code inside your synchronous constructor or Dispose.

    The correct way to do this in XUnit is through the IAsyncLifetime interface:

    This interface provides you an async alternative to manage the lifetime of your test context.  It can be used inside your test classes directly but also works in Class and Collection fixtures.

    Monday, September 9, 2019

    Team Foundation Server–402 (Payment Required)

    As part of our build process we push some of our artifacts as NuGet packages to Team Foundation Server.

    After adding a new build agent, executing the push failed with the following log output:

    2019-09-02T13:30:07.1191540Z ##[section]Starting: dotnet push

    2019-09-02T13:30:07.1191540Z ==============================================================================

    2019-09-02T13:30:07.1191540Z Task         : .NET Core

    2019-09-02T13:30:07.1191540Z Description  : Build, test, package, or publish a dotnet application, or run a custom dotnet command. For package commands, supports NuGet.org and authenticated feeds like Package Management and MyGet.

    2019-09-02T13:30:07.1191540Z Version      : 2.131.0

    2019-09-02T13:30:07.1191540Z Author       : Microsoft Corporation

    2019-09-02T13:30:07.1191540Z Help         : [More Information](https://go.microsoft.com/fwlink/?linkid=832194)

    2019-09-02T13:30:07.1191540Z ==============================================================================

    2019-09-02T13:30:07.5722786Z SYSTEMVSSCONNECTION exists true

    2019-09-02T13:30:07.6035200Z Saving NuGet.config to a temporary config file.

    2019-09-02T13:30:07.6035200Z Saving NuGet.config to a temporary config file.

    2019-09-02T13:30:07.6035200Z [command]"C:\Program Files\dotnet\dotnet.exe" nuget push F:\Build\_work\12\s\Hub\Hub.Interfaces\bin\Development\Hub.Interfaces.1.0.0.nupkg --source http://tfs:8080/tfs/DefaultCollection/_packaging/892779dc-d854-4c9f-8b26-833d52585ae6/nuget/v3/index.json --api-key VSTS

    2019-09-02T13:30:07.9941927Z error: Unable to load the service index for source http://tfs:8080/tfs/DefaultCollection/_packaging/892779dc-d854-4c9f-8b26-833d52585ae6/nuget/v3/index.json.

    2019-09-02T13:30:07.9941927Z error:   Response status code does not indicate success: 402 (Payment Required - The user does not have a license for the extension ms.feed. (TFS Activity ID: ACE122BD-49E4-4927-B927-1C2A37F8D18B)).

    2019-09-02T13:30:08.0253954Z ##[error]Error: C:\Program Files\dotnet\dotnet.exe failed with return code: 1

    2019-09-02T13:30:08.0253954Z ##[error]Packages failed to publish

    2019-09-02T13:30:08.0253954Z ##[section]Finishing: dotnet push

    The problem is that this new agent is running using a service account that isn’t know inside Team Foundation Server. Let’s fix this…

    • Browse to your Team Foundation Server collection
    • On the Start page click on the Gear icon and select Users from the dropdown

    • On the Users tab, select Package Management on the left.

    • Afterwards click on Assign to add extra users.

    Remark: The steps above apply to Team Foundation Server not to Azure DevOps(the latest incarnation of TFS)

    Thursday, September 5, 2019

    FluentValidation–Validating collections

    On one of my projects we are using FluentValidation. After upgrading to FluentValidation 8 the way we had to validate our collections changed.

    Before we were using this:

    Now we had to use the RuleForEach method:

    But what if you also want to validate if the collection itself is empty or not. This is still possible through RuleFor:

    Wednesday, September 4, 2019

    Entity Framework Core Query Objects

    Although the name suggests otherwise you are not limited to using entity types (=objects with an id) in EF Core. EF Core 2.1 introduces support for query types, which can be used to query data that isn’t mapped to entity types.

    This allows you to directly map your query results to DTO’s, something especially useful on the query side when you are having a CQRS style architecture.

    Let’s try this!

    First we have to create our DTO:

    Next step is to make EF Core aware of the existence of this query type. We’ll use the fluent configuration api but instead of calling the Entity<T>() method on the modelBuilder we’ll use the Query<T>() method:

    Next, we configure the DbContext to include the DbQuery<T>:

    You can then define queries directly in OnModelBuilding using a LINQ query through ToQuery():

    Of course you can also map a query type to a view using ToView():

    A last option is to directly map it to a raw SQL query using the FromSql method on your DbContext:

    Remark: The query type must be registered with the DbContext as a DbQuery and all columns in the SQL need to match all of the properties in the query type.

    More information: https://docs.microsoft.com/en-us/ef/core/modeling/query-types

    Tuesday, September 3, 2019

    NSubstitute Roslyn Analyzers

    Quick tip if you are using NSubstitute (like me) as your mocking library. Certainly don’t forget to include the NSubstitute Roslyn analyzers as well.

    This allows you to detect some cases where NSubstitute will fail at runtime (e.g. when you attempt to mock non-virtual members).

    For the full list of rules that are validated, take a look at the documentation:

    https://github.com/nsubstitute/NSubstitute.Analyzers/tree/master/documentation

    Monday, September 2, 2019

    .NET Memory management posters

    If you finally want to be able to answer the dreaded “explain memory management in .NET” interview question, I have a solution for you…

    Take a look at the .NET Memory Management posters created by Konrad Kokosa: