Last week I blogged about how you can run dotnet monitor as a sidecar in your Kubernetes cluster. Although the yaml file I shared worked on my local cluster (inside Minikube), it didn’t work when I tried to deploy it to AKS. Nothing happened when I tried to connect to the specified URL’s.
To fix this I had to take multiple steps:
- First I had to explicitly set the ‘—urls’ argument inside the manifest:
apiVersion: apps/v1 | |
kind: Deployment | |
metadata: | |
name: dotnet-monitor-example | |
spec: | |
replicas: 1 | |
selector: | |
matchLabels: | |
app: dotnet-monitor-example | |
template: | |
metadata: | |
labels: | |
app: dotnet-monitor-example | |
spec: | |
volumes: | |
- name: diagnostics | |
emptyDir: {} | |
containers: | |
- name: server | |
image: mcr.microsoft.com/dotnet/core/samples:aspnetapp | |
ports: | |
- containerPort: 80 | |
volumeMounts: | |
- mountPath: /tmp | |
name: diagnostics | |
- name: sidecar | |
image: mcr.microsoft.com/dotnet/monitor:5.0.0-preview.5 | |
ports: | |
- containerPort: 52323 | |
args: ["--urls", "http://*:52323", "--metricUrls", "http://*:52325"] | |
volumeMounts: | |
- name: diagnostics | |
mountPath: /tmp |
- Now I was able to connect to the url but it still failed. When I took a look at the logs I noticed the following message:
{"Timestamp":"2021-07-27T18:48:29.6522095Z","EventId":7,"LogLevel":"Information","Category":"Microsoft.Diagnostics.Tools.Monitor.ApiKeyAuthenticationHandler","Message":"MonitorApiKey was not authenticated. Failure message: API key authentication not configured.","State":{"Message":"MonitorApiKey was not authenticated. Failure message: API key authentication not configured.","AuthenticationScheme":"MonitorApiKey","FailureMessage":"API key authentication not configured.","{OriginalFormat}":"{AuthenticationScheme} was not authenticated. Failure message: {FailureMessage}"},"Scopes":[{"Message":"ConnectionId:0HMAH5RL3D6BM","ConnectionId":"0HMAH5RL3D6BM"},{"Message":"RequestPath:/processes RequestId:0HMAH5RL3D6BM:00000001, SpanId:|fe3ec0c2-46980a5b9b2602e2., TraceId:fe3ec0c2-46980a5b9b2602e2, ParentId:","RequestId":"0HMAH5RL3D6BM:00000001","RequestPath":"/processes","SpanId":"|fe3ec0c2-46980a5b9b2602e2.","TraceId":"fe3ec0c2-46980a5b9b2602e2","ParentId":""}]}
- We need to create an API key secret and mount it as a volume to our sidecar. Here is the code to generate a secret:
kubectl create secret generic apikey \ --from-literal=ApiAuthentication__ApiKeyHash=$hash \ --from-literal=ApiAuthentication__ApiKeyHashType=SHA256 \ --dry-run=client -o yaml \ | kubectl apply -f -
- Now we need to mount the secret as a volume. Here is the updated manifest:
apiVersion: apps/v1 | |
kind: Deployment | |
metadata: | |
name: dotnet-monitor-example | |
spec: | |
replicas: 1 | |
selector: | |
matchLabels: | |
app: dotnet-monitor-example | |
template: | |
metadata: | |
labels: | |
app: dotnet-monitor-example | |
spec: | |
volumes: | |
- name: diagnostics | |
emptyDir: {} | |
- name: apikey | |
secret: | |
secretName: apikey | |
containers: | |
- name: server | |
image: mcr.microsoft.com/dotnet/core/samples:aspnetapp | |
ports: | |
- containerPort: 80 | |
volumeMounts: | |
- mountPath: /tmp | |
name: diagnostics | |
- name: sidecar | |
image: mcr.microsoft.com/dotnet/monitor:5.0.0-preview.5 | |
ports: | |
- containerPort: 52323 | |
args: ["--urls", "http://*:52323", "--metricUrls", "http://*:52325"] | |
volumeMounts: | |
- name: diagnostics | |
mountPath: /tmp | |
- name: apikey | |
mountPath: /etc/dotnet-monitor |
If you want to learn more, I could recommend the following video as a good introduction: