Last week we had a production issue at one of my customers where a server went offline due to high memory usage. So far the bad news. The good news is that we had Azure Application Insights monitoring in place, so we could easily validate that the high memory usage was causing the issue as our Application Insight logs showed a long list of OutOfMemoryException
s.
However as a separate Application Insights instance was used per application, we couldn’t easily pinpoint which application was the main culprit.
Remark: Unfortunately it isn’t possible to show multiple resources on the Metrics tab, so that is not an option(you can upvote the feature if you like it):
I could go through every Application Insights resource one by one, but that wouldn’t be very efficient. Therefore I decided to turn to KQL and write a query on top of the Log Analytics workspace where all the data was consolidated.
Here is the query I used in the end:
And here is how the result looked like when I executed the query. I think it is clear that we found the culprit.
Time to further investigate…
More information
Overview of Log Analytics in Azure Monitor - Azure Monitor | Microsoft Learn