Skip to main content

Posts

Showing posts with the label Azure

Upgrading Dependency Track from v4 to v5

Dependency Track is an open source component analysis platform from OWASP. You upload the SBOM of your application, and Dependency Track keeps track of the components inside it. It checks those components against vulnerability sources like the National Vulnerability Database, GitHub Advisories and OSV. It also lets you define policies and send notifications when something new shows up. In short: it tells you which of your applications are affected when the next vulnerable library hits the news. Recently Dependency Track got an upgrade and version 5 was released. So, time to upgrade! However, that turned out not to be as easy as expected. It took us 2 attempts. Our first attempt failed completely, so we took a different route. Here is what we tried and the two things that cost us the most time. Big shout out to Jef, who looked over my shoulder during the upgrade and helped tackling the issues when we got stuck. Two approaches in the documentation The Dependency Track documenta...

Enabling the Application Insights Profiler

Note: this is part 2 of a series on the Azure Monitor Profiler. Part 1 covered what the Profiler is and where to find the results - this post covers actually turning it on. There are two ways to enable the profiler: through app settings on App Service, or by wiring it into your code directly. Which one you need depends on where your app runs and how much control you want over the setup. Option 1: codeless enablement on App Service If your app runs on App Service (Windows) and your Application Insights resource is in the same subscription, this is the easiest path - no code changes, no redeploy. From the portal: In your App Service instance, select Monitoring > Application Insights Select Turn on Application Insights , then Enable Scroll down to the .NET or .NET Core tab Set Collection level to Recommended Under Profiler and Code Optimizations , select On Apply , then confirm with Yes Or skip the portal entirely and set the app settings directly...

Our P95 spiked, now what?

Yesterday one of our Application Insights dashboard showed a P95 latency spike and we had no idea why. The telemetry told us that a request was slow. But it didn't tell us why . Was it a database call? A CPU-bound loop? Lock contention? A GC pause? The naive approach would have to been to add logging statements around the code you suspect, redeploy, wait for the issue to reproduce, and repeat. In production, that's slow and it doesn't scale - you're guessing, and every guess costs a deployment cycle. We stayed away from all that guess work and reached out to the Application Insights Profiler . Instead of reasoning from logs, you get actual flame graphs of real production requests, showing exactly where time was spent. What the profiler actually captures The profiler runs as an agent alongside your application and periodically captures traces of live requests - not synthetic load, actual production traffic. For each captured request, it builds a trace you can in...

Fixing "Selected tag uses an invalid operating system " error when deploying to Azure Container Apps

Yesterday I tried deploying a Docker image to Azure Container Apps and hit a wall with a cryptic error about an invalid operating system.  It took me a lot of time to find the root cause and fix it.Here's what happened, why it happens, and the exact command to fix it. The error After pushing an image to Azure Container Registry and pointing a Container App at it, the deployment failed with this message: Selected tag uses an invalid operating system ''. Error when deploying an Azure Container App image. The error is frustrating because it gave me almost nothing to work with; an empty string where the OS name should be. The image built and pushed just fine, so what's going on? Root cause The culprit is provenance attestation — a feature that Docker BuildKit enables by default when using docker buildx . When provenance is enabled, Docker generates an extra manifest layer containing build metadata. This results in a multi-platform image manifest (an OCI image i...

Python vs PySpark notebooks in MS Fabric

Being new to Microsoft Fabric I noticed that you have multiple options when writing notebooks using Python: run your code with PySpark (backed by a Spark cluster) or with Python (running natively on the notebook's compute). Both options look almost identical on the surface — you're still writing Python syntax either way — but under the hood they behave very differently, and picking the wrong one can cost you time, money, and unnecessary complexity. In this post I try to identify the key differences and give you some heuristics for deciding which engine to reach for. Python vs PySpark: what's actually different? When you select PySpark in a Fabric notebook, your code runs on a distributed Apache Spark cluster. Fabric spins up a cluster, distributes your data across multiple worker nodes, and executes transformations in parallel. The core abstraction is the DataFrame (or RDD), and operations are lazy — nothing actually runs until you trigger an action like .show() ...

How to work with OneLake files locally using Python

Last week I shared how you could use the  OneLake File Explorer to sync your Lakehouse tables to your local machine. It's a convenient way to get your Parquet and Delta Lake files off the cloud and onto disk — but what do you actually do with them once they're there? In this post, I’ll walk you through how to interact with your locally synced OneLake files using Python. We'll cover four practical approaches, with real code you can drop straight into a notebook. Where are your files? When OneLake File Explorer syncs your files, they land in a path that looks something like this: C:\Users\<you>\OneLake - <workspace name>\<lakehouse name>.Lakehouse\Tables\<table name> Keep that path in mind— you'll be passing it into every example below. Delta Lake tables are stored as folders containing multiple Parquet files plus a _delta_log/ directory, so make sure you're pointing at the table's root folder, not an individual file. Readin...

Accessing Microsoft Fabric data locally with OneLake file explorer

If you've spent any time working with Microsoft Fabric, you know that navigating to the web portal every time you need to inspect, upload, or tweak a file gets old fast. OneLake File Explorer is Microsoft's answer to that friction — a lightweight Windows application that mounts your entire Fabric data estate directly in Windows File Explorer, the same way OneDrive handles your documents. One..what? OneLake is the unified data lake underpinning every Microsoft Fabric tenant. Unlike traditional architectures where teams maintain separate data lakes per domain or business unit, every Fabric tenant gets exactly one OneLake — one place where Lakehouses, Warehouses, KQL databases, and other Fabric items store their data. There's no need to copy data between engines; Spark, SQL, and Power BI all read from the same underlying storage. The organizational hierarchy is straightforward: Tenant → Workspaces → Items (Lakehouses, Warehouses, etc.) → Files/Tables . This maps neat...

How I built a custom agent skill to configure Application Insights

If you've ever found yourself repeating the same Azure setup ritual — adding the Application Insights SDK, wiring up telemetry, configuring sampling rules — you already know the pain. It's not hard, but it's tedious. Every new service needs the same scaffolding. Every new team member has to learn the same conventions. That's exactly what I solved with a custom skill. Now, when I need to instrument a service, I just tell Copilot to configure Application Insights, and it does everything exactly the way our team expects. No extra prompting, no re-explaining our conventions. It just works. This post explains what Skills are, how they work inside VS Code, and how to build one for your own team — using my Application Insights skill as a hands-on example. What is an agent skill? An agent skill is a folder of instructions, scripts, and reference files that teaches your AI agent how to handle a specific task. Think of it as institutional knowledge made executable. Instea...

Building an end-to-end monitoring solution with Azure Arc, Log Analytics and Workbooks - Part 5: Putting it all together

Wow! We covered a lot in this series. Part 1 - Overview & Architecture Part 2 – Data collection with Azure Arc Part 3 – Data persistence in Log Analytics Part 4 -  Data visualization with Azure Workbooks Time for a wrap up and some troubleshooting Let's trace the data flow from start to finish to make sure everything connects: The Azure Monitor Agent runs on each Arc-enabled on-prem VM. The Data Collection Rule tells the agent what health data to gather — application pools, Windows services, and scheduled tasks. The agent collects that data on a regular interval and ships it to Azure. The DCR routes the incoming data to our custom table ( OnPremHealthStatus_CL ) in the Log Analytics Workspace. The Workbook queries that table and renders the dashboard. If any link in that chain breaks, data stops flowing. The troubleshooting section below covers the most common failure points. Troubleshooting checklist No data appearing in the workbook: ...

Building an end-to-end monitoring solution with Azure Arc, Log Analytics and Workbooks–Part 4: Data visualisation with Azure Workbooks

In part 1 I explained that we want to setup an application health dashboard to gain insights on the availability and health of the on-premise parts of our applications. Specifically, we want to monitor our application pools, scheduled tasks and windows services. I introduced the overall architecture and explained the building blocks. Part 2 was all about the data collection part using Azure Arc Data Collection rules. I continued in Part 3 with our custom table in Log Analytics to persist our data. And today it is time for Part 4 were I share how visualize all this info using Azure Workbooks. What we're visualizing The workbook is the user-facing piece. Our goal is a dashboard that lets an operator quickly answer three questions: What's running? What's stopped or failed? Which machines need attention? A good health dashboard has two modes: the "glance" mode where an operator can immediately see if anything is wrong, and the "investigate" mod...

Building an end-to-end monitoring solution with Azure Arc, Log Analytics and Workbooks–Part 3: Data persistence in Log Analytics

In part 1 I explained that we want to setup an application health dashboard to gain insights on the availability and health of the on-premise parts of our applications. Specifically we want to monitor our application pools, scheduled tasks and windows services. I introduced the overall architecture and explained the building blocks. Part 2 was all about the data collection part using Azure Arc Data Collection rules. Today I’ll focus on how we used a custom table in Log Analytics to persist our data. Why a custom table The built-in Windows event logs in Log Analytics (the Event table) contain a lot of data, but the format isn't optimized for health-status queries. Parsing event log XML to extract service states or scheduled task results on every query adds latency and complexity. When you query the Event table for service state changes, you're filtering through thousands of rows, parsing semi-structured XML from the EventData column, and then correlating multiple ev...

Building an end-to-end monitoring solution with Azure Arc, Log Analytics and Workbooks–Part 2: Data collection with Azure Arc

In part 1 I explained that we want to setup an application health dashboard to gain insights on the availability and health of the on-premise parts of our applications. Specifically we want to monitor our application pools, scheduled tasks and windows services. I introduced the overall architecture and explained the building blocks. Today we'll dive in the first one of these blocks; the data collection part using Azure Arc Data Collection rules. Understanding Data Collection rules A Data Collection Rule (DCR) is a declarative configuration object in Azure that defines the full lifecycle of telemetry: what to collect, how to transform it, and where to send it. It's the connective tissue between the Azure Monitor Agent running on your VMs and the Log Analytics Workspace where the data lands. DCRs replaced the older model where agents were configured locally via XML files. The new model is centralized — you define the DCR in Azure, associate it with your VMs, and the agent...