Monitoring GitHub Copilot usage with the Aspire Dashboard

When talking to developers I get wildly different opinions about how "good" GitHub Copilot actually was for them. Some love it, some find it slow, some weren't sure it was doing anything useful at all(just kidding). The problem: nobody had any data. We were talking about token usage, latency and tool calls purely from gut feeling.

GitHub Copilot can export all of that as OpenTelemetry traces, metrics and events. And the easiest way to look at it locally, without spinning up a cloud backend, is the Aspire Dashboard. Let's set that up.

Why do we need this?

Copilot isn't just autocomplete anymore. Every agent turn is a small orchestration: it calls a model, the model asks for tools, the tools run, the model answers. If you want to know where the time and the tokens go, you need to see that orchestration, not just the end result.

That's exactly what the Open Telemetry integration gives you. Every agent interaction produces a span tree:

invoke_agent copilot                           [~15s]
  ├── chat gpt-4o                              [~3s]  (LLM requests tool calls)
  ├── execute_tool readFile                    [~50ms]
  ├── execute_tool runCommand                  [~2s]
  ├── chat gpt-4o                              [~4s]  (LLM generates final response)
  └── (span ends)

Remark: Open Telemetry is off by default. No data is emitted, and the SDK isn't even loaded, until you explicitly enable it. Nothing to worry about on the privacy side unless you turn it on yourself.

Getting the Aspire Dashboard running

There are multiple ways to collect and visualize the Open Telemetry data. An easy way in is the Aspire Dashboard. It is a single container with a built-in OTLP endpoint and a trace viewer. No Azure subscription, no collector to configure. Run it with the Aspire CLI:

aspire dashboard run

Or, if you'd rather run the container image directly:

docker run --rm -d -p 18888:18888 -p 4318:18890 --name aspire-dashboard \
  mcr.microsoft.com/dotnet/aspire-dashboard:latest

That's it. Open http://localhost:18888 and you'll land on an empty Traces view, waiting for data.

Pointing Copilot Chat at the dashboard

Now tell Copilot Chat to actually emit telemetry. Open your VS Code settings.json and add:

{

  "github.copilot.chat.otel.enabled": true,
  "github.copilot.chat.otel.captureContent": true
}

enabled turns telemetry on. captureContent is optional — it makes Copilot include the actual prompt and response text in the spans, instead of just token counts and durations. Handy while you're exploring locally, less handy on a shared machine.

Reload the window (settings changes don't apply to an already-running one), then have a normal chat conversation with Copilot. Ask it to read a file, run a command, whatever you'd do anyway.

Tip: enabling the setting alone doesn't emit anything. You need to actually send a chat message, and OTel batches before sending, so give it 10-30 seconds after your chat turn before you go looking in the dashboard.

Reading the traces

Back in http://localhost:18888, go to Traces.

You should see a trace per agent turn, named invoke_agent copilot (or invoke_agent claude if you're running Claude agent sessions through the Copilot Chat extension). Expand one and you get the full breakdown: how long the model call took, which tools ran, how long each of those took.

This is where the "Copilot feels slow" complaints get an actual answer. Sometimes it's the model call. Sometimes it's a tool — a slow runCommand, an MCP server doing something expensive — hiding behind what looks like one long agent turn.

The spans carry the GenAI semantic-convention attributes, so the same data works with any OTel backend later:

• gen_ai.usage.input_tokens / gen_ai.usage.output_tokens:  token spend per call
• gen_ai.usage.cache_read.input_tokens: how much you're benefiting from prompt caching
• gen_ai.response.model: which model actually answered (Copilot silently reroutes sometimes)
• gen_ai.tool.name: which tool ran inside execute_tool spans

Remark: if you're running the Copilot CLI in a separate terminal session rather than the VS Code chat panel, those traces show up as independent root traces under the github-copilot service instead of copilot-chat. They're not linked to your chat panel traces, but they land in the same dashboard, so filter by service.name if you want to tell them apart.

Going beyond gRPC vs HTTP

The default exporter is otlp-http, which is what the config above uses. If you'd rather use gRPC:

{
  "github.copilot.chat.otel.enabled": true,
  "github.copilot.chat.otel.exporterType": "otlp-grpc",
  "github.copilot.chat.otel.otlpEndpoint": http://localhost:18888
}

Remark: the Copilot CLI terminal runtime only speaks otlp-http, even if you configure gRPC. The Aspire Dashboard happens to serve both protocols on the same port, so this works transparently — you don't need two separate setups for chat-panel and CLI traces.

From local dashboard to team dashboard

The Aspire Dashboard is great for "let me quickly see what just happened on my machine," but it doesn't persist data across restarts and it's not meant for a whole team. Once you want aggregated numbers (token spend per team, cache hit rate over a sprint,...) you're looking at forwarding the same OTLP stream to Application Insights, Grafana, or another long-term backend through an OTel Collector. 

Same settings, different endpoint. That's maybe a topic for another post.

For now: enable the setting, point it at a local Aspire Dashboard, and actually look at what your agent sessions are doing. Simple, useful, done.

More information

• Monitor agent usage with OpenTelemetry - the official VS Code docs this post is based on
• Aspire Dashboard standalone docs
• OTel GenAI Semantic Conventions