Skip to main content

GitHub Copilot auto mode: should you disable models?

An architect in one of our teams selected auto mode in GitHub Copilot and ended up on a high-cost model. Nothing was broken, auto did what it is designed to do. But it was the trigger for a broader discussion:

Should we disable some (of the more expensive) models?

What does auto actually do?

Auto looks at every prompt and selects the model that is best suited for it. The choice is based on task complexity and model availability. Usage is charged based on the model auto selects, so a heavy reasoning model means a heavy bill.

The models auto can choose from are limited by your plan and by policy. Organization owners can enable or disable a model under Settings > Copilot > Models. However, be aware that by default every model that becomes generally available is on by default for GitHub Copilot Business and Enterprise. If you want to approve every model yourself, disable the "Default availability for released models" policy.

Disable a model here and auto will never select it. That answers the question, but it comes with a trade-off: disabling a model removes it for manual selection as well.

Remark: There is a feature request in the Copilot CLI repository that asks for a separate model pool for auto, because disabling a model globally removes it from manual workflows too. It does not exist today.

Auto optimization levels

Before you start with disabling models, you should first have a look at an improvement introduced in Auto mode. With the latest update three tiers are added to auto: efficiency, balance and intelligence:

  • Efficiency: keeps costs low and suits fast, straightforward tasks
  • Balance: weighs cost, quality and latency together, for everyday work
  • Intelligence: prioritizes quality and is built for complex tasks

Remark: A tier is a preference, not a limit. All three tiers use the same set of available models, and each prompt is still evaluated on its own. The reverse also holds: a simple task can still go to a small model on the intelligence tier. The only hard limit is the list of enabled models.

HydraFusion

A second option you could try before disabling models is Hydrafusion, an experimental research preview. Auto selects one model per request. HydraFusion runs multiple models in a single turn, has them review and critique one another, and fuses the result. So it picks a workflow instead of a model.

It is only available in Copilot CLI for now.

To try it, open the Copilot CLI and make sure you have the latest version:

/update
/experimental on
/model

Then select HydraFusion (Research Preview).

For each request it currently chooses one of three execution patterns:

  • Single: one model solves the task directly
  • Cascade: an efficient model drafts a solution and a quality gate decides whether to accept it or escalate to a stronger model
  • Critique: one model drafts, an independent read-only critic from a different model family reviews it, and the drafting model revises once

The critique pattern follows the same idea as the rubber duck agent I wrote about before.

There is no separate HydraFusion charge. You pay for the models it runs, so the cost of a turn is the sum of its phases.

GitHub reports that on TerminalBench 2.1 it improved verified task quality by 4.9 percentage points at 67% lower estimated cost compared with Claude Opus 5.

Does disabling models still work? For auto, yes. For HydraFusion, the FAQ says you cannot select or exclude the models it draws from. "Not today", but GitHub is looking into it.

So, should you disable models?

My take:

  • Don't use disabling as a guardrail for individual developers. Focus on educating them first on model selection, cost saving options, best practices.
  • Only disable models that nobody in your organization should use, for cost or compliance reasons. That is a policy decision, not a fix for one incident.
  • Let developers start with auto mode balance or efficiency, and switch to intelligence on purpose.
  • Try HydraFusion on non-critical work and watch your usage. You cannot steer which models it uses.

More information

Popular posts from this blog

Podman– Command execution failed with exit code 125

After updating WSL on one of the developer machines, Podman failed to work. When we took a look through Podman Desktop, we noticed that Podman had stopped running and returned the following error message: Error: Command execution failed with exit code 125 Here are the steps we tried to fix the issue: We started by running podman info to get some extra details on what could be wrong: >podman info OS: windows/amd64 provider: wsl version: 5.3.1 Cannot connect to Podman. Please verify your connection to the Linux system using `podman system connection list`, or try `podman machine init` and `podman machine start` to manage a new Linux VM Error: unable to connect to Podman socket: failed to connect: dial tcp 127.0.0.1:2655: connectex: No connection could be made because the target machine actively refused it. That makes sense as the podman VM was not running. Let’s check the VM: >podman machine list NAME         ...

Cache stampede: when our cache turned against us

While investigating some performance issues, we ran into an ASP.NET Core API that cached a fairly expensive aggregation query for 60 seconds. Under normal load, that was fine: one request rebuilds the cache, everyone else reads from it. Under peak load, dozens of requests would arrive in that same expiry window, all see a cache miss, and all fire the same expensive query in parallel. The database didn't like that. That was the moment when our caching layer stopped helping and started hurting. A burst of requests comes in at the same time, all miss the cache, and all go hammer the database or the downstream API at once. That's a cache stampede . The cache was supposed to protect our backend, and for a few hundred milliseconds it did the opposite. Why this happens IMemoryCache.GetOrCreate (and its async sibling) looks like it protects you, but it doesn't add any locking on its own. Look at the naive version: public async Task<Report> GetReportAsync(string key) ...

The role of ActivitySource in OpenTelemetry for .NET

While doing some pair programming to integrate OpenTelemetry tracing to a .NET application, we had a discussion on how to use the ActivitySource . It looks simple. You new one up, give it a name, start an activity, done. The discussion started when we added a second ActivitySource with the exact same name in a different class. This made us wonder: "Are we duplicating traces now? Is this a memory leak? Do we need a singleton?" So we decided to dig deeper. This post is what we learned… What ActivitySource actually is ActivitySource is part of System.Diagnostics , not part of the OpenTelemetry NuGet packages. Microsoft built tracing primitives directly into the BCL, and OpenTelemetry's .NET SDK simply listens to them. This is why you can add distributed tracing to a library without taking a dependency on OpenTelemetry at all. An ActivitySource is a factory for Activity objects, and an Activity is .NET's name for what OpenTelemetry calls a span.(don’t ask m...