While investigating some performance issues, we ran into an ASP.NET Core API that cached a fairly expensive aggregation query for 60 seconds. Under normal load, that was fine: one request rebuilds the cache, everyone else reads from it. Under peak load, dozens of requests would arrive in that same expiry window, all see a cache miss, and all fire the same expensive query in parallel. The database didn't like that.
That was the moment when our caching layer stopped helping and started hurting. A burst of requests comes in at the same time, all miss the cache, and all go hammer the database or the downstream API at once. That's a cache stampede. The cache was supposed to protect our backend, and for a few hundred milliseconds it did the opposite.
Why this happens
IMemoryCache.GetOrCreate (and its async sibling) looks like it protects you, but it doesn't add any locking on its own. Look at the naive version:
public async Task<Report> GetReportAsync(string key)
{
if (_cache.TryGetValue(key, out Report cached))
{
return cached;
}
var report = await _reportService.BuildExpensiveReportAsync(key);
_cache.Set(key, report, TimeSpan.FromSeconds(60));
return report;
}If ten requests hit this at the exact same moment after expiry, all ten see TryGetValue return false, and all ten call BuildExpensiveReportAsync. Nothing here coordinates them. That's the stampede in a nutshell.
Remark: this isn't specific to IMemoryCache. Distributed caches like Redis have the exact same problem. The cache itself doesn't know or care that nine other callers are about to ask for the same key.
Fix 1: a per-key lock with SemaphoreSlim
The simplest fix is to make concurrent callers for the same key wait for the first one to finish, instead of all racing to rebuild.
private static readonly ConcurrentDictionary<string, SemaphoreSlim> _locks = new();
public async Task<Report> GetReportAsync(string key)
{
if (_cache.TryGetValue(key, out Report cached))
{
return cached;
}
var keyLock = _locks.GetOrAdd(key, _ => new SemaphoreSlim(1, 1));
await keyLock.WaitAsync();
try
{
// Re-check: another request may have already rebuilt it
// while we were waiting on the semaphore.
if (_cache.TryGetValue(key, out cached))
{
return cached;
}
var report = await _reportService.BuildExpensiveReportAsync(key);
_cache.Set(key, report, TimeSpan.FromSeconds(60));
return report;
}
finally
{
keyLock.Release();
}
}The double check (TryGetValue before and after acquiring the lock) matters. Without it, every waiting request would still rebuild the report one after another instead of reusing the result the first one just produced.
Remark: the ConcurrentDictionary<string, SemaphoreSlim> grows forever if you don't clean it up. For a small, bounded set of keys this is fine. For a large or unbounded key space, you'll want to evict unused semaphores, or switch to the library-based approach below.
Fix 2: let HybridCache do it for you
Writing your own per-key locking is fine for one or two hot spots. Once you have this pattern in five services, it's worth reaching for something that bakes it in. Microsoft.Extensions.Caching.Hybrid.HybridCache is Microsoft's own answer here, and stampede protection is one of the main reasons it exists: it explicitly coordinates concurrent callers for the same key so only one factory call runs at a time.
Register it like this:
services.AddHybridCache(options =>
{
options.DefaultEntryOptions = new HybridCacheEntryOptions
{
Expiration = TimeSpan.FromSeconds(60),
LocalCacheExpiration = TimeSpan.FromSeconds(60)
};
});And use it like this:
public async Task<Report> GetReportAsync(string key, HybridCache cache)
{
return await cache.GetOrCreateAsync(
key,
async cancel => await _reportService.BuildExpensiveReportAsync(key, cancel),
cancellationToken: default);
}GetOrCreateAsync guarantees only one factory call runs per key at a time. Every other concurrent caller for that key awaits the same in-flight task instead of starting a new one. That's exactly the behavior we hand-rolled with the SemaphoreSlim above, minus the bookkeeping and it ships in the box, no extra NuGet dependency to evaluate.
Remark: HybridCache also gives you a two-level cache for free; an in-process ("local") layer plus an optional distributed layer (Redis, SQL Server, whatever IDistributedCache backend you already have configured). The stampede protection applies at the local layer on each node; it doesn't stop two different nodes in a cluster from both rebuilding the same key at the same time, since there's no cross-machine lock. For that you'd still need a distributed lock, or accept the occasional double rebuild across nodes as a smaller-scale version of the same problem.
Fix 3: don't let everything expire at the same instant
Even with per-key locking, you can still get a milder version of the problem if many different keys all happen to expire at the same time. For example, if you warm up a cache with a fixed TTL for a batch of items at startup. A small trick here is to add jitter to the expiration:
var jitter = TimeSpan.FromSeconds(Random.Shared.Next(0, 10));
_cache.Set(key, report, TimeSpan.FromSeconds(60) + jitter);Spreading expirations out by a few seconds turns one big spike into a series of small ones.
Putting it together
For a single hot key, a SemaphoreSlim per key gets you most of the way. For anything broader, or if you also want fail-safe behavior when the backend is down, FusionCache (or a similar library) is worth the dependency. And regardless of which approach you pick, jitter on the TTL is a cheap insurance policy against synchronized expirations.
That's it!