We built the cache because the alternative cost more than an engineer. Two years on, it has paid for itself twice — and taught us three things we would rather have known on day one.
The cold start problem is a people problem
A cold cache is not slow because of the network. It is slow because nobody agrees on what a cache key should contain, so every team quietly adds one more variable until the hit rate collapses.
We fixed it by making the key a reviewed artefact: one file, one owner, and a test that fails when the key changes without a note explaining why.
A cache with a 40% hit rate is not a cache. It is a tax with extra steps.
Eviction beats capacity
For a year we answered every slowdown by adding storage. The fix was the opposite: evict aggressively, keep the index small enough to stay in memory, and accept a few extra misses on old branches.
Measure misses, not hits
Hit rate is a vanity metric — it goes up when builds repeat, which is exactly when nobody is waiting. What matters is the time a miss costs on the critical path, per branch, at the moment somebody is watching the pipeline.
Was it worth building?
For us, yes — but only because the cache is boring now. If you are still tuning yours every month, the honest answer is that you are running a small distributed systems project on the side of your actual work.