This is the second post in the ON Labs series. The first covered the WebGL engine: shaders, draw calls, the GPU-level stuff. This one is about what happens before a single pixel renders: fetching project data from an external API, caching it and making sure concurrent requests don’t trip over each other.
The original problem
Labs pulls project boards and media assets from Air.inc. Early on, every page request made a fresh API call. It worked fine in development and immediately hit rate limits in production. Multiple serverless instances spinning up at once meant dozens of identical requests hitting the same endpoints within seconds.
You can’t just “add a cache” when you don’t control how many instances are running or when they start.
One request rebuilds, the rest wait
The core idea is coordination: when the cache expires, only one instance rebuilds while the others wait. If the rebuild takes too long or fails entirely, stale data gets served instead of an error. A crashed process can’t hold the lock forever; it expires on its own.
One request rebuilds. The rest wait or serve stale. No stampede
It’s stale-while-revalidate implemented in application code, because the hosting layer doesn’t offer it natively for API-driven data. The storage layer doubles as both cache and coordination mechanism between instances that have no other way to communicate.
Cache versioning and the video pipeline
The cached payload carries a version number. When the data shape changes (new fields, restructured assets) the version bumps and every instance treats the old cache as expired. No manual purging, no deploy-time clears.
A separate store handles optimised video URLs. Each asset’s source maps to transcoded variants at different quality levels, cached per-asset with their own expiry. A concurrency limit keeps the transcoding lookups from overwhelming the provider.
Labs loads quickly. Transitions run smoothly. Filters respond without delay. Nobody sees lock negotiation or stale fallbacks or version checks. They see a portfolio that works.
What the data layer protects: a fast, reliable first impression
Why build this yourself
Most caching is straightforward: set a TTL, serve from cache, refresh when it expires. The complication is concurrency across serverless instances with no shared memory. Off-the-shelf HTTP caching doesn’t solve that. Coordination in a stateless environment means leaning on a shared store and building the locking yourself.
Not glamorous work. But without it, two people visiting at the same time could bring the whole thing down. That’s not hypothetical.
