The invisible shield: How Contentstack Launch protects every customer at scale
Share
At Contentstack Launch, our philosophy is simple: rate limits shouldn’t bother you. They're designed to ensure every customer gets a fair share of system resources while staying largely invisible during normal usage. But attacks are different. When someone tries to overwhelm your application with malicious traffic, protecting the Launch platform from unexpected traffic spikes and malicious requests is our responsibility. We don't want customers worrying about infrastructure-level rate limiting or how to keep one application's traffic from affecting another. Our goal is to minimize disruption so your teams can stay focused on building great digital experiences.
On the open web, developers can't control who accesses their applications or how often. Bots, automated traffic and attackers can all consume resources or attempt to disrupt services.
You're probably already familiar with Distributed Denial of Service (DDoS) attacks, where thousands of machines across the globe flood an application with requests. Unlike a single-source attack that can be blocked with simple IP filtering, a DDoS is designed to resemble legitimate traffic from legitimate locations, making it much more difficult to distinguish and stop.
The challenge we face as the builders of Contentstack Launch is one every hosting platform faces: how do you let legitimate traffic through while ensuring one customer's traffic doesn't affect another's? In this post, we'll walk through the approaches we considered and the architecture we ultimately built to solve that problem. While this is an engineering deep dive, every design decision supports a simple goal: delivering a fast, reliable experience for every Launch customer.
1. The shield that keeps every customer protected
We strongly believe that one customer’s traffic flood shouldn’t be another customer’s problem.
For this requirement, rate limiting controls how many requests a caller can make within a given time window. Requests beyond that threshold are rejected before they ever reach our infrastructure. This is what makes isolation work and ensures that no single organization, whether under attack or just surging, can consume more than its allocation and spill into someone else's capacity. For Contentstack Launch, each organization has a defined request-per-second limit. If traffic exceeds this limit, whether because of a legitimate surge in traffic or due to a DDoS attack, excess requests are gracefully rejected before they can impact other customers on the platform.
That's the shield. The harder question is: how do you build one that actually works at scale? The goal is simple: traffic from one organization shouldn't affect the experience of another.
2. Fast, but not smart enough
Rate limiting is often solved as close to the edge as possible, usually at the CDN (Content Delivery Network) layer. CDNs are fast, globally distributed content servers that sit right in the path of every request. On paper, the CDN seems like the perfect place to implement rate limiting, but doing so comes with significant constraints to consider.
A modern CDN operates through hundreds of Points of Presence (POPs) spread across the globe. Each POP is an independent node with its own local request counter -- meaning that without coordination across the nodes, a single organization could consume far more than its intended limit. You can work around this by designating one POP as a global coordinator, but now every request has to round-trip to that central node for counting. This introduces latency into a layer that was supposed to significantly reduce it.
CDN-level rate limiting works well for broad, IP-based abuse protection, but it falls short as a scalable strategy for shared resources. For accurate, per-organization enforcement with dynamic plan-aware limits and no added latency, we had to go deeper. The result is an architecture that helps us enforce organization-level limits without introducing unnecessary latency for customer requests.
3. Three strategies, one winner
There are three classic approaches for enforcing global per-organization limits:
| Strategy | How It Works | Strength | Weakness |
| Fixed Window | Count requests per fixed window (0–1s, 1–2s) | Simple | Boundary bursts: 2× limit possible at window edges |
| Sliding Window Log | Track every request timestamp, compute rolling count | Accurate | Expensive at scale log storage per request |
| Token Bucket | Tokens replenish at fixed rate, one consumed per request | Burst-friendly, smooth | Complexity in a distributed system. Where do you store the bucket? |
Token Bucket is the approach most production-grade rate limiters converge on and so did we. But as the table hints, the hard part wasn't choosing the algorithm. It was figuring out where to store it.
4. What we built: a tiered token bucket
The naive implementation of a tiered bucket strategy stores the bucket entirely in a centralized database that all servers share. That technically works, but it means that every single request triggers a database round-trip. At high, enterprise-level request volumes, that's a tremendous amount of trips. This adds latency in the critical path and creates a bottleneck under load.
We wanted to solve this problem and implement a smarter token allocation process. We created what we call a Tiered Token Bucket, a two-tier architecture that keeps a small local token reserve on each server and a shared global pool in a database.
Here's how it works:
Every server holds a per-organization bucket in local memory
Each bucket is allocated enough tokens to serve a burst of requests instantly without a trip to the database
When local tokens run out, the server reaches out to the global database, which atomically checks the organization's total usage against their plan limit and allocates more tokens if available
The server stores the refilled tokens locally and continues serving until the next refill is retrieved from the database
This design means, in our measurements ~95% of requests never touch the database at all. That translates into better request handling while reducing load on shared infrastructure. The fast path is entirely in-memory, on the same server handling the request. The database is only called when local reserves need replenishing, or when the global limit is close.
Figure 1: Request flow — fast path (local decision, zero network calls) vs slow path (global coordination on refill)
5. The problem we didn't see coming
Getting here wasn't straightforward. Our first approach used a pre-allocation method, where token buckets were created when needed. The moment our infrastructure received the first request for an organization, it would create a bucket and immediately claim a full batch of tokens from the global counter. Not just the 1 token needed for that request, but the entire batch, upfront.
The problem? In any distributed system, traffic is never perfectly balanced. Some servers are hot, receiving a disproportionate share of requests, while others go cold as the load balancer routes traffic elsewhere. A server that claims a full batch of tokens from the global database and then goes cold still holds onto those tokens, while the global counter has already been decremented. Those tokens are effectively stranded. From the system's perspective, the capacity is consumed, but in reality it was never used.
Org A :- 200 req/s limit, 10 pods, 20-token per batch:
Figure 2: Pre-allocation — stranded tokens on cold pods drain the global counter, causing false denials on hot pods
The above chart shows an example of underutilization and how it causes false rejections at scale. In theory, you could sidestep this by setting limits high enough that stranded tokens never matter. But there is some risk with that approach. The gap between the configured limit and your effective limit grows with pod count and traffic skew, both of which are outside your control. A deployment, a traffic shift, a scaling event could silently widen that gap. Our promise is that rate limits shouldn't exist from your perspective under normal usage. Underutilization breaks that promise not because your traffic was too high, but because the infrastructure state made it look that way.
What if we flipped the model to post-allocation?
Post-allocation means that each bucket now starts with a number of tokens that live entirely in local memory. This bypasses the need to retrieve them from the global counter and allows our infrastructure to instantly begin serving requests. Only once those initial tokens are exhausted is there a trip to the database to deduct from the global pool and refill the bucket. If a server goes cold before that point, then the window expires, the local tokens evaporate and the global counter is never touched.
Same 10-pods example with post-allocation
Org A :- 200 req/s limit, 10 pods, 20-token per batch:
Figure 3: Post-allocation — cold pods expire without touching the global counter; hot pods refill from full available capacity
This approach comes with a small, intentional tradeoff. During brief periods of activity, an organization may temporarily exceed its exact allocation before global coordination occurs. We believe this tradeoff delivers the best balance between performance, fairness and customer experience while continuing to protect the platform at scale.
6. How this protects our customers
When we set out to build this part of our hosting platform, we did it with one central question in mind: how do we provide the best experience for customers while also ensuring high performance at scale?
Getting this balance right isn't easy. Rate limiting that's too strict can lead to valid traffic getting blocked. And too lax rate limiting is as good as no rate limiting. We took an approach that is both globally accurate and locally fast, which means our customers are shielded without ever feeling it.
With this approach:
Your plan limit is enforced globally: Not per-server, not per-region, but across the entire platform. Your allocation is yours and no other customer can consume it.
DDoS traffic is handled early: Requests beyond configured limits are rejected before they consume additional platform resources, helping reduce the impact on customer workloads.
Always take the fastest path: Most requests are served from in-memory local tokens. No database, no network hop, no added latency.
Limits adapt to your plan: As your organization grows and your plan changes, your rate limit updates automatically. No configuration, no redeployment.
A platform you can rely on
Rate limiting is one of those features that, when it works well, you never think about. Traffic comes in, requests are served and everything just works. The complexity is invisible by design.
At Contentstack Launch, rate limit protection is built into our infrastructure from the ground up. Every organization, regardless of size or traffic pattern, gets a consistent, reliable experience. We built our platform to ensure it.
Whether you're running a quiet staging environment, or absorbing the stampede of a successful flash sale, Launch is built to keep your applications available and responsive, even during periods of unexpected demand.
