Stackbook Logo
← Back to System Architectures

Designing a Multi-Tenant SaaS Platform

Design a multi-tenant SaaS platform covering tenant isolation models, noisy-neighbor control, per-tenant scaling, billing metering and safe migrations.

5 min read

A multi-tenant platform serves many customers from shared infrastructure. The economics depend on sharing; the trust depends on isolation. Every design decision balances those two forces.

Requirements

Functional

  • Tenant onboarding, users, roles and SSO per tenant.
  • Tenant-scoped data, configuration and branding.
  • Usage metering, plans and limits; admin and support tooling.

Non-functional

  • Strict data isolation: no cross-tenant read or write, ever.
  • 10,000 tenants, with a long tail of small ones and a few large ones holding 1,000x more data.
  • 99.95% availability; p99 API latency under 300 ms.
  • Noisy neighbors must not degrade others by more than a few percent.
  • Onboarding in under a minute for standard tenants.

Capacity Estimation

Assume 10,000 tenants, 1M users and 5,000 API requests per second at peak. Median tenant has 5 GB, the top 1% have 500 GB each: 50 TB for the top tier plus about 50 TB for the rest, so about 100 TB. A single database cannot hold this comfortably, so shard. If each shard holds 2 TB and runs at 60% capacity, you need roughly 80 shards, with big tenants possibly dedicated.

High-Level Design

Client -> CDN -> API Gateway (authN, tenant resolution, rate limit per tenant)
                     |
              Service Layer (stateless, tenant context in every call)
                     |
          Tenant Directory (tenant -> shard, tier, region, limits)
                     |
     +---------------+----------------+
 Shared pool shards    Dedicated DB (enterprise)    Regional cells
 (tenant_id column)    (schema or database)
                     |
          Queue / Workers (per-tenant fair scheduling)
          Metering pipeline -> Billing

Data Model

  • tenants(tenant_id, plan, tier, shard_id, region, status, created_at)
  • users(tenant_id, user_id, email, role) with a composite key starting with tenant_id
  • tenant_config(tenant_id, key, value)
  • usage_events(event_id, tenant_id, metric, qty, ts)
  • Every business table has tenant_id as the first column of its primary key and indexes.

Deep Dives

Isolation models

There are three standard choices. A shared schema with a tenant_id column is cheapest and scales to thousands of tenants but relies on every query being scoped. Schema per tenant gives easier per-tenant backup and customization, but migrations multiply and catalog size limits appear around tens of thousands. Database or cell per tenant gives the strongest isolation and independent scaling at the highest cost. A tiered approach works well: the long tail shares pools, and enterprise customers pay for dedicated resources, all behind the same tenant directory.

Enforcing scope

Scoping bugs cause the worst incidents. Defense in depth: the gateway resolves the tenant from the token and injects it into a request context; the data layer refuses queries without a tenant filter; and the database enforces it with row-level security keyed on a session variable set per connection. Add automated tests that try cross-tenant access, and canary tenants that detect leakage.

Noisy neighbors

Apply limits at several layers: per-tenant rate limits at the gateway using token buckets, connection pool quotas per tenant, query timeouts, and fair-share queues for background jobs so one tenant's bulk import does not starve others. Weighted fair queueing by plan gives paying tiers predictable throughput. Heavy tenants are automatically promoted to their own shard through a live migration.

Idempotency and concurrency

Tenant-scoped idempotency keys are unique on tenant_id plus key, so one tenant cannot collide with another. Use optimistic concurrency on mutable records. For tenant-level operations such as plan changes or migrations, take a per-tenant lock with a lease, so two workers cannot reshape one tenant simultaneously.

Migrations and schema change

Use expand and contract: add nullable columns, deploy code that writes both, backfill in batches throttled per shard, switch reads, then drop the old column. Roll out shard by shard, starting with internal and low-risk tenants, with automatic halt on error rate increases.

Tenant moves

Moving a tenant between shards uses snapshot, change capture to catch up, brief write freeze of seconds, then directory flip. The directory is cached with short TTL and versioned so stale routes retry after redirect.

Metering

Emit usage events to a stream with deterministic ids so reprocessing is idempotent. Aggregate hourly per tenant and metric, reconcile against invoices, and enforce soft and hard limits from near-real-time counters.

Failure Modes and Trade-offs

  • Shard outage: only its tenants are affected; cells limit blast radius to a slice of customers.
  • Directory outage: cache routes locally with a long TTL; the directory itself is replicated and rarely changes.
  • Bad deploy: progressive rollout by cell, with fast rollback.
  • Per-tenant customization versus uniformity: configuration over code branches keeps the fleet maintainable.
  • Data residency: pin tenants to regions at creation; cross-region analytics operate on aggregates.
  • Cost versus isolation: dedicated databases raise price and operations load but simplify compliance conversations.

Key takeaways

  • Choose a tiered isolation model with a tenant directory that maps tenants to shards.
  • Make tenant context mandatory and enforce it at the gateway, data layer and database.
  • Control noisy neighbors with per-tenant rate limits, quotas and fair scheduling.
  • Deploy and migrate cell by cell using expand and contract changes.
  • Meter usage with idempotent events and enforce limits from fast counters.
  • Keep tenant moves routine so hot tenants can be isolated on demand.