Stackbook Logo
← Back to System Architectures

Designing a Feature Flag Service

Design a feature flag service with fast local evaluation, targeting rules, gradual rollouts, streaming updates and safe behavior when the control plane fails.

5 min read

Feature flags let teams change behavior without deploying. Because flags sit on the hot path of every request in every service, the design goal is evaluation that is fast, local and independent of the flag service being up.

Requirements

Functional

  • Boolean, string, number and JSON flags per environment.
  • Targeting by user attributes, segments and percentage rollouts.
  • Change history, approval workflow, kill switches, scheduled changes.
  • Client SDKs for backend, web and mobile.

Non-functional

  • Evaluation adds under 1 ms and never blocks on the network.
  • Flag changes reach 99% of SDK instances within 5 seconds.
  • Evaluation continues if the flag service is entirely down.
  • Support 100,000 SDK instances and 1M evaluations per second across the fleet.

Capacity Estimation

Assume 5,000 flags per environment, each ruleset averaging 2 KB: the full ruleset for an environment is 10 MB, often compressed to 1 to 2 MB. With 100,000 SDK connections, a full download on startup is 150 GB if all restart at once, so serve from CDN and use incremental diffs. Changes are rare, about 100 per hour across the company, each diff a few KB. Streaming to 100,000 connections costs only about 100 messages per hour per connection. Evaluation events, if sent, are the heavy part: sample or aggregate them in the SDK.

High-Level Design

Admin UI/API -> Control Plane (auth, validation, approvals) -> Flag DB (versioned)
                                                            -> Change Log -> Publisher
Publisher -> CDN (full ruleset JSON, versioned) 
          -> Streaming Service (SSE/WebSocket, diffs) -> Server SDKs
Server SDK: in-memory ruleset + local evaluator
Client SDK: ruleset fetch by context via Edge Evaluator, cached on device
SDK -> Event Collector (aggregated counts) -> Analytics / Stale flag detection

Data Model

  • flags(flag_id, project, key, type, description, owner, created_at)
  • flag_configs(flag_id, env, version, enabled, default_variation, off_variation, rules_json, fallthrough_json)
  • segments(segment_id, env, rules_json, version)
  • audit_log(id, actor, flag_id, env, before, after, ts)
  • Environment ruleset snapshot: a single versioned document with all flags and segments.

Deep Dives

Local evaluation

Server SDKs download the full ruleset and evaluate in memory. The evaluator walks the targeting rules in order: individual targets, segment membership, attribute conditions, and finally the fallthrough, which may be a percentage rollout. No network call occurs per evaluation. Mobile and browser clients, which should not receive all rules, call an edge evaluator with a context and cache the resulting values.

Deterministic percentage rollouts

Bucket by hashing flag key, salt and the user key to a number from 0 to 99,999, then compare against cumulative variation weights. The same user always gets the same result, and raising a rollout from 10% to 20% only adds users, never flips existing ones. Different flags use different salts so the same users are not always in every experiment.

Propagation and consistency

Each environment snapshot has a monotonically increasing version. SDKs open a stream, receive diffs and apply them only if the version is exactly the next one; otherwise they refetch the full snapshot from the CDN. A periodic poll every 30 to 60 seconds guarantees convergence if the stream silently drops. Flag changes are eventually consistent: two services may disagree for a few seconds, so avoid flags that must flip atomically across services. When that is required, use a two-step rollout with compatible code on both sides.

Concurrency in the control plane

Two editors may change the same flag. Updates carry the expected version and fail on mismatch with a clear diff. Every change writes the audit entry in the same transaction. Dangerous flags need approvals and can be locked in production.

Failure behavior

SDKs hold the last known ruleset in memory and optionally persist it to disk or fall back to a bootstrap file baked into the build. If startup fetch fails, the SDK returns the code-supplied default variation. Defaults must be the safe behavior. The kill switch is just a flag, so it must work even under degraded conditions: keep the data path on CDN, which is highly available, rather than on the control-plane database.

Hygiene

Stale flags become debt. Track the last evaluated time and owner, flag temporary flags with expiry dates, and open cleanup tickets when a flag serves a single variation for 30 days.

Failure Modes and Trade-offs

  • Bad config change: validate rules schema, run a dry-run against sample contexts, and support instant rollback to a prior version.
  • Streaming outage: fallback polling via CDN preserves freshness within a minute.
  • Large ruleset size: split by project or use per-flag delta fetch.
  • Privacy: evaluate sensitive attributes locally rather than sending them to the service.
  • Server-side versus edge evaluation for clients: edge evaluation hides rules but adds a network hop; shipping rules to the client is faster but exposes targeting logic.
  • Event volume versus insight: aggregate counts cheaply, and sample full contexts only for experiments.

Key takeaways

  • Evaluate locally from an in-memory ruleset so flags never add network latency or a dependency.
  • Use hash-based bucketing for stable, monotonic percentage rollouts.
  • Combine streaming with CDN snapshots and polling for guaranteed convergence.
  • Version every ruleset and write audit records transactionally with changes.
  • Choose safe code defaults and make the data plane independent of the control plane.
  • Manage flag lifecycle actively to avoid stale-flag debt.