scalingestablished · medium operational burden

Fan-out on Write

Also known as: push-model-feed, write-time-fan-out, timeline-precompute

Intent

Push new content into each follower feed at write time so reads are cheap.

Problem

Building a feed on read means querying and merging posts from everyone a user follows, which is slow and expensive for read-heavy products. Feeds are opened far more often than posts are created.

Forces

•Feed reads vastly outnumber posts
•Celebrity accounts have millions of followers
•Feeds must feel instant to open
•Write amplification costs storage and throughput

Solution

Overview

Fan-out on write, the push model, does the merging when content is created. Each follower's feed is a ready-to-read list. Fan-out on read, the pull model, does the opposite and merges at request time. Most large feed systems combine the two.

How it works

post created
   ├─ store post (id, author, body)
   └─ enqueue fan-out(author, post_id)
         worker: for follower in followers(author):
                   LPUSH feed:follower post_id
                   LTRIM feed:follower 0 799

read feed(user): ids = LRANGE feed:user 0 49; return hydrate(ids)

The feed stores only ids, so edits and deletes apply to a single post record; hydration filters out deleted posts. Workers fan out in chunks and in parallel, and they prioritize active followers first.

The hybrid threshold is typically defined by follower count. If an author has more than, say, 100000 followers, their posts are not pushed. At read time the service fetches the feed list and merges in the latest posts of followed high-fan-out authors, then sorts by time or rank.

Inactive users do not need materialized feeds. Skip them and rebuild on their next login from the pull path.

Pitfalls

Treating all authors equally. One account can generate more writes than the rest of the system.
Fan-out failures that leave some followers without the post. Make jobs idempotent and retry.
Unbounded feed length and memory growth.
Follow and unfollow changes that leave stale entries in feeds.
No visibility into fan-out lag, the key freshness metric.

Key takeaways

Push for typical authors, pull for very popular ones.
Store ids in feeds and hydrate at read time.
Fan-out must be asynchronous, idempotent and rate-controlled.
Monitor queue lag and cap feed size.

✓ When to Use

•Social timelines, notification inboxes and activity feeds
•Read-heavy systems with moderate follower counts
•Products where feed latency is a key metric

✗ When Not to Use

•Accounts with huge audiences, unless using a hybrid model
•Write-heavy systems with rarely opened feeds
•Feeds that are personalized by ranking computed at read time

Pros

+Feed reads are a single fast lookup
+Read latency independent of follow count
+Expensive merging work moved off the request path

Cons

−Write amplification proportional to followers
−Celebrity posts cause massive bursts
−Storage for materialized feeds, including inactive users

Cost Profile

Infrastructure

High: feed storage and fan-out workers

Operational

Medium: queues, lag monitoring, rebuilds

Cognitive

Medium: hybrid logic and consistency

Failure Modes

Fan-out queue lag makes new posts appear late in feeds
A celebrity post saturates workers and delays everyone else
Deleted or edited posts remain visible in precomputed feeds

Real-World Examples

Alternatives

Related Patterns

backpressure

Competency Domains

data statedistribution communication