Stackbook Logo
← Back to System Architectures

Designing a Photo Sharing Service

Design a photo sharing service for 500M users: direct-to-storage uploads, thumbnail pipeline, CDN delivery, metadata sharding and feed integration.

4 min read

A photo sharing service stores user images, generates multiple sizes, and serves them quickly worldwide alongside social features such as likes and feeds. The design separates cheap, durable blob storage from fast, queryable metadata.

Requirements

Functional

  • Upload photos with captions and tags.
  • View photos in multiple resolutions; user profile galleries.
  • Follow users, like and comment, see a feed.
  • Delete photos and honour privacy settings.

Non-functional

  • 500M users, 100M daily active, 50M uploads per day.
  • Upload acknowledged within 2 seconds; image load under 200 ms from nearby edges.
  • 11 nines durability for originals; high availability for reads.
  • Reads outnumber writes about 100 to 1.

Capacity estimation

  • 50M uploads per day is roughly 580 per second, peak 2,000.
  • Original averages 3 MB: 150 TB per day, about 55 PB per year.
  • Derived sizes (thumbnail, small, medium, large) add roughly 30 percent after compression.
  • Reads: 5B image views per day, about 58,000 per second, nearly all CDN hits.
  • Metadata: 1 KB per photo gives 50 GB per day, about 18 TB per year, which needs sharding but is small next to the blobs.

High-level design

Client -> API Gateway -> Upload Service -- pre-signed URL --> Object Store
                               |                                  |
                               v                                  v event
                         Metadata DB (sharded)           Image Processing Workers
                               ^                          (resize, strip EXIF, scan)
                               |                                  |
Client <- CDN <- Origin (object store, derived sizes) <-----------+
Client -> Feed / Social services (follows, likes, comments)

Data model

photos(photo_id PK, owner_id, storage_key, caption, visibility, created_at, status)
photo_variants(photo_id, size, storage_key, width, height)
likes(photo_id, user_id)      comments(photo_id, comment_id, user_id, body)

Shard photos by owner_id so a profile gallery is a single-shard query. Photo IDs are time-sortable. Like counts live in a counter cache and are flushed periodically to the database; the likes table with a composite key prevents duplicates.

Deep dives

Upload flow. The client asks the API for an upload slot and receives a pre-signed URL. It sends bytes straight to object storage, which keeps application servers light. A storage event enqueues processing. The photo row starts in a pending status and becomes active after processing. For a snappy feel, the client shows the local image immediately.

Processing pipeline. Workers validate the file type by content, not extension, strip location EXIF data unless the user opts in, correct orientation, generate variants (WebP or AVIF for modern clients, JPEG fallback), compute a perceptual hash for duplicate and abuse detection, and run safety scans. The pipeline is idempotent so retries are safe.

Serving. URLs encode photo ID and size. The CDN caches variants with long max-age and immutable headers because a photo's bytes never change; edits create a new version key. Support responsive sizes via srcset so mobile clients fetch small images. On-the-fly resizing at the edge is an alternative to pre-generating every size, trading CPU for storage.

Storage tiering. Recent and popular photos stay on standard storage; photos untouched for a year move to infrequent-access tiers. Originals are replicated across zones, and across regions for durability.

Privacy. Private photos use signed, short-lived CDN URLs and an authorisation check at URL issue time. Deletion removes the metadata row immediately, then asynchronously purges blobs and invalidates CDN entries.

Feed integration. New photos publish an event to the feed service, which fans out using the hybrid push and pull strategy for followers. The feed carries only photo IDs.

Failure modes and trade-offs

  • Processing backlog: the photo stays pending. Show the original to the owner, and autoscale workers on queue depth.
  • Orphaned blobs: upload succeeds but the metadata write fails. A periodic reconciler deletes blobs with no row after a grace period.
  • CDN miss storm after a purge: shield caching and staggered invalidation protect the origin.
  • Hot photo or user: viral content is fine on a CDN, but hot metadata rows need replicas and cached counters.
  • Pre-generate versus on-demand resizing: pre-generation costs storage but gives predictable latency; on-demand saves storage but needs strong caching.
  • Hash-based dedupe saves space but complicates deletion because shared blobs need reference counting.

Key takeaways

  • Store blobs in object storage and metadata in a sharded database; never mix them.
  • Pre-signed direct uploads keep application servers out of the data path.
  • Immutable, versioned URLs make CDN caching trivial and safe.
  • An idempotent processing pipeline handles resizing, EXIF stripping and safety checks.
  • Tier old photos and reconcile orphans to keep a 55 PB per year growth affordable.