Stackbook Logo
data-stateestablished · high operational burden

Polyglot Persistence

Also known as: multi-model-storage, right-database-for-the-job

Intent

Use different storage technologies for different data shapes and access patterns within one system.

Problem

A single database rarely serves transactions, search, caching, analytics and time-series data equally well. Forcing every workload into one engine leads to awkward schemas and poor performance. Yet each extra datastore adds operational and consistency complexity.

Forces

  • •Access patterns differ widely across features
  • •Each datastore needs expertise, backups and monitoring
  • •Data copied between stores can drift
  • •Teams want autonomy but the organization needs a manageable footprint

Solution

Overview

Polyglot persistence means picking storage per use case: PostgreSQL for orders, Redis for sessions, Elasticsearch for search, a time-series database for metrics, object storage for blobs. The benefit is fit; the cost is everything that comes with running more systems.

How it works

Keep one source of truth and derive the rest.

Service --write--> PostgreSQL (system of record)
                        |  CDC / outbox events
          +-------------+-------------+
          v             v             v
     Search index   Redis cache   Analytics store

Derived stores are disposable: if one is lost, rebuild it from the source by replaying changes.

A practical governance approach is a paved road. The platform team supports a short list of engines, for instance one relational database, one cache, one search engine and one object store, each with templates for backups, alerts and access control. Teams may choose outside the list only by owning the full operational cost. Pair this with data contracts between stores: say how fresh a derived store must be, who can rebuild it, and what happens to user-facing features while it is behind or unavailable.

Pitfalls

  • Dual writes from application code, which fail partially and cause drift.
  • Adopting a database for résumé reasons rather than a measured need.
  • No owner or runbook for a store, so restores are untested.
  • Joining data across stores in application code on the hot path.

Key takeaways

  • Start with one general-purpose database and add engines only for proven needs.
  • Make exactly one store authoritative per piece of data.
  • Feed secondary stores from a log so they are rebuildable.
  • Count operational cost, not only query performance.

✓ When to Use

  • •Distinct workloads such as transactions, full-text search and metrics in one product
  • •Read patterns that a relational schema serves poorly
  • •Organizations with the staff to run several datastores

✗ When Not to Use

  • •Early-stage products where one relational database suffices
  • •Small teams without on-call capacity for multiple engines
  • •Cases where a single engine extension already meets the need

Pros

  • +Each workload gets a fit-for-purpose engine
  • +Specialized stores scale independently
  • +Avoids contorting data models

Cons

  • −More systems to operate, secure and back up
  • −Cross-store consistency is eventual
  • −Higher cognitive load and hiring bar

Cost Profile

Infrastructure

High - several clusters with their own capacity

Operational

High - multiple upgrade, backup and tuning regimes

Cognitive

High - many models and query languages

Failure Modes

  • Secondary store silently diverges from the system of record

  • Each team adopts a new database and nobody can operate it

  • Dual writes without a log leave stores inconsistent after partial failure

Real-World Examples

Alternatives

    Related Patterns

    • cdc
    • outbox-pattern
    • materialized-view

    Competency Domains

    data stateeconomics evolution