Stackbook Logo
data-stateemerging · high operational burden

Feature Store

Also known as: ml-feature-store, online-offline-feature-store

Intent

Centralize definition, computation and serving of machine learning features for consistent training and inference.

Problem

Data scientists compute features in notebooks for training while engineers reimplement them for production serving. The mismatch, called training-serving skew, degrades model accuracy and teams duplicate work.

Forces

  • •Same feature logic for training and serving
  • •Low-latency lookups online versus large scans offline
  • •Point-in-time correctness to avoid label leakage
  • •Feature reuse across teams and models

Solution

Overview

A feature store manages the inputs to machine learning models as shared, versioned assets. It addresses two problems: duplicated feature code and mismatches between how features are computed for training versus production.

How it works

Features are declared against an entity such as user or merchant, with a source and transformation. Materialization jobs compute values and write them to both stores.

sources --> transform (batch/stream)
              |--> offline store (history, columnar)  --> training sets
              +--> online store (latest per key)       --> inference API
model request: get_features(user_id=42, ["orders_7d", "avg_basket"])

Point-in-time joins are central. For a label at time T, the offline store returns the feature value as known at T, never later. Without this, training includes information unavailable in production, and offline metrics look better than reality.

Freshness varies by feature: daily batch for slow signals, streaming aggregation for counters such as clicks in the last ten minutes. Some features are computed on request from the incoming payload and do not need storage.

Pitfalls

  • Skew from separate code paths: keep one transformation definition feeding both stores.
  • Label leakage: joining on latest values instead of as-of values inflates offline accuracy.
  • Silent staleness: monitor feature age and null rates at serving time, and alert when materialization lags.
  • Online store cost: only materialize features that models actually use online, with TTLs.
  • Changing a feature definition in place breaks deployed models; version features.

Key takeaways

  • One definition, two stores: offline for training, online for serving.
  • Enforce point-in-time correctness.
  • Monitor freshness and distribution drift.
  • Adopt when features are shared or skew is a real problem.

✓ When to Use

  • •Multiple models share features
  • •Online inference needs low-latency feature lookups
  • •Training-serving skew has caused incidents

✗ When Not to Use

  • •One model with simple, request-derived inputs
  • •No online serving requirement
  • •Team too small to operate another platform component

Pros

  • +Consistent features across training and serving
  • +Reuse and discoverability
  • +Point-in-time correct training data

Cons

  • −Significant platform to run
  • −Dual stores must be kept in sync
  • −Tight coupling of models to feature definitions

Cost Profile

Infrastructure

Medium - online and offline stores

Operational

High - materialization pipelines and monitoring

Cognitive

Medium - point-in-time semantics

Failure Modes

  • Online and offline values diverge causing skew

  • Training data leaks future values

  • Stale online features after a failed materialization job

Real-World Examples

Alternatives

    Related Patterns

      Competency Domains

      data statereliability ops