Stackbook Logo
data-stateestablished · medium operational burden

ETL vs ELT

Also known as: extract-transform-load, extract-load-transform

Intent

Choose whether to transform data before loading into the target or inside the target after loading.

Problem

Moving data from operational sources into analytics storage requires cleaning and modeling. Doing it in the wrong place causes brittle pipelines, lost raw data, or runaway warehouse cost.

Forces

  • •Warehouse compute is elastic and SQL-friendly
  • •Sensitive data may need masking before landing
  • •Raw retention enables reprocessing
  • •Transformation logic needs testing and version control

Solution

Overview

ETL and ELT differ in where the transform step runs. ETL transforms in a dedicated engine before loading; ELT loads raw data first and transforms inside the destination using its own compute. The shift toward ELT followed cheap cloud storage and scalable warehouses.

How it works

In ELT, an ingestion tool copies tables or files into a raw schema. Transformation models then build layered views and tables.

ETL:  source --> [transform engine] --> warehouse (modeled only)
ELT:  source --> warehouse.raw --> [SQL models] --> warehouse.analytics

An ELT model is a SELECT statement stored in version control:

-- models/orders_clean.sql
SELECT order_id, customer_id,
       CAST(total AS DECIMAL(12,2)) AS total,
       LOWER(status) AS status
FROM raw.orders
WHERE order_id IS NOT NULL

The orchestrator builds models in dependency order, runs tests for uniqueness and nulls, and records lineage. ETL remains valuable when data must be tokenized or filtered before landing, or when transforms need general-purpose code.

Pitfalls

  • Loading raw sensitive data into broadly readable schemas. Restrict raw access and mask columns.
  • Full refreshes of large models every run; prefer incremental models keyed on a watermark.
  • No tests or code review on SQL models, so metric changes go unnoticed.
  • Mixing styles: some logic in the ingestion tool, some in SQL, some in dashboards, so nobody knows where a number comes from.
  • Ignoring warehouse cost: schedule heavy transforms and monitor spend per model.

Key takeaways

  • ELT is the default for analytics on elastic warehouses; ETL for pre-landing compliance or complex code.
  • Keep transformations in version control with tests.
  • Retain raw data, but control who can read it.
  • Use incremental builds to control cost.

✓ When to Use

  • •Cloud warehouse with scalable compute (ELT)
  • •Compliance requires removing fields before landing (ETL)
  • •Analysts own transformations in SQL (ELT)

✗ When Not to Use

  • •Tiny datasets where either adds needless tooling
  • •Transformations need non-SQL logic like ML models inline with ELT only
  • •Target compute cost is prohibitive for heavy transforms

Pros

  • +ELT keeps raw data for reprocessing
  • +ELT lets analysts iterate in SQL
  • +ETL can minimize sensitive data in the target

Cons

  • −ELT stores raw data and raises storage and privacy exposure
  • −ETL needs separate processing infrastructure
  • −Transform logic location can become inconsistent

Cost Profile

Infrastructure

Medium - transform compute in warehouse or separate cluster

Operational

Medium - orchestration and testing

Cognitive

Low - concept is simple

Failure Modes

  • Raw PII lands unprotected in an ELT warehouse

  • Transform jobs inflate warehouse bills

  • Untested model changes silently alter metrics

Real-World Examples

Alternatives

    Related Patterns

      Competency Domains

      data stateeconomics evolution