Problem
Moving data from operational sources into analytics storage requires cleaning and modeling. Doing it in the wrong place causes brittle pipelines, lost raw data, or runaway warehouse cost.
Also known as: extract-transform-load, extract-load-transform
Choose whether to transform data before loading into the target or inside the target after loading.
Moving data from operational sources into analytics storage requires cleaning and modeling. Doing it in the wrong place causes brittle pipelines, lost raw data, or runaway warehouse cost.
ETL and ELT differ in where the transform step runs. ETL transforms in a dedicated engine before loading; ELT loads raw data first and transforms inside the destination using its own compute. The shift toward ELT followed cheap cloud storage and scalable warehouses.
In ELT, an ingestion tool copies tables or files into a raw schema. Transformation models then build layered views and tables.
ETL: source --> [transform engine] --> warehouse (modeled only)
ELT: source --> warehouse.raw --> [SQL models] --> warehouse.analytics
An ELT model is a SELECT statement stored in version control:
-- models/orders_clean.sql
SELECT order_id, customer_id,
CAST(total AS DECIMAL(12,2)) AS total,
LOWER(status) AS status
FROM raw.orders
WHERE order_id IS NOT NULL
The orchestrator builds models in dependency order, runs tests for uniqueness and nulls, and records lineage. ETL remains valuable when data must be tokenized or filtered before landing, or when transforms need general-purpose code.
Medium - transform compute in warehouse or separate cluster
Medium - orchestration and testing
Low - concept is simple
Raw PII lands unprotected in an ELT warehouse
Transform jobs inflate warehouse bills
Untested model changes silently alter metrics