dbt Incremental Model Pattern
Build efficient dbt incremental models that process only new or changed data instead of full refreshes.
-- models/marts/fct_orders.sql
{{
config(
materialized='incremental',
unique_key='order_id',
incremental_strategy='merge',
on_schema_change='append_new_columns'
)
}}
WITH source_orders AS (
SELECT
order_id,
customer_id,
order_date,
amount,
status,
updated_at
FROM {{ ref('stg_orders') }}
{% if is_incremental() %}
-- Only process rows newer than the last run
WHERE updated_at > (
SELECT COALESCE(MAX(updated_at), '1900-01-01')
FROM {{ this }}
)
{% endif %}
),
enriched AS (
SELECT
o.*,
c.segment AS customer_segment,
c.region,
o.amount * COALESCE(fx.rate, 1) AS amount_usd
FROM source_orders o
LEFT JOIN {{ ref('dim_customers') }} c
ON o.customer_id = c.customer_id
LEFT JOIN {{ ref('stg_fx_rates') }} fx
ON DATE(o.order_date) = fx.rate_date
)
SELECT
*,
CURRENT_TIMESTAMP AS _loaded_at
FROM enrichedSponsored
dbt Cloud
Use Cases
- Efficient data warehouse builds processing only deltas
- dbt model optimization for large datasets
- Merge-based upserts in analytics pipelines
Tags
Related Snippets
Similar patterns you can reuse in the same workflow.
Snowflake MERGE with Slowly Changing Dim
Implement SCD Type 2 in Snowflake using MERGE to track historical changes in dimension tables.
Best for: Tracking full history of dimension changes
dbt Source Freshness and Testing
Configure dbt source freshness checks and schema tests to validate upstream data pipelines.
Best for: Ensuring upstream data sources are fresh
dbt Run and Test — CI/CD Pipeline Script
Bash script for running dbt build with testing, documentation generation, and failure notifications.
Best for: Automating dbt builds in CI/CD pipelines
Python ETL Pipeline Example
Complete extract-transform-load pipeline with error handling, logging, and incremental processing.
Best for: Automating data ingestion from CSV to warehouse