#etl
6 snippets tagged with #etl
SQL MERGE (Standard Upsert)
Use the SQL MERGE statement for atomic insert-or-update operations with matched/not-matched clauses.
Python ETL Pipeline Example
Complete extract-transform-load pipeline with error handling, logging, and incremental processing.
Python Batch Processing Script
Process large files in configurable batches with progress tracking, error handling, and resume support.
Database Sync Script in Python
Sync data between two databases with upsert logic, batch processing, and change detection.
SQL Incremental Load Pattern
Incremental data load using watermark tracking to process only new and updated records efficiently.
SQL Data Deduplication Techniques
Remove duplicate records using ROW_NUMBER, DISTINCT ON, and self-join deduplication strategies.