pythonbeginner
SQLite + Pandas Local Data Pipeline
Run a lightweight local ETL with SQLite and pandas: load CSV, transform, persist to SQLite.
pythonPress ⌘/Ctrl + Shift + C to copy
import sqlite3
import pandas as pd
df = pd.read_csv('sales.csv', parse_dates=['date'])
df['month'] = df['date'].dt.to_period('M')
df['revenue'] = df['price'] * df['qty']
monthly = df.groupby('month')['revenue'].sum().reset_index()
monthly['month'] = monthly['month'].astype(str)
con = sqlite3.connect('analytics.db')
monthly.to_sql('monthly_revenue', con, if_exists='replace', index=False)
print(pd.read_sql('SELECT * FROM monthly_revenue LIMIT 5', con))
con.close()Use Cases
- local analytics
- prototyping
- lightweight data pipelines
Tags
Related Snippets
Similar patterns you can reuse in the same workflow.
pythonintermediate
Read Large CSV in Chunks with Pandas
Process CSV files larger than RAM by reading in chunks — memory-efficient ETL pattern for data pipelines.
Best for: Processing multi-GB CSV files without running out of memory
#pandas#csv
pythonintermediate
Pandas Vectorised Operations vs Apply
Compare apply vs vectorised pandas operations for performance-critical column transformations.
Best for: feature engineering
#pandas#vectorization
pythonbeginner
Flatten Nested JSON with pandas
Use pd.json_normalize to flatten deeply nested API responses into a flat DataFrame.
Best for: API response flattening
#pandas#json
pythonintermediate
Pandas Method Chaining with .pipe()
Use the .pipe() method to create clean, readable pandas transformation chains.
Best for: clean ETL code
#pandas#pipe