pythonbeginner
Read Files from S3 with fsspec
Access S3 files directly with fsspec and pandas without boto3 boilerplate.
pythonPress ⌘/Ctrl + Shift + C to copy
import pandas as pd
df = pd.read_csv(
's3://my-bucket/data/sales.csv',
storage_options={'key': 'AWS_KEY', 'secret': 'AWS_SECRET'},
)
print(df.shape)
df2 = pd.read_parquet(
's3://my-bucket/warehouse/events/',
storage_options={'anon': False},
filters=[('year', '==', 2024)],
)
print(df2.dtypes)Use Cases
- cloud data access
- S3 reading
- serverless ETL
Tags
Related Snippets
Similar patterns you can reuse in the same workflow.
typescriptintermediate
AWS SDK v3 — S3 Operations in TypeScript
Perform S3 operations with AWS SDK v3: upload, download, list, presigned URLs, and multipart upload.
Best for: Server-side file uploads to S3
#aws#s3
pythonadvanced
Python ETL Pipeline Example
Complete extract-transform-load pipeline with error handling, logging, and incremental processing.
Best for: Automating data ingestion from CSV to warehouse
#etl#pipeline
pythonintermediate
Python Batch Processing Script
Process large files in configurable batches with progress tracking, error handling, and resume support.
Best for: Processing large CSV files that don't fit in memory
#batch-processing#python
pythonadvanced
Database Sync Script in Python
Sync data between two databases with upsert logic, batch processing, and change detection.
Best for: Replicating data between databases
#database#sync