</>
pythonbeginner

Pydantic Settings for Pipeline Config

Manage pipeline configuration from environment variables and .env files with Pydantic Settings.

python
from pydantic_settings import BaseSettings, SettingsConfigDict
from pydantic import PostgresDsn, RedisDsn
from typing import Optional

class PipelineConfig(BaseSettings):
    model_config = SettingsConfigDict(env_file='.env', env_file_encoding='utf-8')

    database_url:   PostgresDsn
    redis_url:      Optional[RedisDsn] = None
    batch_size:     int = 5000
    workers:        int = 4
    dry_run:        bool = False
    s3_bucket:      str

cfg = PipelineConfig(_env_file='.env')
print(f'batch_size={cfg.batch_size}, workers={cfg.workers}, dry_run={cfg.dry_run}')

Use Cases

  • pipeline configuration
  • 12-factor apps
  • environment management

Tags

#pydantic#config#settings#env

Related Snippets

Similar patterns you can reuse in the same workflow.

pythonbeginner

Type-Safe Settings with Pydantic

Load and validate environment variables into a typed settings object using pydantic-settings with defaults.

Best for: Application configuration

#pydantic#settings
typescriptbeginner

Environment Secrets Loader

Loads and validates required environment variables at startup and throws meaningful errors for missing ones.

Best for: Application bootstrap

#env#config
pythonintermediate

Data Validation with Pydantic

Validate and parse data records using Pydantic models with custom validators and error reporting.

Best for: Validating incoming data before warehouse loading

#validation#pydantic
pythonintermediate

Pydantic Models for ETL Validation

Parse and validate raw JSON records against Pydantic models before inserting into a database.

Best for: input validation

#pydantic#validation