</>
pythonintermediate

Pydantic Models for ETL Validation

Parse and validate raw JSON records against Pydantic models before inserting into a database.

python
from pydantic import BaseModel, EmailStr, field_validator
from datetime import datetime
from typing import Optional

class OrderRecord(BaseModel):
    order_id:   int
    customer:   str
    email:      EmailStr
    amount:     float
    currency:   str = 'USD'
    created_at: datetime
    notes:      Optional[str] = None

    @field_validator('amount')
    @classmethod
    def positive_amount(cls, v: float) -> float:
        if v <= 0:
            raise ValueError('amount must be > 0')
        return round(v, 2)

raw = {'order_id':1,'customer':'Alice','email':'alice@example.com','amount':99.99,'created_at':'2024-01-15T10:00:00'}
print(OrderRecord(**raw).model_dump())

Use Cases

  • input validation
  • ETL pipeline gates
  • API data parsing

Tags

#pydantic#validation#etl#schema

Related Snippets

Similar patterns you can reuse in the same workflow.

pythonintermediate

Data Validation with Pydantic

Validate and parse data records using Pydantic models with custom validators and error reporting.

Best for: Validating incoming data before warehouse loading

#validation#pydantic
pythonintermediate

Pandera DataFrame Schema Validation

Use Pandera to validate DataFrame schemas with type checks, value constraints, and custom checks.

Best for: pipeline input validation

#pandera#validation
pythonintermediate

Pandera @check_input and @check_output

Decorate pipeline functions with Pandera schema validators to enforce input and output contracts.

Best for: contract testing

#pandera#validation
typescriptintermediate

Schema Validation with Zod-like Patterns

Build a minimal schema validation library inspired by Zod using TypeScript type inference.

Best for: API request validation

#nodejs#validation