</>
pythonadvanced

NumPy Structured Arrays for Records

Use NumPy structured arrays to store heterogeneous record types efficiently without pandas overhead.

python
import numpy as np

dtype = np.dtype([('id', np.int32), ('price', np.float32), ('qty', np.int16), ('label', 'U10')])

records = np.array([(1, 9.99, 3, 'widget'), (2, 24.99, 1, 'gadget'), (3, 4.99, 10, 'bolt')], dtype=dtype)

print(records['price'])
print(records[records['qty'] > 2])

total_revenue = (records['price'] * records['qty']).sum()
print(f'Total revenue: {total_revenue:.2f}')

Use Cases

  • binary record storage
  • memory-efficient arrays
  • C interop

Tags

#numpy#structured-array#memory#records

Related Snippets

Similar patterns you can reuse in the same workflow.

pythonintermediate

Read Large CSV in Chunks with Pandas

Process CSV files larger than RAM by reading in chunks — memory-efficient ETL pattern for data pipelines.

Best for: Processing multi-GB CSV files without running out of memory

#pandas#csv
pythonintermediate

Pandas Memory Reduction via Dtypes

Reduce DataFrame memory by 60-80% by downcasting numeric types and using categorical columns.

Best for: large dataset loading

#pandas#memory
pythonintermediate

Stream Large SQL Query in Chunks

Read millions of rows from SQL in memory-safe chunks using pandas read_sql with chunksize.

Best for: large table extraction

#pandas#sql
pythonintermediate

NumPy Advanced Indexing Patterns

Use fancy indexing, boolean masks, and np.where for fast array transformations without loops.

Best for: numerical computing

#numpy#indexing