Pydantic v2's Rust core hides a costly secret at the Python boundary

Libraries Run Rust Inside Python (With PyO3)

Pydantic v2's Rust core hides a costly secret at the Python boundary

Pydantic v2's speed comes from pydantic-core, a Rust extension built with PyO3. This post walks through a JSON parser in Rust exposed to Python via PyO3 and maturin. The surprising bottleneck: converting the parsed Rust tree into Python objects can cost more than the parsing itself. For large documents, that materialization loop dominates end-to-end time. The advice: profile the boundary, not just the algorithm, and consider lazy views instead of eagerly building the whole tree.

A document with 100,000 values means on the order of 100,000 Python objects being created at the boundary, all after parsing is completely done.

More from this day

2026-09-13