Python Interop with PyO3

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

The reference material consulted while preparing these pages is listed in the section bibliography.

This page is written against Rust 1.99.0, PyO3 0.29.3, maturin 1.15 and CPython 3.13, checked on 2026-10-06. PyO3 binds Rust and Python in both directions: you can write Python extension modules in Rust — a common way to speed up a hot loop in a Python service without rewriting the service — and you can embed a Python interpreter in a Rust program. maturin builds and publishes the resulting wheels. Every example below was built with maturin develop and tested with pytest.

A first extension module

[package]
name = "pyshelf"
version = "0.1.0"
edition = "2024"

[lib]
name = "pyshelf"
crate-type = ["cdylib"]

[dependencies]
pyo3 = "0.29.3"

Source: PyO3 guide: Installation

[build-system]
requires = ["maturin>=1.15,<2"]
build-backend = "maturin"

[project]
name = "pyshelf"
requires-python = ">=3.9"
dynamic = ["version"]

[tool.maturin]
features = ["pyo3/extension-module"]

Source: PyO3 guide: Installation

use pyo3::exceptions::PyValueError;
use pyo3::prelude::*;

/// Normalises an ISBN-13 (removes hyphens and spaces) and validates its check digit.
#[pyfunction]
fn normalise_isbn(text: &str) -> PyResult<String> {
    let digits: String = text.chars().filter(|c| !matches!(c, '-' | ' ')).collect();
    if digits.len() != 13 || !digits.bytes().all(|b| b.is_ascii_digit()) {
        return Err(PyValueError::new_err(format!("not an ISBN-13: {text:?}")));
    }
    let sum: u32 = digits.bytes().enumerate().map(|(i, b)| u32::from(b - b'0') * if i % 2 == 0 { 1 } else { 3 }).sum();
    if !sum.is_multiple_of(10) {
        return Err(PyValueError::new_err("bad check digit"));
    }
    Ok(digits)
}

/// Scores every title against a query. CPU-bound: detaches from the interpreter so other
/// Python threads can run while Rust computes.
#[pyfunction]
fn score_titles(py: Python<'_>, titles: Vec<String>, query: &str) -> Vec<(String, u32)> {
    let query = query.to_lowercase();
    py.detach(|| {
        titles
            .into_iter()
            .map(|t| {
                let score = t.to_lowercase().matches(query.as_str()).count() as u32;
                (t, score)
            })
            .collect()
    })
}

/// A catalogue object usable from Python.
#[pyclass]
struct Catalog {
    titles: Vec<String>,
}

#[pymethods]
impl Catalog {
    #[new]
    fn new() -> Self {
        Catalog { titles: Vec::new() }
    }

    fn add(&mut self, title: String) {
        self.titles.push(title);
    }

    fn search(&self, query: &str) -> Vec<String> {
        let q = query.to_lowercase();
        self.titles.iter().filter(|t| t.to_lowercase().contains(&q)).cloned().collect()
    }

    fn __len__(&self) -> usize {
        self.titles.len()
    }

    fn __repr__(&self) -> String {
        format!("Catalog({} titles)", self.titles.len())
    }
}

/// Calls back into Python: uses the `json` module from Rust.
#[pyfunction]
fn to_json(py: Python<'_>, titles: Vec<String>) -> PyResult<String> {
    let json = py.import("json")?;
    json.call_method1("dumps", (titles,))?.extract()
}

/// The Python module `pyshelf`.
#[pymodule]
mod pyshelf {
    #[pymodule_export]
    use super::{Catalog, normalise_isbn, score_titles, to_json};
}

Source: The PyO3 user guide

The pieces:

#[pyfunction] exposes a Rust function. Arguments and return values are converted automatically: &str, String, integers, floats, bool, Vec<T>, HashMap<K, V>, tuples and Option<T> (as None) all map to the obvious Python types.
#[pyclass] turns a struct into a Python class; #[pymethods] exposes its methods, #[new] is the constructor, and dunder methods (__len__, __repr__, __iter__, …) implement Python protocols.
#[pymodule] on an inline mod declares the module; each #[pymodule_export] item becomes an attribute of it. The module name must match lib.name in Cargo.toml and the package name in pyproject.toml.
Functions that need the interpreter take a Python<'py> token, the proof that the current thread is attached to the Python interpreter.

Building with maturin

python -m venv .venv && . .venv/bin/activate
pip install maturin pytest
maturin develop               # build in debug mode and install into the active virtualenv
maturin develop --release     # optimised build: always use this before measuring performance
python -c "import pyshelf; print(pyshelf.normalise_isbn('978-0-441-01359-3'))"
# 9780441013593
maturin build --release       # build wheels into target/wheels/ for distribution

Source: maturin user guide

maturin build produces platform-specific wheels; enable the abi3-py39-style features of PyO3 to build one wheel that works on all CPython versions from 3.9 up (the stable ABI), and use maturin’s GitHub Actions template to build wheels for Linux (manylinux/musllinux), macOS and Windows.

Types, Bound<'py, T> and errors

Values owned by Python are accessed through smart pointers tied to the Python<'py> token:

Bound<'py, T> — a reference to a Python object valid while the thread is attached (the everyday type: Bound<'py, PyList>, Bound<'py, PyAny>);
Py<T> — an owned reference that can be stored in Rust structs and sent between threads, and turned back into a Bound with .bind(py);
Borrowed<'a, 'py, T> — a borrowed reference used in some low-level APIs.

The FromPyObject and IntoPyObject traits drive conversions (.extract::<T>() and automatic argument conversion). Errors are PyErr; a function returning PyResult<T> (an alias for Result<T, PyErr>) raises a Python exception on Err. Create exceptions with PyValueError::new_err(...), PyKeyError, etc., or implement From<MyError> for PyErr so ? converts your own errors. A Rust panic inside an extension becomes a pyo3_runtime.PanicException in Python.

Source: PyO3 guide: Python object types

The GIL, attaching and detaching

CPython’s standard build has a Global Interpreter Lock: only one thread runs Python bytecode at a time. Rust code holding the Python<'py> token is attached to the interpreter and, in the standard build, holds the GIL. For CPU-bound Rust work that does not touch Python objects, detach from the interpreter so other Python threads can run in parallel: py.detach(|| ...), as score_titles does above. Calling into Python from a thread that is not attached uses Python::attach(|py| ...).

PyO3 0.26 renamed these APIs to reflect that, with free-threaded Python (PEP 703, officially supported from CPython 3.14), there may be no GIL at all: Python::with_gil became Python::attach and allow_threads became detach. Since PyO3 0.28 extension modules support free-threaded interpreters by default; a module that relies on the GIL for thread safety can opt out with #[pymodule(gil_used = true)]. #[pyclass] types must then be Sync (or use unsendable), because they can genuinely be accessed from several threads at once.

Source: PyO3 guide: Supporting free-threaded Python

Testing with pytest

Test the extension the way Python users will use it: build it into a virtualenv and run pytest. This also lets an existing Python test suite verify a Rust rewrite of a module (Mara & Holmes, Refactoring to Rust, chapter 7):

import pytest
import pyshelf


def test_normalise_isbn():
    assert pyshelf.normalise_isbn("978-0-441-01359-3") == "9780441013593"


def test_bad_isbn_raises_value_error():
    with pytest.raises(ValueError, match="bad check digit"):
        pyshelf.normalise_isbn("978-0-441-01359-4")


def test_catalog_class():
    c = pyshelf.Catalog()
    c.add("Dune")
    c.add("Dune Messiah")
    assert len(c) == 2
    assert c.search("messiah") == ["Dune Messiah"]
    assert repr(c) == "Catalog(2 titles)"


def test_score_titles_and_json():
    assert pyshelf.score_titles(["Dune", "Emma"], "dune") == [("Dune", 1), ("Emma", 0)]
    assert pyshelf.to_json(["Dune"]) == '["Dune"]'

Source: pytest: How to write and report assertions in tests

maturin develop && python -m pytest -q tests
# ....                                                                     [100%]
# 4 passed in 0.01s

Source: maturin: Local Development

Rust-side unit tests of pure functions run with cargo test as usual; tests that need the interpreter use Python::attach with the auto-initialize feature.

Embedding Python in Rust

The other direction: a Rust binary that runs Python code — plugins, configuration scripts, or a library that only exists in Python. Enable auto-initialize (which links against libpython) and attach to the interpreter:

[dependencies]
pyo3 = { version = "0.29.3", features = ["auto-initialize"] }

Source: PyO3 guide: Features reference

use pyo3::prelude::*;
use pyo3::types::PyDict;

fn main() -> PyResult<()> {
    Python::attach(|py| {
        let statistics = py.import("statistics")?;
        let ratings = vec![5, 3, 4, 4];
        let mean: f64 = statistics.call_method1("mean", (ratings,))?.extract()?;

        let locals = PyDict::new(py);
        locals.set_item("title", "Dune")?;
        let shout: String = py.eval(c"title.upper() + '!'", None, Some(&locals))?.extract()?;
        println!("mean rating {mean}, {shout}");
        Ok(())
    })
}

Source: PyO3 guide: Calling Python from Rust

When is a Rust extension worth it?

A Rust extension pays off when a measurable hot spot is CPU-bound and spends its time in Python-level loops: parsing, validation, scoring, numeric kernels that NumPy cannot vectorise, or work that can run in parallel once detached from the interpreter. It does not help I/O-bound code (an async framework already waits efficiently), and every call crosses a conversion boundary, so batch work into few calls with large inputs rather than many tiny calls. Profile first; the FastAPI section’s Performance & Observability page discusses PyPy, Cython and Rust extensions as options for a Python service, and Incremental Adoption & Refactoring compares in-process extensions with running Rust as a separate service.

Where the books are outdated

Mara & Holmes (Refactoring to Rust, chapters 6-8) use PyO3 0.14, whose API has changed substantially:

the GIL-ref types (&PyModule, &PyList, &PyAny) were replaced by the Bound<'py, T> smart pointer (0.21-0.23);
Python::with_gil and allow_threads are now Python::attach and py.detach (0.26);
modules can be declared as inline #[pymodule] mod blocks with #[pymodule_export] instead of a function that calls m.add_function(wrap_pyfunction!(...));
free-threaded Python is supported by default (0.28), which requires #[pyclass] types to be thread-safe.

The concepts in the book — maturin, #[pyfunction], releasing the GIL for parallel work, testing with pytest — remain valid; the code needs these updates.

References

The PyO3 user guide (getting started, functions, classes, modules, conversions, error handling, free-threading, Python from Rust)
PyO3 API documentation
maturin user guide
PEP 703: Making the Global Interpreter Lock Optional in CPython
pytest documentation
Lily Mara and Joel Holmes, Refactoring to Rust (Manning, 2025)