I have been experimenting with combining Rust and Python in a few open source projects, mainly to bring together the strengths of both languages.
https://github.com/kannandreams/tuff
The pattern that worked well for me was to keep Python as the engineer-facing layer, where users define workflows, configure behaviour, and interact with the project, while moving selected performance-sensitive parts into Rust.
Instead of rewriting the entire project in Rust, I treated Rust as the engine underneath Python. Python continued to provide the familiar interface, while moving selected performance-sensitive parts into Rust to take advantage of its performance, memory safety, and efficient concurrency.
This article shares what I learned from that approach, how the integration works at a fundamental level, and how to build a small project where Python acts as the public interface and Rust powers the underlying implementation.
The most widely used toolchain for this architecture is:
PyO3 for defining the interface between Rust and Python.
maturin for compiling, packaging, installing, and publishing the extension.
Cargo for managing the Rust code.
Python package manager such as
uvfor managing the Python project.
1. Example Project
Rather than jumping straight into PyO3 or maturin, let’s first understand what we’re building.
Throughout this guide we’ll use a simple project called FastStats. It exposes a Python API for calculating statistical summaries while delegating the computation to a Rust implementation. Although the example is intentionally small, it demonstrates the same architecture used by many production Python packages that rely on Rust for performance-critical components.
A typical architecture might look like this:
From the Python developer's perspective, this looks like an ordinary function call. But what actually happens after calculate_summary() is invoked? How does Python execute Rust code, and how does the result make its way back into Python?
Rust performs the underlying computation:
fn calculate_summary(values: Vec<f64>) -> Summary {
// Native implementation
}The boundary between them is handled by PyO3. This is different from running a Rust program as a separate process using subprocess. With PyO3, the Rust code is loaded directly into the Python application, allowing Python to call Rust functions just like regular Python functions.
2. Fundamental Architecture
When we write import faststats. Python starts looking for something named faststats that it can import. For a typical Python package, that usually means finding Python source files like in below project structure and reads these .py files and executes them.
faststats/
├── __init__.py
├── api.py
└── utils.pyA Rust-powered package works a little differently. Instead of loading only Python source files, Python can also load a compiled library produced by Rust, for example:
faststats/
├── __init__.py
└── _core.cpython-314-darwin.soHere, _core.cpython-314-darwin.so is the Rust code that has already been compiled into machine code. Python loads this file when the package is imported, making the Rust functions available just like ordinary Python functions.
The filename looks unusual because it contains information about where the library can run:
cpython-314indicates it was built for CPython 3.14. This compatibility is defined by Python’s Application Binary Interface (ABI), which specifies how compiled libraries communicate with the Python interpreter at the binary level.darwinindicates it was built for macOS. On Linux or Windows, or for a different Python version, the filename would look different.
Python locates and loads the compiled module.
PyO3 converts Python arguments into Rust types.
Rust executes the native function.
PyO3 converts the Rust result back into a Python value.
3. Toolchains
PyO3
If you're new to Rust, one concept worth understanding is macros. Unlike functions, which run when your program executes, macros generate or transform Rust code during compilation. They are commonly used to reduce boilerplate and add functionality without requiring you to write repetitive code yourself.
PyO3 takes advantage of Rust's procedural macros to expose Rust code as Python objects. By adding annotations such as #[pyfunction], #[pyclass], and #[pymodule], PyO3 generates the necessary binding code that allows Python to call Rust functions and work with Rust types.
For example:
#[pyfunction]
fn add(left: i64, right: i64) -> i64 {
left + right
}Here, #[pyfunction] isn't part of the Rust language itself. It's a procedural macro provided by PyO3. During compilation, it generates the glue code needed to expose the add function as a normal Python function like this.
from faststats import add
print(add(10, 20))These macros let ordinary Rust functions, structs, methods, and modules appear as Python objects.
PyO3 supports both creating native Python modules in Rust and embedding Python inside a Rust application.
maturin
PyO3 makes it possible for Rust and Python to communicate, but it doesn't build or package your project. Once you've written the Rust code and defined the Python bindings, you still need a way to compile the project, integrate it with Python's packaging ecosystem, and produce a package that users can install. That's where maturin comes in. Its main commands include
maturin new
// compiles the project and installs it directly into the active Python environment
maturin develop
// creates distributable wheel files, normally under target/wheels
maturin buildCargo
Cargo.toml
src/lib.rs
src/algorithm.rs
tests/Cargo remains responsible for the Rust side.
Rust dependencies continue to come from crates.io and are declared in Cargo.toml
pyproject.toml
The Python package metadata and build backend are configured in pyproject.toml.
[build-system]
requires = ["maturin>=1.8,<2.0"]
build-backend = "maturin"4. Project Structure
A simple mixed layout works well for many Python and Rust projects. For larger codebases, separating the pure Rust core from the PyO3 bindings creates a cleaner architecture and makes the Rust logic easier to reuse and test independently.
Source code
The complete faststats example used throughout this article is available on GitHub. The Rust implementation lives in src/summary.rs, while the PyO3 binding layer is implemented in src/lib.rs.
GitHub : https://github.com/kannandreams/when-engineers-meet-ai/tree/main/code/faststats
The Python package is stored under python/faststats. The compiled Rust extension will be exposed internally as:
faststats._coreSuppose your Rust extension is compiled as:
faststats._coreThis means the native Rust module can be imported directly:
from faststats._core import calculate_summaryTechnically, this works. However, we usually don’t want users to depend on _core. Instead, we create a clean public API in __init__.py (or api.py):
from ._core import calculate_summary
__all__ = ["calculate_summary"]Now users write:
from faststats import calculate_summaryThis process is called re-exporting. The Python package simply exposes selected functions from the internal _core module as part of the public package interface.
In Cargo.toml file,
[package]
name = "faststats"
version = "0.1.0"
edition = "2024"
[lib]
name = "_core"
crate-type = ["cdylib"]
[dependencies]
pyo3 = { version = "0.29", features = ["extension-module"] }Why cdylib?
By default, Rust builds libraries that are intended to be used by other Rust programs. Python, however, expects a shared library that it can load at runtime. Setting the crate type to cdylib tells Rust to build the project in a format that Python can import as a native extension.
Why must the name be _core ?
The Rust library name, the PyO3 module name, and the Python package configuration all need to agree on the same module name.
// Cargo TOML
[lib]
name = "_core"
// Rust Code
#[pymodule]
fn _core(...) {
...
}
// pyproject.toml
[tool.maturin]
module-name = "faststats._core"All three use the name _core.
5. The Cost Model
Rust is not automatically faster merely because the function body is written in Rust. The total execution time is closer to:
Total time = Python-side preparation + argument conversion + boundary crossing + Rust execution + result conversion + Python-side processing
Rust integration works best when:
The computation is substantial.
Calls are batched.
Data conversion is limited.
Parallel execution or native libraries provide a meaningful advantage.
6. Python Interpreter Lock and Rust Parallelism
Most Python installations today still use the traditional Global Interpreter Lock (GIL), which allows only one thread to execute Python bytecode at a time. While Rust code can run concurrently, any interaction with Python objects must follow the interpreter's threading rules. Starting with Python 3.13, a free-threaded build is available that removes the traditional GIL.
For CPU-intensive work, PyO3 allows Rust to temporarily detach from the Python interpreter while executing native code. This gives Rust the freedom to use multiple threads without unnecessarily holding onto Python runtime resources.
use pyo3::prelude::*;
use rayon::prelude::*;
#[pyfunction]
fn parallel_sum(py: Python<'_>, values: Vec<f64>) -> f64 {
py.detach(|| {
values.par_iter().copied().sum()
})
}Notice that the data is first converted into native Rust types (Vec<f64>). Once inside the detached block, the computation no longer depends on Python objects, allowing Rust to fully leverage its own concurrency libraries such as Rayon.
7. Packaging and Wheels
A pure Python wheel may run on many platforms because it contains Python source.
A Rust extension wheel contains native machine code, so distributions usually need to be built for combinations such as:
// Linux x86-64, Linux ARM64
// macOS Intel, macOS Apple Silicon
// Windows x86-64
uv run maturin build --release
//The generated file will appear under:
target/wheels/
// Install it for testing:
uv pip install target/wheels/faststats-*.whl7. Mental Model
The cleanest way to understand a mixed Python–Rust project is to view it as four layers:
