Python and Rust occupy seemingly opposite ends of the language spectrum. Python, interpreted at runtime, offers developers a flexible and comfortable programming environment, but at the cost of raw speed. Rust provides the speed, plus guarantees of memory safety, but requires that you learn a new paradigm for handling memory operations.
In theory, these languages shouldn't compete; they should cooperate. And in practice, they can. Rust can benefit from Python's ease of use, and Python can benefit from Rust's speed and safety.
If you want to use Rust with Python, or Python with Rust, you'll need to have at least passing familiarity with both languages to get the best results. You'll also need to decide which of the two is your primary language, as the options for each approach are significantly different.
Calling Rust from Python with PyO3
If Python is your primary language, integrating with Rust works in conceptually the same way as integrating Python with C. The default implementation of Python, written in C, uses extensions either written in C or using a C-compatible ABI. Extensions written in Rust that use the same ABI will also work, although that isn't automatic—you have to use crates designed to provide bindings for Rust functions to the Python C API.
Creating Rust bindings in Python
The most widely recognized project for creating Rust bindings in Python is PyO3. It can be used to write Python modules in Rust, or to embed the Python runtime in a Rust binary.
PyO3 leverages another project, Maturin, which is a tool for authoring Rust crates with Python packaging and bindings. When installed in a Python virtual environment, Maturin can be used from the command line to initialize a new Rust project with Python bindings enabled. The developer uses directives in the Rust code to indicate what Rust functions to expose to Python, and how to expose the whole of the Rust project to Python as an importable module.
Mapping Rust and Python types
One of PyO3's useful aspects is its mappings between Rust and Python types. Functions written in Rust can accept either native Python types or Rust types converted from Python types. For instance, a bytearray or bytes object in Python can map elegantly to a Vec<u8> in Rust, and a str in Python can be rendered as a Rust String.
Converting from Python to Rust incurs a per-call cost, but it frees you from having to use Python types entirely in the Rust code. In the Cython world, this is akin to the conversions to C types: there's a cost for each conversion, but they bring major speedups if your goal is numerical processing entirely in C.
Calling Python from Rust with the cpython crate
If you're primarily a Rust developer but want to use Python inside a Rust application, the cpython crate is a straightforward way to do it. The cpython crate provides Rust bindings to the CPython interpreter, which is the most common Python runtime (so named because it's written in C).
Rust programs can invoke the CPython interpreter and work with it, allowing you to create and manipulate Python objects in Rust and make library calls. One example in the documentation shows how to initialize the Python runtime, import modules, create Python objects, and execute method calls.
The cpython crate also includes a few useful macros. The py_fn! macro, for instance, wraps a Rust function so that it's callable from Python. The py_class! macro lets you generate Rust classes as Python class objects.
If you're more familiar with Rust than Python, it's a good idea to have at least passing familiarity with the Python C API and the various Python object types before diving in.
Performance tip
An important caveat with both cpython and PyO3 is to always minimize the number of times data is passed back and forth between the two languages. Each call from Python to Rust or vice versa incurs some overhead. If the overhead outweighs the work you're doing in Rust, you won't see any significant performance improvement.
As an example, if you're looping over an object collection, send the object to Rust and perform the looping there. This is more efficient than looping on the Python side and calling the Rust code with each iteration of the loop.
This guideline also applies generally to integrations between Python and other code that uses the Python C ABI, such as Cython modules.