The Python programming language releases new versions yearly, with a feature-locked beta release in the first half of the year and the final release toward the end of the year.
Python 3.11 has just been released, and developers are encouraged to try out this latest version on non-production code, both to verify that it works with your programs and to get an idea of whether your code will benefit from its performance enhancements.
Here's a rundown of the most significant new features in Python 3.11 and what they mean for Python developers.
Speed improvements
Many individual performance improvements landed in Python 3.11, but the single biggest addition is the specializing adaptive interpreter. Since an object's type rarely changes, the interpreter now attempts to analyze running code and replace general bytecodes with type-specific ones. For instance, binary operations (add, subtract, etc.) can be replaced with specialized versions for integers, floats, and strings.
Python function calls also require less overhead in Python 3.11. Stack frames for function calls now use less memory and are more efficiently designed. Also, while recursive calls aren't tail-optimized (which probably isn't possible in Python, anyway), they are more efficient than in previous versions. The Python interpreter itself also starts faster, and core modules needed for the Python runtime are stored and loaded more efficiently.
According to the official Python benchmark suite, Python 3.11 runs around 1.25 times faster than Python 3.10. Note that this speedup is an aggregate measure. Some things run much faster, but many others run only slightly faster or about the same. Still, the best part about these improvements is that they come for free. You don't need to make any code changes for Python programs to take advantage of Python 3.11's speedups.
Enhanced error information
Another immediately useful feature in Python 3.11 is more detailed error messages. Python 3.10 already had better error reporting, thanks to the new parser used in the interpreter. Now, Python 3.11 expands on that by providing detailed feedback about what specific part of a given expression caused an error.
Consider the following code, which throws an error:
x = [1,2,3] z = x[1][0]
In Python 3.10, we'd receive the following error message, which is not very helpful:
File "C:\Python311\code.py", line 2, in <module>
z = x[1][0]
TypeError: 'int' object is not subscriptable
Rather than leave us wondering which int is not scriptable, the error trace in Python 3.11 points to the exact part of the line that generates the error:
File "C:\Python311\code.py", line 2, in <module>
z = x[1][0]
~~~~^^^
TypeError: 'int' object is not subscriptable
Now, there is no ambiguity about where the problem lies.
Exception improvements
Exceptions, Python's error-handling mechanism, have received many new features in Python 3.11:
- Multiple exceptions can be raised and handled at once with the new
except*syntax and the newExceptionGroupexception type. This allows the elegant handling of issues where multiple errors can be raised together, such as when dealing with asynchronous or concurrent methods or when dealing with multiple failures when retrying an operation. - "Zero-cost" exceptions: Exceptions now have no cost to a program unless they are actually raised. This means the default path for a
try/exceptblock is faster and uses less memory. - The time needed to catch an exception has been reduced by around 10%.
- Exceptions can be enriched with contextual notes, separate from the text of the exception itself.
Typing improvements
Python's type-hinting features make larger codebases easier to manage and analyze, and have increased significantly with each revision since Python 3.5. Python 3.11 brings in several new type-hinting additions.
The Self type
Class methods that return self previously required obtuse and verbose annotations to be useful. typing.Self lets you annotate the return value of a class method as, simply, Self. You get useful and predictable results from your analysis tools for such methods.
Arbitrary string literal type
Previously, type annotations had no way to indicate a given variable needed to be a string literal—that is, a string defined in source code. The new typing.LiteralString annotation fixes that. Using the new annotation, linters can test for a variable being either a string defined in source or a new string composed of only source-defined strings.
Dataclass transforms
Since Python 3.7, dataclasses have made it easier to define classes that followed common patterns for creating properties based on their initialization parameters. But there was no standard mechanism for allowing things that behaved like dataclasses (but weren't dataclasses themselves) to use type annotations to declare their behavior. Dataclass transforms adds the typing.dataclass_transform decorator to indicate how a given function, class, or metaclass behaves like a dataclass.
Variadic generics
The original proposal for type hints included TypeVar, a way to specify a generic function using a single parameterized type—for example, a type T that could be an int or a float. Python 3.11 adds TypeVarTuple, or "variadic generics," which you can use to specify a placeholder for not just one type but a series of them, expressed as a tuple. This would be especially useful in libraries like NumPy, where you could perform ahead-of-time checks for errors like whether a supplied array was the correct shape.
TOML read-only support in stdlib
Python uses TOML, or Tom's Obvious Minimal Language, as a configuration format (as in pyproject.toml), but doesn't expose the ability to read TOML-format files as a standard library module. Python 3.11 adds tomllib to address that problem. Note that tomllib doesn't create or write TOML files; for that you need a third-party module like Tomli-W or TOML Kit.
Atomic grouping and speedups for regex
Python's re module, for working with regular expressions, has lacked a few features found in other implementations of regular expressions. One is atomic grouping, widely supported in other languages. Python 3.11 now supports this pattern using the common syntax for atomic groupings (as an example, (?>...)).
The re module's pattern matching engine has also been rewritten somewhat, and runs about 10% faster.
Removing 'dead batteries' from the standard library
PEP 594 kicked off an effort to remove many so-called dead batteries, or modules that are obsolete or unmaintained, from the Python standard library. As of Python 3.11, those libraries are marked as deprecated but not yet removed; they will be removed entirely in Python 3.13.
Other Python 3.11 additions, fixes, and changes
Many more smaller improvements also landed in Python 3.11:
- Python objects require less memory, as their namespaces are now lazily created, and their namespace dictionaries now share keys whenever possible.
- Dictionaries where all keys are Unicode no longer need to store hashes, thus reducing the size of the dictionary and allowing more cache efficiency.
- The CPython runtime, the reference interpreter for Python, now has experimental support for being compiled to WebAssembly. This may aid the future development of projects like PyScript, which allow a Wasm-compiled Python runtime to operate in the browser.