What’s new in Python 3.7

Python 3.7, the latest version of the language aimed at making complex tasks simple, is now in production release. The most significant additions and improvements to Python 3.7 include:

• Data classes that reduce boilerplate when working with data in classes.
• A potentially backward-incompatible change involving the handling of exceptions in generators.
• A “development mode” for the interpreter.
• Nanosecond-resolution time objects.
• UTF-8 mode that uses UTF-8 encoding by default in the environment.
• A new built-in for triggering the debugger.

Where to download Python 3.7

You can download Python 3.7 from the Python Software Foundation.

The new features in Python 3.7

Python data classes

Python is known far and wide as a fast and convenient way to manipulate structured data. Python provides classes to structure data and attach common behaviors to instances of that data, but classes with many initializers have long suffered from needing a lot of boilerplate code to instantiate them. For example:

class User():
    def __init__(self, name, user_id, just_joined=True):
        self.name = name
        self.id = user_id
        self.just_joined = just_joined

To automate this kind of class instantiation, Python 3.7 introduces a new module, dataclasses, as described in PEP 557. It provides a decorator that allows the above behavior to be reproduced in a much simpler way, using the type hinting syntax that was introduced in Python 3.5:

@dataclass
class User():
    name: str
    user_id: int
    just_joined: bool=True

The resulting class works like a regular Python class. You can also declare certain fields “frozen” or immutable, and automate (or manually override) the creation of special methods on properties, like __hash__ or __repr__.

Python generator exception handling

A change that has been in the works for some time now, as outlined in PEP 479, is designed to make it easier to debug a StopIteration exception raised by a Python generator. Previously, it was too easy for a generator to raise a StopIteration when it encountered another problem, instead of because it had run out of things to iterate through. This created a whole class of bugs that were hard to trace.

With Python 3.7, when a generator yields up a StopIteration exception, the StopIteration exception is translated into a RuntimeError exception so that it doesn’t silently bubble all the way up through the application’s stack frames. This means some programs that weren’t too discerning about how they handled generator behaviors will throw a RuntimeError in Python 3.7. In Python 3.6, this behavior produced a deprecation warning; in Python 3.7, it produces a full error.

One quick fix is to use a try/except block to catch the StopIteration before it propagates outside the generator. A better solution is to rethink how generators are constructed—for example, to use a return statement to terminate a generator instead of manually raising StopIteration. See PEP 469 for more details on how to remedy this in existing code and how to guard against it in new code.

Python development mode

A new command-line switch for the Python interpreter, -X, lets the developer set a number of low-level options for the interpreter. With Python 3.7, the option -X dev enables “development mode,” a slew of runtime checks that normally have a big impact on performance, but are useful for a developer during the debugging process.

Options activated by -X dev include:

• Debug mode for the asyncio module. This provides more detailed logging and exception handling for asynchronous operations, which can be difficult to debug or reason about.
• Debug hooks for memory allocators. This is useful for those writing CPython extensions. It enables more explicit runtime checks on how CPython allocates and frees memory internally.
• Enabling the faulthandler module, so that tracebacks are always dumped after a crash.

Python time functions with nanosecond resolution

A new class of time functions in Python 3.7 return time values with nanosecond precision. Even though Python is an interpreted language, Python core developer Victor Stinner made a case for reporting time with nanosecond accuracy. The biggest reason is to avoid losing precision when dealing with converting time values that are recorded by other programs, such as databases.

The new time functions use the suffix _ns. For example, the nanosecond version of time.process_time() is time.process_time_ns(). Note that not all time functions have nanosecond equivalents, as some of them don’t benefit from it.

Python UTF-8 mode

Python has long supported UTF-8 for easy handling of strings and text. But the locale in the surrounding environment is sometimes still ASCII, not UTF-8, and the mechanisms for detecting locale aren’t always reliable.

Python 3.7 adds what’s called “UTF-8 mode,” enabled by the -X command line switch, that assumes UTF-8 is the locale provided by the environment. In Posix locales, UTF-8 mode is enabled by default, but it’s disabled by default elsewhere to avoid breaking backward compatibility. It’s worth experimenting with having UTF-8 mode on by default, but it shouldn’t be enabled in production until you’re confident that all interactions between Python and the surrounding environment use UTF-8.

Built-in breakpoint() function

Python comes with its own built-in debugger, although it can hook into third-party debugging tools as well, as long as they can speak to Python’s internal debugging API. What Python has lacked until now, though, is a standardized way to programmatically trigger the debugger from within a Python app.

Python 3.7 adds breakpoint(), a built-in function that causes execution to switch to the debugger when called. The debugger in question doesn’t have to be Python’s own pdb; it can be any debugger that has previously been set as the debugger of choice. Previously, the debugger would have had to be set manually, then invoked, which made for more verbose code. breakpoint() makes it possible to invoke the debugger in a single command, and to keep a clean separation from setting up the debugger and invoking it.

Other new Python 3.7 features

Dozens of other changes have landed in Python 3.7. Here are some others you’re likely to encounter when working with the latest version of Python:

C API for thread-local storage support. The Thread Specific Storage (TSS) API, described in PEP 539, replaces the legacy Thread Local Storage (TLS) API. People customizing CPython, or writing CPython extensions that use the interpreter’s internal APIs, will need to be aware of this.

Module attribute access customization. When you create a module in a Python program, you can now customize the behavior of attribute access on instances of that module. You can do this by just creating a __getattr__ method inside the module, same as you would for a class. This way, things like requests for nonexistent functions or methods inside the module can be intercepted, flagged, or proxied.

Python importlib resources.

The importlib module can now be used to read “resources,” or binary artifacts shipped with a Python application such as a data file. This way, a developer can access those files through importlib’s abstractions, so it doesn’t matter if they’re stored in a .zip file or in a directory somewhere on the system.

Under-the-hood optimizations. Many individual operations are now faster:

• Method calls are up to 20 percent thanks to new opcodes. (You don’t need to worry about the implications of this unless you’re writing code that manipulates Python opcodes directly.)
• Faster case-insensitive matching in regular expressions, sometimes as much as 20 times faster.
• Some constants in source code are now optimized more efficiently.

For the complete rundown of what’s new in Python 3.7, the Python Software Foundation has published a document covering all the details.