Python 3.7 有什么新变化

编者:Elvis Pranskevichus <elvis@magic.io>

本文解释了 Python 3.7 相比 3.6 的新增特性。Python 3.7 于 2018 年 6 月 27 日发布。完整的详情可参阅 更新日志

摘要 - 发布重点

新的语法特性:

  • PEP 563, 类型注解延迟评估.

向后不兼容的语法更改:

新的库模块:

新的内置特性:

对 Python 数据模型的改进:

  • PEP 562, 自定义可访问的模块属性。
  • PEP 560, typing模块和泛型类型的核心支持。
  • dict 对象会保持插入时的顺序这个特性 正式宣布 成为 Python 语言官方规范的一部分。

标准库中的重大改进:

CPython 实现的改进:

C API 的改进:

  • PEP 539,用于线程本地存储的新 C API

文档的改进:

此版本在诸多方面有显著的性能改进。Optimizations 章节详细列出了它们。

和之前的 Python 版本存在兼容性的更改列表,请参阅 Porting to Python 3.7 章节。

新的特性

PEP 563:延迟的标注求值

随着 PEP 3107 加入标注功能并在 PEP 526 进一步细化,Python 中类型提示的出现揭示了两个明显的可用性问题:

  • 标注只能使用在当前作用域中已经存在的名称,也就是说,它们不支持任何形式的前向引用;而且——
  • 标注源码对 Python 程序的启动时间有不利的影响。

这两个问题都可以通过延迟标注求值来解决。在定义标注的时候,编译器并不会编译执行相应表达式的代码,而是保存与相应表达式的 AST 等价的字符串形式。如果有需要,标注可以在运行时使用 typing.get_type_hints() 进行解析。在不需要这种解析的通常情况下,标注的存储成本更低(因为解析器只需处理较短的字符串)且启动时间更短。

在可用性方面,标注现在支持向前引用,以使以下句法有效:

class C:
    @classmethod
    def from_string(cls, source: str) -> C:
        ...

    def validate_b(self, obj: B) -> bool:
        ...

class B:
    ...

由于此修改会破坏兼容性,在 Python 3.7 中此种新的行为需要在每个模块层级上使用 __future__ 导入来启用:

from __future__ import annotations

它将在 Python 4.0 中成为默认行为。

参见

PEP 563 -- 延迟的标注求值
PEP 由 Łukasz Langa 撰写并实现。

PEP 538: 传统 C 区域强制转换

Python 3 系列有一个持续的挑战就是为处理 7 比特位 ASCII 文本的假定编码确定合理的默认策略,目前的设定是在非 Windows 平台上使用默认的 C 或 POSIX 区域设置。

PEP 538 更新了默认的解释器命令行接口,以自动将上述区域强制转换为可用的基于 UTF-8 的区域,具体描述可参见有关新增环境变量 PYTHONCOERCECLOCALE 的文档。 以这种方式自动设置 LC_CTYPE 意味着核心解释器和能感知区域的 C 扩展 (例如 readline) 都将会假定 UTF-8 已被用作默认的文本编码,而不再是 ASCII。

PEP 11 中的平台支持定义也已被更新以限制完整文本处理支持适当配置的基于非 ASCII 的语言区域。

作为此更改的一部分,当使用任何已定义的强制转换目标区域时 (目前为 C.UTF-8, C.utf8UTF-8) stdinstdout 默认的处理器现在将为 surrogateescape (而不是 strict)。 而无论是什么区域,stderr 默认的处理器仍为 backslashreplace

默认情况下区域强制转换会静默进行,但为了辅助调试潜在的区域相关集成问题,可以通过设置 PYTHONCOERCECLOCALE=warn 来请求显式地启用警告信息(直接在 stderr 上发出)。 此设置还会使得 Python 运行时在核心解释器初始化时如果传统 C 区域仍然处于激活状态时发出警告。

虽然 PEP 538 的区域强制转换的好处在于它还会同时影响扩展模块 (例如 GNU readline) 以及子进程 (包括运行非 Python 应用和旧版本 Python 的子进程),但它也存在需要所运行系统必须存在适合的目标区域的缺点。 为了更好地处理没有可用适合的目标区域的情况 (例如在 RHEL/CentOS 7 上就会出现此情况),Python 3.7 还实现了 PEP 540: 强制 UTF-8 运行时模式

参见

PEP 538 -- 强制转换传统 C 区域到基于 UTF-8 的区域
PEP 由 Nick Coghlan 撰写并实现。

PEP 540: 强制 UTF-8 运行时模式

新的 -X utf8 命令行选项和 PYTHONUTF8 环境变量可被用来启用 CPython 的 UTF-8 模式

当处于 UTF-8 模式时,CPython 会忽略区域设置,并默认使用 UTF-8 编码。 用于 sys.stdinsys.stdout 流的错误处理器将设置为 surrogateescape

强制 UTF-8 模式可被用来在嵌入的 Python 解释器中改变文本处理行为,而不会改变嵌入方应用的区域设置。

PEP 540 的 UTF-8 模式的好处是不必关心运行所在系统中有哪些可用区域即可工作,但它也存在对扩展模块 (例如 GNU readline)、运行非 Python 应用的子进程以及运行旧版本 Python 的子进程不起作用的缺点。 为了减小与这些组件通信时破坏文本数据的风险,Python 3.7 还实现了 PEP 540: 强制 UTF-8 运行时模式)。

UTF-8 模式在语言区域为 CPOSIX 并且 PEP 538 区域强制转换特性无法将其修改为某种基于 UTF-8 的替代项时会被默认启用(无论修改失败是由于设置了 PYTHONCOERCECLOCALE=0, LC_ALL 还是由于缺少适合的目标区域)。

参见

PEP 540 -- 增加了新的 UTF-8 模式
PEP 由 Victor Stinner 撰写并实现

PEP 553: 内置的 breakpoint()

Python 3.7 包含了新的内置 breakpoint() 函数,作为一种简单方便地进入 Python 调试器的方式。

内置 breakpoint() 会调用 sys.breakpointhook()。 在默认情况下后者会导入 pdb 然后再调用 pdb.set_trace(),但是通过将 sys.breakpointhook() 绑定到你选定的函数,breakpoint() 可以进入任何调试器。 此外,环境变量 PYTHONBREAKPOINT 可被设置为你选定的调试器的可调用对象。 设置 PYTHONBREAKPOINT=0 会完全禁用内置 breakpoint()

参见

PEP 553 -- 内置的 breakpoint()
PEP 由 Barry Warsaw 撰写并实现

PEP 539: 用于线程局部存储的新 C API

虽然 Python 已提供了用于线程局部存储支持的 C API;但原有的 线程局部存储 (TLS) API 使用 int 来表示所有平台上的 TLS 密钥。 对于官方支持的平台而言这通常不是问题,但这既不符合 POSIX 标准,也不具备任何实际意义上的可移植性。

PEP 539 通过向 CPython 提供了一个新的 线程特定存储 (TSS) API 来改变这一点,它取代了原有的 CPython 内部 TLS API 的使用,并且原有 API 已弃用。 TSS API 使用一种新类型 Py_tss_t 而非 int 来表示 TSS 密钥 -- 这是一种不透明类型,其定义可能依赖于下层的 TLS 实现。 因此,这将允许在以无法安全地转换为 int 的方式定义原生 TLS 密钥的平台上构建 CPython。

请注意在原生 TLS 密钥定义方式无法被安全地转换为 int 的平台上,原有 TLS API 中的全部函数将无法执行并会立即返回失败信息。 这样能清楚地表明原有 API 在无法可靠使用的平台上不受支持,并且不会再尝试添加此类支持。

参见

PEP 539 -- 在 CPython 中用于线程局部存储的新 C-API
PEP 由 Erik M. Bray 撰写;由 Masayuki Yamamoto 实现。

PEP 562: 定制对模块属性的访问

Python 3.7 允许在模块上定义 __getattr__() 并且当以其他方式找不到某个模块属性时将会调用它。 在模块上定义 __dir__() 现在也是允许的。

一个典型的可能有用的例子是已弃用模块属性和惰性加载。

参见

PEP 562 -- 模块的 __getattr____dir__
PEP 由 Ivan Levkivskyi 撰写并实现

PEP 564: 具有纳秒级精度的新时间函数

现代系统的时钟精度可以超过由 time.time() 函数及其变化形式所返回的浮点数的有限精度。 为了避免精度损失,PEP 564time 模块中增加了原有计时器函数的六个新“纳秒版”变化形式:

这些新函数会以整数值的形式返回纳秒数。

测量 表明在 Linux 和 Windows 上 time.time_ns() 的精度大约比 time.time() 要高 3 倍。

参见

PEP 564 -- 增加具有纳秒级精度的新时间函数
PEP 由 Victor Stinner 撰写并实现

PEP 565: 在 __main__ 中显示 DeprecationWarning

DeprecationWarning 的默认处理方式已经被更改,这此警告默认只显示一次,仅有当直接在 __main__ 模块中运行的代码触发它们时才会再次显示。 因此,单文件脚本开发者以及 Python 交互模式使用者应该会再次开始看到针对他们所使用 API 的已弃用警告,但被导入应用、库和框架模块所触发的已弃用警告默认将继续隐藏。

作为此项更改的结果,标准库现在允许开发者在三种不同的已弃用警告行为之间进行选择:

  • FutureWarning: 默认情况下总是会显示,建议用于应用程序最终用户应该看到的警告信息(例如对于已弃用的应用程序配置的设置选项)。
  • DeprecationWarning: 默认情况下仅在 __main__ 中以及当运行测试时会显示,建议用于其他 Python 开发者应该看到的警告信息,提示版本升级可能导致行为改变或者错误。
  • PendingDeprecationWarning: 默认情况下仅在运行测试时会显示,可用于提示未来版本升级将会改变警告类别为 DeprecationWarningFutureWarning 的情况。

在此之前 DeprecationWarningPendingDeprecationWarning 都仅在运行测试时可见,这意味着主要编写单文件脚本或使用 Python 交互模式的开发者可能会因他们所用 API 突然出现的改变而感到惊讶。

参见

PEP 565 -- 在 __main__ 中显示 DeprecationWarning
PEP 由 Nick Coghlan 撰写并实现

PEP 560: 对 typing 模块和泛型类型的核心支持

PEP 484 最初的设计方式使其不会向核心 CPython 解释器引入 任何 更改。 现在类型提示和 typing 模块已被社区广泛使用,因此这个限制已被取消。 这个 PEP 引入了两个特殊方法 __class_getitem__()__mro_entries__,这些方法现在被 typing 中的大多数类和特殊构造所使用。 结果就是与类型相关的各类操作的速度提升了 7 倍,泛型类型可以在没有元类冲突的情况下被使用,而 typing 模块中几个长期存在的错误也已被修正。

参见

PEP 560 -- 对 typing 模块和泛型类型的核心支持
PEP 由 Ivan Levkivskyi 撰写并实现

PEP 552: 基于哈希值的 .pyc 文件

传统上 Python 检查字节码缓存文件 (即 .pyc 文件) 是否最新的方式是通过对源码元数据 (最后更改的时间戳和大小)和生成缓存时保存在其文件头中的源码元数据进行比较。 这种检查方法虽然有效,但也存在缺点。 当文件系统的时间戳太粗糙时,Python 有可能错过源码更新,导致用户感到困惑。 此外,在缓存文件中存在时间戳对于 构建可再现 并且基于内容的构建系统来说是有问题的。

PEP 552 扩展了 pyc 格式以允许使用源文件的哈希值而非源文件的时间戳来检查有效性。 这种 .pyc 文件就称为“基于哈希值的”。 默认情况下,Python 仍然使用基于时间戳的有效性检查,不会在运行时生成基于哈希值的 .pyc 文件。 基于哈希值的 .pyc 文件可以使用 py_compilecompileall 来生成。

基于哈希值的 .pyc 文件包含两种变体:已选定和未选定。 Python 会在运行时针对相应源码文件验证已选定基于哈希值的 .pyc 文件,但对未选定基于哈希值的 pyc 文件则不会这样做。 未选定基于哈希值的 .pyc 文件对于需要由 Python 外部的系统(例如构建系统)负责使 .pyc 文件保持最新的环境来说是一种有用的性能优化。

请参阅 已缓存字节码的失效 了解更多信息。

参见

PEP 552 -- 确定性的 pyc 文件
PEP 由 Benjamin Peterson 撰写并实现

PEP 545: Python 文档翻译

PEP 545 描述了创建和维护 Python 文档翻译的整个过程。

新增了三个新的翻译版本:

参见

PEP 545 -- Python 文档翻译
PEP 由 Julien Palard, Inada Naoki 和 Victor Stinner 撰写并实现。

开发运行时模式: -X dev

新的 -X dev 命令行选项或新的 PYTHONDEVMODE 环境变量可被用来启用 CPython 的 开发模式。 在开发模式下,CPython 会执行额外的在默认情况下开销过大的运行时检查。 请参阅 -X dev 文档查看对于此模式效果的完整描述。

其他语言特性修改

  • 现在可以将超过 255 个参数传递给一个函数,而现在一个函数也可以拥有超过 255 个形参。 (由 Serhiy Storchaka 在 bpo-12844bpo-18896 中贡献。)
  • 现在 bytes.fromhex()bytearray.fromhex() 会忽略所有 ASCII 空白符,而非仅是空格符. (由 Robert Xiao 在 bpo-28927 中贡献。)
  • str, bytesbytearray 获得了对新 isascii() 方法的支持,该方法可被用来测试是个字符串或字节串是否仅包含 ASCII 字符。 (由 INADA Naoki 在 bpo-32677 中贡献。)
  • 现在当 from ... import ... 失败时 ImportError 会显示模块名称和模块 __file__ 路径。 (由 Matthias Bussonnier 在 bpo-29546 中贡献。)
  • 现在已支持涉及将子模块绑定到一个名称的绝对导入的循环导入。 (由 Serhiy Storchaka 在 bpo-30024 中贡献。)
  • 现在 object.__format__(x, '') 等价于 str(x) 而非 format(str(self), '')。 (由 Serhiy Storchaka d bpo-28974 中贡献。)
  • 为更好地支持栈跟踪的动态创建,现在 types.TracebackType 可以从 Python 代码中被实例化,并且 回溯对象tb_next 属性现在是可写的。 (由 Nathaniel J. Smith 在 bpo-30579 中贡献。)
  • 当使用 -m 开关时,现在 sys.path[0] 会主动扩展为完整的起始目录路径,而不是保持为空目录(这将允许在发生导入时从 当前 工作目录导入) (由 Nick Coghlan 在 bpo-33053 中贡献。)
  • 新的 -X importtime 选项或 PYTHONPROFILEIMPORTTIME 环境变量可被用来显示每次模块导入的时间。 (由 Victor Stinner 在 bpo-31415 中贡献。)

新增模块

contextvars

新的 contextvars 模块和一组 新的 C API 引入了对 上下文变量 的支持。 上下文变量在概念上类似于线程局部变量。 与 TLS 不同,上下文变量能正确地支持异步代码。

asynciodecimal 已得到更新以使用和支持开箱即用的上下文变量。 特别是激活的 decimal 上下文现在将存储在上下文变量中,它允许十进制运算在异步代码中使用正确的上下文。

参见

PEP 567 -- 上下文变量
PEP 由 Yury Selivanov 撰写并实现

dataclasses

新的 dataclass() 装饰器提供了一种声明 数据类 的方式。 数据类使用变量标注来描述其属性。 它的构造器和其他魔术方法例如 __repr__(), __eq__() 以及 __hash__() 会自动地生成。

示例:

@dataclass
class Point:
    x: float
    y: float
    z: float = 0.0

p = Point(1.5, 2.5)
print(p)   # produces "Point(x=1.5, y=2.5, z=0.0)"

参见

PEP 557 -- 数据类
PEP 由 Eric V. Smith 撰写并实现

importlib.resources

新的 importlib.resources 模块提供了一些新的 API 和一个新的 ABC 用于访问、打开和读取包内的 资源。 资源基本上类似于包内的文件,但它们不一定是物理文件系统中实际的文件。 模块加载器可以提供 get_resource_reader() 函数,它会返回一个 importlib.abc.ResourceReader 实例来支持这个新 API。 内置的文件路径加载器和 zip 文件加载器都支持此特性。

由 Barry Warsaw 和 Brett Cannon 在 bpo-32248 中贡献。

参见

importlib_resources -- 用于先前 Python 版本的 PyPI 下层接口。

改进的模块

argparse

新的 ArgumentParser.parse_intermixed_args() 方法允许混合选项与位置参数。 (由 paul.j3 在 bpo-14191 中提供。)

asyncio

asyncio 模块获得了许多新的特性、可用性和 性能提升。 重要的改变包括:

Several asyncio APIs have been deprecated.

binascii

The b2a_uu() function now accepts an optional backtick keyword argument. When it's true, zeros are represented by '`' instead of spaces. (Contributed by Xiang Zhang in bpo-30103.)

calendar

The HTMLCalendar class has new class attributes which ease the customization of CSS classes in the produced HTML calendar. (Contributed by Oz Tiram in bpo-30095.)

collections

collections.namedtuple() now supports default values. (Contributed by Raymond Hettinger in bpo-32320.)

compileall

compileall.compile_dir() learned the new invalidation_mode parameter, which can be used to enable hash-based .pyc invalidation. The invalidation mode can also be specified on the command line using the new --invalidation-mode argument. (Contributed by Benjamin Peterson in bpo-31650.)

concurrent.futures

ProcessPoolExecutor and ThreadPoolExecutor now support the new initializer and initargs constructor arguments. (Contributed by Antoine Pitrou in bpo-21423.)

The ProcessPoolExecutor can now take the multiprocessing context via the new mp_context argument. (Contributed by Thomas Moreau in bpo-31540.)

contextlib

The new nullcontext() is a simpler and faster no-op context manager than ExitStack. (Contributed by Jesse-Bakker in bpo-10049.)

The new asynccontextmanager(), AbstractAsyncContextManager, and AsyncExitStack have been added to complement their synchronous counterparts. (Contributed by Jelle Zijlstra in bpo-29679 and bpo-30241, and by Alexander Mohr and Ilya Kulakov in bpo-29302.)

cProfile

The cProfile command line now accepts -m module_name as an alternative to script path. (Contributed by Sanyam Khurana in bpo-21862.)

crypt

The crypt module now supports the Blowfish hashing method. (Contributed by Serhiy Storchaka in bpo-31664.)

The mksalt() function now allows specifying the number of rounds for hashing. (Contributed by Serhiy Storchaka in bpo-31702.)

datetime

The new datetime.fromisoformat() method constructs a datetime object from a string in one of the formats output by datetime.isoformat(). (Contributed by Paul Ganssle in bpo-15873.)

The tzinfo class now supports sub-minute offsets. (Contributed by Alexander Belopolsky in bpo-5288.)

dbm

dbm.dumb now supports reading read-only files and no longer writes the index file when it is not changed.

decimal

The decimal module now uses context variables to store the decimal context. (Contributed by Yury Selivanov in bpo-32630.)

dis

The dis() function is now able to disassemble nested code objects (the code of comprehensions, generator expressions and nested functions, and the code used for building nested classes). The maximum depth of disassembly recursion is controlled by the new depth parameter. (Contributed by Serhiy Storchaka in bpo-11822.)

distutils

README.rst is now included in the list of distutils standard READMEs and therefore included in source distributions. (Contributed by Ryan Gonzalez in bpo-11913.)

enum

The Enum learned the new _ignore_ class property, which allows listing the names of properties which should not become enum members. (Contributed by Ethan Furman in bpo-31801.)

In Python 3.8, attempting to check for non-Enum objects in Enum classes will raise a TypeError (e.g. 1 in Color); similarly, attempting to check for non-Flag objects in a Flag member will raise TypeError (e.g. 1 in Perm.RW); currently, both operations return False instead and are deprecated. (Contributed by Ethan Furman in bpo-33217.)

functools

functools.singledispatch() now supports registering implementations using type annotations. (Contributed by Łukasz Langa in bpo-32227.)

gc

The new gc.freeze() function allows freezing all objects tracked by the garbage collector and excluding them from future collections. This can be used before a POSIX fork() call to make the GC copy-on-write friendly or to speed up collection. The new gc.unfreeze() functions reverses this operation. Additionally, gc.get_freeze_count() can be used to obtain the number of frozen objects. (Contributed by Li Zekun in bpo-31558.)

hmac

The hmac module now has an optimized one-shot digest() function, which is up to three times faster than HMAC(). (Contributed by Christian Heimes in bpo-32433.)

http.client

HTTPConnection and HTTPSConnection now support the new blocksize argument for improved upload throughput. (Contributed by Nir Soffer in bpo-31945.)

http.server

SimpleHTTPRequestHandler now supports the HTTP If-Modified-Since header. The server returns the 304 response status if the target file was not modified after the time specified in the header. (Contributed by Pierre Quentel in bpo-29654.)

SimpleHTTPRequestHandler accepts the new directory argument, in addition to the new --directory command line argument. With this parameter, the server serves the specified directory, by default it uses the current working directory. (Contributed by Stéphane Wirtel and Julien Palard in bpo-28707.)

The new ThreadingHTTPServer class uses threads to handle requests using ThreadingMixin. It is used when http.server is run with -m. (Contributed by Julien Palard in bpo-31639.)

idlelib and IDLE

Multiple fixes for autocompletion. (Contributed by Louie Lu in bpo-15786.)

Module Browser (on the File menu, formerly called Class Browser), now displays nested functions and classes in addition to top-level functions and classes. (Contributed by Guilherme Polo, Cheryl Sabella, and Terry Jan Reedy in bpo-1612262.)

The Settings dialog (Options, Configure IDLE) has been partly rewritten to improve both appearance and function. (Contributed by Cheryl Sabella and Terry Jan Reedy in multiple issues.)

The font sample now includes a selection of non-Latin characters so that users can better see the effect of selecting a particular font. (Contributed by Terry Jan Reedy in bpo-13802.) The sample can be edited to include other characters. (Contributed by Serhiy Storchaka in bpo-31860.)

The IDLE features formerly implemented as extensions have been reimplemented as normal features. Their settings have been moved from the Extensions tab to other dialog tabs. (Contributed by Charles Wohlganger and Terry Jan Reedy in bpo-27099.)

Editor code context option revised. Box displays all context lines up to maxlines. Clicking on a context line jumps the editor to that line. Context colors for custom themes is added to Highlights tab of Settings dialog. (Contributed by Cheryl Sabella and Terry Jan Reedy in bpo-33642, bpo-33768, and bpo-33679.)

On Windows, a new API call tells Windows that tk scales for DPI. On Windows 8.1+ or 10, with DPI compatibility properties of the Python binary unchanged, and a monitor resolution greater than 96 DPI, this should make text and lines sharper. It should otherwise have no effect. (Contributed by Terry Jan Reedy in bpo-33656.)

New in 3.7.1:

Output over N lines (50 by default) is squeezed down to a button. N can be changed in the PyShell section of the General page of the Settings dialog. Fewer, but possibly extra long, lines can be squeezed by right clicking on the output. Squeezed output can be expanded in place by double-clicking the button or into the clipboard or a separate window by right-clicking the button. (Contributed by Tal Einat in bpo-1529353.)

The changes above have been backported to 3.6 maintenance releases.

importlib

The importlib.abc.ResourceReader ABC was introduced to support the loading of resources from packages. See also importlib.resources. (Contributed by Barry Warsaw, Brett Cannon in bpo-32248.)

importlib.reload() now raises ModuleNotFoundError if the module lacks a spec. (Contributed by Garvit Khatri in bpo-29851.)

importlib.find_spec() now raises ModuleNotFoundError instead of AttributeError if the specified parent module is not a package (i.e. lacks a __path__ attribute). (Contributed by Milan Oberkirch in bpo-30436.)

The new importlib.source_hash() can be used to compute the hash of the passed source. A hash-based .pyc file embeds the value returned by this function.

io

The new TextIOWrapper.reconfigure() method can be used to reconfigure the text stream with the new settings. (Contributed by Antoine Pitrou in bpo-30526 and INADA Naoki in bpo-15216.)

ipaddress

The new subnet_of() and supernet_of() methods of ipaddress.IPv6Network and ipaddress.IPv4Network can be used for network containment tests. (Contributed by Michel Albert and Cheryl Sabella in bpo-20825.)

itertools

itertools.islice() now accepts integer-like objects as start, stop, and slice arguments. (Contributed by Will Roberts in bpo-30537.)

locale

The new monetary argument to locale.format_string() can be used to make the conversion use monetary thousands separators and grouping strings. (Contributed by Garvit in bpo-10379.)

The locale.getpreferredencoding() function now always returns 'UTF-8' on Android or when in the forced UTF-8 mode.

logging

Logger instances can now be pickled. (Contributed by Vinay Sajip in bpo-30520.)

The new StreamHandler.setStream() method can be used to replace the logger stream after handler creation. (Contributed by Vinay Sajip in bpo-30522.)

It is now possible to specify keyword arguments to handler constructors in configuration passed to logging.config.fileConfig(). (Contributed by Preston Landers in bpo-31080.)

math

The new math.remainder() function implements the IEEE 754-style remainder operation. (Contributed by Mark Dickinson in bpo-29962.)

mimetypes

The MIME type of .bmp has been changed from 'image/x-ms-bmp' to 'image/bmp'. (Contributed by Nitish Chandra in bpo-22589.)

msilib

The new Database.Close() method can be used to close the MSI database. (Contributed by Berker Peksag in bpo-20486.)

multiprocessing

The new Process.close() method explicitly closes the process object and releases all resources associated with it. ValueError is raised if the underlying process is still running. (Contributed by Antoine Pitrou in bpo-30596.)

The new Process.kill() method can be used to terminate the process using the SIGKILL signal on Unix. (Contributed by Vitor Pereira in bpo-30794.)

Non-daemonic threads created by Process are now joined on process exit. (Contributed by Antoine Pitrou in bpo-18966.)

os

os.fwalk() now accepts the path argument as bytes. (Contributed by Serhiy Storchaka in bpo-28682.)

os.scandir() gained support for file descriptors. (Contributed by Serhiy Storchaka in bpo-25996.)

The new register_at_fork() function allows registering Python callbacks to be executed at process fork. (Contributed by Antoine Pitrou in bpo-16500.)

Added os.preadv() (combine the functionality of os.readv() and os.pread()) and os.pwritev() functions (combine the functionality of os.writev() and os.pwrite()). (Contributed by Pablo Galindo in bpo-31368.)

The mode argument of os.makedirs() no longer affects the file permission bits of newly-created intermediate-level directories. (Contributed by Serhiy Storchaka in bpo-19930.)

os.dup2() now returns the new file descriptor. Previously, None was always returned. (Contributed by Benjamin Peterson in bpo-32441.)

The structure returned by os.stat() now contains the st_fstype attribute on Solaris and its derivatives. (Contributed by Jesús Cea Avión in bpo-32659.)

pathlib

The new Path.is_mount() method is now available on POSIX systems and can be used to determine whether a path is a mount point. (Contributed by Cooper Ry Lees in bpo-30897.)

pdb

pdb.set_trace() now takes an optional header keyword-only argument. If given, it is printed to the console just before debugging begins. (Contributed by Barry Warsaw in bpo-31389.)

pdb command line now accepts -m module_name as an alternative to script file. (Contributed by Mario Corchero in bpo-32206.)

py_compile

py_compile.compile() -- and by extension, compileall -- now respects the SOURCE_DATE_EPOCH environment variable by unconditionally creating .pyc files for hash-based validation. This allows for guaranteeing reproducible builds of .pyc files when they are created eagerly. (Contributed by Bernhard M. Wiedemann in bpo-29708.)

pydoc

The pydoc server can now bind to an arbitrary hostname specified by the new -n command-line argument. (Contributed by Feanil Patel in bpo-31128.)

queue

The new SimpleQueue class is an unbounded FIFO queue. (Contributed by Antoine Pitrou in bpo-14976.)

re

The flags re.ASCII, re.LOCALE and re.UNICODE can be set within the scope of a group. (Contributed by Serhiy Storchaka in bpo-31690.)

re.split() now supports splitting on a pattern like r'\b', '^$' or (?=-) that matches an empty string. (Contributed by Serhiy Storchaka in bpo-25054.)

Regular expressions compiled with the re.LOCALE flag no longer depend on the locale at compile time. Locale settings are applied only when the compiled regular expression is used. (Contributed by Serhiy Storchaka in bpo-30215.)

FutureWarning is now emitted if a regular expression contains character set constructs that will change semantically in the future, such as nested sets and set operations. (Contributed by Serhiy Storchaka in bpo-30349.)

Compiled regular expression and match objects can now be copied using copy.copy() and copy.deepcopy(). (Contributed by Serhiy Storchaka in bpo-10076.)

signal

The new warn_on_full_buffer argument to the signal.set_wakeup_fd() function makes it possible to specify whether Python prints a warning on stderr when the wakeup buffer overflows. (Contributed by Nathaniel J. Smith in bpo-30050.)

socket

The new socket.getblocking() method returns True if the socket is in blocking mode and False otherwise. (Contributed by Yury Selivanov in bpo-32373.)

The new socket.close() function closes the passed socket file descriptor. This function should be used instead of os.close() for better compatibility across platforms. (Contributed by Christian Heimes in bpo-32454.)

The socket module now exposes the socket.TCP_CONGESTION (Linux 2.6.13), socket.TCP_USER_TIMEOUT (Linux 2.6.37), and socket.TCP_NOTSENT_LOWAT (Linux 3.12) constants. (Contributed by Omar Sandoval in bpo-26273 and Nathaniel J. Smith in bpo-29728.)

Support for socket.AF_VSOCK sockets has been added to allow communication between virtual machines and their hosts. (Contributed by Cathy Avery in bpo-27584.)

Sockets now auto-detect family, type and protocol from file descriptor by default. (Contributed by Christian Heimes in bpo-28134.)

socketserver

socketserver.ThreadingMixIn.server_close() now waits until all non-daemon threads complete. socketserver.ForkingMixIn.server_close() now waits until all child processes complete.

Add a new socketserver.ForkingMixIn.block_on_close class attribute to socketserver.ForkingMixIn and socketserver.ThreadingMixIn classes. Set the class attribute to False to get the pre-3.7 behaviour.

sqlite3

sqlite3.Connection now exposes the backup() method when the underlying SQLite library is at version 3.6.11 or higher. (Contributed by Lele Gaifax in bpo-27645.)

The database argument of sqlite3.connect() now accepts any path-like object, instead of just a string. (Contributed by Anders Lorentsen in bpo-31843.)

ssl

The ssl module now uses OpenSSL's builtin API instead of match_hostname() to check a host name or an IP address. Values are validated during TLS handshake. Any certificate validation error including failing the host name check now raises SSLCertVerificationError and aborts the handshake with a proper TLS Alert message. The new exception contains additional information. Host name validation can be customized with SSLContext.host_flags. (Contributed by Christian Heimes in bpo-31399.)

注解

The improved host name check requires a libssl implementation compatible with OpenSSL 1.0.2 or 1.1. Consequently, OpenSSL 0.9.8 and 1.0.1 are no longer supported (see Platform Support Removals for more details). The ssl module is mostly compatible with LibreSSL 2.7.2 and newer.

The ssl module no longer sends IP addresses in SNI TLS extension. (Contributed by Christian Heimes in bpo-32185.)

match_hostname() no longer supports partial wildcards like www*.example.org. SSLContext.host_flags has partial wildcard matching disabled by default. (Contributed by Mandeep Singh in bpo-23033 and Christian Heimes in bpo-31399.)

The default cipher suite selection of the ssl module now uses a blacklist approach rather than a hard-coded whitelist. Python no longer re-enables ciphers that have been blocked by OpenSSL security updates. Default cipher suite selection can be configured at compile time. (Contributed by Christian Heimes in bpo-31429.)

Validation of server certificates containing internationalized domain names (IDNs) is now supported. As part of this change, the SSLSocket.server_hostname attribute now stores the expected hostname in A-label form ("xn--pythn-mua.org"), rather than the U-label form ("pythön.org"). (Contributed by Nathaniel J. Smith and Christian Heimes in bpo-28414.)

The ssl module has preliminary and experimental support for TLS 1.3 and OpenSSL 1.1.1. At the time of Python 3.7.0 release, OpenSSL 1.1.1 is still under development and TLS 1.3 hasn't been finalized yet. The TLS 1.3 handshake and protocol behaves slightly differently than TLS 1.2 and earlier, see TLS 1.3. (Contributed by Christian Heimes in bpo-32947, bpo-20995, bpo-29136, bpo-30622 and bpo-33618)

SSLSocket and SSLObject no longer have a public constructor. Direct instantiation was never a documented and supported feature. Instances must be created with SSLContext methods wrap_socket() and wrap_bio(). (Contributed by Christian Heimes in bpo-32951)

OpenSSL 1.1 APIs for setting the minimum and maximum TLS protocol version are available as SSLContext.minimum_version and SSLContext.maximum_version. Supported protocols are indicated by several new flags, such as HAS_TLSv1_1. (Contributed by Christian Heimes in bpo-32609.)

字符串

string.Template now lets you to optionally modify the regular expression pattern for braced placeholders and non-braced placeholders separately. (Contributed by Barry Warsaw in bpo-1198569.)

subprocess

The subprocess.run() function accepts the new capture_output keyword argument. When true, stdout and stderr will be captured. This is equivalent to passing subprocess.PIPE as stdout and stderr arguments. (Contributed by Bo Bayles in bpo-32102.)

The subprocess.run function and the subprocess.Popen constructor now accept the text keyword argument as an alias to universal_newlines. (Contributed by Andrew Clegg in bpo-31756.)

On Windows the default for close_fds was changed from False to True when redirecting the standard handles. It's now possible to set close_fds to true when redirecting the standard handles. See subprocess.Popen. This means that close_fds now defaults to True on all supported platforms. (Contributed by Segev Finer in bpo-19764.)

The subprocess module is now more graceful when handling KeyboardInterrupt during subprocess.call(), subprocess.run(), or in a Popen context manager. It now waits a short amount of time for the child to exit, before continuing the handling of the KeyboardInterrupt exception. (Contributed by Gregory P. Smith in bpo-25942.)

sys

The new sys.breakpointhook() hook function is called by the built-in breakpoint(). (Contributed by Barry Warsaw in bpo-31353.)

On Android, the new sys.getandroidapilevel() returns the build-time Android API version. (Contributed by Victor Stinner in bpo-28740.)

The new sys.get_coroutine_origin_tracking_depth() function returns the current coroutine origin tracking depth, as set by the new sys.set_coroutine_origin_tracking_depth(). asyncio has been converted to use this new API instead of the deprecated sys.set_coroutine_wrapper(). (Contributed by Nathaniel J. Smith in bpo-32591.)

time

PEP 564 adds six new functions with nanosecond resolution to the time module:

New clock identifiers have been added:

  • time.CLOCK_BOOTTIME (Linux): Identical to time.CLOCK_MONOTONIC, except it also includes any time that the system is suspended.
  • time.CLOCK_PROF (FreeBSD, NetBSD and OpenBSD): High-resolution per-process CPU timer.
  • time.CLOCK_UPTIME (FreeBSD, OpenBSD): Time whose absolute value is the time the system has been running and not suspended, providing accurate uptime measurement.

The new time.thread_time() and time.thread_time_ns() functions can be used to get per-thread CPU time measurements. (Contributed by Antoine Pitrou in bpo-32025.)

The new time.pthread_getcpuclockid() function returns the clock ID of the thread-specific CPU-time clock.

tkinter

The new tkinter.ttk.Spinbox class is now available. (Contributed by Alan Moore in bpo-32585.)

tracemalloc

tracemalloc.Traceback behaves more like regular tracebacks, sorting the frames from oldest to most recent. Traceback.format() now accepts negative limit, truncating the result to the abs(limit) oldest frames. To get the old behaviour, use the new most_recent_first argument to Traceback.format(). (Contributed by Jesse Bakker in bpo-32121.)

types

The new WrapperDescriptorType, MethodWrapperType, MethodDescriptorType, and ClassMethodDescriptorType classes are now available. (Contributed by Manuel Krebber and Guido van Rossum in bpo-29377, and Serhiy Storchaka in bpo-32265.)

The new types.resolve_bases() function resolves MRO entries dynamically as specified by PEP 560. (Contributed by Ivan Levkivskyi in bpo-32717.)

unicodedata

The internal unicodedata database has been upgraded to use Unicode 11. (Contributed by Benjamin Peterson.)

unittest

The new -k command-line option allows filtering tests by a name substring or a Unix shell-like pattern. For example, python -m unittest -k foo runs foo_tests.SomeTest.test_something, bar_tests.SomeTest.test_foo, but not bar_tests.FooTest.test_something. (Contributed by Jonas Haag in bpo-32071.)

unittest.mock

The sentinel attributes now preserve their identity when they are copied or pickled. (Contributed by Serhiy Storchaka in bpo-20804.)

The new seal() function allows sealing Mock instances, which will disallow further creation of attribute mocks. The seal is applied recursively to all attributes that are themselves mocks. (Contributed by Mario Corchero in bpo-30541.)

urllib.parse

urllib.parse.quote() has been updated from RFC 2396 to RFC 3986, adding ~ to the set of characters that are never quoted by default. (Contributed by Christian Theune and Ratnadeep Debnath in bpo-16285.)

uu

The uu.encode() function now accepts an optional backtick keyword argument. When it's true, zeros are represented by '`' instead of spaces. (Contributed by Xiang Zhang in bpo-30103.)

uuid

The new UUID.is_safe attribute relays information from the platform about whether generated UUIDs are generated with a multiprocessing-safe method. (Contributed by Barry Warsaw in bpo-22807.)

uuid.getnode() now prefers universally administered MAC addresses over locally administered MAC addresses. This makes a better guarantee for global uniqueness of UUIDs returned from uuid.uuid1(). If only locally administered MAC addresses are available, the first such one found is returned. (Contributed by Barry Warsaw in bpo-32107.)

warnings

The initialization of the default warnings filters has changed as follows:

  • warnings enabled via command line options (including those for -b and the new CPython-specific -X dev option) are always passed to the warnings machinery via the sys.warnoptions attribute.

  • warnings filters enabled via the command line or the environment now have the following order of precedence:

    • the BytesWarning filter for -b (or -bb)
    • any filters specified with the -W option
    • any filters specified with the PYTHONWARNINGS environment variable
    • any other CPython specific filters (e.g. the default filter added for the new -X dev mode)
    • any implicit filters defined directly by the warnings machinery
  • in CPython debug builds, all warnings are now displayed by default (the implicit filter list is empty)

(Contributed by Nick Coghlan and Victor Stinner in bpo-20361, bpo-32043, and bpo-32230.)

Deprecation warnings are once again shown by default in single-file scripts and at the interactive prompt. See PEP 565: 在 __main__ 中显示 DeprecationWarning for details. (Contributed by Nick Coghlan in bpo-31975.)

xml.etree

ElementPath predicates in the find() methods can now compare text of the current node with [. = "text"], not only text in children. Predicates also allow adding spaces for better readability. (Contributed by Stefan Behnel in bpo-31648.)

xmlrpc.server

SimpleXMLRPCDispatcher.register_function can now be used as a decorator. (Contributed by Xiang Zhang in bpo-7769.)

zipapp

Function create_archive() now accepts an optional filter argument to allow the user to select which files should be included in the archive. (Contributed by Irmen de Jong in bpo-31072.)

Function create_archive() now accepts an optional compressed argument to generate a compressed archive. A command line option --compress has also been added to support compression. (Contributed by Zhiming Wang in bpo-31638.)

zipfile

ZipFile now accepts the new compresslevel parameter to control the compression level. (Contributed by Bo Bayles in bpo-21417.)

Subdirectories in archives created by ZipFile are now stored in alphabetical order. (Contributed by Bernhard M. Wiedemann in bpo-30693.)

C API Changes

A new API for thread-local storage has been implemented. See PEP 539: 用于线程局部存储的新 C API for an overview and Thread Specific Storage (TSS) API for a complete reference. (Contributed by Masayuki Yamamoto in bpo-25658.)

The new context variables functionality exposes a number of new C APIs.

The new PyImport_GetModule() function returns the previously imported module with the given name. (Contributed by Eric Snow in bpo-28411.)

The new Py_RETURN_RICHCOMPARE macro eases writing rich comparison functions. (Contributed by Petr Victorin in bpo-23699.)

The new Py_UNREACHABLE macro can be used to mark unreachable code paths. (Contributed by Barry Warsaw in bpo-31338.)

The tracemalloc now exposes a C API through the new PyTraceMalloc_Track() and PyTraceMalloc_Untrack() functions. (Contributed by Victor Stinner in bpo-30054.)

The new import__find__load__start() and import__find__load__done() static markers can be used to trace module imports. (Contributed by Christian Heimes in bpo-31574.)

The fields name and doc of structures PyMemberDef, PyGetSetDef, PyStructSequence_Field, PyStructSequence_Desc, and wrapperbase are now of type const char * rather of char *. (Contributed by Serhiy Storchaka in bpo-28761.)

The result of PyUnicode_AsUTF8AndSize() and PyUnicode_AsUTF8() is now of type const char * rather of char *. (Contributed by Serhiy Storchaka in bpo-28769.)

The result of PyMapping_Keys(), PyMapping_Values() and PyMapping_Items() is now always a list, rather than a list or a tuple. (Contributed by Oren Milman in bpo-28280.)

Added functions PySlice_Unpack() and PySlice_AdjustIndices(). (Contributed by Serhiy Storchaka in bpo-27867.)

PyOS_AfterFork() is deprecated in favour of the new functions PyOS_BeforeFork(), PyOS_AfterFork_Parent() and PyOS_AfterFork_Child(). (Contributed by Antoine Pitrou in bpo-16500.)

The PyExc_RecursionErrorInst singleton that was part of the public API has been removed as its members being never cleared may cause a segfault during finalization of the interpreter. Contributed by Xavier de Gaye in bpo-22898 and bpo-30697.

Added C API support for timezones with timezone constructors PyTimeZone_FromOffset() and PyTimeZone_FromOffsetAndName(), and access to the UTC singleton with PyDateTime_TimeZone_UTC. Contributed by Paul Ganssle in bpo-10381.

The type of results of PyThread_start_new_thread() and PyThread_get_thread_ident(), and the id parameter of PyThreadState_SetAsyncExc() changed from long to unsigned long. (Contributed by Serhiy Storchaka in bpo-6532.)

PyUnicode_AsWideCharString() now raises a ValueError if the second argument is NULL and the wchar_t* string contains null characters. (Contributed by Serhiy Storchaka in bpo-30708.)

Changes to the startup sequence and the management of dynamic memory allocators mean that the long documented requirement to call Py_Initialize() before calling most C API functions is now relied on more heavily, and failing to abide by it may lead to segfaults in embedding applications. See the Porting to Python 3.7 section in this document and the Before Python Initialization section in the C API documentation for more details.

The new PyInterpreterState_GetID() returns the unique ID for a given interpreter. (Contributed by Eric Snow in bpo-29102.)

Py_DecodeLocale(), Py_EncodeLocale() now use the UTF-8 encoding when the UTF-8 mode is enabled. (Contributed by Victor Stinner in bpo-29240.)

PyUnicode_DecodeLocaleAndSize() and PyUnicode_EncodeLocale() now use the current locale encoding for surrogateescape error handler. (Contributed by Victor Stinner in bpo-29240.)

The start and end parameters of PyUnicode_FindChar() are now adjusted to behave like string slices. (Contributed by Xiang Zhang in bpo-28822.)

Build Changes

Support for building --without-threads has been removed. The threading module is now always available. (Contributed by Antoine Pitrou in bpo-31370.).

A full copy of libffi is no longer bundled for use when building the _ctypes module on non-OSX UNIX platforms. An installed copy of libffi is now required when building _ctypes on such platforms. (Contributed by Zachary Ware in bpo-27979.)

The Windows build process no longer depends on Subversion to pull in external sources, a Python script is used to download zipfiles from GitHub instead. If Python 3.6 is not found on the system (via py -3.6), NuGet is used to download a copy of 32-bit Python for this purpose. (Contributed by Zachary Ware in bpo-30450.)

The ssl module requires OpenSSL 1.0.2 or 1.1 compatible libssl. OpenSSL 1.0.1 has reached end of lifetime on 2016-12-31 and is no longer supported. LibreSSL is temporarily not supported as well. LibreSSL releases up to version 2.6.4 are missing required OpenSSL 1.0.2 APIs.

Optimizations

The overhead of calling many methods of various standard library classes implemented in C has been significantly reduced by porting more code to use the METH_FASTCALL convention. (Contributed by Victor Stinner in bpo-29300, bpo-29507, bpo-29452, and bpo-29286.)

Various optimizations have reduced Python startup time by 10% on Linux and up to 30% on macOS. (Contributed by Victor Stinner, INADA Naoki in bpo-29585, and Ivan Levkivskyi in bpo-31333.)

Method calls are now up to 20% faster due to the bytecode changes which avoid creating bound method instances. (Contributed by Yury Selivanov and INADA Naoki in bpo-26110.)

The asyncio module received a number of notable optimizations for commonly used functions:

  • The asyncio.get_event_loop() function has been reimplemented in C to make it up to 15 times faster. (Contributed by Yury Selivanov in bpo-32296.)
  • asyncio.Future callback management has been optimized. (Contributed by Yury Selivanov in bpo-32348.)
  • asyncio.gather() is now up to 15% faster. (Contributed by Yury Selivanov in bpo-32355.)
  • asyncio.sleep() is now up to 2 times faster when the delay argument is zero or negative. (Contributed by Andrew Svetlov in bpo-32351.)
  • The performance overhead of asyncio debug mode has been reduced. (Contributed by Antoine Pitrou in bpo-31970.)

As a result of PEP 560 work, the import time of typing has been reduced by a factor of 7, and many typing operations are now faster. (Contributed by Ivan Levkivskyi in bpo-32226.)

sorted() and list.sort() have been optimized for common cases to be up to 40-75% faster. (Contributed by Elliot Gorokhovsky in bpo-28685.)

dict.copy() is now up to 5.5 times faster. (Contributed by Yury Selivanov in bpo-31179.)

hasattr() and getattr() are now about 4 times faster when name is not found and obj does not override object.__getattr__() or object.__getattribute__(). (Contributed by INADA Naoki in bpo-32544.)

Searching for certain Unicode characters (like Ukrainian capital "Є") in a string was up to 25 times slower than searching for other characters. It is now only 3 times slower in the worst case. (Contributed by Serhiy Storchaka in bpo-24821.)

The collections.namedtuple() factory has been reimplemented to make the creation of named tuples 4 to 6 times faster. (Contributed by Jelle Zijlstra with further improvements by INADA Naoki, Serhiy Storchaka, and Raymond Hettinger in bpo-28638.)

date.fromordinal() and date.fromtimestamp() are now up to 30% faster in the common case. (Contributed by Paul Ganssle in bpo-32403.)

The os.fwalk() function is now up to 2 times faster thanks to the use of os.scandir(). (Contributed by Serhiy Storchaka in bpo-25996.)

The speed of the shutil.rmtree() function has been improved by 20--40% thanks to the use of the os.scandir() function. (Contributed by Serhiy Storchaka in bpo-28564.)

Optimized case-insensitive matching and searching of regular expressions. Searching some patterns can now be up to 20 times faster. (Contributed by Serhiy Storchaka in bpo-30285.)

re.compile() now converts flags parameter to int object if it is RegexFlag. It is now as fast as Python 3.5, and faster than Python 3.6 by about 10% depending on the pattern. (Contributed by INADA Naoki in bpo-31671.)

The modify() methods of classes selectors.EpollSelector, selectors.PollSelector and selectors.DevpollSelector may be around 10% faster under heavy loads. (Contributed by Giampaolo Rodola' in bpo-30014)

Constant folding has been moved from the peephole optimizer to the new AST optimizer, which is able perform optimizations more consistently. (Contributed by Eugene Toder and INADA Naoki in bpo-29469 and bpo-11549.)

Most functions and methods in abc have been rewritten in C. This makes creation of abstract base classes, and calling isinstance() and issubclass() on them 1.5x faster. This also reduces Python start-up time by up to 10%. (Contributed by Ivan Levkivskyi and INADA Naoki in bpo-31333)

Significant speed improvements to alternate constructors for datetime.date and datetime.datetime by using fast-path constructors when not constructing subclasses. (Contributed by Paul Ganssle in bpo-32403)

The speed of comparison of array.array instances has been improved considerably in certain cases. It is now from 10x to 70x faster when comparing arrays holding values of the same integer type. (Contributed by Adrian Wielgosik in bpo-24700.)

The math.erf() and math.erfc() functions now use the (faster) C library implementation on most platforms. (Contributed by Serhiy Storchaka in bpo-26121.)

Other CPython Implementation Changes

  • Trace hooks may now opt out of receiving the line and opt into receiving the opcode events from the interpreter by setting the corresponding new f_trace_lines and f_trace_opcodes attributes on the frame being traced. (Contributed by Nick Coghlan in bpo-31344.)
  • Fixed some consistency problems with namespace package module attributes. Namespace module objects now have an __file__ that is set to None (previously unset), and their __spec__.origin is also set to None (previously the string "namespace"). See bpo-32305. Also, the namespace module object's __spec__.loader is set to the same value as __loader__ (previously, the former was set to None). See bpo-32303.
  • The locals() dictionary now displays in the lexical order that variables were defined. Previously, the order was undefined. (Contributed by Raymond Hettinger in bpo-32690.)
  • The distutils upload command no longer tries to change CR end-of-line characters to CRLF. This fixes a corruption issue with sdists that ended with a byte equivalent to CR. (Contributed by Bo Bayles in bpo-32304.)

Deprecated Python Behavior

Yield expressions (both yield and yield from clauses) are now deprecated in comprehensions and generator expressions (aside from the iterable expression in the leftmost for clause). This ensures that comprehensions always immediately return a container of the appropriate type (rather than potentially returning a generator iterator object), while generator expressions won't attempt to interleave their implicit output with the output from any explicit yield expressions. In Python 3.7, such expressions emit DeprecationWarning when compiled, in Python 3.8 this will be a SyntaxError. (Contributed by Serhiy Storchaka in bpo-10544.)

Returning a subclass of complex from object.__complex__() is deprecated and will be an error in future Python versions. This makes __complex__() consistent with object.__int__() and object.__float__(). (Contributed by Serhiy Storchaka in bpo-28894.)

Deprecated Python modules, functions and methods

aifc

aifc.openfp() has been deprecated and will be removed in Python 3.9. Use aifc.open() instead. (Contributed by Brian Curtin in bpo-31985.)

asyncio

Support for directly await-ing instances of asyncio.Lock and other asyncio synchronization primitives has been deprecated. An asynchronous context manager must be used in order to acquire and release the synchronization resource. (Contributed by Andrew Svetlov in bpo-32253.)

The asyncio.Task.current_task() and asyncio.Task.all_tasks() methods have been deprecated. (Contributed by Andrew Svetlov in bpo-32250.)

collections

In Python 3.8, the abstract base classes in collections.abc will no longer be exposed in the regular collections module. This will help create a clearer distinction between the concrete classes and the abstract base classes. (Contributed by Serhiy Storchaka in bpo-25988.)

dbm

dbm.dumb now supports reading read-only files and no longer writes the index file when it is not changed. A deprecation warning is now emitted if the index file is missing and recreated in the 'r' and 'w' modes (this will be an error in future Python releases). (Contributed by Serhiy Storchaka in bpo-28847.)

enum

In Python 3.8, attempting to check for non-Enum objects in Enum classes will raise a TypeError (e.g. 1 in Color); similarly, attempting to check for non-Flag objects in a Flag member will raise TypeError (e.g. 1 in Perm.RW); currently, both operations return False instead. (Contributed by Ethan Furman in bpo-33217.)

gettext

Using non-integer value for selecting a plural form in gettext is now deprecated. It never correctly worked. (Contributed by Serhiy Storchaka in bpo-28692.)

importlib

Methods MetaPathFinder.find_module() (replaced by MetaPathFinder.find_spec()) and PathEntryFinder.find_loader() (replaced by PathEntryFinder.find_spec()) both deprecated in Python 3.4 now emit DeprecationWarning. (Contributed by Matthias Bussonnier in bpo-29576)

The importlib.abc.ResourceLoader ABC has been deprecated in favour of importlib.abc.ResourceReader.

locale

locale.format() has been deprecated, use locale.format_string() instead. (Contributed by Garvit in bpo-10379.)

macpath

The macpath is now deprecated and will be removed in Python 3.8. (Contributed by Chi Hsuan Yen in bpo-9850.)

threading

dummy_threading and _dummy_thread have been deprecated. It is no longer possible to build Python with threading disabled. Use threading instead. (Contributed by Antoine Pitrou in bpo-31370.)

socket

The silent argument value truncation in socket.htons() and socket.ntohs() has been deprecated. In future versions of Python, if the passed argument is larger than 16 bits, an exception will be raised. (Contributed by Oren Milman in bpo-28332.)

ssl

ssl.wrap_socket() is deprecated. Use ssl.SSLContext.wrap_socket() instead. (Contributed by Christian Heimes in bpo-28124.)

sunau

sunau.openfp() has been deprecated and will be removed in Python 3.9. Use sunau.open() instead. (Contributed by Brian Curtin in bpo-31985.)

sys

Deprecated sys.set_coroutine_wrapper() and sys.get_coroutine_wrapper().

The undocumented sys.callstats() function has been deprecated and will be removed in a future Python version. (Contributed by Victor Stinner in bpo-28799.)

wave

wave.openfp() has been deprecated and will be removed in Python 3.9. Use wave.open() instead. (Contributed by Brian Curtin in bpo-31985.)

Deprecated functions and types of the C API

Function PySlice_GetIndicesEx() is deprecated and replaced with a macro if Py_LIMITED_API is not set or set to a value in the range between 0x03050400 and 0x03060000 (not inclusive), or is 0x03060100 or higher. (Contributed by Serhiy Storchaka in bpo-27867.)

PyOS_AfterFork() has been deprecated. Use PyOS_BeforeFork(), PyOS_AfterFork_Parent() or PyOS_AfterFork_Child() instead. (Contributed by Antoine Pitrou in bpo-16500.)

Platform Support Removals

  • FreeBSD 9 and older are no longer officially supported.

  • For full Unicode support, including within extension modules, *nix platforms are now expected to provide at least one of C.UTF-8 (full locale), C.utf8 (full locale) or UTF-8 (LC_CTYPE-only locale) as an alternative to the legacy ASCII-based C locale.

  • OpenSSL 0.9.8 and 1.0.1 are no longer supported, which means building CPython 3.7 with SSL/TLS support on older platforms still using these versions requires custom build options that link to a more recent version of OpenSSL.

    Notably, this issue affects the Debian 8 (aka "jessie") and Ubuntu 14.04 (aka "Trusty") LTS Linux distributions, as they still use OpenSSL 1.0.1 by default.

    Debian 9 ("stretch") and Ubuntu 16.04 ("xenial"), as well as recent releases of other LTS Linux releases (e.g. RHEL/CentOS 7.5, SLES 12-SP3), use OpenSSL 1.0.2 or later, and remain supported in the default build configuration.

    CPython's own CI configuration file provides an example of using the SSL compatibility testing infrastructure in CPython's test suite to build and link against OpenSSL 1.1.0 rather than an outdated system provided OpenSSL.

API and Feature Removals

The following features and APIs have been removed from Python 3.7:

  • The os.stat_float_times() function has been removed. It was introduced in Python 2.3 for backward compatibility with Python 2.2, and was deprecated since Python 3.1.
  • Unknown escapes consisting of '\' and an ASCII letter in replacement templates for re.sub() were deprecated in Python 3.5, and will now cause an error.
  • Removed support of the exclude argument in tarfile.TarFile.add(). It was deprecated in Python 2.7 and 3.2. Use the filter argument instead.
  • The splitunc() function in the ntpath module was deprecated in Python 3.1, and has now been removed. Use the splitdrive() function instead.
  • collections.namedtuple() no longer supports the verbose parameter or _source attribute which showed the generated source code for the named tuple class. This was part of an optimization designed to speed-up class creation. (Contributed by Jelle Zijlstra with further improvements by INADA Naoki, Serhiy Storchaka, and Raymond Hettinger in bpo-28638.)
  • Functions bool(), float(), list() and tuple() no longer take keyword arguments. The first argument of int() can now be passed only as positional argument.
  • Removed previously deprecated in Python 2.4 classes Plist, Dict and _InternalDict in the plistlib module. Dict values in the result of functions readPlist() and readPlistFromBytes() are now normal dicts. You no longer can use attribute access to access items of these dictionaries.
  • The asyncio.windows_utils.socketpair() function has been removed. Use the socket.socketpair() function instead, it is available on all platforms since Python 3.5. asyncio.windows_utils.socketpair was just an alias to socket.socketpair on Python 3.5 and newer.
  • asyncio no longer exports the selectors and _overlapped modules as asyncio.selectors and asyncio._overlapped. Replace from asyncio import selectors with import selectors.
  • Direct instantiation of ssl.SSLSocket and ssl.SSLObject objects is now prohibited. The constructors were never documented, tested, or designed as public constructors. Users were supposed to use ssl.wrap_socket() or ssl.SSLContext. (Contributed by Christian Heimes in bpo-32951.)
  • The unused distutils install_misc command has been removed. (Contributed by Eric N. Vander Weele in bpo-29218.)

Module Removals

The fpectl module has been removed. It was never enabled by default, never worked correctly on x86-64, and it changed the Python ABI in ways that caused unexpected breakage of C extensions. (Contributed by Nathaniel J. Smith in bpo-29137.)

Windows-only Changes

The python launcher, (py.exe), can accept 32 & 64 bit specifiers without having to specify a minor version as well. So py -3-32 and py -3-64 become valid as well as py -3.7-32, also the -m-64 and -m.n-64 forms are now accepted to force 64 bit python even if 32 bit would have otherwise been used. If the specified version is not available py.exe will error exit. (Contributed by Steve Barnes in bpo-30291.)

The launcher can be run as py -0 to produce a list of the installed pythons, with default marked with an asterisk. Running py -0p will include the paths. If py is run with a version specifier that cannot be matched it will also print the short form list of available specifiers. (Contributed by Steve Barnes in bpo-30362.)

Porting to Python 3.7

This section lists previously described changes and other bugfixes that may require changes to your code.

Changes in Python Behavior

  • async and await names are now reserved keywords. Code using these names as identifiers will now raise a SyntaxError. (Contributed by Jelle Zijlstra in bpo-30406.)

  • PEP 479 is enabled for all code in Python 3.7, meaning that StopIteration exceptions raised directly or indirectly in coroutines and generators are transformed into RuntimeError exceptions. (Contributed by Yury Selivanov in bpo-32670.)

  • object.__aiter__() methods can no longer be declared as asynchronous. (Contributed by Yury Selivanov in bpo-31709.)

  • Due to an oversight, earlier Python versions erroneously accepted the following syntax:

    f(1 for x in [1],)
    
    class C(1 for x in [1]):
        pass
    

    Python 3.7 now correctly raises a SyntaxError, as a generator expression always needs to be directly inside a set of parentheses and cannot have a comma on either side, and the duplication of the parentheses can be omitted only on calls. (Contributed by Serhiy Storchaka in bpo-32012 and bpo-32023.)

  • When using the -m switch, the initial working directory is now added to sys.path, rather than an empty string (which dynamically denoted the current working directory at the time of each import). Any programs that are checking for the empty string, or otherwise relying on the previous behaviour, will need to be updated accordingly (e.g. by also checking for os.getcwd() or os.path.dirname(__main__.__file__), depending on why the code was checking for the empty string in the first place).

Changes in the Python API

  • socketserver.ThreadingMixIn.server_close() now waits until all non-daemon threads complete. Set the new socketserver.ThreadingMixIn.block_on_close class attribute to False to get the pre-3.7 behaviour. (Contributed by Victor Stinner in bpo-31233 and bpo-33540.)

  • socketserver.ForkingMixIn.server_close() now waits until all child processes complete. Set the new socketserver.ForkingMixIn.block_on_close class attribute to False to get the pre-3.7 behaviour. (Contributed by Victor Stinner in bpo-31151 and bpo-33540.)

  • The locale.localeconv() function now temporarily sets the LC_CTYPE locale to the value of LC_NUMERIC in some cases. (Contributed by Victor Stinner in bpo-31900.)

  • pkgutil.walk_packages() now raises a ValueError if path is a string. Previously an empty list was returned. (Contributed by Sanyam Khurana in bpo-24744.)

  • A format string argument for string.Formatter.format() is now positional-only. Passing it as a keyword argument was deprecated in Python 3.5. (Contributed by Serhiy Storchaka in bpo-29193.)

  • Attributes key, value and coded_value of class http.cookies.Morsel are now read-only. Assigning to them was deprecated in Python 3.5. Use the set() method for setting them. (Contributed by Serhiy Storchaka in bpo-29192.)

  • The mode argument of os.makedirs() no longer affects the file permission bits of newly-created intermediate-level directories. To set their file permission bits you can set the umask before invoking makedirs(). (Contributed by Serhiy Storchaka in bpo-19930.)

  • The struct.Struct.format type is now str instead of bytes. (Contributed by Victor Stinner in bpo-21071.)

  • parse_multipart() now accepts the encoding and errors arguments and returns the same results as FieldStorage: for non-file fields, the value associated to a key is a list of strings, not bytes. (Contributed by Pierre Quentel in bpo-29979.)

  • Due to internal changes in socket, calling socket.fromshare() on a socket created by socket.share in older Python versions is not supported.

  • repr for BaseException has changed to not include the trailing comma. Most exceptions are affected by this change. (Contributed by Serhiy Storchaka in bpo-30399.)

  • repr for datetime.timedelta has changed to include the keyword arguments in the output. (Contributed by Utkarsh Upadhyay in bpo-30302.)

  • Because shutil.rmtree() is now implemented using the os.scandir() function, the user specified handler onerror is now called with the first argument os.scandir instead of os.listdir when listing the directory is failed.

  • Support for nested sets and set operations in regular expressions as in Unicode Technical Standard #18 might be added in the future. This would change the syntax. To facilitate this future change a FutureWarning will be raised in ambiguous cases for the time being. That include sets starting with a literal '[' or containing literal character sequences '--', '&&', '~~', and '||'. To avoid a warning, escape them with a backslash. (Contributed by Serhiy Storchaka in bpo-30349.)

  • The result of splitting a string on a regular expression that could match an empty string has been changed. For example splitting on r'\s*' will now split not only on whitespaces as it did previously, but also on empty strings before all non-whitespace characters and just before the end of the string. The previous behavior can be restored by changing the pattern to r'\s+'. A FutureWarning was emitted for such patterns since Python 3.5.

    For patterns that match both empty and non-empty strings, the result of searching for all matches may also be changed in other cases. For example in the string 'a\n\n', the pattern r'(?m)^\s*?$' will not only match empty strings at positions 2 and 3, but also the string '\n' at positions 2--3. To match only blank lines, the pattern should be rewritten as r'(?m)^[^\S\n]*$'.

    re.sub() now replaces empty matches adjacent to a previous non-empty match. For example re.sub('x*', '-', 'abxd') returns now '-a-b--d-' instead of '-a-b-d-' (the first minus between 'b' and 'd' replaces 'x', and the second minus replaces an empty string between 'x' and 'd').

    (Contributed by Serhiy Storchaka in bpo-25054 and bpo-32308.)

  • Change re.escape() to only escape regex special characters instead of escaping all characters other than ASCII letters, numbers, and '_'. (Contributed by Serhiy Storchaka in bpo-29995.)

  • tracemalloc.Traceback frames are now sorted from oldest to most recent to be more consistent with traceback. (Contributed by Jesse Bakker in bpo-32121.)

  • On OSes that support socket.SOCK_NONBLOCK or socket.SOCK_CLOEXEC bit flags, the socket.type no longer has them applied. Therefore, checks like if sock.type == socket.SOCK_STREAM work as expected on all platforms. (Contributed by Yury Selivanov in bpo-32331.)

  • On Windows the default for the close_fds argument of subprocess.Popen was changed from False to True when redirecting the standard handles. If you previously depended on handles being inherited when using subprocess.Popen with standard io redirection, you will have to pass close_fds=False to preserve the previous behaviour, or use STARTUPINFO.lpAttributeList.

  • importlib.machinery.PathFinder.invalidate_caches() -- which implicitly affects importlib.invalidate_caches() -- now deletes entries in sys.path_importer_cache which are set to None. (Contributed by Brett Cannon in bpo-33169.)

  • In asyncio, loop.sock_recv(), loop.sock_sendall(), loop.sock_accept(), loop.getaddrinfo(), loop.getnameinfo() have been changed to be proper coroutine methods to match their documentation. Previously, these methods returned asyncio.Future instances. (Contributed by Yury Selivanov in bpo-32327.)

  • asyncio.Server.sockets now returns a copy of the internal list of server sockets, instead of returning it directly. (Contributed by Yury Selivanov in bpo-32662.)

  • Struct.format is now a str instance instead of a bytes instance. (Contributed by Victor Stinner in bpo-21071.)

  • ast.literal_eval() is now stricter. Addition and subtraction of arbitrary numbers are no longer allowed. (Contributed by Serhiy Storchaka in bpo-31778.)

  • Calendar.itermonthdates will now consistently raise an exception when a date falls outside of the 0001-01-01 through 9999-12-31 range. To support applications that cannot tolerate such exceptions, the new Calendar.itermonthdays3 and Calendar.itermonthdays4 can be used. The new methods return tuples and are not restricted by the range supported by datetime.date. (Contributed by Alexander Belopolsky in bpo-28292.)

  • collections.ChainMap now preserves the order of the underlying mappings. (Contributed by Raymond Hettinger in bpo-32792.)

  • The submit() method of concurrent.futures.ThreadPoolExecutor and concurrent.futures.ProcessPoolExecutor now raises a RuntimeError if called during interpreter shutdown. (Contributed by Mark Nemec in bpo-33097.)

  • The configparser.ConfigParser constructor now uses read_dict() to process the default values, making its behavior consistent with the rest of the parser. Non-string keys and values in the defaults dictionary are now being implicitly converted to strings. (Contributed by James Tocknell in bpo-23835.)

  • Several undocumented internal imports were removed. One example is that os.errno is no longer available; use import errno directly instead. Note that such undocumented internal imports may be removed any time without notice, even in micro version releases.

Changes in the C API

The function PySlice_GetIndicesEx() is considered unsafe for resizable sequences. If the slice indices are not instances of int, but objects that implement the __index__() method, the sequence can be resized after passing its length to PySlice_GetIndicesEx(). This can lead to returning indices out of the length of the sequence. For avoiding possible problems use new functions PySlice_Unpack() and PySlice_AdjustIndices(). (Contributed by Serhiy Storchaka in bpo-27867.)

CPython bytecode changes

There are two new opcodes: LOAD_METHOD and CALL_METHOD. (Contributed by Yury Selivanov and INADA Naoki in bpo-26110.)

The STORE_ANNOTATION opcode has been removed. (Contributed by Mark Shannon in bpo-32550.)

Windows-only Changes

The file used to override sys.path is now called <python-executable>._pth instead of 'sys.path'. See Finding modules for more information. (Contributed by Steve Dower in bpo-28137.)

Other CPython implementation changes

In preparation for potential future changes to the public CPython runtime initialization API (see PEP 432 for an initial, but somewhat outdated, draft), CPython's internal startup and configuration management logic has been significantly refactored. While these updates are intended to be entirely transparent to both embedding applications and users of the regular CPython CLI, they're being mentioned here as the refactoring changes the internal order of various operations during interpreter startup, and hence may uncover previously latent defects, either in embedding applications, or in CPython itself. (Initially contributed by Nick Coghlan and Eric Snow as part of bpo-22257, and further updated by Nick, Eric, and Victor Stinner in a number of other issues). Some known details affected:

  • PySys_AddWarnOptionUnicode() is not currently usable by embedding applications due to the requirement to create a Unicode object prior to calling Py_Initialize. Use PySys_AddWarnOption() instead.
  • warnings filters added by an embedding application with PySys_AddWarnOption() should now more consistently take precedence over the default filters set by the interpreter

Due to changes in the way the default warnings filters are configured, setting Py_BytesWarningFlag to a value greater than one is no longer sufficient to both emit BytesWarning messages and have them converted to exceptions. Instead, the flag must be set (to cause the warnings to be emitted in the first place), and an explicit error::BytesWarning warnings filter added to convert them to exceptions.

Due to a change in the way docstrings are handled by the compiler, the implicit return None in a function body consisting solely of a docstring is now marked as occurring on the same line as the docstring, not on the function's header line.

The current exception state has been moved from the frame object to the co-routine. This simplified the interpreter and fixed a couple of obscure bugs caused by having swap exception state when entering or exiting a generator. (Contributed by Mark Shannon in bpo-25612.)

Notable changes in Python 3.7.1

Starting in 3.7.1, Py_Initialize() now consistently reads and respects all of the same environment settings as Py_Main() (in earlier Python versions, it respected an ill-defined subset of those environment variables, while in Python 3.7.0 it didn't read any of them due to bpo-34247). If this behavior is unwanted, set Py_IgnoreEnvironmentFlag to 1 before calling Py_Initialize().

In 3.7.1 the C API for Context Variables was updated to use PyObject pointers. See also bpo-34762.

In 3.7.1 the tokenize module now implicitly emits a NEWLINE token when provided with input that does not have a trailing new line. This behavior now matches what the C tokenizer does internally. (Contributed by Ammar Askar in bpo-33899.)

Notable changes in Python 3.7.2

In 3.7.2, venv on Windows no longer copies the original binaries, but creates redirector scripts named python.exe and pythonw.exe instead. This resolves a long standing issue where all virtual environments would have to be upgraded or recreated with each Python update. However, note that this release will still require recreation of virtual environments in order to get the new scripts.