定义扩展模块

A C extension for CPython is a shared library (for example, a .so file on Linux, .pyd DLL on Windows), which is loadable into the Python process (for example, it is compiled with compatible compiler settings), and which exports an initialization function.

要在默认情况下可被导入 (也就是说,通过 importlib.machinery.ExtensionFileLoader),共享库必须在 sys.path 中可用,并且必须命名为模块名之后加一个在 importlib.machinery.EXTENSION_SUFFIXES 中列出的扩展名。

备注

构建、打包和分发扩展模块最好使用第三方工具完成,并且超出了本文的范围。一个合适的工具是 Setuptools,其文档可以在 https://setuptools.pypa.io/en/latest/setuptools.html 上找到。

Normally, the initialization function returns a module definition initialized using PyModuleDef_Init(). This allows splitting the creation process into several phases:

  • Before any substantial code is executed, Python can determine which capabilities the module supports, and it can adjust the environment or refuse loading an incompatible extension.

  • By default, Python itself creates the module object -- that is, it does the equivalent of object.__new__() for classes. It also sets initial attributes like __package__ and __loader__.

  • Afterwards, the module object is initialized using extension-specific code -- the equivalent of __init__() on classes.

This is called multi-phase initialization to distinguish it from the legacy (but still supported) single-phase initialization scheme, where the initialization function returns a fully constructed module. See the single-phase-initialization section below for details.

在 3.5 版本发生变更: 增加了对多阶段初始化的支持 (PEP 489)。

多个模块实例

By default, extension modules are not singletons. For example, if the sys.modules entry is removed and the module is re-imported, a new module object is created, and typically populated with fresh method and type objects. The old module is subject to normal garbage collection. This mirrors the behavior of pure-Python modules.

额外的模块实例可能会在 子解释器 中或者 Python 运行时重新初始化之后 (Py_Finalize()Py_Initialize()) 被创建。在这些情况下,模块实例间共享 Python 对象可能导致程序崩溃或未定义的行为。

为避免这种问题,每个扩展模块的实例都应当是 隔离的: 对一个实例的修改不应隐式地影响其他的实例,以及模块所拥有的全部状态,包括对 Python 对象的引用,都应当是特定模块实例专属的。请参阅 隔离扩展模块 了解更多的细节和实用的指南。

一个避免这些问题的简单方式是 针对重复的初始化引发一个错误

所有模块都应当支持 子解释器,否则就要显式地发出缺乏支持的信号。 这往往是通过隔离或阻止重复的初始化,如上文所述。一个模块也可以使用 Py_mod_multiple_interpreters 槽位将其限制于主解释器中。

Initialization function

The initialization function defined by an extension module has the following signature:

PyObject *PyInit_modulename(void)

Its name should be PyInit_<name>, with <name> replaced by the name of the module.

For modules with ASCII-only names, the function must instead be named PyInit_<name>, with <name> replaced by the name of the module. When using 多阶段初始化, non-ASCII module names are allowed. In this case, the initialization function name is PyInitU_<name>, with <name> encoded using Python's punycode encoding with hyphens replaced by underscores. In Python:

def initfunc_name(name):
    try:
        suffix = b'_' + name.encode('ascii')
    except UnicodeEncodeError:
        suffix = b'U_' + name.encode('punycode').replace(b'-', b'_')
    return b'PyInit' + suffix

It is recommended to define the initialization function using a helper macro:

PyMODINIT_FUNC

声明一个扩展模块初始化函数。这个宏:

  • 指定了 PyObject* 返回类型,

  • 添加平台所需的任何特殊链接声明,以及

  • 对于 C++,将函数声明为 extern "C"

例如,一个名为 spam 的模块可以这样定义:

static struct PyModuleDef spam_module = {
    .m_base = PyModuleDef_HEAD_INIT,
    .m_name = "spam",
    ...
};

PyMODINIT_FUNC
PyInit_spam(void)
{
    return PyModuleDef_Init(&spam_module);
}

It is possible to export multiple modules from a single shared library by defining multiple initialization functions. However, importing them requires using symbolic links or a custom importer, because by default only the function corresponding to the filename is found. See the Multiple modules in one library section in PEP 489 for details.

The initialization function is typically the only non-static item defined in the module's C source.

多阶段初始化

Normally, the initialization function (PyInit_modulename) returns a PyModuleDef instance with non-NULL m_slots. Before it is returned, the PyModuleDef instance must be initialized using the following function:

PyObject *PyModuleDef_Init(PyModuleDef *def)
属于 稳定 ABI 自 3.5 版起.

确保模块定义是一个正确初始化的 Python 对象,并正确报告其类型以及引用计数。

返回强制转换为 PyObject*def,或者如果出现错误则返回 NULL

Calling this function is required for 多阶段初始化. It should not be used in other contexts.

请注意 Python 会假定 PyModuleDef 结构体是静态分配的。此函数可以返回一个新引用或借用引用;这个引用不可被释放。

Added in version 3.5.

旧式的单阶段初始化

注意

单阶段初始化是一种用于初始化扩展模块的旧机制,它具有已知的缺点和设计瑕疵。建议扩展模块作者改用多阶段初始化。

In single-phase initialization, the initialization function (PyInit_modulename) should create, populate and return a module object. This is typically done using PyModule_Create() and functions like PyModule_AddObjectRef().

单阶段初始化与 默认方式 的主要区别如下:

  • 单阶段模块本质上是(更准确地说,包含)"单例对象"。

    当模块首次初始化时,Python 会保存该模块 __dict__ 中的内容(通常包括模块的函数和类型等)。

    对于后续导入操作,Python 不会再次调用初始化函数,而是创建一个带有新 __dict__ 的模块对象,并将已保存的内容复制到其中。例如,假设有一个单阶段模块 _testsinglephase [1] 定义了函数 sum 和异常类 error:

    >>> import sys
    >>> import _testsinglephase as one
    >>> del sys.modules['_testsinglephase']
    >>> import _testsinglephase as two
    >>> one is two
    False
    >>> one.__dict__ is two.__dict__
    False
    >>> one.sum is two.sum
    True
    >>> one.error is two.error
    True
    

    该具体行为应被视为 CPython 的实现细节。

  • 为解决 PyInit_modulename 函数不接受 spec 参数的限制,导入机制会保存部分状态,并在 PyInit_modulename 调用期间将其应用于首个匹配的模块对象。具体表现为:当导入子模块时,该机制会将父包名称自动前置到模块名前。

    单阶段 PyInit_modulename 函数应当尽早创建"其所属"模块对象,该操作需在任何其他模块对象创建之前完成。

  • 不支持非 ASCII 模块名 (PyInitU_modulename)。

  • 单阶段模块支持模块查找函数如 PyState_FindModule()