サブプロセス

このセクションはサブプロセスの生成と管理を行う高水準の async/await asyncio API について説明します。

以下は、 asyncio モジュールがどのようにシェルコマンドを実行し、結果を取得できるかを示す例です:

import asyncio

async def run(cmd):
    proc = await asyncio.create_subprocess_shell(
        cmd,
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE)

    stdout, stderr = await proc.communicate()

    print(f'[{cmd!r} exited with {proc.returncode}]')
    if stdout:
        print(f'[stdout]\n{stdout.decode()}')
    if stderr:
        print(f'[stderr]\n{stderr.decode()}')

asyncio.run(run('ls /zzz'))

このサンプルコードは以下を出力します:

['ls /zzz' exited with 1]
[stderr]
ls: /zzz: No such file or directory

Because all asyncio subprocess functions are asynchronous and asyncio provides many tools to work with such functions, it is easy to execute and monitor multiple subprocesses in parallel. It is indeed trivial to modify the above example to run several commands simultaneously:

async def main():
    await asyncio.gather(
        run('ls /zzz'),
        run('sleep 1; echo "hello"'))

asyncio.run(main())

See also the Examples subsection.

Creating Subprocesses

coroutine asyncio.create_subprocess_exec(program, *args, stdin=None, stdout=None, stderr=None, loop=None, limit=None, **kwds)

サブプロセスを作成します。

The limit argument sets the buffer limit for StreamReader wrappers for Process.stdout and Process.stderr (if subprocess.PIPE is passed to stdout and stderr arguments).

Process のインスタンスを返します。

See the documentation of loop.subprocess_exec() for other parameters.

coroutine asyncio.create_subprocess_shell(cmd, stdin=None, stdout=None, stderr=None, loop=None, limit=None, **kwds)

Run the cmd shell command.

The limit argument sets the buffer limit for StreamReader wrappers for Process.stdout and Process.stderr (if subprocess.PIPE is passed to stdout and stderr arguments).

Process のインスタンスを返します。

See the documentation of loop.subprocess_shell() for other parameters.

重要

It is the application's responsibility to ensure that all whitespace and special characters are quoted appropriately to avoid shell injection vulnerabilities. The shlex.quote() function can be used to properly escape whitespace and special shell characters in strings that are going to be used to construct shell commands.

注釈

The default asyncio event loop implementation on Windows does not support subprocesses. Subprocesses are available for Windows if a ProactorEventLoop is used. See Subprocess Support on Windows for details.

参考

asyncio also has the following low-level APIs to work with subprocesses: loop.subprocess_exec(), loop.subprocess_shell(), loop.connect_read_pipe(), loop.connect_write_pipe(), as well as the Subprocess Transports and Subprocess Protocols.

定数

asyncio.subprocess.PIPE

Can be passed to the stdin, stdout or stderr parameters.

If PIPE is passed to stdin argument, the Process.stdin attribute will point to a StreamWriter instance.

If PIPE is passed to stdout or stderr arguments, the Process.stdout and Process.stderr attributes will point to StreamReader instances.

asyncio.subprocess.STDOUT

Special value that can be used as the stderr argument and indicates that standard error should be redirected into standard output.

asyncio.subprocess.DEVNULL

Special value that can be used as the stdin, stdout or stderr argument to process creation functions. It indicates that the special file os.devnull will be used for the corresponding subprocess stream.

Interacting with Subprocesses

Both create_subprocess_exec() and create_subprocess_shell() functions return instances of the Process class. Process is a high-level wrapper that allows communicating with subprocesses and watching for their completion.

class asyncio.subprocess.Process

An object that wraps OS processes created by the create_subprocess_exec() and create_subprocess_shell() functions.

This class is designed to have a similar API to the subprocess.Popen class, but there are some notable differences:

このクラスは スレッド安全ではありません。

サブプロセスとスレッド 節も参照してください。

coroutine wait()

Wait for the child process to terminate.

Set and return the returncode attribute.

注釈

This method can deadlock when using stdout=PIPE or stderr=PIPE and the child process generates so much output that it blocks waiting for the OS pipe buffer to accept more data. Use the communicate() method when using pipes to avoid this condition.

coroutine communicate(input=None)

Interact with process:

  1. send data to stdin (if input is not None);

  2. read data from stdout and stderr, until EOF is reached;

  3. wait for process to terminate.

The optional input argument is the data (bytes object) that will be sent to the child process.

Return a tuple (stdout_data, stderr_data).

If either BrokenPipeError or ConnectionResetError exception is raised when writing input into stdin, the exception is ignored. This condition occurs when the process exits before all data are written into stdin.

If it is desired to send data to the process' stdin, the process needs to be created with stdin=PIPE. Similarly, to get anything other than None in the result tuple, the process has to be created with stdout=PIPE and/or stderr=PIPE arguments.

Note, that the data read is buffered in memory, so do not use this method if the data size is large or unlimited.

send_signal(signal)

子プロセスにシグナル signal を送信します。

注釈

Windows では、SIGTERM は terminate() の別名になります。CTRL_C_EVENT および CTRL_BREAK_EVENT で、creationflags で始まり、CREATE_NEW_PROCESS_GROUP を含むパラメータをプロセスに送信することができます。

terminate()

Stop the child process.

On POSIX systems this method sends signal.SIGTERM to the child process.

On Windows the Win32 API function TerminateProcess() is called to stop the child process.

kill()

Kill the child.

On POSIX systems this method sends SIGKILL to the child process.

On Windows this method is an alias for terminate().

stdin

Standard input stream (StreamWriter) or None if the process was created with stdin=None.

stdout

Standard output stream (StreamReader) or None if the process was created with stdout=None.

stderr

Standard error stream (StreamReader) or None if the process was created with stderr=None.

警告

Use the communicate() method rather than process.stdin.write(), await process.stdout.read() or await process.stderr.read. This avoids deadlocks due to streams pausing reading or writing and blocking the child process.

pid

Process identification number (PID).

Note that for processes created by the create_subprocess_shell() function, this attribute is the PID of the spawned shell.

returncode

Return code of the process when it exits.

A None value indicates that the process has not terminated yet.

負の値 -N は子プロセスがシグナル N により中止させられたことを示します (POSIX のみ)。

Subprocess and Threads

Standard asyncio event loop supports running subprocesses from different threads, but there are limitations:

  • An event loop must run in the main thread.

  • The child watcher must be instantiated in the main thread before executing subprocesses from other threads. Call the get_child_watcher() function in the main thread to instantiate the child watcher.

Note that alternative event loop implementations might not share the above limitations; please refer to their documentation.

参考

asyncio-multithreading

使用例

An example using the Process class to control a subprocess and the StreamReader class to read from its standard output.

The subprocess is created by the create_subprocess_exec() function:

import asyncio
import sys

async def get_date():
    code = 'import datetime; print(datetime.datetime.now())'

    # Create the subprocess; redirect the standard output
    # into a pipe.
    proc = await asyncio.create_subprocess_exec(
        sys.executable, '-c', code,
        stdout=asyncio.subprocess.PIPE)

    # Read one line of output.
    data = await proc.stdout.readline()
    line = data.decode('ascii').rstrip()

    # Wait for the subprocess exit.
    await proc.wait()
    return line

if sys.platform == "win32":
    asyncio.set_event_loop_policy(
        asyncio.WindowsProactorEventLoopPolicy())

date = asyncio.run(get_date())
print(f"Current date: {date}")

See also the same example written using low-level APIs.