diff options
Diffstat (limited to 'Doc/library/asyncio-task.rst')
| -rw-r--r-- | Doc/library/asyncio-task.rst | 1022 |
1 files changed, 455 insertions, 567 deletions
diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 2b480d4be3..670e4a5fbe 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -1,97 +1,119 @@ .. currentmodule:: asyncio -Tasks and coroutines + +==================== +Coroutines and Tasks ==================== -**Source code:** :source:`Lib/asyncio/tasks.py` +This section outlines high-level asyncio APIs to work with coroutines +and Tasks. + +.. contents:: + :depth: 1 + :local: -**Source code:** :source:`Lib/asyncio/coroutines.py` .. _coroutine: Coroutines ----------- +========== -Coroutines used with :mod:`asyncio` may be implemented using the -:keyword:`async def` statement, or by using :term:`generators <generator>`. -The :keyword:`async def` type of coroutine was added in Python 3.5, and -is recommended if there is no need to support older Python versions. +Coroutines declared with async/await syntax is the preferred way of +writing asyncio applications. For example, the following snippet +of code prints "hello", waits 1 second, and then prints "world":: -Generator-based coroutines should be decorated with :func:`@asyncio.coroutine -<asyncio.coroutine>`, although this is not strictly enforced. -The decorator enables compatibility with :keyword:`async def` coroutines, -and also serves as documentation. Generator-based -coroutines use the ``yield from`` syntax introduced in :pep:`380`, -instead of the original ``yield`` syntax. + >>> import asyncio -The word "coroutine", like the word "generator", is used for two -different (though related) concepts: + >>> async def main(): + ... print('hello') + ... await asyncio.sleep(1) + ... print('world') -- The function that defines a coroutine - (a function definition using :keyword:`async def` or - decorated with ``@asyncio.coroutine``). If disambiguation is needed - we will call this a *coroutine function* (:func:`iscoroutinefunction` - returns ``True``). + >>> asyncio.run(main()) + hello + world -- The object obtained by calling a coroutine function. This object - represents a computation or an I/O operation (usually a combination) - that will complete eventually. If disambiguation is needed we will - call it a *coroutine object* (:func:`iscoroutine` returns ``True``). +Note that simply calling a coroutine will not schedule it to +be executed:: -Things a coroutine can do: + >>> main() + <coroutine object main at 0x1053bb7c8> -- ``result = await future`` or ``result = yield from future`` -- - suspends the coroutine until the - future is done, then returns the future's result, or raises an - exception, which will be propagated. (If the future is cancelled, - it will raise a ``CancelledError`` exception.) Note that tasks are - futures, and everything said about futures also applies to tasks. +To actually run a coroutine asyncio provides three main mechanisms: -- ``result = await coroutine`` or ``result = yield from coroutine`` -- - wait for another coroutine to - produce a result (or raise an exception, which will be propagated). - The ``coroutine`` expression must be a *call* to another coroutine. +* The :func:`asyncio.run` function to run the top-level + entry point "main()" function (see the above example.) -- ``return expression`` -- produce a result to the coroutine that is - waiting for this one using :keyword:`await` or ``yield from``. +* Awaiting on a coroutine. The following snippet of code will + print "hello" after waiting for 1 second, and then print "world" + after waiting for *another* 2 seconds:: -- ``raise exception`` -- raise an exception in the coroutine that is - waiting for this one using :keyword:`await` or ``yield from``. + import asyncio + import time -Calling a coroutine does not start its code running -- -the coroutine object returned by the call doesn't do anything until you -schedule its execution. There are two basic ways to start it running: -call ``await coroutine`` or ``yield from coroutine`` from another coroutine -(assuming the other coroutine is already running!), or schedule its execution -using the :func:`ensure_future` function or the :meth:`AbstractEventLoop.create_task` -method. + async def say_after(delay, what): + await asyncio.sleep(delay) + print(what) + async def main(): + print('started at', time.strftime('%X')) -Coroutines (and tasks) can only run when the event loop is running. + await say_after(1, 'hello') + await say_after(2, 'world') -.. decorator:: coroutine + print('finished at', time.strftime('%X')) - Decorator to mark generator-based coroutines. This enables - the generator use :keyword:`!yield from` to call :keyword:`async - def` coroutines, and also enables the generator to be called by - :keyword:`async def` coroutines, for instance using an - :keyword:`await` expression. + asyncio.run(main()) - There is no need to decorate :keyword:`async def` coroutines themselves. + Expected output:: - If the generator is not yielded from before it is destroyed, an error - message is logged. See :ref:`Detect coroutines never scheduled - <asyncio-coroutine-not-scheduled>`. + started at 17:13:52 + hello + world + finished at 17:13:55 -.. note:: +* The :func:`asyncio.create_task` function to run coroutines + concurrently as asyncio :class:`Tasks <Task>`. + + Let's modify the above example and run two "set_after" coroutines + *concurrently*:: + + async def main(): + task1 = asyncio.create_task( + say_after(1, 'hello')) + + task2 = asyncio.create_task( + say_after(2, 'world')) + + print('started at', time.strftime('%X')) + + # Wait until both tasks are completed (should take + # around 2 seconds.) + await task1 + await task2 + + print('finished at', time.strftime('%X')) + + Note that expected output now shows that the snippet runs + 1 second faster than before:: + + started at 17:14:32 + hello + world + finished at 17:14:34 + +Note that in this documentation the term "coroutine" can be used for +two closely related concepts: + +* a *coroutine function*: an :keyword:`async def` function; - In this documentation, some methods are documented as coroutines, - even if they are plain Python functions returning a :class:`Future`. - This is intentional to have a freedom of tweaking the implementation - of these functions in the future. If such a function is needed to be - used in a callback-style code, wrap its result with :func:`ensure_future`. +* a *coroutine object*: object returned by calling a + *coroutine function*. +Running an asyncio Program +========================== + .. function:: run(coro, \*, debug=False) This function runs the passed coroutine, taking care of @@ -101,45 +123,46 @@ Coroutines (and tasks) can only run when the event loop is running. This function cannot be called when another asyncio event loop is running in the same thread. - If debug is True, the event loop will be run in debug mode. + If *debug* is ``True``, the event loop will be run in debug mode. This function always creates a new event loop and closes it at the end. It should be used as a main entry point for asyncio programs, and should ideally only be called once. .. versionadded:: 3.7 - **Important:** this has been been added to asyncio in Python 3.7 - on a :term:`provisional basis <provisional api>`. + **Important:** this function has been added to asyncio in + Python 3.7 on a :term:`provisional basis <provisional api>`. -.. _asyncio-hello-world-coroutine: +Creating Tasks +============== -Example: Hello World coroutine -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +.. function:: create_task(coro) -Example of coroutine displaying ``"Hello World"``:: + Wrap the *coro* :ref:`coroutine <coroutine>` into a task and schedule + its execution. Return the task object. - import asyncio + The task is executed in the loop returned by :func:`get_running_loop`, + :exc:`RuntimeError` is raised if there is no running loop in + current thread. - async def hello_world(): - print("Hello World!") + .. versionadded:: 3.7 - asyncio.run(hello_world()) -.. seealso:: +Sleeping +======== - The :ref:`Hello World with call_soon() <asyncio-hello-world-callback>` - example uses the :meth:`AbstractEventLoop.call_soon` method to schedule a - callback. +.. coroutinefunction:: sleep(delay, result=None, \*, loop=None) + Block for *delay* seconds. -.. _asyncio-date-coroutine: + If *result* is provided, it is returned to the caller + when the coroutine completes. -Example: Coroutine displaying the current date -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + .. _asyncio_example_sleep: -Example of coroutine displaying the current date every second during 5 seconds -using the :meth:`sleep` function:: + Example of coroutine displaying the current date every second + for 5 seconds:: import asyncio import datetime @@ -155,408 +178,283 @@ using the :meth:`sleep` function:: asyncio.run(display_date()) -.. seealso:: - - The :ref:`display the current date with call_later() - <asyncio-date-callback>` example uses a callback with the - :meth:`AbstractEventLoop.call_later` method. - - -Example: Chain coroutines -^^^^^^^^^^^^^^^^^^^^^^^^^ - -Example chaining coroutines:: - - import asyncio - - async def compute(x, y): - print("Compute %s + %s ..." % (x, y)) - await asyncio.sleep(1.0) - return x + y - - async def print_sum(x, y): - result = await compute(x, y) - print("%s + %s = %s" % (x, y, result)) - - loop = asyncio.get_event_loop() - loop.run_until_complete(print_sum(1, 2)) - loop.close() - -``compute()`` is chained to ``print_sum()``: ``print_sum()`` coroutine waits -until ``compute()`` is completed before returning its result. - -Sequence diagram of the example: - -.. image:: tulip_coro.png - :align: center - -The "Task" is created by the :meth:`AbstractEventLoop.run_until_complete` method -when it gets a coroutine object instead of a task. - -The diagram shows the control flow, it does not describe exactly how things -work internally. For example, the sleep coroutine creates an internal future -which uses :meth:`AbstractEventLoop.call_later` to wake up the task in 1 second. - - -InvalidStateError ------------------ - -.. exception:: InvalidStateError - - The operation is not allowed in this state. - - -TimeoutError ------------- - -.. exception:: TimeoutError - - The operation exceeded the given deadline. - -.. note:: - - This exception is different from the builtin :exc:`TimeoutError` exception! - - -Future ------- - -.. class:: Future(\*, loop=None) - - This class is *almost* compatible with :class:`concurrent.futures.Future`. - - Differences: - - - :meth:`result` and :meth:`exception` do not take a timeout argument and - raise an exception when the future isn't done yet. - - - Callbacks registered with :meth:`add_done_callback` are always called - via the event loop's :meth:`~AbstractEventLoop.call_soon`. - - - This class is not compatible with the :func:`~concurrent.futures.wait` and - :func:`~concurrent.futures.as_completed` functions in the - :mod:`concurrent.futures` package. - - This class is :ref:`not thread safe <asyncio-multithreading>`. - - .. method:: cancel() - - Cancel the future and schedule callbacks. - - If the future is already done or cancelled, return ``False``. Otherwise, - change the future's state to cancelled, schedule the callbacks and return - ``True``. - - .. method:: cancelled() - - Return ``True`` if the future was cancelled. - - .. method:: done() - - Return ``True`` if the future is done. - - Done means either that a result / exception are available, or that the - future was cancelled. - - .. method:: result() - - Return the result this future represents. - - If the future has been cancelled, raises :exc:`CancelledError`. If the - future's result isn't yet available, raises :exc:`InvalidStateError`. If - the future is done and has an exception set, this exception is raised. - - .. method:: exception() - - Return the exception that was set on this future. - - The exception (or ``None`` if no exception was set) is returned only if - the future is done. If the future has been cancelled, raises - :exc:`CancelledError`. If the future isn't done yet, raises - :exc:`InvalidStateError`. - .. method:: add_done_callback(callback, *, context=None) +Running Tasks Concurrently +========================== - Add a callback to be run when the future becomes done. +.. function:: gather(\*fs, loop=None, return_exceptions=False) - The *callback* is called with a single argument - the future object. If the - future is already done when this is called, the callback is scheduled - with :meth:`~AbstractEventLoop.call_soon`. + Return a Future aggregating results from the given coroutine objects, + Tasks, or Futures. - An optional keyword-only *context* argument allows specifying a custom - :class:`contextvars.Context` for the *callback* to run in. The current - context is used when no *context* is provided. + If all Tasks/Futures are completed successfully, the result is an + aggregate list of returned values. The result values are in the + order of the original *fs* sequence. - :ref:`Use functools.partial to pass parameters to the callback - <asyncio-pass-keywords>`. For example, - ``fut.add_done_callback(functools.partial(print, "Future:", - flush=True))`` will call ``print("Future:", fut, flush=True)``. + All coroutines in the *fs* list are automatically + scheduled as :class:`Tasks <Task>`. - .. versionchanged:: 3.7 - The *context* keyword-only parameter was added. See :pep:`567` - for more details. + If *return_exceptions* is ``True``, exceptions in the Tasks/Futures + are treated the same as successful results, and gathered in the + result list. Otherwise, the first raised exception is immediately + propagated to the returned Future. - .. method:: remove_done_callback(fn) + If the outer Future is *cancelled*, all submitted Tasks/Futures + (that have not completed yet) are also *cancelled*. - Remove all instances of a callback from the "call when done" list. + If any child is *cancelled*, it is treated as if it raised + :exc:`CancelledError` -- the outer Future is **not** cancelled in + this case. This is to prevent the cancellation of one submitted + Task/Future to cause other Tasks/Futures to be cancelled. - Returns the number of callbacks removed. + All futures must share the same event loop. - .. method:: set_result(result) + .. versionchanged:: 3.7 + If the *gather* itself is cancelled, the cancellation is + propagated regardless of *return_exceptions*. - Mark the future done and set its result. + .. _asyncio_example_gather: - If the future is already done when this method is called, raises - :exc:`InvalidStateError`. + Example:: - .. method:: set_exception(exception) + import asyncio - Mark the future done and set an exception. + async def factorial(name, number): + f = 1 + for i in range(2, number + 1): + print(f"Task {name}: Compute factorial({i})...") + await asyncio.sleep(1) + f *= i + print(f"Task {name}: factorial({number}) = {f}") - If the future is already done when this method is called, raises - :exc:`InvalidStateError`. + async def main(): + await asyncio.gather( + factorial("A", 2), + factorial("B", 3), + factorial("C", 4), + )) - .. method:: get_loop() + asyncio.run(main()) - Return the event loop the future object is bound to. + # Expected output: + # + # Task A: Compute factorial(2)... + # Task B: Compute factorial(2)... + # Task C: Compute factorial(2)... + # Task A: factorial(2) = 2 + # Task B: Compute factorial(3)... + # Task C: Compute factorial(3)... + # Task B: factorial(3) = 6 + # Task C: Compute factorial(4)... + # Task C: factorial(4) = 24 - .. versionadded:: 3.7 +Shielding Tasks From Cancellation +================================= -Example: Future with run_until_complete() -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +.. coroutinefunction:: shield(fut, \*, loop=None) -Example combining a :class:`Future` and a :ref:`coroutine function -<coroutine>`:: + Wait for a Future/Task while protecting it from being cancelled. - import asyncio + *fut* can be a coroutine, a Task, or a Future-like object. If + *fut* is a coroutine it is automatically scheduled as a + :class:`Task`. - async def slow_operation(future): - await asyncio.sleep(1) - future.set_result('Future is done!') + The statement:: - loop = asyncio.get_event_loop() - future = asyncio.Future() - asyncio.ensure_future(slow_operation(future)) - loop.run_until_complete(future) - print(future.result()) - loop.close() + res = await shield(something()) -The coroutine function is responsible for the computation (which takes 1 second) -and it stores the result into the future. The -:meth:`~AbstractEventLoop.run_until_complete` method waits for the completion of -the future. + is equivalent to:: -.. note:: - The :meth:`~AbstractEventLoop.run_until_complete` method uses internally the - :meth:`~Future.add_done_callback` method to be notified when the future is - done. + res = await something() + *except* that if the coroutine containing it is cancelled, the + Task running in ``something()`` is not cancelled. From the point + of view of ``something()``, the cancellation did not happen. + Although its caller is still cancelled, so the "await" expression + still raises a :exc:`CancelledError`. -Example: Future with run_forever() -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + If ``something()`` is cancelled by other means (i.e. from within + itself) that would also cancel ``shield()``. -The previous example can be written differently using the -:meth:`Future.add_done_callback` method to describe explicitly the control -flow:: + If it is desired to completely ignore cancellation (not recommended) + the ``shield()`` function should be combined with a try/except + clause, as follows:: - import asyncio - - async def slow_operation(future): - await asyncio.sleep(1) - future.set_result('Future is done!') + try: + res = await shield(something()) + except CancelledError: + res = None - def got_result(future): - print(future.result()) - loop.stop() - loop = asyncio.get_event_loop() - future = asyncio.Future() - asyncio.ensure_future(slow_operation(future)) - future.add_done_callback(got_result) - try: - loop.run_forever() - finally: - loop.close() +Timeouts +======== -In this example, the future is used to link ``slow_operation()`` to -``got_result()``: when ``slow_operation()`` is done, ``got_result()`` is called -with the result. +.. coroutinefunction:: wait_for(fut, timeout, \*, loop=None) + Wait for a coroutine, Task, or Future to complete with timeout. -Task ----- + *fut* can be a coroutine, a Task, or a Future-like object. If + *fut* is a coroutine it is automatically scheduled as a + :class:`Task`. -.. function:: create_task(coro) + *timeout* can either be ``None`` or a float or int number of seconds + to wait for. If *timeout* is ``None``, block until the future + completes. - Wrap a :ref:`coroutine <coroutine>` *coro* into a task and schedule - its execution. Return the task object. + If a timeout occurs, it cancels the task and raises + :exc:`asyncio.TimeoutError`. - The task is executed in :func:`get_running_loop` context, - :exc:`RuntimeError` is raised if there is no running loop in - current thread. + To avoid the task cancellation, wrap it in :func:`shield`. - .. versionadded:: 3.7 + The function will wait until the future is actually cancelled, + so the total wait time may exceed the *timeout*. -.. class:: Task(coro, \*, loop=None) + If the wait is cancelled, the future *fut* is also cancelled. - A unit for concurrent running of :ref:`coroutines <coroutine>`, - subclass of :class:`Future`. + .. _asyncio_example_waitfor: - A task is responsible for executing a coroutine object in an event loop. If - the wrapped coroutine yields from a future, the task suspends the execution - of the wrapped coroutine and waits for the completion of the future. When - the future is done, the execution of the wrapped coroutine restarts with the - result or the exception of the future. + Example:: - Event loops use cooperative scheduling: an event loop only runs one task at - a time. Other tasks may run in parallel if other event loops are - running in different threads. While a task waits for the completion of a - future, the event loop executes a new task. + async def eternity(): + # Sleep for one hour + await asyncio.sleep(3600) + print('yay!') - The cancellation of a task is different from the cancellation of a - future. Calling :meth:`cancel` will throw a - :exc:`~concurrent.futures.CancelledError` to the wrapped - coroutine. :meth:`~Future.cancelled` only returns ``True`` if the - wrapped coroutine did not catch the - :exc:`~concurrent.futures.CancelledError` exception, or raised a - :exc:`~concurrent.futures.CancelledError` exception. + async def main(): + # Wait for at most 1 second + try: + await asyncio.wait_for(eternity(), timeout=1.0) + except asyncio.TimeoutError: + print('timeout!') - If a pending task is destroyed, the execution of its wrapped :ref:`coroutine - <coroutine>` did not complete. It is probably a bug and a warning is - logged: see :ref:`Pending task destroyed <asyncio-pending-task-destroyed>`. + asyncio.run(main()) - Don't directly create :class:`Task` instances: use the :func:`create_task` - function or the :meth:`AbstractEventLoop.create_task` method. + # Expected output: + # + # timeout! - Tasks support the :mod:`contextvars` module. When a Task - is created it copies the current context and later runs its coroutine - in the copied context. See :pep:`567` for more details. + .. versionchanged:: 3.7 + When *fut* is cancelled due to a timeout, ``wait_for`` waits + for *fut* to be cancelled. Previously, it raised + :exc:`asyncio.TimeoutError` immediately. - This class is :ref:`not thread safe <asyncio-multithreading>`. - .. versionchanged:: 3.7 - Added support for the :mod:`contextvars` module. +Waiting Primitives +================== - .. classmethod:: all_tasks(loop=None) +.. coroutinefunction:: wait(fs, \*, loop=None, timeout=None,\ + return_when=ALL_COMPLETED) - Return a set of all tasks for an event loop. + Wait for a set of coroutines, Tasks, or Futures to complete. - By default all tasks for the current event loop are returned. - If *loop* is ``None``, :func:`get_event_loop` function - is used to get the current loop. + *fs* is a list of coroutines, Futures, and/or Tasks. Coroutines + are automatically scheduled as :class:`Tasks <Task>`. - .. classmethod:: current_task(loop=None) + Returns two sets of Tasks/Futures: ``(done, pending)``. - Return the currently running task in an event loop or ``None``. + *timeout* (a float or int), if specified, can be used to control + the maximum number of seconds to wait before returning. - By default the current task for the current event loop is returned. + Note that this function does not raise :exc:`asyncio.TimeoutError`. + Futures or Tasks that aren't done when the timeout occurs are simply + returned in the second set. - ``None`` is returned when called not in the context of a :class:`Task`. + *return_when* indicates when this function should return. It must + be one of the following constants: - .. method:: cancel() + .. tabularcolumns:: |l|L| - Request that this task cancel itself. + +-----------------------------+----------------------------------------+ + | Constant | Description | + +=============================+========================================+ + | :const:`FIRST_COMPLETED` | The function will return when any | + | | future finishes or is cancelled. | + +-----------------------------+----------------------------------------+ + | :const:`FIRST_EXCEPTION` | The function will return when any | + | | future finishes by raising an | + | | exception. If no future raises an | + | | exception then it is equivalent to | + | | :const:`ALL_COMPLETED`. | + +-----------------------------+----------------------------------------+ + | :const:`ALL_COMPLETED` | The function will return when all | + | | futures finish or are cancelled. | + +-----------------------------+----------------------------------------+ - This arranges for a :exc:`~concurrent.futures.CancelledError` to be - thrown into the wrapped coroutine on the next cycle through the event - loop. The coroutine then has a chance to clean up or even deny the - request using try/except/finally. + Unlike :func:`~asyncio.wait_for`, ``wait()`` does not cancel the + futures when a timeout occurs. - Unlike :meth:`Future.cancel`, this does not guarantee that the task - will be cancelled: the exception might be caught and acted upon, delaying - cancellation of the task or preventing cancellation completely. The task - may also return a value or raise a different exception. + Usage:: - Immediately after this method is called, :meth:`~Future.cancelled` will - not return ``True`` (unless the task was already cancelled). A task will - be marked as cancelled when the wrapped coroutine terminates with a - :exc:`~concurrent.futures.CancelledError` exception (even if - :meth:`cancel` was not called). + done, pending = await asyncio.wait(fs) - .. method:: get_stack(\*, limit=None) - Return the list of stack frames for this task's coroutine. +.. function:: as_completed(fs, \*, loop=None, timeout=None) - If the coroutine is not done, this returns the stack where it is - suspended. If the coroutine has completed successfully or was - cancelled, this returns an empty list. If the coroutine was - terminated by an exception, this returns the list of traceback - frames. + Return an iterator of awaitables which return + :class:`Future` instances. - The frames are always ordered from oldest to newest. + Raises :exc:`asyncio.TimeoutError` if the timeout occurs before + all Futures are done. - The optional limit gives the maximum number of frames to return; by - default all available frames are returned. Its meaning differs depending - on whether a stack or a traceback is returned: the newest frames of a - stack are returned, but the oldest frames of a traceback are returned. - (This matches the behavior of the traceback module.) + Example:: - For reasons beyond our control, only one stack frame is returned for a - suspended coroutine. + for f in as_completed(fs): + result = await f + # ... - .. method:: print_stack(\*, limit=None, file=None) - Print the stack or traceback for this task's coroutine. +Scheduling From Other Threads +============================= - This produces output similar to that of the traceback module, for the - frames retrieved by get_stack(). The limit argument is passed to - get_stack(). The file argument is an I/O stream to which the output - is written; by default output is written to sys.stderr. +.. function:: run_coroutine_threadsafe(coro, loop) + Submit a coroutine to the given event loop. Thread-safe. -Example: Parallel execution of tasks -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + Return a :class:`concurrent.futures.Future` to access the result. -Example executing 3 tasks (A, B, C) in parallel:: + This function is meant to be called from a different OS thread + than the one where the event loop is running. Example:: - import asyncio + # Create a coroutine + coro = asyncio.sleep(1, result=3) - async def factorial(name, number): - f = 1 - for i in range(2, number+1): - print("Task %s: Compute factorial(%s)..." % (name, i)) - await asyncio.sleep(1) - f *= i - print("Task %s: factorial(%s) = %s" % (name, number, f)) + # Submit the coroutine to a given loop + future = asyncio.run_coroutine_threadsafe(coro, loop) - loop = asyncio.get_event_loop() - loop.run_until_complete(asyncio.gather( - factorial("A", 2), - factorial("B", 3), - factorial("C", 4), - )) - loop.close() + # Wait for the result with an optional timeout argument + assert future.result(timeout) == 3 -Output:: + If an exception is raised in the coroutine, the returned Future + will be notified. It can also be used to cancel the task in + the event loop:: - Task A: Compute factorial(2)... - Task B: Compute factorial(2)... - Task C: Compute factorial(2)... - Task A: factorial(2) = 2 - Task B: Compute factorial(3)... - Task C: Compute factorial(3)... - Task B: factorial(3) = 6 - Task C: Compute factorial(4)... - Task C: factorial(4) = 24 + try: + result = future.result(timeout) + except asyncio.TimeoutError: + print('The coroutine took too long, cancelling the task...') + future.cancel() + except Exception as exc: + print('The coroutine raised an exception: {!r}'.format(exc)) + else: + print('The coroutine returned: {!r}'.format(result)) -A task is automatically scheduled for execution when it is created. The event -loop stops when all tasks are done. + See the :ref:`concurrency and multithreading <asyncio-multithreading>` + section of the documentation. + Unlike other asyncio functions this functions requires the *loop* + argument to be passed explicitly. -Task functions --------------- + .. versionadded:: 3.5.1 -.. note:: - In the functions below, the optional *loop* argument allows explicitly setting - the event loop object used by the underlying task or coroutine. If it's - not provided, the default event loop is used. +Introspection +============= .. function:: current_task(loop=None) - Return the current running :class:`Task` instance or ``None``, if + Return the currently running :class:`Task` instance, or ``None`` if no task is running. If *loop* is ``None`` :func:`get_running_loop` is used to get @@ -567,246 +465,236 @@ Task functions .. function:: all_tasks(loop=None) - Return a set of :class:`Task` objects created for the loop. + Return a set of not yet finished :class:`Task` objects run by + the loop. If *loop* is ``None``, :func:`get_running_loop` is used for getting - current loop (contrary to the deprecated :meth:`Task.all_tasks` method - that uses :func:`get_event_loop`.) + current loop. .. versionadded:: 3.7 -.. function:: as_completed(fs, \*, loop=None, timeout=None) - - Return an iterator whose values, when waited for, are :class:`Future` - instances. +Task Object +=========== - Raises :exc:`asyncio.TimeoutError` if the timeout occurs before all Futures - are done. - - Example:: - - for f in as_completed(fs): - result = await f # The 'await' may raise - # Use result - - .. note:: - - The futures ``f`` are not necessarily members of fs. +.. class:: Task(coro, \*, loop=None) -.. function:: ensure_future(coro_or_future, \*, loop=None) + A :class:`Future`-like object that wraps a Python + :ref:`coroutine <coroutine>`. Not thread-safe. - Schedule the execution of a :ref:`coroutine object <coroutine>`: wrap it in - a future. Return a :class:`Task` object. + Tasks are used to run coroutines in event loops. + If a coroutine awaits on a Future, the Task suspends + the execution of the coroutine and waits for the completion + of the Future. When the Future is *done*, the execution of + the wrapped coroutine resumes. - If the argument is a :class:`Future`, it is returned directly. + Event loops use cooperative scheduling: an event loop runs + one Task at a time. While a Task awaits for the completion of a + Future, the event loop runs other Tasks, callbacks, or performs + IO operations. - .. versionadded:: 3.4.4 + Use the high-level :func:`asyncio.create_task` function to create + Tasks, or the low-level :meth:`loop.create_task` or + :func:`ensure_future` functions. Manual instantiation of Tasks + is discouraged. - .. versionchanged:: 3.5.1 - The function accepts any :term:`awaitable` object. + To cancel a running Task use the :meth:`cancel` method. Calling it + will cause the Task to throw a :exc:`CancelledError` exception into + the wrapped coroutine. If a coroutine is awaiting on a Future + object during cancellation, the Future object will be cancelled. - .. note:: + :meth:`cancelled` can be used to check if the Task was cancelled. + The method returns ``True`` if the wrapped coroutine did not + suppress the :exc:`CancelledError` exception and was actually + cancelled. - :func:`create_task` (added in Python 3.7) is the preferable way - for spawning new tasks. + :class:`asyncio.Task` inherits from :class:`Future` all of its + APIs except :meth:`Future.set_result` and + :meth:`Future.set_exception`. - .. seealso:: + Tasks support the :mod:`contextvars` module. When a Task + is created it copies the current context and later runs its + coroutine in the copied context. - The :func:`create_task` function and - :meth:`AbstractEventLoop.create_task` method. + .. versionchanged:: 3.7 + Added support for the :mod:`contextvars` module. -.. function:: wrap_future(future, \*, loop=None) + .. method:: cancel() - Wrap a :class:`concurrent.futures.Future` object in a :class:`Future` - object. + Request the Task to be cancelled. -.. function:: gather(\*coros_or_futures, loop=None, return_exceptions=False) + This arranges for a :exc:`CancelledError` exception to be thrown + into the wrapped coroutine on the next cycle of the event loop. - Return a future aggregating results from the given coroutine objects or - futures. + The coroutine then has a chance to clean up or even deny the + request by suppressing the exception with a :keyword:`try` ... + ... ``except CancelledError`` ... :keyword:`finally` block. + Therefore, unlike :meth:`Future.cancel`, :meth:`Task.cancel` does + not guarantee that the Task will be cancelled, although + suppressing cancellation completely is not common and is actively + discouraged. - All futures must share the same event loop. If all the tasks are done - successfully, the returned future's result is the list of results (in the - order of the original sequence, not necessarily the order of results - arrival). If *return_exceptions* is true, exceptions in the tasks are - treated the same as successful results, and gathered in the result list; - otherwise, the first raised exception will be immediately propagated to the - returned future. + .. _asyncio_example_task_cancel: - Cancellation: if the outer Future is cancelled, all children (that have not - completed yet) are also cancelled. If any child is cancelled, this is - treated as if it raised :exc:`~concurrent.futures.CancelledError` -- the - outer Future is *not* cancelled in this case. (This is to prevent the - cancellation of one child to cause other children to be cancelled.) + The following example illustrates how coroutines can intercept + the cancellation request:: - .. versionchanged:: 3.7.0 - If the *gather* itself is cancelled, the cancellation is propagated - regardless of *return_exceptions*. + async def cancel_me(): + print('cancel_me(): before sleep') -.. function:: iscoroutine(obj) + try: + # Wait for 1 hour + await asyncio.sleep(3600) + except asyncio.CancelledError: + print('cancel_me(): cancel sleep') + raise + finally: + print('cancel_me(): after sleep') - Return ``True`` if *obj* is a :ref:`coroutine object <coroutine>`, - which may be based on a generator or an :keyword:`async def` coroutine. + async def main(): + # Create a "cancel_me" Task + task = asyncio.create_task(cancel_me()) -.. function:: iscoroutinefunction(func) + # Wait for 1 second + await asyncio.sleep(1) - Return ``True`` if *func* is determined to be a :ref:`coroutine function - <coroutine>`, which may be a decorated generator function or an - :keyword:`async def` function. + task.cancel() + try: + await task + except asyncio.CancelledError: + print("main(): cancel_me is cancelled now") -.. function:: run_coroutine_threadsafe(coro, loop) + asyncio.run(main()) - Submit a :ref:`coroutine object <coroutine>` to a given event loop. + # Expected output: + # + # cancel_me(): before sleep + # cancel_me(): cancel sleep + # cancel_me(): after sleep + # main(): cancel_me is cancelled now - Return a :class:`concurrent.futures.Future` to access the result. + .. method:: cancelled() - This function is meant to be called from a different thread than the one - where the event loop is running. Usage:: + Return ``True`` if the Task is *cancelled*. - # Create a coroutine - coro = asyncio.sleep(1, result=3) - # Submit the coroutine to a given loop - future = asyncio.run_coroutine_threadsafe(coro, loop) - # Wait for the result with an optional timeout argument - assert future.result(timeout) == 3 + The Task is *cancelled* when the cancellation was requested with + :meth:`cancel` and the wrapped coroutine propagated the + :exc:`CancelledError` exception thrown into it. - If an exception is raised in the coroutine, the returned future will be - notified. It can also be used to cancel the task in the event loop:: + .. method:: done() - try: - result = future.result(timeout) - except asyncio.TimeoutError: - print('The coroutine took too long, cancelling the task...') - future.cancel() - except Exception as exc: - print('The coroutine raised an exception: {!r}'.format(exc)) - else: - print('The coroutine returned: {!r}'.format(result)) + Return ``True`` if the Task is *done*. - See the :ref:`concurrency and multithreading <asyncio-multithreading>` - section of the documentation. + A Task is *done* when the wrapped coroutine either returned + a value, raised an exception, or the Task was cancelled. - .. note:: + .. method:: get_stack(\*, limit=None) - Unlike other functions from the module, - :func:`run_coroutine_threadsafe` requires the *loop* argument to - be passed explicitly. + Return the list of stack frames for this Task. - .. versionadded:: 3.5.1 + If the wrapped coroutine is not done, this returns the stack + where it is suspended. If the coroutine has completed + successfully or was cancelled, this returns an empty list. + If the coroutine was terminated by an exception, this returns + the list of traceback frames. -.. coroutinefunction:: sleep(delay, result=None, \*, loop=None) + The frames are always ordered from oldest to newest. - Create a :ref:`coroutine <coroutine>` that completes after a given - time (in seconds). If *result* is provided, it is produced to the caller - when the coroutine completes. + Only one stack frame is returned for a suspended coroutine. - The resolution of the sleep depends on the :ref:`granularity of the event - loop <asyncio-delayed-calls>`. + The optional *limit* argument sets the maximum number of frames + to return; by default all available frames are returned. + The ordering of the returned list differs depending on whether + a stack or a traceback is returned: the newest frames of a + stack are returned, but the oldest frames of a traceback are + returned. (This matches the behavior of the traceback module.) - This function is a :ref:`coroutine <coroutine>`. + .. method:: print_stack(\*, limit=None, file=None) -.. coroutinefunction:: shield(arg, \*, loop=None) + Print the stack or traceback for this Task. - Wait for a future, shielding it from cancellation. + This produces output similar to that of the traceback module + for the frames retrieved by :meth:`get_stack`. - The statement:: + The *limit* argument is passed to :meth:`get_stack` directly. - res = await shield(something()) + The *file* argument is an I/O stream to which the output + is written; by default output is written to :data:`sys.stderr`. - is exactly equivalent to the statement:: + .. classmethod:: all_tasks(loop=None) - res = await something() + Return a set of all tasks for an event loop. - *except* that if the coroutine containing it is cancelled, the task running - in ``something()`` is not cancelled. From the point of view of - ``something()``, the cancellation did not happen. But its caller is still - cancelled, so the yield-from expression still raises - :exc:`~concurrent.futures.CancelledError`. Note: If ``something()`` is - cancelled by other means this will still cancel ``shield()``. + By default all tasks for the current event loop are returned. + If *loop* is ``None``, the :func:`get_event_loop` function + is used to get the current loop. - If you want to completely ignore cancellation (not recommended) you can - combine ``shield()`` with a try/except clause, as follows:: + This method is **deprecated** and will be removed in + Python 3.9. Use the :func:`all_tasks` function instead. - try: - res = await shield(something()) - except CancelledError: - res = None + .. classmethod:: current_task(loop=None) + Return the currently running task or ``None``. -.. coroutinefunction:: wait(futures, \*, loop=None, timeout=None,\ - return_when=ALL_COMPLETED) + If *loop* is ``None``, the :func:`get_event_loop` function + is used to get the current loop. - Wait for the Futures and coroutine objects given by the sequence *futures* - to complete. Coroutines will be wrapped in Tasks. Returns two sets of - :class:`Future`: (done, pending). + This method is **deprecated** and will be removed in + Python 3.9. Use the :func:`current_task` function instead. - The sequence *futures* must not be empty. - *timeout* can be used to control the maximum number of seconds to wait before - returning. *timeout* can be an int or float. If *timeout* is not specified - or ``None``, there is no limit to the wait time. +.. _asyncio_generator_based_coro: - *return_when* indicates when this function should return. It must be one of - the following constants of the :mod:`concurrent.futures` module: +Generator-based Coroutines +========================== - .. tabularcolumns:: |l|L| +.. note:: - +-----------------------------+----------------------------------------+ - | Constant | Description | - +=============================+========================================+ - | :const:`FIRST_COMPLETED` | The function will return when any | - | | future finishes or is cancelled. | - +-----------------------------+----------------------------------------+ - | :const:`FIRST_EXCEPTION` | The function will return when any | - | | future finishes by raising an | - | | exception. If no future raises an | - | | exception then it is equivalent to | - | | :const:`ALL_COMPLETED`. | - +-----------------------------+----------------------------------------+ - | :const:`ALL_COMPLETED` | The function will return when all | - | | futures finish or are cancelled. | - +-----------------------------+----------------------------------------+ + Support for generator-based coroutines is **deprecated** and + is scheduled for removal in Python 4.0. - Unlike :func:`~asyncio.wait_for`, ``wait()`` will not cancel the futures - when a timeout occurs. +Generator-based coroutines predate async/await syntax. They are +Python generators that use ``yield from`` expressions to await +on Futures and other coroutines. - This function is a :ref:`coroutine <coroutine>`. +Generator-based coroutines should be decorated with +:func:`@asyncio.coroutine <asyncio.coroutine>`, although this is not +enforced. - Usage:: - done, pending = await asyncio.wait(fs) +.. decorator:: coroutine - .. note:: + Decorator to mark generator-based coroutines. - This does not raise :exc:`asyncio.TimeoutError`! Futures that aren't done - when the timeout occurs are returned in the second set. + This decorator enables legacy generator-based coroutines to be + compatible with async/await code:: + @asyncio.coroutine + def old_style_coroutine(): + yield from asyncio.sleep(1) -.. coroutinefunction:: wait_for(fut, timeout, \*, loop=None) + async def main(): + await old_style_coroutine() - Wait for the single :class:`Future` or :ref:`coroutine object <coroutine>` - to complete with timeout. If *timeout* is ``None``, block until the future - completes. + This decorator is **deprecated** and is scheduled for removal in + Python 4.0. - Coroutine will be wrapped in :class:`Task`. + This decorator should not be used for :keyword:`async def` + coroutines. - Returns result of the Future or coroutine. When a timeout occurs, it - cancels the task and raises :exc:`asyncio.TimeoutError`. To avoid the task - cancellation, wrap it in :func:`shield`. The function will wait until - the future is actually cancelled, so the total wait time may exceed - the *timeout*. +.. function:: iscoroutine(obj) - If the wait is cancelled, the future *fut* is also cancelled. + Return ``True`` if *obj* is a :ref:`coroutine object <coroutine>`. - This function is a :ref:`coroutine <coroutine>`, usage:: + This method is different from :func:`inspect.iscoroutine` because + it returns ``True`` for generator-based coroutines decorated with + :func:`@coroutine <coroutine>`. - result = await asyncio.wait_for(fut, 60.0) +.. function:: iscoroutinefunction(func) - .. versionchanged:: 3.4.3 - If the wait is cancelled, the future *fut* is now also cancelled. + Return ``True`` if *func* is a :ref:`coroutine function + <coroutine>`. - .. versionchanged:: 3.7 - When *fut* is cancelled due to a timeout, ``wait_for`` now waits - for *fut* to be cancelled. Previously, - it raised :exc:`~asyncio.TimeoutError` immediately. + This method is different from :func:`inspect.iscoroutinefunction` + because it returns ``True`` for generator-based coroutine functions + decorated with :func:`@coroutine <coroutine>`. |