Merge "Rework the state documentation"

1 files changed, 112 insertions, 78 deletions

diff --git a/doc/source/states.rst b/doc/source/states.rst
index 432e007..805c10c 100644
--- a/doc/source/states.rst
+++ b/doc/source/states.rst
@@ -4,6 +4,15 @@ States
 
 .. _engine states:
 
+.. note::
+
+  The code contains explicit checks during transitions using the models
+  described below. These checks ensure that a transition is valid, if the
+  transition is determined to be invalid the transitioning code will raise
+  a :py:class:`~taskflow.exceptions.InvalidState` exception. This exception
+  being triggered usually means there is some kind of bug in the code or some
+  type of misuse/state violation is occurring, and should be reported as such.
+
 Engine
 ======
 
@@ -41,56 +50,60 @@ Flow
    :align: left
    :alt: Flow state transitions
 
-**PENDING** - A flow starts its life in this state.
-
-**RUNNING** - In this state flow makes a progress, executes and/or reverts its
-atoms.
-
-**SUCCESS** - Once all atoms have finished successfully the flow transitions to
-the SUCCESS state.
+**PENDING** - A flow starts its execution lifecycle in this state (it has no
+state prior to being ran by an engine, since flow(s) are just pattern(s)
+that define the semantics and ordering of their contents and flows gain
+state only when they are executed).
 
-**REVERTED** - The flow transitions to this state when it has been reverted
-successfully after the failure.
+**RUNNING** - In this state the engine running a flow progresses through the
+flow.
 
-**FAILURE** - The flow transitions to this state when it can not be reverted
-after the failure.
+**SUCCESS** - Transitioned to once all of the flows atoms have finished
+successfully.
 
-**SUSPENDING** - In the RUNNING state the flow can be suspended. When this
-happens, flow transitions to the SUSPENDING state immediately. In that state
-the engine running the flow waits for running atoms to finish (since the engine
-can not preempt atoms that are active).
+**REVERTED** - Transitioned to once all of the flows atoms have been reverted
+successfully after a failure.
 
-**SUSPENDED** - When no atoms are running and all results received so far are
-saved, the flow transitions from the SUSPENDING state to SUSPENDED. Also it may
-go to the SUCCESS state if all atoms were in fact ran, or to the REVERTED state
-if the flow was reverting and all atoms were reverted while the engine was
-waiting for running atoms to finish, or to the FAILURE state if atoms were run
-or reverted and some of them failed.
+**FAILURE** - The engine will transition the flow to this state when it can not
+be reverted after a single failure or after multiple failures (greater than
+one failure *may* occur when running in parallel).
 
-**RESUMING** - When the flow is interrupted 'in a hard way' (e.g. server
-crashed), it can be loaded from storage in any state. If the state is not
-PENDING (aka, the flow was never ran) or SUCCESS, FAILURE or REVERTED (in which
-case the flow has already finished), the flow gets set to the RESUMING state
-for the short time period while it is being loaded from backend storage [a
-database, a filesystem...] (this transition is not shown on the diagram). When
-the flow is finally loaded, it goes to the SUSPENDED state.
+**SUSPENDING** - In the ``RUNNING`` state the engine running the flow can be
+suspended. When this happens, the engine attempts to transition the flow
+to the ``SUSPENDING`` state immediately. In that state the engine running the
+flow waits for running atoms to finish (since the engine can not preempt
+atoms that are actively running).
 
-From the SUCCESS, FAILURE or REVERTED states the flow can be ran again (and
-thus it goes back into the RUNNING state). One of the possible use cases for
-this transition is to allow for alteration of a flow or flow details associated
-with a previously ran flow after the flow has finished, and client code wants
-to ensure that each atom from this new (potentially updated) flow has its
-chance to run.
+**SUSPENDED** - When no atoms are running and all results received so far have
+been saved, the engine transitions the flow from the ``SUSPENDING`` state
+to the ``SUSPENDED`` state.
 
 .. note::
 
-  The current code also contains strong checks during each flow state
-  transition using the model described above and raises the
-  :py:class:`~taskflow.exceptions.InvalidState` exception if an invalid
-  transition is attempted. This exception being triggered usually means there
-  is some kind of bug in the engine code or some type of misuse/state violation
-  is occurring, and should be reported as such.
-
+  The engine may transition the flow to the ``SUCCESS`` state (from the
+  ``SUSPENDING`` state) if all atoms were in fact running (and completed)
+  before the suspension request was able to be honored (this is due to the lack
+  of preemption) or to the ``REVERTED`` state if the engine was reverting and
+  all atoms were reverted while the engine was waiting for running atoms to
+  finish or to the ``FAILURE`` state if atoms were running or reverted and
+  some of them had failed.
+
+**RESUMING** - When the engine running a flow is interrupted *'in a
+hard way'* (e.g. server crashed), it can be loaded from storage in *any*
+state (this is required since it is can not be known what state was last
+successfully saved). If the loaded state is not ``PENDING`` (aka, the flow was
+never ran) or ``SUCCESS``, ``FAILURE`` or ``REVERTED`` (in which case the flow
+has already finished), the flow gets set to the ``RESUMING`` state for the
+short time period while it is being loaded from backend storage [a database, a
+filesystem...] (this transition is not shown on the diagram). When the flow is
+finally loaded, it goes to the ``SUSPENDED`` state.
+
+From the ``SUCCESS``, ``FAILURE`` or ``REVERTED`` states the flow can be ran
+again; therefore it is allowable to go back into the ``RUNNING`` state
+immediately. One of the possible use cases for this transition is to allow for
+alteration of a flow or flow details associated with a previously ran flow
+after the flow has finished, and client code wants to ensure that each atom
+from this new (potentially updated) flow has its chance to run.
 
 Task
 ====
@@ -100,63 +113,84 @@ Task
    :align: left
    :alt: Task state transitions
 
-**PENDING** - When a task is added to a flow, it starts in the PENDING state,
-which means it can be executed immediately or waits for all of task it depends
-on to complete.  The task transitions to the PENDING state after it was
-reverted and its flow was restarted or retried.
+**PENDING** - A task starts its execution lifecycle in this state (it has no
+state prior to being ran by an engine, since tasks(s) are just objects that
+represent how to accomplish a piece of work). Once it has been transitioned to
+the ``PENDING`` state by the engine this means it can be executed immediately
+or if needed will wait for all of the atoms it depends on to complete.
 
-**RUNNING** - When flow starts to execute the task, it transitions to the
-RUNNING state, and stays in this state until its
-:py:meth:`execute() <taskflow.task.BaseTask.execute>` method returns.
+.. note::
 
-**SUCCESS** - The task transitions to this state after it was finished
-successfully.
+  A engine running a task also transitions the task to the ``PENDING`` state
+  after it was reverted and its containing flow was restarted or retried.
 
-**FAILURE** - The task transitions to this state after it was finished with
-error. When the flow containing this task is being reverted, all its tasks are
-walked in particular order.
+**RUNNING** - When an engine running the task starts to execute the task, the
+engine will transition the task to the ``RUNNING`` state, and the task will
+stay in this state until the tasks :py:meth:`~taskflow.task.BaseTask.execute`
+method returns.
 
-**REVERTING** - The task transitions to this state when the flow starts to
-revert it and its :py:meth:`revert() <taskflow.task.BaseTask.revert>` method
-is called. Only tasks in the SUCCESS or FAILURE state can be reverted. If this
-method fails (raises exception), the task goes to the FAILURE state.
+**SUCCESS** - The engine running the task transitions the task to this state
+after the task has finished successfully (ie no exception/s were raised during
+execution).
 
-**REVERTED** - A task that has been reverted appears in this state.
+**FAILURE** - The engine running the task transitions the task to this state
+after it has finished with an error.
+
+**REVERTING** - The engine running a task transitions the task to this state
+when the containing flow the engine is running starts to revert and
+its :py:meth:`~taskflow.task.BaseTask.revert` method is called. Only tasks in
+the ``SUCCESS`` or ``FAILURE`` state can be reverted. If this method fails (ie
+raises an exception), the task goes to the ``FAILURE`` state (if it was already
+in the ``FAILURE`` state then this is a no-op).
 
+**REVERTED** - A task that has been reverted appears in this state.
 
 Retry
 =====
 
+.. note::
+
+  A retry has the same states as a task and one additional state.
+
 .. image:: img/retry_states.svg
    :width: 660px
    :align: left
    :alt: Retry state transitions
 
-Retry has the same states as a task and one additional state.
+**PENDING** - A retry starts its execution lifecycle in this state (it has no
+state prior to being ran by an engine, since retry(s) are just objects that
+represent how to retry an associated flow). Once it has been transitioned to
+the ``PENDING`` state by the engine this means it can be executed immediately
+or if needed will wait for all of the atoms it depends on to complete (in the
+retry case the retry object will also be consulted when failures occur in the
+flow that the retry is associated with by consulting its
+:py:meth:`~taskflow.retry.Decider.on_failure` method).
+
+.. note::
 
-**PENDING** - When a retry is added to a flow, it starts in the PENDING state,
-which means it can be executed immediately or waits for all of task it depends
-on to complete.  The retry transitions to the PENDING state after it was
-reverted and its flow was restarted or retried.
+  A engine running a retry also transitions the retry to the ``PENDING`` state
+  after it was reverted and its associated flow was restarted or retried.
 
-**RUNNING** - When flow starts to execute the retry, it transitions to the
-RUNNING state, and stays in this state until its
-:py:meth:`execute() <taskflow.retry.Retry.execute>` method returns.
+**RUNNING** - When a engine starts to execute the retry, the engine
+transitions the retry to the ``RUNNING`` state, and the retry stays in this
+state until its :py:meth:`~taskflow.retry.Retry.execute` method returns.
 
-**SUCCESS** - The retry transitions to this state after it was finished
-successfully.
+**SUCCESS** - The engine running the retry transitions it to this state after
+it was finished successfully (ie no exception/s were raised during
+execution).
 
-**FAILURE** - The retry transitions to this state after it was finished with
-error. When the flow containing this retry is being reverted, all its tasks are
-walked in particular order.
+**FAILURE** - The engine running the retry transitions it to this state after
+it has finished with an error.
 
-**REVERTING** - The retry transitions to this state when the flow starts to
-revert it and its :py:meth:`revert() <taskflow.retry.Retry.revert>` method is
-called. Only retries in SUCCESS or FAILURE state can be reverted. If this
-method fails (raises exception), the retry goes to the FAILURE state.
+**REVERTING** - The engine running the retry transitions to this state when
+the associated flow the engine is running starts to revert it and its
+:py:meth:`~taskflow.retry.Retry.revert` method is called. Only retries
+in ``SUCCESS`` or ``FAILURE`` state can be reverted. If this method fails (ie
+raises an exception), the retry goes to the ``FAILURE`` state (if it was
+already in the ``FAILURE`` state then this is a no-op).
 
 **REVERTED** - A retry that has been reverted appears in this state.
 
-**RETRYING** - If flow that is managed by the current retry was failed and
-reverted, the engine prepares it for the next run and transitions to the
-RETRYING state.
+**RETRYING** - If flow that is associated with the current retry was failed and
+reverted, the engine prepares the flow for the next run and transitions the
+retry to the ``RETRYING`` state.