delta/openstack/taskflow.git/taskflow/tests/unit/test_engines.py, branch 4.6.2

Update URLs in documents according to document migration

2017-07-13T04:05:18+00:00

Change-Id: I9ca92fdcec388e02462332e04fe7c1bf8b5f64b8

Fix process based executor task proxying-back events

2017-07-11T03:14:04+00:00

Let's dive into what the problem is here.

First a description of what happens to a task that
is to be executed in a external (but local) process
via the process executor mechanism.

When a task is about to be sent to execute in the
external (but local) process its first cloned, this
is mainly done so that its notification callbacks can
be altered in a safe manner (ie not altering the
original task object to do this) and that clone has
its notifier emptied out.

What replaces the clone's notifier callbacks though
is a new object (that has a __call__ method so it
looks like just another callback) that will send
messages to the parent process (the one that has
the engine in it) over a secure(ish) channel whenever
the local task triggers its notifier notify() method.

This allows for callbacks in the parent process to
get triggered because once the messages recieved the
original tasks notifier object has its notify() method
called (therefore those callbacks do not really know
the task they are getting messages from is executing out
of process).

The issue though is that if the ANY(*) event type is registered
due to how it works in the notifier is that if the child/cloned
notifier has the ANY event type registered and the cloned task
calls notify() with a specific event this will cause the ANY
callback (in the clone) to transmit a message *and* it will
cause the *specific* event callback to also transmit a message
back to the parent process.

On the engine process side it will get 2 messages and trigger
the callbacks 3 times (twice for the specific event callback
because how the local notifier has the ANY callback registered
and one more time when the local process also sends the same
event based on its registration of the ANY event in the child
process).

This is not what is expected (the message rcved on the engine
process should only trigger one callback to get triggered
if the engine process task has no ANY callback registered or two
engine process callbacks to get triggered if the engine process
task has the ANY callback registered).

Closes-Bug: #1537948

Change-Id: I271bf1f23ad73df6c177cf00fd902c4881ba44ae

Replace assertRaisesRegexp with assertRaisesRegex

2017-06-03T06:40:12+00:00

assertRaisesRegexp was renamed to assertRaisesRegex in Py3.2
For more details, please check:
https://docs.python.org/3/library/
unittest.html#unittest.TestCase.assertRaisesRegex

Change-Id: I89cce19e80b04074aab9f49a76c7652acace78b3
Closes-Bug: #1436957

Instead of a multiprocessing queue use sockets via asyncore

2016-05-24T23:16:56+00:00

For a local process based executor usage currently to ensure
that task emitted notifications are proxied we use the multi
processing library and use its queue concept. This sadly creates
a proxy process that gets associated, and this proxy process
handles the queue and messages sent to and from it. Instead of
doing this we can instead just create a temporary local socket
using a random socket and have tasks (which are running in
different processes) use that to communicate back any emitted
notifications instead (and we can use the asyncore module to handle
the emitted notifications since it handles the lower level socket
reading, polling and dispatching).

To ensure that the socket created is somewhat secure we use a
similar process as the multi-processing library uses where we
sign all messages with a hmac that uses a one time key that only
the main process and the child process know about (and reject
any messages that do not validate using this key).

Change-Id: Iff9180054bf14495e5667af00ae2fafbdbc23791

Allow for specifying green threaded to parallel engine

2016-05-03T18:00:51+00:00

Currently when a string is passed to the parallel engine
it will only know how to create a process or a native thread
based executor. The futurist library also supports making
a green thread based executor, so support creating it.

This will save glance some code that they have to create
a executor based on different options (one of those is a
green option).

Change-Id: I15c164a38b4445d28eb6062aed6c56cce0e0364b

Merge "Ensure that the engine finishes up even under sent-in failures"

2016-04-28T20:50:08+00:00

Merge "Use 'addCleanup' instead of 'tearDown' in engine(s) test"

2016-02-03T21:14:07+00:00

Ensure that the engine finishes up even under sent-in failures

2016-01-21T01:11:00+00:00

Change-Id: Ic16c854d285398c688f132697c3bb7e637feb9a8

Allow for alterations in decider 'area of influence'

2016-01-10T06:42:17+00:00

Christmas came early.

Closes-Bug: #1479466

Change-Id: I931d826690c925f022dbfffe9afb7bf41345b1d0

Allow provided flow to be empty

2015-11-20T20:56:51+00:00

If someone really wants to provide a flow to run that is
empty that is there prerogative so it doesn't seem that valuable
to blow up if they do this.

Change-Id: I0ad89b0ade85a64f6ec107e2686454ef6dc97353