summaryrefslogtreecommitdiff
path: root/taskflow/engines/action_engine
Commit message (Collapse)AuthorAgeFilesLines
* Merge "Quote string representations"5.0.0Zuul2022-06-131-1/+2
|\
| * Quote string representationsMichael Johnson2022-06-101-1/+2
| | | | | | | | | | | | | | | | networkx 2.8.3 will raise errors if certain characters, such as a colon, in the node names and the string is not quoted. This patch double quotes flow, node, and task string representations to make sure there are not issues with these characters occuring in the names. Story: 2010083 Change-Id: Ib0941cddf14dde5d6b9f97fe0224d6e6f3975226
* | Remove sixTakashi Kajinami2022-05-187-51/+20
|/ | | | | | | | This library no longer supports Python 2, thus usage of six can be removed. This also removes workaround about pickle library used in Python 2 only. Change-Id: I19d298cf0f402d65f0b142dea0bf35cf992332a9
* Fix minor typo in ActionEngine exception messageEric Harney2021-12-021-1/+1
| | | | Change-Id: I66b9937df5c58561792328e08f836dc05927e7f1
* Merge "Update TaskFlow for networkx 2.x"3.8.0Zuul2019-11-156-16/+16
|\
| * Update TaskFlow for networkx 2.xMichael Johnson2019-10-186-16/+16
| | | | | | | | | | | | | | | | | | | | | | The networkx 2.x series has been out for two years now and supports python 3.6 and greater[1]. This patch updates TaskFlow to require a minimum of networkx 2.1. It also updates the code to support recent deprecation expiration introduced in the 2.4 release. [1] https://networkx.github.io/documentation/stable/news.html Change-Id: Ife31d353ba80824ebc63c8b21ee90943badc8da3
* | Fix python3.8 hmac compatibilityEric Harney2019-09-031-1/+2
|/ | | | | | | | | The "digestmod" parameter must be passed in for Python 3.8. md5 was chosen here because this is what was implicitly used for Python < 3.8. Change-Id: I17eca0692c4a89ff90cfa9d4ef3947d2c9ee23e7
* Fix process based executor task proxying-back eventsJoshua Harlow2017-07-111-9/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Let's dive into what the problem is here. First a description of what happens to a task that is to be executed in a external (but local) process via the process executor mechanism. When a task is about to be sent to execute in the external (but local) process its first cloned, this is mainly done so that its notification callbacks can be altered in a safe manner (ie not altering the original task object to do this) and that clone has its notifier emptied out. What replaces the clone's notifier callbacks though is a new object (that has a __call__ method so it looks like just another callback) that will send messages to the parent process (the one that has the engine in it) over a secure(ish) channel whenever the local task triggers its notifier notify() method. This allows for callbacks in the parent process to get triggered because once the messages recieved the original tasks notifier object has its notify() method called (therefore those callbacks do not really know the task they are getting messages from is executing out of process). The issue though is that if the ANY(*) event type is registered due to how it works in the notifier is that if the child/cloned notifier has the ANY event type registered and the cloned task calls notify() with a specific event this will cause the ANY callback (in the clone) to transmit a message *and* it will cause the *specific* event callback to also transmit a message back to the parent process. On the engine process side it will get 2 messages and trigger the callbacks 3 times (twice for the specific event callback because how the local notifier has the ANY callback registered and one more time when the local process also sends the same event based on its registration of the ANY event in the child process). This is not what is expected (the message rcved on the engine process should only trigger one callback to get triggered if the engine process task has no ANY callback registered or two engine process callbacks to get triggered if the engine process task has the ANY callback registered). Closes-Bug: #1537948 Change-Id: I271bf1f23ad73df6c177cf00fd902c4881ba44ae
* python3.0 has deprecated LOG.warnxhzhf2017-03-071-4/+4
| | | | | | | | python3.0 has deprecated LOG.warn https://docs.python.org/3/library/logging.html#logging.warning Closes-Bug: #1650843 Change-Id: Ib8473cbdb84547f385e619fe08ed723b1ee7bcdb
* Protect storage better against external concurrent accessJoshua Harlow2017-01-091-44/+50
| | | | | | | | | Lock down the various state machine action handling functions so that they are ensured correct (and consistent) access to the storage layer when they are modifiying and/or reading it. Change-Id: Ie893a44aa963ab515f19e77f9904f49c843cb4e5
* Rename engine analyzer to be named selectorJoshua Harlow2017-01-035-67/+67
| | | | | | | | | | | | | This moves out the engine next to run (or revert) selection process to a single class that just does this and moves out the common functions the analyzer class provided to the runtime object (which all components can access). This makes it easier to adjust the selection algorithm in different ways. Change-Id: I091c69297a7bff60729791d3ca6c3fae14d6eea5
* Add logging around metadata, ignore tallying + historyJoshua Harlow2016-08-012-24/+47
| | | | | | | | This can be useful to look at for debugging purposes when trying to understand how (and if) a atom is being ignored by a set of deciders. Change-Id: I550a2f7f361e516c255683d725b2f3a8130a0a86
* remove unused LOGji-xuepeng2016-07-102-6/+0
| | | | | | This is to remove unused LOG to keep code clean. Change-Id: Ie44659f3b7af9612f127f7d46bb2c1481b6a634f
* Merge "Use the full 'get_execute_failures' vs the shortname"Jenkins2016-07-081-6/+6
|\
| * Use the full 'get_execute_failures' vs the shortnameJoshua Harlow2016-05-281-6/+6
| | | | | | | | | | | | | | | | The usage of the full name is easier to read and easier to understand in the surronding code so prefer to use it. Change-Id: I1c43704ae7dfdba90217ce9a1c60cef8e2ce8632
* | Fix some misspellings in the function name and descriptionsPablo Iranzo Gómez2016-06-302-5/+5
| | | | | | | | Change-Id: I7e3451feb94b1f25b00c5e7b197bb6b527548306
* | Split revert/execute missing args messagesGreg Hill2016-05-261-9/+12
|/ | | | | | | | | To make it more obvious when a revert method is missing a dependency, we now identify which method was missing the required paramater. Change-Id: I24b0d0b01dbd239b92ba444f6ef44273c5ac25d3 Closes-Bug: 1581616
* Instead of a multiprocessing queue use sockets via asyncoreJoshua Harlow2016-05-243-412/+724
| | | | | | | | | | | | | | | | | | | | | | For a local process based executor usage currently to ensure that task emitted notifications are proxied we use the multi processing library and use its queue concept. This sadly creates a proxy process that gets associated, and this proxy process handles the queue and messages sent to and from it. Instead of doing this we can instead just create a temporary local socket using a random socket and have tasks (which are running in different processes) use that to communicate back any emitted notifications instead (and we can use the asyncore module to handle the emitted notifications since it handles the lower level socket reading, polling and dispatching). To ensure that the socket created is somewhat secure we use a similar process as the multi-processing library uses where we sign all messages with a hmac that uses a one time key that only the main process and the child process know about (and reject any messages that do not validate using this key). Change-Id: Iff9180054bf14495e5667af00ae2fafbdbc23791
* Merge "Allow for specifying green threaded to parallel engine"1.32.0Jenkins2016-05-062-0/+27
|\
| * Allow for specifying green threaded to parallel engineJoshua Harlow2016-05-032-0/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | Currently when a string is passed to the parallel engine it will only know how to create a process or a native thread based executor. The futurist library also supports making a green thread based executor, so support creating it. This will save glance some code that they have to create a executor based on different options (one of those is a green option). Change-Id: I15c164a38b4445d28eb6062aed6c56cce0e0364b
* | Attempt to cancel active futures when suspending is underwayJoshua Harlow2016-05-031-9/+40
|/ | | | | | | | | | | | | | Instead of waiting on futures to complete that may still be in an executors backlog (aka, not even started yet) we can avoid even executing (and waiting on these to finish) by forcing them to be cancelled and handling this when analyzing the futures result (by doing nothing with the cancelled result of a future). This saves some time when doing suspension with executors that have a very small pool of workers that actually run tasks/atoms. Change-Id: Ie79466c44ac6af5ff8936f192ebce4428aa12d98
* Merge "Have the storage class provide a 'change_flow_state' method"Jenkins2016-04-281-13/+9
|\
| * Have the storage class provide a 'change_flow_state' methodJoshua Harlow2015-08-151-13/+9
| | | | | | | | | | | | | | | | | | | | Instead of needing the engine method to lock, fetch, check and then update the flow state it is more appropriate if the storage class does all these actions itself (therefore hiding this change from the engine code, making the engine code that much smaller). Change-Id: I2a289c2bcabe76728fa8eb26265ce168abf81b7c
* | Merge "Ensure that the engine finishes up even under sent-in failures"Jenkins2016-04-281-0/+7
|\ \
| * | Ensure that the engine finishes up even under sent-in failuresJoshua Harlow2016-01-201-0/+7
| | | | | | | | | | | | Change-Id: Ic16c854d285398c688f132697c3bb7e637feb9a8
* | | Merge "Add some basic/initial engine statistics"Jenkins2016-04-282-21/+71
|\ \ \
| * | | Add some basic/initial engine statisticsJoshua Harlow2016-02-012-21/+71
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It can be quite nice to expose a basic set of metrics about the internals of an engine, including the time in each state, and how long the engine is active for and likely more in the future. To start add a engine statistics property and gather some basic timing data and place this data into this property for access (and/or introspection) by users. Part of blueprint gather-engine-statistics Change-Id: Ibc3c78755bd8ae779b52fc4772519f243a521576
* | | | Merge "Allow for revert to have a different argument list from execute"Jenkins2016-04-283-10/+21
|\ \ \ \
| * | | | Allow for revert to have a different argument list from executeGreg Hill2016-02-263-10/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Also allows for people to create Atom's with a different rebind or requires structure for the revert method, if desired. Implements blueprint: seperate-revert-args Change-Id: Ie7d13c8000ef08ff303481d486d1ba1cfbdeea44
* | | | | Add the ability to skip resolving from activatingJoshua Harlow2016-04-042-10/+25
|/ / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It can be useful for debugging a failure to be able to stop when a failure is detected so that the failure can be examined in more detail. This adds the ability to have a 'never_resolve' engine option which will skip such resolution attempts for these types of scenarios. Change-Id: If2386cba970e14c7168a6ff2d4fd4786bcd8e989
* | | | Refactor Atom/BaseTask/Task/Retry class hierarchyGreg Hill2016-02-181-1/+1
|/ / / | | | | | | | | | | | | | | | | | | | | | * Moved common argument mapping logic to Atom * Removed BaseTask * Removed duplicated logic from subclasses Change-Id: I275745adb80cecb0c35c1230eac76436bf3b0157
* | | Merge "Use helper function for post-atom-completion work"Jenkins2016-02-012-38/+51
|\ \ \
| * | | Use helper function for post-atom-completion workJoshua Harlow2016-01-202-38/+51
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | For allowing for future changes (where the future 'add_done_callback' can be used if we deem it to be ok to do this) make the completion of an atom a small helper function that is called (and can be called elsewhere in the future). Change-Id: Ief2746debf496b017331a88c7913d9321aac0774
* | | Merge "Add comment as to why we continue when tallying edge decider nay voters"Jenkins2016-02-011-1/+5
|\ \ \
| * | | Add comment as to why we continue when tallying edge decider nay votersJoshua Harlow2016-01-111-1/+5
| | | | | | | | | | | | | | | | Change-Id: Iaee621e15330979d80285430be97af91934b0357
* | | | Merge "Pass through run timeout in engine run()"Jenkins2016-01-261-2/+8
|\ \ \ \ | |_|/ / |/| | |
| * | | Pass through run timeout in engine run()Joshua Harlow2015-12-031-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It can be quite useful to control the delay/timeout used during waiting periods via the run() method and not requiring the usage of run_iter() to achieve the same thing. This change makes it possible to pass the same timeout used in run_iter() to run() and have it be used in the engine internals. Change-Id: I3798ee14600a2d30e009abf1f273b5edbbe3f5ed
* | | | Merge "Add useful/helpful comment to retry scheduler"Jenkins2016-01-201-0/+3
|\ \ \ \
| * | | | Add useful/helpful comment to retry schedulerJoshua Harlow2016-01-191-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is hopefully useful to understand why this happens in this order. Change-Id: I8ff63417489a8d4bada12acc91dbc6bca69df2ba
* | | | | Some additional engine loggingJoshua Harlow2016-01-132-8/+31
| |_|/ / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | Add more trace logging that can be useful to log when things start to work in the wrong way (which can sometimes happen). Change-Id: I7f36735d29c94e382f790feef283e538a9a6a8ff
* | | | Allow for alterations in decider 'area of influence'Joshua Harlow2016-01-099-242/+558
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Christmas came early. Closes-Bug: #1479466 Change-Id: I931d826690c925f022dbfffe9afb7bf41345b1d0
* | | | Fix wrong usage of iter_utils.unique_seenJoshua Harlow2016-01-092-4/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This wasn't actually using the right data to derive uniqueness, so fix it to actually use the right entry to determine what to skip or what not to skip. Closes-Bug: #1525379 Change-Id: Ic23b6d03877f7714f6a3fb74adac0ba58cd97f0d
* | | | Merge "Use 'match_type' utility function instead of staticmethod"Jenkins2016-01-091-15/+11
|\ \ \ \
| * | | | Use 'match_type' utility function instead of staticmethodJoshua Harlow2015-12-141-15/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The utility method allows for moving the logic of which item compiler handles which type to the main compiler class instead of having it in each per compiler instance. Change-Id: Iec578b846a879cbff40f2c50ce865eb260b073e8
* | | | | Cache atom name -> actions and provide accessor functionJoshua Harlow2015-12-302-21/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of doing repeated type checks in the completer engine internal component/object just use the runtime compilation routine to cache the mapping of atom names to actions and then use an exposed function to fetch the needed action in the completer object as needed. Change-Id: I07161e7956d039cf89d057b8082e12b82adcd82f
* | | | | Move all internal blather usage/calls to trace usage/callsJoshua Harlow2015-12-176-41/+41
| | | | | | | | | | | | | | | | | | | | Change-Id: I415a81d3b6b15b17a9a91cc2a0681c159172a4e1
* | | | | Merge "Change name of misc.ensure_dict to misc.safe_copy_dict"Jenkins2015-12-151-1/+1
|\ \ \ \ \ | |/ / / / |/| | | |
| * | | | Change name of misc.ensure_dict to misc.safe_copy_dictJoshua Harlow2015-12-141-1/+1
| | | | | | | | | | | | | | | | | | | | Change-Id: I2ff3b9bb4d0b8143163e4bf370f10c74e79dbd31
* | | | | Merge "Change engine 'self._check' into a decorator"Jenkins2015-12-141-14/+38
|\ \ \ \ \ | |_|/ / / |/| | | |
| * | | | Change engine 'self._check' into a decoratorJoshua Harlow2015-12-021-14/+38
| | |/ / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of having 'self._check' be called at the stat of every using function it can be made into a decorator and placed on the function instead. This way it is a little more explicit than having it called at the start of each method. This also adds a check to ensure the engine has been validated before it is ran (and the decorator takes into this into account as well). Change-Id: I67ee8fe8629392003f7777d4d4fab918b1a071dd
View Lorry Controller Status