summaryrefslogtreecommitdiff
path: root/taskflow
Commit message (Collapse)AuthorAgeFilesLines
* Merge "Improve docstrings in graph flow to denote exceptions raised"Jenkins2015-08-071-4/+27
|\
| * Improve docstrings in graph flow to denote exceptions raisedJoshua Harlow2015-08-051-4/+27
| | | | | | | | | | | | | | | | | | | | | | Link the existing exceptions mentioned to there sphinx doc about them and also adds a section about when the 'DependencyFailure' exception is raised. Also tweaks the class docstring a little, to make it easier to understand. Change-Id: Ie4b989444c5ad73660cc61c0c3b2b1702b669087
* | Merge "Show intermediary compilation(s) when BLATHER is enabled"Jenkins2015-08-071-13/+16
|\ \ | |/ |/|
| * Show intermediary compilation(s) when BLATHER is enabledJoshua Harlow2015-08-021-13/+16
| | | | | | | | | | | | | | | | | | This information can be useful for analyzing why/what is generated during each intermediary flow/subflow and task compilation call so include showing it when and only when the BLATHER level logging is on. Change-Id: I8e9508b8250533a4830fe78705d867139b1eab36
* | Give the GC more of a break with regard to cycles1.18.0Joshua Harlow2015-08-013-12/+12
|/ | | | | | | | | | | | | We can avoid creating reference cycles relatively easily which will make the GC have to do less to garbage collect these objects so let's just give it a break to start. This is *safe* to do since the runtime components have the same lifetime as the runtime itself and they will never outlive the runtime objects existence (a runtime objects lifetime is directly the same as the engine objects lifetime). Change-Id: I7f1ee91e04f29dd27da1e57a462573e068aee45c
* Merge "Use the action engine '_check' helper method"Jenkins2015-08-011-4/+1
|\
| * Use the action engine '_check' helper methodJoshua Harlow2015-07-251-4/+1
| | | | | | | | Change-Id: I0822bbf1caf28a8fd2b4e914643aca61ae0c7f45
* | Merge "Base class for deciders"Jenkins2015-08-011-16/+29
|\ \
| * | Base class for decidersTimofey Durakov2015-07-311-16/+29
| | | | | | | | | | | | | | | | | | | | | To align decider classes interface new base class introduced in this patch Change-Id: I42c69d3daa89153f1f3f9da32bccaf8d840ab1be
* | | Merge "Avoid adding 1 to a failure (if it gets triggered)"Jenkins2015-07-311-1/+5
|\ \ \
| * | | Avoid adding 1 to a failure (if it gets triggered)Joshua Harlow2015-07-271-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Sometimes the CI gate times out and instead of returning normal results a failure object is returned, so be more careful on adding integers to those objects. Closes-Bug: 1478744 Change-Id: Ibdb9d30266d2a7f3bfeacc39e74cf61b44025a56
* | | | Merge "Replace the tree 'pformat()' recursion with non-recursive variant"Jenkins2015-07-315-46/+467
|\ \ \ \
| * | | | Replace the tree 'pformat()' recursion with non-recursive variantJoshua Harlow2015-07-275-46/+467
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adjusted variant creates the same output but is hopefully easier to understand and follow than the recursive version. This version is also not limited by the python stack limit which is a general good thing to avoid being limited by. It also adds a bunch of tests to make sure the format is as expected under various tree structures. Change-Id: I2ae42c7c1bf72794800929250bcf6ccbe658230b
* | | | | Remove extra runner layer and just use use machine in engineJoshua Harlow2015-07-284-216/+211
| |_|/ / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Just directly use the built machine in the action engine and avoid having another layer of abstraction that does not provide that much value. This makes the code cleaner, and more easy to understand (and so-on). Change-Id: Iae1279098112254338258c1941c15889f1ad1a79
* | | | Merge "Bump futurist and remove waiting code in taskflow"Jenkins2015-07-294-147/+6
|\ \ \ \
| * | | | Bump futurist and remove waiting code in taskflowJoshua Harlow2015-07-254-147/+6
| | |_|/ | |/| | | | | | | | | | Change-Id: Ifc9780aa129a4a2804cead301a519895c2bfc0b5
* | | | Merge "Remove legacy py2.6 backwards logging compat. code"Jenkins2015-07-281-39/+1
|\ \ \ \ | |_|_|/ |/| | |
| * | | Remove legacy py2.6 backwards logging compat. codeJoshua Harlow2015-07-231-39/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | We no longer provide support for py2.6 so we don't need the logging compatibility code to exist anymore. Change-Id: Iaefab67fd8b4e222475d99f57c2c3a7a5ce07d6e
* | | | Merge "Remove no longer used '_was_failure' static method"Jenkins2015-07-271-5/+0
|\ \ \ \ | |_|_|/ |/| | |
| * | | Remove no longer used '_was_failure' static methodJoshua Harlow2015-07-231-5/+0
| |/ / | | | | | | | | | Change-Id: I74765d376cdaa2c23a6aaa4a74517da4e2df7ad8
* | | Merge "Use io.open vs raw open"1.17.0Jenkins2015-07-261-4/+4
|\ \ \ | |_|/ |/| |
| * | Use io.open vs raw openJoshua Harlow2015-07-101-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | The io.open call can take in a encoding so we don't need to read in binary mode, then convert it since it can just do that on our behalf. Change-Id: I0cce2841b40f1566ba07ff95a553cb18ea9059ee
* | | Merge "Add ability to reset an engine via a `reset` method"Jenkins2015-07-254-3/+214
|\ \ \
| * | | Add ability to reset an engine via a `reset` methodJoshua Harlow2015-07-184-3/+214
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If an engines work was previously partially completed and it is desired to reset it (and re-run) so that partially completed or ignored (or other) work inside of it can run again make that possible by exposing and documenting a new `reset` method (and use it internally as well). Change-Id: I47f82010a2108d5d8fd5e42ca9f7e5f165e65488
* | | | Merge "Modify listeners to handle the results now possible from revert()"Jenkins2015-07-252-23/+11
|\ \ \ \
| * | | | Modify listeners to handle the results now possible from revert()Joshua Harlow2015-07-232-23/+11
| | |_|/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now that the REVERT and REVERT_FAILURE states can produce results or failure objects we need to take that into account in listeners that were not expecting those states to produce anything; this change adjusts the built-in listeners so that they now handle these states and the results they produce. Also removes some no longer needed py2.6 code used in the logging listener, as that is not needed anymore since we dropped py2.6 support. Change-Id: I0d0a9759648b2a2f27a97c68e19c7cdb6375a4f2
* | | | Merge "Create and use a serial retry executor"Jenkins2015-07-2512-101/+120
|\ \ \ \
| * | | | Create and use a serial retry executorJoshua Harlow2015-07-2112-101/+120
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To make it easily possible to change the retry atom execution from being in the engine thread this creates a retry executor (which is similar to the task executor) and provide that a serial executor (which it will use to execute with). This makes the retry and task actions closer to being the same and makes the surrounding code that much similar (which makes understanding it easier). Change-Id: I993e938280df3bd97f8076293183ef21989e2dba
* | | | | Merge "Make currently implemented jobs use @functools.total_ordering"Jenkins2015-07-242-3/+28
|\ \ \ \ \ | |_|/ / / |/| | | |
| * | | | Make currently implemented jobs use @functools.total_orderingJoshua Harlow2015-07-082-3/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows jobs to be fully be comparable using the total ordering function to provide the complexity around the various functions needed to achieve/perform comparisons. Also fixes up the various job __eq__ and __lt__ methods to correctly return 'NotImplemented' on unknown types and adds a __hash__ to the redis job (so that it can be used in hashable collections, just like the zookeeper job class). Change-Id: I8820d5cc6b2e7f346ac329f011f41b76fa94b777
* | | | | Fix lack of space between functionsJoshua Harlow2015-07-221-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Somehow this passed through the gate and now it is causing related failures, so fix it so that those other failures will not happen. Change-Id: Idb046b0e4e23af49c947a80cf6f77fef3a9ec0c8
* | | | | Merge "Remove **most** usage of taskflow.utils in examples"Jenkins2015-07-2112-134/+159
|\ \ \ \ \ | |_|/ / / |/| | | |
| * | | | Remove **most** usage of taskflow.utils in examplesJoshua Harlow2015-07-1712-134/+159
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It appears folks are using the taskflow.utils code in there own code-bases (likely taking it from the examples) which we do not want to encourage, so remove the usage of **most** of taskflow.utils code from the examples so that people are less likely to copy/paste/reference it. Change-Id: I0ce3c520de347e3e746e7912aa1366a515458424
* | | | | Merge "Document more of the retry subclasses special keyword arguments"Jenkins2015-07-211-1/+38
|\ \ \ \ \
| * | | | | Document more of the retry subclasses special keyword argumentsJoshua Harlow2015-06-151-1/+38
| | | | | | | | | | | | | | | | | | | | | | | | Change-Id: Iaa3949da61c95ffe697fd80cef3ee8a6febd2a8c
* | | | | | Merge "Remove kazoo hack/fix for issue no longer needed"Jenkins2015-07-211-7/+1
|\ \ \ \ \ \
| * | | | | | Remove kazoo hack/fix for issue no longer neededJoshua Harlow2015-06-291-7/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Depends-On: I5cf30d2952850de140f4bcc8bb3eac100ee8001e Change-Id: Ifdf6ec863a3596b6b5e2e58ea383112484b47c26
* | | | | | | Merge "Use encodeutils for exception -> string function"Jenkins2015-07-214-34/+76
|\ \ \ \ \ \ \
| * | | | | | | Use encodeutils for exception -> string functionJoshua Harlow2015-06-304-34/+76
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The oslo.utils library now provides a better version of this that always returns a unicode exception message, so update our usage to use it (and remove our own local function). This guarantee of unicode also means we have to update a few other places to make sure we get back bytes or unicode as needed. Change-Id: I924380408aaf6d2aec418ceaaf623c75900268f7
* | | | | | | | Merge "Unify the zookeeper/redis jobboard iterators"Jenkins2015-07-213-96/+105
|\ \ \ \ \ \ \ \ | |_|_|_|_|_|/ / |/| | | | | | |
| * | | | | | | Unify the zookeeper/redis jobboard iteratorsJoshua Harlow2015-07-123-96/+105
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To make the zookeeper jobboard and redis jobboard iterjobs and wait functions that much similar have both return iterator objects from the same class iterator. This makes the iterator code and the jobboard code that much easier to follow and understand. Change-Id: Ia772cde881c2631002140e06684521fd42441534
* | | | | | | | Merge "Update 'make_client' kazoo docs and link to them"Jenkins2015-07-173-1/+57
|\ \ \ \ \ \ \ \
| * | | | | | | | Update 'make_client' kazoo docs and link to themJoshua Harlow2015-07-173-1/+57
| |/ / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since the creation of a client is somewhat important and knowing what the options that are transfereed to kazoo are we should explicitly document what keys are and what the values should be. Change-Id: I1a5037b274828190270ea5c402be8b2100306de4
* | | | | | | | Merge "Run the '99_bottles.py' demo at a fast rate when activated"Jenkins2015-07-171-4/+21
|\ \ \ \ \ \ \ \
| * | | | | | | | Run the '99_bottles.py' demo at a fast rate when activatedJoshua Harlow2015-07-101-4/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When this is ran without any arguments just run it locally by starting a producer, then a conductor, then stopping after it finishes the first song request. This allows this example to be ran during unit testing to make sure it functions as expected (with zero return code). Change-Id: I26c210e2c993e770955985c9c779d303eb8c0616
* | | | | | | | | Merge "Address concurrent mutation of sqlalchemy backend"Jenkins2015-07-171-13/+8
|\ \ \ \ \ \ \ \ \
| * | | | | | | | | Address concurrent mutation of sqlalchemy backendJoshua Harlow2015-06-151-13/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In order to prevent a thread from closing a backend while another thread is getting a connection (which can result in an engine being created) stop this kind of concurrent mutation by creating a engine (if it was not user provided) in the constructor. In the close the engine dispose is called (which will according to the docs just create a new pool anyway) so there is no need to recreate the full engine object from its same configuration again. Change-Id: Id1fa3001b3ebbe76bbcdb08ed4add6a9e16ea96b
* | | | | | | | | | Merge "Compile lists of retry/task atoms at runtime compile time"Jenkins2015-07-172-7/+24
|\ \ \ \ \ \ \ \ \ \
| * | | | | | | | | | Compile lists of retry/task atoms at runtime compile timeJoshua Harlow2015-07-092-7/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of recompiling and rebuilding this list every iteration of the ``iterate_retries`` function we can just locally cache this information in the runtime compile function and later just use it directly. Change-Id: I70e8409391d655730da61413300db05b25843350
* | | | | | | | | | | Merge "Just make the compiler object at __init__ time"Jenkins2015-07-171-5/+1
|\ \ \ \ \ \ \ \ \ \ \
View Lorry Controller Status