summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* pylint: enable arguments-differaevri/pylint_ignore_argsdiffAngelos Evripiotis2019-06-121-3/+0
|
* Ignore existing PyLint 'arguments-differ' warningsAngelos Evripiotis2019-06-1216-26/+26
| | | | | | In order to get the 'arguments-differ' warning enabled sooner rather than later, ignore all existing cases. We'll tackle them in separate work.
* Merge branch 'aevri/supersuper' into 'master'bst-marge-bot2019-06-1211-17/+17
|\ | | | | | | | | Refactor 'super(cls, self)' -> 'super()' See merge request BuildStream/buildstream!1391
| * Refactor 'super(cls, self)' -> 'super()'aevri/supersuperAngelos Evripiotis2019-06-1211-17/+17
|/ | | | | | | | | | For most use-cases with modern Python, it's not necessary to supply the 'type' or 'object-or-type' arguments to super(). Replace all our instances with arg-less super, which is less error-prone. https://docs.python.org/3.5/library/functions.html#super
* Merge branch 'bschubert/fix-overnight' into 'master'bst-marge-bot2019-06-111-1/+2
|\ | | | | | | | | | | | | tests: Build wheel before installing BuildStream in overnight tests Closes #1045 See merge request BuildStream/buildstream!1388
| * tests: Build wheel before installing BuildStream in overnight testsBenjamin Schubert2019-06-101-1/+2
|/ | | | | | | BuildStream now requires setuptools and cython when building the wheel and thus can't be installed directly with pip install --no-index. Building a wheel beforehand allows us to install without dependencies.
* Merge branch 'bschubert/remove-useless-condition' into 'master'bst-marge-bot2019-06-102-2/+2
|\ | | | | | | | | plugins/sources: Remove 'or None' call that will never be true See merge request BuildStream/buildstream!1387
| * plugins/sources: Remove 'or None' call that will never be trueBenjamin Schubert2019-06-102-2/+2
|/ | | | | Since node_get_member has a default value passed in, it is impossible for the method to return 'None'
* Merge branch 'bschubert/optimize-extract-depends-node' into 'master'bst-marge-bot2019-06-075-52/+59
|\ | | | | | | | | rewrite _extract_depends_from_node in Cython and optimize See merge request BuildStream/buildstream!1383
| * _loader/types: Use range() instead of enumerate in extract_depends_from_nodeBenjamin Schubert2019-06-071-2/+2
| | | | | | | | | | range() and access to the list can be optimized better by cython than the enumerate() call.
| * _loader/types: Use an accumulator to speed up _extract_depends_from_nodeBenjamin Schubert2019-06-071-15/+8
| | | | | | | | | | | | | | We need to call this function three times in a row then concatenate the three results. Using an accumulator is slightly faster.
| * _loader/types: add type information on _extract_depends_from_nodeBenjamin Schubert2019-06-071-11/+15
| |
| * _loader/types: Use helper function for extract_depends_from_nodeBenjamin Schubert2019-06-071-17/+30
| | | | | | | | | | | | | | extract_depends_from_node has two different behaviors depending on whether it is calling itself or something else is calling it. Extracting the inner calls to a helper function helps speed up the code and makes it more understandable.
| * _loader: Move extract_depends_from_node from loadelement to typesBenjamin Schubert2019-06-075-52/+49
|/ | | | | | | | | | | extract_depends_from_node is only depending on things declared in types, loadelement doesn't have any dependency on it. It makes more sense to have it in types.pyx. Moreover, this allows us to cythonize the function, reducing its total runtime impact. - _yaml: expose node_del as public api for Cython
* Merge branch 'bschubert/stricter-node-api' into 'master'bst-marge-bot2019-06-075-23/+22
|\ | | | | | | | | Make the Node api stricter, by only accepting nodes See merge request BuildStream/buildstream!1384
| * NEWS: restrict `Plugin.node_items` to be called on `Node`s.bschubert/stricter-node-apiBenjamin Schubert2019-06-072-3/+6
| | | | | | | | | | Also update documentation in plugin for APIs that do accept Node and not `dict`.
| * _yaml: Don't check for none node in __trim_list_provenanceBenjamin Schubert2019-06-071-2/+2
| | | | | | | | | | By being more restrictive when creating new nodes, we can remove checks for not-node values.
| * _yaml: Ensure every element passed to list_copy or node_set is a NodeBenjamin Schubert2019-06-071-4/+6
| | | | | | | | | | This is to restrict the API to always work with nodes in order to sanitize the API
| * _yaml: Restrict parameter of node_items and node_keys to be 'Node'Benjamin Schubert2019-06-073-14/+8
|/ | | | | In order to move to a cleaner Node api, we need to ensure that what we are treating is correctly set up as being Nodes.
* Merge branch 'tpollard/bzrfix' into 'master'bst-marge-bot2019-06-071-0/+1
|\ | | | | | | | | testing/_utils/site.py: Fix missing BZR assignment See merge request BuildStream/buildstream!1385
| * testing/_utils/site.py: Fix missing BZR assignmentTom Pollard2019-06-071-0/+1
|/ | | | | | Ensure that BZR is assigned to None in exception handling when not available on the host. Needed for tests/testutils/repo/bzr.py import.
* Merge branch 'jennis/push_based_pipeline' into 'master'bst-marge-bot2019-06-077-68/+202
|\ | | | | | | | | | | | | Push based pipeline Closes #1002 and #943 See merge request BuildStream/buildstream!1344
| * queue.py: Use heapq for the ready queuejennis/push_based_pipelineJames Ennis2019-06-073-3/+21
| | | | | | | | | | | | | | This patch includes setting a _depth to each element once the pipeline has been sorted. This is necessary as we need to store elements in the heapq sorted by their depth.
| * queue.py: Push-based queuesJames Ennis2019-06-075-55/+90
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Queue.enqueue() and Queue.harvest_jobs() now exhibit push-based behaviour. Although most of the logic from Queue.enqueue() has been moved to Queue._enqueue_element() * QueueStatus.WAIT has been replaced with QueueStatus.PENDING to reflect the new push-based nature of the queues * There now exists a virtual method in Queue: register_pending_element which is used to register am element which is not immediately ready to be processed in the queue with specific callbacks. These callbacks will enqueue the element when called.
| * element.py: Add callbacks to be used upon specific state changesJames Ennis2019-06-071-2/+82
| | | | | | | | | | | | | | This commit introduces three callback functions which will be used to callback to relevant queues upon specific state changes: when an element becomes required, when an element becomes able to query the cache and when an element becomes buildable.
| * element.py: Speed up _buildable()Jürg Billeter2019-06-071-10/+1
| |
| * element.py: Introduce __ready_for_runtime_and_cachedJames Ennis2019-06-071-1/+11
|/ | | | | | | In order to know whether an element is buildable, we need to know whether it's dependencies are cached as well as whether they have been marked as ready for runtime. This patch introduces a second "ready_for_runtime" check.
* Merge branch 'aevri/job_msg_enum' into 'master'bst-marge-bot2019-06-0610-37/+52
|\ | | | | | | | | Use enums for Job-related constants See merge request BuildStream/buildstream!1380
| * Use 'is' when comparing against JobStatusaevri/job_msg_enumAngelos Evripiotis2019-06-069-11/+11
| | | | | | | | | | | | Since JobStatus is an enum, it's clearer to compare using 'is' - equality comparison will fail in the same cases, but might lull folks into thinking that comparison with integer would also work.
| * _scheduler/jobs/job: make JobStatus an enumAngelos Evripiotis2019-06-061-1/+2
| | | | | | | | | | | | | | | | This provides some minor guards against mistakes, and we'll be able to do type-checking later. This does open the possibility of problems if folks mistakenly try to pass off an integer as a JobStatus.
| * _scheduler/jobs/job: use enum for return codesAngelos Evripiotis2019-06-061-14/+18
| |
| * _scheduler/jobs/job: use enum for message typesAngelos Evripiotis2019-06-061-11/+21
|/
* Merge branch 'bschubert/optimize-loader-types' into 'master'bst-marge-bot2019-06-065-21/+35
|\ | | | | | | | | Optimize _loader/types.py See merge request BuildStream/buildstream!1379
| * _loader/types: type values wherever possibleBenjamin Schubert2019-06-061-8/+13
| | | | | | | | | | That way, cython can make better inference on the code and does not need to be too conservative.
| * _loader/types: Make Dependency a cdef classBenjamin Schubert2019-06-061-1/+6
| | | | | | | | | | Moving this class to Cython gives a non-negligeable speedup on 'show' operations.
| * _loader/types: cimport yaml functions for better speedBenjamin Schubert2019-06-064-15/+17
| | | | | | | | | | | | - _yaml: export node_validate function as Cython, as it was not done before. This requires rewriting the function to remove a closure. - Optimize node check by not calling is_node().
| * _loader/types: move to a cython packageBenjamin Schubert2019-06-063-0/+2
| | | | | | | | | | | | Types is a simple module that accounts for a few percent of a basic 'show' operation. Having it cythonized allows us to get better performance without too much wokr
| * _loader/types: use type(x) is str instead of isintanceBenjamin Schubert2019-06-061-1/+1
|/ | | | | We don't expected anything else than `str` or `Node`, so type() should be enough
* Merge branch 'bschubert/pylint-artifactcache' into 'master'bst-marge-bot2019-06-068-14/+34
|\ | | | | | | | | Ensure pylint runs in tests/artifactcache See merge request BuildStream/buildstream!1377
| * tests/artifactcache: add __init__.py file to make it a packageBenjamin Schubert2019-06-061-0/+0
| | | | | | | | | | | | | | | | tests/integration was lacking a __init__.py file, meaning it was not recognized as a python package, and thus, pylint would not check anything inside the directory. Adding __init__.py ensures we have correct checks here.
| * tests/artifactcache/push: remove unused importBenjamin Schubert2019-06-061-1/+0
| |
| * tests/artifactcache/pull: Remove unused variableBenjamin Schubert2019-06-061-1/+0
| |
| * tests/artifactcache/config: Fix pylint issuesBenjamin Schubert2019-06-061-4/+10
| |
| * tests/artifactcache/artifactservice: Fix pylint issuesBenjamin Schubert2019-06-061-3/+1
| |
| * tests/artifactcache: Disable 'unused-import' checks that are pytest fixturesBenjamin Schubert2019-06-066-6/+6
| | | | | | | | | | | | | | Some imports are fixtures, that need to be in the current namespace when pytest runs. However, pylint does not know this. Disabling pylint checks on those imports
| * tests/artifactcache: Silence all 'redefined-outer-name' pylint errorsBenjamin Schubert2019-06-066-0/+18
|/ | | | | | | | | | This is due to pytest fixtures having to be named the same as the test arguments. This is a pre-requisite to enable pylint on this directory We need to do this per file as we can't blanket disable for directories. See upstream issue: https://github.com/PyCQA/pylint/issues/618
* Merge branch 'bschubert/site-consolidation' into 'master'bst-marge-bot2019-06-0642-135/+91
|\ | | | | | | | | Remove tests/testutils/site.py and move everything to buildstream/testing/_utils/site.py See merge request BuildStream/buildstream!1375
| * tests/examples/flatpak-autotools: rename HAVE_OSTREE to HAVE_OSTREE_PLUGINBenjamin Schubert2019-06-061-4/+6
| | | | | | | | | | | | What this checks does is check if the ostree plugin is installed and fully configured. Renaming to HAVE_OSTREE_PLUGIN makes it slightly more explicit
| * tests/testutils/site: Remove completely and migrate HAVE_OSTREE to examplesBenjamin Schubert2019-06-062-10/+7
| | | | | | | | | | | | The only place needing to know about OSTREE is the flatpak-autotools for the documentation. We can therefore move the OSTREE business here and remove completely the site file, ending up with a single one left.
| * tests/testutils/site: Remove unused HAVE_OSTREE_CLI variableBenjamin Schubert2019-06-061-9/+0
| |