| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
| |
It was saying "There is not enough space to build the given element.",
this makes me think the error is associated to a specific element, but
this does not make sense to show up in a cleanup task.
Instead say "There is not enough space to complete the build.", which
should be more clear that even after cleaning up there is not enough
space.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This branch makes the following changes:
* jobs/job.py: No longer stores any interested resource list
Jobs are ephemeral again, they only ever exist while they
are running.
* queues/queue.py: Revert to only handling lists of elements
Elements pass through the queues, Queue.harvest_jobs()
replaces Queue.pop_ready_jobs() and now the Queue stops
creating jobs as soon as there are not enough resources
for the job.
Also removed unused `prepare()` abstract method.
* queues/buildqueue.py: Adapt the part where we launch a job
This part needs to be reworked anyway, just touch it up for
now so that it doesnt break with the surrounding changes.
* jobs/{cachesize,cleanup}job.py: Expose uniform complete callback
Allows the scheduler to manage resource deallocation for these
two job completions as a custom thing, at the same phase
that the Queues take care of their own resource deallocation.
* resources.py: No longer has knowledge of the job
Since jobs are ephemeral, they are not a suitable place
to store the resource identifiers, these must be provided
by the callers wherever needed.
Now the main Resources object is owned by the Scheduler
but shared with Queues, each take care of managing the
resources of the jobs they create through the same
resource API.
* scheduler.py: Reverted to only creating jobs on demand
This changes the flow of the scheduler such that whenever
jobs complete, the queues are interrogated for as many
jobs which can run at the moment but not more; and this
completely removes the waiting list.
For the internal cache management jobs, we handle this
with a little state instead of having a waiting list
and only launch when the resources permit it.
By abolishing the scheduler waiting list and creating jobs
on demand, we fix the order of element processing and consequently
fix issue #712.
|
| |
|
|
| |
This is redundant now that we report it through the JobStatus.
|
| |
|
|
|
|
|
|
|
|
| |
This changes the deepest callback from when a Job completes to
propagate a JobStatus value instead of a simple boolean, and updates
all of the effected code paths which used to receive a boolean
to now handle the JobStatus values.
This further improves the situation for issue #753, as now we avoid
queueing cache size jobs for pull jobs which are skipped.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When queuing the special cache management related cleanup and
cache size jobs, we now treat these jobs as special and do the
following:
* Avoid queueing a cleanup/cache_size job if one is already queued
We just drop redundantly queued jobs here.
* Ensure that jobs of this type only run one at a time
This could have been done with the Resources mechanics,
however as these special jobs have the same properties and
are basically owned by the Scheduler, it seemed more straight
forward to handle the behaviors of these special jobs together.
This fixes issue #753
|
| |
|
|
|
|
|
|
| |
This is not used anywhere outside of the Scheduler, currently
only the Scheduler itself is allowed to queue a job at this level.
If the highlevel business logic for automatic queueing of auxiliary
jobs moves to another location, we can make this public again.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* utils.py:_kill_process_tree(): Ignore NoSuchProcess errors
These are caused because we issue SIGTERM, and if the process
has not exited after a timeout, we kill it.
* _scheduler/jobs/job.py: Stop handling NoSuchProcess errors here
redundantly, they are already ignored.
It seems that we were ignoring it after sleeping when terminating
tasks from the scheduler... but we were not ignoring it when performing
the same pattern in the `Plugin.call()` -> `utils._call()` path, so
we were still getting these exceptions at termination time from host
tool processes launched by source plugins.
|
| |
|
|
| |
Remove unneeded cruft.
|
| |\
| |
| |
| |
| | |
.gitlab-ci.yml: Use same version for all image
See merge request BuildStream/buildstream!1282
|
| |/ |
|
| |\
| |
| |
| |
| | |
.gitlab-ci.yml: Add tests for Fedora 30
See merge request BuildStream/buildstream!1281
|
| |/ |
|
| |\
| |
| |
| |
| | |
[1.2] .gitlab-ci.yml: Remove deprecated fedora-27 and add current stable fedora-29
See merge request BuildStream/buildstream!1276
|
| | | |
|
| |/
|
|
|
|
| |
needed
This will allow tests to pass in fedora28 (python3.6) and fedora29 (python3.7)
|
| |\
| |
| |
| |
| | |
tox.ini: pin [testenv:docs] sphinx dependency to 1.8.5
See merge request BuildStream/buildstream!1277
|
| |/
|
|
|
|
| |
In sphinx 2.0.0 sphinx.apidoc has moved to sphinx.ext.apidoc,
which causes a build import error. Pin the version to 1.8.5
to allow docs to be built until issues can be fixed.
|
| | |
|
| |\
| |
| |
| |
| | |
Tristan/backport update state changes 1.2
See merge request BuildStream/buildstream!1256
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
- Tests that the target is still built even when a workspace is open
on a runtime dependency of a build-only dependency.
- Tests that the target is still built even when a workspace is open
on a runtime dependency of a runtime dependency of a build-only
dependency
This adds the regression test provided by Matthew Yates for issue #919
This test differs from the one committed in master as:
- We have an orthogonal bug in 1.2.x where buildable elements show up
in a waiting state instead of a buildable state
- Some new APIs were used in the test case added in master, adjusted
to use only 1.2 APIs for the test case.
|
| | |
| |
| |
| | |
This is to be consistent will all other queues that follow this setup.
|
| | |
| |
| |
| |
| | |
Since we know when an object's state can change, we don't need
to update the state of all objects for consistency.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Statuses of an element can be changed when:
1) It is pulled
2) It is fetched
3) It is workspaced and it finished building
4) One of its dependencies is tracked
5) One of its dependencies is workspaced and finished building
We can therefore update the statuses at those moments and we don't
need to check all the time.
This reduces considerably the calls to update_states that are done
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
_unique_id is set at instantiation and never modified afterwards.
Since the guildelines for the project is to never modify directly
an object's state, accessing _unique_id as an attribute is safe.
Moreover this will save us some cost of calling functions.
|
| |/
|
|
|
| |
This encapsulates the logic of how we handle ids and the table in the
plugin class itself, making it easier to refactor afterwards
|
| |\
| |
| |
| |
| | |
.gitlab-ci.yml: Update CI to use images from GitLab Registry
See merge request BuildStream/buildstream!1258
|
| |/
|
|
|
|
|
|
|
| |
The testsuite images have been deprecated and removed from Docker Hub.
Instead, we need to use their counterparts from the GitLab Container
Registry.
We also need to mark `systemd-udev` as user-installed so that we can
remove `ostree` etc.
|
| |\
| |
| |
| |
| | |
Removing toplevel Makefile
See merge request BuildStream/buildstream!1242
|
| |/
|
|
|
|
| |
This seems to be erronously added to the toplevel, the Makefile
in the `requirements/` subdirectory is the correct one and is the
latest version.
|
| |\
| |
| |
| |
| | |
Improve error reporting when files are not found (1.2)
See merge request BuildStream/buildstream!1217
|
| | |
| |
| |
| |
| |
| |
| | |
boundaries
These include assertions for the expected provenance in the errors,
protecting against regressions of #947
|
| | |
| |
| |
| |
| | |
These also assert that the provenance of the references to missing
files are reported, guarding for regressions of issue #947
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
appropriate
When a file is missing in a subproject, it is not particularly meaningful
to specify the filesystem path to the elements directory of the subproject,
as this temporary staging directory belongs to BuildStream and not the user.
Instead, when a file is missing in a subproject, specifying the junction name
is more useful.
This fixes an aspect of #947
|
| | |
| |
| |
| | |
This fixes issue #947
|
| |/
|
|
|
|
|
|
|
| |
When printing the provenance, show the fully qualified element name
including the junction prefix in the case that the provenance comes
from a subproject.
This makes much more sensible error reporting when reporting errors
which originate from a subproject.
|
| |
|
|
|
|
| |
freedesktop-sdk/bootstrap/project.conf doesn't exist anymore
This completes e57a654239ad0301481b1f41d1dc9cad96d71a7f
|
| |\
| |
| |
| |
| | |
.gitlab-ci.yml: Use current freedesktop-sdk release: freedesktop-sdk-18.08.28
See merge request BuildStream/buildstream!1220
|
| |/ |
|
| |\
| |
| |
| |
| | |
Coverage optional in 1.2.x
See merge request BuildStream/buildstream!1203
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This change simply makes it possible to run the tests without
coverage and without any coverage related dependencies present
in the tox venvs.
To run tests without coverage, simply choose the python version
and append "-nocover" to the env name, e.g.:
tox -e py36-nocover
|
| | |
| |
| |
| |
| |
| |
| | |
This disables any attempt to use coverage when running tests
directly against installed dependencies with setup.py.
This fixes #916
|
| | | |
|
| | | |
|
| |/
|
|
|
|
|
| |
This scaffolding needs to manually tell coverage to cleanup
when sigterm happens in the process in order to collect
the coverage report, but we need to do this conditionally
in case we are running tests without coverage support.
|
| |\
| |
| |
| |
| | |
Observe fail-on-overlap policy on building element's project
See merge request BuildStream/buildstream!1172
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
This test ensures the overlap failure vs warning policy in one
project only ever affects the artifacts created for the project
which declares it and does not force it's policy onto another
consuming project.
A regression test against issue #926
|
| |/
|
|
|
|
|
|
| |
Instead of observing the policy on the element which introduces
overlapping files, which is still the correct element to observe
for whitelists.
This fixes #926
|
| | |
|
| |\
| |
| |
| |
| | |
buildstream/_artifactcache/cascache.py: Set 0644 rights to pulled files
See merge request BuildStream/buildstream!1132
|
| |/
|
|
| |
This was broken by 9252a18180ce79d70c193768293baa0f0eff9981.
|
