| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
| |
This fixes an issue with distbuild-status and distbuild-cancel crashing
due to their appropriate Initiator classes not handling 'build-failed'
messages
Change-Id: Ia35c8e14a30e3a9bdea1e44f7726181db75dfbe5
|
|
|
|
| |
Change-Id: If3f6abdaab6518e77da911bfe1952c8ffe4bda34
|
|
|
|
| |
Change-Id: Ia644ddfaa5138f0ad459099cf26f51b545a9f9ca
|
|
|
|
| |
Change-Id: If74c97ccd81aa4d92ef247d2be59282f9552d4a1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The `morph anchor`, `morph build-morphology` and a potential `morph
diff` command would all benefit from having a unified way to parse the
argv for the systems it must operate on, especially in the case of the
potential `morph diff`, which needs to be able to handle being given two
sets of systems.
`morph anchor` may make use of it now by passing the list of systems to
the Source resolver, but `morph build-morphology` would have to iterate
over the systems and graph each independently.
Change-Id: I91ab4764ffca3aa16f144f89f68f37cc21b9f643
|
|
|
|
|
|
|
| |
Remove extra job set line as self._current_job no longer exists
in worker_build_scheduler.py
Change-Id: I8849742587f11f83ebba64f48eaf97fac83e6589
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The existing Source resolution code handles resolution of multiple
systems sufficiently.
It is not appropriate to then take this source pool and attempt to
create a build graph from it though, as the logical structure of the
input of what we want to build, and the logical structure of what we
will produce are conflated in the Source object.
If we do not intend to create a build graph from the Source Pool we
generate, then it is an appropriate data structure that may be used to
analyse changes in the input to a build.
Change-Id: If8e4a726f16f8aca000e59ecbbeb7d926cc08391
|
|
|
|
|
|
|
|
|
|
| |
Passing a single ref is still accepted, but if you have multiple refs
you need to check from the same repository, it is more appropriate to do
it in one call to get_updated_repo, as otherwise there will be
unnecessary output about it not needing to be updated in multiple
places.
Change-Id: I194d7c0e3e84c4917518ba37672b508505c71b8e
|
|
|
|
| |
Change-Id: I0e0b8d352eb4ef1ab6c50e0ba0162263d9bac09d
|
|
|
|
|
|
|
|
|
|
| |
Previously it would not attempt to make commits it needed locally
available if the commit was available on the remote repo cache.
Now it will do the update if the commit is not available locally, and
will obey --no-git-update.
Change-Id: I80f1e351ce334641e2ef733fa4c9a6ab967f9b67
|
|
|
|
|
|
|
| |
This is useful for representing lists of items in status or exception
messages.
Change-Id: I530eecdc311ac77fca9922dab063f550ea810c06
|
|
|
|
| |
Change-Id: I60e808d7f42890dac2e1470a994e1a31a92401e7
|
|
|
|
|
|
| |
The * should not be in quotes.
Change-Id: Ieebdc7532ba1bff5ba9742f72440ed00b0c0de2a
|
|
|
|
|
|
|
|
|
|
|
|
| |
If a stratum build-depends on itself, the build graph calculation
gets stuck in an infinite loop as it adds the same stratum to the
queue of morphologies to inspect over and over again.
This commit causes MorphologyLoader.validate_stratum to raise an
error if a stratum contains itself in it's build-depends, as
depending on itself makes no sense and will cause the above problem.
Change-Id: I76df5b7d63d010ae3b17f72bfa39b273e74279dd
|
|
|
|
|
|
| |
This fixes an error caused by not enough parameters being given
to the InitiatorBuildCommand constructor in distbuild-morphology.
Change-Id: I133bd2f267fd06cfe88a1cbf4711cc79ad00d209
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If the initiator sends an invalid build-request message, it will now
exit with the following sort of error:
ERROR: Failed to build baserock:baserock/definitions
f2d78e9b7221bca65cba53af3f3b50d50d90628f
systems/build-system-x86_64.morph: Invalid build-request message. Check
you are using a supported version of Morph. This distbuild network uses
protocol version 2.
Previously, the controller would log an error to its log file, but it
would not send any response to the initiator so the initiator would
hang forever.
Behaviour is the same as before for the case where the initiator sends a
build-request message with the wrong protocol version: the initiator
will exit with an error message.
Change-Id: I94fdee02bc701d4a679a0261b3c46dbdf14cfcaf
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Commit 807e6a90876c5469d242 changed the behaviour of sysroot.write to
avoid deleting the contents of the sysroot. This was done so if you
accidentally set 'sysroot=/' it wouldn't delete your whole system.
It turns out that SDK deployments like
clusters/sdk-example-cluster.morph depended on the contents of the
directory being deleted. The system
armv7lhf-cross-toolchain-system-x86_64.morph has a bunch of files
installed by the cross-toolchain in
/usr/armv7lhf-baserock-linux-gnueabi/sys-root. Previously sysroot.write
would delete these, but since commit 807e6a90876c5469d242 it would
fail with several errors like:
mv: can't rename '/src/tmp/deployments/usr/armv7l.../sys-root/sbin'
If we use 'cp -a' instead of 'mv' then it is slower to deploy, but there
are no errors.
I am still unsure why files from the cross-toolchain system are
installed and then deleted. Although this patch fixes the immediate
issue, I don't know if it's the right thing to do. It seems better to
not install those files in the first place, if we do not need them.
This commit also removes the check for the sysroot target location being
empty. This doesn't work, because it runs /before/ the system being
deployed is unpacked.
Change-Id: I10671c2f3b2060cfb36f880675b83351c6cdd807
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Although in theory a worker should only ever have one job at once, in
practice this assumption doesn't hold, and can cause serious confusion.
The worker (implemented in the JsonRouter class) will actually queue up
exec-request messages and run the oldest one first. I saw a case where,
due to a build not being correctly cancelled, the
WorkerConnection.current_job attribute got out of sync with what the
worker was actually building. This lead to an error when trying to fetch
the built artifacts, as the controller tried to fetch artifacts for
something that wasn't actually built yet, and everything got stuck.
To prevent this from happening, we either need to remove the
exec-request queue in the worker-daemon process, or make the
WorkerConnection class cope with multiple jobs at once. The latter seems
like the more robust approach, so I have done that.
Another bug this fixes is the issue where, if the 'Computing build
graph' (serialise-artifact) step of a build completes on the controller
while one of its WorkerConnection objects is waiting for artifacts to
be fetched by the shared cache from the worker, the build hangs. This
would happen because the WorkerConnection assumed that any
HelperResponse message it saw was the result of its request, so would
send a _JobFinished before caching had actually finished if there was
an unrelated HelperResponse received in the meantime. It now checks
the request ID of the HelperResponse before calling the code that is
now in the new _handle_helper_result_for_job() function.
Change-Id: Ia961f333f9dae77405b58c82c99a56e4c43e1628
|
|
|
|
|
|
|
| |
Rather than generating IDs for each job, identify them by what artifact
is going to be built. Artifact cache IDs need to be unique in any case.
Change-Id: I37a0277931c45a8fb6e37ae7c2a6a942ae732fdd
|
|
|
|
|
|
|
| |
This is a bit more comprehensive than the previous approach of using
public instance attributes, and I find it easier to reason about.
Change-Id: I2942ecf53c95e29893dc0982d38aec689ebfa614
|
|
|
|
|
|
|
| |
The intention is to allow workers to use this class for job tracking, in
addition to the controller.
Change-Id: I355861086764476b383266bab7e850af5e05bc54
|
|
|
|
|
|
| |
Handy for log messages.
Change-Id: I4336866c456a6225a6f3ecbfef10dfc7b864ac59
|
|
|
|
|
|
|
|
|
|
|
| |
Previously it would crash with a backtrace if there were unexpected
filenames in the directory. It's still not amazingly robust, but I
don't have time to rewrite the whole thing now.
This code seems to have ignored that cachefs.walkfiles() returns
filenames with a preceeding '/', which I have fixed now.
Change-Id: I98b3094bd6c82b26984513ee81a1eab9bf253a34
|
|
|
|
|
|
|
| |
This commit undoes behaviour changes from commit
aa6dfcbb70c03dfeb3f9af02.
Change-Id: Ie677fb9c4e6bcd6edeba2cdd87f4f6125dcae7a4
|
|
|
|
|
|
|
|
|
|
|
|
| |
290483010cfc7945cd4483fadd1d98c3b83efb3 broke morph checkout, which
uses set_fetch_url on a repository that has been cloned, hence has its
origin remote url config already on-disk.
The fix prevents it changing the push_url when the fetch_url is set,
unless it is an unnamed remote, as if the config is on-disk, this
already does the right thing.
Change-Id: I6204f664407bab3d7f8ecf0fcca72f5015dee55e
|
|
|
|
|
|
|
|
|
| |
Commit 84096556ea54d4af236f1fe5f7ccf61c1343016f changed the protocol
without changing the protocol version. Versions of Morph between
that one and this one may hang forever in 'morph distbuild' if trying
to build on an incompatible distbuild network.
Change-Id: I9194657f59a4b4a61a6fde7bd85105b56ca1a78d
|
|
|
|
| |
Change-Id: I77a8a3aab887f5d14a372690502df3fdeba6db10
|
|
|
|
| |
Change-Id: If9d92d7c75b9c4276b69c482c076c6fc1d4ccbbf
|
|
|
|
| |
Change-Id: I1df58c33987597d4aa5a8eb241b4de4ac72fe250
|
|
|
|
|
|
|
| |
"the user gets the repo" does not set exit-morph, so it is not valid to
check whether it exited successfully.
Change-Id: I05e2d5c1919eee6b714269642eb9c39bcf578bbc
|
|
|
|
|
|
|
| |
There's no API to do it in one push yet, but we can send a delete for
all the branches that *did* commit.
Change-Id: I671e9384b84657a3e9034d62818caa0ac0d8de1e
|
|
|
|
| |
Change-Id: I500cb81fd0f133bd9f4e76d46bc0ff8a4f57fe50
|
|
|
|
| |
Change-Id: I5dec13df6c28eeb4e8c83ec41fb4bd119e2eebb1
|
|
|
|
|
|
|
| |
87f8dbefda89bf6cb9e4b88f23a5317b054da0d4 added a method that used
_gitdir, but the patch to change it to gitdir was merged afterwards.
Change-Id: Ibd9bff73a0fe69b3c1c2ff6acd02df6cea4a13de
|
|
|
|
|
|
| |
os.symlink will fail if the origin file/link already exist
Change-Id: I8175c8dce699e55c3e39e35dfd45c0c19b8bd96d
|
|
|
|
| |
Change-Id: Iaab620f3d9ebc037fe024db933b03e8f40ca40a4
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit allows the specification of one or more strata/chunks
in a deployment entry in a cluster morphology to deploy instead of
the full system if --partial is set. These are listed in a
'partial-deploy-components' field in each deployment definition.
The components must be in the system, and this only works for
tarball or sysroot deployments. It SHOULD NOT be used when
deploying production systems, as it has a number of limitations.
Change-Id: I04ac58af57216335d9257f6620d09f18f61ea714
|
|
|
|
| |
Change-Id: I09e9b17ef2e0fb94dbf5a96dca91062d64433add
|
|
|
|
| |
Change-Id: I4e7888cbff2e4708154538f8f0a48aeaa1a8a811
|
|
|
|
|
|
|
|
|
|
| |
Currently, attempting to distbuild a component which is not in
the given system or doesn't exist at all will cause the full
system to be built, rather than an error raised. This is because
the logic which checks that all components were found is
completely nonsensical. This commit makes it actually check the
right thing.
Change-Id: Ide4d7e3fa5f71e433f3a7b7c8c387fe594c92e43
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds a `morph generate-manifest-csv` command which creates
a CSV manifest for a system containing license information,
upstream URL and a guess at the version of each component in a
system.
The existing `morph generate-manifest` is also renamed to
`morph generate-manifest-genivi` to improve the distinction
between the two commands which generate a manifest.
The basis of this commit was done by
Michael Drake <michael.drake@codethink.co.uk>.
Change-Id: I712de11eaa6e1564c5bae5f5f396e5e77484972f
|
|
|
|
| |
Change-Id: I501632c4d7acde06e391df3752275a3afc334ba6
|
|
|
|
|
|
|
|
|
|
| |
If a user had the wrong setting for artifact-cache-server then they
could build a system with 'morph distbuild', then be told it wasn't
built by 'morph deploy'. The logs do not show any info about which
cache server was used, so it looks like a Morph bug. This change should
make such a configuration problem much easier to spot.
Change-Id: I932ca3c5caa46990f9bcd77d75b3862f49e0dfb8
|
|
|
|
|
|
|
| |
commit-tree is usually sufficient, but I have a need to be able to
create a commit directly.
Change-Id: I80ba63eb9601aa1190554bb07522465ffb2cb5d9
|
|
|
|
| |
Change-Id: Ife3a809cbf241d8d5a283d7f010ccb5e4d7ea292
|
|
|
|
|
|
| |
It's useful to be able to use this.
Change-Id: Ib32d27ddb637f1b8ed683bdd8ec1db108529a163
|
|
|
|
| |
Change-Id: I048687305054f08b0d8e01daa1366170b3eec323
|
|
|
|
| |
Change-Id: If1b156a309236956d7a35136db35a8fd0ab3ad71
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Once building starts we close the json machine on the initiator,
but we may have received build progress events between processing
our build-started event and closing the json machine, since there
is not a nice way to tell the different types of build progress
apart (they all use BuildProgress) we will ignore all BuildProgress
messages for now.
A possible fix for this is to introduce GraphProgress messages so that
we can report the building of the graph without reporting other types
of BuildProgress ("Waiting for worker" or "Transferring artifact to cache")
that we're not interested in.
Note that we will still report build failures or build success, so if
there's a mistake in the definitions this will be reported before the
detach can occur, similarly if the system is already built this will
be reported before the detach happens.
Change-Id: Ia006ccfba826d2c91f4dea6c028ecdcb5a2b02d6
|
|
|
|
| |
Change-Id: Icfc3d1aa125196e208d7ac35f43f06c5f5a21ba4
|