| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
| |
This is a bit stupid and requires tests to be adjusted (which is done in
this commit as well).
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
This is done to not print confusing errors in situations where one of
the bundles or base URLs works and everything is fine. If all of them
are failing though, we now print all the error messages in order to give
users an overview over what might have gone wrong.
|
| | |
|
| |
|
|
| |
Otherwise we have to deal with git's not liking colons in filenames.
|
| | |
|
| |
|
|
|
|
| |
This is needed because if there are any connection errors during
bootstrap, we do not want that to affect the build. It is mostly a hack
that should not be necessary. But it is and that's life.
|
| | |
|
| | |
|
| |
|
|
| |
This order makes most sense in all regards.
|
| | |
|
| |
|
|
| |
This is to improve logging, error reporting and readability of the code.
|
| | |
|
| |
|
|
|
|
|
| |
If the SourceManager is told to not update repositories, then it should
always assume that we have the repo cached for at least one of the base
URLs. In that case it should never try to download a bundle from a
bundle server and just skip all bundles.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
| |
This is the WRONG thing to do, in the long run, but for now, without this I can't
ever get the bootstrap build to finish more than once in a while, since git or
gitorious or something keeps failing during the git updates.
|
| |
|
|
| |
We don't have wget in the bootstrap chroot, so we need to deal with this.
|
| |
|
|
|
|
| |
For some reason, "git bundle unbundle" doesn't work for me, but "git clone" does.
Then, adding a remote fails, because the cloned bundle already has one, so now
we'll use "git remote set-url" and "git remote update".
|
| |
|
|
|
|
| |
Previously, urljoin wouldn't add the bundle server, since the pathname at the
server contains a colon. We now use a simpler way of joining (lowly string
catenation) which should always work.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit introduces the following new classes:
morphlib.git.Submodules:
* takes a parent repo treeish
* parses the .gitmodule file into morphlib.git.Submodule objects
* provides iterator/container functionality for submodules
morphlib.git.Submodule:
* represents a single entry in a .gitmodules file
* stores a Treeish for the corresponding repository
In addition to this, the exception classes InvalidTreeish and
SourceNotFound where renamed to InvalidReferenceError and
RepositoryUpdateError. Several new exception classes were added for when
resolving submodules fails.
The SourceManager now resolves the Submodules and Submodule objects for
the submodules of a Treeish in SourceManager.get_treeish() and also
takes care of caching submodule repositories whenever necessary.
|
| | |
|
| |
|
|
|
|
| |
Also, avoid raising a KeyError when reaching the end of the base URL
traversal without having had success in updating a repository. Instead,
raise a SourceNotFound error.
|
| |
|
|
|
|
| |
This is useful because we still have a mix of sudo and fakeroot, which
can cause permission issues in repos. This way, we can work around this
problem in some situations.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit introduces four new classes:
BuildController:
* takes an app instance and a tempdir
* allows to add BuildWorker objects
* provides a build() method that takes a set of blobs and
a build order that is then built by assigning work to the
build workers as needed
* the build() method takes care of polling the workers for
their state, moving them between busy and idle states
reliably, collect and print their output in a non-confusing
order, and makes sure to wait for all workers to finish
before processing the next group in the build order.
* at this point, when waiting for one or more workers to
become idle to assign them another blob to build, the
controller always picks the worker that has been idling
for the longest period of time. this can be changed later.
BuildWorker:
* base class for all worker classes
* takes a name and an app instance
* has a idle_since datetime property
* provides a build() method that takes a Blob object and builds
it in whatever way the subclasses implement it
* provides a check_complete(timeout) method that checks whether
the worker has finished building the blob yet or not
LocalBuildWorker:
* worker class for local builds that don't go through SSH
* it uses morphlib.execute.Execute to run morph in a child process
in build()
* at the moment, this class executes "./morph" instead of "morph"
as it assumes the user to run morph from its source tree. obviously,
this will have to be fixed later.
RemoteBuildWorker:
* doesn't implement anything yet, will be used for distributing
work to other machines running morph via SSH
Notes:
* At the moment, there is a degree of undesired redundancy when
building a stratum in a worker, as this will cause the worker to
rebuild all its dependencies. This will have to be fixed as it is
avoidable and wastes a lot of time and processing power.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
This affects pretty much every part of morph, so this might not be fully
working and stable yet.
This commit also introduces the "update-gits" command that can be used
to update all cached repositories from the list of base URLs. The tree
walk when resolving the Treeish objects in Builder.get_cache_id() is a
bit similar to what we do in BuildDependencyGraph, maybe we can merge
that one day.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
