| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The "unicode fix" worked for the subset of cases relevant, and only
broke distbuild because its tests have not been integrated with ./check,
so the fact that it broke for any string ending with a \ escaped notice,
if you will excuse the pun.
During json.load, the encode option is for specifying the character
encoding of the file or string that is being loaded.
During json.dump, the encode option is for the encoding of `str` keys
and values.
The fact that it worked for the set of cases we cared about is a small
mystery, probably caused by the strings we happened to give it being
valid unicode-escape encoded `str`ings.
A full fix would require either converting all these cases to a
different format, such as YAML, which will handle input data not being
valid Unicode, or pre-processing the data that is passed to `json.dump`
to convert all `str` instances to an appropriately escaped `unicode`,
and converting back on `json.load`, but this is a quick fix to get the
distbuild code working again.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
json only accepts unicode. Various APIs such as file paths and environment
variables allow binary data, so we need to support this properly.
This patch changes every[1] use of json.load or json.dump to escape
non-unicode data strings. This appears exactly as it used to if the
input was valid unicode, if it isn't it will insert \xabcd escapes in
the place of non-unicode data.
When loading back in, if json.load is told to unescape it with
`encoding='unicode-escape'` then it will convert it back correctly.
This change was primarily to support file paths that weren't valid
unicode, where this would choke and die. Now it works, but any tools
that parsed the metadata need to unescape the paths.
[1]: The interface to the remote repo cache uses json data, but I haven't
changes its json.load calls to unescape the data, since the repo
caches haven't been made to escape the data.
|
| | |
|
| | |
|
| |\
| |
| |
| |
| |
| |
| | |
Reviewed by:
Sam Thursfield
Richard Maw
Lars Wirzenius
|
| | |
| |
| |
| |
| |
| | |
The contents of the message has changed for several events,
event messages that need to be sent to several initiators have a list
of ids instead of a single id.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
There are two new messages:
WorkerBuildStepAlreadyStarted tells the initiator that the artifact
they want to build is already being built, e.g.
'eglibc-misc is already building on 172.17.1.37:3434'
WorkerBuildWaiting tells the initiator that the artifact they want
to build can't be built yet because there aren't any workers free, e.g.
'Ready to build eglibc-misc: waiting for a worker to become available'
|
| |/ |
|
| |
|
|
|
|
| |
Most of the time knowing the state of the build
graph isn't that useful for debugging distbuild,
but it may be useful in some situations.
|
| |
|
|
|
|
| |
The controller should check the response event is
actually the response it was waiting for before
logging that there was a cache response
|
| |\
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Conflicts:
distbuild/build_controller.py
Reviewed by:
Lars Wirzenius
Daniel Silverstone
Sam Thursfield
|
| | |
| |
| |
| | |
We now get the state of all artifacts with a single request.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
There are cases where a state machine handles an event but stays in the
same state. A callback is registered which filters messages further
before possibly taking action. There have been bugs caused by this
pattern being incorrectly implemented (where the callback is expected to
filter the message, but a transition takes place anyway). Hopefully a
consistent naming convention will make the pattern clearer.
|
| | |
| |
| |
| |
| |
| | |
There is always one BuildController object per InitiatorConnection.
By coupling the objects slightly closer we can simplify some transitions
in BuildController.
|
| | |
| |
| |
| |
| | |
This is similar to the issue fixed by commit
c38b77bed86acc8b90f253ce354f3ecf98e475e7.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Each BuildController instance sets itself up to receive all messages from
all workers (via the WorkerConnection instances). In the case of a build
failure, all BuildController objects would transition to 'None' state
(causing them all to be destroyed) on any WorkerBuildFailed message.
This meant that if any one build failed on a distbuild network:
- the user whose build actually failed would receive the error
messages correctly
- any concurrent users would see no further build messages from the
controller, making it look like their builds had hung.
Ctrl+C from the 'hung' users would still be correctly handled by the
controller, as their InitiatorConnection instance would still be alive.
|
| | |
| |
| |
| | |
Makes it easier to see what they mean at a glance.
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
New DistbuildSocket class that wraps socket.socket(), providing a
descriptive repr() handler showing where the socket is connected, and
providing a couple of helper methods for fetching local and remote
endpoint names.
This commit also adds a descriptive repr() handler to a few other
objects (mostly giving socket connection details).
|
| |/ |
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
Whenever the controller finds a source artifact
it wants to build, it changes its state to BUILDING.
We build all a chunk's source artifacts in one go.
So for any chunk artifact, we change the state of
all chunk artifacts that are built from the
same source to BUILDING
|
| |
|
