summaryrefslogtreecommitdiff
path: root/distbuild
Commit message (Collapse)AuthorAgeFilesLines
...
* Don't log a cache response till we're sureRichard Ipsum2014-04-161-1/+2
| | | | | | The controller should check the response event is actually the response it was waiting for before logging that there was a cache response
* Merge branch 'baserock/richardipsum/distbuild_improve_annotation3'Richard Ipsum2014-04-153-55/+75
|\ | | | | | | | | | | | | | | | | | | Conflicts: distbuild/build_controller.py Reviewed by: Lars Wirzenius Daniel Silverstone Sam Thursfield
| * Annotation using single post requestRichard Ipsum2014-04-111-53/+70
| | | | | | | | We now get the state of all artifacts with a single request.
| * Set body and headers in messageRichard Ipsum2014-04-111-1/+2
| | | | | | | | body and headers must now be specified for http-request message.
| * Add body and headers to http-request messageRichard Ipsum2014-04-081-0/+2
| |
* | Fix lines longer than 80 characters (my fault)Sam Thursfield2014-04-143-5/+8
| |
* | distbuild: Consistently name all maybe_ message callbacksSam Thursfield2014-04-142-25/+25
| | | | | | | | | | | | | | | | | | There are cases where a state machine handles an event but stays in the same state. A callback is registered which filters messages further before possibly taking action. There have been bugs caused by this pattern being incorrectly implemented (where the callback is expected to filter the message, but a transition takes place anyway). Hopefully a consistent naming convention will make the pattern clearer.
* | distbuild: Make some transitions more specificSam Thursfield2014-04-142-23/+18
| | | | | | | | | | | | There is always one BuildController object per InitiatorConnection. By coupling the objects slightly closer we can simplify some transitions in BuildController.
* | distbuild: Fix possible 'one user disconnecting breaks all builds' issueSam Thursfield2014-04-141-2/+2
| | | | | | | | | | This is similar to the issue fixed by commit c38b77bed86acc8b90f253ce354f3ecf98e475e7.
* | distbuild: Fix 'one build failure causes all concurrent builds to hang'Sam Thursfield2014-04-141-8/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Each BuildController instance sets itself up to receive all messages from all workers (via the WorkerConnection instances). In the case of a build failure, all BuildController objects would transition to 'None' state (causing them all to be destroyed) on any WorkerBuildFailed message. This meant that if any one build failed on a distbuild network: - the user whose build actually failed would receive the error messages correctly - any concurrent users would see no further build messages from the controller, making it look like their builds had hung. Ctrl+C from the 'hung' users would still be correctly handled by the controller, as their InitiatorConnection instance would still be alive.
* | distbuild: Clarify InitiatorConnection docstringSam Thursfield2014-04-141-3/+6
| |
* | distbuild: Label state machine transition tablesSam Thursfield2014-04-1410-0/+11
| | | | | | | | Makes it easier to see what they mean at a glance.
* | distbuild: worker-build failure does not indicate an error in the controllerSam Thursfield2014-04-141-1/+1
| |
* | distbuild: Log IPs of initiators and workers in more placesSam Thursfield2014-04-102-32/+31
| | | | | | | | This makes tracking multiple builds through one log file a bit easier.
* | distbuild: Improve logging of connections and objectsSam Thursfield2014-04-109-7/+93
| | | | | | | | | | | | | | | | | | | | New DistbuildSocket class that wraps socket.socket(), providing a descriptive repr() handler showing where the socket is connected, and providing a couple of helper methods for fetching local and remote endpoint names. This commit also adds a descriptive repr() handler to a few other objects (mostly giving socket connection details).
* | distbuild: Remove debug logging from serialise-artifact codeSam Thursfield2014-04-101-7/+0
| | | | | | | | | | We still log all messages sent to workers, which include the output of the serialise-artifact code in full. There's no need for these status messages.
* | distbuild: Disable extra debugging by defaultSam Thursfield2014-04-102-2/+2
|/
* Fix _maybe_abort in distbuildRichard Ipsum2014-03-281-1/+1
|
* Transfer all chunk artifacts after they are builtRichard Ipsum2014-03-261-10/+15
| | | | | We want to be able to transfer all source artifacts in a single transaction
* Fix build controllerRichard Ipsum2014-03-261-6/+34
| | | | | | | | | | Whenever the controller finds a source artifact it wants to build, it changes its state to BUILDING. We build all a chunk's source artifacts in one go. So for any chunk artifact, we change the state of all chunk artifacts that are built from the same source to BUILDING
* Make serialise work with artifact splittingRichard Ipsum2014-03-261-26/+118
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Serialisation was simple when we only had 1 artifact per source. However, to allow smaller systems, we need artifact splitting to produce multiple artifacts per chunk source. So now the new serialisation format has a separate list of artifacts and sources, rather than the Source being generated from the artifact's serialisation. Python's id() function is used to encode the references between the various Sources and Artifacts, these are replaced with a reference to the new object after deserialisation. Previously the cache-key was used, but this is no longer sufficient to uniquely identify an Artifact. The resultant build graph after deserialisation is a little different to what went in: Strata end up with a different Source per Artifact, so it _is_ a 1 to 1 mapping, as opposed to Chunks, where it's many to 1. We serialise strata and chunks differently because stratum artifacts from the same source can have different dependencies, for example core-devel can have different dependencies to core-runtime. Without intervention we would serialise core-devel and core-devel's dependencies without including core-runtime's dependencies. To solve this we've decided to encode stratum artifacts completely indepedently: each stratum artifact has its own source. This is safe because stratum artifacts can be constructed independently, as opposed to Chunks where all the Artifacts for a Source are produced together. This is a little hacky in its current form, but it simplifies matters later in distbuild with regards to how it handles expressing that every Artifact that shares a Source is built together. Arguably, this should be the output of producing the build graph anyway, since it more helpfully represents which Artifacts are built together than checking the morphology kind all the time, but more assumptions need checking in morph before it's safe to make this change across the whole of the morph codebase.
* Don't print worker in initiatorRichard Ipsum2014-03-261-2/+0
| | | | Worker name is not sent in message
* Add the distbuild libsRichard Ipsum2014-03-2128-0/+3886
View Lorry Controller Status