summaryrefslogtreecommitdiff
path: root/distbuild
Commit message (Collapse)AuthorAgeFilesLines
* Merge remote-tracking branch 'origin/sam/distbuild-logs-2'Sam Thursfield2014-05-141-0/+1
|\ | | | | | | | | Reviewed-By: Richard Ipsum <richard.ipsum@codethink.co.uk> Reviewed-By: Lars Wirzenius <lars.wirzenius@codethink.co.uk>
| * distbuild: Include .build-log when copying chunk artifacts to the TroveSam Thursfield2014-05-141-0/+1
| | | | | | | | | | Users need to be able to see logs of all builds, not just those that failed.
* | Make distbuild put worker logs onto stdoutRichard Ipsum2014-05-141-0/+1
|/
* Add _ExecFailed eventRichard Ipsum2014-05-061-4/+24
| | | | To cancel jobs cleanly we need to know when a job has failed.
* Use messages to update job stateRichard Ipsum2014-05-061-3/+24
|
* Add cancelling to WorkerBuildSchedulerRichard Ipsum2014-05-061-13/+93
|
* Remove unused import and methodRichard Ipsum2014-05-061-3/+0
| | | | add_initiator() isn't necessary given lists have a remove method.
* Rename vars in dequeue_eventsRichard Ipsum2014-05-061-2/+2
|
* Add cancelling to build controllerRichard Ipsum2014-05-051-7/+19
|
* Remove route mapRichard Ipsum2014-04-241-1/+0
|
* Merge branch 'baserock/richardipsum/distbuild_jobs'Richard Ipsum2014-04-246-103/+262
|\ | | | | | | | | | | | | Reviewed by: Sam Thursfield Richard Maw Lars Wirzenius
| * Import all the thingsRichard Ipsum2014-04-231-1/+4
| |
| * Initiator: Handle step-already-started messageRichard Ipsum2014-04-231-0/+7
| |
| * InitiatorConnection: Handle _step_already_startedRichard Ipsum2014-04-231-1/+21
| |
| * Check whether request id is in the event's id listRichard Ipsum2014-04-231-6/+8
| | | | | | | | | | | | The contents of the message has changed for several events, event messages that need to be sent to several initiators have a list of ids instead of a single id.
| * Add handling for new messages to BuildControllerRichard Ipsum2014-04-231-1/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | There are two new messages: WorkerBuildStepAlreadyStarted tells the initiator that the artifact they want to build is already being built, e.g. 'eglibc-misc is already building on 172.17.1.37:3434' WorkerBuildWaiting tells the initiator that the artifact they want to build can't be built yet because there aren't any workers free, e.g. 'Ready to build eglibc-misc: waiting for a worker to become available'
| * Add new step-already-started message to protocolRichard Ipsum2014-04-231-0/+5
| |
| * WorkerConnection: _maybe_handle_helper_resultRichard Ipsum2014-04-231-9/+4
| | | | | | | | | | | | | | Put our _exec_response_msg into WorkerBuildFinished event, it's essentially the same as _finished_msg, just a different name Get our artifact's cache key from the job
| * WorkerConnection: _request_cachingRichard Ipsum2014-04-231-9/+6
| | | | | | | | Now we just get everything from the job object
| * WorkerConection: _handle_exec_responseRichard Ipsum2014-04-231-9/+8
| | | | | | | | | | | | | | | | | | | | | | The exec_response_msg also needs to be sent to a number of initiators, so we give it a list of ids not just one. The exec_response_msg will be sent to the controller once the artifacts have been cached successfully. There's no longer any need to use a route map to retrieve the id of the initiator, since this is stored with the job
| * WorkerConnection: _handle_exec_outputRichard Ipsum2014-04-231-2/+2
| | | | | | | | | | msg now contains a list of initiator ids rather than a single one, since BuiltOutput needs to be sent to a number of initiators
| * WorkerBuildQueuer: Use job's artifact and idRichard Ipsum2014-04-231-14/+17
| | | | | | | | | | | | Each job is given a unique id, so we don't need to generate an id for each exec request this means we can remove use of route map since we can use the job's id for the exec request
| * Remove cancelRichard Ipsum2014-04-231-6/+1
| | | | | | | | This method no longer works, we will replace it soon.
| * Change event names backRichard Ipsum2014-04-231-3/+3
| | | | | | | | | | The name change from BuildFailed -> JobFailed etc was unintentionally merged into master, undo this.
| * WorkerConnection: misc attributesRichard Ipsum2014-04-231-0/+6
| | | | | | | | | | | | _job is the job this worker is carrying out _exec_response_msg will contain the response the worker sends back to us when it finishes the build.
| * WorkerBuildQueuer: replace request queue with jobsRichard Ipsum2014-04-231-25/+67
| |
| * Add Jobs and Job classesRichard Ipsum2014-04-231-10/+49
| |
| * Make WorkerBuildCaching carry a list of idsRichard Ipsum2014-04-231-5/+2
| | | | | | | | We need to be able to send this message to a number of initiators
| * Add new build messages to worker build schedulerRichard Ipsum2014-04-231-2/+12
| |
* | Remove duplicated function _artifact_filenameRichard Ipsum2014-04-161-8/+2
|/
* Add flag for debugging state of build graphRichard Ipsum2014-04-161-8/+10
| | | | | | Most of the time knowing the state of the build graph isn't that useful for debugging distbuild, but it may be useful in some situations.
* Don't log a cache response till we're sureRichard Ipsum2014-04-161-1/+2
| | | | | | The controller should check the response event is actually the response it was waiting for before logging that there was a cache response
* Merge branch 'baserock/richardipsum/distbuild_improve_annotation3'Richard Ipsum2014-04-153-55/+75
|\ | | | | | | | | | | | | | | | | | | Conflicts: distbuild/build_controller.py Reviewed by: Lars Wirzenius Daniel Silverstone Sam Thursfield
| * Annotation using single post requestRichard Ipsum2014-04-111-53/+70
| | | | | | | | We now get the state of all artifacts with a single request.
| * Set body and headers in messageRichard Ipsum2014-04-111-1/+2
| | | | | | | | body and headers must now be specified for http-request message.
| * Add body and headers to http-request messageRichard Ipsum2014-04-081-0/+2
| |
* | Fix lines longer than 80 characters (my fault)Sam Thursfield2014-04-143-5/+8
| |
* | distbuild: Consistently name all maybe_ message callbacksSam Thursfield2014-04-142-25/+25
| | | | | | | | | | | | | | | | | | There are cases where a state machine handles an event but stays in the same state. A callback is registered which filters messages further before possibly taking action. There have been bugs caused by this pattern being incorrectly implemented (where the callback is expected to filter the message, but a transition takes place anyway). Hopefully a consistent naming convention will make the pattern clearer.
* | distbuild: Make some transitions more specificSam Thursfield2014-04-142-23/+18
| | | | | | | | | | | | There is always one BuildController object per InitiatorConnection. By coupling the objects slightly closer we can simplify some transitions in BuildController.
* | distbuild: Fix possible 'one user disconnecting breaks all builds' issueSam Thursfield2014-04-141-2/+2
| | | | | | | | | | This is similar to the issue fixed by commit c38b77bed86acc8b90f253ce354f3ecf98e475e7.
* | distbuild: Fix 'one build failure causes all concurrent builds to hang'Sam Thursfield2014-04-141-8/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Each BuildController instance sets itself up to receive all messages from all workers (via the WorkerConnection instances). In the case of a build failure, all BuildController objects would transition to 'None' state (causing them all to be destroyed) on any WorkerBuildFailed message. This meant that if any one build failed on a distbuild network: - the user whose build actually failed would receive the error messages correctly - any concurrent users would see no further build messages from the controller, making it look like their builds had hung. Ctrl+C from the 'hung' users would still be correctly handled by the controller, as their InitiatorConnection instance would still be alive.
* | distbuild: Clarify InitiatorConnection docstringSam Thursfield2014-04-141-3/+6
| |
* | distbuild: Label state machine transition tablesSam Thursfield2014-04-1410-0/+11
| | | | | | | | Makes it easier to see what they mean at a glance.
* | distbuild: worker-build failure does not indicate an error in the controllerSam Thursfield2014-04-141-1/+1
| |
* | distbuild: Log IPs of initiators and workers in more placesSam Thursfield2014-04-102-32/+31
| | | | | | | | This makes tracking multiple builds through one log file a bit easier.
* | distbuild: Improve logging of connections and objectsSam Thursfield2014-04-109-7/+93
| | | | | | | | | | | | | | | | | | | | New DistbuildSocket class that wraps socket.socket(), providing a descriptive repr() handler showing where the socket is connected, and providing a couple of helper methods for fetching local and remote endpoint names. This commit also adds a descriptive repr() handler to a few other objects (mostly giving socket connection details).
* | distbuild: Remove debug logging from serialise-artifact codeSam Thursfield2014-04-101-7/+0
| | | | | | | | | | We still log all messages sent to workers, which include the output of the serialise-artifact code in full. There's no need for these status messages.
* | distbuild: Disable extra debugging by defaultSam Thursfield2014-04-102-2/+2
|/
* Fix _maybe_abort in distbuildRichard Ipsum2014-03-281-1/+1
|
* Transfer all chunk artifacts after they are builtRichard Ipsum2014-03-261-10/+15
| | | | | We want to be able to transfer all source artifacts in a single transaction
View Lorry Controller Status