summaryrefslogtreecommitdiff
path: root/distbuild/initiator.py
Commit message (Collapse)AuthorAgeFilesLines
* distbuild: When a build finishes, say which worker it was built onSam Thursfield2015-10-071-0/+4
| | | | Change-Id: I493fced8cf2664283923f6f41097ca991d3fc3de
* distbuild: Condense Initiator class to remove unnecessary duplicationLauren Perry2015-05-151-127/+7
| | | | | | | | | Create an InitiatorCommand class that accepts message_type and status_text parameters to be used by the distbuild-list-jobs, distbuild-status and distbuild-cancel commands to send request messages to the distbuild network Change-Id: Ib686dcd7c370d802b612e9aaa1e3df76f0275fae
* Explain how to cancel a distbuildAdam Coldrick2015-05-121-9/+12
| | | | | | | | | | | | | | | | Cancelling a distbuild with ctrl+c no longer cancels the build itself. This commit adds some output explaining what should be done to cancel the build as well as the local process. This commit also fixes a bug where the BuildStarted event would be sent each time a chunk finished building, since it was being sent in _queue_worker_builds. This is fixed by adding a new function to be called when the build graph annotation is complete which sends BuildStarted and then calls _queue_worker_builds, which no longer sends the BuildStarted event. Change-Id: I26ddea2c9080887f449e87004411ddffe4e583b7
* Add GraphProgress messagesRichard Ipsum2015-05-121-0/+17
| | | | | | | Adds distinct message types to give us more flexibility over message handling now that we have multiple initiator types with different requirements. Change-Id: Ib2af8736b83d66ef20a8e37591ca68c9441b6497
* distbuild: Fix protocol version checking for distbuild commandsLauren Perry2015-05-111-0/+14
| | | | | | | | This fixes an issue with distbuild-status and distbuild-cancel crashing due to their appropriate Initiator classes not handling 'build-failed' messages Change-Id: Ia35c8e14a30e3a9bdea1e44f7726181db75dfbe5
* Ignore BuildProgress messagesRichard Ipsum2015-04-291-1/+0
| | | | | | | | | | | | | | | | | | | | Once building starts we close the json machine on the initiator, but we may have received build progress events between processing our build-started event and closing the json machine, since there is not a nice way to tell the different types of build progress apart (they all use BuildProgress) we will ignore all BuildProgress messages for now. A possible fix for this is to introduce GraphProgress messages so that we can report the building of the graph without reporting other types of BuildProgress ("Waiting for worker" or "Transferring artifact to cache") that we're not interested in. Note that we will still report build failures or build success, so if there's a mistake in the definitions this will be reported before the detach can occur, similarly if the system is already built this will be reported before the detach happens. Change-Id: Ia006ccfba826d2c91f4dea6c028ecdcb5a2b02d6
* distbuild: Add distbuild status commandLauren Perry2015-04-291-0/+54
| | | | | | | | | Adds a command to get the status of all recently ran distbuilds for a given server (e.g. Running, Finished, Failed, Cancelled), so as to tell if a build running via distbuild-start has finished or otherwise exited without going through the server's log files Change-Id: I5ce9fe54ae7b1bd8fe3e0d629f615042be8827ed
* distbuild: Add distbuild start and cancel functionalityLauren Perry2015-04-291-4/+116
| | | | | | | | | | | Add command for distbuild-start to build_plugin in morphlib, and create a boolean parameter to inform the initiator whether to disconnect the controller and leave the build running remotely. Add distbuild-cancel command to parse currently-running distbuild build-request IDs and cancel the one matching the given argument Change-Id: I458a5767bb768ceb2b4d8876adf1c86075d452bd
* distbuild: Add protocol version checking for list-jobs commandLauren Perry2015-04-291-2/+12
| | | | | | | | | | | | | | | Currently, the distbuild-list-jobs command will fail if morph is outdated (i.e. protocol version for client and distbuild network don't match); a protocol_version field has been added to the list-jobs request message to fix this. Moved version check outside build-request message to reduce duplication in new functions. Generalised the list-request output to reduce duplication for any further additions that may require a message output. Change-Id: I28e733cbfe8c89e8c11427df5d40ab275abd313c
* distbuild: Don't create a directory for build output until we get someSam Thursfield2015-04-211-9/+17
| | | | | | | | Currently, it leaves around empty directories called build-00, build-01, etc. when you run a distbuild that fails to get as far as building something, which is annoying. Change-Id: Id3466e248c327dedaf973bc2fe22d42e5c5570d4
* distbuild: Add distbuild-list-jobs functionLauren Perry2015-04-171-3/+57
| | | | | | | | | | Add InitiatorListJobs class and list-jobs message template, add distbuild-list-jobs to morph commandlist, send running job information back to initiator, split out handling of build request and list-jobs messages to separate functions and change generating a random integer to UUID for message identification Change-Id: Id02604f2c1201dbc10f6bbd7f501b8ce1ce0deae
* Implement partial distbuildsAdam Coldrick2015-04-021-1/+7
| | | | | | | | | | | | | | In addition to partial builds we also want to be able to do partial distbuilds, and distbuild uses a different codepath. This commit updates the distbuild code to know what to do if a partial build is requested. It only builds up to the latest chunk/stratum that was requested, and displays where to find the artifacts for each of the chunks/strata requested upon completion of the build. The usage is the same as for local builds. Change-Id: I0537f74e2e65c7aefe5e71795f17999e2415fce5
* Use the modern way of the GPL copyright header: URL instead real addressJavier Jardón2015-03-161-2/+1
| | | | Change-Id: I992dc0c1d40f563ade56a833162d409b02be90a0
* Merge branch 'sam/distbuild-build-logs'Sam Thursfield2015-03-111-23/+38
|\ | | | | | | | | Reviewed-By: Adam Coldrick <adam.coldrick@codethink.co.uk> Reviewed-By: Richard Maw <richard.maw@codethink.co.uk>
| * distbuild: Log in build-step-xx.log files when initiator cancels buildSam Thursfield2015-02-181-14/+18
| | | | | | | | | | This makes it easier to spot if an incomplete build was due to the user cancelling, or if it represents a dropped connection or internal error.
| * distbuild: Remove the build-steps messageSam Thursfield2015-02-181-8/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This message was hundreds of kilobytes in size, as it contained a recursive list of dependencies for each artifact in the build graph. It was used in the initiator only to print this message: Build steps in total: 592 This message is now gone. The 'Need to build %d artifacts' build-progress message now indicates the total build steps instead: Need to build 300 artifacts, of 592 total This is a compatible change to the distbuild protocol: old initiators will continue to work as normal with new controllers that don't send the build-steps message.
| * distbuild: Create a new directory to store build logs for each build.Sam Thursfield2015-02-181-1/+20
| | | | | | | | | | | | | | | | | | | | | | | | It gets messy having hundreds of build-step-xx.log files in the current directory, and if two builds are run in parallel from the same directory the logs for a given chunk will be mixed together in one file. Now, a new directory named build-0, build-1, build-2 etc is created for each new build. If the user passes --initiator-step-output-dir the logs will be placed in that directory, instead. This behaviour is the same as before.
* | Add protocol versioning for distbuild systemsLauren Perry2015-02-251-2/+3
|/
* Fix copyright yearsSam Thursfield2015-02-111-1/+1
|
* distbuild: Write name of build worker to build-step-xx.log filesSam Thursfield2015-02-111-15/+33
| | | | | | | Knowing which worker built something is useful for debugging, and right now that information is only present on the initiator's console. It's good to have it in the build-step-xx.log file too so the information doesn't get lost.
* Fix lines longer than 79 charactersSam Thursfield2014-10-291-1/+2
|
* Fix distbuild to allow passing a commit instead of a named ref to be builtSam Thursfield2014-10-271-2/+4
| | | | | | | | | | | The recent changes to the BuildCommand.build() function caused distbuild to break, because I didn't make the same change to the InitiatorBuildCommand.build() function but did change how it was called. This commit adds the ability to have optional fields in distbuild messages. This is used to add an optional 'original_ref' field, which will get passed to `morph serialise-artifact` by new distbuild controllers, and will be ignored by older ones.
* Allow distbuilds to choose where to put logsRichard Maw2014-10-081-1/+7
|
* Fix copyright years of distbuild code.Sam Thursfield2014-09-111-1/+1
|
* Initiator: Handle step-already-started messageRichard Ipsum2014-04-231-0/+7
|
* distbuild: Label state machine transition tablesSam Thursfield2014-04-141-0/+1
| | | | Makes it easier to see what they mean at a glance.
* distbuild: Disable extra debugging by defaultSam Thursfield2014-04-101-1/+1
|
* Don't print worker in initiatorRichard Ipsum2014-03-261-2/+0
| | | | Worker name is not sent in message
* Add the distbuild libsRichard Ipsum2014-03-211-0/+195
View Lorry Controller Status