summaryrefslogtreecommitdiff
path: root/distbuild/serialise.py
Commit message (Collapse)AuthorAgeFilesLines
* distbuild: Remove debug logging from serialise-artifact codeSam Thursfield2014-04-101-7/+0
| | | | | We still log all messages sent to workers, which include the output of the serialise-artifact code in full. There's no need for these status messages.
* Make serialise work with artifact splittingRichard Ipsum2014-03-261-26/+118
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Serialisation was simple when we only had 1 artifact per source. However, to allow smaller systems, we need artifact splitting to produce multiple artifacts per chunk source. So now the new serialisation format has a separate list of artifacts and sources, rather than the Source being generated from the artifact's serialisation. Python's id() function is used to encode the references between the various Sources and Artifacts, these are replaced with a reference to the new object after deserialisation. Previously the cache-key was used, but this is no longer sufficient to uniquely identify an Artifact. The resultant build graph after deserialisation is a little different to what went in: Strata end up with a different Source per Artifact, so it _is_ a 1 to 1 mapping, as opposed to Chunks, where it's many to 1. We serialise strata and chunks differently because stratum artifacts from the same source can have different dependencies, for example core-devel can have different dependencies to core-runtime. Without intervention we would serialise core-devel and core-devel's dependencies without including core-runtime's dependencies. To solve this we've decided to encode stratum artifacts completely indepedently: each stratum artifact has its own source. This is safe because stratum artifacts can be constructed independently, as opposed to Chunks where all the Artifacts for a Source are produced together. This is a little hacky in its current form, but it simplifies matters later in distbuild with regards to how it handles expressing that every Artifact that shares a Source is built together. Arguably, this should be the output of producing the build graph anyway, since it more helpfully represents which Artifacts are built together than checking the morphology kind all the time, but more assumptions need checking in morph before it's safe to make this change across the whole of the morph codebase.
* Add the distbuild libsRichard Ipsum2014-03-211-0/+166