summaryrefslogtreecommitdiff
path: root/buildstream/source.py
Commit message (Collapse)AuthorAgeFilesLines
* Remove shebangs from python filesremove_shebangsGökçen Nurlu2018-06-181-1/+0
| | | | Fixes #424
* doc: Overhaul of page names and titlesTristan Van Berkom2018-06-131-1/+1
| | | | | | | | | | | | | | | | o Giving main pages simple word titles This makes the main page: * About * Installing * Using * Reference * Contributing o Now named all rst files with their parent page name as a prefix. o Also changed some titles to make overall consistent titles.
* doc/source/core_framework.rst: Use a toctree instead of linksTristan Van Berkom2018-06-091-2/+2
|
* Load and save junctioned source refs from/to junction.refsTristan Van Berkom2018-04-251-6/+24
| | | | | | | | | | | | o _projectrefs.py: Additional constructor option to choose the base name o _project.py: Load two ProjectRefs objects, one for the junctions o source.py: Load and save junctioned source refs with the appropriate ProjectRefs object o tests: Updated some tests to expect junctions to be stored in junction.refs This fixes issue #361
* source.py, element.py, _pipeline.py: Streamling preflighting.Tristan Van Berkom2018-04-161-0/+9
| | | | | | | | | | | Instead of having the pipeline preflight all sources separately from elements, have the element preflight it's sources. This is in order to simplify the shared code path for the pipeline and the loader to use for instantiating elements. Also updated tests to expect the new ElementError and SourceError instead of the PipelineError which was raised for preflighting before.
* source.py: Adhere to policy on private symbolsTristan Van Berkom2018-04-141-16/+17
| | | | This is a part of issue #285
* source.py: Enhanced documentationTristan Van Berkom2018-04-131-73/+136
| | | | | Added new Abstract Methods section here, and rearranged sources such that abstract methods are on top.
* element.py, source.py: Cleanup how Source cache keys are calculated.Tristan Van Berkom2018-04-111-10/+3
| | | | | | | | Recently after a refactor we kept the Source adding workspace keys to the source keys because, now clean this up to have the workspace key added directly in the Element cache key calculation. This breaks cache keys.
* element.py, source.py: Element consumes the rest of workspace logic.Tristan Van Berkom2018-04-091-37/+13
| | | | And Source no longer has any __workspace handle.
* element.py, source.py: Removing knowledge of assemble scheduling state from ↵Tristan Van Berkom2018-04-091-33/+0
| | | | | | source And considering the workspace related edge cases in Element instead of Source.
* element.py: Manage scheduled tracking state in ElementTristan Van Berkom2018-04-091-16/+0
| | | | | | | | | | | | | | | This removes the scheduled state of tracking from Sources, as this is really an element wide thing. To be consistent with assembly, now this comes with: o Element._schedule_tracking() o Element._tracking_done() o Element.__tracking_scheduled o Element.__tracking_done Updated the TrackQueue() to call Element._tracking_done() similarly to how we have BuildQueue() call Element._assemble_done().
* element.py: Cache source consistency logic and refactor workspace logic into ↵Tristan Van Berkom2018-04-091-10/+0
| | | | | | | Element Workspaces are now element wide, so consistency edge cases must be handled at the element level instead of the source level.
* _project.py: Adhere to policy on private symbolsTristan Van Berkom2018-04-031-7/+7
| | | | | | | | | | | | | | | And adjust all surrounding sources for changed symbols. Additional details: o Added Project.get_shell_config() to report the shell configuration, instead of making those members all public o Moved assertions about project specified format versions required of plugins out of Plugin.__init__, and into Project.create_element() and Project.create_source(), so we can keep more things private This is a part of issue #285
* _context.py: Adhere to policy on private symbolsTristan Van Berkom2018-04-031-3/+3
| | | | | | And adjust all surrounding sources for changed symbols. This is a part of issue #285
* source.py: Remove unused method Source._del_workspace()Tristan Van Berkom2018-04-021-9/+0
| | | | | | | | | | This was never particularly useful, there is no circumstances under which a workspace needs to be deleted, and a cache key invalidated, in the course of a session. A workspace is deleted only atomically as a part of `bst workspace close`, which does not even load a pipeline anymore, so the pipeline state need not be adjusted in this case.
* Make workspaces use objects instead of pipeline helper methodsTristan Maat2018-03-271-72/+18
|
* source.py: Source._save_ref() should return saved stateTristan Van Berkom2018-03-211-0/+5
| | | | So we can report proper processed status from tracking queues
* source.py and loading: Refactoring of origin handlesTristan Van Berkom2018-03-201-9/+5
| | | | | | | | | | | The Source object previously stored the __origin_node, __origin_toplevel and __origin_filename, this is from a time when we did not hold on to the plugin's Provenance object explicitly. Since this information comes from the same place, let's just use Plugin._get_provenance() to derive these values, instead of redundantly carrying them along separately.
* Fix #248 - Support project.refs in the core.Tristan Van Berkom2018-03-201-2/+152
| | | | | | | | | | | | | | | | | | | | | | | | | This adds a new Source.load_ref() API which is technically optional to implement, projects which make use of a project.refs file must only use source plugins which implement the new load_ref() method. * source.py: Added load_ref() API to load a ref from a specified node. This also adds _load_ref() and _save_ref() wrappers which handle the logistics of when to load and save a ref to which location. This also fixes _set_ref() to apply the ref to the node unconditionally, this must be done independantly of whether the ref actually changed. o Modifications to the loading process such that Source now can have access to the element name and source index. o _pipeline.py: Delegate abstract loading of source refs to Source._load_ref() - Print a summarized warning about redundant source references - Assert that one cannot track cross-junction elements without project.refs. o _scheduler/trackqueue.py: Delegate saving refs to Source._save_ref()
* source.py: Document ref accessors to mandate support for None values.Tristan Van Berkom2018-03-201-3/+12
| | | | | | | This is technically an API break, but will be transparant for the vast majority of the current hand full of source implementations which exist at this time. This is a lesser evil than bloating the API with new methods.
* pylint - dealt with superfluous-parens warningJames Ennis2018-03-141-3/+3
|
* Silence messages from Source cache interrogationTristan Van Berkom2018-03-071-1/+5
| | | | | | | | | | | Source interrogation usually involves calling out to host tools to quickly check if a given ref exists. This has however regressed over time when running `bst build --track`. This patch adds a new context manager to silence the messages, and silences messages while calling `Source.get_consistency()` Fixes #280
* Add a 'sources' field to project.conf to override defaultsJonathan Maw2018-02-191-1/+28
| | | | | Equivalent to the 'elements' field, but slightly different because sources don't have accompanying yaml.
* source.py: fix comment typoAngelos Evripiotis2018-02-131-1/+1
|
* source.py: Fix typo in workspace warningJürg Billeter2018-02-131-1/+1
| | | | Fixes #250
* Raise warning if workspaced source's ref is updatedincremental-buildChandan Singh2018-01-251-0/+4
| | | | | | | In case a source with an open workspace is tracked and it's ref gets updated, BuildStream should inform the user that the new ref will not be picked up so long as the workspace is open. To start using the updated ref, the existing workspace will have to be closed.
* Add support for doing incremental buildsChandan Singh2018-01-251-5/+42
| | | | | | | | | | | | | | | | | | | | | | | | | This functionality is only supported for sources which have an open workspace. When such sources are present, the workspace directory will be mounted directly inside the sandbox. As opposed to the default behavior, which is to copy files inside the sandbox. This will save time when building large projects as only those files will need be re-compiled that have been modified during two consecutive builds (assuming the underlying build system supports such behavior). A few things to note regarding this behavior: - If there are any `configure-commands` present, they will run only once for each open workspace. If an element has multiple workspaces and any one of them is opened/closed, they will be executed again on the next run. But, modifying the contents of a workspace will not trigger the `configure-commands` to be executed on the next run. - Workspaced builds still leverage the cache. So, if no changes are made to the workspace, i.e. no files are modified, then it will not force a rebuild. Fixes #192.
* source.py: Properly handle Source.track() returning NoneJürg Billeter2018-01-181-0/+4
| | | | | | | Source.track() may return None when tracking is not available. Handle this identical to the case where track() returns the current ref. Fixes #201
* Use explicit source state updatesjuerg/source-stateJürg Billeter2018-01-161-17/+17
| | | | | This adds the _update_state() method to the Source class, similar to the corresponding method in the Element class.
* Rename _force_inconsistent() to _schedule_tracking()Jürg Billeter2018-01-161-1/+1
| | | | | _force_inconsistent is too low level. Keep that detail contained in the Source class.
* source.py: Allow specifying detailed message in SourceError()Tristan Van Berkom2018-01-111-3/+4
|
* source.py & element.py: Fixing SourceError() and ElementError() constructors.Tristan Van Berkom2018-01-041-3/+6
| | | | | | | | Recently I added the `reason` member which can be used to set machine readable error reason strings for the purpose of testing. Forgot to add the necessary `*` argument, forcing `reason` to be a keyword-only argument.
* source.py: Raise proper SourceError() when failing to create the staging ↵Tristan Van Berkom2018-01-021-1/+7
| | | | | | | directory This changes the UX to report a better human readable error, which is otherwise a BUG message with stack trace.
* Exceptions refactoringTristan Van Berkom2017-12-291-2/+3
| | | | | | | | | | | | | | | Outline of changes to exceptions: o Now BstError base class has a `domain` and `reason` member, reason members are optional. o All derived error classes now specify their `domain` o For LoadError, LoadErrorReason defines the error's `reason` o Now the scheduler `job` class takes care of sending the error reason and domain back to the main process where the last exception which caused a child task to fail can be discretely stored.
* source: clarify note on why to use this tmpdir()Angelos Evripiotis2017-12-131-3/+4
|
* source.py: Cache the workspace keyAntoine Wacheux2017-12-131-11/+26
| | | | | | In order to avoid multiple traversals of the file system when the workspace key is requested multiple times, it is now cached in the source element. The cache is invalidated if the workspace is deleted or moved.
* source.py: Implement _get_workspace_pathAntoine Wacheux2017-12-121-3/+8
|
* Remove unused importsGökçen Nurlu2017-12-071-4/+2
|
* Switch old-style string formattings to new '.format()'Gökçen Nurlu2017-11-171-5/+5
|
* Change origin to point at source repoDaniel Playle2017-11-111-3/+34
| | | | | | | | | | | | This changes workspaces created with the git source element so that the origin remote points to the source repository of the build element as opposed to the internal repository in the bst cache. This introduces an addition of the init_workspace method in the source API. This method, which defaults to calling stage, is for the setup of the workspace after the creation of the workspace directory. This is a part of issue #53
* Refactor: Move context.py -> _context.pyTristan Van Berkom2017-11-081-1/+1
| | | | | | | | | | Consequently: o Changed Plugin.get_context() to a private Plugin._get_context() accessor. o Updated anything which imports Context to do so from private _context module o Updated docs to exclude the now private Context
* refactoring: Made Project option privateTristan Van Berkom2017-11-061-3/+28
| | | | | | | | | | | | | | This required adding two new APIs to make up for it on the Source o get_project_directory() Added here because elements should not be accessing external resources, Sources needed for local files and GPG keys and such o translate_url() Used by sources to mish-mash the project aliases and create real urls.
* Refactoring: Renamed _BstError -> BstErrorTristan Van Berkom2017-11-061-2/+2
| | | | | | Base class for exceptions is now a part of the already private _exceptions module Also moved PipelineError from _pipeline -> _exceptions module
* Refactoring: Move exceptions module to be privateTristan Van Berkom2017-11-061-2/+1
| | | | Hide all of buildstream's internal exceptions from the API surface.
* Refactoring: Move ElementError and SourceError to their respective classes, ↵Tristan Van Berkom2017-11-061-1/+10
| | | | | | | | | | create SandboxError These errors are a part of public facing API, and the exceptions module contains a lot of internal details to be hidden from public API. This move required creating SandboxError because sandbox related code had previously been hijacking the ElementError and raising that.
* source.py: Dont error out if staging directory is not empty.Tristan Van Berkom2017-10-221-3/+0
| | | | | This is the wrong place for the check, it needs to be done once for the toplevel staging directory, not for each source.
* Add warnings when staging to /buildstream/build74-prevent-artifacts-from-containing-files-in-buildstream-buildTristan Maat2017-10-191-1/+4
|
* Add source plugin node validationsTristan Maat2017-09-141-0/+7
|
* source.py: Use utils.sha256sum() when calculating workspace keysTristan Van Berkom2017-09-121-11/+7
|
* source.py: Consider workspaced sources with missing content inconsistentTristan Van Berkom2017-09-041-2/+13
| | | | | | | Avoid trying to calculate cache keys and then running into an error by just considering workspaces with missing content to be inconsistent. This fixes issue #80
View Lorry Controller Status