summaryrefslogtreecommitdiff
path: root/buildstream/source.py
Commit message (Collapse)AuthorAgeFilesLines
...
* source.py: Remove unused method Source._del_workspace()Tristan Van Berkom2018-04-021-9/+0
| | | | | | | | | | This was never particularly useful, there is no circumstances under which a workspace needs to be deleted, and a cache key invalidated, in the course of a session. A workspace is deleted only atomically as a part of `bst workspace close`, which does not even load a pipeline anymore, so the pipeline state need not be adjusted in this case.
* Make workspaces use objects instead of pipeline helper methodsTristan Maat2018-03-271-72/+18
|
* source.py: Source._save_ref() should return saved stateTristan Van Berkom2018-03-211-0/+5
| | | | So we can report proper processed status from tracking queues
* source.py and loading: Refactoring of origin handlesTristan Van Berkom2018-03-201-9/+5
| | | | | | | | | | | The Source object previously stored the __origin_node, __origin_toplevel and __origin_filename, this is from a time when we did not hold on to the plugin's Provenance object explicitly. Since this information comes from the same place, let's just use Plugin._get_provenance() to derive these values, instead of redundantly carrying them along separately.
* Fix #248 - Support project.refs in the core.Tristan Van Berkom2018-03-201-2/+152
| | | | | | | | | | | | | | | | | | | | | | | | | This adds a new Source.load_ref() API which is technically optional to implement, projects which make use of a project.refs file must only use source plugins which implement the new load_ref() method. * source.py: Added load_ref() API to load a ref from a specified node. This also adds _load_ref() and _save_ref() wrappers which handle the logistics of when to load and save a ref to which location. This also fixes _set_ref() to apply the ref to the node unconditionally, this must be done independantly of whether the ref actually changed. o Modifications to the loading process such that Source now can have access to the element name and source index. o _pipeline.py: Delegate abstract loading of source refs to Source._load_ref() - Print a summarized warning about redundant source references - Assert that one cannot track cross-junction elements without project.refs. o _scheduler/trackqueue.py: Delegate saving refs to Source._save_ref()
* source.py: Document ref accessors to mandate support for None values.Tristan Van Berkom2018-03-201-3/+12
| | | | | | | This is technically an API break, but will be transparant for the vast majority of the current hand full of source implementations which exist at this time. This is a lesser evil than bloating the API with new methods.
* pylint - dealt with superfluous-parens warningJames Ennis2018-03-141-3/+3
|
* Silence messages from Source cache interrogationTristan Van Berkom2018-03-071-1/+5
| | | | | | | | | | | Source interrogation usually involves calling out to host tools to quickly check if a given ref exists. This has however regressed over time when running `bst build --track`. This patch adds a new context manager to silence the messages, and silences messages while calling `Source.get_consistency()` Fixes #280
* Add a 'sources' field to project.conf to override defaultsJonathan Maw2018-02-191-1/+28
| | | | | Equivalent to the 'elements' field, but slightly different because sources don't have accompanying yaml.
* source.py: fix comment typoAngelos Evripiotis2018-02-131-1/+1
|
* source.py: Fix typo in workspace warningJürg Billeter2018-02-131-1/+1
| | | | Fixes #250
* Raise warning if workspaced source's ref is updatedincremental-buildChandan Singh2018-01-251-0/+4
| | | | | | | In case a source with an open workspace is tracked and it's ref gets updated, BuildStream should inform the user that the new ref will not be picked up so long as the workspace is open. To start using the updated ref, the existing workspace will have to be closed.
* Add support for doing incremental buildsChandan Singh2018-01-251-5/+42
| | | | | | | | | | | | | | | | | | | | | | | | | This functionality is only supported for sources which have an open workspace. When such sources are present, the workspace directory will be mounted directly inside the sandbox. As opposed to the default behavior, which is to copy files inside the sandbox. This will save time when building large projects as only those files will need be re-compiled that have been modified during two consecutive builds (assuming the underlying build system supports such behavior). A few things to note regarding this behavior: - If there are any `configure-commands` present, they will run only once for each open workspace. If an element has multiple workspaces and any one of them is opened/closed, they will be executed again on the next run. But, modifying the contents of a workspace will not trigger the `configure-commands` to be executed on the next run. - Workspaced builds still leverage the cache. So, if no changes are made to the workspace, i.e. no files are modified, then it will not force a rebuild. Fixes #192.
* source.py: Properly handle Source.track() returning NoneJürg Billeter2018-01-181-0/+4
| | | | | | | Source.track() may return None when tracking is not available. Handle this identical to the case where track() returns the current ref. Fixes #201
* Use explicit source state updatesjuerg/source-stateJürg Billeter2018-01-161-17/+17
| | | | | This adds the _update_state() method to the Source class, similar to the corresponding method in the Element class.
* Rename _force_inconsistent() to _schedule_tracking()Jürg Billeter2018-01-161-1/+1
| | | | | _force_inconsistent is too low level. Keep that detail contained in the Source class.
* source.py: Allow specifying detailed message in SourceError()Tristan Van Berkom2018-01-111-3/+4
|
* source.py & element.py: Fixing SourceError() and ElementError() constructors.Tristan Van Berkom2018-01-041-3/+6
| | | | | | | | Recently I added the `reason` member which can be used to set machine readable error reason strings for the purpose of testing. Forgot to add the necessary `*` argument, forcing `reason` to be a keyword-only argument.
* source.py: Raise proper SourceError() when failing to create the staging ↵Tristan Van Berkom2018-01-021-1/+7
| | | | | | | directory This changes the UX to report a better human readable error, which is otherwise a BUG message with stack trace.
* Exceptions refactoringTristan Van Berkom2017-12-291-2/+3
| | | | | | | | | | | | | | | Outline of changes to exceptions: o Now BstError base class has a `domain` and `reason` member, reason members are optional. o All derived error classes now specify their `domain` o For LoadError, LoadErrorReason defines the error's `reason` o Now the scheduler `job` class takes care of sending the error reason and domain back to the main process where the last exception which caused a child task to fail can be discretely stored.
* source: clarify note on why to use this tmpdir()Angelos Evripiotis2017-12-131-3/+4
|
* source.py: Cache the workspace keyAntoine Wacheux2017-12-131-11/+26
| | | | | | In order to avoid multiple traversals of the file system when the workspace key is requested multiple times, it is now cached in the source element. The cache is invalidated if the workspace is deleted or moved.
* source.py: Implement _get_workspace_pathAntoine Wacheux2017-12-121-3/+8
|
* Remove unused importsGökçen Nurlu2017-12-071-4/+2
|
* Switch old-style string formattings to new '.format()'Gökçen Nurlu2017-11-171-5/+5
|
* Change origin to point at source repoDaniel Playle2017-11-111-3/+34
| | | | | | | | | | | | This changes workspaces created with the git source element so that the origin remote points to the source repository of the build element as opposed to the internal repository in the bst cache. This introduces an addition of the init_workspace method in the source API. This method, which defaults to calling stage, is for the setup of the workspace after the creation of the workspace directory. This is a part of issue #53
* Refactor: Move context.py -> _context.pyTristan Van Berkom2017-11-081-1/+1
| | | | | | | | | | Consequently: o Changed Plugin.get_context() to a private Plugin._get_context() accessor. o Updated anything which imports Context to do so from private _context module o Updated docs to exclude the now private Context
* refactoring: Made Project option privateTristan Van Berkom2017-11-061-3/+28
| | | | | | | | | | | | | | This required adding two new APIs to make up for it on the Source o get_project_directory() Added here because elements should not be accessing external resources, Sources needed for local files and GPG keys and such o translate_url() Used by sources to mish-mash the project aliases and create real urls.
* Refactoring: Renamed _BstError -> BstErrorTristan Van Berkom2017-11-061-2/+2
| | | | | | Base class for exceptions is now a part of the already private _exceptions module Also moved PipelineError from _pipeline -> _exceptions module
* Refactoring: Move exceptions module to be privateTristan Van Berkom2017-11-061-2/+1
| | | | Hide all of buildstream's internal exceptions from the API surface.
* Refactoring: Move ElementError and SourceError to their respective classes, ↵Tristan Van Berkom2017-11-061-1/+10
| | | | | | | | | | create SandboxError These errors are a part of public facing API, and the exceptions module contains a lot of internal details to be hidden from public API. This move required creating SandboxError because sandbox related code had previously been hijacking the ElementError and raising that.
* source.py: Dont error out if staging directory is not empty.Tristan Van Berkom2017-10-221-3/+0
| | | | | This is the wrong place for the check, it needs to be done once for the toplevel staging directory, not for each source.
* Add warnings when staging to /buildstream/build74-prevent-artifacts-from-containing-files-in-buildstream-buildTristan Maat2017-10-191-1/+4
|
* Add source plugin node validationsTristan Maat2017-09-141-0/+7
|
* source.py: Use utils.sha256sum() when calculating workspace keysTristan Van Berkom2017-09-121-11/+7
|
* source.py: Consider workspaced sources with missing content inconsistentTristan Van Berkom2017-09-041-2/+13
| | | | | | | Avoid trying to calculate cache keys and then running into an error by just considering workspaces with missing content to be inconsistent. This fixes issue #80
* source.py: Consider directory when calculating cache keyTristan Van Berkom2017-09-011-4/+12
| | | | | | | | NOTE: This commit changes cache keys for everything ! The core parses a 'directory' parameter which effects how all sources are staged. This obviously changes the build but until now this was ignored in cache key calculation.
* documentation: Set page titles manually in python modules.Tristan Van Berkom2017-08-311-0/+4
|
* source.py: Fix missing files when calculating sha256sumTristan Maat2017-07-171-4/+9
|
* source.py: Add workspace APITristan Maat2017-07-171-1/+77
|
* source.py: Use the new utils._tempdir() utility to implement Source.tempdir()Tristan Van Berkom2017-07-091-11/+2
|
* source.py: Ensure 'directory' exists in _stageJonathan Maw2017-06-221-0/+1
| | | | | | | | | It is usually created by integration commands when staging dependencies, but we cannot rely on this as some elements have no dependencies, and some sources handle the non-existence of 'directory' better than others. To avoid mandating that every source must be able to cope with 'directory's not existence, source will ensure it exists.
* source.py: Added _force_inconsistent() methodTristan Van Berkom2017-05-291-1/+15
| | | | | For initializing a pipeline in a forcefully inconsistent state, this is required for pipelines which execute TrackQueues.
* source.py: Allow forceful reinterrogation of consistency stateTristan Van Berkom2017-05-281-2/+2
|
* source.py: Added tempdir() context managerTristan Van Berkom2017-04-081-1/+27
| | | | | | This is for Source implementations to use a temp directory, this should be used instead of python tempdir APIs because it will cleanup the tempdir automatically in the case of user provoked termination.
* Refactor: Untangling element/source namesTristan Van Berkom2017-03-181-2/+2
| | | | | | | | | | | | | | | | | | | | | | | o The metaelements and metasources now carry the name, the loader resolves source names now. o Element/Source factories dont require a name anymore as they are already in the meta objects o Pipeline no longer composes names o Element.name is now the original project relative filename, this allows plugins to identify that name in their dependencies, allowing one to express configuration which identifies elements by the same name that the user used in the dependencies. o Removed plugin._get_display_name() in favor of the plugin.name o Added Element.normal_name, for the cases where we need to have a normalized name for creating directories and log files o Updated frontend and test cases and all callers to use the new naming
* source.py: Fix _set_ref() to not try to manually recalculate the cacheTristan Van Berkom2017-01-251-1/+0
| | | | This was missed in the previous changes regarding caching
* source.py: Removed recalculate argument from _get_consistency()Tristan Van Berkom2017-01-201-4/+5
|
* source.py: API change consistent() -> get_consistency()Tristan Van Berkom2017-01-191-18/+42
| | | | Now adds Consistency class for consistency options
* source.py: Removed refresh(), added reference accessor/mutatorTristan Van Berkom2017-01-181-26/+62
| | | | | | | | | | | | | | | Now we have a different API around refreshing tracking references: o get_ref(): Source should return currently set reference o set_ref(): Source should update current ref, and update the passed yaml node o track(): Return the latest reference for a symbolic tracking ref, this may involve network usage enough to track the new ref. This way we dont try to serialize entire yaml documents over the subprocess pipe, some of the formatting I think may have been lost in serialization.
View Lorry Controller Status