summaryrefslogtreecommitdiff
path: root/buildstream/source.py
Commit message (Collapse)AuthorAgeFilesLines
* Raise warning if workspaced source's ref is updatedincremental-buildChandan Singh2018-01-251-0/+4
| | | | | | | In case a source with an open workspace is tracked and it's ref gets updated, BuildStream should inform the user that the new ref will not be picked up so long as the workspace is open. To start using the updated ref, the existing workspace will have to be closed.
* Add support for doing incremental buildsChandan Singh2018-01-251-5/+42
| | | | | | | | | | | | | | | | | | | | | | | | | This functionality is only supported for sources which have an open workspace. When such sources are present, the workspace directory will be mounted directly inside the sandbox. As opposed to the default behavior, which is to copy files inside the sandbox. This will save time when building large projects as only those files will need be re-compiled that have been modified during two consecutive builds (assuming the underlying build system supports such behavior). A few things to note regarding this behavior: - If there are any `configure-commands` present, they will run only once for each open workspace. If an element has multiple workspaces and any one of them is opened/closed, they will be executed again on the next run. But, modifying the contents of a workspace will not trigger the `configure-commands` to be executed on the next run. - Workspaced builds still leverage the cache. So, if no changes are made to the workspace, i.e. no files are modified, then it will not force a rebuild. Fixes #192.
* source.py: Properly handle Source.track() returning NoneJürg Billeter2018-01-181-0/+4
| | | | | | | Source.track() may return None when tracking is not available. Handle this identical to the case where track() returns the current ref. Fixes #201
* Use explicit source state updatesjuerg/source-stateJürg Billeter2018-01-161-17/+17
| | | | | This adds the _update_state() method to the Source class, similar to the corresponding method in the Element class.
* Rename _force_inconsistent() to _schedule_tracking()Jürg Billeter2018-01-161-1/+1
| | | | | _force_inconsistent is too low level. Keep that detail contained in the Source class.
* source.py: Allow specifying detailed message in SourceError()Tristan Van Berkom2018-01-111-3/+4
|
* source.py & element.py: Fixing SourceError() and ElementError() constructors.Tristan Van Berkom2018-01-041-3/+6
| | | | | | | | Recently I added the `reason` member which can be used to set machine readable error reason strings for the purpose of testing. Forgot to add the necessary `*` argument, forcing `reason` to be a keyword-only argument.
* source.py: Raise proper SourceError() when failing to create the staging ↵Tristan Van Berkom2018-01-021-1/+7
| | | | | | | directory This changes the UX to report a better human readable error, which is otherwise a BUG message with stack trace.
* Exceptions refactoringTristan Van Berkom2017-12-291-2/+3
| | | | | | | | | | | | | | | Outline of changes to exceptions: o Now BstError base class has a `domain` and `reason` member, reason members are optional. o All derived error classes now specify their `domain` o For LoadError, LoadErrorReason defines the error's `reason` o Now the scheduler `job` class takes care of sending the error reason and domain back to the main process where the last exception which caused a child task to fail can be discretely stored.
* source: clarify note on why to use this tmpdir()Angelos Evripiotis2017-12-131-3/+4
|
* source.py: Cache the workspace keyAntoine Wacheux2017-12-131-11/+26
| | | | | | In order to avoid multiple traversals of the file system when the workspace key is requested multiple times, it is now cached in the source element. The cache is invalidated if the workspace is deleted or moved.
* source.py: Implement _get_workspace_pathAntoine Wacheux2017-12-121-3/+8
|
* Remove unused importsGökçen Nurlu2017-12-071-4/+2
|
* Switch old-style string formattings to new '.format()'Gökçen Nurlu2017-11-171-5/+5
|
* Change origin to point at source repoDaniel Playle2017-11-111-3/+34
| | | | | | | | | | | | This changes workspaces created with the git source element so that the origin remote points to the source repository of the build element as opposed to the internal repository in the bst cache. This introduces an addition of the init_workspace method in the source API. This method, which defaults to calling stage, is for the setup of the workspace after the creation of the workspace directory. This is a part of issue #53
* Refactor: Move context.py -> _context.pyTristan Van Berkom2017-11-081-1/+1
| | | | | | | | | | Consequently: o Changed Plugin.get_context() to a private Plugin._get_context() accessor. o Updated anything which imports Context to do so from private _context module o Updated docs to exclude the now private Context
* refactoring: Made Project option privateTristan Van Berkom2017-11-061-3/+28
| | | | | | | | | | | | | | This required adding two new APIs to make up for it on the Source o get_project_directory() Added here because elements should not be accessing external resources, Sources needed for local files and GPG keys and such o translate_url() Used by sources to mish-mash the project aliases and create real urls.
* Refactoring: Renamed _BstError -> BstErrorTristan Van Berkom2017-11-061-2/+2
| | | | | | Base class for exceptions is now a part of the already private _exceptions module Also moved PipelineError from _pipeline -> _exceptions module
* Refactoring: Move exceptions module to be privateTristan Van Berkom2017-11-061-2/+1
| | | | Hide all of buildstream's internal exceptions from the API surface.
* Refactoring: Move ElementError and SourceError to their respective classes, ↵Tristan Van Berkom2017-11-061-1/+10
| | | | | | | | | | create SandboxError These errors are a part of public facing API, and the exceptions module contains a lot of internal details to be hidden from public API. This move required creating SandboxError because sandbox related code had previously been hijacking the ElementError and raising that.
* source.py: Dont error out if staging directory is not empty.Tristan Van Berkom2017-10-221-3/+0
| | | | | This is the wrong place for the check, it needs to be done once for the toplevel staging directory, not for each source.
* Add warnings when staging to /buildstream/build74-prevent-artifacts-from-containing-files-in-buildstream-buildTristan Maat2017-10-191-1/+4
|
* Add source plugin node validationsTristan Maat2017-09-141-0/+7
|
* source.py: Use utils.sha256sum() when calculating workspace keysTristan Van Berkom2017-09-121-11/+7
|
* source.py: Consider workspaced sources with missing content inconsistentTristan Van Berkom2017-09-041-2/+13
| | | | | | | Avoid trying to calculate cache keys and then running into an error by just considering workspaces with missing content to be inconsistent. This fixes issue #80
* source.py: Consider directory when calculating cache keyTristan Van Berkom2017-09-011-4/+12
| | | | | | | | NOTE: This commit changes cache keys for everything ! The core parses a 'directory' parameter which effects how all sources are staged. This obviously changes the build but until now this was ignored in cache key calculation.
* documentation: Set page titles manually in python modules.Tristan Van Berkom2017-08-311-0/+4
|
* source.py: Fix missing files when calculating sha256sumTristan Maat2017-07-171-4/+9
|
* source.py: Add workspace APITristan Maat2017-07-171-1/+77
|
* source.py: Use the new utils._tempdir() utility to implement Source.tempdir()Tristan Van Berkom2017-07-091-11/+2
|
* source.py: Ensure 'directory' exists in _stageJonathan Maw2017-06-221-0/+1
| | | | | | | | | It is usually created by integration commands when staging dependencies, but we cannot rely on this as some elements have no dependencies, and some sources handle the non-existence of 'directory' better than others. To avoid mandating that every source must be able to cope with 'directory's not existence, source will ensure it exists.
* source.py: Added _force_inconsistent() methodTristan Van Berkom2017-05-291-1/+15
| | | | | For initializing a pipeline in a forcefully inconsistent state, this is required for pipelines which execute TrackQueues.
* source.py: Allow forceful reinterrogation of consistency stateTristan Van Berkom2017-05-281-2/+2
|
* source.py: Added tempdir() context managerTristan Van Berkom2017-04-081-1/+27
| | | | | | This is for Source implementations to use a temp directory, this should be used instead of python tempdir APIs because it will cleanup the tempdir automatically in the case of user provoked termination.
* Refactor: Untangling element/source namesTristan Van Berkom2017-03-181-2/+2
| | | | | | | | | | | | | | | | | | | | | | | o The metaelements and metasources now carry the name, the loader resolves source names now. o Element/Source factories dont require a name anymore as they are already in the meta objects o Pipeline no longer composes names o Element.name is now the original project relative filename, this allows plugins to identify that name in their dependencies, allowing one to express configuration which identifies elements by the same name that the user used in the dependencies. o Removed plugin._get_display_name() in favor of the plugin.name o Added Element.normal_name, for the cases where we need to have a normalized name for creating directories and log files o Updated frontend and test cases and all callers to use the new naming
* source.py: Fix _set_ref() to not try to manually recalculate the cacheTristan Van Berkom2017-01-251-1/+0
| | | | This was missed in the previous changes regarding caching
* source.py: Removed recalculate argument from _get_consistency()Tristan Van Berkom2017-01-201-4/+5
|
* source.py: API change consistent() -> get_consistency()Tristan Van Berkom2017-01-191-18/+42
| | | | Now adds Consistency class for consistency options
* source.py: Removed refresh(), added reference accessor/mutatorTristan Van Berkom2017-01-181-26/+62
| | | | | | | | | | | | | | | Now we have a different API around refreshing tracking references: o get_ref(): Source should return currently set reference o set_ref(): Source should update current ref, and update the passed yaml node o track(): Return the latest reference for a symbolic tracking ref, this may involve network usage enough to track the new ref. This way we dont try to serialize entire yaml documents over the subprocess pipe, some of the formatting I think may have been lost in serialization.
* source.py: Added _stage() wrapper method.Tristan Van Berkom2017-01-181-0/+9
| | | | | This internal method acts as a controller method and does the conversion for the 'directory' parameter.
* source.py: INFO message when a source ref changesTristan Van Berkom2017-01-141-0/+2
|
* source.py: Now constructor takes a display_nameTristan Van Berkom2017-01-121-2/+2
|
* source.py: Wrap some abstract methodsTristan Van Berkom2017-01-111-0/+26
| | | | | Use _refresh() and _consistent() internally, now we cache the consistent state making things a bit less loopy
* source.py: Added consistent() abstract method.Tristan Van Berkom2017-01-091-0/+13
| | | | | | | This is expected to return True if the ref is set and the source can be fetched and staged, otherwise False. If a source is inconsistent, it means that it needs to be sucessfully refreshed (from any track that might be supported in the source).
* source.py: Document ->refresh() to return a booleanTristan Van Berkom2017-01-071-0/+3
| | | | Now refresh returns True if the refresh resulted in any change.
* source.py: Derive from new Plugin common classTristan Van Berkom2017-01-071-91/+7
|
* source.py: Allow Source object to be stringified.Tristan Van Berkom2016-12-191-0/+6
| | | | Same should be done for Elements.
* source.py: Added get_mirror_directory() and enhanced docsTristan Van Berkom2016-12-191-2/+17
| | | | | | The get_mirror_directory() method is just convenience for the Source implementation to get it's own private mirroring base directory, which is a subdirectory of the user specified sources directory.
* Simplify exceptions a bitTristan Van Berkom2016-12-181-6/+20
| | | | | | | | | | | Instead of declaring per operation exceptions, like PreflightError and FetchError etc, just declare exceptions per domain. Replaced the former errors with SourceError for Source implementations and ElementError for element implementations. The core will be able to report errors differently depending on what they asked given plugins to do, and handle a reduced set of exceptions.
* source.py: Fleshing out initial Source plugin facing APITristan Van Berkom2016-12-151-1/+135
|
View Lorry Controller Status