summaryrefslogtreecommitdiff
path: root/buildstream/sandbox
Commit message (Collapse)AuthorAgeFilesLines
* element: Make "--sysroot" take a bare directoryJonathan Maw2018-10-302-5/+14
| | | | | | | | | | | | | | | | | | i.e. instead of taking a directory that must contain "root" and "scratch", and treating "root" as the root, use the directory directly. In element.py: * __sandbox takes the `bare_sandbox` arg, to pass into the sandbox's constructor In sandbox.py: * If bare_sandbox, `_root` is the passed-in directory, and `__scratch` is None. * Trying to use `__scratch` when bare_sandbox is True is a bug. In _mount.py: * Don't get the value of `__scratch` if it's not needed. This is part of #539
* sandbox.py: Remove redundant Sandbox.__directoryJonathan Maw2018-10-301-2/+1
|
* Check if command is a str and replace by list before checking existenceBenjamin Schubert2018-10-263-8/+12
| | | | | | | | Currently we would make sure the sandbox had a command before converting it to a list if it was given as a string. That meant that a string command would never exist and the check be invalid. This also adds the same logic in the dummy sandbox for consistency.
* sandbox/_sandboxchroot.py: Silence warning about dict.getDaniel Silverstone2018-10-251-1/+1
| | | | | | | Sometimes `dict.get()` is preferable to an `if` statement, but this time it's clearer if we keep the structure. As such, silence the warning. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* sandbox/_sandboxchroot.py: Silence warning about preexec_fnDaniel Silverstone2018-10-251-1/+1
| | | | | | | We are super-careful to not use threads in places where we might use Popen and as such this warning is save to quash. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* sandbox/_sandboxchroot.py: Silence warning about unused variableDaniel Silverstone2018-10-251-1/+0
| | | | | | The variable was unused, remove it. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* sandbox/_sandboxbwrap.py: Disable lint about dict.getDaniel Silverstone2018-10-251-1/+1
| | | | | | | Sometimes `dict.get()` is preferable, in this instance it's less clear so we'll disable that lint Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* sandbox/_mounter.py: Remove useless inheritance on `object`Daniel Silverstone2018-10-251-1/+1
| | | | | | In Python 3, inheriting from `object` explicitly is pointless. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* sandbox/_mount.py: Do not use dict literals in argument defaultsDaniel Silverstone2018-10-251-3/+6
| | | | | | | The use of dictionary literals in argument defaults is disrecommended due to the way that they are static and thus potentially very confusing. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* _sandboxremote.py: Try to reopen operation steam on failuremablanch/630-remote-execution-reconnMartin Blanchard2018-10-231-15/+43
| | | | | | | | The REAPI allows a client to reconnect to an ongoing operation stream by providing a WaitExecution(). If implemented on server side, BuildStream will try to recover from connection errors using it. https://gitlab.com/BuildStream/buildstream/issues/630
* Fix spelling of it's and itsRichard Dale2018-10-161-1/+1
|
* Sandbox: CWD was not being created for workspacesWilliam Salmon2018-10-163-11/+8
| | | | | | | | | | | | The code was creating the cwd folder but when the workspace was mounted in to the buildroot it was hiding the folder created in it behind the bind mounted workspace. However by using the bubblewarp `--dir` directive to ensure that cwd exists we can cover both workspace and non workspace situations with the same method. For issue #512 in Gitlab.
* sandbox/_sandboxdummy.py: Take a reason for useDaniel Silverstone2018-10-041-1/+2
| | | | | | | | | Since the SandboxDummy sandbox is only used in very specific circumstances and with good reason, this adjusts the SandboxDummy class to take a reason and to return it if `.run()` has to raise an exception because this is a dummy sandbox. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* sandbox/_sandboxremote.py: Use helpers to set up cwd and envJürg Billeter2018-10-021-9/+5
| | | | This matches the other sandbox implementations.
* Import sandbox subclass when instantiating sandboxJames Ennis2018-09-271-2/+0
| | | | | This patch was required due to not being able to execute `bst` on a Linux machine without FUSE available.
* Adding Dummy sandboxknownexus2018-09-272-0/+41
| | | | | This is to allow platforms that do not support sandboxing To error in a controlled/known way
* sandbox/_sandboxremote.py: Migrate to Context.artifactcacheJürg Billeter2018-09-271-7/+6
|
* _sandboxremote.py: Handle remote build execution exit codemablanch/668-remote-build-failureMartin Blanchard2018-09-271-6/+15
| | | | https://gitlab.com/BuildStream/buildstream/issues/668
* sandbox.py: Stop caching get_virtual_directory if get_directory is usedJim MacArthur2018-09-251-7/+15
| | | | | Also update the documentation for get_virtual_directory to make the implications of using both methods clearer.
* _sandboxremote.py: Ignore return value of push_directoryJim MacArthur2018-09-211-2/+2
| | | | The return value is always upload_vdir.ref.
* FUSE: Mount with -odev in chroot sandboxRichard Maw2018-09-182-6/+11
| | | | | | | | | | | | | This is needed to permit access to the device nodes added to /dev on Linux when FUSE is used as root. The chroot sandbox only works with all privileges, so there's no explicit check for being root or having the appropriate capabilities. A check for whether it's running as root isn't needed on Linux with bubblewrap because /dev or its devices are mounted on top of the FUSE layer, so device nodes are accessed directly rather than through the FUSE layer.
* sandbox/_sandboxremote.py: Acquire artifact cache via PlatformDaniel Silverstone2018-09-181-12/+7
| | | | | | | | | The SandboxRemote used to construct its own CASCache which was considered dangerous. This patch replaces that with acquisition of the cache via the Platform singleton, hopefully eliminating issues from having more than one artifact cache object in a single process. Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* sandbox: deduplicate default environment logicRichard Maw2018-09-163-33/+29
|
* subprocesses: Ensure PWD is set in process environmentRichard Maw2018-09-102-3/+15
| | | | | | | | | | | | | | | Naive getcwd implementations (such as in bash 4.4) can break when bind-mounts to different paths on the same filesystem are present, since the algorithm needs to know whether it's a mount-point to know whether it can trust the inode value from the readdir result or to use stat on the directory. Less naive implementations (such as in glibc) iterate again using stat in the case of not finding the directory because the inode in readdir was wrong, though a Linux-specific implementation could use name_to_handle_at. Letting the command know what directory it is in makes it unnecessary for it to call the faulty getcwd in the first place.
* sandbox/__init__.py: Add SandboxRemoteJim MacArthur2018-09-071-0/+1
| | | | https://gitlab.com/BuildStream/buildstream/issues/454
* _sandboxremote.py: Implement the REAPI clientJim MacArthur2018-09-071-0/+226
| | | | | | | | The remote execution client is implemented as a remote sandbox that sends sources and build commands to a REAPI server and fetches results once remotely executed. New file. https://gitlab.com/BuildStream/buildstream/issues/454
* sandbox.py: Allow setting the virtual directoryJim MacArthur2018-09-071-3/+20
| | | | | | | | This is for use after remote execution has finished, since remote execution produces a new output directory rather than modifying the initial directory. https://gitlab.com/BuildStream/buildstream/issues/454
* Sandbox: Return a CasBasedDirectory when an environment variable is setJim MacArthur2018-08-151-2/+10
|
* Directory API: add mark_changedJim MacArthur2018-08-152-0/+2
|
* Convert uses of external_directory to get_underlying_directory()Jim MacArthur2018-08-152-3/+5
|
* Set environment in bwrap command line instead of its environmentValentin David2018-08-101-3/+7
| | | | Fixes #498
* sandbox.py: Use _root in _has_commandjmac/virtual_directoriesJim MacArthur2018-08-011-2/+2
| | | | | | | This will need more attention when we bring in another virtual directory backend, however, we've said it is acceptable for the sandbox itself to access the underlying directory, and this is the best fix in the meantime.
* sandbox: Remove instances of get_directoryJim MacArthur2018-08-012-2/+5
| | | | | sandbox/_mount.py, sandbox/_sandboxbwrap.py: Remove instances of get_directory
* sandbox.py: Add function to get virtual directory.Jim MacArthur2018-08-011-0/+15
|
* Sandbox.py: Rename __root to _root.Jim MacArthur2018-08-012-5/+6
| | | | | | | | This is to allow to allow its use by subclasses. Since access to get_directories is now blocked for some plugins, and the subclasses of Sandbox do not have configuration defined by YAML files, they need another way to get at the root directory.
* Add BST_VIRTUAL_DIRECTORY flag for element pluginsJim MacArthur2018-08-011-3/+10
|
* Provide better error message on missing commandsTiago Gomes2018-07-023-0/+33
| | | | | | | Before running a command in the sandbox, check its existence and fail early if it does not. This fixes issue #289.
* Remove shebangs from python filesGökçen Nurlu2018-06-197-7/+0
| | | | Fixes #424
* doc/source/core_framework.rst: Use a toctree instead of linksTristan Van Berkom2018-06-091-3/+2
|
* _sandboxbwrap.py: Fix post-bwrap cleanup behaviourGökçen Nurlu2018-06-051-1/+1
| | | | | | | | | The cleanup was supposed not to remove folders (`/dev`, `/tmp`, `/proc`) if they already existed before bwrap but it did the opposite: it tried to remove them if they existed before, and didn't remove them if they were created during bwrap. This was caused by a `not` clause, and this removes it. Fixes #379
* buildstream/sandbox/_mount.py: Fixing unused variablesTristan Van Berkom2018-04-191-1/+1
|
* mount: Wrap yields in context managers with try-finallyRichard Maw2018-04-121-11/+14
| | | | | | | | | | | | | | | The terminator context will only clean up on a signal, so if another exception causes context manager cleanup then unmount won't be called unless it's part of another context manager or is wrapped in a try block's except or finally. Everywhere else's unmounts are handled by delegating to another context manager but these were what needed to be fixed. The change in buildstream/_fuse/mount.py would cause lockups since the build worker process still having a subprocess blocked its termination from completing which in turn caused the pipeline manager process to block indefinitely on it.
* Consider new sandbox configuration in cache keys.Tristan Van Berkom2018-03-251-0/+34
| | | | | | | | | | | | | This adds SandboxConfig.get_unique_key() to delegate the cache key contribution to the SandboxConfig. Further, this moves over the OS and Machine Architecture parts of cache key calculation into SandboxConfig.get_unique_key(), removing the comment which speaks of later delegating this part of cache key calculation to sandboxes. Cache key calculation algorithm is backward compatible, and so the artifact version remains unchanged.
* sandbox/_config.py: Replace docstring with comment.Tristan Van Berkom2018-03-251-5/+4
| | | | | For consistency, we document private things with comments, and public things with docstrings.
* Add 'sandbox' configuration key and build-uid/build-gid elementsJim MacArthur2018-03-234-3/+54
| | | | | | | | | | | | | | | | | | | | This only affects SandboxBWrap at the moment. buildstream/_loader.py: Add Symbol.SANDBOX and allow it in validation buildstream/_metaelement.py: Add 'sandbox' variable and store it in the object buildstream/_project.py: Add 'sandbox' configuration key and load it from project.conf. buildstream/data/projectconfig.yaml: Default build-uid/build-gid values of 0 for 'sandbox'. buildstream/element.py: Add __extract_sandbox_config to find the final sandbox configuration. Pass this to the sandbox constructor. buildstream/sandbox/_sandboxbwrap.py: If sandbox configuration was supplied, use it for uid and gid instead of the default 0. buildstream/sandbox/_sandboxchroot.py: Throw exception if non-0 uid/gid were supplied. buildstream/sandbox/__init__.py: Import SandboxConfig. buildstream/sandbox/_private.py: New file, containing SandboxConfig. Made private to avoid documentation for this class.
* _sandboxchroot.py: Ensure the cwd existsJürg Billeter2018-03-151-0/+6
| | | | This matches SandboxBwrap.
* pylint - dealt with redefined-argument-from-local warningJames Ennis2018-03-141-2/+2
|
* pylint - dealt with import warningsJames Ennis2018-03-141-1/+1
|
* pylint - dealt with unidiomatic-typecheck warningJames Ennis2018-03-141-1/+1
|
* _sandboxbwrap.py: Force cleanup when in interactive modeTristan Van Berkom2018-03-021-7/+24
| | | | | | | | | | | | | | | | | | | | When running a sandbox in interactive mode (implies `bst shell` was launched or an interactive shell for debugging), dont bail out when cleaning up directories which are not empty. We treat this as a bug, if any of the base sandbox directories (/dev, /tmp or /proc) are not empty when tearing down the sandbox, because it would indicate something is wrong with bwrap. When in interactive mode however, the user/project may have mounted additional directories inside these base directories; for which we need to create intermediate directories for the mount. Instead of keeping track of every intermediate directory, just force remove in interactive mode, as this is safe. Ideally, we should fix upstream bwrap to cleanup the debris it creates when exiting.
View Lorry Controller Status