summaryrefslogtreecommitdiff
path: root/layer
Commit message (Collapse)AuthorAgeFilesLines
* layer: NewStoreFromOptions(): include driver-name in error messageSebastiaan van Stijn2023-03-061-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | When reading through some bug reports, I noticed that the error-message for unsupported storage drivers is not very informative, as it does not include the actual storage driver. Some of these errors are used as sentinel errors internally, so improving the error returned by graphdriver.New() may need some additional work, but this patch makes a start by including the name of the graphdriver (if set) in the error-message. Before this patch: dockerd --storage-driver=foobar ... failed to start daemon: error initializing graphdriver: driver not supported With this patch: dockerd --storage-driver=foobar ... failed to start daemon: error initializing graphdriver: driver not supported: foobar It's worth noting that there may be code "in the wild" that perform string- matching on this error (e.g. [balena][1]), which is why I included the name as a separate "component" in the output, to allow matching parts of the error. [1]: https://github.com/balena-io-modules/balena-preload/blob/3d5c77a4668b688dd21e9d7aec7c333943680adb/lib/preload.ts#L34-L35 Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* use errors.Is() to handle image store errorsSebastiaan van Stijn2023-01-021-1/+2
| | | | | | | | The image store's used are an interface, so there's no guarantee that implementations don't wrap the errors. Make sure to catch such cases by using errors.Is. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* layer: layerStore.registerWithDescriptor: rename var to prevent shadowingSebastiaan van Stijn2022-11-251-19/+19
| | | | | | | | | | This `err` is special (as described at the top of the function), but due to its name is easy to overlook, which risks the chance of inadvertently shadowing it. This patch renames the variable to reduce the chance of this happening. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* Remove uses of deprecated go-digest.NewDigestFromHex, go-digest.Digest.HexSebastiaan van Stijn2022-11-083-5/+5
| | | | | | | | | | Both of these were deprecated in https://github.com/opencontainers/go-digest/commit/55f675811a1b915549933c64571fd86b2676ba76, but the format of the GoDoc comments didn't follow the correct format, which caused them not being picked up by tools as "deprecated". This patch updates uses in the codebase to use the alternatives. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* layer: gofumptSebastiaan van Stijn2022-10-087-66/+67
| | | | Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* layer: use strconv instead of fmt.Sprintf()Sebastiaan van Stijn2022-10-081-3/+1
| | | | Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* runconfig, oci, image, layer, distribution: fix empty-lines (revive)Sebastiaan van Stijn2022-09-281-1/+0
| | | | | | | | | | | | | | | | | | | | | | | runconfig/config_test.go:23:46: empty-lines: extra empty line at the start of a block (revive) runconfig/config_test.go:75:55: empty-lines: extra empty line at the start of a block (revive) oci/devices_linux.go:57:34: empty-lines: extra empty line at the start of a block (revive) oci/devices_linux.go:60:69: empty-lines: extra empty line at the start of a block (revive) image/fs_test.go:53:38: empty-lines: extra empty line at the end of a block (revive) image/tarexport/save.go:88:29: empty-lines: extra empty line at the end of a block (revive) layer/layer_unix_test.go:21:34: empty-lines: extra empty line at the end of a block (revive) distribution/xfer/download.go:302:9: empty-lines: extra empty line at the end of a block (revive) distribution/manifest_test.go:154:99: empty-lines: extra empty line at the end of a block (revive) distribution/manifest_test.go:329:52: empty-lines: extra empty line at the end of a block (revive) distribution/manifest_test.go:354:59: empty-lines: extra empty line at the end of a block (revive) registry/config_test.go:323:42: empty-lines: extra empty line at the end of a block (revive) registry/config_test.go:350:33: empty-lines: extra empty line at the end of a block (revive) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* pkg/containerfs: drop ContainerFS type aliasCory Snider2022-09-234-14/+10
| | | | Signed-off-by: Cory Snider <csnider@mirantis.com>
* pkg/containerfs: alias ContainerFS to stringCory Snider2022-09-234-20/+20
| | | | | | Drop the constructor and redundant string() type-casts. Signed-off-by: Cory Snider <csnider@mirantis.com>
* pkg/containerfs: simplify ContainerFS typeCory Snider2022-09-234-19/+19
| | | | | | Iterate towards dropping the type entirely. Signed-off-by: Cory Snider <csnider@mirantis.com>
* pkg/containerfs: drop Driver abstractionCory Snider2022-09-232-16/+17
| | | | | | | | | | The Driver abstraction was needed for Linux Containers on Windows, support for which has since been removed. There is no direct equivalent to Lchmod() in the standard library so continue to use the containerd/continuity version. Signed-off-by: Cory Snider <csnider@mirantis.com>
* pkg/containerfs: drop PathDriver abstractionCory Snider2022-09-232-14/+15
| | | | | | | With LCOW support removed, there is no need to support non-native file paths any longer. Signed-off-by: Cory Snider <csnider@mirantis.com>
* layer: layerstore.Cleanup(): improve some loggingSebastiaan van Stijn2022-06-211-3/+5
| | | | | | | | Improve consistency for the logs, and remove a redundant log: time="2022-06-07T15:37:24.418470152Z" level=debug msg="found 0 orphan layers" Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* Merge pull request #43366 from corhere/finish-identitymapping-refactorSebastiaan van Stijn2022-03-252-4/+3
|\ | | | | Finish refactor of UID/GID usage to a new struct
| * Finish refactor of UID/GID usage to a new structCory Snider2022-03-142-4/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Finish the refactor which was partially completed with commit 34536c498d56, passing around IdentityMapping structs instead of pairs of []IDMap slices. Existing code which uses []IDMap relies on zero-valued fields to be valid, empty mappings. So in order to successfully finish the refactoring without introducing bugs, their replacement therefore also needs to have a useful zero value which represents an empty mapping. Change IdentityMapping to be a pass-by-value type so that there are no nil pointers to worry about. The functionality provided by the deprecated NewIDMappingsFromMaps function is required by unit tests to to construct arbitrary IdentityMapping values. And the daemon will always need to access the mappings to pass them to the Linux kernel. Accommodate these use cases by exporting the struct fields instead. BuildKit currently depends on the UIDs and GIDs methods so we cannot get rid of them yet. Signed-off-by: Cory Snider <csnider@mirantis.com>
* | layer: remove layerstore.setOS(), layerstore.getOS()Sebastiaan van Stijn2022-03-114-62/+1
|/ | | | | | | | This removes the `setOS()` / `getOS()` functions from the layer store, which were added in fc21bf280bac39377d3a236efa87f5c8cbadfb9f and 0380fbff37922cadf294851b1546f4c212c7f364 in support of LCOW. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* Merge pull request #43302 from thaJeztah/layer_remove_getwithoutlockSebastiaan van Stijn2022-03-052-12/+10
|\ | | | | layers: remove layerStore.getWithoutLock()
| * layers: remove layerStore.getWithoutLock()Sebastiaan van Stijn2022-02-282-12/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This function was abstracting things a bit too much; the layerStore had a exported `.Get()` which called `.getWithoutLock()`, but also a non-exported `.get()`, which also called `.getWithoutLock()`. While it's common to have a non-exported variant (without locking), the naming of `.get()` could easily be confused for that variant (which it wasn't). All locations where `.get()` was called were already handling locks for `releaseLayer()`, so moving the actual locking inline for `.get()` makes it more visible where locking happens. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* | remove unneeded "digest" alias for "go-digest"Sebastiaan van Stijn2022-03-048-8/+8
| | | | | | | | | | | | | | | | I think this was there for historic reasons (may have been goimports expected this, and we used to have a linter that wanted it), but it's not needed, so let's remove it (to make my IDE less complaining about unneeded aliases). Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* | layer: remove unused ErrActiveMount, ErrNotMounted, ErrNotSupportedSebastiaan van Stijn2022-03-021-13/+0
|/ | | | | | | These errors were added in 500e77bad0b19b3b1c8e6ac195485adcb70daef1, but were never used. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* Merge pull request #43182 from thaJeztah/layer_remove_unused_errorSebastiaan van Stijn2022-02-177-42/+21
|\ | | | | layer: remove unused error return from .Size() and .DiffSize()
| * layer: remove unused error return from .Size() and .DiffSize()Sebastiaan van Stijn2022-01-247-42/+21
| | | | | | | | | | | | | | None of the implementations used return an error, so removing the error return can simplify using these. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* | layer: remove OS from layerstoreSebastiaan van Stijn2022-01-254-16/+10
|/ | | | | | | | This was added in commits fc21bf280bac39377d3a236efa87f5c8cbadfb9f and 0380fbff37922cadf294851b1546f4c212c7f364 in support of LCOW, but was now always set to runtime.GOOS. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* refactor: move from io/ioutil to io and os packageEng Zer Jun2021-08-278-43/+36
| | | | | | | | The io/ioutil package has been deprecated in Go 1.16. This commit replaces the existing io/ioutil functions with their new definitions in io and os packages. Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* Update to Go 1.17.0, and gofmt with Go 1.17Sebastiaan van Stijn2021-08-243-0/+3
| | | | Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* fix typo in commentsKaijie Chen2021-03-021-1/+1
| | | | Signed-off-by: Kaijie Chen <chen@kaijie.org>
* replace pkg/locker with github.com/moby/lockerSebastiaan van Stijn2020-09-101-1/+1
| | | | Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* Remove "could not get checksum with tar-split" debug messageSebastiaan van Stijn2020-03-061-1/+0
| | | | | | | | | | | | | | | | | This code was originally written for v1.10 migration where it signified that layers were recomputed from pre 1.8 layout and could possibly change the tarball checksums. It's now being repurposed in the BuildKit adapter but there it doesn't have any warn condition as all data generated by builder is new anyway. Currently, debug log entries as the one below may appear in the daemon logs: [2019-10-02T10:00:06.690674253Z] could not get checksum for "x128nsj79yzfx4j5h6em2w2on" with tar-split: "no tar-split file" This patch removes the debug log, as it may confuse users ("we couldn't validate what we downloaded, but we're gonna run it anyway?") Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* Make sure that layers are removed from layerdb after succefull layer removalOlli Janatuinen2019-11-282-3/+3
| | | | Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
* Remove unused functions, variables, fieldsSebastiaan van Stijn2019-09-182-5/+0
| | | | | | | | | | | | | opts/env_test: suppress a linter warning this one: > opts/env_test.go:95:4: U1000: field `err` is unused (unused) > err error > ^ Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com> Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* goimports: fix importsSebastiaan van Stijn2019-09-187-7/+7
| | | | | | | Format the source according to latest goimports. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com> Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* Merge pull request #39715 from olljanat/getorplan-unit-testSebastiaan van Stijn2019-09-161-0/+48
|\ | | | | Unit test for getOrphan
| * Unit test for getOrphanOlli Janatuinen2019-08-101-0/+48
| | | | | | | | Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
* | Reimplement iteration over fileInfos in getOrphan.Vikram bir Singh2019-09-061-23/+31
|/ | | | | | | | | | 1. Reduce complexity due to nested if blocks by using early return/continue 2. Improve logging Changes suggested as a part of code review comments in 39748 Signed-off-by: Vikram bir Singh <vikrambir.singh@docker.com>
* First step to implement full garbage collector for image layersOlli Janatuinen2019-06-052-5/+111
| | | | | | | | | | | Refactored exiting logic on way that layers are first marked to be under removal so if actual removal fails they can be found from disk and cleaned up. Full garbage collector will be implemented as part of containerd migration. Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
* layer: protect from same-name racesKir Kolyshkin2019-05-211-17/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As pointed out by Tonis, there's a race between ReleaseRWLayer() and GetRWLayer(): ``` ----- goroutine 1 ----- ----- goroutine 2 ----- ReleaseRWLayer() m := ls.mounts[l.Name()] ... m.deleteReference(l) m.hasReferences() ... GetRWLayer() ... mount := ls.mounts[id] ls.driver.Remove(m.mountID) ls.store.RemoveMount(m.name) return mount.getReference() delete(ls.mounts, m.Name()) ----------------------- ----------------------- ``` When something like this happens, GetRWLayer will return an RWLayer without a storage. Oops. There might be more races like this, and it seems the best solution is to lock by layer id/name by using pkg/locker. With this in place, name collision could not happen, so remove the part of previous commit that protected against it in CreateRWLayer (temporary nil assigmment and associated rollback). So, now we have * layerStore.mountL sync.Mutex to protect layerStore.mount map[] (against concurrent access); * mountedLayer's embedded `sync.Mutex` to protect its references map[]; * layerStore.layerL (which I haven't touched); * per-id locker, to avoid name conflicts and concurrent operations on the same rw layer. The whole rig seems to look more readable now (mutexes use is straightforward, no nested locks). Reported-by: Tonis Tiigi <tonistiigi@gmail.com> Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com> Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
* layer/CreateRWLayerByGraphID: removeKir Kolyshkin2019-05-212-219/+0
| | | | | | | | This is an additon to commit 1fea38856a ("Remove v1.10 migrator") aka PR #38265. Since that one, CreateRWLayerByGraphID() is not used anywhere, so let's drop it. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
* layer: optimize layerStore mountLXinfeng Liu2019-05-092-21/+35
| | | | | | | | | | | | | | | | | | | | Goroutine stack analisys shown some lock contention while doing massively (100 instances of `docker rm`) parallel image removal, with many goroutines waiting for the mountL mutex. Optimize it. With this commit, the above operation is about 3x faster, with no noticeable change to container creation times (tested on aufs and overlay2). kolyshkin@: - squashed commits - added description - protected CreateRWLayer against name collisions by temporary assiging nil to ls.mounts[name], and treating nil as "non-existent" in all the other functions. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
* layer: protect mountedLayer.referencesKir Kolyshkin2019-05-061-1/+13
| | | | | | Add a mutex to protect concurrent access to mountedLayer.references map. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
* add ApplyDiff to RWLayerEvan Hazlett2019-04-253-0/+66
| | | | Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
* layer/layer_store: ensure NewInputTarStream resources are releasedSergio Lopez2018-12-211-3/+4
| | | | | | | | | | | | In applyTar, if the driver's ApplyDiff returns an error, the function returns early without calling io.Copy. As a consequence, the resources (a goroutine and some buffers holding the uncompressed image, the digest, etc...) allocated or referenced by NewInputTarStream above aren't released, as the worker goroutine only finishes when it finds EOF or a closed pipe. Signed-off-by: Sergio Lopez <slp@redhat.com>
* Fix a typos in layer_windows.gozhenhai gao2018-11-181-1/+1
| | | | Signed-off-by: zhenhai gao <gaozh1988@live.com>
* Add ADD/COPY --chown flag support to WindowsSalahuddin Khan2018-08-131-3/+3
| | | | | | | | | | | | | | | | | | | | This implements chown support on Windows. Built-in accounts as well as accounts included in the SAM database of the container are supported. NOTE: IDPair is now named Identity and IDMappings is now named IdentityMapping. The following are valid examples: ADD --chown=Guest . <some directory> COPY --chown=Administrator . <some directory> COPY --chown=Guests . <some directory> COPY --chown=ContainerUser . <some directory> On Windows an owner is only granted the permission to read the security descriptor and read/write the discretionary access control list. This fix also grants read/write and execute permissions to the owner. Signed-off-by: Salahuddin Khan <salah@docker.com>
* layer: relax graphdriver ID formatTonis Tiigi2018-06-101-3/+3
| | | Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* daemon: access to distribution internalsTonis Tiigi2018-06-102-0/+8
| | | Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
* Remove metadata store interfaceDerek McGowan2018-03-067-81/+26
| | | | | | | | | Layer metadata storage has not been implemented outside of the layer store and will be deprecated by containerd metadata storage. To prepare for this and freeze the current metadata storage, remove the exported interface and make it internal to the layer store. Signed-off-by: Derek McGowan <derek@mcgstyle.net>
* Add canonical import commentDaniel Nephin2018-02-0519-19/+19
| | | | Signed-off-by: Daniel Nephin <dnephin@docker.com>
* Golint fix with ro_layer.goYong Tang2018-01-241-5/+1
| | | | | | A small golint fix with ro_layer.go. Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Remove OS() from layer interfaceJohn Howard2018-01-185-23/+0
| | | | Signed-off-by: John Howard <jhoward@microsoft.com>
* Address feedback from TonisJohn Howard2018-01-185-12/+12
| | | | Signed-off-by: John Howard <jhoward@microsoft.com>
View Lorry Controller Status