summaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* Merge pull request #4662 from pks-t/pks/gitfile-apiEdward Thomson2018-06-092-129/+71
|\ | | | | path: unify `git_path_is_*` APIs
| * path: unify `git_path_is_*` APIsPatrick Steinhardt2018-06-012-129/+71
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Right now, there's quite a lot of different function calls to determine whether a path component matches a specific name after normalization from the filesystem. We have a function for each of {gitattributes, gitmodules, gitignore} multiplicated with {generic, NTFS, HFS} checks. In the long time, this is unmaintainable in case there are e.g. new filesystems with specific semantics, blowing up the number of functions we need to implement. Replace all functions with a simple `git_path_is_gitfile` function, which accepts an enum pointing out the filename that is to be checked against as well as the filesystem normalizations to check for. This greatly simplifies implementation at the expense of the caller having to invoke a somewhat longer function call.
* | Merge pull request #4670 from pks-t/pks/ignore-leadingdirEdward Thomson2018-06-092-17/+13
|\ \ | | | | | | Fix negative gitignore rules with leading directories
| * | ignore: remove now-useless check for LEADINGDIRPatrick Steinhardt2018-06-061-14/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When checking whether a rule negates another rule, we were checking whether a rule had the `GIT_ATTR_FNMATCH_LEADINGDIR` flag set and, if so, added a "/*" to its end before passing it to `fnmatch`. Our code now sets `GIT_ATTR_FNMATCH_NOLEADINGDIR`, thus the `LEADINGDIR` flag shall never be set. Furthermore, due to the `NOLEADINGDIR` flag, trailing globs do not get consumed by our ignore parser anymore. Clean up code by just dropping this now useless logic.
| * | ignore: fix negative leading directory rules unignoring subdirectory filesPatrick Steinhardt2018-06-062-3/+10
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When computing whether a file is ignored, we simply search for the first matching rule and return whether it is a positive ignore rule (the file is really ignored) or whether it is a negative ignore rule (the file is being unignored). Each rule has a set of flags which are being passed to `fnmatch`, depending on what kind of rule it is. E.g. in case it is a negative ignore we add a flag `GIT_ATTR_FNMATCH_NEGATIVE`, in case it contains a glob we set the `GIT_ATTR_FNMATCH_HASGLOB` flag. One of these flags is the `GIT_ATTR_FNMATCH_LEADINGDIR` flag, which is always set in case the pattern has a trailing "/*" or in case the pattern is negative. The flag causes the `fnmatch` function to return a match in case a string is a leading directory of another, e.g. "dir/" matches "dir/foo/bar.c". In case of negative patterns, this is wrong in certain cases. Take the following simple example of a gitignore: dir/ !dir/ The `LEADINGDIR` flag causes "!dir/" to match "dir/foo/bar.c", and we correctly unignore the directory. But take this example: *.test !dir/* We expect everything in "dir/" to be unignored, but e.g. a file in a subdirectory of dir should be ignored, as the "*" does not cross directory hierarchies. With `LEADINGDIR`, though, we would just see that "dir/" matches and return that the file is unignored, even if it is contained in a subdirectory. Instead, we want to ignore leading directories here and check "*.test". Afterwards, we have to iterate up to the parent directory and do the same checks. To fix the issue, disallow matching against leading directories in gitignore files. This can be trivially done by just adding the `GIT_ATTR_FNMATCH_NOLEADINGDIR` to the spec passed to `git_attr_fnmatch__parse`. Due to a bug in that function, though, this flag is being ignored for negative patterns, which is fixed in this commit, as well. As a last fix, we need to ignore rules that are supposed to match a directory when our path itself is a file. All together, these changes fix the described error case.
* | settings: allow swapping out memory allocatorPatrick Steinhardt2018-06-072-0/+14
| | | | | | | | | | | | | | Tie in the newly created infrastructure for swapping out memory allocators into our settings code. A user can now simply use the new option "GIT_OPT_SET_ALLOCATOR" with `git_libgit2_opts`, passing in an already initialized allocator structure as vararg.
* | alloc: make memory allocators use function pointersPatrick Steinhardt2018-06-077-115/+142
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, our memory allocators are being redirected to the correct implementation at compile time by simply using macros. In order to make them swappable at runtime, this commit reshuffles that by instead making use of a global "git_allocator" structure, whose pointers are set up to reference the allocator functions. Like this, it becomes easy to swap out allocators by simply setting these function pointers. In order to initialize a "git_allocator", our provided allocators "stdalloc" and "crtdbg" both provide an init function. This is being called to initialize a passed in allocator struct and set up its members correctly. No support is yet included to enable users of libgit2 to switch out the memory allocator at a global level.
* | stdalloc: extend allocators by file and linePatrick Steinhardt2018-06-073-27/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Our desired architecture would make allocators completely pluggable, such that users of libgit2 can swap out memory allocators at runtime. While making e.g. debugging easier by not having to do a separate build, this feature can also help maintainers of bindings for libgit2 by tying the memory allocations into the other language's memory system. In order to do so, though, we first need to make our two different pre-existing allocators "stdalloc" and "crtdbg" have the same function signatures, as the "crtdbg" allocators all have an additional file and line argument. This is required to build correct stack traces for debugging memory allocations. As that feature may also be interesting to authors of other applications for debugging libgit2, we now simply add these arguments to our standard allocators. Obviously, this may come with a performance penalty. During some simple benchmarks no real impact could be measured though in contrast to a simple pluggable allocator. The following table summarizes the benchmarks. There were three different builds with our current standard allocator ("standard"), with pluggable authenticators accessed via function pointers ("pluggable") and for pluggable authenticators with file and line being added ("fileline"). Furthermore, there were three scenarios for 100.000.000 allocations of 100B ("small alloc"), 100.000.000 allocations of 100KB ("medium alloc"), and 1.000.000 allocations of 100MB. All results are best of 10 runs. |------------|-------------------|-------------------|-------------------| | build/test | small alloc | medium alloc | big alloc | |------------|-------------------|-------------------|-------------------| | standard | 4539779566, +0.0% | 5912927186, +0.0% | 5166935308, +0.0% | |------------|-------------------|-------------------|-------------------| | pluggable | 4611074505, +1.5% | 5979185308, +1.1% | 5388776352, +4.2% | |------------|-------------------|-------------------|-------------------| | fileline | 4588338192, +1.1% | 6004951910, +1.5% | 4942528135, -4.4% | |------------|-------------------|-------------------|-------------------| As can be seen, there is a performance overhead for pluggable allocators. Furthermore, it can also be seen that there is some big variance between runs, especially in the "big alloc" scenario. This is probably being caused by nondeterministic behaviour in the kernel for dynamic allocations. Still, it can be observed that there should be no real difference between the "pluggable" and "fileline" allocators.
* | util: extract allocators into its own "alloc.h" headerPatrick Steinhardt2018-06-072-56/+71
| | | | | | | | | | | | | | | | | | | | Our "util.h" header is a grabbag of various different functions, where many don't have a clear group they belong to. Our set of allocator functions though can be clearly singled out as a single group of functions that always belongs together. Furthermore, we will need to implement additional functions relating to our allocators subsystem when moving to pluggable allocators. Thus, we should just move these functions into their own "alloc" module.
* | util: extract `stdalloc` allocator into its own modulePatrick Steinhardt2018-06-073-91/+137
| | | | | | | | | | | | | | | | | | | | | | Right now, the standard allocator is being declared as part of the "util.h" header as a set of inline functions. As with the crtdbg allocator functions, these inline functions make it hard to convert to function pointers for our allocators. Create a new "stdalloc" module containing our standard allocations functions to split these out. Convert the existing allocators to macros which make use of the stdalloc functions.
* | win32: crtdbg: provide independent `free` functionPatrick Steinhardt2018-06-073-2/+10
| | | | | | | | | | | | | | | | | | | | | | | | Currently, the `git__free` function is being defined in a single place, only, disregarding whether we use our standard allocators or the crtdbg allocators. This makes it a bit harder to convert our code base to use pluggable allocators, and furthermore makes the border between our two allocators a bit more blurry. Implement a separate `git__crtdbg__free` function for the crtdbg allocator in order to completely separate both allocator implementations.
* | win32: crtdbg: internalize implementation of allocatorsPatrick Steinhardt2018-06-072-74/+82
| | | | | | | | | | | | | | | | | | | | | | The crtdbg allocators are currently being implemented as inline functions as part of the "w32_crtdbg_stacktrace.h" header. As we are moving towards pluggable allocators with the help of function pointers, though, we cannot make use of inlining anymore. Instead, we can only have a single implementation of these allocating functions. Move all implementations of the crtdbg allocators into "w32_crtdbg_stacktrace.c".
* | Merge pull request #4655 from glaubitz/alignmentPatrick Steinhardt2018-06-071-21/+21
|\ \ | | | | | | index: Fix alignment issues in write_disk_entry()
| * | index: Fix alignment issues in write_disk_entry()John Paul Adrian Glaubitz2018-06-011-21/+21
| |/ | | | | | | | | | | In order to avoid alignment issues on certain target architectures, it is necessary to use memcpy() when modifying elements of a struct inside a buffer returned by git_filebuf_reserve().
* | Merge pull request #4665 from neithernut/fix-refdb-globPatrick Steinhardt2018-06-061-2/+6
|\ \ | | | | | | refdb_fs: fix regression: failure when globbing for non-existant references
| * | refdb_fs: test whether the base directory exists when globbingJulian Ganz2018-06-011-2/+6
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit fixes a regression introduced by 20a2b02d9a1bcb4825ec49605146223c565dcacf The commit introduced an optimization for finding references using a glob: rather than iterating over all references and matching each one against the glob, we would iterate only over references within the directory common to all possible references which may match against the glob. However, contrary to the `ref/` directory, which was the previous entry point for the iteration, this directory may not exist. In this case, the optimization causes an error (`ENOENT`) rather than the iterator simply yielding no references. This patch fixes the regression by checkign for this specific case.
* | Merge pull request #4641 from pks-t/pks/submodule-names-memleakPatrick Steinhardt2018-06-061-29/+31
|\ \ | |/ |/| Detect duplicated submodules for the same path
| * submodule: remove useless mask computationsPatrick Steinhardt2018-05-301-18/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previous to dfda2f68e (submodule: remove the per-repo cache, 2015-04-27), we tried to cache our submodules per repository to avoid having to reload it too frequently. As it created some headaches with regards to multithreading, we removed that cache. Previous to that removal, we had to compute what submodule status to refresh. The mask computation was not removed, though, resulting in confusing and actually dead code. While it seems like the mask is currently in use in a conditional, it is not, as we unconditionally assign to the mask previous to that condition. Remove all mask computations to clean up stale code.
| * submodule: refactor loading submodule namesPatrick Steinhardt2018-05-301-12/+24
| | | | | | | | | | | | | | | | | | The function `load_submodule_names` was always being called with a newly allocated string map, which was then getting filled by the function. Move the string map allocation into `load_submodule_names`, instead, and pass the whole map back to the caller in case no error occurs. This change helps to avoid misuse by handing in pre-populated maps.
| * submodule: detect duplicated submodule pathsPatrick Steinhardt2018-05-301-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | When loading submodule names, we build a map of submodule paths and their respective names. While looping over the configuration keys, we do not check though whether a submodule path was seen already. This leads to a memory leak in case we have multiple submodules with the same path, as we just overwrite the old value in the map in that case. Fix the error by verifying that the path to be added is not yet part of the string map. Git does not allow to have multiple submodules for a path anyway, so we now do the same and detect this duplication, reporting it to the user.
* | Merge pull request #4661 from laomaiweng/patch-1Patrick Steinhardt2018-05-301-1/+1
|\ \ | |/ |/| streams: openssl: add missing check on OPENSSL_LEGACY_API
| * streams: openssl: add missing check on OPENSSL_LEGACY_APIQuentin Minster2018-05-301-1/+1
| | | | | | The `CRYPTO_THREADID` type is no longer available in OpenSSL ≥ 1.1.0 with deprecated features disabled, and causes build failures. Since the `threadid_cb()` function is only ever called by `git_openssl_set_locking()` when `defined(OPENSSL_LEGACY_API)`, only define it then.
* | Merge pull request #4656 from tiennou/fix/mbedtls-no-pkgconfigPatrick Steinhardt2018-05-301-2/+4
|\ \ | |/ |/| mbedtls: don't require mbedtls from our pkgconfig file
| * mbedtls: don't require mbedtls from our pkgconfig fileEtienne Samson2018-05-251-2/+4
| | | | | | | | mbedTLS has no pkgconfig file, hence we can't require it. For now, pass its link flags as our own.
* | Merge pull request #4660 from libgit2/cmn/submodule-traversalCarlos Martín Nieto2018-05-298-42/+357
|\ \ | |/ |/| Fixes for CVE 2018-11235
| * submodule: plug leaks from the escape detectionCarlos Martín Nieto2018-05-241-3/+10
| |
| * submodule: replace index with strchr which exists on WindowsCarlos Martín Nieto2018-05-241-1/+1
| |
| * submodule: the repostiory for _name_is_valid should not be constCarlos Martín Nieto2018-05-242-4/+3
| | | | | | | | | | We might modify caches due to us trying to load the configuration to figure out what kinds of filesystem protections we should have.
| * path: check for a symlinked .gitmodules in fs-agnostic codeCarlos Martín Nieto2018-05-231-8/+32
| | | | | | | | | | We still compare case-insensitively to protect more thoroughly as we don't know what specifics we'll see on the system and it's the behaviour from git.
| * path: reject .gitmodules as a symlinkCarlos Martín Nieto2018-05-237-18/+28
| | | | | | | | | | | | | | | | Any part of the library which asks the question can pass in the mode to have it checked against `.gitmodules` being a symlink. This is particularly relevant for adding entries to the index from the worktree and for checking out files.
| * index: stat before creating the entryCarlos Martín Nieto2018-05-231-7/+30
| | | | | | | | | | This is so we have it available for the path validity checking. In a later commit we will start rejecting `.gitmodules` files as symlinks.
| * path: accept the name length as a parameterCarlos Martín Nieto2018-05-222-36/+43
| | | | | | | | | | We may take in names from the middle of a string so we want the caller to let us know how long the path component is that we should be checking.
| * path: expose dotgit detection functions per filesystemCarlos Martín Nieto2018-05-222-3/+84
| | | | | | | | | | These will be used by the checkout code to detect them for the particular filesystem they're on.
| * path: hide the dotgit file functionsCarlos Martín Nieto2018-05-181-0/+21
| | | | | | | | | | These can't go into the public API yet as we don't want to introduce API or ABI changes in a security release.
| * path: add functions to detect .gitconfig and .gitattributesCarlos Martín Nieto2018-05-181-0/+10
| |
| * path: add a function to detect an .gitmodules fileCarlos Martín Nieto2018-05-181-0/+13
| | | | | | | | | | | | | | | | Given a path component it knows what to pass to the filesystem-specific functions so we're protected even from trees which try to use the 8.3 naming rules to get around us matching on the filename exactly. The logic and test strings come from the equivalent git change.
| * path: provide a generic function for checking dogit files on NTFSCarlos Martín Nieto2018-05-181-0/+53
| | | | | | | | | | It checks against the 8.3 shortname variants, including the one which includes the checksum as part of its name.
| * path: provide a generic dogit checking function for HFSCarlos Martín Nieto2018-05-181-6/+19
| | | | | | | | This lets us check for other kinds of reserved files.
| * submodule: also validate Windows-separated paths for validityCarlos Martín Nieto2018-05-141-9/+28
| | | | | | | | | | | | | | | | Otherwise we would also admit `..\..\foo\bar` as a valid path and fail to protect Windows users. Ideally we would check for both separators without the need for the copied string, but this'll get us over the RCE.
| * submodule: ignore submodules which include path traversal in their nameCarlos Martín Nieto2018-05-092-3/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | If the we decide that the "name" of the submodule (i.e. its path inside `.git/modules/`) is trying to escape that directory or otherwise trick us, we ignore the configuration for that submodule. This leaves us with a half-configured submodule when looking it up by path, but it's the same result as if the configuration really were missing. The name check is potentially more strict than it needs to be, but it lets us re-use the check we're doing for the checkout. The function that encapsulates this logic is ready to be exported but we don't want to do that in a security release so it remains internal for now.
* | Merge pull request #4642 from pks-t/pks/cmake-resolve-pkgconfigPatrick Steinhardt2018-05-091-10/+2
|\ \ | | | | | | cmake: resolve libraries found by pkg-config
| * | cmake: remove now-useless LIBGIT2_LIBDIRS handlingPatrick Steinhardt2018-05-091-6/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With the recent change of always resolving pkg-config libraries to their full path, we do not have to manage the LIBGIT2_LIBDIRS variable anymore. The only other remaining user of LIBGIT2_LIBDIRS is winhttp, which is a CMake-style library target and can thus be resolved by CMake automatically. Remove the variable to simplify our build system a bit.
| * | cmake: resolve libraries found by pkg-configPatrick Steinhardt2018-05-091-4/+2
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Libraries found by CMake modules are usually handled with their full path. This makes linking against those libraries a lot more robust when it comes to libraries in non-standard locations, as otherwise we might mix up libraries from different locations when link directories are given. One excemption are libraries found by PKG_CHECK_MODULES. Instead of returning libraries with their complete path, it will return the variable names as well as a set of link directories. In case where multiple sets of the same library are installed in different locations, this can lead the compiler to link against the wrong libraries in the end, when link directories of other dependencies are added. To fix this shortcoming, we need to manually resolve library paths returned by CMake against their respective library directories. This is an easy task to do with `FIND_LIBRARY`.
* | Merge pull request #4629 from neithernut/enhance-glob-perfPatrick Steinhardt2018-05-091-3/+30
|\ \ | | | | | | refdb_fs: enhance performance of globbing
| * | refdb_fs: enable root arbitration for fixed portion of globsJulian Ganz2018-04-271-0/+24
| | | | | | | | | | | | | | | | | | | | | A glob used for iteration may start with an entire path containing no special characters. If we start scanning for references within that path rather than in `refs/`, we may end up scanning only a small fraction of all references.
| * | refdb_fs: prepare arbitration of the root used for ref iterationJulian Ganz2018-04-271-3/+6
| | | | | | | | | | | | | | | Instead of a hardcoded "refs", we may choose a different directory within the git directory as the root from which we look for references.
* | | Merge pull request #4645 from pks-t/pks/racy-init-deinitPatrick Steinhardt2018-05-091-10/+11
|\ \ \ | | | | | | | | global: adjust init count under lock
| * | | global: adjust init count under lockPatrick Steinhardt2018-05-041-10/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Our global initialization functions `git_libgit2_init()` and `git_libgit2_shutdown()` both adjust a global init counter to determine whether we are the first respectively last user of libgit2. On Unix-systems do not do so under lock, though, which opens the possibility of a race between these two functions: Thread 1 Thread 2 git__n_inits = 0; git_libgit2_init(); git_atomic_inc(&git__n_inits); /* git__n_inits == 1 */ git_libgit2_shutdown(); if (git_atomic_dec(&git__n_inits) != 0) /* git__n_inits == 0, no early exit here */ pthread_mutex_lock(&_init_mutex); shutdown_common(); pthread_mutex_unlock(&_init_mutex); pthread_mutex_lock(&_init_mutex); init_once(); pthread_mutex_unlock(&_init_mutex); So we can end up in a situation where we try to shutdown shared data structures before they have been initialized. Fix the race by always locking `_init_mutex` before incrementing or decrementing `git__n_inits`.
* | | | Merge pull request #4646 from pks-t/pks/gcc-8.1-warningsPatrick Steinhardt2018-05-091-1/+2
|\ \ \ \ | | | | | | | | | | Fix GCC 8.1 warnings
| * | | | streams: openssl: fix bogus warning on unused parameterPatrick Steinhardt2018-05-041-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Our provided callback function `threadid_cb(CRYPTO_THREADID *threadid)` sets up a unique thread ID by asking pthread for the current thread ID. Since openssl version 1.1, `CRYPTO_THREADID_set_numeric` is simply a no-op macro, leaving the `threadid` argument unused after the preprocessor has processed the macro. GCC does not account for that situation and will thus complain about `threadid` being unused. Silence this warning by using `GIT_UNUSED(threadid)`.
View Lorry Controller Status