| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
| |
There's nothing bash-y about our build and test scripts. Use sh instead
of bash for bash-deprived systems like FreeBSD.
|
| |
|
|
| |
This reverts commit 1fe3fa5e59818c851d50efc6563db5f8a5d7ae9b.
|
| | |
|
| |\ |
|
| | | |
|
| | | |
|
| | | |
|
| |\ \
| | |
| | | |
Have git_branch_lookup accept GIT_BRANCH_ALL
|
| | | | |
|
| | | | |
|
| |\ \ \
| | | |
| | | | |
Rename git_transfer_progress to git_indexer_progress
|
| | | | |
| | | |
| | | |
| | | |
| | | | |
Safely deprecate `git_push_transfer_progress`, forwarding it to the new
`git_push_transfer_progress_cb` name.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
The `git_push_transfer_progress` is a callback and as such should be
suffixed with `_cb` for consistency. Rename
`git_push_transfer_progress` to `git_push_transfer_progress_cb`.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Safely deprecate `git_transfer_progress` and `git_transfer_progress_cb`
types, forwarding them to the new `git_indexer_progress` and
`git_indexer_progress_cb`.
|
| | | | |
| | | |
| | | |
| | | |
| | | | |
Update internal usage of `git_transfer_progress` to
`git_indexer_progreses`.
|
| |/ / /
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
The name `git_transfer_progress` does not reflect its true purpose.
It suggests that it's progress for a non-existence `git_transfer`
object, and is used for indexing callbacks more broadly than just
during transfers.
Rename `git_transfer_progress` to `git_indexer_progress`.
|
| |\ \ \
| | | |
| | | | |
High-level map APIs
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Remove the low-level interface that was exposing implementation details of
`git_idxmap` to callers. From now on, only the high-level functions shall be
used to retrieve or modify values of a map. Adjust remaining existing callers.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Remove the low-level interface that was exposing implementation details of
`git_oidmap` to callers. From now on, only the high-level functions shall be
used to retrieve or modify values of a map. Adjust remaining existing callers.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Remove the low-level interface that was exposing implementation details of
`git_offmap` to callers. From now on, only the high-level functions shall be
used to retrieve or modify values of a map. Adjust remaining existing callers.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Remove the low-level interface that was exposing implementation details of
`git_strmap` to callers. From now on, only the high-level functions shall be
used to retrieve or modify values of a map. Adjust remaining existing callers.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
To relieve us from memory pressure, we may regularly call `cache_evict_entries`
to remove some entries from it. Unfortunately, our cache does not support a
least-recently-used mode or something similar, which is why we evict entries
completeley at random right now. Thing is, this is only possible due to the map
interfaces exposing the entry indices, and we intend to completely remove those
to decouple map users from map implementations. As soon as that is done, we are
unable to do this random eviction anymore.
Convert this to make use of an iterator for now. Obviously, there is no random
eviction possible like that anymore, but we'll always start by evicting from the
beginning of the map. Due to hashing, one may hope that the selected buckets
will be evicted at least in some way unpredictably. But more likely than not,
this will not be the case. But let's see what happens and if any users complain
about degraded performance. If so, we might come up with a different scheme than
random removal, e.g. by using an LRU cache.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
To compute whether there are objects missing in a packfile, the indexer keeps
around a map of OIDs that it still expects to see. This map does not store any
values at all, but in fact the keys are owned by the map itself. Right now, we
free these keys by iterating over the map and freeing the key itself, which is
kind of awkward as keys are expected to be constant.
We can make this a bit prettier by inserting the OID as value, too. As we
already store the `NULL` pointer either way, this does not increase memory
usage, but makes the code a tad more clear. Furthermore, we convert the
previously existing map iteration via indices to make use of an iterator,
instead.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Currently, our headers need to leak some implementation details of maps due to
their direct use of indices in the implementation of their foreach macros. This
makes it impossible to completely hide the map structures away, and also makes
it impossible to include the khash implementation header in the C files of the
respective map only.
This is now being fixed by providing a high-level iteration interface
`map_iterate`, which takes as inputs the map that shall be iterated over, an
iterator as well as the locations where keys and values shall be put into. For
simplicity's sake, the iterator is a simple `size_t` that shall initialized to
`0` on the first call. All existing foreach macros are then adjusted to make use
of this new function.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Some callers were still using the tightly-coupled pattern of `lookup_index` and
`valid_index` to verify that an entry exists in a map. Instead, use the more
high-level `exists` functions to decouple map users from its implementation.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Currently, the delete functions of maps do not provide a return value. Like
this, it is impossible to tell whether the entry has really been deleted or not.
Change the implementation to provide either a return value of zero if the entry
has been successfully deleted or `GIT_ENOTFOUND` if the key could not be found.
Convert callers to the `delete_at` functions to instead use this higher-level
interface.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
The currently existing function `git_idxmap_resize` and
`git_idxmap_icase_resize` do not return any error codes at all due to their
previous implementation making use of a macro. Due to that, it is impossible to
see whether the resize operation might have failed due to an out-of-memory
situation.
Fix this by providing a proper error code. Adjust callers to make use of it.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Currently, one would use the function `git_idxmap_insert` to insert key/value
pairs into a map. This function has historically been a macro, which is why its
syntax is kind of weird: instead of returning an error code directly, it instead
has to be passed a pointer to where the return value shall be stored. This does
not match libgit2's common idiom of directly returning error codes.
Introduce a new function `git_idxmap_set`, which takes as parameters the map,
key and value and directly returns an error code. Convert all callers of
`git_idxmap_insert` to make use of it.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
The current way of looking up an entry from a map is tightly coupled with the
map implementation, as one first has to look up the index of the key and then
retrieve the associated value by using the index. As a caller, you usually do
not care about any indices at all, though, so this is more complicated than
really necessary. Furthermore, it invites for errors to happen if the correct
error checking sequence is not being followed.
Introduce new high-level functions `git_idxmap_get` and `git_idxmap_icase_get`
that take a map and a key and return a pointer to the associated value if such a
key exists. Otherwise, a `NULL` pointer is returned. Adjust all callers that can
trivially be converted.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Currently, there is only one caller that adds entries into an offset map, and
this caller first uses `git_offmap_put` to add a key and then set the value at
the returned index by using `git_offmap_set_value_at`. This is just too tighlty
coupled with implementation details of the map as it exposes the index of
inserted entries, which we really do not care about at all.
Introduce a new function `git_offmap_set`, which takes as parameters the map,
key and value and directly returns an error code. Convert the caller to make use
of it instead.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
The current way of looking up an entry from a map is tightly coupled with the
map implementation, as one first has to look up the index of the key and then
retrieve the associated value by using the index. As a caller, you usually do
not care about any indices at all, though, so this is more complicated than
really necessary. Furthermore, it invites for errors to happen if the correct
error checking sequence is not being followed.
Introduce a new high-level function `git_offmap_get` that takes a map and a key
and returns a pointer to the associated value if such a key exists. Otherwise,
a `NULL` pointer is returned. Adjust all callers that can trivially be
converted.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Currently, one would use either `git_oidmap_insert` to insert key/value pairs
into a map or `git_oidmap_put` to insert a key only. These function have
historically been macros, which is why their syntax is kind of weird: instead of
returning an error code directly, they instead have to be passed a pointer to
where the return value shall be stored. This does not match libgit2's common
idiom of directly returning error codes.Furthermore, `git_oidmap_put` is tightly
coupled with implementation details of the map as it exposes the index of
inserted entries.
Introduce a new function `git_oidmap_set`, which takes as parameters the map,
key and value and directly returns an error code. Convert all trivial callers of
`git_oidmap_insert` and `git_oidmap_put` to make use of it.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
The current way of looking up an entry from a map is tightly coupled with the
map implementation, as one first has to look up the index of the key and then
retrieve the associated value by using the index. As a caller, you usually do
not care about any indices at all, though, so this is more complicated than
really necessary. Furthermore, it invites for errors to happen if the correct
error checking sequence is not being followed.
Introduce a new high-level function `git_oidmap_get` that takes a map and a key
and returns a pointer to the associated value if such a key exists. Otherwise,
a `NULL` pointer is returned. Adjust all callers that can trivially be
converted.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Currently, one would use the function `git_strmap_insert` to insert key/value
pairs into a map. This function has historically been a macro, which is why its
syntax is kind of weird: instead of returning an error code directly, it instead
has to be passed a pointer to where the return value shall be stored. This does
not match libgit2's common idiom of directly returning error codes.
Introduce a new function `git_strmap_set`, which takes as parameters the map,
key and value and directly returns an error code. Convert all callers of
`git_strmap_insert` to make use of it.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
The current way of looking up an entry from a map is tightly coupled with the
map implementation, as one first has to look up the index of the key and then
retrieve the associated value by using the index. As a caller, you usually do
not care about any indices at all, though, so this is more complicated than
really necessary. Furthermore, it invites for errors to happen if the correct
error checking sequence is not being followed.
Introduce a new high-level function `git_strmap_get` that takes a map and a key
and returns a pointer to the associated value if such a key exists. Otherwise,
a `NULL` pointer is returned. Adjust all callers that can trivially be
converted.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
There currently exist two different function names for getting the entry count
of maps, where offmaps offset and string maps use `num_entries` and OID maps use
`size`. In most programming languages with built-in map types, this is simply
called `size`, which is also shorter to type. Thus, this commit renames the
other two functions `num_entries` to match the common way and adjusts all
callers.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Currently, the lifecycle functions for maps (allocation, deallocation, resize)
are not named in a uniform way and do not have a uniform function signature.
Rename the functions to fix that, and stick to libgit2's naming scheme of saying
`git_foo_new`. This results in the following new interface for allocation:
- `int git_<t>map_new(git_<t>map **out)` to allocate a new map, returning an
error code if we ran out of memory
- `void git_<t>map_free(git_<t>map *map)` to free a map
- `void git_<t>map_clear(git<t>map *map)` to remove all entries from a map
This commit also fixes all existing callers.
|
| |\ \ \ \
| | | | |
| | | | | |
refdb_fs: fix loose/packed refs lookup racing with repacks
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Right now, we first load the packed refs cache and only afterwards load the
loose references. This is susceptible to a race when the loose ref is being
migrated to a packed cache by e.g. git-pack-refs(1):
libgit2 git-pack-refs
1. We load the packed ref, which
does not yet have the migrated
reference.
2. git-pack-refs updates the packed ref
file to have the migrated ref.
3. git-pack-refs deletes the old loose
ref.
4. We look up the loose ref.
So we now do not find the reference at all and will never iterate over it.
Fix the issue by reversing the order: instead of first loading the packed refs,
we will now look up the loose reference first. If it has already been deleted,
then it must already be present in the packed-refs by definition, as git.git
will only delete the reference after updating the packed refs file.
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Right now, loading loose refs has the side-effect of setting the
`PACKREF_SHADOWED` flag for references that exist both in the loose and the
packed refs. Because of this, we are force do first look up packed refs and only
afterwards loading the packed refs. This is susceptible to a race, though, when
refs are being repacked: when first loading the packed cache, then it may not
yet have the migrated loose ref. But when now trying to look up the loose
reference afterwards, then it may already have been migrated. Thus, we would
fail to find this reference in this scenario.
Remove this ordering dependency to allow fixing the above race. Instead of
setting the flag when loading loose refs, we will now instead set it lazily when
iterating over the loose refs. This even has the added benefit of not requiring
us to lock the packed refs cache, as we already have an owned copy of it.
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
When creating a new iterator, we eagerly load loose refs but only lazily create
a copy of packed refs. The lazy load only happens as soon as we have iterated
over all loose refs, opening up a potentially wide window for races. This
may lead to an inconsistent view e.g. when the caller decides to reload packed
references somewhen between iterating the loose refs, which is unexpected.
Fix the issue by eagerly copying the sorted cache. Note that right now, we are
heavily dependent on ordering here: we first need to reload packed refs, then we
have to load loose refs and only as a last step are we allowed to copy the
cache. This is because loading loose refs has the side-effect of setting the
`PACKED_SHADOWED` flag in the packed refs cache, which we require to avoid
outputting packed refs that already exist as loose refs.
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Refactor the error handling in `refdb_fs_backend__iterator` to always return the
correct error code returned by the failing function.
|
| | |/ / /
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
When repacking references, git.git will first update the packed refs and only
afterwards delete any existing loose references that have now been moved to the
new packed refs file. Due to this, there is a potential for racing if one first
reads the packfile (which has not been updated yet) and only then trying to read
the loose reference (which has just been deleted). In this case, one will
incorrectly fail to lookup the reference and it will be reported as missing.
Naturally, this is exactly what we've been doing in `refdb_fs_backend__exists`.
Fix the race by reversing the lookup: we will now first check if the loose
reference exists and only afterwards refresh the packed file.
|
| |\ \ \ \
| | | | |
| | | | | |
Allocator restructuring
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
In commit 6e0dfc6ff (Make stdalloc__reallocarray call
stdalloc__realloc, 2019-02-16), we have changed the stdalloc
allocator to reuse `stdalloc__realloc` to implement
`stdalloc__reallocarray`. This commit is making the same change
for the Windows-specific crtdbg allocator to avoid code
duplication.
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
The Windows-specific crtdbg allocator is currently mixed into the
crtdbg stacktracing compilation unit, making it harder to find
than necessary. Extract it and move it into the new "allocators/"
subdirectory to improve discoverability.
This change means that the crtdbg compilation unit is now
compiled unconditionally, whereas it has previously only been
compiled on Windows platforms. Thus we now have additional guards
around the code so that it will only be compiled if
GIT_MSVC_CRTDBG is defined. This also allows us to move over the
fallback-implementation of `git_win32_crtdbg_init_allocator` into
the same compilation unit.
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Right now, our two allocator implementations are scattered around
the tree in "stdalloc.h" and "win32/w32_crtdbg_stacktrace.h".
Start grouping them together in a single directory "allocators/",
similar to how e.g. our streams are organized.
|
| |\ \ \ \ \
| | | | | |
| | | | | | |
cache: fix misnaming of `git_cache_free`
|
| | |/ / / /
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Functions that free a structure's contents but not the structure
itself shall be named `dispose` in the libgit2 project, but the
function `git_cache_free` does not follow this naming pattern.
Fix this by renaming it to `git_cache_dispose` and adjusting all
callers to make use of the new name.
|
| |\ \ \ \ \
| |/ / / /
|/| | | | |
examples: produce single cgit2 binary
|
