| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
| |
Like we want to separate libgit2 and utility source code, we want to
separate libgit2 and utility tests. Start by moving all the tests into
libgit2.
|
| | |
|
| |
|
|
|
| |
Users may want to validate raw object content; provide them a function
to do so.
|
| | |
|
| |
|
|
|
|
| |
Introduce `git_fs_path`, which operates on generic filesystem paths.
`git_path` will be kept for only git-specific path functionality (for
example, checking for `.git` in a path).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
libgit2 has two distinct requirements that were previously solved by
`git_buf`. We require:
1. A general purpose string class that provides a number of utility APIs
for manipulating data (eg, concatenating, truncating, etc).
2. A structure that we can use to return strings to callers that they
can take ownership of.
By using a single class (`git_buf`) for both of these purposes, we have
confused the API to the point that refactorings are difficult and
reasoning about correctness is also difficult.
Move the utility class `git_buf` to be called `git_str`: this represents
its general purpose, as an internal string buffer class. The name also
is an homage to Junio Hamano ("gitstr").
The public API remains `git_buf`, and has a much smaller footprint. It
is generally only used as an "out" param with strict requirements that
follow the documentation. (Exceptions exist for some legacy APIs to
avoid breaking callers unnecessarily.)
Utility functions exist to convert a user-specified `git_buf` to a
`git_str` so that we can call internal functions, then converting it
back again.
|
| |
|
|
|
|
| |
Separate the concerns of the hash functions from the git_oid functions.
The git_oid structure will need to understand either SHA1 or SHA256; the
hash functions should only deal with the appropriate one of these.
|
| | |
|
| |
|
|
|
| |
Instead of a `git_bom_t` that a `git_buf` function returns, let's keep
it `git_buf_bom_t`.
|
| |
|
|
|
| |
The `git_buf_text` namespace is unnecessary and strange. Remove it,
just keep the functions prefixed with `git_buf`.
|
| | |
|
| | |
|
| |
|
|
|
|
| |
We _dispose_ the contents of objects; we _free_ objects (and their
contents). Update `git_strarray_free` to be `git_strarray_dispose`.
`git_strarray_free` remains as a deprecated proxy function.
|
| |
|
|
|
|
|
|
|
| |
In our test case object::cache::fast_thread_rush, we're creating 100
concurrent threads opening a repository and reading objects from it.
This test actually fails on ARM32 with an out-of-memory error, which
isn't entirely unexpected.
Work around the issue by halving the number of threads.
|
| | |
|
| |
|
|
|
| |
Ensure that the new protection around .git::$INDEX_ALLOCATION rules are
enabled for using the treebuilder when core.protectNTFS is set.
|
| |
|
|
| |
Prefer `off64_t` internally.
|
| |
|
|
|
|
|
|
|
| |
Our file utils functions all have a "futils" prefix, e.g.
`git_futils_touch`. One would thus naturally guess that their
definitions and implementation would live in files "futils.h" and
"futils.c", respectively, but in fact they live in "fileops.h".
Rename the files to match expectations.
|
| |
|
|
|
| |
Don't calculate 4 GiB as that will produce a compiler warning on MSVC.
Just hardcode it.
|
| |
|
|
|
| |
Don't try to feed 4 GB of data to APIs that only take a `size_t` on
32-bit platforms.
|
| |
|
|
|
|
| |
The majority of functions are named `from_something` (with an
underscore) instead of `fromsomething`. Update the index functions for
consistency with the rest of the library.
|
| |
|
|
|
|
| |
The majority of functions are named `from_something` (with an
underscore) instead of `fromsomething`. Update the blob functions for
consistency with the rest of the library.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The `p_fallocate` platform is currently in use in our tests,
only, but it proved to be quite burdensome to get it implemented
in a cross-platform way. The only "real" user is the test
object::tree::read::largefile, where it's used to allocate a
large file in the filesystem only to commit it to the repo and
read its object back again. We can simplify this quite a bit by
just using an in-memory buffer of 4GB. Sure, this cannot be used
on platforms with low resources. But creating 4GB files is not
any better, and we already skip the test if the environment
variable "GITTEST_INVASIVE_FS_SIZE" is not set. So we're arguably
not worse off than before.
|
| |
|
|
|
|
|
|
|
|
|
| |
The object::cache test module has two tests that do nearly the
same thing: given a cache limit, load a certain set of objects
and verify if those objects have been cached or not.
Convert those tests to the new data-driven initializers to
demonstrate how these are to be used. Furthermore, add some
additional test data. This conversion is mainly done to show this
new facility.
|
| | |
|
| | |
|
| |
|
|
|
| |
Move to the `git_error` name in the internal API for error-related
functions.
|
| |
|
|
|
|
|
| |
We use the term "invalid" to refer to bad or malformed data, eg
`GIT_REF_INVALID` and `GIT_EINVALIDSPEC`. Since we're changing the
names of the `git_object_t`s in this release, update it to be
`GIT_OBJECT_INVALID` instead of `BAD`.
|
| |
|
|
| |
Use the new object_type enumeration names within the codebase.
|
| |
|
|
| |
Use the new-style index names throughout our own codebase.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The `parse_mode` option uses an open-coded octal number parser. The
parser is quite naive in that it simply parses until hitting a character
that is not in the accepted range of '0' - '7', completely ignoring the
fact that we can at most accept a 16 bit unsigned integer as filemode.
If the filemode is bigger than UINT16_MAX, it will thus overflow and
provide an invalid filemode for the object entry.
Fix the issue by using `git__strntol32` instead and doing a bounds
check. As this function already handles overflows, it neatly solves the
problem.
Note that previously, `parse_mode` was also skipping the character
immediately after the filemode. In proper trees, this should be a simple
space, but in fact the parser accepted any character and simply skipped
over it. As a consequence of using `git__strntol32`, we now need to an
explicit check for a trailing whitespace after having parsed the
filemode. Because of the newly introduced error message, the test
object::tree::parse::mode_doesnt_cause_oob_read needs adjustment to its
error message check, which in fact is a good thing as it demonstrates
that we now fail looking for the whitespace immediately following the
filemode.
Add a test that shows that we will fail to parse such invalid filemodes
now.
|
| |
|
|
|
|
|
|
|
|
|
| |
When parsing a tree entry's mode, we will eagerly parse until we hit a
character that is not in the accepted set of octal digits '0' - '7'. If
the provided buffer is not a NUL terminated one, we may thus read
out-of-bounds.
Fix the issue by passing the buffer length to `parse_mode` and paying
attention to it. Note that this is not a vulnerability in our usual code
paths, as all object data read from the ODB is NUL terminated.
|
| |
|
|
|
|
|
|
|
|
|
| |
We currently don't have any tests that directly exercise the tree
parser. This is due to the fact that the parsers for raw object data has
only been recently introduce with commit ca4db5f4a (object: implement
function to parse raw data, 2017-10-13), and previous to that the setup
simply was too cumbersome as it always required going through the ODB.
Now that we have the infrastructure, add a suite of tests that directly
exercise the tree parser and various edge cases.
|
| |
|
|
|
|
|
|
|
|
|
| |
The commit message encoding is currently being parsed by the
`git__prefixcmp` function. As this function does not accept a buffer
length, it will happily skip over a buffer's end if it is not `NUL`
terminated.
Fix the issue by using `git__prefixncmp` instead. Add a test that
verifies that we are unable to parse the encoding field if it's cut off
by the supplied buffer length.
|
| |
|
|
|
|
| |
We currently do not have any test suites dedicated to parsing commits
from their raw representations. Add one based on `git_object__from_raw`
to be able to test special cases more easily.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When parsing tags, we skip all unknown fields that appear before the tag
message. This skipping is done by using a plain `strstr(buffer, "\n\n")`
to search for the two newlines that separate tag fields from tag
message. As it is not possible to supply a buffer length to `strstr`,
this call may skip over the buffer's end and thus result in an out of
bounds read. As `strstr` may return a pointer that is out of bounds, the
following computation of `buffer_end - buffer` will overflow and result
in an allocation of an invalid length.
Fix the issue by using `git__memmem` instead. Add a test that verifies
parsing the tag fails not due to the allocation failure but due to the
tag having no message.
|
| |
|
|
|
|
|
|
| |
While the tests in object::tag::read exercises reading and parsing valid
tags from the ODB, they barely try to verify that the parser fails in a
sane way when parsing invalid tags. Create a new test suite
object::tag::parse that directly exercise the parser by using
`git_object__from_raw` and add various tests for valid and invalid tags.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
When we add entries to a treebuilder we validate them. But we validate even
those that we're adding because they exist in the base tree. This disables
using the normal mechanisms on these trees, even to fix them.
Keep track of whether the entry we're appending comes from an existing tree and
bypass the name and id validation if it's from existing data.
|
| |
|
|
|
|
|
|
|
| |
C++ style comment ("//") are not specified by the ISO C90 standard and
thus do not conform to it. While libgit2 aims to conform to C90, we did
not enforce it until now, which is why quite a lot of these
non-conforming comments have snuck into our codebase. Do a tree-wide
conversion of all C++ style comments to the supported C style comments
to allow us enforcing strict C90 compliance in a later commit.
|
| | |
|
| |
|
|
|
|
| |
Instead of laving it uninitialized and relying on luck for it to be non-zero,
let's give it a dummy hash so we make valgrind happy (in this case the hash
comes from `sha1sum </dev/null`.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
In commit a96d3cc3f (cache-tree: reject entries with null sha1,
2017-04-21), the git.git project has changed its stance on null OIDs in
tree objects. Previously, null OIDs were accepted in tree entries to
help tools repair broken history. This resulted in some problems though
in that many code paths mistakenly passed null OIDs to be added to a
tree, which was not properly detected.
Align our own code base according to the upstream change and reject
writing tree entries early when the OID is all-zero.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
Verifying hashsums of objects we are reading from the ODB may be costly
as we have to perform an additional hashsum calculation on the object.
Especially when reading large objects, the penalty can be as high as
35%, as can be seen when executing the equivalent of `git cat-file` with
and without verification enabled. To mitigate for this, we add a global
option for libgit2 which enables the developer to turn off the
verification, e.g. when he can be reasonably sure that the objects on
disk won't be corrupted.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The upstream git.git project verifies objects when looking them up from
disk. This avoids scenarios where objects have somehow become corrupt on
disk, e.g. due to hardware failures or bit flips. While our mantra is
usually to follow upstream behavior, we do not do so in this case, as we
never check hashes of objects we have just read from disk.
To fix this, we create a new error class `GIT_EMISMATCH` which denotes
that we have looked up an object with a hashsum mismatch. `odb_read_1`
will then, after having read the object from its backend, hash the
object and compare the resulting hash to the expected hash. If hashes do
not match, it will return an error.
This obviously introduces another computation of checksums and could
potentially impact performance. Note though that we usually perform I/O
operations directly before doing this computation, and as such the
actual overhead should be drowned out by I/O. Running our test suite
seems to confirm this guess. On a Linux system with best-of-five
timings, we had 21.592s with the check enabled and 21.590s with the
ckeck disabled. Note though that our test suite mostly contains very
small blobs only. It is expected that repositories with bigger blobs may
notice an increased hit by this check.
In addition to a new test, we also had to change the
odb::backend::nonrefreshing test suite, which now triggers a hashsum
mismatch when looking up the commit "deadbeef...". This is expected, as
the fake backend allocated inside of the test will return an empty
object for the OID "deadbeef...", which will obviously not hash back to
"deadbeef..." again. We can simply adjust the hash to equal the hash of
the empty object here to fix this test.
|
| |
|
|
|
|
| |
We currently have no tests which check whether we fail reading corrupted
objects. Add one which modifies contents of an object stored on disk and
then tries to read the object.
|
| |
|
|
|
|
|
|
|
|
|
| |
The object::lookup tests do use the "testrepo.git" repository in a
read-only way, so we do not set up the repository as a sandbox but
simply open it. But in a future commit, we will want to test looking up
objects which are corrupted in some way, which requires us to modify the
on-disk data. Doing this in a repository without creating the sandbox
will modify contents of our libgit2 repository, though.
Create the repository in a sandbox to avoid this.
|
| |
|
|
|
| |
We do not currently use the sorted version of this input in the
function, which means we produce bad results.
|
| | |
|
