<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/libgit2.git/src/odb.h, branch peff/binary-search-do-while</title>
<subtitle>github.com: libgit2/libgit2.git
</subtitle>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/'/>
<entry>
<title>odb: add option to turn off hash verification</title>
<updated>2017-04-28T12:05:45+00:00</updated>
<author>
<name>Patrick Steinhardt</name>
<email>ps@pks.im</email>
</author>
<published>2017-04-21T07:31:56+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=35079f507b5287a2a4726cae7753439f753453e2'/>
<id>35079f507b5287a2a4726cae7753439f753453e2</id>
<content type='text'>
Verifying hashsums of objects we are reading from the ODB may be costly
as we have to perform an additional hashsum calculation on the object.
Especially when reading large objects, the penalty can be as high as
35%, as can be seen when executing the equivalent of `git cat-file` with
and without verification enabled. To mitigate for this, we add a global
option for libgit2 which enables the developer to turn off the
verification, e.g. when he can be reasonably sure that the objects on
disk won't be corrupted.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Verifying hashsums of objects we are reading from the ODB may be costly
as we have to perform an additional hashsum calculation on the object.
Especially when reading large objects, the penalty can be as high as
35%, as can be seen when executing the equivalent of `git cat-file` with
and without verification enabled. To mitigate for this, we add a global
option for libgit2 which enables the developer to turn off the
verification, e.g. when he can be reasonably sure that the objects on
disk won't be corrupted.
</pre>
</div>
</content>
</entry>
<entry>
<title>odb: verify object hashes</title>
<updated>2017-04-28T12:05:45+00:00</updated>
<author>
<name>Patrick Steinhardt</name>
<email>ps@pks.im</email>
</author>
<published>2017-04-10T07:30:08+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=28a0741f1ae6f5e1261c8e73854dda69e7a61067'/>
<id>28a0741f1ae6f5e1261c8e73854dda69e7a61067</id>
<content type='text'>
The upstream git.git project verifies objects when looking them up from
disk. This avoids scenarios where objects have somehow become corrupt on
disk, e.g. due to hardware failures or bit flips. While our mantra is
usually to follow upstream behavior, we do not do so in this case, as we
never check hashes of objects we have just read from disk.

To fix this, we create a new error class `GIT_EMISMATCH` which denotes
that we have looked up an object with a hashsum mismatch. `odb_read_1`
will then, after having read the object from its backend, hash the
object and compare the resulting hash to the expected hash. If hashes do
not match, it will return an error.

This obviously introduces another computation of checksums and could
potentially impact performance. Note though that we usually perform I/O
operations directly before doing this computation, and as such the
actual overhead should be drowned out by I/O. Running our test suite
seems to confirm this guess. On a Linux system with best-of-five
timings, we had 21.592s with the check enabled and 21.590s with the
ckeck disabled. Note though that our test suite mostly contains very
small blobs only. It is expected that repositories with bigger blobs may
notice an increased hit by this check.

In addition to a new test, we also had to change the
odb::backend::nonrefreshing test suite, which now triggers a hashsum
mismatch when looking up the commit "deadbeef...". This is expected, as
the fake backend allocated inside of the test will return an empty
object for the OID "deadbeef...", which will obviously not hash back to
"deadbeef..." again. We can simply adjust the hash to equal the hash of
the empty object here to fix this test.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The upstream git.git project verifies objects when looking them up from
disk. This avoids scenarios where objects have somehow become corrupt on
disk, e.g. due to hardware failures or bit flips. While our mantra is
usually to follow upstream behavior, we do not do so in this case, as we
never check hashes of objects we have just read from disk.

To fix this, we create a new error class `GIT_EMISMATCH` which denotes
that we have looked up an object with a hashsum mismatch. `odb_read_1`
will then, after having read the object from its backend, hash the
object and compare the resulting hash to the expected hash. If hashes do
not match, it will return an error.

This obviously introduces another computation of checksums and could
potentially impact performance. Note though that we usually perform I/O
operations directly before doing this computation, and as such the
actual overhead should be drowned out by I/O. Running our test suite
seems to confirm this guess. On a Linux system with best-of-five
timings, we had 21.592s with the check enabled and 21.590s with the
ckeck disabled. Note though that our test suite mostly contains very
small blobs only. It is expected that repositories with bigger blobs may
notice an increased hit by this check.

In addition to a new test, we also had to change the
odb::backend::nonrefreshing test suite, which now triggers a hashsum
mismatch when looking up the commit "deadbeef...". This is expected, as
the fake backend allocated inside of the test will return an empty
object for the OID "deadbeef...", which will obviously not hash back to
"deadbeef..." again. We can simply adjust the hash to equal the hash of
the empty object here to fix this test.
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge pull request #4030 from libgit2/ethomson/fsync</title>
<updated>2017-03-22T20:29:22+00:00</updated>
<author>
<name>Edward Thomson</name>
<email>ethomson@edwardthomson.com</email>
</author>
<published>2017-03-22T20:29:22+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=6fd6c67824d2caad9f8716a676820d8319f79a9d'/>
<id>6fd6c67824d2caad9f8716a676820d8319f79a9d</id>
<content type='text'>
fsync all the things</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
fsync all the things</pre>
</div>
</content>
</entry>
<entry>
<title>git_commit_create: freshen tree objects in commit</title>
<updated>2017-03-03T14:12:00+00:00</updated>
<author>
<name>Edward Thomson</name>
<email>ethomson@github.com</email>
</author>
<published>2017-03-03T13:26:29+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=52d03f37f70ebde3e0a7794a60dbf68d0e69b9e8'/>
<id>52d03f37f70ebde3e0a7794a60dbf68d0e69b9e8</id>
<content type='text'>
Freshen the tree object that a commit points to during commit time.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Freshen the tree object that a commit points to during commit time.
</pre>
</div>
</content>
</entry>
<entry>
<title>Honor `core.fsyncObjectFiles`</title>
<updated>2017-03-02T09:11:33+00:00</updated>
<author>
<name>Edward Thomson</name>
<email>ethomson@github.com</email>
</author>
<published>2017-02-28T12:29:29+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=1c04a96b25da048221f31ecee0227d960dc00489'/>
<id>1c04a96b25da048221f31ecee0227d960dc00489</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>odb: improved not found error messages</title>
<updated>2016-03-07T15:20:01+00:00</updated>
<author>
<name>Edward Thomson</name>
<email>ethomson@github.com</email>
</author>
<published>2016-03-04T06:18:30+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=e10144ae57e50798e43515cb469722a4e825c23c'/>
<id>e10144ae57e50798e43515cb469722a4e825c23c</id>
<content type='text'>
When looking up an abbreviated oid, show the actual (abbreviated) oid
the caller passed instead of a full (but ambiguously truncated) oid.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
When looking up an abbreviated oid, show the actual (abbreviated) oid
the caller passed instead of a full (but ambiguously truncated) oid.
</pre>
</div>
</content>
</entry>
<entry>
<title>odb: make the writestream's size a git_off_t</title>
<updated>2015-05-13T07:34:20+00:00</updated>
<author>
<name>Carlos Martín Nieto</name>
<email>cmn@dwim.me</email>
</author>
<published>2015-05-12T11:06:33+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=77b339f7b6c7e167ecaf9374eb6876b498d8cb83'/>
<id>77b339f7b6c7e167ecaf9374eb6876b498d8cb83</id>
<content type='text'>
Restricting files to size_t is a silly limitation. The loose backend
writes to a file directly, so there is no issue in using 63 bits for the
size.

We still assume that the header is going to fit in 64 bytes, which does
mean quite a bit smaller files due to the run-length encoding, but it's
still a much larger size than you would want Git to handle.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Restricting files to size_t is a silly limitation. The loose backend
writes to a file directly, so there is no issue in using 63 bits for the
size.

We still assume that the header is going to fit in 64 bytes, which does
mean quite a bit smaller files due to the run-length encoding, but it's
still a much larger size than you would want Git to handle.
</pre>
</div>
</content>
</entry>
<entry>
<title>Create public filter object and use it</title>
<updated>2013-09-17T16:30:06+00:00</updated>
<author>
<name>Russell Belfer</name>
<email>rb@github.com</email>
</author>
<published>2013-08-28T23:44:04+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=85d5481206a932d747b2d5587b6d4c7f69993ba6'/>
<id>85d5481206a932d747b2d5587b6d4c7f69993ba6</id>
<content type='text'>
This creates include/sys/filter.h with a basic definition of a
git_filter and then converts the internal code to use it.  There
are related internal objects (git_filter_list) that we will want
to publish at some point, but this is a first step.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This creates include/sys/filter.h with a basic definition of a
git_filter and then converts the internal code to use it.  There
are related internal objects (git_filter_list) that we will want
to publish at some point, but this is a first step.
</pre>
</div>
</content>
</entry>
<entry>
<title>Add callback to git_objects_table</title>
<updated>2013-04-22T14:51:40+00:00</updated>
<author>
<name>Russell Belfer</name>
<email>rb@github.com</email>
</author>
<published>2013-04-15T07:05:44+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=786062639f05e361da977f3f1f6286141fa12fca'/>
<id>786062639f05e361da977f3f1f6286141fa12fca</id>
<content type='text'>
This adds create and free callback to the git_objects_table so
that more of the creation and destruction of objects can be table
driven instead of using switch statements.  This also makes the
semantics of certain object creation functions consistent so that
we can make better use of function pointers.  This also fixes a
theoretical error case where an object allocation fails and we
end up storing NULL into the cache.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This adds create and free callback to the git_objects_table so
that more of the creation and destruction of objects can be table
driven instead of using switch statements.  This also makes the
semantics of certain object creation functions consistent so that
we can make better use of function pointers.  This also fixes a
theoretical error case where an object allocation fails and we
end up storing NULL into the cache.
</pre>
</div>
</content>
</entry>
<entry>
<title>What has science done.</title>
<updated>2013-04-22T14:50:50+00:00</updated>
<author>
<name>Vicent Marti</name>
<email>tanoku@gmail.com</email>
</author>
<published>2013-04-03T20:30:07+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=8842c75f172ed94be4ad11521d4083e97d740785'/>
<id>8842c75f172ed94be4ad11521d4083e97d740785</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
</feed>
