<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/libgit2.git/tests/pack, branch ethomson/codeql</title>
<subtitle>github.com: libgit2/libgit2.git
</subtitle>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/'/>
<entry>
<title>tests: ifdef out unused function in no-thread builds</title>
<updated>2020-12-05T22:07:49+00:00</updated>
<author>
<name>Edward Thomson</name>
<email>ethomson@edwardthomson.com</email>
</author>
<published>2020-12-05T22:03:59+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=0450e3134b543e047eb391e0a26bce760c62ebb1'/>
<id>0450e3134b543e047eb391e0a26bce760c62ebb1</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Make the pack and mwindow implementations data-race-free</title>
<updated>2020-11-29T03:40:56+00:00</updated>
<author>
<name>lhchavez</name>
<email>lhchavez@lhchavez.com</email>
</author>
<published>2020-08-02T01:24:41+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=322c15ee858622f2e3def514d3e7e1b47023950e'/>
<id>322c15ee858622f2e3def514d3e7e1b47023950e</id>
<content type='text'>
This change fixes a packfile heap corruption that can happen when
interacting with multiple packfiles concurrently across multiple
threads. This is exacerbated by setting a lower mwindow open file limit.

This change:

* Renames most of the internal methods in pack.c to clearly indicate
  that they expect to be called with a certain lock held, making
  reasoning about the state of locks a bit easier.
* Splits the `git_pack_file` lock in two: the one in `git_pack_file`
  only protects the `index_map`. The protection to `git_mwindow_file` is
  now in that struct.
* Explicitly checks for freshness of the `git_pack_file` in
  `git_packfile_unpack_header`: this allows the mwindow implementation
  to close files whenever there is enough cache pressure, and
  `git_packfile_unpack_header` will reopen the packfile if needed.
* After a call to `p_munmap()`, the `data` and `len` fields are poisoned
  with `NULL` to make use-after-frees more evident and crash rather than
  being open to the possibility of heap corruption.
* Adds a test case to prevent this from regressing in the future.

Fixes: #5591
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change fixes a packfile heap corruption that can happen when
interacting with multiple packfiles concurrently across multiple
threads. This is exacerbated by setting a lower mwindow open file limit.

This change:

* Renames most of the internal methods in pack.c to clearly indicate
  that they expect to be called with a certain lock held, making
  reasoning about the state of locks a bit easier.
* Splits the `git_pack_file` lock in two: the one in `git_pack_file`
  only protects the `index_map`. The protection to `git_mwindow_file` is
  now in that struct.
* Explicitly checks for freshness of the `git_pack_file` in
  `git_packfile_unpack_header`: this allows the mwindow implementation
  to close files whenever there is enough cache pressure, and
  `git_packfile_unpack_header` will reopen the packfile if needed.
* After a call to `p_munmap()`, the `data` and `len` fields are poisoned
  with `NULL` to make use-after-frees more evident and crash rather than
  being open to the possibility of heap corruption.
* Adds a test case to prevent this from regressing in the future.

Fixes: #5591
</pre>
</div>
</content>
</entry>
<entry>
<title>midx: Support multi-pack-index files in odb_pack.c</title>
<updated>2020-11-27T11:40:02+00:00</updated>
<author>
<name>lhchavez</name>
<email>lhchavez@lhchavez.com</email>
</author>
<published>2020-02-16T02:00:56+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=f847fa7b34abdc7850b8739747e1d25eefafa5f2'/>
<id>f847fa7b34abdc7850b8739747e1d25eefafa5f2</id>
<content type='text'>
This change adds support for reading multi-pack-index files from the
packfile odb backend. This also makes git_pack_file objects open their
backing failes lazily in more scenarios, since the multi-pack-index can
avoid having to open them in some cases (yay!).

This change also refreshes the documentation found in src/odb_pack.c to
match the updated code.

Part of: #5399
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change adds support for reading multi-pack-index files from the
packfile odb backend. This also makes git_pack_file objects open their
backing failes lazily in more scenarios, since the multi-pack-index can
avoid having to open them in some cases (yay!).

This change also refreshes the documentation found in src/odb_pack.c to
match the updated code.

Part of: #5399
</pre>
</div>
</content>
</entry>
<entry>
<title>runtime: move init/shutdown into the "runtime"</title>
<updated>2020-10-11T19:13:04+00:00</updated>
<author>
<name>Edward Thomson</name>
<email>ethomson@edwardthomson.com</email>
</author>
<published>2020-05-15T10:47:09+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=e316b0d3d64eb8f65f4109c1565d929b29e1d33a'/>
<id>e316b0d3d64eb8f65f4109c1565d929b29e1d33a</id>
<content type='text'>
Provide a mechanism for system components to register for initialization
and shutdown of the libgit2 runtime.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Provide a mechanism for system components to register for initialization
and shutdown of the libgit2 runtime.
</pre>
</div>
</content>
</entry>
<entry>
<title>multipack: Introduce a parser for multi-pack-index files</title>
<updated>2020-10-05T12:08:38+00:00</updated>
<author>
<name>lhchavez</name>
<email>lhchavez@lhchavez.com</email>
</author>
<published>2020-02-23T22:28:52+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=005e77157d5eef9d9c0765ff201e6ec07e7f5d00'/>
<id>005e77157d5eef9d9c0765ff201e6ec07e7f5d00</id>
<content type='text'>
This change is the first in a series to add support for git's
multi-pack-index. This should speed up large repositories significantly.

Part of: #5399
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change is the first in a series to add support for git's
multi-pack-index. This should speed up large repositories significantly.

Part of: #5399
</pre>
</div>
</content>
</entry>
<entry>
<title>Minor nits and style formatting</title>
<updated>2020-07-12T16:53:10+00:00</updated>
<author>
<name>lhchavez</name>
<email>lhchavez@lhchavez.com</email>
</author>
<published>2020-07-12T16:53:10+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=92d42eb3d83a28febbbb50df7c398e32677da28a'/>
<id>92d42eb3d83a28febbbb50df7c398e32677da28a</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Create the repository within the test</title>
<updated>2020-06-27T14:38:02+00:00</updated>
<author>
<name>lhchavez</name>
<email>lhchavez@lhchavez.com</email>
</author>
<published>2020-06-27T14:34:36+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=d88994da6058f4dc8260e3c16898ad7ab1b61855'/>
<id>d88994da6058f4dc8260e3c16898ad7ab1b61855</id>
<content type='text'>
This change moves the humongous static repository with 1025 commits into
the test file, now with a more modest 16 commits.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change moves the humongous static repository with 1025 commits into
the test file, now with a more modest 16 commits.
</pre>
</div>
</content>
</entry>
<entry>
<title>Review feedback</title>
<updated>2020-06-26T23:45:25+00:00</updated>
<author>
<name>lhchavez</name>
<email>lhchavez@lhchavez.com</email>
</author>
<published>2020-06-26T23:10:30+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=eab2b0448e47801090c50fe9bb9e72e377f0317a'/>
<id>eab2b0448e47801090c50fe9bb9e72e377f0317a</id>
<content type='text'>
* Change the default of the file limit to 0 (unlimited).
* Changed the heuristic to close files to be the file that contains the
  least-recently-used window such that the window is the
  most-recently-used in the file, and the file does not have in-use
  windows.
* Parameterized the filelimit test to check for a limit of 1 and 100
  open windows.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Change the default of the file limit to 0 (unlimited).
* Changed the heuristic to close files to be the file that contains the
  least-recently-used window such that the window is the
  most-recently-used in the file, and the file does not have in-use
  windows.
* Parameterized the filelimit test to check for a limit of 1 and 100
  open windows.
</pre>
</div>
</content>
</entry>
<entry>
<title>mwindow: set limit on number of open files</title>
<updated>2020-06-21T14:15:26+00:00</updated>
<author>
<name>lhchavez</name>
<email>lhchavez@lhchavez.com</email>
</author>
<published>2020-02-08T20:47:24+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=9679df573604571d1372d2850116e66e1f4cec23'/>
<id>9679df573604571d1372d2850116e66e1f4cec23</id>
<content type='text'>
There are some cases in which repositories accrue a large number of
packfiles. The existing mwindow limit applies only to the total size of
mmap'd files, not on their number. This leads to a situation in which
having lots of small packfiles could exhaust the allowed number of open
files, particularly on macOS, where the default ulimit is very low
(256).

This change adds a new configuration parameter
(GIT_OPT_SET_MWINDOW_FILE_LIMIT) that sets the maximum number of open
packfiles, with a default of 128. This is low enough so that even macOS
users should not hit it during normal use.

Based on PR #5386, originally written by @josharian.

Fixes: #2758
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
There are some cases in which repositories accrue a large number of
packfiles. The existing mwindow limit applies only to the total size of
mmap'd files, not on their number. This leads to a situation in which
having lots of small packfiles could exhaust the allowed number of open
files, particularly on macOS, where the default ulimit is very low
(256).

This change adds a new configuration parameter
(GIT_OPT_SET_MWINDOW_FILE_LIMIT) that sets the maximum number of open
packfiles, with a default of 128. This is low enough so that even macOS
users should not hit it during normal use.

Based on PR #5386, originally written by @josharian.

Fixes: #2758
</pre>
</div>
</content>
</entry>
<entry>
<title>git_packbuilder_write: Allow setting path to NULL to use the default path</title>
<updated>2020-05-23T23:07:54+00:00</updated>
<author>
<name>Josh Triplett</name>
<email>josh@joshtriplett.org</email>
</author>
<published>2020-05-23T23:07:54+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=5278a0061100e2527e83f4cb159b40fc58bc786c'/>
<id>5278a0061100e2527e83f4cb159b40fc58bc786c</id>
<content type='text'>
If given a NULL path, write to the object path of the repository.

Add tests for the new behavior.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
If given a NULL path, write to the object path of the repository.

Add tests for the new behavior.
</pre>
</div>
</content>
</entry>
</feed>
