<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/libgit2.git/src/mwindow.h, branch ethomson/ssl_refactor</title>
<subtitle>github.com: libgit2/libgit2.git
</subtitle>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/'/>
<entry>
<title>Make the pack and mwindow implementations data-race-free</title>
<updated>2020-11-29T03:40:56+00:00</updated>
<author>
<name>lhchavez</name>
<email>lhchavez@lhchavez.com</email>
</author>
<published>2020-08-02T01:24:41+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=322c15ee858622f2e3def514d3e7e1b47023950e'/>
<id>322c15ee858622f2e3def514d3e7e1b47023950e</id>
<content type='text'>
This change fixes a packfile heap corruption that can happen when
interacting with multiple packfiles concurrently across multiple
threads. This is exacerbated by setting a lower mwindow open file limit.

This change:

* Renames most of the internal methods in pack.c to clearly indicate
  that they expect to be called with a certain lock held, making
  reasoning about the state of locks a bit easier.
* Splits the `git_pack_file` lock in two: the one in `git_pack_file`
  only protects the `index_map`. The protection to `git_mwindow_file` is
  now in that struct.
* Explicitly checks for freshness of the `git_pack_file` in
  `git_packfile_unpack_header`: this allows the mwindow implementation
  to close files whenever there is enough cache pressure, and
  `git_packfile_unpack_header` will reopen the packfile if needed.
* After a call to `p_munmap()`, the `data` and `len` fields are poisoned
  with `NULL` to make use-after-frees more evident and crash rather than
  being open to the possibility of heap corruption.
* Adds a test case to prevent this from regressing in the future.

Fixes: #5591
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This change fixes a packfile heap corruption that can happen when
interacting with multiple packfiles concurrently across multiple
threads. This is exacerbated by setting a lower mwindow open file limit.

This change:

* Renames most of the internal methods in pack.c to clearly indicate
  that they expect to be called with a certain lock held, making
  reasoning about the state of locks a bit easier.
* Splits the `git_pack_file` lock in two: the one in `git_pack_file`
  only protects the `index_map`. The protection to `git_mwindow_file` is
  now in that struct.
* Explicitly checks for freshness of the `git_pack_file` in
  `git_packfile_unpack_header`: this allows the mwindow implementation
  to close files whenever there is enough cache pressure, and
  `git_packfile_unpack_header` will reopen the packfile if needed.
* After a call to `p_munmap()`, the `data` and `len` fields are poisoned
  with `NULL` to make use-after-frees more evident and crash rather than
  being open to the possibility of heap corruption.
* Adds a test case to prevent this from regressing in the future.

Fixes: #5591
</pre>
</div>
</content>
</entry>
<entry>
<title>mwindow: use GIT_ASSERT</title>
<updated>2020-11-27T11:09:20+00:00</updated>
<author>
<name>Edward Thomson</name>
<email>ethomson@edwardthomson.com</email>
</author>
<published>2020-04-05T16:18:20+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=a3e8b7cd3e49bdcffc9d9d5bda1d134a448932e2'/>
<id>a3e8b7cd3e49bdcffc9d9d5bda1d134a448932e2</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>mwindow: localize mutex</title>
<updated>2020-10-11T13:43:37+00:00</updated>
<author>
<name>Edward Thomson</name>
<email>ethomson@edwardthomson.com</email>
</author>
<published>2020-05-13T09:48:13+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=8aa69f887afaced1389d1ae965d5f313b10da66b'/>
<id>8aa69f887afaced1389d1ae965d5f313b10da66b</id>
<content type='text'>
Move the mwindow mutex into the mwindow code itself, initializing it in
the mwindow global initialization function instead of in the global
initializer.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Move the mwindow mutex into the mwindow code itself, initializing it in
the mwindow global initialization function instead of in the global
initializer.
</pre>
</div>
</content>
</entry>
<entry>
<title>internal: use off64_t instead of git_off_t</title>
<updated>2019-11-25T02:18:29+00:00</updated>
<author>
<name>Edward Thomson</name>
<email>ethomson@edwardthomson.com</email>
</author>
<published>2019-06-23T17:13:29+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=6460e8abcfda7692af9bd267bddf6e11a78f8130'/>
<id>6460e8abcfda7692af9bd267bddf6e11a78f8130</id>
<content type='text'>
Prefer `off64_t` internally.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Prefer `off64_t` internally.
</pre>
</div>
</content>
</entry>
<entry>
<title>Make sure to always include "common.h" first</title>
<updated>2017-07-03T08:51:48+00:00</updated>
<author>
<name>Patrick Steinhardt</name>
<email>ps@pks.im</email>
</author>
<published>2017-06-30T11:39:01+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=0c7f49dd4316b332f30b4ea72a657bace41e1245'/>
<id>0c7f49dd4316b332f30b4ea72a657bace41e1245</id>
<content type='text'>
Next to including several files, our "common.h" header also declares
various macros which are then used throughout the project. As such, we
have to make sure to always include this file first in all
implementation files. Otherwise, we might encounter problems or even
silent behavioural differences due to macros or defines not being
defined as they should be. So in fact, our header and implementation
files should make sure to always include "common.h" first.

This commit does so by establishing a common include pattern. Header
files inside of "src" will now always include "common.h" as its first
other file, separated by a newline from all the other includes to make
it stand out as special. There are two cases for the implementation
files. If they do have a matching header file, they will always include
this one first, leading to "common.h" being transitively included as
first file. If they do not have a matching header file, they instead
include "common.h" as first file themselves.

This fixes the outlined problems and will become our standard practice
for header and source files inside of the "src/" from now on.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Next to including several files, our "common.h" header also declares
various macros which are then used throughout the project. As such, we
have to make sure to always include this file first in all
implementation files. Otherwise, we might encounter problems or even
silent behavioural differences due to macros or defines not being
defined as they should be. So in fact, our header and implementation
files should make sure to always include "common.h" first.

This commit does so by establishing a common include pattern. Header
files inside of "src" will now always include "common.h" as its first
other file, separated by a newline from all the other includes to make
it stand out as special. There are two cases for the implementation
files. If they do have a matching header file, they will always include
this one first, leading to "common.h" being transitively included as
first file. If they do not have a matching header file, they instead
include "common.h" as first file themselves.

This fixes the outlined problems and will become our standard practice
for header and source files inside of the "src/" from now on.
</pre>
</div>
</content>
</entry>
<entry>
<title>mwindow: init mwindow files in git_libgit2_init</title>
<updated>2016-08-04T14:30:48+00:00</updated>
<author>
<name>Edward Thomson</name>
<email>ethomson@github.com</email>
</author>
<published>2016-08-03T21:01:48+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=2381d9e4900050f879cedf851c0329440db7c5e3'/>
<id>2381d9e4900050f879cedf851c0329440db7c5e3</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>pack: clean up error returns</title>
<updated>2014-06-25T19:35:58+00:00</updated>
<author>
<name>Carlos Martín Nieto</name>
<email>cmn@dwim.me</email>
</author>
<published>2014-06-25T19:35:58+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=c19b1c0442b52e07f1c9f48b65ef0b3823eaa526'/>
<id>c19b1c0442b52e07f1c9f48b65ef0b3823eaa526</id>
<content type='text'>
Set a message when we fail to lock.

Also make the put function void, since it's called from free, which
cannot report errors. The only errors we can experience here are
internal state corruption, so we assert that we are trying to put a
pack which we have previously got.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Set a message when we fail to lock.

Also make the put function void, since it's called from free, which
cannot report errors. The only errors we can experience here are
internal state corruption, so we assert that we are trying to put a
pack which we have previously got.
</pre>
</div>
</content>
</entry>
<entry>
<title>Share packs across repository instances</title>
<updated>2014-06-23T19:50:36+00:00</updated>
<author>
<name>Carlos Martín Nieto</name>
<email>cmn@dwim.me</email>
</author>
<published>2014-06-18T15:13:12+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=b3b66c57930358467395fc3a5bca87edefd25cf4'/>
<id>b3b66c57930358467395fc3a5bca87edefd25cf4</id>
<content type='text'>
Opening the same repository multiple times will currently open the same
file multiple times, as well as map the same region of the file multiple
times. This is not necessary, as the packfile data is immutable.

Instead of opening and closing packfiles directly, introduce an
indirection and allocate packfiles globally. This does mean locking on
each packfile open, but we already use this lock for the global mwindow
list so it doesn't introduce a new contention point.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Opening the same repository multiple times will currently open the same
file multiple times, as well as map the same region of the file multiple
times. This is not necessary, as the packfile data is immutable.

Instead of opening and closing packfiles directly, introduce an
indirection and allocate packfiles globally. This does mean locking on
each packfile open, but we already use this lock for the global mwindow
list so it doesn't introduce a new contention point.
</pre>
</div>
</content>
</entry>
<entry>
<title>update copyrights</title>
<updated>2013-01-08T23:31:27+00:00</updated>
<author>
<name>Edward Thomson</name>
<email>ethomson@edwardthomson.com</email>
</author>
<published>2013-01-08T23:07:25+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=359fc2d241ac407bdf9bf0d28715705f01ca6360'/>
<id>359fc2d241ac407bdf9bf0d28715705f01ca6360</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>git_mwindow_file_deregister() shouldn't return errors</title>
<updated>2013-01-06T02:27:24+00:00</updated>
<author>
<name>Scott J. Goldman</name>
<email>scottjg@github.com</email>
</author>
<published>2013-01-06T02:20:42+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=f9b55bcb5f9fd8c41c74d4be58ebaad13aa9b7f3'/>
<id>f9b55bcb5f9fd8c41c74d4be58ebaad13aa9b7f3</id>
<content type='text'>
As a function that appears to only be called on error paths, I don't
think it makes sense for it to return an error, or clobber the global
giterr. Note that no existing callsites actually check the return
code.

In my own application, there are errors where the real error ends
up being hidden, as git_mwindow_file_deregister() clobbers the
global giterr. I'm not sure this error is even relevant?
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
As a function that appears to only be called on error paths, I don't
think it makes sense for it to return an error, or clobber the global
giterr. Note that no existing callsites actually check the return
code.

In my own application, there are errors where the real error ends
up being hidden, as git_mwindow_file_deregister() clobbers the
global giterr. I'm not sure this error is even relevant?
</pre>
</div>
</content>
</entry>
</feed>
