<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/libgit2.git/src/alloc.h, branch ethomson/proxy</title>
<subtitle>github.com: libgit2/libgit2.git
</subtitle>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/'/>
<entry>
<title>alloc: make memory allocators use function pointers</title>
<updated>2018-06-07T10:57:39+00:00</updated>
<author>
<name>Patrick Steinhardt</name>
<email>ps@pks.im</email>
</author>
<published>2018-03-20T14:23:49+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=9865cd1696ac4a3f47991a9a1d79b17cef5edc89'/>
<id>9865cd1696ac4a3f47991a9a1d79b17cef5edc89</id>
<content type='text'>
Currently, our memory allocators are being redirected to the correct
implementation at compile time by simply using macros. In order to make
them swappable at runtime, this commit reshuffles that by instead making
use of a global "git_allocator" structure, whose pointers are set up to
reference the allocator functions. Like this, it becomes easy to swap
out allocators by simply setting these function pointers.

In order to initialize a "git_allocator", our provided allocators
"stdalloc" and "crtdbg" both provide an init function. This is being
called to initialize a passed in allocator struct and set up its members
correctly.

No support is yet included to enable users of libgit2 to switch out the
memory allocator at a global level.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Currently, our memory allocators are being redirected to the correct
implementation at compile time by simply using macros. In order to make
them swappable at runtime, this commit reshuffles that by instead making
use of a global "git_allocator" structure, whose pointers are set up to
reference the allocator functions. Like this, it becomes easy to swap
out allocators by simply setting these function pointers.

In order to initialize a "git_allocator", our provided allocators
"stdalloc" and "crtdbg" both provide an init function. This is being
called to initialize a passed in allocator struct and set up its members
correctly.

No support is yet included to enable users of libgit2 to switch out the
memory allocator at a global level.
</pre>
</div>
</content>
</entry>
<entry>
<title>stdalloc: extend allocators by file and line</title>
<updated>2018-06-07T10:57:39+00:00</updated>
<author>
<name>Patrick Steinhardt</name>
<email>ps@pks.im</email>
</author>
<published>2018-03-14T10:43:00+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=08b318c0645b27393a70c2289a65b949ea4b45ed'/>
<id>08b318c0645b27393a70c2289a65b949ea4b45ed</id>
<content type='text'>
Our desired architecture would make allocators completely pluggable,
such that users of libgit2 can swap out memory allocators at runtime.
While making e.g. debugging easier by not having to do a separate build,
this feature can also help maintainers of bindings for libgit2 by tying
the memory allocations into the other language's memory system.

In order to do so, though, we first need to make our two different
pre-existing allocators "stdalloc" and "crtdbg" have the same function
signatures, as the "crtdbg" allocators all have an additional file and
line argument. This is required to build correct stack traces for
debugging memory allocations. As that feature may also be interesting to
authors of other applications for debugging libgit2, we now simply add
these arguments to our standard allocators.

Obviously, this may come with a performance penalty. During some simple
benchmarks no real impact could be measured though in contrast to a
simple pluggable allocator. The following table summarizes the
benchmarks. There were three different builds with our current standard
allocator ("standard"), with pluggable authenticators accessed via
function pointers ("pluggable") and for pluggable authenticators with
file and line being added ("fileline"). Furthermore, there were three
scenarios for 100.000.000 allocations of 100B ("small alloc"),
100.000.000 allocations of 100KB ("medium alloc"), and 1.000.000
allocations of 100MB. All results are best of 10 runs.

|------------|-------------------|-------------------|-------------------|
| build/test | small alloc       | medium alloc      | big alloc         |
|------------|-------------------|-------------------|-------------------|
| standard   | 4539779566, +0.0% | 5912927186, +0.0% | 5166935308, +0.0% |
|------------|-------------------|-------------------|-------------------|
| pluggable  | 4611074505, +1.5% | 5979185308, +1.1% | 5388776352, +4.2% |
|------------|-------------------|-------------------|-------------------|
| fileline   | 4588338192, +1.1% | 6004951910, +1.5% | 4942528135, -4.4% |
|------------|-------------------|-------------------|-------------------|

As can be seen, there is a performance overhead for pluggable
allocators. Furthermore, it can also be seen that there is some big
variance between runs, especially in the "big alloc" scenario. This is
probably being caused by nondeterministic behaviour in the kernel for
dynamic allocations. Still, it can be observed that there should be no
real difference between the "pluggable" and "fileline" allocators.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Our desired architecture would make allocators completely pluggable,
such that users of libgit2 can swap out memory allocators at runtime.
While making e.g. debugging easier by not having to do a separate build,
this feature can also help maintainers of bindings for libgit2 by tying
the memory allocations into the other language's memory system.

In order to do so, though, we first need to make our two different
pre-existing allocators "stdalloc" and "crtdbg" have the same function
signatures, as the "crtdbg" allocators all have an additional file and
line argument. This is required to build correct stack traces for
debugging memory allocations. As that feature may also be interesting to
authors of other applications for debugging libgit2, we now simply add
these arguments to our standard allocators.

Obviously, this may come with a performance penalty. During some simple
benchmarks no real impact could be measured though in contrast to a
simple pluggable allocator. The following table summarizes the
benchmarks. There were three different builds with our current standard
allocator ("standard"), with pluggable authenticators accessed via
function pointers ("pluggable") and for pluggable authenticators with
file and line being added ("fileline"). Furthermore, there were three
scenarios for 100.000.000 allocations of 100B ("small alloc"),
100.000.000 allocations of 100KB ("medium alloc"), and 1.000.000
allocations of 100MB. All results are best of 10 runs.

|------------|-------------------|-------------------|-------------------|
| build/test | small alloc       | medium alloc      | big alloc         |
|------------|-------------------|-------------------|-------------------|
| standard   | 4539779566, +0.0% | 5912927186, +0.0% | 5166935308, +0.0% |
|------------|-------------------|-------------------|-------------------|
| pluggable  | 4611074505, +1.5% | 5979185308, +1.1% | 5388776352, +4.2% |
|------------|-------------------|-------------------|-------------------|
| fileline   | 4588338192, +1.1% | 6004951910, +1.5% | 4942528135, -4.4% |
|------------|-------------------|-------------------|-------------------|

As can be seen, there is a performance overhead for pluggable
allocators. Furthermore, it can also be seen that there is some big
variance between runs, especially in the "big alloc" scenario. This is
probably being caused by nondeterministic behaviour in the kernel for
dynamic allocations. Still, it can be observed that there should be no
real difference between the "pluggable" and "fileline" allocators.
</pre>
</div>
</content>
</entry>
<entry>
<title>util: extract allocators into its own "alloc.h" header</title>
<updated>2018-06-07T10:57:39+00:00</updated>
<author>
<name>Patrick Steinhardt</name>
<email>ps@pks.im</email>
</author>
<published>2018-03-14T10:36:14+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/libgit2.git/commit/?id=d2e996fa17c0c39afe32a703a88b5f1f7b256c1c'/>
<id>d2e996fa17c0c39afe32a703a88b5f1f7b256c1c</id>
<content type='text'>
Our "util.h" header is a grabbag of various different functions, where
many don't have a clear group they belong to. Our set of allocator
functions though can be clearly singled out as a single group of
functions that always belongs together. Furthermore, we will need to
implement additional functions relating to our allocators subsystem when
moving to pluggable allocators. Thus, we should just move these
functions into their own "alloc" module.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Our "util.h" header is a grabbag of various different functions, where
many don't have a clear group they belong to. Our set of allocator
functions though can be clearly singled out as a single group of
functions that always belongs together. Furthermore, we will need to
implement additional functions relating to our allocators subsystem when
moving to pluggable allocators. Thus, we should just move these
functions into their own "alloc" module.
</pre>
</div>
</content>
</entry>
</feed>
