summaryrefslogtreecommitdiff
path: root/doc/debuginfod.8
Commit message (Collapse)AuthorAgeFilesLines
* debuginfod.8: Tweak wording of fdcache operation & parameters.Frank Ch. Eigler2022-05-091-17/+19
| | | | Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* PR28661: debuginfo connection thread pool supportFrank Ch. Eigler2021-12-101-0/+17
| | | | | | | | | | | | Add an option -C, which activates libmicrohttpd's thread-pool mode for handling incoming http connections. Add libmicrohttpd error-logging callback function so as to receive indication of its internal errors, and relay counts to our metrics. Some of these internal errors tipped us off to a microhttpd bug that thread pooling works around. Document in debuginfod.8 page. Hand-tested against "ulimit -u NNN" shells, and with a less strenuous new test case. Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* PR28430: debuginfod: support --passive modeFrank Ch. Eigler2021-11-101-19/+35
| | | | | | | | Add support for a limited mode for debuginfod that uses a pure read-only sqlite index. This mode is useful for load spreading based on naively shared or replicated databases. Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* debuginfod: PR27277 - Describe retrieved files when verboseNoah Sanci2021-09-161-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | Allow users, with enough verbosity, to print the HTTP response headers upon retrieving a file. These files may include several custome http response headers such as X-DEBUGINFOD-FILE, X-DEBUGINFOD-SIZE, and X-DEBUGINFOD-ARCHIVE. These headers are added from the daemon, in debuginfod.cxx. E.g output: HTTP/1.1 200 OK Connection: Keep-Alive Content-Length: 4095072 Cache-Control: public Last-Modified: Thu, 09 Sep 2021 19:06:40 GMT X-FILE: debuginfod X-FILE-SIZE: 4095072 Content-Type: application/octet-stream Date: Fri, 10 Sep 2021 16:38:06 GMT https://sourceware.org/bugzilla/show_bug.cgi?id=27277 Signed-off-by: Noah Sanci <nsanci@redhat.com>
* debuginfod: Turn -d ":memory:" into -d "file::memory:?cache=shared"Mark Wielaard2021-08-281-2/+4
| | | | | | | | | | | | | debuginfod opens the database twice, once in read/wrote and once in read-only mode. This means the magic ":memory:" in-memory database cannot be used as is because the two connections don't really share the underlying database. Fix this by turning ":memory:" into ":file::memory:?cache=shared" which makes the in-memory database shared. See https://sqlite.org/inmemorydb.html Document this in debuginfod.8 and make some tests use -d :memory: Signed-off-by: Mark Wielaard <mark@klomp.org>
* debuginfod: PR27917 - protect against federation loopsDi Chen2021-08-271-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | If someone misconfigures a debuginfod federation to have loops, and a nonexistent buildid lookup is attempted, bad things will happen, as is documented. This patch aims to reduce the risk by adding an option to debuginfod that functions kind of like an IP packet's TTL: a limit on the length of XFF: header that debuginfod is willing to process. If X-Forwarded-For: exceeds N hops, it will not delegate a local lookup miss to upstream debuginfods. Commit ab38d167c40c99 causes federation loops for non-existent resources to result in multiple temporary deadlocks, each lasting for $DEBUGINFOD_TIMEOUT seconds. Since concurrent requests for each unique resource are now serialized, federation loops can result in one server thread waiting to acquire a lock while the server thread holding the lock waits for the first thread to respond to an http request. This PR can help protect against the above multiple temporary deadlocks behaviour. Ex. if --forwarded-ttl-limit=0 then the timeout behaviour of local loops should be avoided. https://sourceware.org/bugzilla/show_bug.cgi?id=27917 Signed-off-by: Di Chen <dichen@redhat.com>
* debuginfod-doc: PR27950 - Remove redanduncies in man page.Alice Zhang2021-08-111-42/+3
| | | | | | | | | | | | Create a new file, debuginfod-client-config.7, that holds all environment variables and cache control files related info. Get rid of repetitive definitions in three other files, instead, those files will include the content of new file. Any future modification related to environment variables and cache files will only require changes in one file. Signed-off-by: Alice Zhang <alizhang@redhat.com> Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* debuginfod: PR28034 - client-side %-escape url charactersNoah Sanci2021-07-221-0/+4
| | | | | | | | | | | | | | | | | | | When requesting some source files, some URL-inconvenient chars sometimes pop up. Example from f33 libstdc++: /buildid/44d8485cb75512c2ca5c8f70afbd475cae30af4f/source/usr/src/debug/ gcc-10.3.1-1.fc33.x86_64/obj-x86_64-redhat-linux/x86_64-redhat-linux/ libstdc++-v3/src/c++11/../../../../../libstdc++-v3/src/c++11/ condition_variable.cc As this URL is passed into debuginfod's handler_cb, it appears that the + signs are helpfully unescaped to spaces by libmicrohttpd, which 'course breaks everything. In order to ensure the server properly parses urls such as this one, %-escape characters on the client side so that the correct url is preserved and properly processed on the server side. https://sourceware.org/bugzilla/show_bug.cgi?id=28034 Signed-off-by: Noah Sanci <nsanci@redhat.com>
* debuginfod: PR25978 - Created the prefetch fdcacheNoah2021-07-141-0/+10
| | | | | | | | | | | | | | | The debuginfod fdcache-prefetch logic has been observed to show some degeneracies in operation. Since fdcache evictions are done frequently, and freshly prefetched archive elements are put at the back of lru[], each eviction round can summarily nuke things that were just prefetched .... and are just going to be prefetched again. It would be better to have two lru lists, or being able to insert newly prefetched entries somewhere in the middle of the list rather than at the very very end. https://sourceware.org/bugzilla/show_bug.cgi?id=25978 Signed-off-by: Noah Sanci <nsanci@redhat.com>
* debuginfod: PR27711 - Use -I/-X regexes during groom phaseNoah2021-07-081-0/+3
| | | | | | | | | | | | | | | | | | | | | The debuginfod -I/-X regexes operate during traversal to identify those files in need of scanning. The regexes are not used during grooming. This means that if from run to run, the regex changes so that formerly indexed files are excluded from traversal, the data is still retained in the index. This is both good and bad. On one hand, if the underlying data is still available, grooming will preserve the data, and let clients ask for it. On the other hand, if the growing index size is a problem, and one wishes to age no-longer-regex-matching index data out, there is no way. Let's add a debuginfod flag to use regexes during grooming. Specifically, in groom(), where the stat() test exists, also check for regex matching as in scan_source_paths(). Treat failure of the regex the same way as though the file didn't exist. Signed-off-by: Noah Sanci <nsanci@redhat.com>
* doc/debuginfod.8: fix wrong parameter name for .deb files in descriptionFrank Ch. Eigler2021-04-041-1/+1
| | | | | | | One place in the description referred to the parameter for .deb/.ddeb files as "-D" as opposed to the correct "-U". Signed-off-by: Andreas Ziegler <andreas.ziegler@fau.de>
* PR27092: debuginfod low-memory handlingFrank Ch. Eigler2021-02-051-0/+11
| | | | | | | | | | | | | | | | | | | | | | A couple of closely related pieces of work allow more early warning about low storage/memory conditions: - New prometheus metrics to track filesystem freespace, and more details about some errors. - Frequent checking of $TMPDIR freespace, to trigger fdcache emergency flushes. - Switch to floating point prometheus metrics, to communicate fractions - and short time intervals - accurately. - Fix startup-time pthread-creation error handling. Testing is smoke-test-level only as it is hard to create free-space-limited $TMPDIRs. Locally tested against tiny through medium tmpfs filesystems, with or without sqlite db also there. Shows a pleasant stream of diagnostics and metrics during shortage but generally does not fail outright. However, catching an actual libstdc++- or kernel-level OOM is beyond our ken. Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* doc: fix spelling typosDmitry V. Levin2020-12-111-1/+1
| | | | | | | | | | | filesytem -> filesystem requries -> requires hte -> the recieve -> receive varients -> variants preceeded -> preceded Signed-off-by: Dmitry V. Levin <ldv@altlinux.org>
* PR26775: make grooming progress visible & interruptibleFrank Ch. Eigler2020-10-301-2/+4
| | | | | | | | | | | On very large servers, it's desirable to be able to interrupt a rescan or groom cycle. SIGUSR[12] now do that. (Unfortunately, this is not practically testable in the testsuite, since these cycles are so fast on that small dataset.) We also expose more internal progress count about the grooming pass, so the administrator can assess possible need to interrupt. Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* PR25728: disable debuginfod --port=0Mark Wielaard2020-03-281-3/+4
| | | | | | | | | | | | When starting debuginfod with --port=0 it would start using a random port (possibly different for ipv4 and ipv6). This seems to be an accidental and not very useful functionality. Just produce an error when started with --port=0. https://sourceware.org/bugzilla/show_bug.cgi?id=25728 Signed-off-by: Mark Wielaard <mark@klomp.org> Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* debuginfod: document and workaround fedora31 zstd compressionFrank Ch. Eigler2020-03-261-1/+4
| | | | | | | | | | Old enough (even RHEL8 era) rpm/libarchive tools cannot grok fedora31 zstd-compressed rpms. Disable those tests if necessary. Document -Z based workaround for these debuginfod users. https://fedoraproject.org/wiki/Changes/Switch_RPMs_to_zstd_compression Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* PR25548: support canonicalized source-path names in debuginfod webapiFrank Ch. Eigler2020-03-261-4/+5
| | | | | | | | | | | | | Programs are sometimes compiled with source path names containing noise like /./ or // or /foo/../, and these survive into DWARF. This code allows either raw or canonicalized pathnames in the webapi, by letting the client pass things verbatim, and letting the server store/accept both raw and canonicalized path names for source files. Tests included & docs updated. Signed-off-by: Frank Ch. Eigler <fche@redhat.com> Reported-by: Eli Schwartz <eschwartz@archlinux.org> Tested-by: Eli Schwartz <eschwartz@archlinux.org>
* debuginfod-client: default to XDG cache.Aaron Merey2020-02-281-3/+7
| | | | | | | | | | | | PR25502: debuginfod client should default to XDG cache Location of client cache now defaults to $XDG_CACHE_HOME/debuginfod_client. If XDG_CACHE_HOME is not set then fallback to $HOME/.cache/debuginfod_client. Also maintain backwards compatibility with the previous default path- if $HOME/.debuginfod_client_cache exists, use that instead of the new defaults. Signed-off-by: Aaron Merey <amerey@redhat.com>
* debuginfod PR25583: map -R to -Z.rpmFrank Ch. Eigler2020-02-251-1/+2
| | | | | | | | | | It was reported that libarchive (bsdtar) at least as far back as rhel7 (3.1.2) can natively process RPM files, so there's no need to mediate those accesses through rpm2cpio. There's no noteworthy performance or testing impact. Signed-off-by: Frank Ch. Eigler <fche@redhat.com> Signed-off-by: Mark Wielaard <mark@klomp.org>
* PR25375: fdcache prefetching to reduce repeated archive decompressionFrank Ch. Eigler2020-02-251-7/+10
| | | | | | | Introduce new option --fdcache-prefetch to accelerate repeated queries from the same debuginfo archive. Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* debuginfod: generalized archive supportFrank Ch. Eigler2020-02-101-15/+32
| | | | | | | | | Add a '-Z EXT[=CMD]' option to debuginfod, which lets it scan any given extension and run CMD on it to unwrap distro archives. For example, for arch-linux pacman files, -Z '.tar.zst=zstdcat' lets debuginfod grok debug and source content in split-debuginfo files. Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* debuginfod: extracted-from-archive file cacheFrank Ch. Eigler2020-01-111-0/+10
| | | | | | | | | | | Add a facility to service webapi and dwz/altdebug requests that resolve to archives via a $TMPDIR file cache. This permits instantaneous dwz resolution during -debuginfo rpm scanning, and also instantanous duplicate webapi requests. The cache is limited both in number of entries and in storage space. Heuristics provide serviceable defaults. Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* debuginfod: rework threading model for file/archive scanningFrank Ch. Eigler2020-01-111-46/+50
| | | | | | | | | | We switch from a thread per supplied PATH, with a semaphore based concurrency control, to a fixed number of worker threads collecting the result of a plain directory traversal being put into a work queue. This allows maximal continuous concurrency, even if the PATH directories are dramatically differently sized. There is no more need to use concurrency-motivated subdirectory wildcards for PATH entries: just a single top level directory will work fast. doc & tests incl.
* debuginfod: Use DEBUGINFOD_TIMEOUT as seconds to get at least 100K.Mark Wielaard2020-01-101-2/+4
| | | | | | | Use just one timeout using CURLOPT_LOW_SPEED_TIME (default 90 seconds) and CURLOPT_LOW_SPEED_LIMIT (100K). Signed-off-by: Mark Wielaard <mark@klomp.org>
* debuginfod server: support .deb/.ddeb archivesFrank Ch. Eigler2019-12-221-31/+40
| | | | | | | | | Add support for scanning .deb / .ddeb files, enabled with a new command line option "-U". Using a synthetic .deb/.ddeb from a Ubuntu 18 machine, extend the debuginfod testsuite with some .deb processing, if the dpkg-deb binary is installed. Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* debuginfod 5: add /metrics endpointFrank Ch. Eigler2019-11-231-2/+11
| | | | | | | | This webapi extensions allows admins to hook up debuginfod to a prometheus-compatible monitoring system for general situational statistics. The metrics are simple enough that local curl requests can give a user a sense of what's going on. The metrics are documented as unstable with respect to future versions.
* debuginfod 4: symbolic link traversal modeFrank Ch. Eigler2019-11-221-0/+9
| | | | | | In order to support file/rpm archives that are organized via symlink trees, add an "-L" option to debuginfod, meaning about the same as for find(1) or ls(1): to traverse rather than ignore symlinks.
* debuginfod 2/2: server sideFrank Ch. Eigler2019-11-221-0/+369
Add the server to the debuginfod/ subdirectory. This is a highly multithreaded c++11 program (still buildable on rhel7's gcc 4.8, which is only partly c++11 compliant). Includes an initial suite of tests, man pages, and a sample systemd service. Signed-off-by: Frank Ch. Eigler <fche@redhat.com> Signed-off-by: Aaron Merey <amerey@redhat.com>
View Lorry Controller Status