summaryrefslogtreecommitdiff
path: root/debuginfod
Commit message (Collapse)AuthorAgeFilesLines
* PR30316: debuginfod wal checkpointingusers/fche/try-filecompFrank Ch. Eigler2023-05-091-1/+113
| | | | | | | | | | | | Add a "--scan-checkpoint=NUM" option to debuginfod to control forced synchronization & sqlite -wal checkpointing for the multithreaded scanning process. In absence of this, a server that's busy with other read & write operations can accumulate potentially large SQLITE WAL temporary files. This option causes the server to take intermittent quiescent breaks during scanning, during which the -wal file can be processed and truncated. Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* debuginfod: PR30378: better compression for _files tableFrank Ch. Eigler2023-05-091-81/+180
| | | | | | | | | | | | | | | | | | | Split the _files table into two links into a new _fileparts table, interning the dirname and basename of each file name string. This reduces storage requirements for many thousands of almost-identical long paths that are evident in large builds like kernels. This is unfortunately a schema-breaking change, so requires reindexing of the corpus. While in the vicinity, the file scan work queue is changed from a <set> to an <unordered_set>. The intent is that files be scanned in a more random sequence instead of sorted. If they're sorted, then files that contain errors will tend to be retried over and over again at the next scan cycle, even at the expense of making progress on the other files in the queue. Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* PR30377: fix debuginfod -r -X combinationFrank Ch. Eigler2023-04-222-1/+5
| | | | | | | | Until this fix, debuginfod -r -X '.*' didn't trigger groom-time removal of everything, because the -I include regex overrode it. Corrected logic to match the scan-time tie-breaking between -I / -X. Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* PR30348: debuginfod: retry partial archive scansFrank Ch. Eigler2023-04-132-0/+21
| | | | | | | | | | | | | On some public debuginfod servers, it was observed that errors may occur during individual archive scanning operations. That's fine, but previous code still went ahead and marked the archive "done" by inserting a record into the *_file_mtime_scanned table. New code ensures that exceptions propagate for these cases, and an archive that encountered an error while scanning will be retried later. Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* debuginfod: When retrieving files from cache, update atime manuallyJan Alexander Steffens (heftig)2023-03-302-0/+20
| | | | | | | | | The cache cleaning logic requires atime to be correct (strictatime or relatime) but some users on Linux only have noatime. Attempt to update the atime manually so that the cache works properly. Signed-off-by: Jan Alexander Steffens (heftig) <heftig@archlinux.org>
* debuginfod: Don't touch access time of new filesJan Alexander Steffens (heftig)2023-03-303-10/+22
| | | | | | | | Instead of copying the mtime, which might be far in the past, don't touch the access time. This will prevent cache cleaning from considering the file as old immediately. Signed-off-by: Jan Alexander Steffens (heftig) <heftig@archlinux.org>
* debuginfod-client.c: Avoid sscanf on mixed-case component of stringAaron Merey2023-03-302-2/+7
| | | | | | | | | | | sscanf is used to get the value of x-debuginfod-size from the http headers. The format string used assumes that the header field name is entirely lower case. However mixed-case field names are possible, resulting in the value not being read. Fix this by removing "x-debuginfod-size" from the format string. Signed-off-by: Aaron Merey <amerey@redhat.com>
* debuginfod: Replace futimes with futimensJan Alexander Steffens (heftig)2023-03-293-10/+14
| | | | | | | Similar to what 8c4aa0ef998191ed828a37190dc179b91649938a did for ar and strip, replace the non-standard futimes with the POSIX futimens. Signed-off-by: Jan Alexander Steffens (heftig) <heftig@archlinux.org>
* debuginfod-client.c: Fix download size not correctly fallbacks to ↵lilydjwg2023-03-292-3/+9
| | | | | | x-debuginfod-size header Signed-off-by: lilydjwg <lilydjwg@gmail.com>
* debuginfod-client.c: Skip empty file creation for cancelled queriesAaron Merey2023-03-172-3/+8
| | | | | | | | | | | | | | | | | Empty files in the client cache are used to indicate that contacted servers could not find a requested resource. Future queries for this resource will not be attempted until the cache_miss_s duration has passed. Currently these empty files are also created when a query is cancelled through the client's progressfn. This can occur, for example, when a user cancels a download with ctrl-c. This prevents user-cancelled queries from being retried promptly without having to modify cache_miss_s. Fix this by skipping the creation of an empty cache file when progressfn cancels a query. Signed-off-by: Aaron Merey <amerey@redhat.com>
* configure: Add --enable-sanitize-memoryIlya Leoshkevich2023-02-141-1/+2
| | | | | | | | | | | | | | | | | | | | Add support for clang Memory Sanitizer [1], which detects the usage of uninitialized values. While elfutils itself is already checked with valgrind, checking code that depends on elfutils requires elfutils to be built with MSan. MSan is not linked into shared libraries, and is linked into executables statically. Therefore, unlike the other sanitizers, MSan needs to be configured fairly early, since we need to drop -D_FORTIFY_SOURCE [2], -Wl,-z,defs and --no-undefined. Disable a few tests that run for more than 5 minutes due to test files being statically linked with MSan. [1] https://clang.llvm.org/docs/MemorySanitizer.html [2] https://github.com/google/sanitizers/issues/247 Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
* printversion: Fix unused variableIlya Leoshkevich2023-02-141-2/+1
| | | | | | | | | | | | | | | | | | clang complains: debuginfod.cxx:354:1: error: unused variable 'apba__' [-Werror,-Wunused-const-variable] ARGP_PROGRAM_BUG_ADDRESS_DEF = PACKAGE_BUGREPORT; ^ ../lib/printversion.h:47:21: note: expanded from macro 'ARGP_PROGRAM_BUG_ADDRESS_DEF' const char *const apba__ __asm ("argp_program_bug_address") ^ The default linkage for consts in C++ is internal, so declare them extern. While at it, remove the debuginfod workaround for argp_program_version_hook. Co-developed-by: Mark Wielaard <mark@klomp.org> Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
* debuginfod-client.c: Download section even if cached executable didn't ↵Aaron Merey2023-02-072-10/+21
| | | | | | | | | | | | | | | | | | | | | contain it. Before attempting to download a section, cache_find_section tries to extract the section from existing files in the cache. If it's determined that the section must not exist, cache_find_section returns -ENOENT to indicate that the download should be skipped. This patch fixes a bug where cache_find_section returns -ENOENT even though the section exists. If the cache contains the executable but not the debuginfo with the given build-id and the section only exists in the debuginfo (such as any of the .debug_* sections), then debuginfod_find_section returns -ENOENT even if the section could be downloaded. Fix this by having cache_find_section not return -ENOENT unless cached debuginfo was able to be read. Signed-off-by: Aaron Merey <amerey@redhat.com>
* debuginfod: Make sure there is only one typedef for debuginfod_clientMark Wielaard2023-02-062-0/+8
| | | | | | | | | | | | Both debuginfod.h and libdwfl.h have a simple typedef for struct debuginfod_client. Some compilers pedantically warn when including both headers that such typedefs are only officially supported in C11. So guard them with _ELFUTILS_DEBUGINFOD_CLIENT_TYPEDEF to make them happy. https://sourceware.org/bugzilla/show_bug.cgi?id=30077 Signed-off-by: Mark Wielaard <mark@klomp.org>
* Fix typos from codespellSam James2023-01-131-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | debuginfod/ChangeLog: * debuginfod-client.c: Typo simultaniously. doc/ChangeLog: * debuginfod.8: Typo succesfully. lib/ChangeLog: * dynamicsizehash_concurrent.c: Typo modul. * system.h: Typo dependend. libdwfl/ChangeLog: * open.c: Typo non-existant. src/ChangeLog: * nm.c: Typo Covert. * strings.c: Likewise. tests/ChangeLog: * elfstrmerge.c: Typo outselves. * run-debuginfod-extraction.sh: Typo accidentially. * run-debuginfod-fd-prefetch-caches.sh: Likewise. Signed-off-by: Sam James <sam@gentoo.org>
* debuginfod PR29975 & PR29976: decrease default concurrencyFrank Ch. Eigler2023-01-112-2/+63
| | | | | | | ... based on rlimit (rlimig -n NUM) ... based on cpu-affinity (taskset -c A,B,C,D ...) Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* debuginfod-client: Use CURLOPT_PROTOCOLS_STR for libcurl >= 7.85.0Mark Wielaard2023-01-112-0/+10
| | | | | | https://sourceware.org/bugzilla/show_bug.cgi?id=29926 Signed-off-by: Mark Wielaard <mark@klomp.org>
* debuginfod: Define CURL_AT_LEAST_VERSION if necessaryMark Wielaard2022-12-212-0/+11
| | | | | | | | Older curl.h don't define CURL_AT_LEAST_VERSION, so define it ourselves because it is nicer than doing hex encoded version comparisons. Signed-off-by: Mark Wielaard <mark@klomp.org>
* PR29926: debuginfod: Fix usage of deprecated CURLINFO_*Andrew Paprocki2022-12-212-2/+6
| | | | | | | | | | | | | | | | | The `CURLINFO_SIZE_DOWNLOAD_T` and `CURLINFO_CONTENT_LENGTH_DOWNLOAD_T` identifiers are `enum`s, not pre-processor definitions, so the current `#ifdef` logic is not selecting the newer API. This results in the older identifiers being used and they now generate errors when compiled against Curl 7.87, which has silently deprecated them, causing GCC to emit `-Werror=deprecated-declarations`. Instead, the newer identifiers were added in Curl 7.55, so explicitly check for `CURL_AT_LEAST_VERSION(7, 55, 0)` instead of the current logic. This eliminates the error when compiling against Curl 7.87. Ref: https://github.com/curl/curl/pull/1511 Signed-off-by: Andrew Paprocki <andrew@ishiboo.com>
* debuginfod: Initialize response_data early in debuginfod-client queryMark Wielaard2022-11-212-2/+7
| | | | | | | | On error going to out2, the response_data is freed. So initialize the response_data to NULL immediately after allocation or when going back to query_in_parallel. Signed-off-by: Mark Wielaard <mark@klomp.org>
* debuginfod-client.c: Don't treat 0 as an error code.Aaron Merey2022-11-072-2/+7
| | | | | | | Replace 'fd > 0' with 'fd >= 0' to avoid treating a possible file descriptor as an error code. Signed-off-by: Aaron Merey <amerey@redhat.com>
* debuginfod_find_section: Always update rc with most recent error codeAaron Merey2022-11-042-1/+10
| | | | | | | | | debuginfod_find_section may attempt to download both the debuginfo and executable matching the given build-id. If neither of these files can be found, update rc to ensure that we always return an accurate error code in this case. Signed-off-by: Aaron Merey <amerey@redhat.com>
* debuginfod.cxx: fix coverity-found use-after-release errorFrank Ch. Eigler2022-11-032-73/+75
| | | | | | | The debuginfod_client object lifetime needs more careful handling, made easier with the defer_dtor<> gadget. Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* debuginfod: Mark extract_section function staticMark Wielaard2022-11-022-1/+5
| | | | | | | The extract_section function in debuginfod-client.c is an internal function and should not be exported. Mark it as static. Signed-off-by: Mark Wielaard <mark@klomp.org>
* Changelog: Update entries from previous commits.Aaron Merey2022-11-011-0/+4
| | | | | | Update Changelogs with details from commits 04b1a3aa and 054b3bde9. Signed-off-by: Aaron Merey <amerey@redhat.com>
* debuginfod-client: Fix out-of-bounds writeAaron Merey2022-11-011-2/+1
| | | | | | | Return early from path_escape when '\0' is seen in order to prevent an out-of-bounds write to the dest buffer. Signed-off-by: Aaron Merey <amerey@redhat.com>
* debuginfod: Support queries for ELF/DWARF sectionsAaron Merey2022-11-017-96/+696
| | | | | | | | | | | | | | | | | | | | | | Add new function debuginfod_find_section which queries debuginfod servers for the raw binary contents of the specified ELF/DWARF section in a file matching the given build-id. Extend the server webapi to support section queries. Section query URLS have the following format: /buildid/BUILDID/section/SECTION The server will attempt to extract the section from a debuginfo file matching the given build-id. If the debuginfo file cannot be found or the section has type SHT_NOBITS, the server will attempt to extract the section from the executable file matching the build-id. If the server is built without section query support, the client will attempt to download the debuginfo matching the build-id and extract the section. If the debuginfo file cannot be found or the section has type SHT_NOBITS, the server will attempt to download the executable file matching the build-id and extract the section. Signed-off-by: Aaron Merey <amerey@redhat.com>
* debuginfod-client: Add DEBUGINFOD_HEADERS_FILE.Daniel Thornburgh2022-10-283-0/+51
| | | | | | | | This DEBUGINFOD_HEADERS_FILE environment variable names a file to supply HTTP headers to outgoing requests. Notably, this allows for Authorization headers to be added from a file under OS access control. Signed-off-by: Daniel Thornburgh <dthorn@google.com>
* debuginfod: report libmicrohttpd version on startupFrank Ch. Eigler2022-10-172-0/+6
| | | | | | To assist troubleshooting with intermittent bugs. Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* Move the #include <libintl.h> into eu-config.hYonggang Luo2022-10-161-1/+0
| | | | | | | | | | So we do not need include in each file. And indeed the macro #define _(Str) dgettext ("elfutils", Str) access libintl function dgettext, so it's make more sense #include <libintl.h> in file eu-config.h Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
* debuginfod-client: Ensure only negative error codes returned.Aaron Merey2022-09-282-2/+7
| | | | | Switch a couple error codes from positive to negative so they aren't interpreted as file descriptors by the caller.
* Retrive 64bit timestamp from curl_easy_getinfo on _TIME_BITS=64 environment河辺 岳人2022-09-271-0/+4
| | | | | | | | | | | | | | | | | | | | | On 32bit system compiled with -D_TIME_BITS=64, following tests failed: FAIL: run-debuginfod-archive-test.sh FAIL: run-debuginfod-archive-groom.sh FAIL: run-debuginfod-archive-rename.sh FAIL: run-debuginfod-federation-sqlite.sh FAIL: run-debuginfod-federation-metrics.sh FAIL: run-debuginfod-federation-link.sh due to tests/debuginfod-subr.sh:archive_test failing on "test $fileame -ot `pwd`", with LHS file having a bogus timestamp. LHS is generated by debuginfo-find as a cache file. I'm not sure the fix is the correct way, but since -D_TIME_BITS=64 mandates -D_FILE_OFFSET_BITS=64 in glibc, this does work on glibc environment. Signed-off-by: Taketo Kabe <kabe@sra-tohoku.co.jp>
* PR28284 cont'd, ->winning_headers reset at start of new queryFrank Ch. Eigler2022-09-082-1/+8
| | | | | | | Mark noted we need to ditch the last client ->winning_headers along with the last ->url. Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* add debuginfod_get_headers if DUMMY_LIBDEBUGINFOD is usedMartin Liska2022-09-082-0/+6
| | | | Signed-off-by: Martin Liska <mliska@suse.cz>
* PR28284: add tweaks on previous debuginfod x-debuginfod* header forwarding workFrank Ch. Eigler2022-09-076-30/+47
| | | | | | | | Embrace case-independent headers, more fully document, handle HTTP \r. In addition to test case, hand-tested against fedora debuginfod instances, running federated servers under valgrind. Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* PR28284 - Debuginfod header functionality implementedNoah Sanci2022-09-066-2/+54
| | | | | | | | Debuginfod and debuginfod clients are now equipped to send and receive http headers prefixed with X-DEBUGINFOD and print them in verbose mode for more context Signed-off-by: Noah Sanci <nsanci@redhat.com>
* debuginfod: Use auto-sized connection pool when -C is not given with argAaron Merey2022-09-052-12/+17
| | | | | | | | | | | | | | | | | | Since commit 4b42d9ad, libmicrohttpd's epoll event loop is used when available in which case we must disable its setting for spawning a thread per request. This contradicts the debuginfod doc's description of '-C', which indicates that if this command line option is not given then the thread pool size is unbounded. Fix this by using an auto-sized thread pool when '-C' is not given, just as we do when it's given with no argument. Update the doc's description of '-C'. Also use a fixed-size pool even if epoll is not supported. The unbounded pool config cannot be considered entirely reliable as it appears to cause random fails in the run-debuginfod-webapi-concurrency test. Signed-off-by: Aaron Merey <amerey@redhat.com>
* Add missing changelog entries.Martin Liska2022-08-171-0/+9
|
* debuginfod: fix http_requests_total{type="debuginfo"} when dwz is usedMartin Liska2022-08-171-2/+4
| | | | | | | | When dwarf_extract_source_paths is called, it can call handle_buildid when a rpm file used dwz. Ignore such internal request in http_requests_total statistics. Signed-off-by: Martin Liska <mliska@suse.cz>
* debuginfod: print filename for "cannot open archive" errorMartin Liska2022-08-171-2/+8
| | | | | | Report the file that has such a problem so that one can inspect it. Signed-off-by: Martin Liska <mliska@suse.cz>
* PR29474: debuginfodFrank Ch. Eigler2022-08-152-1/+8
| | | | | | | | | | | | | | | | | | | | | | | Previous code sometimes confused debuginfod with concurrent queries targeting the same RPM. One thread fetching & prefetching, the other thread misinterpreted the sudden presence of its target file in the fdcache as a mere unnecessary prefetch duplicate. But if it was the other thread's target file, previous code would -skip- it completely, resulting in a 404 error. New code allows the other thread to also decompress the target file and return it, and still continue to its own prefetching process for other files. There's a performance trade-off here. Another option would be for the other thread to check the fdcache regularly and abort its own prefetch/fetch/prefetch loop early in case of a sudden target file hit. That'd save CPU probably but it'd stop prefetching of later segments of the input archive, which could save future time. Automated testing is too time/load sensitive to attempt. Confirmed working with Martin's stress tester. Reported-By: Martin Liška <mliska@suse.cz> Signed-Off-By: Frank Ch. Eigler <fche@redhat.com>
* debuginfod: optimize regular expressions in groom()Josef Cejka2022-08-032-3/+13
| | | | | | | Check if applying of -I and -X during grooming is enabled before the regular expressions are matched. Signed-off-by: Josef Cejka <jcejka@suse.de>
* debuginfod: create indexes to speed up groomingJosef Cejka2022-07-292-0/+8
| | | | | | | | | | | | | | | Create indexes on _r_de and _f_de tables to speed up delete operations called by groom() function. Primary keys of those tables are optimalized to search rows by buildids so delete by file and mtime attributes has to scan the whole table. On large database can single delete query take minutes and grooming will be aborted before completion by time limit. New indexes are compatible with current sqlite schema. Signed-off-by: Josef Cejka <jcejka@suse.de>
* NEWS & ChangeLog: add debuginfod blurbsFrank Ch. Eigler2022-06-031-0/+5
| | | | Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
* debuginfod: add --disable-source-scan optionMichael Trapp2022-06-031-1/+8
| | | | | | | | --disable-source-scan disables scanning of the dwarf source info of debuginfo sections. The source info is not required in setups without source code access. Signed-off-by: Michael Trapp <michael.trapp@sap.com>
* debuginfod: update mtime of interval_path as early as possibleMark Wielaard2022-05-152-2/+10
| | | | | | | | | | Call utime on interval_path file as soon as the thread is committed to cleanup the cache files. This will prevent other threads trying to also commit to cleaning the cache files. Having multiple threads try to clean the cache simultaniously doesn't improve cleanup speed because the threads will try to delete the files in the same order. Signed-off-by: Mark Wielaard <mark@klomp.org>
* debuginfod: Remove debuginfod_init_cacheMark Wielaard2022-05-152-52/+15
| | | | | | | | | | | debuginfod_init_cache would create all config files if they didn't exist yet. It always made two stat calls. Then debuginfod_clean_cache would call debuginfod_config_cache which did the same checks and created any missing config files. Just make sure the cache_path directory exists and remove debuginfod_init_cache before calling debuginfod_clean_cache. Signed-off-by: Mark Wielaard <mark@klomp.org>
* debuginfod: Make sure debuginfod_config_cache always returns valid statMark Wielaard2022-05-152-7/+20
| | | | | | | | | | | If the condig file which value was requested from debuginfod_config_cache didn't exist yet, stat would fail and no valid struct stat would be returned even when the file was correctly created. Fix this by always using O_CREAT to open the file, and reuse that file descriptor to call fstat and for either writing the default value or reading the config file value. Signed-off-by: Mark Wielaard <mark@klomp.org>
* debuginfod: Always request servname from getnameinfo for conninfo.Mark Wielaard2022-05-152-5/+14
| | | | | | | | | | | When getting the connection info getnameinfo is called getting the hostname and servname except when the sockaddr is a pure ipv6 address. In that last case only hostname is requested. Since servname is stack allocated and not initialized it might contain garbage which is then put in the log. Just always request both hostname and servname with NI_NUMERICHOST | NI_NUMERICSERV. Signed-off-by: Mark Wielaard <mark@klomp.org>
* debuginfod: Check all curl_easy_setopt callsMark Wielaard2022-05-142-22/+44
| | | | | | | | curl_easy_setup can fail for various reasons. Add a curl_easy_setopt_ck macro to check all curl_easy_setopt calls and provides a human readable error message in verbose mode. Signed-off-by: Mark Wielaard <mark@klomp.org>
View Lorry Controller Status