debuginfod 1/2: client side

Introduce the debuginfod/ subdirectory, containing the client for a
new debuginfo-over-http service, in shared-library and command-line
forms.  Two functions in libdwfl make calls into the client library to
fetch elf/dwarf files by buildid, as a fallback.  Instead of normal
dynamic linking (thus pulling in a variety of curl dependencies),
the libdwfl hooks use dlopen/dlsym.  Server & tests coming in patch 2.

Signed-off-by: Aaron Merey <amerey@redhat.com>
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>

5 files changed, 330 insertions, 1 deletions

diff --git a/doc/Makefile.am b/doc/Makefile.am
index d6f3eca8..23102604 100644
--- a/doc/Makefile.am
+++ b/doc/Makefile.am
@@ -1,7 +1,7 @@
 ## Process this file with automake to create Makefile.in
 ## Configure input file for elfutils.
 ##
-## Copyright (C) 1996-2001, 2002, 2005 Red Hat, Inc.
+## Copyright (C) 1996-2001, 2002, 2005, 2019 Red Hat, Inc.
 ## This file is part of elfutils.
 ##
 ## This file is free software; you can redistribute it and/or modify
@@ -19,3 +19,11 @@
 EXTRA_DIST = COPYING-GFDL README
 dist_man1_MANS=readelf.1 elfclassify.1
 notrans_dist_man3_MANS=elf_update.3 elf_getdata.3 elf_clone.3 elf_begin.3
+
+notrans_dist_man1_MANS=
+
+if DEBUGINFOD
+notrans_dist_man3_MANS += debuginfod_find_debuginfo.3 debuginfod_find_source.3 debuginfod_find_executable.3
+notrans_dist_man1_MANS += debuginfod-find.1
+endif
+
diff --git a/doc/debuginfod-find.1 b/doc/debuginfod-find.1
new file mode 100644
index 00000000..ab2f4d17
--- /dev/null
+++ b/doc/debuginfod-find.1
@@ -0,0 +1,137 @@
+'\"! tbl | nroff \-man
+'\" t macro stdmacro
+
+.de SAMPLE
+.br
+.RS 0
+.nf
+.nh
+..
+.de ESAMPLE
+.hy
+.fi
+.RE
+..
+
+.TH DEBUGINFOD-FIND 1
+.SH NAME
+debuginfod-find \- request debuginfo-related data
+
+.SH SYNOPSIS
+.B debuginfod-find debuginfo \fIBUILDID\fP
+
+.B debuginfod-find executable \fIBUILDID\fP
+
+.B debuginfod-find source \fIBUILDID\fP \fI/FILENAME\fP
+
+.SH DESCRIPTION
+\fBdebuginfod-find\fP queries one or more \fBdebuginfod\fP servers for
+debuginfo-related data.  In case of a match, it saves the the
+requested file into a local cache, prints the file name to standard
+output, and exits with a success status of 0.  In case of any error,
+it exits with a failure status and an error message to standard error.
+
+.\" Much of the following text is duplicated with debuginfod.8
+
+The debuginfod system uses buildids to identify debuginfo-related data.
+These are stored as binary notes in ELF/DWARF files, and are
+represented as lowercase hexadecimal.  For example, for a program
+/bin/ls, look at the ELF note GNU_BUILD_ID:
+
+.SAMPLE
+% readelf -n /bin/ls | grep -A4 build.id
+Note section [ 4] '.note.gnu.buildid' of 36 bytes at offset 0x340:
+Owner          Data size  Type
+GNU                   20  GNU_BUILD_ID
+Build ID: 8713b9c3fb8a720137a4a08b325905c7aaf8429d
+.ESAMPLE
+
+Then the hexadecimal BUILDID is simply:
+
+.SAMPLE
+8713b9c3fb8a720137a4a08b325905c7aaf8429d
+.ESAMPLE
+
+.SS debuginfo \fIBUILDID\fP
+
+If the given buildid is known to a server, this request will result
+in a binary object that contains the customary \fB.*debug_*\fP
+sections.  This may be a split debuginfo file as created by
+\fBstrip\fP, or it may be an original unstripped executable.
+
+.SS executable \fIBUILDID\fP
+
+If the given buildid is known to the server, this request will result
+in a binary object that contains the normal executable segments.  This
+may be a executable stripped by \fBstrip\fP, or it may be an original
+unstripped executable.  \fBET_DYN\fP shared libraries are considered
+to be a type of executable.
+
+.SS source \fIBUILDID\fP \fI/SOURCE/FILE\fP
+
+If the given buildid is known to the server, this request will result
+in a binary object that contains the source file mentioned.  The path
+should be absolute.  Relative path names commonly appear in the DWARF
+file's source directory, but these paths are relative to
+individual compilation unit AT_comp_dir paths, and yet an executable
+is made up of multiple CUs.  Therefore, to disambiguate, debuginfod
+expects source queries to prefix relative path names with the CU
+compilation-directory, followed by a mandatory "/".
+
+Note: the user should not elide \fB../\fP or \fB/./\fP or extraneous
+\fB///\fP sorts of path components in the directory names, because if
+this is how those names appear in the DWARF files, that is what
+debuginfod needs to see too.
+
+For example:
+.TS
+l l.
+#include <stdio.h>	source BUILDID /usr/include/stdio.h
+/path/to/foo.c	source BUILDID /path/to/foo.c
+\../bar/foo.c AT_comp_dir=/zoo/	source BUILDID /zoo//../bar/foo.c
+.TE
+
+.SH "SECURITY"
+
+debuginfod-find \fBdoes not\fP include any particular security
+features.  It trusts that the binaries returned by the debuginfod(s)
+are accurate.  Therefore, the list of servers should include only
+trustworthy ones.  If accessed across HTTP rather than HTTPS, the
+network should be trustworthy.  Authentication information through
+the internal \fIlibcurl\fP library is not currently enabled, except
+for the basic plaintext \%\fIhttp[s]://userid:password@hostname/\fP style.
+(The debuginfod server does not perform authentication, but a front-end
+proxy server could.)
+
+.SH "ENVIRONMENT VARIABLES"
+
+.TP 21
+.B DEBUGINFOD_URLS
+This environment variable contains a list of URL prefixes for trusted
+debuginfod instances.  Alternate URL prefixes are separated by space.
+
+.TP 21
+.B DEBUGINFOD_TIMEOUT
+This environment variable governs the timeout for each debuginfod HTTP
+connection.  A server that fails to respond within this many seconds
+is skipped.  The default is 5.
+
+.TP 21
+.B DEBUGINFOD_CACHE_PATH
+This environment variable governs the location of the cache where
+downloaded files are kept.  It is cleaned periodically as this program
+is reexecuted.  Cache management parameters may be set by files under
+this directory: see the \fBdebuginfod_find_debuginfo(3)\fP man page
+for details.  The default is $HOME/.debuginfod_client_cache.
+
+.SH "FILES"
+.LP
+.PD .1v
+.TP 20
+.B $HOME/.debuginfod_client_cache
+Default cache directory.
+.PD
+
+.SH "SEE ALSO"
+.I "debuginfod(8)"
+.I "debuginfod_find_debuginfod(3)"
diff --git a/doc/debuginfod_find_debuginfo.3 b/doc/debuginfod_find_debuginfo.3
new file mode 100644
index 00000000..e0923d8d
--- /dev/null
+++ b/doc/debuginfod_find_debuginfo.3
@@ -0,0 +1,182 @@
+'\"! tbl | nroff \-man
+'\" t macro stdmacro
+
+.de SAMPLE
+.br
+.RS 0
+.nf
+.nh
+..
+.de ESAMPLE
+.hy
+.fi
+.RE
+..
+
+.TH DEBUGINFOD_FIND_DEBUGINFO 3
+.SH NAME
+debuginfod_find_debuginfo \- request debuginfo from debuginfod
+
+.SH SYNOPSIS
+.nf
+.B #include <elfutils/debuginfod.h>
+.PP
+.BI "debuginfod_find_debuginfo(const unsigned char *" build_id ", int " build_id_len ", char ** " path ");"
+.BI "debuginfod_find_executable(const unsigned char *" build_id ", int " build_id_len ", char ** " path ");"
+.BI "debuginfod_find_source(const unsigned char *" build_id ", int " build_id_len ", const char *" filename ", char ** " path ");"
+
+.SH DESCRIPTION
+.BR debuginfod_find_debuginfo (),
+.BR debuginfod_find_executable (),
+and
+.BR debuginfod_find_source ()
+query the debuginfod server URLs contained in
+.BR $DEBUGINFOD_URLS
+(see below) for the debuginfo, executable or source file with the
+given \fIbuild_id\fP. \fIbuild_id\fP should be a pointer to either
+a null-terminated, lowercase hex string or a binary blob. If
+\fIbuild_id\fP is given as a hex string, \fIbuild_id_len\fP should
+be 0. Otherwise \fIbuild_id_len\fP should be the number of bytes in
+the binary blob.
+
+.BR debuginfod_find_source ()
+also requries a \fIfilename\fP in order to specify a particular
+source file. \fIfilename\fP should be an absolute path that includes
+the compilation directory of the CU associated with the source file.
+Relative path names commonly appear in the DWARF file's source directory,
+but these paths are relative to individual compilation unit AT_comp_dir
+paths, and yet an executable is made up of multiple CUs. Therefore, to
+disambiguate, debuginfod expects source queries to prefix relative path
+names with the CU compilation-directory, followed by a mandatory "/".
+
+Note: the caller should not elide \fB../\fP or \fB/./\fP or extraneous
+\fB///\fP sorts of path components in the directory names, because if
+this is how those names appear in the DWARF files, that is what
+debuginfod needs to see too.
+
+If \fIpath\fP is not NULL and the query is successful, \fIpath\fP is set
+to the path of the file in the cache. The caller must \fBfree\fP() this value.
+
+The URLs in \fB$DEBUGINFOD_URLS\fP are queried in parallel. As soon as a
+debuginfod server begins transfering the target file all of the connections
+to the other servers are closed.
+
+.SH "CACHE"
+If the query is successful, the \fBdebuginfod_find_*\fP() functions save
+the target file to a local cache. The location of the cache is controlled
+by the \fB$DEBUGINFOD_CACHE_PATH\fP environment variable (see below).
+Cleaning of the cache is controlled by the \fIcache_clean_interval_s\fP
+and \fImax_unused_age_s\fP files, which are found in the
+\fB$DEBUGINFOD_CACHE_PATH\fP directory. \fIcache_clean_interval_s\fP controls
+how frequently the cache is traversed for cleaning and \fImax_unused_age_s\fP
+controls how long a file can go unused (fstat(2) atime) before it's
+removed from the cache during cleaning. These files should contain only an
+ASCII decimal integer representing the interval or max unused age in seconds.
+The default is one day and one week, respectively.  Values of zero mean "immediately".
+
+.SH "SECURITY"
+.BR debuginfod_find_debuginfo (),
+.BR debuginfod_find_executable (),
+and
+.BR debuginfod_find_source ()
+\fBdo not\fP include any particular security
+features.  They trust that the binaries returned by the debuginfod(s)
+are accurate.  Therefore, the list of servers should include only
+trustworthy ones.  If accessed across HTTP rather than HTTPS, the
+network should be trustworthy.  Passing user authentication information
+through the internal \fIlibcurl\fP library is not currently enabled, except
+for the basic plaintext \%\fIhttp[s]://userid:password@hostname/\fP style.
+(The debuginfod server does not perform authentication, but a front-end
+proxy server could.)
+
+.SH "ENVIRONMENT VARIABLES"
+
+.TP 21
+.B DEBUGINFOD_URLS
+This environment variable contains a list of URL prefixes for trusted
+debuginfod instances.  Alternate URL prefixes are separated by space.
+
+.TP 21
+.B DEBUGINFOD_TIMEOUT
+This environment variable governs the timeout for each debuginfod HTTP
+connection.  A server that fails to respond within this many seconds
+is skipped.  The default is 5.
+
+.TP 21
+.B DEBUGINFOD_CACHE_PATH
+This environment variable governs the location of the cache where
+downloaded files are kept.  It is cleaned periodically as this
+program is reexecuted.  The default is $HOME/.debuginfod_client_cache.
+
+.SH "RETURN VALUE"
+If the query is successful, these functions save the target file
+to the client cache and return a file descriptor to that file.
+Otherwise an error code is returned.
+
+.SH "ERRORS"
+The following list is not comprehensive. Error codes may also
+originate from calls to various C Library functions.
+
+.TP
+.BR EACCESS
+Denied access to resource located at the URL.
+
+.TP
+.BR ECONNREFUSED
+Unable to connect to remote host.
+
+.TP
+.BR ECONNRESET
+Unable to either send or recieve network data.
+
+.TP
+.BR EHOSTUNREACH
+Unable to resolve remote host.
+
+.TP
+.BR EINVAL
+One or more arguments are incorrectly formatted. \fIbuild_id\fP may
+be too long (greater than 256 characters), \fIfilename\fP may not
+be an absolute path or a debuginfod URL is malformed.
+
+.TP
+.BR EIO
+Unable to write data received from server to local file.
+
+.TP
+.BR EMLINK
+Too many HTTP redirects.
+
+.TP
+.BR ENETUNREACH
+Unable to initialize network connection.
+
+.TP
+.BR ENOENT
+Could not find the resource located at URL. Often this error code
+indicates that a debuginfod server was successfully contacted but
+the server could not find the target file.
+
+.TP
+.BR ENOMEM
+System is unable to allocate resources.
+
+.TP
+.BR ENOSYS
+\fB$DEBUGINFOD_URLS\fP is not defined.
+
+.TP
+.BR ETIME
+Query failed due to timeout. \fB$DEBUGINFOD_TIMEOUT\fP controls
+the timeout duration. See debuginfod(8) for more information.
+
+.SH "FILES"
+.LP
+.PD .1v
+.TP 20
+.B $HOME/.debuginfod_client_cache
+Default cache directory.
+.PD
+
+.SH "SEE ALSO"
+.I "debuginfod(8)"
diff --git a/doc/debuginfod_find_executable.3 b/doc/debuginfod_find_executable.3
new file mode 100644
index 00000000..16279936
--- /dev/null
+++ b/doc/debuginfod_find_executable.3
@@ -0,0 +1 @@
+.so man3/debuginfod_find_debuginfo.3
diff --git a/doc/debuginfod_find_source.3 b/doc/debuginfod_find_source.3
new file mode 100644
index 00000000..16279936
--- /dev/null
+++ b/doc/debuginfod_find_source.3
@@ -0,0 +1 @@
+.so man3/debuginfod_find_debuginfo.3