| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| ... | |
| |
|
|
|
|
| |
This reverts commit 8b65880b5ad817156b58c58b5133aafc99b0a264.
See <https://github.com/ccache/ccache/pull/1158#issuecomment-1268748557>.
|
| |
|
| |
Co-authored-by: Luboš Luňák <l.lunak@centrum.cz>
|
| |
|
|
| |
Closes #1010.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There is a feature request to be able not to use a local cache at all,
only a network cache. With such a feature, the names "primary storage"
and "secondary storage" make less sense since ccache would be operating
in "secondary only" mode, but then that storage would of course become
the primary (and only).
Let's rename "primary storage" to "local storage" and "secondary
storage" to "remote storage" – operating in "remote only" mode then
makes sense.
One of the original motivations to call networked storage "secondary" is
that the file storage can be used for local file systems as well, making
such storage "not quite remote", but in practice I guess the file
storage backend used primarily for network file systems.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Cache entries are now fully read into memory before (de)compressing,
checksumming and parsing, instead of streaming data like before. While
this increases memory usage when working with large object files, it
also simplifies the code a lot. Another motivation for this change is
that cache entry data is not streamed from secondary storage anyway, and
it makes sense to keep the architecture simple and similar for primary
and secondary storage code paths.
The cache entry format has modified so that the checksum covers the
potentially compressed payload (plus the header), not the uncompressed
payload (plus the header) like before. The checksum is now also stored
in an uncompressed epilogue. Since the cache entry format has been
changed, the input hash has been changed as well.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
| |
Regression in fec405539bca90ebd0992ae6606bd10eec3a64da.
Fixes #1155.
|
| |
|
|
|
| |
clang-format 14 chokes on a construction in unittest/test_Stat.cpp. Work
around this by moving ifdefs around.
|
| |
|
|
|
| |
This allows for writing to a file that already exists with read-only
permissions.
|
| |
|
|
|
| |
I want to introduce more use of nonstd::span<const uint8_t> and I think
that it makes more sense to use unaliased types.
|
| |
|
|
|
| |
Using the FMT macro explicitly makes invalid format message
constructions compile-time errors instead of run-time errors.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
- Only enable the inode cache at compile-time if it's possible to
determine filesystem type.
- Only use the inode cache at run-time if the filesystem type is known
to work with the inode cache instead of refusing just on NFS.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As mentioned in discussion #1086: If the inode cache is enabled,
hash_source_code_file will on an inode cache hit fail to hash time
information if there are temporal macros in the code. This is because
hash_source_code_string (called from hash_source_code_file via
hash_source_code_file_nocache) on an inode cache miss adds time
information to the hash in case one of those macros were found. However,
on an inode cache hit the return value of hash_source_code_file will be
correctly fetched from the cache, but the hash sum will only be updated
with (the hash of) the include file and not the time information.
The fix is to let the inode cache only cache the effects of hashing the
file and checking for macros, not the hashing of time information since
that's volatile.
After the fix:
- The new do_hash_file function performs file hashing and macro
checking. The inode cache caches this step.
- hash_source_code_file returns a Digest instead of adding data to a
Hash.
- hash_source_code_file calls do_hash_file and then potentially hashes
time information. If there are no temporal macros the returned digest
will be identical to the file hash, otherwise the returned digest will
be of a hash of file content hash + time information.
This also improves hashes that are stored in the direct mode manifest:
Previously they were always the hash of the file content hash but now
they are just the file content hash in the common case when there is no
__DATE__ or __TIMESTAMP__ macro in the file.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
[1] added the has_absolute_include_headers variable as a performance
optimization for base_dir/CCACHE_BASEDIR so that the dependency file
only has to be parsed/rewritten when necessary. This is based on the
assumption that if no include file has an absolute path then no rewrite
is needed, but apparently Clang can insert other paths into the
dependency file as well, for instance the asan_blacklist.txt file when
using -fsanitize=address.
Fix this by simply always parsing the dependency file when base_dir is
active.
Fixes #1128.
[1]: 60178b7a8eb8d03b364afb2fb8bd6753924cd9b7
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Cache entries are now shared for different -MT/-MQ options. This is
implemented by (on a cache hit) rewriting the stored dependency file
data to have the correct dependency target before writing it to the
destination. Closes #359.
- An intentional side effect of the above is that the correct dependency
target will be produced even when base_dir/CCACHE_BASEDIR is used -
the dependency target will still be an absolute path. Fixes #1042.
- Buggy support for GCC-specific environment variables
DEPENDENCIES_OUTPUT and SUNPRO_DEPENDENCIES has been removed. When one
of those variables was set, ccache used to store and fetch the content
just as if -MMD or -MD were used. This is however incorrect since GCC
*appends* to the destination file instead of (like -MMD/-MD)
*rewriting* it. Since there is no way for ccache to know what the
compiler appended to the dependency file, we simply can't support it.
Reverts #349.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Ccache will produce a false hit in the following scenario, if "cc" is
GCC without being a symlink to "gcc":
1. ccache cc -c example.c # adds a cache entry
2. ccache cc -c example.c -fcolor-diagnostics # unexpectedly succeeds
(Most major Linux distributions install cc as a symlink to gcc, and then
the problem doesn't show up since ccache then is able to guess that the
compiler is GCC.)
This bug was introduced by [1] which tried to make yet another tweak of
the color diagnostics guessing/probing by no longer rejecting
-fcolor-diagnostics for an unknown compiler, based on the assumption
that the probing call (speculatively adding -fdiagnostics-color) is only
made when we have guessed a GCC compiler. Unfortunately, this broke the
fix in [2] when the compiler is unknown.
All of this is inherently brittle and I don't see any good way to make
it better with the current compiler guessing approach and adding options
speculatively. The cure is to start doing proper compiler identification
instead (#958). Until that is done, the only reasonable fix is to avoid
doing anything related to color diagnostics flags when the compiler is
unknown. This means that an unknown compiler from now on won't produce
colors when used via ccache even if the compiler actually is GCC or
Clang.
[1]: 97a40af49f56e4be2db5e4318d8131e937bd5598
[2]: 61ce8c44c5b1da0be7a8d10c014f1a85b7967433
Closes #1116.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit improves brings two improvements to the lock file
implementation:
1. Support for long-lived locks.
2. Support for non-blocking lock acquisition.
There are now two types of lock file classes: ShortLivedLockFile and
LongLivedLockFile. On Windows the implementations are identical. On
other systems, LongLivedLockFile creates a separate "alive file" whose
mtime will be updated regularly by a helper thread until the lock is
released. This makes it possible for another process to wait for the
lock indefinitely but also to know then a lock is stale and can be
broken. The ShortLivedLockFile class works like the lock file class used
to work before: it considers a lock stale after waiting for two seconds
and noticing that the symlink target has not changed during that time.
ShortLivedLockFile is to be used when the lock is expected to be held
for a very short time so that it's a waste of resources to start a
helper thread to keep the lock alive.
On some systems it would be possible to update mtime of the symlink
itself instead of a separate file, but that does not seem to be portable
enough.
Also worth mentioning is that the reason to not use proper fcntl/flock
locks on non-Windows systems is to continue supporting a cache directory
on NFS since file locks on NFS don't have a good track record.
|
| |
|
|
| |
Clang-Analyzer prefers it that way.
|
| | |
|
| |
|
|
|
|
|
|
| |
Setting up ccache to masquerade as a compiler has always meant using
symbolic links, but there is no technical reason why that has to be the
case. This commit adds support for masquerading via a copy or hard link
of the ccache executable as well. This is mostly useful on platforms or
file systems where symbolic links are not supported.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
[1] added a suitable file extension to the temporary file used for
capturing the preprocessed output by creating a hard link. This fails
when the temporary directory is on a file system that doesn't support
hard links.
Fix this by making it possible to pass a suffix to TemporaryFile and
passing the proper cpp suffix for the tmp stdout file instead of
creating a hard link as an alias.
[1]: 4da9f2b474b7ee39cd54ab4261f90936f3c944ec
Closes #1079.
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The Util::normalize_absolute_path function only works on the syntactic
level, i.e. the result may not actually resolve to the same filesystem
entry (nor to any file system entry for that matter). It was meant to be
used for paths that don’t necessarily exist yet, such as a future
directory in which to write debug files. It may fail in edge cases with
symlinks in the path in combination with .. segments. If the caller
wants to ensure that the resulting path actually makes sense, it needs
to check if the resulting path points to the same file entry as the
original.
To improve on this, Util::normalize_absolute_path has now been renamed
to Util::normalize_abstract_absolute_path and there is a new
Util::normalize_concrete_absolute_path function which returns the
original path if the normalized result doesn't resolve to the same file
system entry as the original path.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When support for caching a compilation with coverage (producing a .gcno
file), the commit[1] made sure to avoid rewriting the input path to
relative with the motivation “also make sure to use the source file
path, since this is in the notes”. However, this seems to be unnecessary
since a relative input file path will be written as is to the .gcno, not
the absolute path. This is the case for at least GCC 4.7+ and Clang
3.6+. Fix this by potentially converting the input path to relative even
when generating coverage.
When investigating the above issue, I noticed that GCC 9+ includes the
current working directory (CWD) in the .gcno file. This means that we
have include the CWD in the hash when compiling with
-ftest-coverage/--coverage in order to replicate what the compiler would
produce. Since this makes it impossible get cache hits when compiling in
different directories, a new gcno_cwd sloppiness has been added for
opting out of hashing the CWD, with the tradeoff of potentially getting
an incorrect directory in the .gcno file.
[1]: 02d3f078bd2495b8db2264ae0b2c692b4c5ba1bd
Fixes #1032.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
The code in Util::read_text_file for converting UTF-16LE to UTF-8 only
works on little-endian machines. This makes the unit test fail on
big-endian machines.
Since the conversion is only needed on Windows (for Visual Studio, which
creates UTF-16LE .rsp files) in practice, work around the problem by
only doing the conversion in Windows builds.
Fixes #1014.
|
