| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
|
|
|
| |
This is a preliminary refactoring for #14335 (supporting plugins in
cross-compilers). In many places the home-unit must be optional because
there won't be one available in the plugin environment (we won't be
compiling anything in this environment). Hence we replace "HomeUnit"
with "Maybe HomeUnit" in a few places and we avoid the use of
"hsc_home_unit" (which is partial) in some few others.
|
| |
|
|
| |
We no longer need it after previous IndefUnitId refactoring.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
At the moment if `-dynamic-too` fails then we rerun the whole pipeline
as if we were just in `-dynamic` mode. I argue this is a misfeature and
we should remove the so-called `DT_Failed` mode.
In what situations do we fall back to `DT_Failed`?
1. If the `dyn_hi` file corresponding to a `hi` file is missing completely.
2. If the interface hash of `dyn_hi` doesn't match the interface hash of `hi`.
What happens in `DT_Failed` mode?
* The whole compiler pipeline is rerun as if the user had just passed `-dynamic`.
* Therefore `dyn_hi/dyn_o` files are used which don't agree with the
`hi/o` files. (As evidenced by `dynamicToo001` test).
* This is very confusing as now a single compiler invocation has
produced further `hi`/`dyn_hi` files which are different to each
other.
Why should we remove it?
* In `--make` mode, which is predominately used `DT_Failed` does not
work (#19782), there can't be users relying on this functionality.
* In `-c` mode, the recovery doesn't fix the root issue, which is the
`dyn_hi` and `hi` files are mismatched. We should instead produce an
error and pass responsibility to the build system using `-c` to ensure
that the prerequisites for `-dynamic-too` (dyn_hi/hi) files are there
before we start compiling.
* It is a misfeature to support use cases like `dynamicToo001` which
allow you to mix different versions of dynamic/non-dynamic interface
files. It's more likely to lead to subtle bugs in your resulting
programs where out-dated build products are used rather than a
deliberate choice.
* In practice, people are usually compiling with `-dynamic-too` rather
than separately with `-dynamic` and `-static`, so the build products
always match and `DT_Failed` is only entered due to compiler bugs (see
!6583)
What should we do instead?
* In `--make` mode, for home packages check during recompilation
checking that `dyn_hi` and `hi` are both present and agree, recompile
the modules if they do not.
* For package modules, when loading the interface check that `dyn_hi`
and `hi` are there and that they agree but fail with an
error message if they are not.
* In `--oneshot` mode, fail with an error message if the right files
aren't already there.
Closes #19782 #20446 #9176 #13616
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Before this patch, plugin units were linked with the target code even
when the unit was passed via `-plugin-package`. This is an issue to
support plugins in cross-compilers (plugins are definitely not ABI
compatible with target code).
We now clearly separate unit dependencies for plugins and unit
dependencies for target code and only link the latter ones.
We've also added a test to ensure that plugin units passed via
`-package` are linked with target code so that `thNameToGhcName` can
still be used in plugins that need it (see T20218b).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If a plugins was specified using the -plugin-package-(id) flag then the
module it applied to was always recompiled.
The recompilation checker was previously using `findImportedModule`,
which looked for packages in the HPT and then in the package database
but only for modules specified using `-package`.
The correct lookup function for plugins is `findPluginModule`, therefore
we check normal imports with `findImportedModule` and plugins with
`findPluginModule`.
Fixes #20417
|
| | |
|
| |
|
|
|
|
| |
* moved deps related code into GHC.Unit.Module.Deps
* refactored Deps module to not export Dependencies constructor to help
maintaining invariants
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Make mkDependencies pure
* Use Sets instead of sorted lists
Notable perf changes:
MultiLayerModules(normal) ghc/alloc 4130851520.0 2981473072.0 -27.8%
T13719(normal) ghc/alloc 4313296052.0 4151647512.0 -3.7%
Metric Decrease:
MultiLayerModules
T13719
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The GHC.Prim module is quite special as there is no interface file,
therefore it doesn't appear in ms_textual_imports, but the ghc-prim
package does appear in the direct package dependencies. This confused
the recompilation checking which couldn't find any modules from ghc-prim
and concluded that the package was no longer a dependency.
The fix is to keep track of whether GHC.Prim is imported separately in
the relevant places.
Fixes #20084
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When TemplateHaskellQuotes is enabled, we also generate programs which
mention symbols from the template-haskell module. So that package is
added conditionally if the extension is turned on.
We should really do the same for other wired-in packages:
* base
* ghc-bignum
* ghc-prim
* rts
When we link an executable, we must also link against these
libraries. In accordance with every other package, these dependencies
should be added into the direct dependencies for a module automatically
and end up in the interface file to record the fact the object file was
created by linking against these packages.
Unfortunately it is not so easy to work out when symbols from each of
these libraries ends up in the generated program. You might think that
`base` would always be used but the `ghc-prim` package doesn't depend
on `base`, so you have to be a bit careful and this futher enhancement
is left to a future patch.
|
| |
|
|
|
|
|
|
| |
Now that Outputable is independent of DynFlags, we can put tracing
functions using SDocs into their own module that doesn't transitively
depend on any GHC.Driver.* module.
A few modules needed to be moved to avoid loops in DEBUG mode.
|
| |
|
|
|
|
|
|
|
|
| |
Now that we hash object files to decide when to recompile due to TH,
this can make a big difference as each interface file in a project will
contain reference to the object files of all package dependencies.
Especially when these are statically linked, hashing them can add up.
The cache is invalidated when `depanalPartial` is called, like the
normal finder cache.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Introduce LogFlags as a independent subset of DynFlags used for logging.
As a consequence in many places we don't have to pass both Logger and
DynFlags anymore.
The main reason for this refactoring is that I want to refactor the
systools interfaces: for now many systools functions use DynFlags both
to use the Logger and to fetch their parameters (e.g. ldInputs for the
linker). I'm interested in refactoring the way they fetch their
parameters (i.e. use dedicated XxxOpts data types instead of DynFlags)
for #19877. But if I did this refactoring before refactoring the Logger,
we would have duplicate parameters (e.g. ldInputs from DynFlags and
linkerInputs from LinkerOpts). Hence this patch first.
Some flags don't really belong to LogFlags because they are subsystem
specific (e.g. most DumpFlags). For example -ddump-asm should better be
passed in NCGConfig somehow. This patch doesn't fix this tight coupling:
the dump flags are part of the UI but they are passed all the way down
for example to infer the file name for the dumps.
Because LogFlags are a subset of the DynFlags, we must update the former
when the latter changes (not so often). As a consequence we now use
accessors to read/write DynFlags in HscEnv instead of using `hsc_dflags`
directly.
In the process I've also made some subsystems less dependent on DynFlags:
- CmmToAsm: by passing some missing flags via NCGConfig (see new fields
in GHC.CmmToAsm.Config)
- Core.Opt.*:
- by passing -dinline-check value into UnfoldingOpts
- by fixing some Core passes interfaces (e.g. CallArity, FloatIn)
that took DynFlags argument for no good reason.
- as a side-effect GHC.Core.Opt.Pipeline.doCorePass is much less
convoluted.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch comprises of four different but closely related ideas. The
net result is fixing a large number of open issues with the driver
whilst making it simpler to understand.
1. Use the hash of the source file to determine whether the source file
has changed or not. This makes the recompilation checking more robust to
modern build systems which are liable to copy files around changing
their modification times.
2. Remove the concept of a "stable module", a stable module was one
where the object file was older than the source file, and all transitive
dependencies were also stable. Now we don't rely on the modification
time of the source file, the notion of stability is moot.
3. Fix TH/plugin recompilation after the removal of stable modules. The
TH recompilation check used to rely on stable modules. Now there is a
uniform and simple way, we directly track the linkables which were
loaded into the interpreter whilst compiling a module. This is an
over-approximation but more robust wrt package dependencies changing.
4. Fix recompilation checking for dynamic object files. Now we actually
check if the dynamic object file exists when compiling with -dynamic-too
Fixes #19774 #19771 #19758 #17434 #11556 #9121 #8211 #16495 #7277 #16093
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit does some de-duplication of logic between the one-shot and --make
modes, and splitting of some of the APIs so that its easier to do the
fine-grained parallelism implementation. This is the first part of the
implementation plan as described in #14095
* compileOne now uses the runPhase pipeline for most of the work.
The Interpreter backend handling has been moved to the runPhase.
* hscIncrementalCompile has been broken down into multiple APIs.
* haddock submodule bump: Rename of variables in html-test ref:
This is caused by a change in ModDetails in case of NoBackend.
Now the initModDetails is used to recreate the ModDetails from interface and
in-memory ModDetails is not used.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit modifies interface files so that *only* direct information
about modules and packages is stored in the interface file.
* Only direct module and direct package dependencies are stored in the
interface files.
* Trusted packages are now stored separately as they need to be checked
transitively.
* hs-boot files below the compiled module in the home module are stored
so that eps_is_boot can be calculated in one-shot mode without loading
all interface files in the home package.
* The transitive closure of signatures is stored separately
This is important for two reasons
* Less recompilation is needed, as motivated by #16885, a lot of
redundant compilation was triggered when adding new imports deep in the
module tree as all the parent interface files had to be redundantly
updated.
* Checking an interface file is cheaper because you don't have to
perform a transitive traversal to check the dependencies are up-to-date.
In the code, places where we would have used the transitive closure, we
instead compute the necessary transitive closure. The closure is not
computed very often, was already happening in checkDependencies, and
was already happening in getLinkDeps.
Fixes #16885
-------------------------
Metric Decrease:
MultiLayerModules
T13701
T13719
-------------------------
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
Replace uses of WARN macro with calls to:
warnPprTrace :: Bool -> SDoc -> a -> a
Remove the now unused HsVersions.h
Bump haddock submodule
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There is no reason to use CPP. __LINE__ and __FILE__ macros are now
better replaced with GHC's CallStack. As a bonus, assert error messages
now contain more information (function name, column).
Here is the mapping table (HasCallStack omitted):
* ASSERT: assert :: Bool -> a -> a
* MASSERT: massert :: Bool -> m ()
* ASSERTM: assertM :: m Bool -> m ()
* ASSERT2: assertPpr :: Bool -> SDoc -> a -> a
* MASSERT2: massertPpr :: Bool -> SDoc -> m ()
* ASSERTM2: assertPprM :: m Bool -> SDoc -> m ()
|
| | |
|
| |
|
|
|
|
|
|
| |
* Make NameCache the mutable one and replace NameCacheUpdater with it
* Remove NameCache related code duplicated into haddock
Bump haddock submodule
|
| |
|
|
|
|
|
|
|
|
| |
In order to support several home-units and several independent
unit-databases, it's easier to explicitly pass UnitState, DynFlags, etc.
to interface loading functions.
This patch converts some functions using monads such as IfG or TcRnIf
with implicit access to HscEnv to use IO instead and to pass them
specific fields of HscEnv instead of an HscEnv value.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Before this patch, the only way to override GHC's default logging
behavior was to set `log_action`, `dump_action` and `trace_action`
fields in DynFlags. This patch introduces a new Logger abstraction and
stores it in HscEnv instead.
This is part of #17957 (avoid storing state in DynFlags). DynFlags are
duplicated and updated per-module (because of OPTIONS_GHC pragma), so
we shouldn't store global state in them.
This patch also fixes a race in parallel "--make" mode which updated
the `generatedDumps` IORef concurrently.
Bump haddock submodule
The increase in MultilayerModules is tracked in #19293.
Metric Increase:
MultiLayerModules
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The unit database cache, the home unit and the unit state were stored in
DynFlags while they ought to be stored in the compiler session state
(HscEnv). This patch fixes this.
It introduces a new UnitEnv type that should be used in the future to
handle separate unit environments (especially host vs target units).
Related to #17957
Bump haddock submodule
|
| |
|
|
|
|
|
|
|
|
| |
Loaded plugins have nothing to do in DynFlags so this patch moves them
into HscEnv (session state).
"DynFlags plugins" become "Driver plugins" to still be able to register
static plugins.
Bump haddock submodule
|
| |
|
|
|
|
|
|
|
| |
In ticket #18733 we noticed a rather serious deficiency in the current
fingerprinting logic for recursive groups. I have described the old
fingerprinting story and its problems in Note [Fingerprinting recursive
groups] and have reworked the story accordingly to avoid these issues.
Fixes #18733.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
1) Don't modify DynFlags (too much) for -dynamic-too: now when we
generate dynamic outputs for "-dynamic-too", we only set "dynamicNow"
boolean field in DynFlags instead of modifying several other fields.
These fields now have accessors that take dynamicNow into account.
2) Use DynamicTooState ADT to represent -dynamic-too state. It's much
clearer than the undocumented "DynamicTooConditional" that was used
before.
As a result, we can finally remove the hscs_iface_dflags field in
HscRecomp. There was a comment on this field saying:
"FIXME (osa): I don't understand why this is necessary, but I spent
almost two days trying to figure this out and I couldn't .. perhaps
someone who understands this code better will remove this later."
I don't fully understand the details, but it was needed because of the
changes made to the DynFlags for -dynamic-too.
There is still something very dubious in GHC.Iface.Recomp: we have to
disable the "dynamicNow" flag at some point for some Backpack's "heinous
hack" to continue to work. It may be because interfaces for indefinite
units are always non-dynamic, or because we mix and match dynamic and
non-dynamic interfaces (#9176), or something else, who knows?
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
I was working on making DynFlags stateless (#17957), especially by
storing loaded plugins into HscEnv instead of DynFlags. It turned out to
be complicated because HscEnv is in GHC.Driver.Types but LoadedPlugin
isn't: it is in GHC.Driver.Plugins which depends on GHC.Driver.Types. I
didn't feel like introducing yet another hs-boot file to break the loop.
Additionally I remember that while we introduced the module hierarchy
(#13009) we talked about splitting GHC.Driver.Types because it contained
various unrelated types and functions, but we never executed. I didn't
feel like making GHC.Driver.Types bigger with more unrelated Plugins
related types, so finally I bit the bullet and split GHC.Driver.Types.
As a consequence this patch moves a lot of things. I've tried to put
them into appropriate modules but nothing is set in stone.
Several other things moved to avoid loops.
* Removed Binary instances from GHC.Utils.Binary for random compiler
things
* Moved Typeable Binary instances into GHC.Utils.Binary.Typeable: they
import a lot of things that users of GHC.Utils.Binary don't want to
depend on.
* put everything related to Units/Modules under GHC.Unit:
GHC.Unit.Finder, GHC.Unit.Module.{ModGuts,ModIface,Deps,etc.}
* Created several modules under GHC.Types: GHC.Types.Fixity, SourceText,
etc.
* Split GHC.Utils.Error (into GHC.Types.Error)
* Finally removed GHC.Driver.Types
Note that this patch doesn't put loaded plugins into HscEnv. It's left
for another patch.
Bump haddock submodule
|
| |
|
|
|
| |
Instead of recreating the HomeUnit from the DynFlags every time we need
it, we store it in the HscEnv.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Data.OldList exports a monomorphized singleton function but
it is not re-exported by Data.List. Adding the export to
Data.List causes a conflict with a 14-year old function of the
same name and type by SPJ in GHC.Utils.Misc. We can't just remove
this function because that leads to a problems when building
GHC with a stage0 compiler that does not have singleton in
Data.List yet. We also can't hide the function in GHC.Utils.Misc
since it is not possible to hide a function from a module if the
module does not export the function. To work around this, all
places where the Utils.Misc singleton was used now use a qualified
version like Utils.singleton and in GHC.Utils.Misc we are very
specific about which version we export.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
FastStrings can be compared in 2 ways: by Unique or lexically. We don't
want to bless one particular way with an "Ord" instance because it leads
to bugs (#18562) or to suboptimal code (e.g. using lexical comparison
while a Unique comparison would suffice).
UTF-8 encoding has the advantage that sorting strings by their encoded
bytes also sorts them by their Unicode code points, without having to
decode the actual code points. BUT GHC uses Modified UTF-8 which
diverges from UTF-8 by encoding \0 as 0xC080 instead of 0x00 (to avoid
null bytes in the middle of a String so that the string can still be
null-terminated). This patch adds a new `utf8CompareShortByteString`
function that performs sorting by bytes but that also takes Modified
UTF-8 into account. It is much more performant than decoding the strings
into [Char] to perform comparisons (which we did in the previous patch).
Bump haddock submodule
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Since Backpack the "home unit" is much more involved than what it was
before (just an identifier obtained with `-this-unit-id`). Now it is
used in conjunction with `-component-id` and `-instantiated-with` to
configure module instantiations and to detect if we are type-checking an
indefinite unit or compiling a definite one.
This patch introduces a new HomeUnit datatype which is much easier to
understand. Moreover to make GHC support several packages in the same
instances, we will need to handle several HomeUnits so having a
dedicated (documented) type is helpful.
Finally in #14335 we will also need to handle the case where we have no
HomeUnit at all because we are only loading existing interfaces for
plugins which live in a different space compared to units used to
produce target code. Several functions will have to be refactored to
accept "Maybe HomeUnit" parameters instead of implicitly querying the
HomeUnit fields in DynFlags. Having a dedicated type will make this
easier.
Bump haddock submodule
|
| |
|
|
|
|
|
|
|
| |
- put panic related functions into GHC.Utils.Panic
- put trace related functions using DynFlags in GHC.Driver.Ppr
One step closer making Outputable fully independent of DynFlags.
Bump haddock submodule
|
| |
|
|
|
|
|
|
|
| |
They both have the same role and Backend name is more explicit.
Metric Decrease:
T3064
Update Haddock submodule
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The terminology changed over time and now package databases contain
"units" (there can be several units compiled from a single Cabal
package: one per-component, one for each option set, one per
instantiation, etc.). We should try to be consistent internally and use
"units": that's what this renaming does. Maybe one day we'll fix the UI
too (e.g. replace -package-id with -unit-id, we already have
-this-unit-id and ghc-pkg has -unit-id...) but it's not done in this
patch.
* rename getPkgFrameworkOpts into getUnitFrameworkOpts
* rename UnitInfoMap into ClosureUnitInfoMap
* rename InstalledPackageIndex into UnitInfoMap
* rename UnusablePackages into UnusableUnits
* rename PackagePrecedenceIndex into UnitPrecedenceMap
* rename PackageDatabase into UnitDatabase
* rename pkgDatabase into unitDatabases
* rename pkgState into unitState
* rename initPackages into initUnits
* rename renamePackage into renameUnitInfo
* rename UnusablePackageReason into UnusableUnitReason
* rename getPackage* into getUnit*
* etc.
|
| |
|
|
|
| |
* rename thisPackage into homeUnit
* document and refactor several Backpack things
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We often have (ModuleName, Bool) or (Module, Bool) pairs for "extended"
module names (without or with a unit id) disambiguating boot and normal
modules. We think this is important enough across the compiler that it
deserves a new nominal product type. We do this with synnoyms and a
functor named with a `Gen` prefix, matching other newly created
definitions.
It was also requested that we keep custom `IsBoot` / `NotBoot` sum type.
So we have it too. This means changing many the many bools to use that
instead.
Updates `haddock` submodule.
|
| |
|
|
| |
accordingly)
|
| |
|
|
|
|
|
| |
Introduce GHC.Unit.* hierarchy for everything concerning units, packages
and modules.
Update Haddock submodule
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Over the years the unit management code has been modified a lot to keep
up with changes in Cabal (e.g. support for several library components in
the same package), to integrate BackPack, etc. I found it very hard to
understand as the terminology wasn't consistent, was referring to past
concepts, etc.
The terminology is now explained as clearly as I could in the Note
"About Units" and the code is refactored to reflect it.
-------------------
Many names were misleading: UnitId is not an Id but could be a virtual
unit (an indefinite one instantiated on the fly), IndefUnitId
constructor may contain a definite instantiated unit, etc.
* Rename IndefUnitId into InstantiatedUnit
* Rename IndefModule into InstantiatedModule
* Rename UnitId type into Unit
* Rename IndefiniteUnitId constructor into VirtUnit
* Rename DefiniteUnitId constructor into RealUnit
* Rename packageConfigId into mkUnit
* Rename getPackageDetails into unsafeGetUnitInfo
* Rename InstalledUnitId into UnitId
Remove references to misleading ComponentId: a ComponentId is just an
indefinite unit-id to be instantiated.
* Rename ComponentId into IndefUnitId
* Rename ComponentDetails into UnitPprInfo
* Fix display of UnitPprInfo with empty version: this is now used for
units dynamically generated by BackPack
Generalize several types (Module, Unit, etc.) so that they can be used
with different unit identifier types: UnitKey, UnitId, Unit, etc.
* GenModule: Module, InstantiatedModule and InstalledModule are now
instances of this type
* Generalize DefUnitId, IndefUnitId, Unit, InstantiatedUnit,
PackageDatabase
Replace BackPack fake "hole" UnitId by a proper HoleUnit constructor.
Add basic support for UnitKey. They should be used more in the future to
avoid mixing them up with UnitId as we do now.
Add many comments.
Update Haddock submodule
|
| |
|
|
|
|
|
| |
Update Haddock submodule
Metric Increase:
haddock.compiler
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* SysTools
* Parser
* GHC.Builtin
* GHC.Iface.Recomp
* Settings
Update Haddock submodule
Metric Decrease:
Naperian
parsing001
|
| |
|
|
| |
Update Haddock submodule
|
| |
|
|
|
|
|
| |
Update Haddock submodule
Metric Increase:
haddock.compiler
|
| |
|
|
| |
Update submodule: haddock
|