Remove DT_Failed statewip/dyn-too-fixes

At the moment if `-dynamic-too` fails then we rerun the whole pipeline
as if we were just in `-dynamic` mode. I argue this is a misfeature and
we should remove the so-called `DT_Failed` mode.

In what situations do we fall back to `DT_Failed`?

1. If the `dyn_hi` file corresponding to a `hi` file is missing completely.
2. If the interface hash of `dyn_hi` doesn't match the interface hash of `hi`.

What happens in `DT_Failed` mode?

* The whole compiler pipeline is rerun as if the user had just passed `-dynamic`.
* Therefore `dyn_hi/dyn_o` files are used which don't agree with the
  `hi/o` files. (As evidenced by `dynamicToo001` test).
* This is very confusing as now a single compiler invocation has
  produced further `hi`/`dyn_hi` files which are different to each
  other.

Why should we remove it?

* In `--make` mode, which is predominately used `DT_Failed` does not
  work (#19782), there can't be users relying on this functionality.
* In `-c` mode, the recovery doesn't fix the root issue, which is the
  `dyn_hi` and `hi` files are mismatched. We should instead produce an
  error and pass responsibility to the build system using `-c` to ensure
  that the prerequisites for `-dynamic-too` (dyn_hi/hi) files are there
  before we start compiling.
* It is a misfeature to support use cases like `dynamicToo001` which
  allow you to mix different versions of dynamic/non-dynamic interface
  files. It's more likely to lead to subtle bugs in your resulting
  programs where out-dated build products are used rather than a
  deliberate choice.
* In practice, people are usually compiling with `-dynamic-too` rather
  than separately with `-dynamic` and `-static`, so the build products
  always match and `DT_Failed` is only entered due to compiler bugs (see
  !6583)

What should we do instead?

* In `--make` mode, for home packages check during recompilation
  checking that `dyn_hi` and `hi` are both present and agree, recompile
  the modules if they do not.
* For package modules, when loading the interface check that `dyn_hi`
  and `hi` are there and that they agree but fail with an
  error message if they are not.
* In `--oneshot` mode, fail with an error message if the right files
  aren't already there.

Closes #19782 #20446 #9176 #13616

3 files changed, 12 insertions, 100 deletions

diff --git a/compiler/GHC/Driver/Main.hs b/compiler/GHC/Driver/Main.hs
index 1d36a83445..365807fad8 100644
--- a/compiler/GHC/Driver/Main.hs
+++ b/compiler/GHC/Driver/Main.hs
@@ -989,7 +989,7 @@ hscMaybeWriteIface logger dflags is_simple iface old_iface mod_location = do
       --
       let no_change = old_iface == Just (mi_iface_hash (mi_final_exts iface))
 
-      dt <- dynamicTooState dflags
+      let dt = dynamicTooState dflags
 
       when (logHasDumpFlag logger Opt_D_dump_if_trace) $ putMsg logger $
         hang (text "Writing interface(s):") 2 $ vcat
@@ -1003,7 +1003,6 @@ hscMaybeWriteIface logger dflags is_simple iface old_iface mod_location = do
             write_iface dflags iface
             case dt of
                DT_Dont   -> return ()
-               DT_Failed -> return ()
                DT_Dyn    -> panic "Unexpected DT_Dyn state when writing simple interface"
                DT_OK     -> write_iface (setDynamicNow dflags) iface
          else case dt of
@@ -1011,7 +1010,6 @@ hscMaybeWriteIface logger dflags is_simple iface old_iface mod_location = do
                DT_OK   | not no_change             -> write_iface dflags iface
                -- FIXME: see no_change' comment above
                DT_Dyn                              -> write_iface dflags iface
-               DT_Failed | not (dynamicNow dflags) -> write_iface dflags iface
                _                                   -> return ()
 
       when (gopt Opt_WriteHie dflags) $ do
diff --git a/compiler/GHC/Driver/Pipeline.hs b/compiler/GHC/Driver/Pipeline.hs
index 59cb28eccc..55b6a28970 100644
--- a/compiler/GHC/Driver/Pipeline.hs
+++ b/compiler/GHC/Driver/Pipeline.hs
@@ -720,63 +720,6 @@ fullPipeline pipe_env hsc_env pp_fn src_flavour = do
     <- use (T_HscRecomp pipe_env hsc_env' input_fn src_flavour)
   hscPipeline pipe_env (hsc_env_with_plugins, mod_sum, hsc_recomp_status)
 
-checkDynamicToo :: P m => HscEnv -> (HscEnv -> m (ModIface, Maybe Linkable)) -> (ModIface, Maybe Linkable) -> m (ModIface, Maybe Linkable)
-checkDynamicToo hsc_env dyn_too_rerun res = do
-  liftIO (dynamicTooState (hsc_dflags hsc_env)) >>= \case
-      DT_Dont   -> return res
-      DT_Dyn    -> return res
-      DT_OK     -> return res
-      -- If we are compiling a Haskell module with -dynamic-too, we
-      -- first try the "fast path": that is we compile the non-dynamic
-      -- version and at the same time we check that interfaces depended
-      -- on exist both for the non-dynamic AND the dynamic way. We also
-      -- check that they have the same hash.
-      --    If they don't, dynamicTooState is set to DT_Failed.
-      --       See GHC.Iface.Load.checkBuildDynamicToo
-      --    If they do, in the end we produce both the non-dynamic and
-      --    dynamic outputs.
-      --
-      -- If this "fast path" failed, we execute the whole pipeline
-      -- again, this time for the dynamic way *only*. To do that we
-      -- just set the dynamicNow bit from the start to ensure that the
-      -- dynamic DynFlags fields are used and we disable -dynamic-too
-      -- (its state is already set to DT_Failed so it wouldn't do much
-      -- anyway).
-      DT_Failed
-          -- NB: Currently disabled on Windows (ref #7134, #8228, and #5987)
-          | OSMinGW32 <- platformOS (targetPlatform dflags) -> return res
-          | otherwise -> do
-              liftIO (debugTraceMsg logger 4
-                        (text "Running the full pipeline again for -dynamic-too"))
-              hsc_env' <- liftIO (resetHscEnv hsc_env)
-              dyn_too_rerun hsc_env'
-  where
-    dflags = hsc_dflags hsc_env
-    logger = hsc_logger hsc_env
-
--- | Enable dynamic-too, reset EPS
-resetHscEnv :: HscEnv -> IO HscEnv
-resetHscEnv hsc_env = do
-  let init_dflags = hsc_dflags hsc_env
-      dflags0 = flip gopt_unset Opt_BuildDynamicToo
-                     $ setDynamicNow                                  -- -dynamic
-                     $ (init_dflags { hiSuf_ = dynHiSuf_ init_dflags  -- -hisuf = -dynhisuf
-                                    , objectSuf_ =  dynObjectSuf_ init_dflags -- -osuf = -dynosuf
-                                    })
-  hsc_env' <- newHscEnv dflags0
-  (dbs,unit_state,home_unit,mconstants) <- initUnits (hsc_logger hsc_env) dflags0 Nothing
-  dflags1 <- updatePlatformConstants dflags0 mconstants
-  unit_env0 <- initUnitEnv (ghcNameVersion dflags1) (targetPlatform dflags1)
-  let unit_env = unit_env0
-        { ue_home_unit = Just home_unit
-        , ue_units     = unit_state
-        , ue_unit_dbs  = Just dbs
-        }
-  let hsc_env'' = hscSetFlags dflags1 $ hsc_env'
-       { hsc_unit_env = unit_env
-       }
-  return hsc_env''
-
 -- | Everything after preprocess
 hscPipeline :: P m => PipeEnv ->  ((HscEnv, ModSummary, HscRecompStatus)) -> m (ModIface, Maybe Linkable)
 hscPipeline pipe_env (hsc_env_with_plugins, mod_sum, hsc_recomp_status) = do
@@ -785,10 +728,7 @@ hscPipeline pipe_env (hsc_env_with_plugins, mod_sum, hsc_recomp_status) = do
     HscRecompNeeded mb_old_hash -> do
       (tc_result, warnings) <- use (T_Hsc hsc_env_with_plugins mod_sum)
       hscBackendAction <- use (T_HscPostTc hsc_env_with_plugins mod_sum tc_result warnings mb_old_hash )
-      res <- hscBackendPipeline pipe_env hsc_env_with_plugins mod_sum hscBackendAction
-      -- Once the pipeline has finished, check to see if -dynamic-too failed and
-      -- rerun again if it failed but just the `--dynamic` way.
-      checkDynamicToo hsc_env_with_plugins (\hsc' -> hscPipeline pipe_env (hsc', mod_sum, hsc_recomp_status)) res
+      hscBackendPipeline pipe_env hsc_env_with_plugins mod_sum hscBackendAction
 
 hscBackendPipeline :: P m => PipeEnv -> HscEnv -> ModSummary -> HscBackendAction -> m (ModIface, Maybe Linkable)
 hscBackendPipeline pipe_env hsc_env mod_sum result =
@@ -801,11 +741,9 @@ hscBackendPipeline pipe_env hsc_env mod_sum result =
     -- Interpreter -> (,) <$> use (T_IO (mkFullIface hsc_env (hscs_partial_iface result) Nothing)) <*> pure Nothing
     _ -> do
       res <- hscGenBackendPipeline pipe_env hsc_env mod_sum result
-      liftIO (dynamicTooState (hsc_dflags hsc_env)) >>= \case
-        DT_OK -> do
+      when (gopt Opt_BuildDynamicToo (hsc_dflags hsc_env)) $ do
           let dflags' = setDynamicNow (hsc_dflags hsc_env) -- set "dynamicNow"
           () <$ hscGenBackendPipeline pipe_env (hscSetFlags dflags' hsc_env) mod_sum result
-        _ -> return ()
       return res
 
 hscGenBackendPipeline :: P m
diff --git a/compiler/GHC/Driver/Session.hs b/compiler/GHC/Driver/Session.hs
index 3342091bfa..8f1cd31ece 100644
--- a/compiler/GHC/Driver/Session.hs
+++ b/compiler/GHC/Driver/Session.hs
@@ -37,7 +37,7 @@ module GHC.Driver.Session (
         xopt_DuplicateRecordFields,
         xopt_FieldSelectors,
         lang_set,
-        DynamicTooState(..), dynamicTooState, setDynamicNow, setDynamicTooFailed,
+        DynamicTooState(..), dynamicTooState, setDynamicNow,
         sccProfilingEnabled,
         DynFlags(..),
         outputFile, objectSuf, ways,
@@ -530,7 +530,6 @@ data DynFlags = DynFlags {
   hiSuf_                :: String,
   hieSuf                :: String,
 
-  dynamicTooFailed      :: IORef Bool,
   dynObjectSuf_         :: String,
   dynHiSuf_             :: String,
 
@@ -1020,33 +1019,21 @@ positionIndependent dflags = gopt Opt_PIC dflags || gopt Opt_PIE dflags
 -- need Template-Haskell and GHC is dynamically linked (cf
 -- GHC.Driver.Pipeline.compileOne').
 --
--- This somewhat explains why we have "dynamicTooFailed :: IORef Bool" in
--- DynFlags: when -dynamic-too is enabled, we try to build the dynamic objects,
--- but we may fail and we shouldn't abort the whole compilation because the user
--- may not even have asked for -dynamic-too in the first place. So instead we
--- use this global variable to indicate that we can't build dynamic objects and
--- compilation continues to build non-dynamic objects only. At the end of the
--- non-dynamic pipeline, if this value indicates that the dynamic compilation
--- failed, we run the whole pipeline again for the dynamic way (except on
--- Windows...). See GHC.Driver.Pipeline.runPipeline.
+-- We used to try and fall back from a dynamic-too failure but this feature
+-- didn't work as expected (#20446) so it was removed to simplify the
+-- implementation and not obscure latent bugs.
 
 data DynamicTooState
    = DT_Dont    -- ^ Don't try to build dynamic objects too
-   | DT_Failed  -- ^ Won't try to generate dynamic objects for some reason
    | DT_OK      -- ^ Will still try to generate dynamic objects
    | DT_Dyn     -- ^ Currently generating dynamic objects (in the backend)
    deriving (Eq,Show,Ord)
 
-dynamicTooState :: MonadIO m => DynFlags -> m DynamicTooState
+dynamicTooState :: DynFlags -> DynamicTooState
 dynamicTooState dflags
-   | not (gopt Opt_BuildDynamicToo dflags) = return DT_Dont
-   | otherwise = do
-      failed <- liftIO $ readIORef (dynamicTooFailed dflags)
-      if failed
-         then return DT_Failed
-         else if dynamicNow dflags
-               then return DT_Dyn
-               else return DT_OK
+   | not (gopt Opt_BuildDynamicToo dflags) = DT_Dont
+   | dynamicNow dflags = DT_Dyn
+   | otherwise = DT_OK
 
 setDynamicNow :: DynFlags -> DynFlags
 setDynamicNow dflags0 =
@@ -1054,21 +1041,12 @@ setDynamicNow dflags0 =
       { dynamicNow = True
       }
 
-setDynamicTooFailed :: MonadIO m => DynFlags -> m ()
-setDynamicTooFailed dflags =
-   liftIO $ writeIORef (dynamicTooFailed dflags) True
-
 -----------------------------------------------------------------------------
 
 -- | Used by 'GHC.runGhc' to partially initialize a new 'DynFlags' value
 initDynFlags :: DynFlags -> IO DynFlags
 initDynFlags dflags = do
- let -- We can't build with dynamic-too on Windows, as labels before
-     -- the fork point are different depending on whether we are
-     -- building dynamically or not.
-     platformCanGenerateDynamicToo
-         = platformOS (targetPlatform dflags) /= OSMinGW32
- refDynamicTooFailed <- newIORef (not platformCanGenerateDynamicToo)
+ let
  refRtldInfo <- newIORef Nothing
  refRtccInfo <- newIORef Nothing
  refRtasmInfo <- newIORef Nothing
@@ -1089,7 +1067,6 @@ initDynFlags dflags = do
        (useColor dflags, colScheme dflags)
  tmp_dir <- normalise <$> getTemporaryDirectory
  return dflags{
-        dynamicTooFailed = refDynamicTooFailed,
         useUnicode    = useUnicode',
         useColor      = useColor',
         canUseColor   = stderrSupportsAnsiColors,
@@ -1163,7 +1140,6 @@ defaultDynFlags mySettings llvmConfig =
         hiSuf_                  = "hi",
         hieSuf                  = "hie",
 
-        dynamicTooFailed        = panic "defaultDynFlags: No dynamicTooFailed",
         dynObjectSuf_           = "dyn_" ++ phaseInputExt StopLn,
         dynHiSuf_               = "dyn_hi",
         dynamicNow              = False,