diff options
author | Andreas Klebinger <klebinger.andreas@gmx.at> | 2022-05-04 10:50:04 +0200 |
---|---|---|
committer | Marge Bot <ben+marge-bot@smart-cactus.org> | 2022-06-27 08:01:39 -0400 |
commit | ac7a7fc88b51f9fb4e84499397e12eb0081ba79e (patch) | |
tree | 075714e3c20f6aa770e8a5cb508112436fe466b5 /compiler/GHC/Core/Opt | |
parent | 38378be3506f0d4f597fcd5aa2d9db3124fbf535 (diff) | |
download | haskell-ac7a7fc88b51f9fb4e84499397e12eb0081ba79e.tar.gz |
Don't mark lambda binders as OtherCon
We used to put OtherCon unfoldings on lambda binders of workers
and sometimes also join points/specializations with with the
assumption that since the wrapper would force these arguments
once we execute the RHS they would indeed be in WHNF.
This was wrong for reasons detailed in #21472. So now we purge
evaluated unfoldings from *all* lambda binders.
This fixes #21472, but at the cost of sometimes not using as efficient a
calling convention. It can also change inlining behaviour as some
occurances will no longer look like value arguments when they did
before.
As consequence we also change how we compute CBV information for
arguments slightly. We now *always* determine the CBV convention
for arguments during tidy. Earlier in the pipeline we merely mark
functions as candidates for having their arguments treated as CBV.
As before the process is described in the relevant notes:
Note [CBV Function Ids]
Note [Attaching CBV Marks to ids]
Note [Never put `OtherCon` unfoldigns on lambda binders]
-------------------------
Metric Decrease:
T12425
T13035
T18223
T18223
T18923
MultiLayerModulesTH_OneShot
Metric Increase:
WWRec
-------------------------
Diffstat (limited to 'compiler/GHC/Core/Opt')
-rw-r--r-- | compiler/GHC/Core/Opt/Arity.hs | 4 | ||||
-rw-r--r-- | compiler/GHC/Core/Opt/Simplify.hs | 66 | ||||
-rw-r--r-- | compiler/GHC/Core/Opt/SpecConstr.hs | 159 | ||||
-rw-r--r-- | compiler/GHC/Core/Opt/WorkWrap.hs | 15 | ||||
-rw-r--r-- | compiler/GHC/Core/Opt/WorkWrap/Utils.hs | 103 |
5 files changed, 158 insertions, 189 deletions
diff --git a/compiler/GHC/Core/Opt/Arity.hs b/compiler/GHC/Core/Opt/Arity.hs index 5edc5e5bb0..84af26e257 100644 --- a/compiler/GHC/Core/Opt/Arity.hs +++ b/compiler/GHC/Core/Opt/Arity.hs @@ -2156,7 +2156,7 @@ eta-reduce that are specific to Core and GHC: See Note [Eta expanding primops]. W. We may not undersaturate StrictWorkerIds. - See Note [Strict Worker Ids] in GHC.CoreToStg.Prep. + See Note [CBV Function Ids] in GHC.CoreToStg.Prep. Here is a list of historic accidents surrounding unsound eta-reduction: @@ -2474,7 +2474,7 @@ canEtaReduceToArity fun dest_join_arity dest_arity = || ( dest_arity < idCbvMarkArity fun ) -- (W) -- Don't undersaturate StrictWorkerIds. - -- See Note [Strict Worker Ids] in GHC.CoreToStg.Prep. + -- See Note [CBV Function Ids] in GHC.CoreToStg.Prep. || isLinearType (idType fun) -- (L) -- Don't perform eta reduction on linear types. diff --git a/compiler/GHC/Core/Opt/Simplify.hs b/compiler/GHC/Core/Opt/Simplify.hs index 1523394be9..de049523cc 100644 --- a/compiler/GHC/Core/Opt/Simplify.hs +++ b/compiler/GHC/Core/Opt/Simplify.hs @@ -78,7 +78,6 @@ import GHC.Utils.Misc import Control.Monad - {- The guts of the simplifier is in this module, but the driver loop for the simplifier is in GHC.Core.Opt.Pipeline @@ -1705,8 +1704,9 @@ simplLamBndr :: SimplEnv -> InBndr -> SimplM (SimplEnv, OutBndr) -- Historically this had a special case for when a lambda-binder -- could have a stable unfolding; -- see Historical Note [Case binders and join points] --- But now it is much simpler! -simplLamBndr env bndr = simplBinder env bndr +-- But now it is much simpler! We now only remove unfoldings. +-- See Note [Never put `OtherCon` unfoldings on lambda binders] +simplLamBndr env bndr = simplBinder env (zapIdUnfolding bndr) simplLamBndrs :: SimplEnv -> [InBndr] -> SimplM (SimplEnv, [OutBndr]) simplLamBndrs env bndrs = mapAccumLM simplLamBndr env bndrs @@ -3130,7 +3130,7 @@ simplAlt env scrut' _ case_bndr' cont' (Alt (LitAlt lit) bndrs rhs) simplAlt env scrut' _ case_bndr' cont' (Alt (DataAlt con) vs rhs) = do { -- See Note [Adding evaluatedness info to pattern-bound variables] let vs_with_evals = addEvals scrut' con vs - ; (env', vs') <- simplLamBndrs env vs_with_evals + ; (env', vs') <- simplBinders env vs_with_evals -- Bind the case-binder to (con args) ; let inst_tys' = tyConAppArgs (idType case_bndr') @@ -3654,37 +3654,59 @@ mkDupableStrictBind env arg_bndr join_rhs res_ty mkDupableAlt :: Platform -> OutId -> JoinFloats -> OutAlt -> SimplM (JoinFloats, OutAlt) -mkDupableAlt _platform case_bndr jfloats (Alt con bndrs' rhs') - | exprIsTrivial rhs' -- See point (2) of Note [Duplicating join points] - = return (jfloats, Alt con bndrs' rhs') +mkDupableAlt _platform case_bndr jfloats (Alt con alt_bndrs alt_rhs_in) + | exprIsTrivial alt_rhs_in -- See point (2) of Note [Duplicating join points] + = return (jfloats, Alt con alt_bndrs alt_rhs_in) | otherwise - = do { let rhs_ty' = exprType rhs' - - final_bndrs' - | isDeadBinder case_bndr = filter abstract_over bndrs' - | otherwise = bndrs' ++ [case_bndr] - - abstract_over bndr - | isTyVar bndr = True -- Abstract over all type variables just in case - | otherwise = not (isDeadBinder bndr) - -- The deadness info on the new Ids is preserved by simplBinders - final_args = varsToCoreExprs final_bndrs' + = do { let rhs_ty' = exprType alt_rhs_in + + bangs + | DataAlt c <- con + = dataConRepStrictness c + | otherwise = [] + + abstracted_binders = abstract_binders alt_bndrs bangs + + abstract_binders :: [Var] -> [StrictnessMark] -> [(Id,StrictnessMark)] + abstract_binders [] [] + -- Abstract over the case binder too if it's used. + | isDeadBinder case_bndr = [] + | otherwise = [(case_bndr,MarkedStrict)] + abstract_binders (alt_bndr:alt_bndrs) marks + -- Abstract over all type variables just in case + | isTyVar alt_bndr = (alt_bndr,NotMarkedStrict) : abstract_binders alt_bndrs marks + abstract_binders (alt_bndr:alt_bndrs) (mark:marks) + -- The deadness info on the new Ids is preserved by simplBinders + -- We don't abstract over dead ids here. + | isDeadBinder alt_bndr = abstract_binders alt_bndrs marks + | otherwise = (alt_bndr,mark) : abstract_binders alt_bndrs marks + abstract_binders _ _ = pprPanic "abstrict_binders - failed to abstract" (ppr $ Alt con alt_bndrs alt_rhs_in) + + filtered_binders = map fst abstracted_binders + -- We want to make any binder with an evaldUnfolding strict in the rhs. + -- See Note [Call-by-value for worker args] (which also applies to join points) + (rhs_with_seqs) = mkStrictFieldSeqs abstracted_binders alt_rhs_in + + final_args = varsToCoreExprs filtered_binders -- Note [Join point abstraction] -- We make the lambdas into one-shot-lambdas. The -- join point is sure to be applied at most once, and doing so -- prevents the body of the join point being floated out by -- the full laziness pass - really_final_bndrs = map one_shot final_bndrs' + final_bndrs = map one_shot filtered_binders one_shot v | isId v = setOneShotLambda v | otherwise = v - join_rhs = mkLams really_final_bndrs rhs' - ; join_bndr <- newJoinId final_bndrs' rhs_ty' + -- No lambda binder has an unfolding, but (currently) case binders can, + -- so we must zap them here. + join_rhs = mkLams (map zapIdUnfolding final_bndrs) rhs_with_seqs + + ; join_bndr <- newJoinId filtered_binders rhs_ty' ; let join_call = mkApps (Var join_bndr) final_args - alt' = Alt con bndrs' join_call + alt' = Alt con alt_bndrs join_call ; return ( jfloats `addJoinFlts` unitJoinFloat (NonRec join_bndr join_rhs) , alt') } diff --git a/compiler/GHC/Core/Opt/SpecConstr.hs b/compiler/GHC/Core/Opt/SpecConstr.hs index 8b303f0316..9ef2d3e3e6 100644 --- a/compiler/GHC/Core/Opt/SpecConstr.hs +++ b/compiler/GHC/Core/Opt/SpecConstr.hs @@ -628,41 +628,12 @@ regardless of size; and then we needed a way to turn that *off*. Now that we have ForceSpecConstr, this NoSpecConstr is probably redundant. (Used only for PArray, TODO: remove?) -Note [SpecConstr and evaluated unfoldings] +Note [SpecConstr and strict fields] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -SpecConstr will attach evaldUnfolding unfoldings to function -arguments representing things that should be fully evaluated -by the time we execute the RHS. - -This primarily concerns strict fields. To give an example in the -containers package we have a merge function with this specialization: - - "SC:$wmerge01" [2] - forall (sc_s5lX :: ghc-prim:GHC.Prim.Int#) - (sc_s5lY :: ghc-prim:GHC.Prim.Int#) - (sc_s5lZ - :: IntMap a_s4UX - Unf=OtherCon []) - (sc_s5m0 - :: IntMap a_s4UX - Unf=OtherCon []) - (sc_s5lW :: ghc-prim:GHC.Prim.Int#) - (sc_s5lU :: ghc-prim:GHC.Prim.Int#) - (sc_s5lV :: a_s4UX). - $wmerge0_s4UK (Data.IntMap.Internal.Tip @a_s4UX sc_s5lU sc_s5lV) - (ghc-prim:GHC.Types.I# sc_s5lW) - (Data.IntMap.Internal.Bin - @a_s4UX sc_s5lX sc_s5lY sc_s5lZ sc_s5m0) - = $s$wmerge0_s5m2 - sc_s5lX sc_s5lY sc_s5lZ sc_s5m0 sc_s5lW sc_s5lU sc_s5lV] - -We give sc_s5lZ and sc_s5m0 a evaluated unfolding since they come out of -strict field fields in the Bin constructor. -This is especially important since tag inference can then use this -information to adjust the calling convention of -`$wmerge0_s4UK` to enforce arguments being passed fully evaluated+tagged. -See Note [Tag Inference], Note [Strict Worker Ids] for more information on -how we can take advantage of this. +We treat strict fields in SpecConstr the same way we do in W/W. +That is we make the specialized function strict in arguments +representing strict fields. See Note [Call-by-value for worker args] +for why we do this. Note [Specialising on dictionaries] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -797,6 +768,7 @@ specConstrProgram guts us <- getUniqueSupplyM (_, annos) <- getFirstAnnotations deserializeWithData guts this_mod <- getModule + -- pprTraceM "specConstrInput" (ppr $ mg_binds guts) let binds' = reverse $ fst $ initUs us $ do -- Note [Top-level recursive groups] (env, binds) <- goEnv (initScEnv (initScOpts dflags this_mod) annos) @@ -1815,7 +1787,7 @@ spec_one :: ScEnv -} spec_one env fn arg_bndrs body (call_pat, rule_number) - | CP { cp_qvars = qvars, cp_args = pats } <- call_pat + | CP { cp_qvars = qvars, cp_args = pats, cp_strict_args = cbv_args } <- call_pat = do { spec_uniq <- getUniqueM ; let env1 = extendScSubstList (extendScInScope env qvars) (arg_bndrs `zip` pats) @@ -1858,17 +1830,14 @@ spec_one env fn arg_bndrs body (call_pat, rule_number) , (spec_lam_args, spec_call_args, _) <- addVoidWorkerArg spec_lam_args1 [] -- needsVoidWorkerArg: usual w/w hack to avoid generating -- a spec_rhs of unlifted type and no args. - -- Unlike W/W we don't turn functions into strict workers - -- immediately here instead letting tidy handle this. - -- For this reason we can ignore the cbv marks. - -- See Note [Strict Worker Ids]. See Note [Tag Inference]. , !spec_arity <- spec_arity1 + 1 , !spec_join_arity <- fmap (+ 1) spec_join_arity1 = (spec_lam_args, spec_call_args, spec_arity, spec_join_arity) | otherwise = (spec_lam_args1, spec_lam_args1, spec_arity1, spec_join_arity1) - spec_id = mkLocalId spec_name Many + spec_id = asWorkerLikeId $ + mkLocalId spec_name Many (mkLamTypes spec_lam_args spec_body_ty) -- See Note [Transfer strictness] `setIdDmdSig` spec_sig @@ -1876,8 +1845,8 @@ spec_one env fn arg_bndrs body (call_pat, rule_number) `setIdArity` spec_arity `asJoinId_maybe` spec_join_arity - -- Conditionally use result of new worker-wrapper transform - spec_rhs = mkLams spec_lam_args spec_body + -- Conditionally use result of new worker-wrapper transform + spec_rhs = mkLams spec_lam_args (mkSeqs cbv_args spec_body_ty spec_body) rule_rhs = mkVarApps (Var spec_id) $ -- This will give us all the arguments we quantify over -- in the rule plus the void argument if present @@ -1889,25 +1858,39 @@ spec_one env fn arg_bndrs body (call_pat, rule_number) rule_name inline_act fn_name qvars pats rule_rhs -- See Note [Transfer activation] - -- ; pprTrace "spec_one {" (vcat [ text "function:" <+> ppr fn <+> ppr (idUnique fn) + -- ; pprTraceM "spec_one {" (vcat [ text "function:" <+> ppr fn <+> braces (ppr (idUnique fn)) -- , text "sc_count:" <+> ppr (sc_count env) -- , text "pats:" <+> ppr pats -- , text "call_pat:" <+> ppr call_pat -- , text "-->" <+> ppr spec_name -- , text "bndrs" <+> ppr arg_bndrs -- , text "extra_bndrs" <+> ppr extra_bndrs + -- , text "cbv_args" <+> ppr cbv_args -- , text "spec_lam_args" <+> ppr spec_lam_args -- , text "spec_call_args" <+> ppr spec_call_args -- , text "rule_rhs" <+> ppr rule_rhs - -- , text "adds_void_worker_arg" <+> ppr adds_void_worker_arg + -- , text "adds_void_worker_arg" <+> ppr add_void_arg -- , text "body" <+> ppr body -- , text "spec_rhs" <+> ppr spec_rhs - -- , text "how_bound" <+> ppr (sc_how_bound env) ]) $ - -- return () + -- , text "how_bound" <+> ppr (sc_how_bound env) ]) ; return (spec_usg, OS { os_pat = call_pat, os_rule = rule , os_id = spec_id , os_rhs = spec_rhs }) } +-- See Note [SpecConstr and strict fields] +mkSeqs :: [Var] -> Type -> CoreExpr -> CoreExpr +mkSeqs seqees res_ty rhs = + foldr addEval rhs seqees + where + addEval :: Var -> CoreExpr -> CoreExpr + addEval arg_id rhs + -- Argument representing strict field and it's worth passing via cbv + | shouldStrictifyIdForCbv arg_id + = Case (Var arg_id) arg_id res_ty ([Alt DEFAULT [] rhs]) + | otherwise + = rhs + + {- Note [SpecConst needs to add void args first] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Consider a function @@ -2197,13 +2180,15 @@ only in kind-casts, but I'm doing the simple thing for now. -} data CallPat = CP { cp_qvars :: [Var] -- Quantified variables - , cp_args :: [CoreExpr] } -- Arguments + , cp_args :: [CoreExpr] -- Arguments + , cp_strict_args :: [Var] } -- Arguments we want to pass unlifted even if they are boxed -- See Note [SpecConstr call patterns] instance Outputable CallPat where - ppr (CP { cp_qvars = qvars, cp_args = args }) + ppr (CP { cp_qvars = qvars, cp_args = args, cp_strict_args = strict }) = text "CP" <> braces (sep [ text "cp_qvars =" <+> ppr qvars <> comma - , text "cp_args =" <+> ppr args ]) + , text "cp_args =" <+> ppr args + , text "cp_strict_args = " <> ppr strict ]) callsToNewPats :: ScEnv -> Id -> SpecInfo @@ -2315,16 +2300,16 @@ callToPats :: ScEnv -> [ArgOcc] -> Call -> UniqSM (Maybe CallPat) callToPats env bndr_occs call@(Call fn args con_env) = do { let in_scope = substInScope (sc_subst env) - ; pairs <- zipWith3M (argToPat env in_scope con_env) args bndr_occs (map (const NotMarkedStrict) args) + ; arg_tripples <- zipWith3M (argToPat env in_scope con_env) args bndr_occs (map (const NotMarkedStrict) args) -- This zip trims the args to be no longer than -- the lambdas in the function definition (bndr_occs) -- Drop boring patterns from the end -- See Note [SpecConstr call patterns] - ; let pairs' | isJoinId fn = pairs - | otherwise = dropWhileEnd is_boring pairs - is_boring (interesting, _) = not interesting - (interesting_s, pats) = unzip pairs' + ; let arg_tripples' | isJoinId fn = arg_tripples + | otherwise = dropWhileEnd is_boring arg_tripples + is_boring (interesting, _,_) = not interesting + (interesting_s, pats, cbv_ids) = unzip3 arg_tripples' interesting = or interesting_s ; let pat_fvs = exprsFreeVarsList pats @@ -2374,7 +2359,7 @@ callToPats env bndr_occs call@(Call fn args con_env) -- text "pat_fvs:" <+> ppr pat_fvs -- ) -- ppr (CP { cp_qvars = qvars', cp_args = pats })) >> - return (Just (CP { cp_qvars = qvars', cp_args = pats })) + return (Just (CP { cp_qvars = qvars', cp_args = pats, cp_strict_args = concat cbv_ids })) else return Nothing } -- argToPat takes an actual argument, and returns an abstracted @@ -2389,8 +2374,8 @@ argToPat :: ScEnv -> CoreArg -- A call arg (or component thereof) -> ArgOcc -> StrictnessMark -- Tells us if this argument is a strict field of a data constructor - -- See Note [SpecConstr and evaluated unfoldings] - -> UniqSM (Bool, CoreArg) + -- See Note [SpecConstr and strict fields] + -> UniqSM (Bool, CoreArg, [Id]) -- Returns (interesting, pat), -- where pat is the pattern derived from the argument @@ -2415,12 +2400,12 @@ argToPat1 :: ScEnv -> Expr CoreBndr -> ArgOcc -> StrictnessMark - -> UniqSM (Bool, Expr CoreBndr) + -> UniqSM (Bool, Expr CoreBndr, [Id]) argToPat1 _env _in_scope _val_env arg@(Type {}) _arg_occ _arg_str - = return (False, arg) + = return (False, arg, []) -argToPat1 env in_scope val_env (Tick _ arg) arg_occ _arg_str - = argToPat env in_scope val_env arg arg_occ _arg_str +argToPat1 env in_scope val_env (Tick _ arg) arg_occ arg_str + = argToPat env in_scope val_env arg arg_occ arg_str -- Note [Tick annotations in call patterns] -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Ignore Notes. In particular, we want to ignore any InlineMe notes @@ -2444,7 +2429,7 @@ argToPat env in_scope val_env (Case scrut _ _ [(_, _, rhs)]) arg_occ argToPat1 env in_scope val_env (Cast arg co) arg_occ arg_str | not (ignoreType env ty2) - = do { (interesting, arg') <- argToPat env in_scope val_env arg arg_occ arg_str + = do { (interesting, arg', strict_args) <- argToPat env in_scope val_env arg arg_occ arg_str ; if not interesting then wildCardPat ty2 arg_str else do @@ -2452,7 +2437,7 @@ argToPat1 env in_scope val_env (Cast arg co) arg_occ arg_str uniq <- getUniqueM ; let co_name = mkSysTvName uniq (fsLit "sg") co_var = mkCoVar co_name (mkCoercionType Representational ty1 ty2) - ; return (interesting, Cast arg' (mkCoVarCo co_var)) } } + ; return (interesting, Cast arg' (mkCoVarCo co_var), strict_args) } } where Pair ty1 ty2 = coercionKind co @@ -2488,17 +2473,12 @@ argToPat1 env in_scope val_env arg arg_occ _arg_str -- ppr rest_args $$ -- ppr (map isTypeArg rest_args)) ; prs <- zipWith3M (argToPat env in_scope val_env) rest_args arg_occs matched_str - ; let args' = map snd prs :: [CoreArg] + ; let args' = map sndOf3 prs :: [CoreArg] ; assertPpr (length con_str == length (filter isRuntimeArg rest_args)) ( ppr con_str $$ ppr rest_args $$ ppr (length con_str) $$ ppr (length rest_args) ) $ return () - -- ; assert (length con_str == length rest_args) $ - -- pprTraceM "argToPat" - -- ( parens (int $ length con_str) <> ppr con_str $$ - -- ppr rest_args $$ - -- ppr prs) - ; return (True, mkConApp dc (ty_args ++ args')) } + ; return (True, mkConApp dc (ty_args ++ args'), concat (map thdOf3 prs)) } where mb_scrut dc = case arg_occ of ScrutOcc bs | Just occs <- lookupUFM bs dc @@ -2526,7 +2506,7 @@ argToPat1 env in_scope val_env arg arg_occ _arg_str -- business of absence analysis, not SpecConstr.) -- (b) we know what its value is -- In that case it counts as "interesting" -argToPat1 env in_scope val_env (Var v) arg_occ _arg_str +argToPat1 env in_scope val_env (Var v) arg_occ arg_str | sc_force env || case arg_occ of { ScrutOcc {} -> True ; UnkOcc -> False ; NoOcc -> False } -- (a) @@ -2535,7 +2515,8 @@ argToPat1 env in_scope val_env (Var v) arg_occ _arg_str -- So sc_keen focused just on f (I# x), where we have freshly-allocated -- box that we can eliminate in the caller , not (ignoreType env (varType v)) - = return (True, Var (setStrUnfolding v MarkedStrict)) + -- See Note [SpecConstr and strict fields] + = return (True, Var v, if isMarkedStrict arg_str then [v] else mempty) where is_value | isLocalId v = v `elemInScopeSet` in_scope @@ -2567,41 +2548,12 @@ argToPat1 env in_scope val_env (Var v) arg_occ _arg_str argToPat1 _env _in_scope _val_env arg _arg_occ arg_str = wildCardPat (exprType arg) arg_str --- We want the given id to be passed call-by-value if it's MarkedCbv. --- For some, but not all ids this can be achieved by giving them an OtherCon unfolding. --- Doesn't touch existing value unfoldings. --- See Note [SpecConstr and evaluated unfoldings] -setStrUnfolding :: Id -> StrictnessMark -> Id --- setStrUnfolding id str = id -setStrUnfolding id str - -- pprTrace "setStrUnfolding" - -- (ppr id <+> ppr (isMarkedCbv str) $$ - -- ppr (idType id) $$ - -- text "boxed:" <> ppr (isBoxedType (idType id)) $$ - -- text "unlifted:" <> ppr (isUnliftedType (idType id)) - -- ) - -- False - -- = undefined - | not (isId id) || isEvaldUnfolding (idUnfolding id) - = id - | isMarkedStrict str - , not $ isUnliftedType (idType id) -- Pointless to stick an evald unfolding on unlifted types - = -- trace "setStrUnfolding2" $ - assert (isId id) $ - assert (not $ hasCoreUnfolding $ idUnfolding id) $ - id `setIdUnfolding` evaldUnfolding - | otherwise - = -- trace "setStrUnfolding3" - id - -- | wildCardPats are always boring -wildCardPat :: Type -> StrictnessMark -> UniqSM (Bool, CoreArg) +wildCardPat :: Type -> StrictnessMark -> UniqSM (Bool, CoreArg, [Id]) wildCardPat ty str = do { id <- mkSysLocalOrCoVarM (fsLit "sc") Many ty - ; let id' = id `setStrUnfolding` str - -- See Note [SpecConstr and evaluated unfoldings] -- ; pprTraceM "wildCardPat" (ppr id' <+> ppr (idUnfolding id')) - ; return (False, varToCoreExpr id') } + ; return (False, varToCoreExpr id, if isMarkedStrict str then [id] else []) } isValue :: ValueEnv -> CoreExpr -> Maybe Value isValue _env (Lit lit) @@ -2659,6 +2611,7 @@ samePat (CP { cp_qvars = vs1, cp_args = as1 }) (CP { cp_qvars = vs2, cp_args = as2 }) = all2 same as1 as2 where + -- If the args are the same, their strictness marks will be too so we don't compare those. same (Var v1) (Var v2) | v1 `elem` vs1 = v2 `elem` vs2 | v2 `elem` vs2 = False diff --git a/compiler/GHC/Core/Opt/WorkWrap.hs b/compiler/GHC/Core/Opt/WorkWrap.hs index 1c7a728d12..8b3e0f0e43 100644 --- a/compiler/GHC/Core/Opt/WorkWrap.hs +++ b/compiler/GHC/Core/Opt/WorkWrap.hs @@ -36,6 +36,7 @@ import GHC.Utils.Panic import GHC.Utils.Panic.Plain import GHC.Utils.Monad import GHC.Utils.Trace +import GHC.Core.DataCon {- We take Core bindings whose binders have: @@ -535,7 +536,7 @@ tryWW ww_opts is_rec fn_id rhs -- See Note [Drop absent bindings] | isAbsDmd (demandInfo fn_info) , not (isJoinId fn_id) - , Just filler <- mkAbsentFiller ww_opts fn_id + , Just filler <- mkAbsentFiller ww_opts fn_id NotMarkedStrict = return [(new_fn_id, filler)] -- See Note [Don't w/w INLINE things] @@ -788,10 +789,10 @@ splitFun ww_opts fn_id rhs mkWWBindPair :: WwOpts -> Id -> IdInfo -> [Var] -> CoreExpr -> Unique -> Divergence - -> ([Demand],[CbvMark], JoinArity, Id -> CoreExpr, Expr CoreBndr -> CoreExpr) + -> ([Demand],JoinArity, Id -> CoreExpr, Expr CoreBndr -> CoreExpr) -> [(Id, CoreExpr)] mkWWBindPair ww_opts fn_id fn_info fn_args fn_body work_uniq div - (work_demands, cbv_marks :: [CbvMark], join_arity, wrap_fn, work_fn) + (work_demands, join_arity, wrap_fn, work_fn) = -- pprTrace "mkWWBindPair" (ppr fn_id <+> ppr wrap_id <+> ppr work_id $$ ppr wrap_rhs) $ [(work_id, work_rhs), (wrap_id, wrap_rhs)] -- Worker first, because wrapper mentions it @@ -821,7 +822,8 @@ mkWWBindPair ww_opts fn_id fn_info fn_args fn_body work_uniq div -- worker is join point iff wrapper is join point -- (see Note [Don't w/w join points for CPR]) - work_id = mkWorkerId work_uniq fn_id (exprType work_rhs) + work_id = asWorkerLikeId $ + mkWorkerId work_uniq fn_id (exprType work_rhs) `setIdOccInfo` occInfo fn_info -- Copy over occurrence info from parent -- Notably whether it's a loop breaker @@ -846,10 +848,7 @@ mkWWBindPair ww_opts fn_id fn_info fn_args fn_body work_uniq div -- arity is consistent with the demand type goes -- through - `setIdCbvMarks` cbv_marks - `asJoinId_maybe` work_join_arity - -- `setIdThing` (undefined cbv_marks) work_arity = length work_demands :: Int @@ -1033,7 +1032,7 @@ splitThunk ww_opts is_rec x rhs = assert (not (isJoinId x)) $ do { let x' = localiseId x -- See comment above ; (useful,_args, wrap_fn, fn_arg) - <- mkWWstr_one ww_opts x' NotMarkedCbv + <- mkWWstr_one ww_opts x' NotMarkedStrict ; let res = [ (x, Let (NonRec x' rhs) (wrap_fn fn_arg)) ] ; if useful then assertPpr (isNonRec is_rec) (ppr x) -- The thunk must be non-recursive return res diff --git a/compiler/GHC/Core/Opt/WorkWrap/Utils.hs b/compiler/GHC/Core/Opt/WorkWrap/Utils.hs index 0351f53ccb..79074a3e05 100644 --- a/compiler/GHC/Core/Opt/WorkWrap/Utils.hs +++ b/compiler/GHC/Core/Opt/WorkWrap/Utils.hs @@ -152,7 +152,6 @@ data WwOpts type WwResult = ([Demand], -- Demands for worker (value) args - [CbvMark], -- Cbv semantics for worker (value) args JoinArity, -- Number of worker (type OR value) args Id -> CoreExpr, -- Wrapper body, lacking only the worker Id CoreExpr -> CoreExpr) -- Worker body, lacking the original function rhs @@ -222,7 +221,7 @@ mkWwBodies opts fun_id arg_vars res_ty demands res_cpr zapped_arg_vars = map zap_var arg_vars (subst, cloned_arg_vars) = cloneBndrs empty_subst uniq_supply zapped_arg_vars res_ty' = GHC.Core.Subst.substTy subst res_ty - init_cbv_marks = map (const NotMarkedCbv) cloned_arg_vars + init_cbv_marks = map (const NotMarkedStrict) cloned_arg_vars ; (useful1, work_args_cbv, wrap_fn_str, fn_args) <- mkWWstr opts cloned_arg_vars init_cbv_marks @@ -243,11 +242,13 @@ mkWwBodies opts fun_id arg_vars res_ty demands res_cpr call_rhs fn_rhs = mkAppsBeta fn_rhs fn_args -- See Note [Join points and beta-redexes] wrapper_body = mkLams cloned_arg_vars . wrap_fn_cpr . wrap_fn_str . call_work - worker_body = mkLams work_lam_args . work_fn_cpr . call_rhs - (worker_args_dmds, work_val_cbvs)= unzip [(idDemandInfo v,cbv) | (v,cbv) <- zipEqual "mkWwBodies" work_call_args work_call_cbv, isId v] + -- See Note [Call-by-value for worker args] + work_seq_str_flds = mkStrictFieldSeqs (zip work_lam_args work_call_cbv) + worker_body = mkLams work_lam_args . work_seq_str_flds . work_fn_cpr . call_rhs + worker_args_dmds= [(idDemandInfo v) | v <- work_call_args, isId v] ; if ((useful1 && not only_one_void_argument) || useful2) - then return (Just (worker_args_dmds, work_val_cbvs, length work_call_args, + then return (Just (worker_args_dmds, length work_call_args, wrapper_body, worker_body)) else return Nothing } @@ -390,12 +391,12 @@ needsVoidWorkerArg fn_id wrap_args work_args -- -- Why as the first argument? See Note [SpecConst needs to add void args first] -- in SpecConstr. -addVoidWorkerArg :: [Var] -> [CbvMark] +addVoidWorkerArg :: [Var] -> [StrictnessMark] -> ([Var], -- Lambda bound args [Var], -- Args at call site - [CbvMark]) -- cbv semantics for the worker args. + [StrictnessMark]) -- cbv semantics for the worker args. addVoidWorkerArg work_args cbv_marks - = (voidArgId : work_args, voidPrimId:work_args, NotMarkedCbv:cbv_marks) + = (voidArgId : work_args, voidPrimId:work_args, NotMarkedStrict:cbv_marks) {- Note [Protecting the last value argument] @@ -617,7 +618,7 @@ wantToUnboxArg do_unlifting fam_envs ty dmd@(n :* sd) -- That is done by 'finaliseArgBoxities'! = Unbox (DataConPatContext dc tc_args co) ds - -- See Note [Strict Worker Ids] + -- See Note [CBV Function Ids] | do_unlifting , isStrUsedDmd dmd , not (isFunTy ty) @@ -788,7 +789,7 @@ code which wasn't fruitful. See https://gitlab.haskell.org/ghc/ghc/-/merge_reque We could still try to do C) in the future by having PAP calls which will evaluate the required arguments before calling the partially applied function. But this would be neither a small nor simple change so we stick with A) and a flag for B) for now. -See also Note [Tag Inference] and Note [Strict Worker Ids] +See also Note [Tag Inference] and Note [CBV Function Ids] -} {- @@ -802,9 +803,9 @@ See also Note [Tag Inference] and Note [Strict Worker Ids] mkWWstr :: WwOpts -> [Var] -- Wrapper args; have their demand info on them -- *Includes type variables* - -> [CbvMark] -- cbv info for arguments + -> [StrictnessMark] -- cbv info for arguments -> UniqSM (Bool, -- Will this result in a useful worker - [(Var,CbvMark)], -- Worker args/their call-by-value semantics. + [(Var,StrictnessMark)], -- Worker args/their call-by-value semantics. CoreExpr -> CoreExpr, -- Wrapper body, lacking the worker call -- and without its lambdas -- This fn adds the unboxing @@ -835,24 +836,24 @@ mkWWstr opts args cbv_info -- See Note [Worker/wrapper for Strictness and Absence] mkWWstr_one :: WwOpts -> Var - -> CbvMark - -> UniqSM (Bool, [(Var,CbvMark)], CoreExpr -> CoreExpr, CoreExpr) -mkWWstr_one opts arg marked_cbv = + -> StrictnessMark + -> UniqSM (Bool, [(Var,StrictnessMark)], CoreExpr -> CoreExpr, CoreExpr) +mkWWstr_one opts arg banged = case wantToUnboxArg True fam_envs arg_ty arg_dmd of _ | isTyVar arg -> do_nothing DropAbsent - | Just absent_filler <- mkAbsentFiller opts arg + | Just absent_filler <- mkAbsentFiller opts arg banged -- Absent case. Dropt the argument from the worker. -- We can't always handle absence for arbitrary -- unlifted types, so we need to choose just the cases we can -- (that's what mkAbsentFiller does) -> return (goodWorker, [], nop_fn, absent_filler) - Unbox dcpc ds -> unbox_one_arg opts arg ds dcpc marked_cbv + Unbox dcpc ds -> unbox_one_arg opts arg ds dcpc banged Unlift -> return ( wwForUnlifting opts - , [(setIdUnfolding arg evaldUnfolding, MarkedCbv)] + , [(arg, MarkedStrict)] , nop_fn , varToCoreExpr arg) @@ -863,15 +864,16 @@ mkWWstr_one opts arg marked_cbv = arg_ty = idType arg arg_dmd = idDemandInfo arg -- Type args don't get cbv marks - arg_cbv = if isTyVar arg then NotMarkedCbv else marked_cbv + arg_cbv = if isTyVar arg then NotMarkedStrict else banged + do_nothing = return (badWorker, [(arg,arg_cbv)], nop_fn, varToCoreExpr arg) unbox_one_arg :: WwOpts -> Var -> [Demand] -> DataConPatContext - -> CbvMark - -> UniqSM (Bool, [(Var,CbvMark)], CoreExpr -> CoreExpr, CoreExpr) + -> StrictnessMark + -> UniqSM (Bool, [(Var,StrictnessMark)], CoreExpr -> CoreExpr, CoreExpr) unbox_one_arg opts arg_var ds DataConPatContext { dcpc_dc = dc, dcpc_tc_args = tc_args , dcpc_co = co } @@ -881,37 +883,32 @@ unbox_one_arg opts arg_var ds -- Create new arguments we get when unboxing dc (ex_tvs', arg_ids) = dataConRepFSInstPat (ex_name_fss ++ repeat ww_prefix) pat_bndrs_uniqs (idMult arg_var) dc tc_args - -- Apply str info to new args. - arg_ids' = zipWithEqual "unbox_one_arg" setIdDemandInfo arg_ids ds + con_str_marks = dataConRepStrictness dc + -- Apply str info to new args. Also remove OtherCon unfoldings so they don't end up in lambda binders + -- of the worker. See Note [Never put `OtherCon` unfoldings on lambda binders] + arg_ids' = map zapIdUnfolding $ zipWithEqual "unbox_one_arg" setIdDemandInfo arg_ids ds unbox_fn = mkUnpackCase (Var arg_var) co (idMult arg_var) dc (ex_tvs' ++ arg_ids') - -- Mark arguments coming out of strict fields as evaluated and give them cbv semantics. See Note [Strict Worker Ids] - cbv_arg_marks = zipWithEqual "unbox_one_arg" bangToMark (dataConRepStrictness dc) arg_ids' - unf_args = zipWith setEvald arg_ids' cbv_arg_marks - cbv_marks = (map (const NotMarkedCbv) ex_tvs') ++ cbv_arg_marks - ; (_sub_args_quality, worker_args, wrap_fn, wrap_args) <- mkWWstr opts (ex_tvs' ++ unf_args) cbv_marks + -- Mark arguments coming out of strict fields so we can make the worker strict on those + -- argumnets later. seq them later. See Note [Call-by-value for worker args] + strict_marks = (map (const NotMarkedStrict) ex_tvs') ++ con_str_marks + ; (_sub_args_quality, worker_args, wrap_fn, wrap_args) <- mkWWstr opts (ex_tvs' ++ arg_ids') strict_marks ; let wrap_arg = mkConApp dc (map Type tc_args ++ wrap_args) `mkCast` mkSymCo co ; return (goodWorker, worker_args, unbox_fn . wrap_fn, wrap_arg) } -- Don't pass the arg, rebox instead - where bangToMark :: StrictnessMark -> Id -> CbvMark - bangToMark NotMarkedStrict _ = NotMarkedCbv - bangToMark MarkedStrict v - | isUnliftedType (idType v) = NotMarkedCbv - | otherwise = MarkedCbv - setEvald var NotMarkedCbv = var - setEvald var MarkedCbv = setIdUnfolding var evaldUnfolding -- | Tries to find a suitable absent filler to bind the given absent identifier -- to. See Note [Absent fillers]. -- -- If @mkAbsentFiller _ id == Just e@, then @e@ is an absent filler with the -- same type as @id@. Otherwise, no suitable filler could be found. -mkAbsentFiller :: WwOpts -> Id -> Maybe CoreExpr -mkAbsentFiller opts arg +mkAbsentFiller :: WwOpts -> Id -> StrictnessMark -> Maybe CoreExpr +mkAbsentFiller opts arg str -- The lifted case: Bind 'absentError' for a nice panic message if we are -- wrong (like we were in #11126). See (1) in Note [Absent fillers] | mightBeLiftedType arg_ty - , not is_strict, not is_evald -- See (2) in Note [Absent fillers] + , not is_strict + , not (isMarkedStrict str) -- See (2) in Note [Absent fillers] = Just (mkAbsentErrorApp arg_ty msg) -- The default case for mono rep: Bind `RUBBISH[rr] arg_ty` @@ -922,7 +919,6 @@ mkAbsentFiller opts arg where arg_ty = idType arg is_strict = isStrictDmd (idDemandInfo arg) - is_evald = isEvaldUnfolding $ idUnfolding arg msg = renderWithContext (defaultSDocContext { sdocSuppressUniques = True }) @@ -1095,29 +1091,28 @@ Needless to say, there are some wrinkles: 2. We also mustn't put an error-thunk (that fills in for an absent value of lifted rep) in a strict field, because #16970 establishes the invariant - that strict fields are always evaluated, by (re-)evaluating what is put in + that strict fields are always evaluated, by possibly (re-)evaluating what is put in a strict field. That's the reason why 'zs' binds a rubbish literal instead of an error-thunk, see #19133. How do we detect when we are about to put an error-thunk in a strict field? - Ideally, we'd just look at the 'StrictnessMark' of the DataCon's field, but - it's quite nasty to thread the marks though 'mkWWstr' and 'mkWWstr_one'. - So we rather look out for a necessary condition for strict fields: + Ideally, we'd just look at the 'StrictnessMark' of the DataCon's field. So that's + what we do! + + There are other necessary conditions for strict fields: Note [Unboxing evaluated arguments] in DmdAnal makes it so that the demand on 'zs' is absent and /strict/: It will get cardinality 'C_10', the empty - interval, rather than 'C_00'. Hence the 'isStrictDmd' check: It guarantees - we never fill in an error-thunk for an absent strict field. + interval, rather than 'C_00'. Hence the 'isStrictDmd' check: It further + guarantees e never fill in an error-thunk for an absent strict field. But that also means we emit a rubbish lit for other args that have cardinality 'C_10' (say, the arg to a bottoming function) where we could've - used an error-thunk, but that's a small price to pay for simplicity. - - In #19766, we discovered that even if the binder has eval cardinality - 'C_00', it may end up in a strict field, with no surrounding seq - whatsoever! That happens if the calling code has already evaluated - said lambda binder, which will then have an evaluated unfolding - ('isEvaldUnfolding'). That in turn tells the Simplifier it is free to drop - the seq. So we better don't fill in an error-thunk for eval'd arguments - either, just in case it ends up in a strict field! + used an error-thunk. + NB from Andreas: But I think using an error thunk there would be dodgy no matter what + for example if we decide to pass the argument to the bottoming function cbv. + As we might do if the function in question is a worker. + See Note [CBV Function Ids] in GHC.CoreToStg.Prep. So I just left the strictness check + in place on top of threading through the marks from the constructor. It's a *really* cheap + and easy check to make anyway. 3. We can only emit a LitRubbish if the arg's type @arg_ty@ is mono-rep, e.g. of the form @TYPE rep@ where @rep@ is not (and doesn't contain) a variable. |