Add and use a new dynamic-library-dirs field in the ghc-pkg info

Summary:
Build systems / package managers want to be able to control the file
layout of installed libraries. In general they may want/need to be able
to put the static libraries and dynamic libraries in different places.
The ghc-pkg library regisrtation needs to be able to handle this.

This is already possible in principle by listing both a static lib dir
and a dynamic lib dir in the library-dirs field (indeed some previous
versions of Cabal did this for shared libs on ELF platforms).

The downside of listing both dirs is twofold. There is a lack of
precision, if we're not careful with naming then we could end up
picking up the wrong library. The more immediate problem however is
that if we list both directories then both directories get included
into the ELF and Mach-O shared object runtime search paths. On ELF this
merely slows down loading of shared libs (affecting prog startup time).
On the latest OSX versions this provokes a much more serious problem:
that there is a rather low limit on the total size of the section
containing the runtime search path (and lib names and related) and thus
listing any unnecessary directories wastes the limited space.

So the solution in this patch is fairly straightforward: split the
static and dynamic library search paths in the ghc-pkg db and its use
within ghc. This is a traditional solution: pkg-config has the same
static / dynamic split (though it describes in in terms of private and
public, but it translates into different behaviour for static and
dynamic linking).

Indeed it would make perfect sense to also have a static/dynamic split
for the list of the libraries to use i.e. to have dynamic variants of
the hs-libraries and extra-libraries fields. These are not immediately
required so this patch does not add it, but it is a reasonable
direction to follow.

To handle compatibility, if the new dynamic-library-dirs field is not
specified then its value is taken from the library-dirs field.

Contains Cabal submodule update.

Test Plan:
Run ./validate

Get christiaanb and carter to test it on OSX Sierra, in combination
with Cabal/cabal-install changes to the default file layout for
libraries.

Reviewers: carter, austin, hvr, christiaanb, bgamari

Reviewed By: christiaanb, bgamari

Subscribers: ezyang, Phyx, thomie

Differential Revision: https://phabricator.haskell.org/D2611

GHC Trac Issues: #12479

1 files changed, 29 insertions, 4 deletions

diff --git a/compiler/main/Packages.hs b/compiler/main/Packages.hs
index e0563da10c..86a37352b0 100644
--- a/compiler/main/Packages.hs
+++ b/compiler/main/Packages.hs
@@ -541,7 +541,8 @@ readPackageConfig dflags conf_file = do
   let
       top_dir = topDir dflags
       pkgroot = takeDirectory conf_file
-      pkg_configs1 = map (mungePackagePaths top_dir pkgroot) proto_pkg_configs
+      pkg_configs1 = map (mungePackageConfig top_dir pkgroot)
+                         proto_pkg_configs
       pkg_configs2 = setBatchPackageFlags dflags pkg_configs1
   --
   return (conf_file, pkg_configs2)
@@ -580,6 +581,22 @@ setBatchPackageFlags dflags pkgs = maybeDistrustAll pkgs
 
     distrust pkg = pkg{ trusted = False }
 
+mungePackageConfig :: FilePath -> FilePath
+                   -> PackageConfig -> PackageConfig
+mungePackageConfig top_dir pkgroot =
+    mungeDynLibFields
+  . mungePackagePaths top_dir pkgroot
+
+mungeDynLibFields :: PackageConfig -> PackageConfig
+mungeDynLibFields pkg =
+    pkg {
+      libraryDynDirs     = libraryDynDirs pkg
+                `orIfNull` libraryDirs pkg
+    }
+  where
+    orIfNull [] flags = flags
+    orIfNull flags _  = flags
+
 -- TODO: This code is duplicated in utils/ghc-pkg/Main.hs
 mungePackagePaths :: FilePath -> FilePath -> PackageConfig -> PackageConfig
 -- Perform path/URL variable substitution as per the Cabal ${pkgroot} spec
@@ -595,6 +612,7 @@ mungePackagePaths top_dir pkgroot pkg =
       importDirs  = munge_paths (importDirs pkg),
       includeDirs = munge_paths (includeDirs pkg),
       libraryDirs = munge_paths (libraryDirs pkg),
+      libraryDynDirs = munge_paths (libraryDynDirs pkg),
       frameworkDirs = munge_paths (frameworkDirs pkg),
       haddockInterfaces = munge_paths (haddockInterfaces pkg),
       haddockHTMLs = munge_urls (haddockHTMLs pkg)
@@ -1459,10 +1477,11 @@ collectIncludeDirs ps = nub (filter notNull (concatMap includeDirs ps))
 -- | Find all the library paths in these and the preload packages
 getPackageLibraryPath :: DynFlags -> [PreloadUnitId] -> IO [String]
 getPackageLibraryPath dflags pkgs =
-  collectLibraryPaths `fmap` getPreloadPackagesAnd dflags pkgs
+  collectLibraryPaths dflags `fmap` getPreloadPackagesAnd dflags pkgs
 
-collectLibraryPaths :: [PackageConfig] -> [FilePath]
-collectLibraryPaths ps = nub (filter notNull (concatMap libraryDirs ps))
+collectLibraryPaths :: DynFlags -> [PackageConfig] -> [FilePath]
+collectLibraryPaths dflags = nub . filter notNull
+                           . concatMap (libraryDirsForWay dflags)
 
 -- | Find all the link options in these and the preload packages,
 -- returning (package hs lib options, extra library options, other flags)
@@ -1514,6 +1533,12 @@ packageHsLibs dflags p = map (mkDynName . addSuffix) (hsLibraries p)
         expandTag t | null t = ""
                     | otherwise = '_':t
 
+-- | Either the 'libraryDirs' or 'libraryDynDirs' as appropriate for the way.
+libraryDirsForWay :: DynFlags -> PackageConfig -> [String]
+libraryDirsForWay dflags
+  | WayDyn `elem` ways dflags = libraryDynDirs
+  | otherwise                 = libraryDirs
+
 -- | Find all the C-compiler options in these and the preload packages
 getPackageExtraCcOpts :: DynFlags -> [PreloadUnitId] -> IO [String]
 getPackageExtraCcOpts dflags pkgs = do