Merge 'upstream/main' into experimental/support-pyproject

14 files changed, 155 insertions, 157 deletions

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 5be824c1..c680fb36 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -24,7 +24,7 @@ jobs:
         platform:
         - ubuntu-latest
         - macos-latest
-        - windows-2019
+        - windows-latest
         include:
         - platform: ubuntu-latest
           python: "3.10"
diff --git a/CHANGES.rst b/CHANGES.rst
index 3c724e47..68b68fbd 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -15,7 +15,7 @@ Documentation changes
   and more prominent mentions to using a revision control system plugin as an
   alternative.
 * #3148: Removed mention to ``pkg_resources`` as the recommended way of accessing data
-  files, in favour of :doc:`importlib.resources`.
+  files, in favour of importlib.resources.
   Additionally more emphasis was put on the fact that *package data files* reside
   **inside** the *package directory* (and therefore should be *read-only*).
 
@@ -207,7 +207,7 @@ v60.4.0
 
 Changes
 ^^^^^^^
-* #2839: Removed `requires` sorting when installing wheels as an egg dir.
+* #2839: Removed ``requires`` sorting when installing wheels as an egg dir.
 * #2953: Fixed a bug that easy install incorrectly parsed Python 3.10 version string.
 * #3006: Fixed startup performance issue of Python interpreter due to imports of
   costly modules in ``_distutils_hack`` -- by :user:`tiran`
@@ -646,7 +646,7 @@ v57.5.0
 
 Changes
 ^^^^^^^
-* #2712: Added implicit globbing support for `[options.data_files]` values.
+* #2712: Added implicit globbing support for ``[options.data_files]`` values.
 
 Documentation changes
 ^^^^^^^^^^^^^^^^^^^^^
@@ -718,7 +718,7 @@ Changes
   ``license_file`` (deprecated) and ``license_files`` options,
   relative to ``.dist-info``. - by :user:`cdce8p`
 * #2678: Moved Setuptools' own entry points into declarative config.
-* #2680: Vendored `more_itertools <https://pypi.org/project/more-itertools>`_ for Setuptools.
+* #2680: Vendored :pypi:`more_itertools` for Setuptools.
 * #2681: Setuptools own setup.py no longer declares setup_requires, but instead expects wheel to be installed as declared by pyproject.toml.
 
 Misc
@@ -1646,7 +1646,7 @@ Breaking Changes
    * eggs are not supported
    * no support for the ``allow_hosts`` easy_install option (``index_url``/``find_links`` are still honored)
    * pip environment variables are honored (and take precedence over easy_install options)
-* #1898: Removed the "upload" and "register" commands in favor of `twine <https://pypi.org/p/twine>`_.
+* #1898: Removed the "upload" and "register" commands in favor of :pypi:`twine`.
 
 Changes
 ^^^^^^^
@@ -1656,7 +1656,7 @@ Changes
   * add support for manylinux2010
   * fix use of removed 'm' ABI flag in Python 3.8 on Windows
 * #1861: Fix empty namespace package installation from wheel.
-* #1877: Setuptools now exposes a new entry point hook "setuptools.finalize_distribution_options", enabling plugins like `setuptools_scm <https://pypi.org/project/setuptools_scm>`_ to configure options on the distribution at finalization time.
+* #1877: Setuptools now exposes a new entry point hook "setuptools.finalize_distribution_options", enabling plugins like :pypi:`setuptools_scm` to configure options on the distribution at finalization time.
 
 
 v41.6.0
@@ -2923,7 +2923,7 @@ v26.1.0
 -------
 
 * #763: ``pkg_resources.get_default_cache`` now defers to the
-  `appdirs project <https://pypi.org/project/appdirs>`_ to
+  :pypi:`appdirs` project to
   resolve the cache directory. Adds a vendored dependency on
   appdirs to pkg_resources.
 
@@ -3915,8 +3915,7 @@ process to fail and PyPI uploads no longer accept files for 13.0.
 
 * Issue #313: Removed built-in support for subversion. Projects wishing to
   retain support for subversion will need to use a third party library. The
-  extant implementation is being ported to `setuptools_svn
-  <https://pypi.org/project/setuptools_svn/>`_.
+  extant implementation is being ported to :pypi:`setuptools_svn`.
 * Issue #315: Updated setuptools to hide its own loaded modules during
   installation of another package. This change will enable setuptools to
   upgrade (or downgrade) itself even when its own metadata and implementation
@@ -4420,8 +4419,7 @@ process to fail and PyPI uploads no longer accept files for 13.0.
 
 * Address security vulnerability in SSL match_hostname check as reported in
   Python #17997.
-* Prefer `backports.ssl_match_hostname
-  <https://pypi.org/project/backports.ssl_match_hostname/>`_ for backport
+* Prefer :pypi:`backports.ssl_match_hostname` for backport
   implementation if present.
 * Correct NameError in ``ssl_support`` module (``socket.error``).
 
diff --git a/changelog.d/3179.change.rst b/changelog.d/3179.change.rst
new file mode 100644
index 00000000..791a327b
--- /dev/null
+++ b/changelog.d/3179.change.rst
@@ -0,0 +1 @@
+Merge with pypa/distutils@267dbd25ac
diff --git a/changelog.d/README.rst b/changelog.d/README.rst
index 49b4d563..6def76b5 100644
--- a/changelog.d/README.rst
+++ b/changelog.d/README.rst
@@ -21,8 +21,7 @@ recorded in the Git history rather than a changelog.
 Alright! So how to add a news fragment?
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-``setuptools`` uses `towncrier <https://pypi.org/project/towncrier/>`_
-for changelog management.
+``setuptools`` uses :pypi:`towncrier` for changelog management.
 To submit a change note about your PR, add a text file into the
 ``changelog.d/`` folder. It should contain an
 explanation of what applying this PR will change in the way
diff --git a/docs/deprecated/easy_install.rst b/docs/deprecated/easy_install.rst
index 76c3f608..3cf3bea9 100644
--- a/docs/deprecated/easy_install.rst
+++ b/docs/deprecated/easy_install.rst
@@ -34,7 +34,7 @@ Using "Easy Install"
 Installing "Easy Install"
 -------------------------
 
-Please see the `setuptools PyPI page <https://pypi.org/project/setuptools/>`_
+Please see the :pypi:`setuptools` on the package index
 for download links and basic installation instructions for each of the
 supported platforms.
 
@@ -1020,10 +1020,7 @@ of the User installation scheme.  "virtualenv" provides a version of ``easy_inst
 scoped to the cloned python install and is used in the normal way. "virtualenv" does offer various features
 that the User installation scheme alone does not provide, e.g. the ability to hide the main python site-packages.
 
-Please refer to the `virtualenv`_ documentation for more details.
-
-.. _virtualenv: https://pypi.org/project/virtualenv/
-
+Please refer to the :pypi:`virtualenv` documentation for more details.
 
 
 Package Index "API"
diff --git a/docs/deprecated/index.rst b/docs/deprecated/index.rst
index ce2ac006..59fc7bef 100644
--- a/docs/deprecated/index.rst
+++ b/docs/deprecated/index.rst
@@ -13,7 +13,6 @@ objectives.
 .. toctree::
     :maxdepth: 1
 
-    python3
     python_eggs
     easy_install
     distutils/index
diff --git a/docs/pkg_resources.rst b/docs/pkg_resources.rst
index c1158189..21ff6dc1 100644
--- a/docs/pkg_resources.rst
+++ b/docs/pkg_resources.rst
@@ -13,8 +13,8 @@ packages.
 Use of ``pkg_resources`` is discouraged in favor of
 `importlib.resources <https://docs.python.org/3/library/importlib.html#module-importlib.resources>`_,
 `importlib.metadata <https://docs.python.org/3/library/importlib.metadata.html>`_,
-and their backports (`resources <https://pypi.org/project/importlib_resources>`_,
-`metadata <https://pypi.org/project/importlib_metadata>`_).
+and their backports (:pypi:`importlib_resources`,
+:pypi:`importlib_metadata`).
 Please consider using those libraries instead of pkg_resources.
 
 
diff --git a/docs/userguide/entry_point.rst b/docs/userguide/entry_point.rst
index ea73bb5e..b97419c4 100644
--- a/docs/userguide/entry_point.rst
+++ b/docs/userguide/entry_point.rst
@@ -120,7 +120,7 @@ and tools like ``pip`` create wrapper scripts that invoke those commands.
 For a project wishing to solicit entry points, Setuptools recommends the
 `importlib.metadata <https://docs.python.org/3/library/importlib.metadata.html>`_
 module (part of stdlib since Python 3.8) or its backport,
-`importlib_metadata <https://pypi.org/project/importlib_metadata>`_.
+:pypi:`importlib_metadata`.
 
 For example, to find the console script entry points from the example above:
 
diff --git a/docs/userguide/extension.rst b/docs/userguide/extension.rst
index d74ca3fe..21fb05b6 100644
--- a/docs/userguide/extension.rst
+++ b/docs/userguide/extension.rst
@@ -194,8 +194,8 @@ Adding Support for Revision Control Systems
 If the files you want to include in the source distribution are tracked using
 Git, Mercurial or SVN, you can use the following packages to achieve that:
 
-- Git and Mercurial: `setuptools_scm <https://pypi.org/project/setuptools_scm/>`_
-- SVN: `setuptools_svn <https://pypi.org/project/setuptools_svn/>`_
+- Git and Mercurial: :pypi:`setuptools_scm`
+- SVN: :pypi:`setuptools_svn`
 
 If you would like to create a plugin for ``setuptools`` to find files tracked
 by another revision control system, you can do so by adding an entry point to
diff --git a/setuptools/_distutils/_macos_compat.py b/setuptools/_distutils/_macos_compat.py
new file mode 100644
index 00000000..17769e91
--- /dev/null
+++ b/setuptools/_distutils/_macos_compat.py
@@ -0,0 +1,12 @@
+import sys
+import importlib
+
+
+def bypass_compiler_fixup(cmd, args):
+    return cmd
+
+
+if sys.platform == 'darwin':
+    compiler_fixup = importlib.import_module('_osx_support').compiler_fixup
+else:
+    compiler_fixup = bypass_compiler_fixup
diff --git a/setuptools/_distutils/sysconfig.py b/setuptools/_distutils/sysconfig.py
index 4a77a431..9fad3835 100644
--- a/setuptools/_distutils/sysconfig.py
+++ b/setuptools/_distutils/sysconfig.py
@@ -436,51 +436,6 @@ def expand_makefile_vars(s, vars):
 _config_vars = None
 
 
-_sysconfig_name_tmpl = '_sysconfigdata_{abi}_{platform}_{multiarch}'
-
-
-def _init_posix():
-    """Initialize the module as appropriate for POSIX systems."""
-    # _sysconfigdata is generated at build time, see the sysconfig module
-    name = os.environ.get(
-        '_PYTHON_SYSCONFIGDATA_NAME',
-        _sysconfig_name_tmpl.format(
-            abi=sys.abiflags,
-            platform=sys.platform,
-            multiarch=getattr(sys.implementation, '_multiarch', ''),
-        ),
-    )
-    try:
-        _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
-    except ImportError:
-        # Python 3.5 and pypy 7.3.1
-        _temp = __import__(
-            '_sysconfigdata', globals(), locals(), ['build_time_vars'], 0)
-    build_time_vars = _temp.build_time_vars
-    global _config_vars
-    _config_vars = {}
-    _config_vars.update(build_time_vars)
-
-
-def _init_nt():
-    """Initialize the module as appropriate for NT"""
-    g = {}
-    # set basic install directories
-    g['LIBDEST'] = get_python_lib(plat_specific=0, standard_lib=1)
-    g['BINLIBDEST'] = get_python_lib(plat_specific=1, standard_lib=1)
-
-    # XXX hmmm.. a normal install puts include files here
-    g['INCLUDEPY'] = get_python_inc(plat_specific=0)
-
-    g['EXT_SUFFIX'] = _imp.extension_suffixes()[0]
-    g['EXE'] = ".exe"
-    g['VERSION'] = get_python_version().replace(".", "")
-    g['BINDIR'] = os.path.dirname(os.path.abspath(sys.executable))
-
-    global _config_vars
-    _config_vars = g
-
-
 def get_config_vars(*args):
     """With no arguments, return a dictionary of all configuration
     variables relevant for the current platform.  Generally this includes
@@ -493,60 +448,7 @@ def get_config_vars(*args):
     """
     global _config_vars
     if _config_vars is None:
-        func = globals().get("_init_" + os.name)
-        if func:
-            func()
-        else:
-            _config_vars = {}
-
-        # Normalized versions of prefix and exec_prefix are handy to have;
-        # in fact, these are the standard versions used most places in the
-        # Distutils.
-        _config_vars['prefix'] = PREFIX
-        _config_vars['exec_prefix'] = EXEC_PREFIX
-
-        if not IS_PYPY:
-            # For backward compatibility, see issue19555
-            SO = _config_vars.get('EXT_SUFFIX')
-            if SO is not None:
-                _config_vars['SO'] = SO
-
-            # Always convert srcdir to an absolute path
-            srcdir = _config_vars.get('srcdir', project_base)
-            if os.name == 'posix':
-                if python_build:
-                    # If srcdir is a relative path (typically '.' or '..')
-                    # then it should be interpreted relative to the directory
-                    # containing Makefile.
-                    base = os.path.dirname(get_makefile_filename())
-                    srcdir = os.path.join(base, srcdir)
-                else:
-                    # srcdir is not meaningful since the installation is
-                    # spread about the filesystem.  We choose the
-                    # directory containing the Makefile since we know it
-                    # exists.
-                    srcdir = os.path.dirname(get_makefile_filename())
-            _config_vars['srcdir'] = os.path.abspath(os.path.normpath(srcdir))
-
-            # Convert srcdir into an absolute path if it appears necessary.
-            # Normally it is relative to the build directory.  However, during
-            # testing, for example, we might be running a non-installed python
-            # from a different directory.
-            if python_build and os.name == "posix":
-                base = project_base
-                if (not os.path.isabs(_config_vars['srcdir']) and
-                    base != os.getcwd()):
-                    # srcdir is relative and we are not in the same directory
-                    # as the executable. Assume executable is in the build
-                    # directory and make srcdir absolute.
-                    srcdir = os.path.join(base, _config_vars['srcdir'])
-                    _config_vars['srcdir'] = os.path.normpath(srcdir)
-
-        # OS X platforms require special customization to handle
-        # multi-architecture, multi-os-version installers
-        if sys.platform == 'darwin':
-            import _osx_support
-            _osx_support.customize_config_vars(_config_vars)
+        _config_vars = sysconfig.get_config_vars().copy()
 
     if args:
         vals = []
diff --git a/setuptools/_distutils/tests/test_unixccompiler.py b/setuptools/_distutils/tests/test_unixccompiler.py
index 4574f77f..c8b4c149 100644
--- a/setuptools/_distutils/tests/test_unixccompiler.py
+++ b/setuptools/_distutils/tests/test_unixccompiler.py
@@ -3,6 +3,7 @@ import os
 import sys
 import unittest
 from test.support import run_unittest
+from unittest.mock import patch
 
 from .py38compat import EnvironmentVarGuard
 
@@ -215,6 +216,42 @@ class UnixCCompilerTestCase(support.TempdirManager, unittest.TestCase):
         self.assertEqual(self.cc.linker_so[0], 'my_cc')
 
     @unittest.skipIf(sys.platform == 'win32', "can't test on Windows")
+    def test_cc_overrides_ldshared_for_cxx_correctly(self):
+        """
+        Ensure that setting CC env variable also changes default linker
+        correctly when building C++ extensions.
+
+        pypa/distutils#126
+        """
+        def gcv(v):
+            if v == 'LDSHARED':
+                return 'gcc-4.2 -bundle -undefined dynamic_lookup '
+            elif v == 'CXX':
+                return 'g++-4.2'
+            return 'gcc-4.2'
+
+        def gcvs(*args, _orig=sysconfig.get_config_vars):
+            if args:
+                return list(map(sysconfig.get_config_var, args))
+            return _orig()
+
+        sysconfig.get_config_var = gcv
+        sysconfig.get_config_vars = gcvs
+        with patch.object(self.cc, 'spawn', return_value=None) as mock_spawn, \
+                patch.object(self.cc, '_need_link', return_value=True), \
+                patch.object(self.cc, 'mkpath', return_value=None), \
+                EnvironmentVarGuard() as env:
+            env['CC'] = 'ccache my_cc'
+            env['CXX'] = 'my_cxx'
+            del env['LDSHARED']
+            sysconfig.customize_compiler(self.cc)
+            self.assertEqual(self.cc.linker_so[0:2], ['ccache', 'my_cc'])
+            self.cc.link(None, [], 'a.out', target_lang='c++')
+            call_args = mock_spawn.call_args[0][0]
+            expected = ['my_cxx', '-bundle', '-undefined', 'dynamic_lookup']
+            assert call_args[:4] == expected
+
+    @unittest.skipIf(sys.platform == 'win32', "can't test on Windows")
     def test_explicit_ldshared(self):
         # Issue #18080:
         # ensure that setting CC env variable does not change
diff --git a/setuptools/_distutils/unixccompiler.py b/setuptools/_distutils/unixccompiler.py
index a07e5988..715408f5 100644
--- a/setuptools/_distutils/unixccompiler.py
+++ b/setuptools/_distutils/unixccompiler.py
@@ -22,9 +22,7 @@ from distutils.ccompiler import \
 from distutils.errors import \
      DistutilsExecError, CompileError, LibError, LinkError
 from distutils import log
-
-if sys.platform == 'darwin':
-    import _osx_support
+from ._macos_compat import compiler_fixup
 
 # XXX Things not currently handled:
 #   * optimization/debug/warning flags; we just use whatever's in Python's
@@ -42,6 +40,66 @@ if sys.platform == 'darwin':
 #     options and carry on.
 
 
+def _split_env(cmd):
+    """
+    For macOS, split command into 'env' portion (if any)
+    and the rest of the linker command.
+
+    >>> _split_env(['a', 'b', 'c'])
+    ([], ['a', 'b', 'c'])
+    >>> _split_env(['/usr/bin/env', 'A=3', 'gcc'])
+    (['/usr/bin/env', 'A=3'], ['gcc'])
+    """
+    pivot = 0
+    if os.path.basename(cmd[0]) == "env":
+        pivot = 1
+        while '=' in cmd[pivot]:
+            pivot += 1
+    return cmd[:pivot], cmd[pivot:]
+
+
+def _split_aix(cmd):
+    """
+    AIX platforms prefix the compiler with the ld_so_aix
+    script, so split that from the linker command.
+
+    >>> _split_aix(['a', 'b', 'c'])
+    ([], ['a', 'b', 'c'])
+    >>> _split_aix(['/bin/foo/ld_so_aix', 'gcc'])
+    (['/bin/foo/ld_so_aix'], ['gcc'])
+    """
+    pivot = os.path.basename(cmd[0]) == 'ld_so_aix'
+    return cmd[:pivot], cmd[pivot:]
+
+
+def _linker_params(linker_cmd, compiler_cmd):
+    """
+    The linker command usually begins with the compiler
+    command (possibly multiple elements), followed by zero or more
+    params for shared library building.
+
+    If the LDSHARED env variable overrides the linker command,
+    however, the commands may not match.
+
+    Return the best guess of the linker parameters by stripping
+    the linker command. If the compiler command does not
+    match the linker command, assume the linker command is
+    just the first element.
+
+    >>> _linker_params('gcc foo bar'.split(), ['gcc'])
+    ['foo', 'bar']
+    >>> _linker_params('gcc foo bar'.split(), ['other'])
+    ['foo', 'bar']
+    >>> _linker_params('ccache gcc foo bar'.split(), 'ccache gcc'.split())
+    ['foo', 'bar']
+    >>> _linker_params(['gcc'], ['gcc'])
+    []
+    """
+    c_len = len(compiler_cmd)
+    pivot = c_len if linker_cmd[:c_len] == compiler_cmd else 1
+    return linker_cmd[pivot:]
+
+
 class UnixCCompiler(CCompiler):
 
     compiler_type = 'unix'
@@ -109,10 +167,8 @@ class UnixCCompiler(CCompiler):
                 raise CompileError(msg)
 
     def _compile(self, obj, src, ext, cc_args, extra_postargs, pp_opts):
-        compiler_so = self.compiler_so
-        if sys.platform == 'darwin':
-            compiler_so = _osx_support.compiler_fixup(compiler_so,
-                                                    cc_args + extra_postargs)
+        compiler_so = compiler_fixup(
+            self.compiler_so, cc_args + extra_postargs)
         try:
             self.spawn(compiler_so + cc_args + [src, '-o', obj] +
                        extra_postargs)
@@ -173,33 +229,22 @@ class UnixCCompiler(CCompiler):
                 ld_args.extend(extra_postargs)
             self.mkpath(os.path.dirname(output_filename))
             try:
-                if target_desc == CCompiler.EXECUTABLE:
-                    linker = self.linker_exe[:]
-                else:
-                    linker = self.linker_so[:]
+                # Select a linker based on context: linker_exe when
+                # building an executable or linker_so (with shared options)
+                # when building a shared library.
+                building_exe = target_desc == CCompiler.EXECUTABLE
+                linker = (self.linker_exe if building_exe else self.linker_so)[:]
+
                 if target_lang == "c++" and self.compiler_cxx:
-                    # skip over environment variable settings if /usr/bin/env
-                    # is used to set up the linker's environment.
-                    # This is needed on OSX. Note: this assumes that the
-                    # normal and C++ compiler have the same environment
-                    # settings.
-                    i = 0
-                    if os.path.basename(linker[0]) == "env":
-                        i = 1
-                        while '=' in linker[i]:
-                            i += 1
-
-                    if os.path.basename(linker[i]) == 'ld_so_aix':
-                        # AIX platforms prefix the compiler with the ld_so_aix
-                        # script, so we need to adjust our linker index
-                        offset = 1
-                    else:
-                        offset = 0
-
-                    linker[i+offset] = self.compiler_cxx[i]
-
-                if sys.platform == 'darwin':
-                    linker = _osx_support.compiler_fixup(linker, ld_args)
+                    env, linker_ne = _split_env(linker)
+                    aix, linker_na = _split_aix(linker_ne)
+                    _, compiler_cxx_ne = _split_env(self.compiler_cxx)
+                    _, linker_exe_ne = _split_env(self.linker_exe)
+
+                    params = _linker_params(linker_na, linker_exe_ne)
+                    linker = env + aix + compiler_cxx_ne + params
+
+                linker = compiler_fixup(linker, ld_args)
 
                 self.spawn(linker + ld_args)
             except DistutilsExecError as msg:
diff --git a/tox.ini b/tox.ini
index a56ea24b..22c796ff 100644
--- a/tox.ini
+++ b/tox.ini
@@ -20,6 +20,10 @@ passenv =
 	windir  # required for test_pkg_resources
 	# honor git config in pytest-perf
 	HOME
+	# workaround for tox-dev/tox#2382
+	PROGRAMDATA
+	PROGRAMFILES
+	PROGRAMFILES(x86)
 
 [testenv:integration]
 deps = {[testenv]deps}
@@ -27,6 +31,10 @@ extras = testing-integration
 passenv =
 	{[testenv]passenv}
 	DOWNLOAD_PATH
+	# workaround for tox-dev/tox#2382
+	PROGRAMDATA
+	PROGRAMFILES
+	PROGRAMFILES(x86)
 setenv =
     PROJECT_ROOT = {toxinidir}
 commands =