Issue #26186: Remove the restriction that built-in and extension

modules  can't be lazily loaded.

Thanks to Python 3.6 allowing for types.ModuleType to have its
__class__ mutated, the restriction can be lifted by calling
create_module() on the wrapped loader.

2 files changed, 37 insertions, 14 deletions

diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst
index 656d51ac0e..c1c3b124d8 100644
--- a/Doc/library/importlib.rst
+++ b/Doc/library/importlib.rst
@@ -379,10 +379,14 @@ ABC hierarchy::
 
        An abstract method that executes the module in its own namespace
        when a module is imported or reloaded.  The module should already
-       be initialized when exec_module() is called.
+       be initialized when ``exec_module()`` is called. When this method exists,
+       :meth:`~importlib.abc.Loader.create_module` must be defined.
 
        .. versionadded:: 3.4
 
+       .. versionchanged:: 3.6
+          :meth:`~importlib.abc.Loader.create_module` must also be defined.
+
     .. method:: load_module(fullname)
 
         A legacy method for loading a module. If the module cannot be
@@ -1200,12 +1204,13 @@ an :term:`importer`.
 
 .. function:: module_from_spec(spec)
 
-   Create a new module based on **spec** and ``spec.loader.create_module()``.
+   Create a new module based on **spec** and
+   :meth:`spec.loader.create_module <importlib.abc.Loader.create_module>`.
 
-   If ``spec.loader.create_module()`` does not return ``None``, then any
-   pre-existing attributes will not be reset. Also, no :exc:`AttributeError`
-   will be raised if triggered while accessing **spec** or setting an attribute
-   on the module.
+   If :meth:`spec.loader.create_module <importlib.abc.Loader.create_module>`
+   does not return ``None``, then any pre-existing attributes will not be reset.
+   Also, no :exc:`AttributeError` will be raised if triggered while accessing
+   **spec** or setting an attribute on the module.
 
    This function is preferred over using :class:`types.ModuleType` to create a
    new module as **spec** is used to set as many import-controlled attributes on
@@ -1267,7 +1272,8 @@ an :term:`importer`.
 
 .. decorator:: set_package
 
-   A :term:`decorator` for :meth:`importlib.abc.Loader.load_module` to set the :attr:`__package__` attribute on the returned module. If :attr:`__package__`
+   A :term:`decorator` for :meth:`importlib.abc.Loader.load_module` to set the
+   :attr:`__package__` attribute on the returned module. If :attr:`__package__`
    is set and has a value other than ``None`` it will not be changed.
 
    .. deprecated:: 3.4
@@ -1300,13 +1306,12 @@ an :term:`importer`.
    This class **only** works with loaders that define
    :meth:`~importlib.abc.Loader.exec_module` as control over what module type
    is used for the module is required. For those same reasons, the loader's
-   :meth:`~importlib.abc.Loader.create_module` method will be ignored (i.e., the
-   loader's method should only return ``None``; this excludes
-   :class:`BuiltinImporter` and :class:`ExtensionFileLoader`). Finally,
-   modules which substitute the object placed into :attr:`sys.modules` will
-   not work as there is no way to properly replace the module references
-   throughout the interpreter safely; :exc:`ValueError` is raised if such a
-   substitution is detected.
+   :meth:`~importlib.abc.Loader.create_module` method must return ``None`` or a
+   type for which its ``__class__`` attribute can be mutated along with not
+   using :term:`slots <__slots__>`. Finally, modules which substitute the object
+   placed into :attr:`sys.modules` will not work as there is no way to properly
+   replace the module references throughout the interpreter safely;
+   :exc:`ValueError` is raised if such a substitution is detected.
 
    .. note::
       For projects where startup time is critical, this class allows for
@@ -1317,6 +1322,11 @@ an :term:`importer`.
 
    .. versionadded:: 3.5
 
+   .. versionchanged:: 3.6
+      Began calling :meth:`~importlib.abc.Loader.create_module`, removing the
+      compatibility warning for :class:`importlib.machinery.BuiltinImporter` and
+      :class:`importlib.machinery.ExtensionFileLoader`.
+
    .. classmethod:: factory(loader)
 
       A static method which returns a callable that creates a lazy loader. This
diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
index 177b17fb72..ea4dd8c5f1 100644
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -291,6 +291,16 @@ The idlelib package is being modernized and refactored to make IDLE look and wor
 In compensation, the eventual result with be that some idlelib classes will be easier to use, with better APIs and docstrings explaining them.  Additional useful information will be added to idlelib when available.
 
 
+importlib
+---------
+
+:class:`importlib.util.LazyLoader` now calls
+:meth:`~importlib.abc.Loader.create_module` on the wrapped loader, removing the
+restriction that :class:`importlib.machinery.BuiltinImporter` and
+:class:`importlib.machinery.ExtensionFileLoader` couldn't be used with
+:class:`importlib.util.LazyLoader`.
+
+
 os
 --
 
@@ -620,6 +630,9 @@ that may require changes to your code.
 Changes in the Python API
 -------------------------
 
+* When :meth:`importlib.abc.Loader.exec_module` is defined,
+  :meth:`importlib.abc.Loader.create_module` must also be defined.
+
 * The format of the ``co_lnotab`` attribute of code objects changed to support
   negative line number delta. By default, Python does not emit bytecode with
   negative line number delta. Functions using ``frame.f_lineno``,