From 4590f72259ecbcea66e12a28af14d867255d2e66 Mon Sep 17 00:00:00 2001 From: Eddie Elizondo Date: Tue, 4 Feb 2020 02:29:25 -0800 Subject: bpo-38076 Clear the interpreter state only after clearing module globals (GH-18039) Currently, during runtime destruction, `_PyImport_Cleanup` is clearing the interpreter state before clearing out the modules themselves. This leads to a segfault on modules that rely on the module state to clear themselves up. For example, let's take the small snippet added in the issue by @DinoV : ``` import _struct class C: def __init__(self): self.pack = _struct.pack def __del__(self): self.pack('I', -42) _struct.x = C() ``` The module `_struct` uses the module state to run `pack`. Therefore, the module state has to be alive until after the module has been cleared out to successfully run `C.__del__`. This happens at line 606, when `_PyImport_Cleanup` calls `_PyModule_Clear`. In fact, the loop that calls `_PyModule_Clear` has in its comments: > Now, if there are any modules left alive, clear their globals to minimize potential leaks. All C extension modules actually end up here, since they are kept alive in the interpreter state. That means that we can't clear the module state (which is used by C Extensions) before we run that loop. Moving `_PyInterpreterState_ClearModules` until after it, fixes the segfault in the code snippet. Finally, this updates a test in `io` to correctly assert the error that it now throws (since it now finds the io module state). The test that uses this is: `test_create_at_shutdown_without_encoding`. Given this test is now working is a proof that the module state now stays alive even when `__del__` is called at module destruction time. Thus, I didn't add a new tests for this. https://bugs.python.org/issue38076 --- Python/import.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) (limited to 'Python/import.c') diff --git a/Python/import.c b/Python/import.c index 9838c3fa04..8bf044827c 100644 --- a/Python/import.c +++ b/Python/import.c @@ -568,8 +568,6 @@ _PyImport_Cleanup(PyThreadState *tstate) _PyErr_Clear(tstate); } Py_XDECREF(dict); - /* Clear module dict copies stored in the interpreter state */ - _PyInterpreterState_ClearModules(interp); /* Collect references */ _PyGC_CollectNoFail(); /* Dump GC stats before it's too late, since it uses the warnings @@ -621,6 +619,9 @@ _PyImport_Cleanup(PyThreadState *tstate) } _PyModule_ClearDict(interp->builtins); + /* Clear module dict copies stored in the interpreter state */ + _PyInterpreterState_ClearModules(interp); + /* Clear and delete the modules directory. Actual modules will still be there only if imported during the execution of some destructor. */ -- cgit v1.2.1