From 2d2af320d94afc6561e8f8adf174c9d3fd9065bc Mon Sep 17 00:00:00 2001 From: Kevin Adler Date: Wed, 14 Oct 2020 20:53:27 -0500 Subject: bpo-41894: Fix UnicodeDecodeError while loading native module (GH-22466) When running in a non-UTF-8 locale, if an error occurs while importing a native Python module (say because a dependent share library is missing), the error message string returned may contain non-ASCII code points causing a UnicodeDecodeError. PyUnicode_DecodeFSDefault is used for buffers which may contain filesystem paths. For consistency with os.strerror(), PyUnicode_DecodeLocale is used for buffers which contain system error messages. While the shortname parameter is always encoded in ASCII according to PEP 489, it is left decoded using PyUnicode_FromString to minimize the changes and since it should not affect the decoding (albeit _potentially_ slower). In dynload_hpux, since the error buffer contains a message generated from a static ASCII string and the module filesystem path, PyUnicode_DecodeFSDefault is used instead of PyUnicode_DecodeLocale as is used elsewhere. * bpo-41894: Fix bugs in dynload error msg handling For both dynload_aix and dynload_hpux, properly handle the possibility that decoding strings may return NULL and when such an error happens, properly decrement any previously decoded strings and return early. In addition, in dynload_aix, ensure that we pass the decoded string *object* pathname_ob to PyErr_SetImportError instead of the original pathname buffer. Co-authored-by: Serhiy Storchaka --- Python/dynload_hpux.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) (limited to 'Python/dynload_hpux.c') diff --git a/Python/dynload_hpux.c b/Python/dynload_hpux.c index 4b964a69d3..e36d608c6d 100644 --- a/Python/dynload_hpux.c +++ b/Python/dynload_hpux.c @@ -36,9 +36,20 @@ dl_funcptr _PyImport_FindSharedFuncptr(const char *prefix, char buf[256]; PyOS_snprintf(buf, sizeof(buf), "Failed to load %.200s", pathname); - PyObject *buf_ob = PyUnicode_FromString(buf); + PyObject *buf_ob = PyUnicode_DecodeFSDefault(buf); + if (buf_ob == NULL) + return NULL; PyObject *shortname_ob = PyUnicode_FromString(shortname); - PyObject *pathname_ob = PyUnicode_FromString(pathname); + if (shortname_ob == NULL) { + Py_DECREF(buf_ob); + return NULL; + } + PyObject *pathname_ob = PyUnicode_DecodeFSDefault(pathname); + if (pathname_ob == NULL) { + Py_DECREF(buf_ob); + Py_DECREF(shortname_ob); + return NULL; + } PyErr_SetImportError(buf_ob, shortname_ob, pathname_ob); Py_DECREF(buf_ob); Py_DECREF(shortname_ob); -- cgit v1.2.1