delta/cpython-git.git/Objects/stringlib/codecs.h, branch faster-bytes-iter

bpo-29882: Add _Py_popcount32() function (GH-20518)

2020-06-08T14:30:33+00:00

* Rename pycore_byteswap.h to pycore_bitutils.h.
* Move popcount_digit() to pycore_bitutils.h as _Py_popcount32().
* _Py_popcount32() uses GCC and clang builtin function if available.
* Add unit tests to _Py_popcount32().

bpo-40302: UTF-32 encoder SWAB4() macro use a|b rather than a+b (GH-19572)

2020-04-17T17:13:34+00:00

bpo-40302: Add pycore_byteswap.h header file (GH-19552)

2020-04-17T15:47:20+00:00

Add a new internal pycore_byteswap.h header file with the following
functions:

* _Py_bswap16()
* _Py_bswap32()
* _Py_bswap64()

Use these functions in _ctypes, sha256 and sha512 modules,
and also use in the UTF-32 encoder.

sha256, sha512 and _ctypes modules are now built with the internal
C API.

bpo-39943: Add the const qualifier to pointers on non-mutable PyUnicode data. (GH-19345)

2020-04-11T07:48:40+00:00

Update some www.unicode.org URLs to use HTTPS. (GH-18912)

2020-03-11T04:10:59+00:00

bpo-39087: Optimize PyUnicode_AsUTF8AndSize() (GH-18327)

2020-02-27T04:48:59+00:00

Avoid using temporary bytes object.

closes bpo-39605: Fix some casts to not cast away const. (GH-18453)

2020-02-12T02:28:35+00:00

gcc -Wcast-qual turns up a number of instances of casting away constness of pointers. Some of these can be safely modified, by either:

Adding the const to the type cast, as in:

-    return _PyUnicode_FromUCS1((unsigned char*)s, size);
+    return _PyUnicode_FromUCS1((const unsigned char*)s, size);

or, Removing the cast entirely, because it's not necessary (but probably was at one time), as in:

-    PyDTrace_FUNCTION_ENTRY((char *)filename, (char *)funcname, lineno);
+    PyDTrace_FUNCTION_ENTRY(filename, funcname, lineno);

These changes will not change code, but they will make it much easier to check for errors in consts

bpo-24214: Fixed the UTF-8 and UTF-16 incremental decoders. (GH-14304)

2019-06-25T08:54:18+00:00

* The UTF-8 incremental decoders fails now fast if encounter
  a sequence that can't be handled by the error handler.
* The UTF-16 incremental decoders with the surrogatepass error
  handler decodes now a lone low surrogate with final=False.

bpo-36775: _PyCoreConfig only uses wchar_t* (GH-13062)

2019-05-02T18:56:30+00:00

_PyCoreConfig: Change filesystem_encoding, filesystem_errors,
stdio_encoding and stdio_errors fields type from char* to wchar_t*.

Changes:

* PyInterpreterState: replace fscodec_initialized (int) with fs_codec
  structure.
* Add get_error_handler_wide() and unicode_encode_utf8() helper
  functions.
* Add error_handler parameter to unicode_encode_locale()
  and unicode_decode_locale().
* Remove _PyCoreConfig_SetString().
* Rename _PyCoreConfig_SetWideString() to _PyCoreConfig_SetString().
* Rename _PyCoreConfig_SetWideStringFromString()
  to _PyCoreConfig_DecodeLocale().

bpo-34523: Support surrogatepass in locale codecs (GH-8995)

2018-08-29T20:21:32+00:00

Add support for the "surrogatepass" error handler in
PyUnicode_DecodeFSDefault() and PyUnicode_EncodeFSDefault()
for the UTF-8 encoding.

Changes:

* _Py_DecodeUTF8Ex() and _Py_EncodeUTF8Ex() now support the
  surrogatepass error handler (_Py_ERROR_SURROGATEPASS).
* _Py_DecodeLocaleEx() and _Py_EncodeLocaleEx() now use
  the _Py_error_handler enum instead of "int surrogateescape" to pass
  the error handler. These functions now return -3 if the error
  handler is unknown.
* Add unit tests on _Py_DecodeLocaleEx() and _Py_EncodeLocaleEx()
  in test_codecs.
* Rename get_error_handler() to _Py_GetErrorHandler() and expose it
  as a private function.
* _freeze_importlib doesn't need config.filesystem_errors="strict"
  workaround anymore.