1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
|
****************************
What's New In Python 3.9
****************************
:Release: |release|
:Date: |today|
.. Rules for maintenance:
* Anyone can add text to this document. Do not spend very much time
on the wording of your changes, because your text will probably
get rewritten to some degree.
* The maintainer will go through Misc/NEWS periodically and add
changes; it's therefore more important to add your changes to
Misc/NEWS than to this file.
* This is not a complete list of every single change; completeness
is the purpose of Misc/NEWS. Some changes I consider too small
or esoteric to include. If such a change is added to the text,
I'll just remove it. (This is another reason you shouldn't spend
too much time on writing your addition.)
* If you want to draw your new text to the attention of the
maintainer, add 'XXX' to the beginning of the paragraph or
section.
* It's OK to just add a fragmentary note about a change. For
example: "XXX Describe the transmogrify() function added to the
socket module." The maintainer will research the change and
write the necessary text.
* You can comment out your additions if you like, but it's not
necessary (especially when a final release is some months away).
* Credit the author of a patch or bugfix. Just the name is
sufficient; the e-mail address isn't necessary.
* It's helpful to add the bug/patch number as a comment:
XXX Describe the transmogrify() function added to the socket
module.
(Contributed by P.Y. Developer in :issue:`12345`.)
This saves the maintainer the effort of going through the Mercurial log
when researching a change.
This article explains the new features in Python 3.9, compared to 3.8.
For full details, see the :ref:`changelog <changelog>`.
.. note::
Prerelease users should be aware that this document is currently in draft
form. It will be updated substantially as Python 3.9 moves towards release,
so it's worth checking back even after reading earlier versions.
Summary -- Release highlights
=============================
.. This section singles out the most important changes in Python 3.9.
Brevity is key.
.. PEP-sized items next.
You should check for DeprecationWarning in your code
====================================================
When Python 2.7 was still supported, many functions were kept for backward
compatibility with Python 2.7. With the end of Python 2.7 support, these
backward compatibility layers have been removed, or will be removed soon.
Most of them emitted a :exc:`DeprecationWarning` warning for several years. For
example, using ``collections.Mapping`` instead of ``collections.abc.Mapping``
emits a :exc:`DeprecationWarning` since Python 3.3, released in 2012.
Test your application with the :option:`-W` ``default`` command-line option to see
:exc:`DeprecationWarning` and :exc:`PendingDeprecationWarning`, or even with
:option:`-W` ``error`` to treat them as errors. :ref:`Warnings Filter
<warning-filter>` can be used to ignore warnings from third-party code.
It has been decided to keep a few backward compatibility layers for one last
release, to give more time to Python projects maintainers to organize the
removal of the Python 2 support and add support for Python 3.9.
Aliases to :ref:`Abstract Base Classes <collections-abstract-base-classes>` in
the :mod:`collections` module, like ``collections.Mapping`` alias to
:class:`collections.abc.Mapping`, are kept for one last release for backward
compatibility. They will be removed from Python 3.10.
More generally, try to run your tests in the :ref:`Python Development Mode
<devmode>` which helps to prepare your code to make it compatible with the
next Python version.
New Features
============
Dictionary Merge & Update Operators
-----------------------------------
Merge (``|``) and update (``|=``) operators have been added to the built-in
:class:`dict` class. See :pep:`584` for a full description.
(Contributed by Brandt Bucher in :issue:`36144`.)
PEP 616: New removeprefix() and removesuffix() string methods
-------------------------------------------------------------
:meth:`str.removeprefix(prefix)<str.removeprefix>` and
:meth:`str.removesuffix(suffix)<str.removesuffix>` have been added
to easily remove an unneeded prefix or a suffix from a string. Corresponding
``bytes``, ``bytearray``, and ``collections.UserString`` methods have also been
added. See :pep:`616` for a full description. (Contributed by Dennis Sweeney in
:issue:`18939`.)
Other Language Changes
======================
* :func:`__import__` now raises :exc:`ImportError` instead of
:exc:`ValueError`, which used to occur when a relative import went past
its top-level package.
(Contributed by Ngalim Siregar in :issue:`37444`.)
* Python now gets the absolute path of the script filename specified on
the command line (ex: ``python3 script.py``): the ``__file__`` attribute of
the :mod:`__main__` module became an absolute path, rather than a relative
path. These paths now remain valid after the current directory is changed
by :func:`os.chdir`. As a side effect, the traceback also displays the
absolute path for :mod:`__main__` module frames in this case.
(Contributed by Victor Stinner in :issue:`20443`.)
* In the :ref:`Python Development Mode <devmode>` and in debug build, the
*encoding* and *errors* arguments are now checked for string encoding and
decoding operations. Examples: :func:`open`, :meth:`str.encode` and
:meth:`bytes.decode`.
By default, for best performance, the *errors* argument is only checked at
the first encoding/decoding error and the *encoding* argument is sometimes
ignored for empty strings.
(Contributed by Victor Stinner in :issue:`37388`.)
* ``"".replace("", s, n)`` now returns ``s`` instead of an empty string for
all non-zero ``n``. It is now consistent with ``"".replace("", s)``.
There are similar changes for :class:`bytes` and :class:`bytearray` objects.
(Contributed by Serhiy Storchaka in :issue:`28029`.)
* Any valid expression can now be used as a :term:`decorator`. Previously, the
grammar was much more restrictive. See :pep:`614` for details.
(Contributed by Brandt Bucher in :issue:`39702`.)
* Improved help for the :mod:`typing` module. Docstrings are now shown for
all special forms and special generic aliases (like ``Union`` and ``List``).
Using :func:`help` with generic alias like ``List[int]`` will show the help
for the correspondent concrete type (``list`` in this case).
(Contributed by Serhiy Storchaka in :issue:`40257`.)
New Modules
===========
* None yet.
Improved Modules
================
ast
---
Added the *indent* option to :func:`~ast.dump` which allows it to produce a
multiline indented output.
(Contributed by Serhiy Storchaka in :issue:`37995`.)
Added :func:`ast.unparse` as a function in the :mod:`ast` module that can
be used to unparse an :class:`ast.AST` object and produce a string with code
that would produce an equivalent :class:`ast.AST` object when parsed.
(Contributed by Pablo Galindo and Batuhan Taskaya in :issue:`38870`.)
Added docstrings to AST nodes that contains the ASDL signature used to
construct that node. (Contributed by Batuhan Taskaya in :issue:`39638`.)
asyncio
-------
Due to significant security concerns, the *reuse_address* parameter of
:meth:`asyncio.loop.create_datagram_endpoint` is no longer supported. This is
because of the behavior of the socket option ``SO_REUSEADDR`` in UDP. For more
details, see the documentation for ``loop.create_datagram_endpoint()``.
(Contributed by Kyle Stanley, Antoine Pitrou, and Yury Selivanov in
:issue:`37228`.)
Added a new :term:`coroutine` :meth:`~asyncio.loop.shutdown_default_executor`
that schedules a shutdown for the default executor that waits on the
:class:`~concurrent.futures.ThreadPoolExecutor` to finish closing. Also,
:func:`asyncio.run` has been updated to use the new :term:`coroutine`.
(Contributed by Kyle Stanley in :issue:`34037`.)
Added :class:`asyncio.PidfdChildWatcher`, a Linux-specific child watcher
implementation that polls process file descriptors. (:issue:`38692`)
concurrent.futures
------------------
Added a new *cancel_futures* parameter to
:meth:`concurrent.futures.Executor.shutdown` that cancels all pending futures
which have not started running, instead of waiting for them to complete before
shutting down the executor.
(Contributed by Kyle Stanley in :issue:`39349`.)
Removed daemon threads from :class:`~concurrent.futures.ThreadPoolExecutor`
and :class:`~concurrent.futures.ProcessPoolExecutor`. This improves
compatibility with subinterpreters and predictability in their shutdown
processes. (Contributed by Kyle Stanley in :issue:`39812`.)
Workers in :class:`~concurrent.futures.ProcessPoolExecutor` are now spawned on
demand, only when there are no available idle workers to reuse. This optimizes
startup overhead and reduces the amount of lost CPU time to idle workers.
(Contributed by Kyle Stanley in :issue:`39207`.)
curses
------
Add :func:`curses.get_escdelay`, :func:`curses.set_escdelay`,
:func:`curses.get_tabsize`, and :func:`curses.set_tabsize` functions.
(Contributed by Anthony Sottile in :issue:`38312`.)
fcntl
-----
Added constants :data:`~fcntl.F_OFD_GETLK`, :data:`~fcntl.F_OFD_SETLK`
and :data:`~fcntl.F_OFD_SETLKW`.
(Contributed by Dong-hee Na in :issue:`38602`.)
ftplib
-------
:class:`~ftplib.FTP` and :class:`~ftplib.FTP_TLS` now raise a :class:`ValueError`
if the given timeout for their constructor is zero to prevent the creation of
a non-blocking socket. (Contributed by Dong-hee Na in :issue:`39259`.)
functools
---------
Add the :class:`functools.TopologicalSorter` class to offer functionality to perform
topological sorting of graphs. (Contributed by Pablo Galindo, Tim Peters and Larry
Hastings in :issue:`17005`.)
gc
--
When the garbage collector makes a collection in which some objects resurrect
(they are reachable from outside the isolated cycles after the finalizers have
been executed), do not block the collection of all objects that are still
unreachable. (Contributed by Pablo Galindo and Tim Peters in :issue:`38379`.)
Added a new function :func:`gc.is_finalized` to check if an object has been
finalized by the garbage collector. (Contributed by Pablo Galindo in
:issue:`39322`.)
http
----
HTTP status codes ``103 EARLY_HINTS``, ``418 IM_A_TEAPOT`` and ``425 TOO_EARLY`` are added to
:class:`http.HTTPStatus`. (Contributed by Dong-hee Na in :issue:`39509` and Ross Rhodes in :issue:`39507`.)
imaplib
-------
:class:`~imaplib.IMAP4` and :class:`~imaplib.IMAP4_SSL` now have
an optional *timeout* parameter for their constructors.
Also, the :meth:`~imaplib.IMAP4.open` method now has an optional *timeout* parameter
with this change. The overridden methods of :class:`~imaplib.IMAP4_SSL` and
:class:`~imaplib.IMAP4_stream` were applied to this change.
(Contributed by Dong-hee Na in :issue:`38615`.)
importlib
---------
To improve consistency with import statements, :func:`importlib.util.resolve_name`
now raises :exc:`ImportError` instead of :exc:`ValueError` for invalid relative
import attempts.
(Contributed by Ngalim Siregar in :issue:`37444`.)
inspect
-------
:attr:`inspect.BoundArguments.arguments` is changed from ``OrderedDict`` to regular
dict. (Contributed by Inada Naoki in :issue:`36350` and :issue:`39775`.)
ipaddress
---------
:mod:`ipaddress` now supports IPv6 Scoped Addresses (IPv6 address with suffix ``%<scope_id>``).
Scoped IPv6 addresses can be parsed using :class:`ipaddress.IPv6Address`.
If present, scope zone ID is available through the :attr:`~ipaddress.IPv6Address.scope_id` attribute.
(Contributed by Oleksandr Pavliuk in :issue:`34788`.)
math
----
Expanded the :func:`math.gcd` function to handle multiple arguments.
Formerly, it only supported two arguments.
(Contributed by Serhiy Storchaka in :issue:`39648`.)
Add :func:`math.lcm`: return the least common multiple of specified arguments.
(Contributed by Mark Dickinson, Ananthakrishnan and Serhiy Storchaka in
:issue:`39479` and :issue:`39648`.)
Add :func:`math.nextafter`: return the next floating-point value after *x*
towards *y*.
(Contributed by Victor Stinner in :issue:`39288`.)
Add :func:`math.ulp`: return the value of the least significant bit
of a float.
(Contributed by Victor Stinner in :issue:`39310`.)
nntplib
-------
:class:`~nntplib.NNTP` and :class:`~nntplib.NNTP_SSL` now raise a :class:`ValueError`
if the given timeout for their constructor is zero to prevent the creation of
a non-blocking socket. (Contributed by Dong-hee Na in :issue:`39259`.)
os
--
Added :data:`~os.CLD_KILLED` and :data:`~os.CLD_STOPPED` for :attr:`si_code`.
(Contributed by Dong-hee Na in :issue:`38493`.)
Exposed the Linux-specific :func:`os.pidfd_open` (:issue:`38692`) and
:data:`os.P_PIDFD` (:issue:`38713`) for process management with file
descriptors.
The :func:`os.unsetenv` function is now also available on Windows.
(Contributed by Victor Stinner in :issue:`39413`.)
The :func:`os.putenv` and :func:`os.unsetenv` functions are now always
available.
(Contributed by Victor Stinner in :issue:`39395`.)
Add :func:`os.waitstatus_to_exitcode` function:
convert a wait status to an exit code.
(Contributed by Victor Stinner in :issue:`40094`.)
pathlib
-------
Added :meth:`pathlib.Path.readlink()` which acts similarly to
:func:`os.readlink`.
(Contributed by Girts Folkmanis in :issue:`30618`)
poplib
------
:class:`~poplib.POP3` and :class:`~poplib.POP3_SSL` now raise a :class:`ValueError`
if the given timeout for their constructor is zero to prevent the creation of
a non-blocking socket. (Contributed by Dong-hee Na in :issue:`39259`.)
pprint
------
:mod:`pprint` can now pretty-print :class:`types.SimpleNamespace`.
(Contributed by Carl Bordum Hansen in :issue:`37376`.)
pydoc
-----
The documentation string is now shown not only for class, function,
method etc, but for any object that has its own ``__doc__`` attribute.
(Contributed by Serhiy Storchaka in :issue:`40257`.)
random
------
Add a new :attr:`random.Random.randbytes` method: generate random bytes.
(Contributed by Victor Stinner in :issue:`40286`.)
signal
------
Exposed the Linux-specific :func:`signal.pidfd_send_signal` for sending to
signals to a process using a file descriptor instead of a pid. (:issue:`38712`)
smtplib
-------
:class:`~smtplib.SMTP` and :class:`~smtplib.SMTP_SSL` now raise a :class:`ValueError`
if the given timeout for their constructor is zero to prevent the creation of
a non-blocking socket. (Contributed by Dong-hee Na in :issue:`39259`.)
:class:`~smtplib.LMTP` constructor now has an optional *timeout* parameter.
(Contributed by Dong-hee Na in :issue:`39329`.)
socket
------
The :mod:`socket` module now exports the :data:`~socket.CAN_RAW_JOIN_FILTERS`
constant on Linux 4.1 and greater.
(Contributed by Stefan Tatschner and Zackery Spytz in :issue:`25780`.)
sys
---
Add a new :attr:`sys.platlibdir` attribute: name of the platform-specific
library directory. It is used to build the path of platform-specific dynamic
libraries and the path of the standard library. It is equal to ``"lib"`` on
most platforms. On Fedora and SuSE, it is equal to ``"lib64"`` on 64-bit
platforms.
(Contributed by Jan Matějek, Matěj Cepl, Charalampos Stratakis and Victor Stinner in :issue:`1294959`.)
typing
------
:pep:`593` introduced an :data:`typing.Annotated` type to decorate existing
types with context-specific metadata and new ``include_extras`` parameter to
:func:`typing.get_type_hints` to access the metadata at runtime. (Contributed
by Till Varoquaux and Konstantin Kashin.)
unicodedata
-----------
The Unicode database has been updated to version 13.0.0. (:issue:`39926`).
venv
----
The activation scripts provided by :mod:`venv` now all specify their prompt
customization consistently by always using the value specified by
``__VENV_PROMPT__``. Previously some scripts unconditionally used
``__VENV_PROMPT__``, others only if it happened to be set (which was the default
case), and one used ``__VENV_NAME__`` instead.
(Contributed by Brett Cannon in :issue:`37663`.)
xml
---
White space characters within attributes are now preserved when serializing
:mod:`xml.etree.ElementTree` to XML file. EOLNs are no longer normalized
to "\n". This is the result of discussion about how to interpret
section 2.11 of XML spec.
(Contributed by Mefistotelis in :issue:`39011`.)
Optimizations
=============
* Optimized the idiom for assignment a temporary variable in comprehensions.
Now ``for y in [expr]`` in comprehensions is as fast as a simple assignment
``y = expr``. For example:
sums = [s for s in [0] for x in data for s in [s + x]]
Unlike to the ``:=`` operator this idiom does not leak a variable to the
outer scope.
(Contributed by Serhiy Storchaka in :issue:`32856`.)
* Optimize signal handling in multithreaded applications. If a thread different
than the main thread gets a signal, the bytecode evaluation loop is no longer
interrupted at each bytecode instruction to check for pending signals which
cannot be handled. Only the main thread of the main interpreter can handle
signals.
Previously, the bytecode evaluation loop was interrupted at each instruction
until the main thread handles signals.
(Contributed by Victor Stinner in :issue:`40010`.)
* Optimize the :mod:`subprocess` module on FreeBSD using ``closefrom()``.
(Contributed by Ed Maste, Conrad Meyer, Kyle Evans, Kubilay Kocak and Victor
Stinner in :issue:`38061`.)
Build and C API Changes
=======================
* New :c:func:`PyThreadState_GetInterpreter` and
:c:func:`PyInterpreterState_Get` functions to get the interpreter.
New :c:func:`PyThreadState_GetFrame` function to get the current frame of a
Python thread state.
New :c:func:`PyThreadState_GetID` function: get the unique identifier of a
Python thread state.
(Contributed by Victor Stinner in :issue:`39947`.)
* Add ``--with-platlibdir`` option to the ``configure`` script: name of the
platform-specific library directory, stored in the new :attr:`sys.platlibdir`
attribute. See :attr:`sys.platlibdir` attribute for more information.
(Contributed by Jan Matějek, Matěj Cepl, Charalampos Stratakis and Victor Stinner in :issue:`1294959`.)
* Add a new public :c:func:`PyObject_CallNoArgs` function to the C API, which
calls a callable Python object without any arguments. It is the most efficient
way to call a callable Python object without any argument.
(Contributed by Victor Stinner in :issue:`37194`.)
* The global variable :c:data:`PyStructSequence_UnnamedField` is now a constant
and refers to a constant string.
(Contributed by Serhiy Storchaka in :issue:`38650`.)
* Exclude ``PyFPE_START_PROTECT()`` and ``PyFPE_END_PROTECT()`` macros of
``pyfpe.h`` from ``Py_LIMITED_API`` (stable API).
(Contributed by Victor Stinner in :issue:`38835`.)
* Remove ``PyMethod_ClearFreeList()`` and ``PyCFunction_ClearFreeList()``
functions: the free lists of bound method objects have been removed.
(Contributed by Inada Naoki and Victor Stinner in :issue:`37340`.)
* Remove ``PyUnicode_ClearFreeList()`` function: the Unicode free list has been
removed in Python 3.3.
(Contributed by Victor Stinner in :issue:`38896`.)
* The ``tp_print`` slot of :ref:`PyTypeObject <type-structs>` has been removed.
It was used for printing objects to files in Python 2.7 and before. Since
Python 3.0, it has been ignored and unused.
(Contributed by Jeroen Demeyer in :issue:`36974`.)
* On non-Windows platforms, the :c:func:`setenv` and :c:func:`unsetenv`
functions are now required to build Python.
(Contributed by Victor Stinner in :issue:`39395`.)
* The ``COUNT_ALLOCS`` special build macro has been removed.
(Contributed by Victor Stinner in :issue:`39489`.)
* Changes in the limited C API (if ``Py_LIMITED_API`` macro is defined):
* Provide :c:func:`Py_EnterRecursiveCall` and :c:func:`Py_LeaveRecursiveCall`
as regular functions for the limited API. Previously, there were defined as
macros, but these macros didn't compile with the limited C API which cannot
access ``PyThreadState.recursion_depth`` field (the structure is opaque in
the limited C API).
* Exclude the following functions from the limited C API:
* ``_Py_CheckRecursionLimit``
* ``_Py_NewReference()``
* ``_Py_ForgetReference()``
* ``_PyTraceMalloc_NewReference()``
* ``_Py_GetRefTotal()``
* The trashcan mechanism which never worked in the limited C API.
* ``PyTrash_UNWIND_LEVEL``
* ``Py_TRASHCAN_BEGIN_CONDITION``
* ``Py_TRASHCAN_BEGIN``
* ``Py_TRASHCAN_END``
* ``Py_TRASHCAN_SAFE_BEGIN``
* ``Py_TRASHCAN_SAFE_END``
* The following static inline functions or macros become regular "opaque"
function to hide implementation details:
* ``_Py_NewReference()``
* ``PyObject_INIT()`` and ``PyObject_INIT_VAR()`` become aliases to
:c:func:`PyObject_Init` and :c:func:`PyObject_InitVar` in the limited C
API, but are overriden with static inline function otherwise. Thanks to
that, it was possible to exclude ``_Py_NewReference()`` from the limited
C API.
* Move following functions and definitions to the internal C API:
* ``_PyDebug_PrintTotalRefs()``
* ``_Py_PrintReferences()``
* ``_Py_PrintReferenceAddresses()``
* ``_Py_tracemalloc_config``
* ``_Py_AddToAllObjects()`` (specific to ``Py_TRACE_REFS`` build)
(Contributed by Victor Stinner in :issue:`38644` and :issue:`39542`.)
* ``PyInterpreterState.eval_frame`` (:pep:`523`) now requires a new mandatory
*tstate* parameter (``PyThreadState*``).
(Contributed by Victor Stinner in :issue:`38500`.)
* Extension modules: :c:member:`~PyModuleDef.m_traverse`,
:c:member:`~PyModuleDef.m_clear` and :c:member:`~PyModuleDef.m_free`
functions of :c:type:`PyModuleDef` are no longer called if the module state
was requested but is not allocated yet. This is the case immediately after
the module is created and before the module is executed
(:c:data:`Py_mod_exec` function). More precisely, these functions are not called
if :c:member:`~PyModuleDef.m_size` is greater than 0 and the module state (as
returned by :c:func:`PyModule_GetState`) is ``NULL``.
Extension modules without module state (``m_size <= 0``) are not affected.
* If :c:func:`Py_AddPendingCall` is called in a subinterpreter, the function is
now scheduled to be called from the subinterpreter, rather than being called
from the main interpreter. Each subinterpreter now has its own list of
scheduled calls.
(Contributed by Victor Stinner in :issue:`39984`.)
* Remove ``_PyRuntime.getframe`` hook and remove ``_PyThreadState_GetFrame``
macro which was an alias to ``_PyRuntime.getframe``. They were only exposed
by the internal C API. Remove also ``PyThreadFrameGetter`` type.
(Contributed by Victor Stinner in :issue:`39946`.)
* The :c:func:`PyModule_AddType` function is added to help adding a type to a module.
(Contributed by Dong-hee Na in :issue:`40024`.)
* The Windows registry is no longer used to initialize :data:`sys.path` when
the ``-E`` option is used. This is significant when embedding Python on
Windows.
(Contributed by Zackery Spytz in :issue:`8901`.)
* Add the functions :c:func:`PyObject_GC_IsTracked` and
:c:func:`PyObject_GC_IsFinalized` to the public API to allow to query if
Python objects are being currently tracked or have been already finalized by
the garbage collector respectively. (Contributed by Pablo Galindo in
:issue:`40241`.)
Deprecated
==========
* The distutils ``bdist_msi`` command is now deprecated, use
``bdist_wheel`` (wheel packages) instead.
(Contributed by Hugo van Kemenade in :issue:`39586`.)
* Currently :func:`math.factorial` accepts :class:`float` instances with
non-negative integer values (like ``5.0``). It raises a :exc:`ValueError`
for non-integral and negative floats. It is now deprecated. In future
Python versions it will raise a :exc:`TypeError` for all floats.
(Contributed by Serhiy Storchaka in :issue:`37315`.)
* The :mod:`parser` module is deprecated and will be removed in future versions
of Python. For the majority of use cases, users can leverage the Abstract Syntax
Tree (AST) generation and compilation stage, using the :mod:`ast` module.
* Using :data:`NotImplemented` in a boolean context has been deprecated,
as it is almost exclusively the result of incorrect rich comparator
implementations. It will be made a :exc:`TypeError` in a future version
of Python.
(Contributed by Josh Rosenberg in :issue:`35712`.)
* The :mod:`random` module currently accepts any hashable type as a
possible seed value. Unfortunately, some of those types are not
guaranteed to have a deterministic hash value. After Python 3.9,
the module will restrict its seeds to :const:`None`, :class:`int`,
:class:`float`, :class:`str`, :class:`bytes`, and :class:`bytearray`.
* Opening the :class:`~gzip.GzipFile` file for writing without specifying
the *mode* argument is deprecated. In future Python versions it will always
be opened for reading by default. Specify the *mode* argument for opening
it for writing and silencing a warning.
(Contributed by Serhiy Storchaka in :issue:`28286`.)
* Deprecated the ``split()`` method of :class:`_tkinter.TkappType` in
favour of the ``splitlist()`` method which has more consistent and
predicable behavior.
(Contributed by Serhiy Storchaka in :issue:`38371`.)
* The explicit passing of coroutine objects to :func:`asyncio.wait` has been
deprecated and will be removed in version 3.11.
(Contributed by Yury Selivanov and Kyle Stanley in :issue:`34790`.)
* binhex4 and hexbin4 standards are now deprecated. The :`binhex` module
and the following :mod:`binascii` functions are now deprecated:
* :func:`~binascii.b2a_hqx`, :func:`~binascii.a2b_hqx`
* :func:`~binascii.rlecode_hqx`, :func:`~binascii.rledecode_hqx`
(Contributed by Victor Stinner in :issue:`39353`.)
* :mod:`ast` classes ``slice``, ``Index`` and ``ExtSlice`` are considered deprecated
and will be removed in future Python versions. ``value`` itself should be
used instead of ``Index(value)``. ``Tuple(slices, Load())`` should be
used instead of ``ExtSlice(slices)``.
(Contributed by Serhiy Storchaka in :issue:`32892`.)
* :mod:`ast` classes ``Suite``, ``Param``, ``AugLoad`` and ``AugStore``
are considered deprecated and will be removed in future Python versions.
They were not generated by the parser and not accepted by the code
generator in Python 3.
(Contributed by Batuhan Taskaya in :issue:`39639` and :issue:`39969`
and Serhiy Storchaka in :issue:`39988`.)
* The :c:func:`PyEval_InitThreads` and :c:func:`PyEval_ThreadsInitialized`
functions are now deprecated and will be removed in Python 3.11. Calling
:c:func:`PyEval_InitThreads` now does nothing. The :term:`GIL` is initialized
by :c:func:`Py_Initialize()` since Python 3.7.
(Contributed by Victor Stinner in :issue:`39877`.)
* Passing ``None`` as the first argument to the :func:`shlex.split` function
has been deprecated. (Contributed by Zackery Spytz in :issue:`33262`.)
Removed
=======
* The erroneous version at :data:`unittest.mock.__version__` has been removed.
* :class:`nntplib.NNTP`: ``xpath()`` and ``xgtitle()`` methods have been removed.
These methods are deprecated since Python 3.3. Generally, these extensions
are not supported or not enabled by NNTP server administrators.
For ``xgtitle()``, please use :meth:`nntplib.NNTP.descriptions` or
:meth:`nntplib.NNTP.description` instead.
(Contributed by Dong-hee Na in :issue:`39366`.)
* :class:`array.array`: ``tostring()`` and ``fromstring()`` methods have been
removed. They were aliases to ``tobytes()`` and ``frombytes()``, deprecated
since Python 3.2.
(Contributed by Victor Stinner in :issue:`38916`.)
* The undocumented ``sys.callstats()`` function has been removed. Since Python
3.7, it was deprecated and always returned :const:`None`. It required a special
build option ``CALL_PROFILE`` which was already removed in Python 3.7.
(Contributed by Victor Stinner in :issue:`37414`.)
* The ``sys.getcheckinterval()`` and ``sys.setcheckinterval()`` functions have
been removed. They were deprecated since Python 3.2. Use
:func:`sys.getswitchinterval` and :func:`sys.setswitchinterval` instead.
(Contributed by Victor Stinner in :issue:`37392`.)
* The C function ``PyImport_Cleanup()`` has been removed. It was documented as:
"Empty the module table. For internal use only."
(Contributed by Victor Stinner in :issue:`36710`.)
* ``_dummy_thread`` and ``dummy_threading`` modules have been removed. These
modules were deprecated since Python 3.7 which requires threading support.
(Contributed by Victor Stinner in :issue:`37312`.)
* ``aifc.openfp()`` alias to ``aifc.open()``, ``sunau.openfp()`` alias to
``sunau.open()``, and ``wave.openfp()`` alias to :func:`wave.open()` have been
removed. They were deprecated since Python 3.7.
(Contributed by Victor Stinner in :issue:`37320`.)
* The :meth:`~threading.Thread.isAlive()` method of :class:`threading.Thread`
has been removed. It was deprecated since Python 3.8.
Use :meth:`~threading.Thread.is_alive()` instead.
(Contributed by Dong-hee Na in :issue:`37804`.)
* Methods ``getchildren()`` and ``getiterator()`` of classes
:class:`~xml.etree.ElementTree.ElementTree` and
:class:`~xml.etree.ElementTree.Element` in the :mod:`~xml.etree.ElementTree`
module have been removed. They were deprecated in Python 3.2.
Use ``iter(x)`` or ``list(x)`` instead of ``x.getchildren()`` and
``x.iter()`` or ``list(x.iter())`` instead of ``x.getiterator()``.
The ``xml.etree.cElementTree`` module has been removed,
use the :mod:`xml.etree.ElementTree` module instead.
Since Python 3.3 the ``xml.etree.cElementTree`` module has been deprecated,
the ``xml.etree.ElementTree`` module uses a fast implementation whenever
available.
(Contributed by Serhiy Storchaka in :issue:`36543`.)
* The old :mod:`plistlib` API has been removed, it was deprecated since Python
3.4. Use the :func:`~plistlib.load`, :func:`~plistlib.loads`, :func:`~plistlib.dump`, and
:func:`~plistlib.dumps` functions. Additionally, the *use_builtin_types* parameter was
removed, standard :class:`bytes` objects are always used instead.
(Contributed by Jon Janzen in :issue:`36409`.)
* The C function ``PyThreadState_DeleteCurrent()`` has been removed. It was not documented.
(Contributed by Joannah Nanjekye in :issue:`37878`.)
* The C function ``PyGen_NeedsFinalizing`` has been removed. It was not
documented, tested, or used anywhere within CPython after the implementation
of :pep:`442`. Patch by Joannah Nanjekye.
(Contributed by Joannah Nanjekye in :issue:`15088`)
* ``base64.encodestring()`` and ``base64.decodestring()``, aliases deprecated
since Python 3.1, have been removed: use :func:`base64.encodebytes` and
:func:`base64.decodebytes` instead.
(Contributed by Victor Stinner in :issue:`39351`.)
* ``fractions.gcd()`` function has been removed, it was deprecated since Python
3.5 (:issue:`22486`): use :func:`math.gcd` instead.
(Contributed by Victor Stinner in :issue:`39350`.)
* The *buffering* parameter of :class:`bz2.BZ2File` has been removed. Since
Python 3.0, it was ignored and using it emitted a :exc:`DeprecationWarning`.
Pass an open file object to control how the file is opened.
(Contributed by Victor Stinner in :issue:`39357`.)
* The *encoding* parameter of :func:`json.loads` has been removed.
As of Python 3.1, it was deprecated and ignored; using it has emitted a
:exc:`DeprecationWarning` since Python 3.8.
(Contributed by Inada Naoki in :issue:`39377`)
* ``with (await asyncio.lock):`` and ``with (yield from asyncio.lock):`` statements are
not longer supported, use ``async with lock`` instead. The same is correct for
``asyncio.Condition`` and ``asyncio.Semaphore``.
(Contributed by Andrew Svetlov in :issue:`34793`.)
* The :func:`sys.getcounts` function, the ``-X showalloccount`` command line
option and the ``show_alloc_count`` field of the C structure
:c:type:`PyConfig` have been removed. They required a special Python build by
defining ``COUNT_ALLOCS`` macro.
(Contributed by Victor Stinner in :issue:`39489`.)
* The ``_field_types`` attribute of the :class:`typing.NamedTuple` class
has been removed. It was deprecated deprecated since Python 3.8. Use
the ``__annotations__`` attribute instead.
(Contributed by Serhiy Storchaka in :issue:`40182`.)
* The :meth:`symtable.SymbolTable.has_exec` method has been removed. It was
deprecated since 2006, and only returning ``False`` when it's called.
(Contributed by Batuhan Taskaya in :issue:`40208`)
Porting to Python 3.9
=====================
This section lists previously described changes and other bugfixes
that may require changes to your code.
Changes in the Python API
-------------------------
* :func:`__import__` and :func:`importlib.util.resolve_name` now raise
:exc:`ImportError` where it previously raised :exc:`ValueError`. Callers
catching the specific exception type and supporting both Python 3.9 and
earlier versions will need to catch both using ``except (ImportError, ValueError):``.
* The :mod:`venv` activation scripts no longer special-case when
``__VENV_PROMPT__`` is set to ``""``.
* The :meth:`select.epoll.unregister` method no longer ignores the
:data:`~errno.EBADF` error.
(Contributed by Victor Stinner in :issue:`39239`.)
* The *compresslevel* parameter of :class:`bz2.BZ2File` became keyword-only,
since the *buffering* parameter has been removed.
(Contributed by Victor Stinner in :issue:`39357`.)
* Simplified AST for subscription. Simple indices will be represented by
their value, extended slices will be represented as tuples.
``Index(value)`` will return a ``value`` itself, ``ExtSlice(slices)``
will return ``Tuple(slices, Load())``.
(Contributed by Serhiy Storchaka in :issue:`34822`.)
* The :mod:`importlib` module now ignores the :envvar:`PYTHONCASEOK`
environment variable when the :option:`-E` or :option:`-I` command line
options are being used.
* The *encoding* parameter has been added to the classes :class:`ftplib.FTP` and
:class:`ftplib.FTP_TLS` as a keyword-only parameter, and the default encoding
is changed from Latin-1 to UTF-8 to follow :rfc:`2640`.
* :func:`inspect.getdoc` no longer returns docstring inherited from the type
of the object or from parent class if it is a class if it is not defined
in the object itself.
(Contributed by Serhiy Storchaka in :issue:`40257`.)
* :meth:`asyncio.loop.shutdown_default_executor` has been added to
:class:`~asyncio.AbstractEventLoop`, meaning alternative event loops that
inherit from it should have this method defined.
(Contributed by Kyle Stanley in :issue:`34037`.)
* The constant values of future flags in the :mod:`__future__` module
is updated in order to prevent collision with compiler flags. Previously
``PyCF_ALLOW_TOP_LEVEL_AWAIT`` was clashing with ``CO_FUTURE_DIVISION``.
(Contributed by Batuhan Taskaya in :issue:`39562`)
CPython bytecode changes
------------------------
* The :opcode:`LOAD_ASSERTION_ERROR` opcode was added for handling the
:keyword:`assert` statement. Previously, the assert statement would not work
correctly if the :exc:`AssertionError` exception was being shadowed.
(Contributed by Zackery Spytz in :issue:`34880`.)
|