Python 3.9 compatibility

Python 3.9 no longer ships the xml.etree.cElementTree module. defusedxml
no longer provides the a fixed module with 3.9 as well.
defusedxml.cElementTree is still available with 3.8.

See: https://bugs.python.org/issue36543
See: https://github.com/python/cpython/pull/19108
Fixes: https://github.com/tiran/defusedxml/issues/50
Signed-off-by: Christian Heimes <christian@python.org>

12 files changed, 72 insertions, 23 deletions

diff --git a/.travis.yml b/.travis.yml
index 5f60377..f901a4b 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -16,8 +16,10 @@ matrix:
       env: TOXENV=py36
     - python: 3.7
       env: TOXENV=py37
-    - python: 3.8-dev
+    - python: 3.8
       env: TOXENV=py38
+    - python: 3.9-dev
+      env: TOXENV=py39
     - python: 3.7
       env: TOXENV=black
     - python: 2.7
diff --git a/CHANGES.txt b/CHANGES.txt
index ae9639e..d6bc61f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,6 +1,16 @@
 Changelog
 =========
 
+defusedxml 0.7.0
+----------------
+
+*Release date: ??-Mar-2020*
+
+- Add support for Python 3.9
+- ``defusedxml.cElementTree`` is not available with Python 3.9.
+- Python 2 is deprecate. Support for Python 2 will be removed in 0.8.0.
+
+
 defusedxml 0.6.0
 ----------------
 
diff --git a/README.md b/README.md
index 056ff6d..e4b84f6 100644
--- a/README.md
+++ b/README.md
@@ -40,7 +40,7 @@ properly implement XML specifications. Application developers must not
 rely that a library is always configured for security and potential
 harmful data by default.
 
-<div class="contents">
+<div class="contents" data-depth="2">
 
 Table of Contents
 
@@ -275,6 +275,8 @@ defuse\_stdlib() (*experimental*)
 
 ### defusedxml.cElementTree
 
+**NOTE** `defusedxml.cElementTree` is not available in Python 3.9+
+
 parse(), iterparse(), fromstring(), XMLParser
 
 ### defusedxml.ElementTree
@@ -304,15 +306,17 @@ parse(), parseString()
 ### defusedxml.xmlrpc
 
 The fix is implemented as monkey patch for the stdlib's xmlrpc package
-(3.x) or xmlrpclib module (2.x). The function monkey\_patch() enables
-the fixes, unmonkey\_patch() removes the patch and puts the code in its
-former state.
+(3.x) or xmlrpclib module (2.x). The function
+<span class="title-ref">monkey\_patch()</span> enables the fixes,
+<span class="title-ref">unmonkey\_patch()</span> removes the patch and
+puts the code in its former state.
 
 The monkey patch protects against XML related attacks as well as
 decompression bombs and excessively large requests or responses. The
 default setting is 30 MB for requests, responses and gzip decompression.
-You can modify the default by changing the module variable MAX\_DATA. A
-value of -1 disables the limit.
+You can modify the default by changing the module variable
+<span class="title-ref">MAX\_DATA</span>. A value of
+<span class="title-ref">-1</span> disables the limit.
 
 ### defusedxml.lxml
 
@@ -379,8 +383,8 @@ new API functions:
     expanded. The limit protects the parser against exponential entity
     expansion attacks (aka billion laughs attack). When the limit is
     exceeded the parser stops and fails with
-    XML\_ERROR\_ENTITY\_INDIRECTIONS. A value of 0 disables the
-    protection.
+    <span class="title-ref">XML\_ERROR\_ENTITY\_INDIRECTIONS</span>. A
+    value of 0 disables the protection.
     
       - Supported range  
         0 .. UINT\_MAX
@@ -394,8 +398,8 @@ new API functions:
     parser variable. The setting protects against quadratic blowup
     attacks (lots of expansions of a large entity declaration). When the
     sum of all entities exceeds the limit, the parser stops and fails
-    with XML\_ERROR\_ENTITY\_EXPANSION. A value of 0 disables the
-    protection.
+    with <span class="title-ref">XML\_ERROR\_ENTITY\_EXPANSION</span>. A
+    value of 0 disables the protection.
     
       - Supported range  
         0 .. UINT\_MAX
@@ -409,7 +413,7 @@ new API functions:
     after the endDoctypeDeclHandler has been called. The flag can be set
     inside the endDoctypeDeclHandler. Without DTD information any entity
     reference in the document body leads to
-    XML\_ERROR\_UNDEFINED\_ENTITY.
+    <span class="title-ref">XML\_ERROR\_UNDEFINED\_ENTITY</span>.
     
       - Supported range  
         0, 1
@@ -640,8 +644,7 @@ default settings. It also does entity resolving when an
 `org.xml.sax.EntityResolver` is configured. I'm not yet sure about the
 default setting here.
 
-Java specialists suggest to have a custom builder
-    factory:
+Java specialists suggest to have a custom builder factory:
 
     DocumentBuilderFactory builderFactory = DocumentBuilderFactory.newInstance();
     builderFactory.setXIncludeAware(False);
@@ -712,6 +715,15 @@ See <https://www.python.org/psf/license> for licensing details.
     Injection](https://www.owasp.org/index.php/Testing_for_XML_Injection_\(OWASP-DV-008\))
 # Changelog
 
+## defusedxml 0.7.0
+
+*Release date: ??-Mar-2020*
+
+  - Add support for Python 3.9
+  - `defusedxml.cElementTree` is not available with Python 3.9.
+  - Python 2 is deprecate. Support for Python 2 will be removed in
+    0.8.0.
+
 ## defusedxml 0.6.0
 
 *Release date: 17-Apr-2019*
diff --git a/README.txt b/README.txt
index eb0f13a..8844fb7 100644
--- a/README.txt
+++ b/README.txt
@@ -317,6 +317,8 @@ defuse_stdlib() (*experimental*)
 defusedxml.cElementTree
 -----------------------
 
+**NOTE** ``defusedxml.cElementTree`` is not available in Python 3.9+
+
 parse(), iterparse(), fromstring(), XMLParser
 
 
diff --git a/SECURITY.md b/SECURITY.md
index b57284b..01122ef 100644
--- a/SECURITY.md
+++ b/SECURITY.md
@@ -4,6 +4,7 @@
 
 | Version | Supported          |
 | ------- | ------------------ |
+| 0.7.0   | :white_check_mark: |
 | 0.6.0   | :white_check_mark: |
 | < 0.6   | :x:                |
 
diff --git a/defusedxml/__init__.py b/defusedxml/__init__.py
index 6f3b758..6e33fc2 100644
--- a/defusedxml/__init__.py
+++ b/defusedxml/__init__.py
@@ -14,6 +14,7 @@ from .common import (
     ExternalReferenceForbidden,
     NotSupportedError,
     _apply_defusing,
+    _HAVE_CELEMENTTREE,
 )
 
 
@@ -24,7 +25,10 @@ def defuse_stdlib():
     """
     defused = {}
 
-    from . import cElementTree
+    if _HAVE_CELEMENTTREE:
+        from . import cElementTree
+    else:
+        cElementTree = None
     from . import ElementTree
     from . import minidom
     from . import pulldom
@@ -36,22 +40,25 @@ def defuse_stdlib():
     xmlrpc.monkey_patch()
     defused[xmlrpc] = None
 
-    for defused_mod in [
-        cElementTree,
+    defused_mods = [
         ElementTree,
         minidom,
         pulldom,
         sax,
         expatbuilder,
         expatreader,
-    ]:
+    ]
+    if _HAVE_CELEMENTTREE:
+        defused_mods.append(cElementTree)
+
+    for defused_mod in defused_mods:
         stdlib_mod = _apply_defusing(defused_mod)
         defused[defused_mod] = stdlib_mod
 
     return defused
 
 
-__version__ = "0.6.0"
+__version__ = "0.7.0.dev1"
 
 __all__ = [
     "DefusedXmlException",
diff --git a/defusedxml/cElementTree.py b/defusedxml/cElementTree.py
index fdc761e..95c3056 100644
--- a/defusedxml/cElementTree.py
+++ b/defusedxml/cElementTree.py
@@ -7,6 +7,11 @@
 """
 from __future__ import absolute_import
 
+from .common import _generate_etree_functions, _HAVE_CELEMENTTREE
+
+if not _HAVE_CELEMENTTREE:
+    raise ImportError("cElementTree has been removed from Python 3.9")
+
 from xml.etree.cElementTree import TreeBuilder as _TreeBuilder
 from xml.etree.cElementTree import parse as _parse
 from xml.etree.cElementTree import tostring
@@ -15,7 +20,6 @@ from xml.etree.cElementTree import tostring
 from xml.etree.ElementTree import iterparse as _iterparse
 
 from .ElementTree import DefusedXMLParser
-from .common import _generate_etree_functions
 
 __origin__ = "xml.etree.cElementTree"
 
diff --git a/defusedxml/common.py b/defusedxml/common.py
index 7e4bb0b..dd750ab 100644
--- a/defusedxml/common.py
+++ b/defusedxml/common.py
@@ -9,6 +9,8 @@ import sys
 import xml.parsers.expat
 
 PY3 = sys.version_info[0] == 3
+# Python 3.9 removed cElementTree module
+_HAVE_CELEMENTTREE = sys.version_info < (3, 9, 0)
 
 # Fail early when pyexpat is not installed correctly
 if not hasattr(xml.parsers.expat, "ParserCreate"):
diff --git a/pyproject.toml b/pyproject.toml
index 22e0c45..ef0550c 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,3 +1,4 @@
 [tool.black]
 line-length = 98
+# black does not yet support py39
 target-version = ['py27', 'py35', 'py36', 'py37', 'py38']
diff --git a/setup.py b/setup.py
index 3b4e4e2..9be51bd 100644
--- a/setup.py
+++ b/setup.py
@@ -59,6 +59,7 @@ setup(
         "Programming Language :: Python :: 3.6",
         "Programming Language :: Python :: 3.7",
         "Programming Language :: Python :: 3.8",
+        "Programming Language :: Python :: 3.9",
         "Topic :: Text Processing :: Markup :: XML",
     ],
     python_requires=">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*",
diff --git a/tests.py b/tests.py
index ddf0794..188200a 100644
--- a/tests.py
+++ b/tests.py
@@ -10,7 +10,7 @@ from xml.sax.saxutils import XMLGenerator
 from xml.sax import SAXParseException
 from pyexpat import ExpatError
 
-from defusedxml import cElementTree, ElementTree, minidom, pulldom, sax, xmlrpc, expatreader
+from defusedxml import ElementTree, minidom, pulldom, sax, xmlrpc, expatreader
 from defusedxml import defuse_stdlib
 from defusedxml import (
     DTDForbidden,
@@ -18,7 +18,13 @@ from defusedxml import (
     ExternalReferenceForbidden,
     NotSupportedError,
 )
-from defusedxml.common import PY3
+from defusedxml.common import PY3, _HAVE_CELEMENTTREE
+
+
+if _HAVE_CELEMENTTREE:
+    from defusedxml import cElementTree
+else:
+    cElementTree = None
 
 
 try:
@@ -205,6 +211,7 @@ class TestDefusedElementTree(BaseTests):
         assert self.module.XMLParse is parser
 
 
+@unittest.skipUnless(_HAVE_CELEMENTTREE, "Python 3.9 has removed cElementTree")
 class TestDefusedcElementTree(TestDefusedElementTree):
     module = cElementTree
 
diff --git a/tox.ini b/tox.ini
index 8bae237..5f5f357 100644
--- a/tox.ini
+++ b/tox.ini
@@ -1,5 +1,5 @@
 [tox]
-envlist = py27,py35,py36,py37,py38,black,pep8py2,pep8py3,doc
+envlist = py27,py35,py36,py37,py38,py39,black,pep8py2,pep8py3,doc
 skip_missing_interpreters = true
 
 [testenv]