Disable "check unicode returns" under Python 3

Disabled the "unicode returns" check that runs on dialect startup when
running under Python 3, which for many years has occurred in order to test
the current DBAPI's behavior for whether or not it returns Python Unicode
or Py2K strings for the VARCHAR and NVARCHAR datatypes.  The check still
occurs by default under Python 2, however the mechanism to test the
behavior will be removed in SQLAlchemy 2.0 when Python 2 support is also
removed.

This logic was very effective when it was needed, however now that Python 3
is standard, all DBAPIs are expected to return Python 3 strings for
character datatypes.  In the unlikely case that a third party DBAPI does
not support this, the conversion logic within :class:`.String` is still
available and the third party dialect may specify this in its upfront
dialect flags by setting the dialect level flag ``returns_unicode_strings``
to one of :attr:`.String.RETURNS_CONDITIONAL` or
:attr:`.String.RETURNS_BYTES`, both of which will enable Unicode conversion
even under Python 3.

As part of this change, disabling testing of the doctest tutorials under
Python 2.

Fixes: #5315
Change-Id: I1260e894611409d3b7fe1a92bd90c52043bbcf19

1 files changed, 45 insertions, 0 deletions

diff --git a/test/sql/test_types.py b/test/sql/test_types.py
index e0c1359b4..a0c4ee022 100644
--- a/test/sql/test_types.py
+++ b/test/sql/test_types.py
@@ -86,6 +86,7 @@ from sqlalchemy.testing.schema import Table
 from sqlalchemy.testing.util import picklers
 from sqlalchemy.testing.util import round_decimal
 from sqlalchemy.util import OrderedDict
+from sqlalchemy.util import u
 
 
 def _all_dialect_modules():
@@ -763,6 +764,50 @@ class UserDefinedTest(
         eq_(a.dialect_specific_args["bar"], "bar")
 
 
+class StringConvertUnicodeTest(fixtures.TestBase):
+    @testing.combinations((Unicode,), (String,), argnames="datatype")
+    @testing.combinations((True,), (False,), argnames="convert_unicode")
+    @testing.combinations(
+        (String.RETURNS_CONDITIONAL,),
+        (String.RETURNS_BYTES,),
+        (String.RETURNS_UNICODE),
+        argnames="returns_unicode_strings",
+    )
+    def test_convert_unicode(
+        self, datatype, convert_unicode, returns_unicode_strings
+    ):
+        s1 = datatype()
+        dialect = mock.Mock(
+            returns_unicode_strings=returns_unicode_strings,
+            encoding="utf-8",
+            convert_unicode=convert_unicode,
+        )
+
+        proc = s1.result_processor(dialect, None)
+
+        string = u("méil")
+        bytestring = string.encode("utf-8")
+
+        if (
+            datatype is Unicode or convert_unicode
+        ) and returns_unicode_strings in (
+            String.RETURNS_CONDITIONAL,
+            String.RETURNS_BYTES,
+        ):
+            eq_(proc(bytestring), string)
+
+            if returns_unicode_strings is String.RETURNS_CONDITIONAL:
+                eq_(proc(string), string)
+            else:
+                if util.py3k:
+                    # trying to decode a unicode
+                    assert_raises(TypeError, proc, string)
+                else:
+                    assert_raises(UnicodeEncodeError, proc, string)
+        else:
+            is_(proc, None)
+
+
 class TypeCoerceCastTest(fixtures.TablesTest):
     __backend__ = True