Marc-Andre's third try at this bulk patch seems to work (except that

his copy of test_contains.py seems to be broken -- the lines he
deleted were already absent).  Checkin messages:


New Unicode support for int(), float(), complex() and long().

- new APIs PyInt_FromUnicode() and PyLong_FromUnicode()
- added support for Unicode to PyFloat_FromString()
- new encoding API PyUnicode_EncodeDecimal() which converts
  Unicode to a decimal char* string (used in the above new
  APIs)
- shortcuts for calls like int(<int object>) and float(<float obj>)
- tests for all of the above

Unicode compares and contains checks:
- comparing Unicode and non-string types now works; TypeErrors
  are masked, all other errors such as ValueError during
  Unicode coercion are passed through (note that PyUnicode_Compare
  does not implement the masking -- PyObject_Compare does this)
- contains now works for non-string types too; TypeErrors are
  masked and 0 returned; all other errors are passed through

Better testing support for the standard codecs.

Misc minor enhancements, such as an alias dbcs for the mbcs codec.

Changes:
- PyLong_FromString() now applies the same error checks as
  does PyInt_FromString(): trailing garbage is reported
  as error and not longer silently ignored. The only characters
  which may be trailing the digits are 'L' and 'l' -- these
  are still silently ignored.
- string.ato?() now directly interface to int(), long() and
  float(). The error strings are now a little different, but
  the type still remains the same. These functions are now
  ready to get declared obsolete ;-)
- PyNumber_Int() now also does a check for embedded NULL chars
  in the input string; PyNumber_Long() already did this (and
  still does)

Followed by:

Looks like I've gone a step too far there... (and test_contains.py
seem to have a bug too).

I've changed back to reporting all errors in PyUnicode_Contains()
and added a few more test cases to test_contains.py (plus corrected
the join() NameError).

1 files changed, 66 insertions, 4 deletions

diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c
index 9c35e2d544..e751bc4334 100644
--- a/Objects/unicodeobject.c
+++ b/Objects/unicodeobject.c
@@ -329,8 +329,14 @@ PyObject *PyUnicode_FromObject(register PyObject *obj)
 	s = PyString_AS_STRING(obj);
 	len = PyString_GET_SIZE(obj);
     }
-    else if (PyObject_AsCharBuffer(obj, &s, &len))
+    else if (PyObject_AsCharBuffer(obj, &s, &len)) {
+	/* Overwrite the error message with something more useful in
+	   case of a TypeError. */
+	if (PyErr_ExceptionMatches(PyExc_TypeError))
+	    PyErr_SetString(PyExc_TypeError,
+			    "coercing to Unicode: need string or charbuffer");
 	return NULL;
+    }
     if (len == 0) {
 	Py_INCREF(unicode_empty);
 	return (PyObject *)unicode_empty;
@@ -1923,6 +1929,60 @@ PyObject *PyUnicode_Translate(PyObject *str,
     return NULL;
 }
     
+/* --- Decimal Encoder ---------------------------------------------------- */
+
+int PyUnicode_EncodeDecimal(Py_UNICODE *s,
+			    int length,
+			    char *output,
+			    const char *errors)
+{
+    Py_UNICODE *p, *end;
+
+    if (output == NULL) {
+	PyErr_BadArgument();
+	return -1;
+    }
+
+    p = s;
+    end = s + length;
+    while (p < end) {
+	register Py_UNICODE ch = *p++;
+	int decimal;
+	
+	if (Py_UNICODE_ISSPACE(ch)) {
+	    *output++ = ' ';
+	    continue;
+	}
+	decimal = Py_UNICODE_TODECIMAL(ch);
+	if (decimal >= 0) {
+	    *output++ = '0' + decimal;
+	    continue;
+	}
+	if (0 < ch < 256) {
+	    *output++ = ch;
+	    continue;
+	}
+	/* All other characters are considered invalid */
+	if (errors == NULL || strcmp(errors, "strict") == 0) {
+	    PyErr_SetString(PyExc_ValueError,
+			    "invalid decimal Unicode string");
+	    goto onError;
+	}
+	else if (strcmp(errors, "ignore") == 0)
+	    continue;
+	else if (strcmp(errors, "replace") == 0) {
+	    *output++ = '?';
+	    continue;
+	}
+    }
+    /* 0-terminate the output string */
+    *output++ = '\0';
+    return 0;
+
+ onError:
+    return -1;
+}
+
 /* --- Helpers ------------------------------------------------------------ */
 
 static 
@@ -2811,12 +2871,14 @@ int PyUnicode_Contains(PyObject *container,
     register Py_UNICODE ch;
 
     /* Coerce the two arguments */
-    u = (PyUnicodeObject *)PyUnicode_FromObject(container);
-    if (u == NULL)
-	goto onError;
     v = (PyUnicodeObject *)PyUnicode_FromObject(element);
     if (v == NULL)
 	goto onError;
+    u = (PyUnicodeObject *)PyUnicode_FromObject(container);
+    if (u == NULL) {
+	Py_DECREF(v);
+	goto onError;
+    }
 
     /* Check v in u */
     if (PyUnicode_GET_SIZE(v) != 1) {