Documented switch to MarkupSafe

--HG--
branch : trunk

7 files changed, 143 insertions, 290 deletions

diff --git a/CHANGES b/CHANGES
index 111e89d..5590be2 100644
--- a/CHANGES
+++ b/CHANGES
@@ -13,6 +13,11 @@ Version 2.5.1
 - babel extraction can now properly extract newstyle gettext calls.
 - using the variable `num` in newstyle gettext for something else
   than the pluralize count will no longer raise a :exc:`KeyError`.
+- removed builtin markup class and switched to markupsafe.  For backwards
+  compatibility the pure Python implementation still exists but is
+  pulled from markupsafe by the Jinja2 developers.  The debug support
+  went into a separate feature called "debugsupport" and is disabled
+  by default because it is only relevant for Python 2.4
 
 Version 2.5
 -----------
diff --git a/docs/faq.rst b/docs/faq.rst
index 5f49087..89186b1 100644
--- a/docs/faq.rst
+++ b/docs/faq.rst
@@ -125,23 +125,33 @@ instead that one can assign to a variable by using set::
 
     {% set comments = get_latest_comments() %}
 
-I don't have the _speedups Module.  Is Jinja slower now?
---------------------------------------------------------
+What is the speedups module and why is it missing?
+--------------------------------------------------
 
 To achieve a good performance with automatic escaping enabled, the escaping
-function is also implemented in pure C and used if Jinja2 was installed with
-the speedups module.  This happens automatically if a C compiler is available
-on the system during installation.
+function was also implemented in pure C in older Jinja2 releases and used if
+Jinja2 was installed with the speedups module.
+
+Because this feature itself is very useful for non-template engines as
+well it was moved into a separate project on PyPI called `MarkupSafe`_.
+
+Jinja2 no longer ships with a C implementation of it but only the pure
+Python implementation.  It will however check if MarkupSafe is available
+and installed, and if it is, use the Markup class from MarkupSafe.
+
+So if you want the speedups, just import MarkupSafe.
+
+.. _MarkupSafe: http://pypi.python.org/pypi/MarkupSafe
 
 My tracebacks look weird.  What's happening?
 --------------------------------------------
 
-If the speedups module is not compiled and you are using a Python installation
-without ctypes (Python 2.4 without ctypes, Jython or Google's AppEngine)
-Jinja2 is unable to provide correct debugging information and the traceback
-may be incomplete.  There is currently no good workaround for Jython or
-the AppEngine as ctypes is unavailable there and it's not possible to use
-the speedups extension.
+If the debugsupport module is not compiled and you are using a Python
+installation without ctypes (Python 2.4 without ctypes, Jython or Google's
+AppEngine) Jinja2 is unable to provide correct debugging information and
+the traceback may be incomplete.  There is currently no good workaround
+for Jython or the AppEngine as ctypes is unavailable there and it's not
+possible to use the debugsupport extension.
 
 Why is there no Python 2.3 support?
 -----------------------------------
diff --git a/docs/intro.rst b/docs/intro.rst
index 0fa2de7..35a01f6 100644
--- a/docs/intro.rst
+++ b/docs/intro.rst
@@ -33,19 +33,11 @@ Prerequisites
 -------------
 
 Jinja2 needs at least **Python 2.4** to run.  Additionally a working C-compiler
-that can create python extensions should be installed for the debugger.  If no
-C-compiler is available and you are using Python 2.4 the `ctypes`_ module
-should be installed.
+that can create python extensions should be installed for the debugger if you
+are using Python 2.4.
 
 If you don't have a working C-compiler and you are trying to install the source
-release with the speedups you will get a compiler error.  This however can be
-circumvented by passing the ``--without-speedups`` command line argument to the
-setup script::
-
-    $ python setup.py --with-speedups install
-
-(As of Jinja 2.2, the speedups are disabled by default and can be enabled
-with ``--with-speedups``.  See :ref:`enable-speedups`)
+release with the debugsupport you will get a compiler error.
 
 .. _ctypes: http://python.net/crew/theller/ctypes/
 
@@ -111,20 +103,35 @@ Or the `easy_install`_ command::
 .. _pip: http://pypi.python.org/pypi/pip
 .. _mercurial: http://www.selenic.com/mercurial/
 
-.. _enable-speedups:
 
-Enable the speedups Module
+More Speed with MarkupSafe
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-By default Jinja2 will not compile the speedups module.  Enabling this
+As of version 2.5.1 Jinja2 will check for an installed `MarkupSafe`_
+module.  If it can find it, it will use the Markup class of that module
+instead of the one that comes with Jinja2.  `MarkupSafe` replaces the
+older speedups module that came with Jinja2 and has the advantage that is
+has a better setup script and will automatically attempt to install the C
+version and nicely fall back to a pure Python implementation if that is
+not possible.
+
+The C implementation of MarkupSafe is much faster and recommended when
+using Jinja2 with autoescaping.
+
+.. _MarkupSafe: http://pypi.python.org/pypi/MarkupSafe
+
+
+Enable the debug support Module
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+By default Jinja2 will not compile the debug support module.  Enabling this
 will fail if you don't have the Python headers or a working compiler.  This
 is often the case if you are installing Jinja2 from a windows machine.
 
-You can enable the speedups extension when installing using the
-``--with-speedups`` flag::
-
-    sudo python setup.py --with-speedups install
+Because the debug support is only necessary for Python 2.4 you will not
+have to do this unless you run 2.4::
 
+    sudo python setup.py --with-debugsupport install
 
 
 Basic API Usage
diff --git a/jinja2/_debugsupport.c b/jinja2/_debugsupport.c
new file mode 100644
index 0000000..e756d8e
--- /dev/null
+++ b/jinja2/_debugsupport.c
@@ -0,0 +1,78 @@
+/**
+ * jinja2._debugsupport
+ * ~~~~~~~~~~~~~~~~~~~~
+ *
+ * C implementation of `tb_set_next`.
+ *
+ * :copyright: (c) 2010 by the Jinja Team.
+ * :license: BSD.
+ */
+
+#include <Python.h>
+
+
+static PyObject*
+tb_set_next(PyObject *self, PyObject *args)
+{
+	PyTracebackObject *tb, *old;
+	PyObject *next;
+
+	if (!PyArg_ParseTuple(args, "O!O:tb_set_next", &PyTraceBack_Type, &tb, &next))
+		return NULL;
+	if (next == Py_None)
+		next = NULL;
+	else if (!PyTraceBack_Check(next)) {
+		PyErr_SetString(PyExc_TypeError,
+				"tb_set_next arg 2 must be traceback or None");
+		return NULL;
+	}
+	else
+		Py_INCREF(next);
+
+	old = tb->tb_next;
+	tb->tb_next = (PyTracebackObject*)next;
+	Py_XDECREF(old);
+
+	Py_INCREF(Py_None);
+	return Py_None;
+}
+
+static PyMethodDef module_methods[] = {
+	{"tb_set_next", (PyCFunction)tb_set_next, METH_VARARGS,
+	 "Set the tb_next member of a traceback object."},
+	{NULL, NULL, 0, NULL}		/* Sentinel */
+};
+
+
+#if PY_MAJOR_VERSION < 3
+
+#ifndef PyMODINIT_FUNC	/* declarations for DLL import/export */
+#define PyMODINIT_FUNC void
+#endif
+PyMODINIT_FUNC
+init_debugsupport(void)
+{
+	Py_InitModule3("jinja2._debugsupport", module_methods, "");
+}
+
+#else /* Python 3.x module initialization */
+
+static struct PyModuleDef module_definition = {
+        PyModuleDef_HEAD_INIT,
+	"jinja2._debugsupport",
+	NULL,
+	-1,
+	module_methods,
+	NULL,
+	NULL,
+	NULL,
+	NULL
+};
+
+PyMODINIT_FUNC
+PyInit__debugsupport(void)
+{
+	return PyModule_Create(&module_definition);
+}
+
+#endif
diff --git a/jinja2/_speedups.c b/jinja2/_speedups.c
deleted file mode 100644
index 5c7ff2d..0000000
--- a/jinja2/_speedups.c
+++ /dev/null
@@ -1,258 +0,0 @@
-/**
- * jinja2._speedups
- * ~~~~~~~~~~~~~~~~
- *
- * This module implements functions for automatic escaping in C for better
- * performance.  Additionally it defines a `tb_set_next` function to patch the
- * debug traceback.  If the speedups module is not compiled a ctypes
- * implementation of `tb_set_next` and Python implementations of the other
- * functions are used.
- *
- * :copyright: (c) 2009 by the Jinja Team.
- * :license: BSD.
- */
-
-#include <Python.h>
-
-#define ESCAPED_CHARS_TABLE_SIZE 63
-#define UNICHR(x) (((PyUnicodeObject*)PyUnicode_DecodeASCII(x, strlen(x), NULL))->str);
-
-#if PY_VERSION_HEX < 0x02050000 && !defined(PY_SSIZE_T_MIN)
-typedef int Py_ssize_t;
-#define PY_SSIZE_T_MAX INT_MAX
-#define PY_SSIZE_T_MIN INT_MIN
-#endif
-
-
-static PyObject* markup;
-static Py_ssize_t escaped_chars_delta_len[ESCAPED_CHARS_TABLE_SIZE];
-static Py_UNICODE *escaped_chars_repl[ESCAPED_CHARS_TABLE_SIZE];
-
-static int
-init_constants(void)
-{
-	PyObject *module;
-	/* happing of characters to replace */
-	escaped_chars_repl['"'] = UNICHR("&#34;");
-	escaped_chars_repl['\''] = UNICHR("&#39;");
-	escaped_chars_repl['&'] = UNICHR("&amp;");
-	escaped_chars_repl['<'] = UNICHR("&lt;");
-	escaped_chars_repl['>'] = UNICHR("&gt;");
-
-	/* lengths of those characters when replaced - 1 */
-	memset(escaped_chars_delta_len, 0, sizeof (escaped_chars_delta_len));
-	escaped_chars_delta_len['"'] = escaped_chars_delta_len['\''] = \
-		escaped_chars_delta_len['&'] = 4;
-	escaped_chars_delta_len['<'] = escaped_chars_delta_len['>'] = 3;
-	
-	/* import markup type so that we can mark the return value */
-	module = PyImport_ImportModule("jinja2.utils");
-	if (!module)
-		return 0;
-	markup = PyObject_GetAttrString(module, "Markup");
-	Py_DECREF(module);
-
-	return 1;
-}
-
-static PyObject*
-escape_unicode(PyUnicodeObject *in)
-{
-	PyUnicodeObject *out;
-	Py_UNICODE *inp = in->str;
-	const Py_UNICODE *inp_end = in->str + in->length;
-	Py_UNICODE *next_escp;
-	Py_UNICODE *outp;
-	Py_ssize_t delta=0, erepl=0, delta_len=0;
-
-	/* First we need to figure out how long the escaped string will be */
-	while (*(inp) || inp < inp_end) {
-		if (*inp < ESCAPED_CHARS_TABLE_SIZE && escaped_chars_delta_len[*inp]) {
-			delta += escaped_chars_delta_len[*inp];
-			++erepl;
-		}
-		++inp;
-	}
-
-	/* Do we need to escape anything at all? */
-	if (!erepl) {
-		Py_INCREF(in);
-		return (PyObject*)in;
-	}
-
-	out = (PyUnicodeObject*)PyUnicode_FromUnicode(NULL, in->length + delta);
-	if (!out)
-		return NULL;
-
-	outp = out->str;
-	inp = in->str;
-	while (erepl-- > 0) {
-		/* look for the next substitution */
-		next_escp = inp;
-		while (next_escp < inp_end) {
-			if (*next_escp < ESCAPED_CHARS_TABLE_SIZE &&
-			    (delta_len = escaped_chars_delta_len[*next_escp])) {
-				++delta_len;
-				break;
-			}
-			++next_escp;
-		}
-		
-		if (next_escp > inp) {
-			/* copy unescaped chars between inp and next_escp */
-			Py_UNICODE_COPY(outp, inp, next_escp-inp);
-			outp += next_escp - inp;
-		}
-
-		/* escape 'next_escp' */
-		Py_UNICODE_COPY(outp, escaped_chars_repl[*next_escp], delta_len);
-		outp += delta_len;
-
-		inp = next_escp + 1;
-	}
-	if (inp < inp_end)
-		Py_UNICODE_COPY(outp, inp, in->length - (inp - in->str));
-
-	return (PyObject*)out;
-}
-
-
-static PyObject*
-escape(PyObject *self, PyObject *text)
-{
-	PyObject *s = NULL, *rv = NULL, *html;
-
-	/* we don't have to escape integers, bools or floats */
-	if (PyLong_CheckExact(text) ||
-#if PY_MAJOR_VERSION < 3
-	    PyInt_CheckExact(text) ||
-#endif
-	    PyFloat_CheckExact(text) || PyBool_Check(text) ||
-	    text == Py_None)
-		return PyObject_CallFunctionObjArgs(markup, text, NULL);
-
-	/* if the object has an __html__ method that performs the escaping */
-	html = PyObject_GetAttrString(text, "__html__");
-	if (html) {
-		rv = PyObject_CallObject(html, NULL);
-		Py_DECREF(html);
-		return rv;
-	}
-
-	/* otherwise make the object unicode if it isn't, then escape */
-	PyErr_Clear();
-	if (!PyUnicode_Check(text)) {
-#if PY_MAJOR_VERSION < 3
-		PyObject *unicode = PyObject_Unicode(text);
-#else
-		PyObject *unicode = PyObject_Str(text);
-#endif
-		if (!unicode)
-			return NULL;
-		s = escape_unicode((PyUnicodeObject*)unicode);
-		Py_DECREF(unicode);
-	}
-	else
-		s = escape_unicode((PyUnicodeObject*)text);
-
-	/* convert the unicode string into a markup object. */
-	rv = PyObject_CallFunctionObjArgs(markup, (PyObject*)s, NULL);
-	Py_DECREF(s);
-	return rv;
-}
-
-
-static PyObject*
-soft_unicode(PyObject *self, PyObject *s)
-{
-	if (!PyUnicode_Check(s))
-#if PY_MAJOR_VERSION < 3
-		return PyObject_Unicode(s);
-#else
-		return PyObject_Str(s);
-#endif
-	Py_INCREF(s);
-	return s;
-}
-
-
-static PyObject*
-tb_set_next(PyObject *self, PyObject *args)
-{
-	PyTracebackObject *tb, *old;
-	PyObject *next;
-
-	if (!PyArg_ParseTuple(args, "O!O:tb_set_next", &PyTraceBack_Type, &tb, &next))
-		return NULL;
-	if (next == Py_None)
-		next = NULL;
-	else if (!PyTraceBack_Check(next)) {
-		PyErr_SetString(PyExc_TypeError,
-				"tb_set_next arg 2 must be traceback or None");
-		return NULL;
-	}
-	else
-		Py_INCREF(next);
-
-	old = tb->tb_next;
-	tb->tb_next = (PyTracebackObject*)next;
-	Py_XDECREF(old);
-
-	Py_INCREF(Py_None);
-	return Py_None;
-}
-
-static PyMethodDef module_methods[] = {
-	{"escape", (PyCFunction)escape, METH_O,
-	 "escape(s) -> markup\n\n"
-	 "Convert the characters &, <, >, ', and \" in string s to HTML-safe\n"
-	 "sequences.  Use this if you need to display text that might contain\n"
-	 "such characters in HTML.  Marks return value as markup string."},
-	{"soft_unicode", (PyCFunction)soft_unicode, METH_O,
-	 "soft_unicode(object) -> string\n\n"
-         "Make a string unicode if it isn't already.  That way a markup\n"
-         "string is not converted back to unicode."},
-	{"tb_set_next", (PyCFunction)tb_set_next, METH_VARARGS,
-	 "Set the tb_next member of a traceback object."},
-	{NULL, NULL, 0, NULL}		/* Sentinel */
-};
-
-
-#if PY_MAJOR_VERSION < 3
-
-#ifndef PyMODINIT_FUNC	/* declarations for DLL import/export */
-#define PyMODINIT_FUNC void
-#endif
-PyMODINIT_FUNC
-init_speedups(void)
-{
-	if (!init_constants())
-		return;
-
-	Py_InitModule3("jinja2._speedups", module_methods, "");
-}
-
-#else /* Python 3.x module initialization */
-
-static struct PyModuleDef module_definition = {
-        PyModuleDef_HEAD_INIT,
-	"jinja2._speedups",
-	NULL,
-	-1,
-	module_methods,
-	NULL,
-	NULL,
-	NULL,
-	NULL
-};
-
-PyMODINIT_FUNC
-PyInit__speedups(void)
-{
-	if (!init_constants())
-		return NULL;
-
-	return PyModule_Create(&module_definition);
-}
-
-#endif
diff --git a/jinja2/debug.py b/jinja2/debug.py
index c2bd07b..35ae24f 100644
--- a/jinja2/debug.py
+++ b/jinja2/debug.py
@@ -60,7 +60,7 @@ class ProcessedTraceback(object):
         self.frames = frames
 
     def chain_frames(self):
-        """Chains the frames.  Requires ctypes or the speedups extension."""
+        """Chains the frames.  Requires ctypes or the debugsupport extension."""
         prev_tb = None
         for tb in self.frames:
             if prev_tb is not None:
@@ -299,7 +299,7 @@ def _init_ugly_crap():
 
 # try to get a tb_set_next implementation
 try:
-    from jinja2._speedups import tb_set_next
+    from jinja2._debugsupport import tb_set_next
 except ImportError:
     try:
         tb_set_next = _init_ugly_crap()
diff --git a/setup.py b/setup.py
index cee83cc..f3b80b6 100644
--- a/setup.py
+++ b/setup.py
@@ -42,7 +42,17 @@ Jinja2==dev``.
 import os
 import sys
 
-from setuptools import setup
+from setuptools import setup, Extension, Feature
+from distutils.command.build_ext import build_ext
+
+debugsupport = Feature(
+    'optional C debug support',
+    standard=False,
+    ext_modules = [
+        Extension('jinja2._debugsupport', ['jinja2/_debugsupport.c']),
+    ],
+)
+
 
 # tell distribute to use 2to3 with our own fixers.
 extra = {}
@@ -86,5 +96,6 @@ setup(
     [babel.extractors]
     jinja2 = jinja2.ext:babel_extract[i18n]
     """,
+    features={'debugsupport': debugsupport},
     **extra
 )