clean up faq, move technical discussions

3 files changed, 82 insertions, 142 deletions

diff --git a/docs/api.rst b/docs/api.rst
index 8b80749..b2db537 100644
--- a/docs/api.rst
+++ b/docs/api.rst
@@ -410,16 +410,19 @@ The Context
     .. automethod:: jinja2.runtime.Context.call(callable, \*args, \**kwargs)
 
 
-.. admonition:: Implementation
+The context is immutable, it prevents modifications, and if it is
+modified somehow despite that those changes may not show up. For
+performance, Jinja does not use the context as data storage for, only as
+a primary data source. Variables that the template does not define are
+looked up in the context, but variables the template does define are
+stored locally.
+
+Instead of modifying the context directly, a function should return
+a value that can be assigned to a variable within the template itself.
 
-    Context is immutable for the same reason Python's frame locals are
-    immutable inside functions.  Both Jinja and Python are not using the
-    context / frame locals as data storage for variables but only as primary
-    data source.
+.. code-block:: jinja
 
-    When a template accesses a variable the template does not define, Jinja
-    looks up the variable in the context, after that the variable is treated
-    as if it was defined in the template.
+    {% set comments = get_latest_comments() %}
 
 
 .. _loaders:
diff --git a/docs/faq.rst b/docs/faq.rst
index ca1f9aa..493dc38 100644
--- a/docs/faq.rst
+++ b/docs/faq.rst
@@ -1,155 +1,75 @@
 Frequently Asked Questions
 ==========================
 
-This page answers some of the often asked questions about Jinja.
-
-.. highlight:: html+jinja
 
 Why is it called Jinja?
 -----------------------
 
-The name Jinja was chosen because it's the name of a Japanese temple and
-temple and template share a similar pronunciation.  It is not named after
-the city in Uganda.
-
-How fast is it?
----------------
-
-We really hate benchmarks especially since they don't reflect much.  The
-performance of a template depends on many factors and you would have to
-benchmark different engines in different situations.  The benchmarks from the
-testsuite show that Jinja has a similar performance to `Mako`_ and is between
-10 and 20 times faster than Django's template engine or Genshi.  These numbers
-should be taken with tons of salt as the benchmarks that took these numbers
-only test a few performance related situations such as looping.  Generally
-speaking the performance of a template engine doesn't matter much as the
-usual bottleneck in a web application is either the database or the application
-code.
-
-.. _Mako: https://www.makotemplates.org/
-
-How Compatible is Jinja with Django?
-------------------------------------
-
-The default syntax of Jinja matches Django syntax in many ways.  However
-this similarity doesn't mean that you can use a Django template unmodified
-in Jinja.  For example filter arguments use a function call syntax rather
-than a colon to separate filter name and arguments.  Additionally the
-extension interface in Jinja is fundamentally different from the Django one
-which means that your custom tags won't work any longer.
+"Jinja" is a Japanese `Shinto shrine`_, or temple, and temple and
+template share a similar English pronunciation. It is not named after
+the `city in Uganda`_.
 
-Generally speaking you will use much less custom extensions as the Jinja
-template system allows you to use a certain subset of Python expressions
-which can replace most Django extensions.  For example instead of using
-something like this::
+.. _Shinto shrine: https://en.wikipedia.org/wiki/Shinto_shrine
+.. _city in Uganda: https://en.wikipedia.org/wiki/Jinja%2C_Uganda
 
-    {% load comments %}
-    {% get_latest_comments 10 as latest_comments %}
-    {% for comment in latest_comments %}
-        ...
-    {% endfor %}
 
-You will most likely provide an object with attributes to retrieve
-comments from the database::
+How fast is Jinja?
+------------------
 
-    {% for comment in models.comments.latest(10) %}
-        ...
-    {% endfor %}
+Jinja is relatively fast among template engines because it compiles and
+caches template code to Python code, so that the template does not need
+to be parsed and interpreted each time. Rendering a template becomes as
+close to executing a Python function as possible.
 
-Or directly provide the model for quick testing::
+Jinja also makes extensive use of caching. Templates are cached by name
+after loading, so future uses of the template avoid loading. The
+template loading itself uses a bytecode cache to avoid repeated
+compiling. The caches can be external to persist across restarts.
+Templates can also be precompiled and loaded as fast Python imports.
 
-    {% for comment in Comment.objects.order_by('-pub_date')[:10] %}
-        ...
-    {% endfor %}
+We dislike benchmarks because they don't reflect real use. Performance
+depends on many factors. Different engines have different default
+configurations and tradeoffs that make it unclear how to set up a useful
+comparison. Often, database access, API calls, and data processing have
+a much larger effect on performance than the template engine.
 
-Please keep in mind that even though you may put such things into templates
-it still isn't a good idea.  Queries should go into the view code and not
-the template!
 
-Isn't it a terrible idea to put Logic into Templates?
------------------------------------------------------
+Isn't it a bad idea to put logic in templates?
+----------------------------------------------
 
 Without a doubt you should try to remove as much logic from templates as
-possible.  But templates without any logic mean that you have to do all
-the processing in the code which is boring and stupid.  A template engine
-that does that is shipped with Python and called `string.Template`.  Comes
-without loops and if conditions and is by far the fastest template engine
-you can get for Python.
-
-So some amount of logic is required in templates to keep everyone happy.
-And Jinja leaves it pretty much to you how much logic you want to put into
-templates.  There are some restrictions in what you can do and what not.
-
-Jinja neither allows you to put arbitrary Python code into templates nor
-does it allow all Python expressions.  The operators are limited to the
-most common ones and more advanced expressions such as list comprehensions
-and generator expressions are not supported.  This keeps the template engine
-easier to maintain and templates more readable.
-
-Why is Autoescaping not the Default?
-------------------------------------
-
-There are multiple reasons why automatic escaping is not the default mode
-and also not the recommended one.  While automatic escaping of variables
-means that you will less likely have an XSS problem it also causes a huge
-amount of extra processing in the template engine which can cause serious
-performance problems.  As Python doesn't provide a way to mark strings as
-unsafe Jinja has to hack around that limitation by providing a custom
-string class (the :class:`~markupsafe.Markup` string) that safely interacts with safe
-and unsafe strings.
-
-With explicit escaping however the template engine doesn't have to perform
-any safety checks on variables.  Also a human knows not to escape integers
-or strings that may never contain characters one has to escape or already
-HTML markup.  For example when iterating over a list over a table of
-integers and floats for a table of statistics the template designer can
-omit the escaping because he knows that integers or floats don't contain
-any unsafe parameters.
-
-Additionally Jinja is a general purpose template engine and not only used
-for HTML/XML generation.  For example you may generate LaTeX, emails,
-CSS, JavaScript, or configuration files.
-
-Why is the Context immutable?
------------------------------
-
-When writing a :func:`pass_context` function, you may have noticed that
-the context tries to stop you from modifying it. If you have managed to
-modify the context by using an internal context API you may have noticed
-that changes in the context don't seem to be visible in the template.
-The reason for this is that Jinja uses the context only as primary data
-source for template variables for performance reasons.
-
-If you want to modify the context write a function that returns a variable
-instead that one can assign to a variable by using set::
-
-    {% set comments = get_latest_comments() %}
-
-
-My Macros are overridden by something
+possible. With less logic, the template is easier to understand, has
+fewer potential side effects, and is faster to compile and render. But a
+template without any logic means processing must be done in code before
+rendering. A template engine that does that is shipped with Python,
+called :class:`string.Template`, and while it's definitely fast it's not
+convenient.
+
+Jinja's features such as blocks, statements, filters, and function calls
+make it much easier to write expressive templates, with very few
+restrictions. Jinja doesn't allow arbitrary Python code in templates, or
+every feature available in the Python language. This keeps the engine
+easier to maintain, and keeps templates more readable.
+
+Some amount of logic is required in templates to keep everyone happy.
+Too much logic in the template can make it complex to reason about and
+maintain. It's up to you to decide how your application will work and
+balance how much logic you want to put in the template.
+
+
+Why is HTML escaping not the default?
 -------------------------------------
 
-In some situations the Jinja scoping appears arbitrary:
-
-layout.tmpl:
-
-.. sourcecode:: jinja
-
-    {% macro foo() %}LAYOUT{% endmacro %}
-    {% block body %}{% endblock %}
-
-child.tmpl:
-
-.. sourcecode:: jinja
-
-    {% extends 'layout.tmpl' %}
-    {% macro foo() %}CHILD{% endmacro %}
-    {% block body %}{{ foo() }}{% endblock %}
+Jinja provides a feature that can be enabled to escape HTML syntax in
+rendered templates. However, it is disabled by default.
 
-This will print ``LAYOUT`` in Jinja.  This is a side effect of having
-the parent template evaluated after the child one.  This allows child
-templates passing information to the parent template.  To avoid this
-issue rename the macro or variable in the parent template to have an
-uncommon prefix.
+Jinja is a general purpose template engine, it is not only used for HTML
+documents. You can generate plain text, LaTeX, emails, CSS, JavaScript,
+configuration files, etc. HTML escaping wouldn't make sense for any of
+these document types.
 
-.. _Jinja 1: https://pypi.org/project/Jinja/
+While automatic escaping means that you are less likely have an XSS
+problem, it also requires significant extra processing during compiling
+and rendering, which can reduce performance. Jinja uses MarkupSafe for
+escaping, which provides optimized C code for speed, but it still
+introduces overhead to track escaping across methods and formatting.
diff --git a/docs/templates.rst b/docs/templates.rst
index 9071ad6..0135ee4 100644
--- a/docs/templates.rst
+++ b/docs/templates.rst
@@ -938,6 +938,23 @@ are available on a macro object:
 If a macro name starts with an underscore, it's not exported and can't
 be imported.
 
+Due to how scopes work in Jinja, a macro in a child template does not
+override a macro in a parent template. The following will output
+"LAYOUT", not "CHILD".
+
+.. code-block:: jinja
+    :caption: ``layout.txt``
+
+    {% macro foo() %}LAYOUT{% endmacro %}
+    {% block body %}{% endblock %}
+
+.. code-block:: jinja
+    :caption: ``child.txt``
+
+    {% extends 'layout.txt' %}
+    {% macro foo() %}CHILD{% endmacro %}
+    {% block body %}{{ foo() }}{% endblock %}
+
 
 .. _call: