Doc: move API docs in their own chapter and add more of it.

10 files changed, 302 insertions, 72 deletions

diff --git a/doc/contents.rst b/doc/contents.rst
index 5e0ae6c1..d3fd3c8e 100644
--- a/doc/contents.rst
+++ b/doc/contents.rst
@@ -18,6 +18,7 @@ Sphinx documentation contents
    theming
    templating
    extensions
+   extdev/index
    websupport
 
    faq
diff --git a/doc/ext/appapi.rst b/doc/extdev/appapi.rst
index a6393d40..b91f6c72 100644
--- a/doc/ext/appapi.rst
+++ b/doc/extdev/appapi.rst
@@ -1,14 +1,27 @@
 .. highlight:: rest
 
-Extension API
-=============
+Application API
+===============
+
+.. module:: sphinx.application
+   :synopsis: Application class and extensibility interface.
 
-.. currentmodule:: sphinx.application
 
 Each Sphinx extension is a Python module with at least a :func:`setup` function.
 This function is called at initialization time with one argument, the
-application object representing the Sphinx process.  This application object has
-the following public API:
+application object representing the Sphinx process.
+
+.. class:: Sphinx
+
+   This application object has the public API described in the following.
+
+Extension setup
+---------------
+
+These methods are usually called in an extension's ``setup()`` function.
+
+Examples of using the Sphinx extension API can be seen in the :mod:`sphinx.ext`
+package.
 
 .. method:: Sphinx.setup_extension(name)
 
@@ -293,6 +306,14 @@ the following public API:
 
    .. versionadded:: 1.1
 
+.. method:: Sphinx.require_sphinx(version)
+
+   Compare *version* (which must be a ``major.minor`` version string,
+   e.g. ``'1.1'``) with the version of the running Sphinx, and abort the build
+   when it is too old.
+
+   .. versionadded:: 1.0
+
 .. method:: Sphinx.connect(event, callback)
 
    Register *callback* to be called when *event* is emitted.  For details on
@@ -306,6 +327,16 @@ the following public API:
 
    Unregister callback *listener_id*.
 
+
+.. exception:: ExtensionError
+
+   All these methods raise this exception if something went wrong with the
+   extension API.
+
+
+Emitting events
+---------------
+
 .. method:: Sphinx.emit(event, *arguments)
 
    Emit *event* and pass *arguments* to the callback functions.  Return the
@@ -319,22 +350,21 @@ the following public API:
 
    .. versionadded:: 0.5
 
-.. method:: Sphinx.require_sphinx(version)
 
-   Compare *version* (which must be a ``major.minor`` version string,
-   e.g. ``'1.1'``) with the version of the running Sphinx, and abort the build
-   when it is too old.
+Producing messages / logging
+----------------------------
 
-   .. versionadded:: 1.0
+The application object also provides support for emitting leveled messages.
 
+.. automethod:: Sphinx.warn
 
-.. exception:: ExtensionError
+.. automethod:: Sphinx.info
 
-   All these functions raise this exception if something went wrong with the
-   extension API.
+.. automethod:: Sphinx.verbose
 
-Examples of using the Sphinx extension API can be seen in the :mod:`sphinx.ext`
-package.
+.. automethod:: Sphinx.debug
+
+.. automethod:: Sphinx.debug2
 
 
 .. _events:
@@ -478,28 +508,27 @@ Use this to adapt your extension to API changes in Sphinx.
       Before version 1.2, check the string ``sphinx.__version__``.
 
 
-.. _template-bridge:
+The Config object
+-----------------
 
-The template bridge
--------------------
-
-.. currentmodule:: sphinx.application
+.. module:: sphinx.config
 
-.. autoclass:: TemplateBridge
-   :members:
+.. class:: Config
 
+   The config object makes the values of all config values available as
+   attributes.
 
-.. _domain-api:
+   It is available as the ``config`` attribute on the application and
+   environment objects.  For example, to get the value of :confval:`language`,
+   use either ``app.config.language`` or ``env.config.language``.
 
-Domain API
-----------
 
-.. module:: sphinx.domains
+.. _template-bridge:
 
-.. autoclass:: Domain
-   :members:
+The template bridge
+-------------------
 
-.. autoclass:: ObjType
+.. currentmodule:: sphinx.application
 
-.. autoclass:: Index
+.. autoclass:: TemplateBridge
    :members:
diff --git a/doc/ext/builderapi.rst b/doc/extdev/builderapi.rst
index 3ace2687..cd8688a3 100644
--- a/doc/ext/builderapi.rst
+++ b/doc/extdev/builderapi.rst
@@ -1,7 +1,7 @@
 .. _writing-builders:
 
-Writing new builders
-====================
+Builder API
+===========
 
 .. todo:: Expand this.
 
diff --git a/doc/extdev/domainapi.rst b/doc/extdev/domainapi.rst
new file mode 100644
index 00000000..d6ecf063
--- /dev/null
+++ b/doc/extdev/domainapi.rst
@@ -0,0 +1,14 @@
+.. _domain-api:
+
+Domain API
+----------
+
+.. module:: sphinx.domains
+
+.. autoclass:: Domain
+   :members:
+
+.. autoclass:: ObjType
+
+.. autoclass:: Index
+   :members:
diff --git a/doc/extdev/envapi.rst b/doc/extdev/envapi.rst
new file mode 100644
index 00000000..56771f73
--- /dev/null
+++ b/doc/extdev/envapi.rst
@@ -0,0 +1,54 @@
+Build environment API
+=====================
+
+.. module:: sphinx.environment
+
+.. class:: BuildEnvironment
+
+   **Attributes**
+
+   .. attribute:: app
+
+      Reference to the application object.
+
+   .. attribute:: config
+
+      Reference to the :class:`.Config` object.
+
+   .. attribute:: srcdir
+
+      Source directory (the directory containing ``conf.py``).
+
+   .. attribute:: doctreedir
+
+      Directory for storing pickled doctrees.
+
+   .. attribute:: found_docs
+
+      A set of all existing docnames.
+
+   .. attribute:: metadata
+
+      Dictionary mapping docnames to "metadata" (see :ref:`metadata`).
+
+   .. attribute:: titles
+
+      Dictionary mapping docnames to the docutils node for their main title.
+
+   .. autoattribute:: docname
+
+   **Utility methods**
+
+   .. automethod:: warn
+
+   .. automethod:: warn_node
+
+   .. automethod:: doc2path
+
+   .. automethod:: relfn2path
+
+   .. automethod:: note_dependency
+
+   .. automethod:: new_serialno
+
+   .. automethod:: note_reread
diff --git a/doc/extdev/index.rst b/doc/extdev/index.rst
new file mode 100644
index 00000000..b76928c0
--- /dev/null
+++ b/doc/extdev/index.rst
@@ -0,0 +1,33 @@
+.. _dev-extensions:
+
+Developing extensions for Sphinx
+================================
+
+Since many projects will need special features in their documentation, Sphinx is
+designed to be extensible on several levels.
+
+This is what you can do in an extension: First, you can add new
+:term:`builder`\s to support new output formats or actions on the parsed
+documents.  Then, it is possible to register custom reStructuredText roles and
+directives, extending the markup.  And finally, there are so-called "hook
+points" at strategic places throughout the build process, where an extension can
+register a hook and run specialized code.
+
+An extension is simply a Python module.  When an extension is loaded, Sphinx
+imports this module and executes its ``setup()`` function, which in turn
+notifies Sphinx of everything the extension offers -- see the extension tutorial
+for examples.
+
+The configuration file itself can be treated as an extension if it contains a
+``setup()`` function.  All other extensions to load must be listed in the
+:confval:`extensions` configuration value.
+
+.. toctree::
+
+   tutorial
+   appapi
+   envapi
+   builderapi
+   markupapi
+   domainapi
+   nodes
diff --git a/doc/extdev/markupapi.rst b/doc/extdev/markupapi.rst
new file mode 100644
index 00000000..8f6c29b1
--- /dev/null
+++ b/doc/extdev/markupapi.rst
@@ -0,0 +1,11 @@
+Docutils markup API
+===================
+
+Roles
+-----
+
+
+Directives
+----------
+
+TODO.
diff --git a/doc/extdev/nodes.rst b/doc/extdev/nodes.rst
new file mode 100644
index 00000000..e67fa3da
--- /dev/null
+++ b/doc/extdev/nodes.rst
@@ -0,0 +1,57 @@
+.. _nodes:
+
+Doctree node classes added by Sphinx
+====================================
+
+.. module:: sphinx.addnodes
+
+Nodes for domain-specific object descriptions
+---------------------------------------------
+
+.. autoclass:: desc
+.. autoclass:: desc_signature
+.. autoclass:: desc_addname
+.. autoclass:: desc_type
+.. autoclass:: desc_returns
+.. autoclass:: desc_name
+.. autoclass:: desc_parameterlist
+.. autoclass:: desc_parameter
+.. autoclass:: desc_optional
+.. autoclass:: desc_annotation
+.. autoclass:: desc_content
+
+New admonition-like constructs
+------------------------------
+
+.. autoclass:: versionmodified
+.. autoclass:: seealso
+
+Other paragraph-level nodes
+-------------------------------
+
+.. autoclass:: compact_paragraph
+
+New inline nodes
+----------------
+
+.. autoclass:: index
+.. autoclass:: pending_xref
+.. autoclass:: literal_emphasis
+.. autoclass:: abbreviation
+.. autoclass:: download_reference
+
+Special nodes
+-------------
+
+.. autoclass:: only
+.. autoclass:: meta
+.. autoclass:: highlightlang
+
+You should not need to generate the nodes below in extensions.
+
+.. autoclass:: glossary
+.. autoclass:: toctree
+.. autoclass:: start_of_file
+.. autoclass:: productionlist
+.. autoclass:: production
+.. autoclass:: termsep
diff --git a/doc/ext/tutorial.rst b/doc/extdev/tutorial.rst
index c09be29a..38264190 100644
--- a/doc/ext/tutorial.rst
+++ b/doc/extdev/tutorial.rst
@@ -12,6 +12,50 @@ include todo entries in the documentation, and collecting these in a central
 place.  (A similar "todo" extension is distributed with Sphinx.)
 
 
+Important objects
+-----------------
+
+There are several key objects whose API you will use while writing an
+extension.  These are:
+
+**Application**
+   The application object (usually called ``app``) is an instance of
+   :class:`.Sphinx`.  It controls the most high-level functionality, such as the
+   setup of extensions, event dispatching and producing output (logging).
+
+   If you have the environment object, the application is available as
+   ``env.app``.
+
+**Environment**
+   The build environment object (usually called ``env``) is an instance of
+   :class:`.BuildEnvironment`.  It is responsible for parsing the source
+   documents stores all metadata about the document collection and is serialized
+   after each build.
+
+   Its API provides methods to do with access to metadata, resolving references,
+   etc.  It can also be used by extensions to cache information that should
+   persist for incremental rebuilds.
+
+   If you have the application or builder object, the environment is available
+   as ``app.env`` or ``builder.env``.
+
+**Builder**
+   The builder object (usually called ``builder``) is an instance of a specific
+   subclass of :class:`.Builder`.  Each builder class knows how to convert the
+   parsed documents into an output format, or otherwise process them (e.g. check
+   external links).
+
+   If you have the application object, the environment is available as
+   ``app.builder``.
+
+**Config**
+   The config object (usually called ``config``) provides the values of
+   configuration values set in :file:`conf.py` as attributes.  It is an instance
+   of :class:`.Config`.
+
+   The config is available as ``app.config`` or ``env.config``.
+
+
 Build Phases
 ------------
 
@@ -29,11 +73,15 @@ in which a Sphinx project is built: this works in several phases.
 
    In Phase 1, all source files (and on subsequent builds, those that are new or
    changed) are read and parsed.  This is the phase where directives and roles
-   are encountered by the docutils, and the corresponding functions are called.
-   The output of this phase is a *doctree* for each source files, that is a tree
-   of docutils nodes.  For document elements that aren't fully known until all
+   are encountered by the docutils, and the corresponding code is executed.  The
+   output of this phase is a *doctree* for each source files, that is a tree of
+   docutils nodes.  For document elements that aren't fully known until all
    existing files are read, temporary nodes are created.
 
+   There are nodes provided by docutils, which are documented `in the docutils
+   documentation <http://docutils.sourceforge.net/docs/ref/doctree.html>`__.
+   Additional nodes are provided by Sphinx and :ref:`documented here <nodes>`.
+
    During reading, the build environment is updated with all meta- and cross
    reference data of the read documents, such as labels, the names of headings,
    described Python objects and index entries.  This will later be used to
@@ -165,21 +213,26 @@ docutils classes defined in :mod:`docutils.nodes`.  ``todo`` inherits from
 ``Admonition`` because it should be handled like a note or warning, ``todolist``
 is just a "general" node.
 
+.. note::
+
+   Many extensions will not have to create their own node classes and work fine
+   with the nodes already provided by `docutils
+   <http://docutils.sourceforge.net/docs/ref/doctree.html>`__ and :ref:`Sphinx
+   <nodes>`.
+
 
 The Directive Classes
 ---------------------
 
 A directive class is a class deriving usually from
-``docutils.parsers.rst.Directive``.  Since the class-based directive interface
-doesn't exist yet in Docutils 0.4, Sphinx has another base class called
-``sphinx.util.compat.Directive`` that you can derive your directive from, and it
-will work with both Docutils 0.4 and 0.5 upwards.  The directive interface is
-covered in detail in the `docutils documentation`_; the important thing is that the
-class has a method ``run`` that returns a list of nodes.
+``docutils.parsers.rst.Directive``.  The directive interface is covered in
+detail in the `docutils documentation`_; the important thing is that the class
+has attributes that configure the allowed markup and a method ``run`` that
+returns a list of nodes.
 
 The ``todolist`` directive is quite simple::
 
-   from sphinx.util.compat import Directive
+   from docutils.parsers.rst import Directive
 
    class TodolistDirective(Directive):
 
@@ -224,9 +277,9 @@ to the build environment instance using ``self.state.document.settings.env``.
 
 Then, to act as a link target (from the todolist), the todo directive needs to
 return a target node in addition to the todo node.  The target ID (in HTML, this
-will be the anchor name) is generated by using ``env.new_serialno`` which is
-returns a new integer directive on each call and therefore leads to unique
-target names.  The target node is instantiated without any text (the first two
+will be the anchor name) is generated by using ``env.new_serialno`` which
+returns a new unique integer on each call and therefore leads to unique target
+names.  The target node is instantiated without any text (the first two
 arguments).
 
 An admonition is created using a standard docutils function (wrapped in Sphinx
diff --git a/doc/extensions.rst b/doc/extensions.rst
index 334e5039..b2adbc1a 100644
--- a/doc/extensions.rst
+++ b/doc/extensions.rst
@@ -3,34 +3,12 @@
 Sphinx Extensions
 =================
 
-.. module:: sphinx.application
-   :synopsis: Application class and extensibility interface.
-
-Since many projects will need special features in their documentation, Sphinx is
-designed to be extensible on several levels.
-
-This is what you can do in an extension: First, you can add new
-:term:`builder`\s to support new output formats or actions on the parsed
-documents.  Then, it is possible to register custom reStructuredText roles and
-directives, extending the markup.  And finally, there are so-called "hook
-points" at strategic places throughout the build process, where an extension can
-register a hook and run specialized code.
-
-An extension is simply a Python module.  When an extension is loaded, Sphinx
-imports this module and executes its ``setup()`` function, which in turn
-notifies Sphinx of everything the extension offers -- see the extension tutorial
-for examples.
-
-The configuration file itself can be treated as an extension if it contains a
-``setup()`` function.  All other extensions to load must be listed in the
-:confval:`extensions` configuration value.
-
-.. toctree::
-
-   ext/tutorial
-   ext/appapi
-   ext/builderapi
+Since many projects will need special features in their documentation, Sphinx
+allows to add "extensions" to the build process, each of which can modify almost
+any aspect of document processing.
 
+This chapter describes the extensions bundled with Sphinx.  For the API
+documentation on writing your own extension, see :ref:`dev-extensions`.
 
 Builtin Sphinx extensions
 -------------------------