From a8add0ec5ef05c26e1641b8310b65ddd75c0fec3 Mon Sep 17 00:00:00 2001
From: Guido van Rossum <guido@python.org>
Date: Mon, 14 May 2007 22:03:55 +0000
Subject: Merged revisions 55270-55324 via svnmerge from
 svn+ssh://pythondev@svn.python.org/python/branches/p3yk

........
  r55271 | fred.drake | 2007-05-11 10:14:47 -0700 (Fri, 11 May 2007) | 3 lines

  remove jpeg, panel libraries for SGI; there is more IRIX stuff left over,
  I guess that should be removed too, but will leave for someone who is sure
........
  r55280 | fred.drake | 2007-05-11 19:11:37 -0700 (Fri, 11 May 2007) | 1 line

  remove mention of file that has been removed
........
  r55301 | brett.cannon | 2007-05-13 17:38:05 -0700 (Sun, 13 May 2007) | 4 lines

  Remove rexec and Bastion from the stdlib.  This also eliminates the need for
  f_restricted on frames.  This in turn negates the need for
  PyEval_GetRestricted() and PyFrame_IsRestricted().
........
  r55303 | brett.cannon | 2007-05-13 19:22:22 -0700 (Sun, 13 May 2007) | 2 lines

  Remove the md5 and sha modules.
........
  r55305 | george.yoshida | 2007-05-13 19:45:55 -0700 (Sun, 13 May 2007) | 2 lines

  fix markup
........
  r55306 | neal.norwitz | 2007-05-13 19:47:57 -0700 (Sun, 13 May 2007) | 1 line

  Get the doc building again after some removals.
........
  r55307 | neal.norwitz | 2007-05-13 19:50:45 -0700 (Sun, 13 May 2007) | 1 line

  Get test_pyclbr passing again after getstatus was removed from commands.  This "test case" was weird since it was just importing a seemingly random module.  Remove the import
........
  r55322 | brett.cannon | 2007-05-14 14:09:20 -0700 (Mon, 14 May 2007) | 3 lines

  Remove the compiler package.  Will eventually need a mechanism to byte compile
  an AST.
........
---
 Doc/lib/compiler.tex | 353 ---------------------------------------------------
 1 file changed, 353 deletions(-)
 delete mode 100644 Doc/lib/compiler.tex

(limited to 'Doc/lib/compiler.tex')

diff --git a/Doc/lib/compiler.tex b/Doc/lib/compiler.tex
deleted file mode 100644
index d4f4124ef7..0000000000
--- a/Doc/lib/compiler.tex
+++ /dev/null
@@ -1,353 +0,0 @@
-\chapter{Python compiler package \label{compiler}}
-
-\sectionauthor{Jeremy Hylton}{jeremy@zope.com}
-
-
-The Python compiler package is a tool for analyzing Python source code
-and generating Python bytecode.  The compiler contains libraries to
-generate an abstract syntax tree from Python source code and to
-generate Python bytecode from the tree.
-
-The \refmodule{compiler} package is a Python source to bytecode
-translator written in Python.  It uses the built-in parser and
-standard \refmodule{parser} module to generated a concrete syntax
-tree.  This tree is used to generate an abstract syntax tree (AST) and
-then Python bytecode.
-
-The full functionality of the package duplicates the builtin compiler
-provided with the Python interpreter.  It is intended to match its
-behavior almost exactly.  Why implement another compiler that does the
-same thing?  The package is useful for a variety of purposes.  It can
-be modified more easily than the builtin compiler.  The AST it
-generates is useful for analyzing Python source code.
-
-This chapter explains how the various components of the
-\refmodule{compiler} package work.  It blends reference material with
-a tutorial.
-
-The following modules are part of the \refmodule{compiler} package:
-
-\localmoduletable
-
-
-\section{The basic interface}
-
-\declaremodule{}{compiler}
-
-The top-level of the package defines four functions.  If you import
-\module{compiler}, you will get these functions and a collection of
-modules contained in the package.
-
-\begin{funcdesc}{parse}{buf}
-Returns an abstract syntax tree for the Python source code in \var{buf}.
-The function raises \exception{SyntaxError} if there is an error in the
-source code.  The return value is a \class{compiler.ast.Module} instance
-that contains the tree.  
-\end{funcdesc}
-
-\begin{funcdesc}{parseFile}{path}
-Return an abstract syntax tree for the Python source code in the file
-specified by \var{path}.  It is equivalent to
-\code{parse(open(\var{path}).read())}.
-\end{funcdesc}
-
-\begin{funcdesc}{walk}{ast, visitor\optional{, verbose}}
-Do a pre-order walk over the abstract syntax tree \var{ast}.  Call the
-appropriate method on the \var{visitor} instance for each node
-encountered.
-\end{funcdesc}
-
-\begin{funcdesc}{compile}{source, filename, mode, flags=None, 
-			dont_inherit=None}
-Compile the string \var{source}, a Python module, statement or
-expression, into a code object that can be executed by the exec
-statement or \function{eval()}. This function is a replacement for the
-built-in \function{compile()} function.
-
-The \var{filename} will be used for run-time error messages.
-
-The \var{mode} must be 'exec' to compile a module, 'single' to compile a
-single (interactive) statement, or 'eval' to compile an expression.
-
-The \var{flags} and \var{dont_inherit} arguments affect future-related
-statements, but are not supported yet.
-\end{funcdesc}
-
-\begin{funcdesc}{compileFile}{source}
-Compiles the file \var{source} and generates a .pyc file.
-\end{funcdesc}
-
-The \module{compiler} package contains the following modules:
-\refmodule[compiler.ast]{ast}, \module{consts}, \module{future},
-\module{misc}, \module{pyassem}, \module{pycodegen}, \module{symbols},
-\module{transformer}, and \refmodule[compiler.visitor]{visitor}.
-
-\section{Limitations}
-
-There are some problems with the error checking of the compiler
-package.  The interpreter detects syntax errors in two distinct
-phases.  One set of errors is detected by the interpreter's parser,
-the other set by the compiler.  The compiler package relies on the
-interpreter's parser, so it get the first phases of error checking for
-free.  It implements the second phase itself, and that implementation is
-incomplete.  For example, the compiler package does not raise an error
-if a name appears more than once in an argument list: 
-\code{def f(x, x): ...}
-
-A future version of the compiler should fix these problems.
-
-\section{Python Abstract Syntax}
-
-The \module{compiler.ast} module defines an abstract syntax for
-Python.  In the abstract syntax tree, each node represents a syntactic
-construct.  The root of the tree is \class{Module} object.
-
-The abstract syntax offers a higher level interface to parsed Python
-source code.  The \refmodule{parser}
-module and the compiler written in C for the Python interpreter use a
-concrete syntax tree.  The concrete syntax is tied closely to the
-grammar description used for the Python parser.  Instead of a single
-node for a construct, there are often several levels of nested nodes
-that are introduced by Python's precedence rules.
-
-The abstract syntax tree is created by the
-\module{compiler.transformer} module.  The transformer relies on the
-builtin Python parser to generate a concrete syntax tree.  It
-generates an abstract syntax tree from the concrete tree.  
-
-The \module{transformer} module was created by Greg
-Stein\index{Stein, Greg} and Bill Tutt\index{Tutt, Bill} for an
-experimental Python-to-C compiler.  The current version contains a
-number of modifications and improvements, but the basic form of the
-abstract syntax and of the transformer are due to Stein and Tutt.
-
-\subsection{AST Nodes}
-
-\declaremodule{}{compiler.ast}
-
-The \module{compiler.ast} module is generated from a text file that
-describes each node type and its elements.  Each node type is
-represented as a class that inherits from the abstract base class
-\class{compiler.ast.Node} and defines a set of named attributes for
-child nodes.
-
-\begin{classdesc}{Node}{}
-  
-  The \class{Node} instances are created automatically by the parser
-  generator.  The recommended interface for specific \class{Node}
-  instances is to use the public attributes to access child nodes.  A
-  public attribute may be bound to a single node or to a sequence of
-  nodes, depending on the \class{Node} type.  For example, the
-  \member{bases} attribute of the \class{Class} node, is bound to a
-  list of base class nodes, and the \member{doc} attribute is bound to
-  a single node.
-  
-  Each \class{Node} instance has a \member{lineno} attribute which may
-  be \code{None}.  XXX Not sure what the rules are for which nodes
-  will have a useful lineno.
-\end{classdesc}
-
-All \class{Node} objects offer the following methods:
-
-\begin{methoddesc}{getChildren}{}
-  Returns a flattened list of the child nodes and objects in the
-  order they occur.  Specifically, the order of the nodes is the
-  order in which they appear in the Python grammar.  Not all of the
-  children are \class{Node} instances.  The names of functions and
-  classes, for example, are plain strings.
-\end{methoddesc}
-
-\begin{methoddesc}{getChildNodes}{}
-  Returns a flattened list of the child nodes in the order they
-  occur.  This method is like \method{getChildren()}, except that it
-  only returns those children that are \class{Node} instances.
-\end{methoddesc}
-
-Two examples illustrate the general structure of \class{Node}
-classes.  The \keyword{while} statement is defined by the following
-grammar production: 
-
-\begin{verbatim}
-while_stmt:     "while" expression ":" suite
-               ["else" ":" suite]
-\end{verbatim}
-
-The \class{While} node has three attributes: \member{test},
-\member{body}, and \member{else_}.  (If the natural name for an
-attribute is also a Python reserved word, it can't be used as an
-attribute name.  An underscore is appended to the word to make it a
-legal identifier, hence \member{else_} instead of \keyword{else}.)
-
-The \keyword{if} statement is more complicated because it can include
-several tests.  
-
-\begin{verbatim}
-if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
-\end{verbatim}
-
-The \class{If} node only defines two attributes: \member{tests} and
-\member{else_}.  The \member{tests} attribute is a sequence of test
-expression, consequent body pairs.  There is one pair for each
-\keyword{if}/\keyword{elif} clause.  The first element of the pair is
-the test expression.  The second elements is a \class{Stmt} node that
-contains the code to execute if the test is true.
-
-The \method{getChildren()} method of \class{If} returns a flat list of
-child nodes.  If there are three \keyword{if}/\keyword{elif} clauses
-and no \keyword{else} clause, then \method{getChildren()} will return
-a list of six elements: the first test expression, the first
-\class{Stmt}, the second text expression, etc.
-
-The following table lists each of the \class{Node} subclasses defined
-in \module{compiler.ast} and each of the public attributes available
-on their instances.  The values of most of the attributes are
-themselves \class{Node} instances or sequences of instances.  When the
-value is something other than an instance, the type is noted in the
-comment.  The attributes are listed in the order in which they are
-returned by \method{getChildren()} and \method{getChildNodes()}.
-
-\input{asttable}
-
-
-\subsection{Assignment nodes}
-
-There is a collection of nodes used to represent assignments.  Each
-assignment statement in the source code becomes a single
-\class{Assign} node in the AST.  The \member{nodes} attribute is a
-list that contains a node for each assignment target.  This is
-necessary because assignment can be chained, e.g. \code{a = b = 2}.
-Each \class{Node} in the list will be one of the following classes: 
-\class{AssAttr}, \class{AssList}, \class{AssName}, or
-\class{AssTuple}. 
-
-Each target assignment node will describe the kind of object being
-assigned to:  \class{AssName} for a simple name, e.g. \code{a = 1}.
-\class{AssAttr} for an attribute assigned, e.g. \code{a.x = 1}.
-\class{AssList} and \class{AssTuple} for list and tuple expansion
-respectively, e.g. \code{a, b, c = a_tuple}.
-
-The target assignment nodes also have a \member{flags} attribute that
-indicates whether the node is being used for assignment or in a delete
-statement.  The \class{AssName} is also used to represent a delete
-statement, e.g. \class{del x}.
-
-When an expression contains several attribute references, an
-assignment or delete statement will contain only one \class{AssAttr}
-node -- for the final attribute reference.  The other attribute
-references will be represented as \class{Getattr} nodes in the
-\member{expr} attribute of the \class{AssAttr} instance.
-
-\subsection{Examples}
-
-This section shows several simple examples of ASTs for Python source
-code.  The examples demonstrate how to use the \function{parse()}
-function, what the repr of an AST looks like, and how to access
-attributes of an AST node.
-
-The first module defines a single function.  Assume it is stored in
-\file{/tmp/doublelib.py}. 
-
-\begin{verbatim}
-"""This is an example module.
-
-This is the docstring.
-"""
-
-def double(x):
-    "Return twice the argument"
-    return x * 2
-\end{verbatim}
-
-In the interactive interpreter session below, I have reformatted the
-long AST reprs for readability.  The AST reprs use unqualified class
-names.  If you want to create an instance from a repr, you must import
-the class names from the \module{compiler.ast} module.
-
-\begin{verbatim}
->>> import compiler
->>> mod = compiler.parseFile("/tmp/doublelib.py")
->>> mod
-Module('This is an example module.\n\nThis is the docstring.\n', 
-       Stmt([Function(None, 'double', ['x'], [], 0,
-                      'Return twice the argument', 
-                      Stmt([Return(Mul((Name('x'), Const(2))))]))]))
->>> from compiler.ast import *
->>> Module('This is an example module.\n\nThis is the docstring.\n', 
-...    Stmt([Function(None, 'double', ['x'], [], 0,
-...                   'Return twice the argument', 
-...                   Stmt([Return(Mul((Name('x'), Const(2))))]))]))
-Module('This is an example module.\n\nThis is the docstring.\n', 
-       Stmt([Function(None, 'double', ['x'], [], 0,
-                      'Return twice the argument', 
-                      Stmt([Return(Mul((Name('x'), Const(2))))]))]))
->>> mod.doc
-'This is an example module.\n\nThis is the docstring.\n'
->>> for node in mod.node.nodes:
-...     print node
-... 
-Function(None, 'double', ['x'], [], 0, 'Return twice the argument',
-         Stmt([Return(Mul((Name('x'), Const(2))))]))
->>> func = mod.node.nodes[0]
->>> func.code
-Stmt([Return(Mul((Name('x'), Const(2))))])
-\end{verbatim}
-
-\section{Using Visitors to Walk ASTs}
-
-\declaremodule{}{compiler.visitor}
-
-The visitor pattern is ...  The \refmodule{compiler} package uses a
-variant on the visitor pattern that takes advantage of Python's
-introspection features to eliminate the need for much of the visitor's
-infrastructure.
-
-The classes being visited do not need to be programmed to accept
-visitors.  The visitor need only define visit methods for classes it
-is specifically interested in; a default visit method can handle the
-rest. 
-
-XXX The magic \method{visit()} method for visitors.
-
-\begin{funcdesc}{walk}{tree, visitor\optional{, verbose}}
-\end{funcdesc}
-
-\begin{classdesc}{ASTVisitor}{}
-
-The \class{ASTVisitor} is responsible for walking over the tree in the
-correct order.  A walk begins with a call to \method{preorder()}.  For
-each node, it checks the \var{visitor} argument to \method{preorder()}
-for a method named `visitNodeType,' where NodeType is the name of the
-node's class, e.g. for a \class{While} node a \method{visitWhile()}
-would be called.  If the method exists, it is called with the node as
-its first argument.
-
-The visitor method for a particular node type can control how child
-nodes are visited during the walk.  The \class{ASTVisitor} modifies
-the visitor argument by adding a visit method to the visitor; this
-method can be used to visit a particular child node.  If no visitor is
-found for a particular node type, the \method{default()} method is
-called. 
-\end{classdesc}
-
-\class{ASTVisitor} objects have the following methods:
-
-XXX describe extra arguments
-
-\begin{methoddesc}{default}{node\optional{, \moreargs}}
-\end{methoddesc}
-
-\begin{methoddesc}{dispatch}{node\optional{, \moreargs}}
-\end{methoddesc}
-
-\begin{methoddesc}{preorder}{tree, visitor}
-\end{methoddesc}
-
-
-\section{Bytecode Generation}
-
-The code generator is a visitor that emits bytecodes.  Each visit method
-can call the \method{emit()} method to emit a new bytecode.  The basic
-code generator is specialized for modules, classes, and functions.  An
-assembler converts that emitted instructions to the low-level bytecode
-format.  It handles things like generator of constant lists of code
-objects and calculation of jump offsets.
-- 
cgit v1.2.1

