diff options
Diffstat (limited to 'Doc/reference')
| -rw-r--r-- | Doc/reference/compound_stmts.rst | 554 | ||||
| -rw-r--r-- | Doc/reference/datamodel.rst | 2118 | ||||
| -rw-r--r-- | Doc/reference/executionmodel.rst | 232 | ||||
| -rw-r--r-- | Doc/reference/expressions.rst | 1283 | ||||
| -rw-r--r-- | Doc/reference/index.rst | 30 | ||||
| -rw-r--r-- | Doc/reference/introduction.rst | 138 | ||||
| -rw-r--r-- | Doc/reference/lexical_analysis.rst | 758 | ||||
| -rw-r--r-- | Doc/reference/simple_stmts.rst | 825 | ||||
| -rw-r--r-- | Doc/reference/toplevel_components.rst | 123 |
9 files changed, 6061 insertions, 0 deletions
diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst new file mode 100644 index 0000000000..e7587f6cc3 --- /dev/null +++ b/Doc/reference/compound_stmts.rst @@ -0,0 +1,554 @@ + +.. _compound: + +******************* +Compound statements +******************* + +.. index:: pair: compound; statement + +Compound statements contain (groups of) other statements; they affect or control +the execution of those other statements in some way. In general, compound +statements span multiple lines, although in simple incarnations a whole compound +statement may be contained in one line. + +The :keyword:`if`, :keyword:`while` and :keyword:`for` statements implement +traditional control flow constructs. :keyword:`try` specifies exception +handlers and/or cleanup code for a group of statements. Function and class +definitions are also syntactically compound statements. + +.. index:: + single: clause + single: suite + +Compound statements consist of one or more 'clauses.' A clause consists of a +header and a 'suite.' The clause headers of a particular compound statement are +all at the same indentation level. Each clause header begins with a uniquely +identifying keyword and ends with a colon. A suite is a group of statements +controlled by a clause. A suite can be one or more semicolon-separated simple +statements on the same line as the header, following the header's colon, or it +can be one or more indented statements on subsequent lines. Only the latter +form of suite can contain nested compound statements; the following is illegal, +mostly because it wouldn't be clear to which :keyword:`if` clause a following +:keyword:`else` clause would belong: :: + + if test1: if test2: print x + +Also note that the semicolon binds tighter than the colon in this context, so +that in the following example, either all or none of the :keyword:`print` +statements are executed:: + + if x < y < z: print x; print y; print z + +Summarizing: + +.. productionlist:: + compound_stmt: `if_stmt` + : | `while_stmt` + : | `for_stmt` + : | `try_stmt` + : | `with_stmt` + : | `funcdef` + : | `classdef` + suite: `stmt_list` NEWLINE | NEWLINE INDENT `statement`+ DEDENT + statement: `stmt_list` NEWLINE | `compound_stmt` + stmt_list: `simple_stmt` (";" `simple_stmt`)* [";"] + +.. index:: + single: NEWLINE token + single: DEDENT token + pair: dangling; else + +Note that statements always end in a ``NEWLINE`` possibly followed by a +``DEDENT``. Also note that optional continuation clauses always begin with a +keyword that cannot start a statement, thus there are no ambiguities (the +'dangling :keyword:`else`' problem is solved in Python by requiring nested +:keyword:`if` statements to be indented). + +The formatting of the grammar rules in the following sections places each clause +on a separate line for clarity. + + +.. _if: + +The :keyword:`if` statement +=========================== + +.. index:: statement: if + +The :keyword:`if` statement is used for conditional execution: + +.. productionlist:: + if_stmt: "if" `expression` ":" `suite` + : ( "elif" `expression` ":" `suite` )* + : ["else" ":" `suite`] + +.. index:: + keyword: elif + keyword: else + +It selects exactly one of the suites by evaluating the expressions one by one +until one is found to be true (see section :ref:`booleans` for the definition of +true and false); then that suite is executed (and no other part of the +:keyword:`if` statement is executed or evaluated). If all expressions are +false, the suite of the :keyword:`else` clause, if present, is executed. + + +.. _while: + +The :keyword:`while` statement +============================== + +.. index:: + statement: while + pair: loop; statement + +The :keyword:`while` statement is used for repeated execution as long as an +expression is true: + +.. productionlist:: + while_stmt: "while" `expression` ":" `suite` + : ["else" ":" `suite`] + +.. index:: keyword: else + +This repeatedly tests the expression and, if it is true, executes the first +suite; if the expression is false (which may be the first time it is tested) the +suite of the :keyword:`else` clause, if present, is executed and the loop +terminates. + +.. index:: + statement: break + statement: continue + +A :keyword:`break` statement executed in the first suite terminates the loop +without executing the :keyword:`else` clause's suite. A :keyword:`continue` +statement executed in the first suite skips the rest of the suite and goes back +to testing the expression. + + +.. _for: + +The :keyword:`for` statement +============================ + +.. index:: + statement: for + pair: loop; statement + +.. index:: object: sequence + +The :keyword:`for` statement is used to iterate over the elements of a sequence +(such as a string, tuple or list) or other iterable object: + +.. productionlist:: + for_stmt: "for" `target_list` "in" `expression_list` ":" `suite` + : ["else" ":" `suite`] + +.. index:: + keyword: in + keyword: else + pair: target; list + +The expression list is evaluated once; it should yield an iterable object. An +iterator is created for the result of the ``expression_list``. The suite is +then executed once for each item provided by the iterator, in the order of +ascending indices. Each item in turn is assigned to the target list using the +standard rules for assignments, and then the suite is executed. When the items +are exhausted (which is immediately when the sequence is empty), the suite in +the :keyword:`else` clause, if present, is executed, and the loop terminates. + +.. index:: + statement: break + statement: continue + +A :keyword:`break` statement executed in the first suite terminates the loop +without executing the :keyword:`else` clause's suite. A :keyword:`continue` +statement executed in the first suite skips the rest of the suite and continues +with the next item, or with the :keyword:`else` clause if there was no next +item. + +The suite may assign to the variable(s) in the target list; this does not affect +the next item assigned to it. + +.. index:: + builtin: range + pair: Pascal; language + +The target list is not deleted when the loop is finished, but if the sequence is +empty, it will not have been assigned to at all by the loop. Hint: the built-in +function :func:`range` returns a sequence of integers suitable to emulate the +effect of Pascal's ``for i := a to b do``; e.g., ``range(3)`` returns the list +``[0, 1, 2]``. + +.. warning:: + + .. index:: + single: loop; over mutable sequence + single: mutable sequence; loop over + + There is a subtlety when the sequence is being modified by the loop (this can + only occur for mutable sequences, i.e. lists). An internal counter is used to + keep track of which item is used next, and this is incremented on each + iteration. When this counter has reached the length of the sequence the loop + terminates. This means that if the suite deletes the current (or a previous) + item from the sequence, the next item will be skipped (since it gets the index + of the current item which has already been treated). Likewise, if the suite + inserts an item in the sequence before the current item, the current item will + be treated again the next time through the loop. This can lead to nasty bugs + that can be avoided by making a temporary copy using a slice of the whole + sequence, e.g., + +:: + + for x in a[:]: + if x < 0: a.remove(x) + + +.. _try: + +The :keyword:`try` statement +============================ + +.. index:: statement: try + +The :keyword:`try` statement specifies exception handlers and/or cleanup code +for a group of statements: + +.. productionlist:: + try_stmt: try1_stmt | try2_stmt + try1_stmt: "try" ":" `suite` + : ("except" [`expression` ["," `target`]] ":" `suite`)+ + : ["else" ":" `suite`] + : ["finally" ":" `suite`] + try2_stmt: "try" ":" `suite` + : "finally" ":" `suite` + +.. versionchanged:: 2.5 + In previous versions of Python, :keyword:`try`...\ :keyword:`except`...\ + :keyword:`finally` did not work. :keyword:`try`...\ :keyword:`except` had to be + nested in :keyword:`try`...\ :keyword:`finally`. + +.. index:: keyword: except + +The :keyword:`except` clause(s) specify one or more exception handlers. When no +exception occurs in the :keyword:`try` clause, no exception handler is executed. +When an exception occurs in the :keyword:`try` suite, a search for an exception +handler is started. This search inspects the except clauses in turn until one +is found that matches the exception. An expression-less except clause, if +present, must be last; it matches any exception. For an except clause with an +expression, that expression is evaluated, and the clause matches the exception +if the resulting object is "compatible" with the exception. An object is +compatible with an exception if it is the class or a base class of the exception +object or a tuple containing an item compatible with the exception. + +If no except clause matches the exception, the search for an exception handler +continues in the surrounding code and on the invocation stack. [#]_ + +If the evaluation of an expression in the header of an except clause raises an +exception, the original search for a handler is canceled and a search starts for +the new exception in the surrounding code and on the call stack (it is treated +as if the entire :keyword:`try` statement raised the exception). + +When a matching except clause is found, the exception is assigned to the target +specified in that except clause, if present, and the except clause's suite is +executed. All except clauses must have an executable block. When the end of +this block is reached, execution continues normally after the entire try +statement. (This means that if two nested handlers exist for the same +exception, and the exception occurs in the try clause of the inner handler, the +outer handler will not handle the exception.) + +.. index:: + module: sys + object: traceback + +Before an except clause's suite is executed, details about the exception are +stored in the :mod:`sys` module and can be access via :func:`sys.exc_info`. +:func:`sys.exc_info` returns a 3-tuple consisting of: ``exc_type`` receives the +object identifying the exception; ``exc_value`` receives the exception's +parameter; ``exc_traceback`` receives a traceback object (see section +:ref:`types`) identifying the point in the program where the exception +occurred. :func:`sys.exc_info` values are restored to their previous values +(before the call) when returning from a function that handled an exception. + +.. index:: + keyword: else + statement: return + statement: break + statement: continue + +The optional :keyword:`else` clause is executed if and when control flows off +the end of the :keyword:`try` clause. [#]_ Exceptions in the :keyword:`else` +clause are not handled by the preceding :keyword:`except` clauses. + +.. index:: keyword: finally + +If :keyword:`finally` is present, it specifies a 'cleanup' handler. The +:keyword:`try` clause is executed, including any :keyword:`except` and +:keyword:`else` clauses. If an exception occurs in any of the clauses and is +not handled, the exception is temporarily saved. The :keyword:`finally` clause +is executed. If there is a saved exception, it is re-raised at the end of the +:keyword:`finally` clause. If the :keyword:`finally` clause raises another +exception or executes a :keyword:`return` or :keyword:`break` statement, the +saved exception is lost. The exception information is not available to the +program during execution of the :keyword:`finally` clause. + +.. index:: + statement: return + statement: break + statement: continue + +When a :keyword:`return`, :keyword:`break` or :keyword:`continue` statement is +executed in the :keyword:`try` suite of a :keyword:`try`...\ :keyword:`finally` +statement, the :keyword:`finally` clause is also executed 'on the way out.' A +:keyword:`continue` statement is illegal in the :keyword:`finally` clause. (The +reason is a problem with the current implementation --- this restriction may be +lifted in the future). + +Additional information on exceptions can be found in section :ref:`exceptions`, +and information on using the :keyword:`raise` statement to generate exceptions +may be found in section :ref:`raise`. + + +.. _with: + +The :keyword:`with` statement +============================= + +.. index:: statement: with + +.. versionadded:: 2.5 + +The :keyword:`with` statement is used to wrap the execution of a block with +methods defined by a context manager (see section :ref:`context-managers`). This +allows common :keyword:`try`...\ :keyword:`except`...\ :keyword:`finally` usage +patterns to be encapsulated for convenient reuse. + +.. productionlist:: + with_stmt: "with" `expression` ["as" `target`] ":" `suite` + +The execution of the :keyword:`with` statement proceeds as follows: + +#. The context expression is evaluated to obtain a context manager. + +#. The context manager's :meth:`__enter__` method is invoked. + +#. If a target was included in the :keyword:`with` statement, the return value + from :meth:`__enter__` is assigned to it. + + .. note:: + + The :keyword:`with` statement guarantees that if the :meth:`__enter__` method + returns without an error, then :meth:`__exit__` will always be called. Thus, if + an error occurs during the assignment to the target list, it will be treated the + same as an error occurring within the suite would be. See step 5 below. + +#. The suite is executed. + +#. The context manager's :meth:`__exit__` method is invoked. If an exception + caused the suite to be exited, its type, value, and traceback are passed as + arguments to :meth:`__exit__`. Otherwise, three :const:`None` arguments are + supplied. + + If the suite was exited due to an exception, and the return value from the + :meth:`__exit__` method was false, the exception is reraised. If the return + value was true, the exception is suppressed, and execution continues with the + statement following the :keyword:`with` statement. + + If the suite was exited for any reason other than an exception, the return value + from :meth:`__exit__` is ignored, and execution proceeds at the normal location + for the kind of exit that was taken. + +.. note:: + + In Python 2.5, the :keyword:`with` statement is only allowed when the + ``with_statement`` feature has been enabled. It will always be enabled in + Python 2.6. This ``__future__`` import statement can be used to enable the + feature:: + + from __future__ import with_statement + + +.. seealso:: + + :pep:`0343` - The "with" statement + The specification, background, and examples for the Python :keyword:`with` + statement. + + +.. _function: + +Function definitions +==================== + +.. index:: + pair: function; definition + statement: def + +.. index:: + object: user-defined function + object: function + +A function definition defines a user-defined function object (see section +:ref:`types`): + +.. productionlist:: + funcdef: [`decorators`] "def" `funcname` "(" [`parameter_list`] ")" ["->" `expression`]? ":" `suite` + decorators: `decorator`+ + decorator: "@" `dotted_name` ["(" [`argument_list` [","]] ")"] NEWLINE + dotted_name: `identifier` ("." `identifier`)* + parameter_list: (`defparameter` ",")* + : ( "*" [`parameter`] ("," `defparameter`)* + : [, "**" `parameter`] + : | "**" `parameter` + : | `defparameter` [","] ) + parameter: `identifier` [":" `expression`] + defparameter: `parameter` ["=" `expression`] + funcname: `identifier` + +.. index:: + pair: function; name + pair: name; binding + +A function definition is an executable statement. Its execution binds the +function name in the current local namespace to a function object (a wrapper +around the executable code for the function). This function object contains a +reference to the current global namespace as the global namespace to be used +when the function is called. + +The function definition does not execute the function body; this gets executed +only when the function is called. + +A function definition may be wrapped by one or more decorator expressions. +Decorator expressions are evaluated when the function is defined, in the scope +that contains the function definition. The result must be a callable, which is +invoked with the function object as the only argument. The returned value is +bound to the function name instead of the function object. Multiple decorators +are applied in nested fashion. For example, the following code:: + + @f1(arg) + @f2 + def func(): pass + +is equivalent to:: + + def func(): pass + func = f1(arg)(f2(func)) + +.. index:: triple: default; parameter; value + +When one or more parameters have the form *parameter* ``=`` *expression*, the +function is said to have "default parameter values." For a parameter with a +default value, the corresponding argument may be omitted from a call, in which +case the parameter's default value is substituted. If a parameter has a default +value, all following parameters up until the "``*``" must also have a default +value --- this is a syntactic restriction that is not expressed by the grammar. + +**Default parameter values are evaluated when the function definition is +executed.** This means that the expression is evaluated once, when the function +is defined, and that that same "pre-computed" value is used for each call. This +is especially important to understand when a default parameter is a mutable +object, such as a list or a dictionary: if the function modifies the object +(e.g. by appending an item to a list), the default value is in effect modified. +This is generally not what was intended. A way around this is to use ``None`` +as the default, and explicitly test for it in the body of the function, e.g.:: + + def whats_on_the_telly(penguin=None): + if penguin is None: + penguin = [] + penguin.append("property of the zoo") + return penguin + +Function call semantics are described in more detail in section :ref:`calls`. A +function call always assigns values to all parameters mentioned in the parameter +list, either from position arguments, from keyword arguments, or from default +values. If the form "``*identifier``" is present, it is initialized to a tuple +receiving any excess positional parameters, defaulting to the empty tuple. If +the form "``**identifier``" is present, it is initialized to a new dictionary +receiving any excess keyword arguments, defaulting to a new empty dictionary. +Parameters after "``*``" or "``*identifier``" are keyword-only parameters and +may only be passed used keyword arguments. + +.. index:: pair: function; annotations + +Parameters may have annotations of the form "``: expression``" following the +parameter name. Any parameter may have an annotation even those of the form +``*identifier`` or ``**identifier``. Functions may have "return" annotation of +the form "``-> expression``" after the parameter list. These annotations can be +any valid Python expression and are evaluated when the function definition is +executed. Annotations may be evaluated in a different order than they appear in +the source code. The presence of annotations does not change the semantics of a +function. The annotation values are available as values of a dictionary keyed +by the parameters' names in the :attr:`__annotations__` attribute of the +function object. + +.. index:: pair: lambda; form + +It is also possible to create anonymous functions (functions not bound to a +name), for immediate use in expressions. This uses lambda forms, described in +section :ref:`lambda`. Note that the lambda form is merely a shorthand for a +simplified function definition; a function defined in a ":keyword:`def`" +statement can be passed around or assigned to another name just like a function +defined by a lambda form. The ":keyword:`def`" form is actually more powerful +since it allows the execution of multiple statements and annotations. + +**Programmer's note:** Functions are first-class objects. A "``def``" form +executed inside a function definition defines a local function that can be +returned or passed around. Free variables used in the nested function can +access the local variables of the function containing the def. See section +:ref:`naming` for details. + + +.. _class: + +Class definitions +================= + +.. index:: + pair: class; definition + statement: class + +.. index:: object: class + +A class definition defines a class object (see section :ref:`types`): + +.. productionlist:: + classdef: "class" `classname` [`inheritance`] ":" `suite` + inheritance: "(" [`expression_list`] ")" + classname: `identifier` + +.. index:: + single: inheritance + pair: class; name + pair: name; binding + pair: execution; frame + +A class definition is an executable statement. It first evaluates the +inheritance list, if present. Each item in the inheritance list should evaluate +to a class object or class type which allows subclassing. The class's suite is +then executed in a new execution frame (see section :ref:`naming`), using a +newly created local namespace and the original global namespace. (Usually, the +suite contains only function definitions.) When the class's suite finishes +execution, its execution frame is discarded but its local namespace is saved. A +class object is then created using the inheritance list for the base classes and +the saved local namespace for the attribute dictionary. The class name is bound +to this class object in the original local namespace. + +**Programmer's note:** Variables defined in the class definition are class +variables; they are shared by all instances. To define instance variables, they +must be given a value in the :meth:`__init__` method or in another method. Both +class and instance variables are accessible through the notation +"``self.name``", and an instance variable hides a class variable with the same +name when accessed in this way. Class variables with immutable values can be +used as defaults for instance variables. For new-style classes, descriptors can +be used to create instance variables with different implementation details. + +.. rubric:: Footnotes + +.. [#] The exception is propogated to the invocation stack only if there is no + :keyword:`finally` clause that negates the exception. + +.. [#] Currently, control "flows off the end" except in the case of an exception or the + execution of a :keyword:`return`, :keyword:`continue`, or :keyword:`break` + statement. + diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst new file mode 100644 index 0000000000..2f6013e48e --- /dev/null +++ b/Doc/reference/datamodel.rst @@ -0,0 +1,2118 @@ + +.. _datamodel: + +********** +Data model +********** + + +.. _objects: + +Objects, values and types +========================= + +.. index:: + single: object + single: data + +:dfn:`Objects` are Python's abstraction for data. All data in a Python program +is represented by objects or by relations between objects. (In a sense, and in +conformance to Von Neumann's model of a "stored program computer," code is also +represented by objects.) + +.. index:: + builtin: id + builtin: type + single: identity of an object + single: value of an object + single: type of an object + single: mutable object + single: immutable object + +Every object has an identity, a type and a value. An object's *identity* never +changes once it has been created; you may think of it as the object's address in +memory. The ':keyword:`is`' operator compares the identity of two objects; the +:func:`id` function returns an integer representing its identity (currently +implemented as its address). An object's :dfn:`type` is also unchangeable. [#]_ +An object's type determines the operations that the object supports (e.g., "does +it have a length?") and also defines the possible values for objects of that +type. The :func:`type` function returns an object's type (which is an object +itself). The *value* of some objects can change. Objects whose value can +change are said to be *mutable*; objects whose value is unchangeable once they +are created are called *immutable*. (The value of an immutable container object +that contains a reference to a mutable object can change when the latter's value +is changed; however the container is still considered immutable, because the +collection of objects it contains cannot be changed. So, immutability is not +strictly the same as having an unchangeable value, it is more subtle.) An +object's mutability is determined by its type; for instance, numbers, strings +and tuples are immutable, while dictionaries and lists are mutable. + +.. index:: + single: garbage collection + single: reference counting + single: unreachable object + +Objects are never explicitly destroyed; however, when they become unreachable +they may be garbage-collected. An implementation is allowed to postpone garbage +collection or omit it altogether --- it is a matter of implementation quality +how garbage collection is implemented, as long as no objects are collected that +are still reachable. (Implementation note: the current implementation uses a +reference-counting scheme with (optional) delayed detection of cyclically linked +garbage, which collects most objects as soon as they become unreachable, but is +not guaranteed to collect garbage containing circular references. See the +documentation of the :mod:`gc` module for information on controlling the +collection of cyclic garbage.) + +Note that the use of the implementation's tracing or debugging facilities may +keep objects alive that would normally be collectable. Also note that catching +an exception with a ':keyword:`try`...\ :keyword:`except`' statement may keep +objects alive. + +Some objects contain references to "external" resources such as open files or +windows. It is understood that these resources are freed when the object is +garbage-collected, but since garbage collection is not guaranteed to happen, +such objects also provide an explicit way to release the external resource, +usually a :meth:`close` method. Programs are strongly recommended to explicitly +close such objects. The ':keyword:`try`...\ :keyword:`finally`' statement +provides a convenient way to do this. + +.. index:: single: container + +Some objects contain references to other objects; these are called *containers*. +Examples of containers are tuples, lists and dictionaries. The references are +part of a container's value. In most cases, when we talk about the value of a +container, we imply the values, not the identities of the contained objects; +however, when we talk about the mutability of a container, only the identities +of the immediately contained objects are implied. So, if an immutable container +(like a tuple) contains a reference to a mutable object, its value changes if +that mutable object is changed. + +Types affect almost all aspects of object behavior. Even the importance of +object identity is affected in some sense: for immutable types, operations that +compute new values may actually return a reference to any existing object with +the same type and value, while for mutable objects this is not allowed. E.g., +after ``a = 1; b = 1``, ``a`` and ``b`` may or may not refer to the same object +with the value one, depending on the implementation, but after ``c = []; d = +[]``, ``c`` and ``d`` are guaranteed to refer to two different, unique, newly +created empty lists. (Note that ``c = d = []`` assigns the same object to both +``c`` and ``d``.) + + +.. _types: + +The standard type hierarchy +=========================== + +.. index:: + single: type + pair: data; type + pair: type; hierarchy + pair: extension; module + pair: C; language + +Below is a list of the types that are built into Python. Extension modules +(written in C, Java, or other languages, depending on the implementation) can +define additional types. Future versions of Python may add types to the type +hierarchy (e.g., rational numbers, efficiently stored arrays of integers, etc.). + +.. index:: + single: attribute + pair: special; attribute + triple: generic; special; attribute + +Some of the type descriptions below contain a paragraph listing 'special +attributes.' These are attributes that provide access to the implementation and +are not intended for general use. Their definition may change in the future. + +None + .. index:: object: None + + This type has a single value. There is a single object with this value. This + object is accessed through the built-in name ``None``. It is used to signify the + absence of a value in many situations, e.g., it is returned from functions that + don't explicitly return anything. Its truth value is false. + +NotImplemented + .. index:: object: NotImplemented + + This type has a single value. There is a single object with this value. This + object is accessed through the built-in name ``NotImplemented``. Numeric methods + and rich comparison methods may return this value if they do not implement the + operation for the operands provided. (The interpreter will then try the + reflected operation, or some other fallback, depending on the operator.) Its + truth value is true. + +Ellipsis + .. index:: object: Ellipsis + + This type has a single value. There is a single object with this value. This + object is accessed through the literal ``...`` or the built-in name + ``Ellipsis``. Its truth value is true. + +Numbers + .. index:: object: numeric + + These are created by numeric literals and returned as results by arithmetic + operators and arithmetic built-in functions. Numeric objects are immutable; + once created their value never changes. Python numbers are of course strongly + related to mathematical numbers, but subject to the limitations of numerical + representation in computers. + + Python distinguishes between integers, floating point numbers, and complex + numbers: + + Integers + .. index:: object: integer + + These represent elements from the mathematical set of integers (positive and + negative). + + There are three types of integers: + + Plain integers + .. index:: + object: plain integer + single: OverflowError (built-in exception) + + These represent numbers in the range -2147483648 through 2147483647. (The range + may be larger on machines with a larger natural word size, but not smaller.) + When the result of an operation would fall outside this range, the result is + normally returned as a long integer (in some cases, the exception + :exc:`OverflowError` is raised instead). For the purpose of shift and mask + operations, integers are assumed to have a binary, 2's complement notation using + 32 or more bits, and hiding no bits from the user (i.e., all 4294967296 + different bit patterns correspond to different values). + + Long integers + .. index:: object: long integer + + These represent numbers in an unlimited range, subject to available (virtual) + memory only. For the purpose of shift and mask operations, a binary + representation is assumed, and negative numbers are represented in a variant of + 2's complement which gives the illusion of an infinite string of sign bits + extending to the left. + + Booleans + .. index:: + object: Boolean + single: False + single: True + + These represent the truth values False and True. The two objects representing + the values False and True are the only Boolean objects. The Boolean type is a + subtype of plain integers, and Boolean values behave like the values 0 and 1, + respectively, in almost all contexts, the exception being that when converted to + a string, the strings ``"False"`` or ``"True"`` are returned, respectively. + + .. index:: pair: integer; representation + + The rules for integer representation are intended to give the most meaningful + interpretation of shift and mask operations involving negative integers and the + least surprises when switching between the plain and long integer domains. Any + operation except left shift, if it yields a result in the plain integer domain + without causing overflow, will yield the same result in the long integer domain + or when using mixed operands. + + .. % Integers + + Floating point numbers + .. index:: + object: floating point + pair: floating point; number + pair: C; language + pair: Java; language + + These represent machine-level double precision floating point numbers. You are + at the mercy of the underlying machine architecture (and C or Java + implementation) for the accepted range and handling of overflow. Python does not + support single-precision floating point numbers; the savings in processor and + memory usage that are usually the reason for using these is dwarfed by the + overhead of using objects in Python, so there is no reason to complicate the + language with two kinds of floating point numbers. + + Complex numbers + .. index:: + object: complex + pair: complex; number + + These represent complex numbers as a pair of machine-level double precision + floating point numbers. The same caveats apply as for floating point numbers. + The real and imaginary parts of a complex number ``z`` can be retrieved through + the read-only attributes ``z.real`` and ``z.imag``. + + .. % Numbers + +Sequences + .. index:: + builtin: len + object: sequence + single: index operation + single: item selection + single: subscription + + These represent finite ordered sets indexed by non-negative numbers. The + built-in function :func:`len` returns the number of items of a sequence. When + the length of a sequence is *n*, the index set contains the numbers 0, 1, + ..., *n*-1. Item *i* of sequence *a* is selected by ``a[i]``. + + .. index:: single: slicing + + Sequences also support slicing: ``a[i:j]`` selects all items with index *k* such + that *i* ``<=`` *k* ``<`` *j*. When used as an expression, a slice is a + sequence of the same type. This implies that the index set is renumbered so + that it starts at 0. + + .. index:: single: extended slicing + + Some sequences also support "extended slicing" with a third "step" parameter: + ``a[i:j:k]`` selects all items of *a* with index *x* where ``x = i + n*k``, *n* + ``>=`` ``0`` and *i* ``<=`` *x* ``<`` *j*. + + Sequences are distinguished according to their mutability: + + Immutable sequences + .. index:: + object: immutable sequence + object: immutable + + An object of an immutable sequence type cannot change once it is created. (If + the object contains references to other objects, these other objects may be + mutable and may be changed; however, the collection of objects directly + referenced by an immutable object cannot change.) + + The following types are immutable sequences: + + Strings + .. index:: + builtin: chr + builtin: ord + object: string + single: character + single: byte + single: ASCII@ASCII + + The items of a string are characters. There is no separate character type; a + character is represented by a string of one item. Characters represent (at + least) 8-bit bytes. The built-in functions :func:`chr` and :func:`ord` convert + between characters and nonnegative integers representing the byte values. Bytes + with the values 0-127 usually represent the corresponding ASCII values, but the + interpretation of values is up to the program. The string data type is also + used to represent arrays of bytes, e.g., to hold data read from a file. + + .. index:: + single: ASCII@ASCII + single: EBCDIC + single: character set + pair: string; comparison + builtin: chr + builtin: ord + + (On systems whose native character set is not ASCII, strings may use EBCDIC in + their internal representation, provided the functions :func:`chr` and + :func:`ord` implement a mapping between ASCII and EBCDIC, and string comparison + preserves the ASCII order. Or perhaps someone can propose a better rule?) + + Unicode + .. index:: + builtin: unichr + builtin: ord + builtin: unicode + object: unicode + single: character + single: integer + single: Unicode + + The items of a Unicode object are Unicode code units. A Unicode code unit is + represented by a Unicode object of one item and can hold either a 16-bit or + 32-bit value representing a Unicode ordinal (the maximum value for the ordinal + is given in ``sys.maxunicode``, and depends on how Python is configured at + compile time). Surrogate pairs may be present in the Unicode object, and will + be reported as two separate items. The built-in functions :func:`unichr` and + :func:`ord` convert between code units and nonnegative integers representing the + Unicode ordinals as defined in the Unicode Standard 3.0. Conversion from and to + other encodings are possible through the Unicode method :meth:`encode` and the + built-in function :func:`unicode`. + + Tuples + .. index:: + object: tuple + pair: singleton; tuple + pair: empty; tuple + + The items of a tuple are arbitrary Python objects. Tuples of two or more items + are formed by comma-separated lists of expressions. A tuple of one item (a + 'singleton') can be formed by affixing a comma to an expression (an expression + by itself does not create a tuple, since parentheses must be usable for grouping + of expressions). An empty tuple can be formed by an empty pair of parentheses. + + .. % Immutable sequences + + Mutable sequences + .. index:: + object: mutable sequence + object: mutable + pair: assignment; statement + single: delete + statement: del + single: subscription + single: slicing + + Mutable sequences can be changed after they are created. The subscription and + slicing notations can be used as the target of assignment and :keyword:`del` + (delete) statements. + + There is currently a single intrinsic mutable sequence type: + + Lists + .. index:: object: list + + The items of a list are arbitrary Python objects. Lists are formed by placing a + comma-separated list of expressions in square brackets. (Note that there are no + special cases needed to form lists of length 0 or 1.) + + .. index:: module: array + + The extension module :mod:`array` provides an additional example of a mutable + sequence type. + + .. % Mutable sequences + + .. % Sequences + +Set types + .. index:: + builtin: len + object: set type + + These represent unordered, finite sets of unique, immutable objects. As such, + they cannot be indexed by any subscript. However, they can be iterated over, and + the built-in function :func:`len` returns the number of items in a set. Common + uses for sets are fast membership testing, removing duplicates from a sequence, + and computing mathematical operations such as intersection, union, difference, + and symmetric difference. + + For set elements, the same immutability rules apply as for dictionary keys. Note + that numeric types obey the normal rules for numeric comparison: if two numbers + compare equal (e.g., ``1`` and ``1.0``), only one of them can be contained in a + set. + + There are currently two intrinsic set types: + + Sets + .. index:: object: set + + These represent a mutable set. They are created by the built-in :func:`set` + constructor and can be modified afterwards by several methods, such as + :meth:`add`. + + Frozen sets + .. index:: object: frozenset + + These represent an immutable set. They are created by the built-in + :func:`frozenset` constructor. As a frozenset is immutable and hashable, it can + be used again as an element of another set, or as a dictionary key. + + .. % Set types + +Mappings + .. index:: + builtin: len + single: subscription + object: mapping + + These represent finite sets of objects indexed by arbitrary index sets. The + subscript notation ``a[k]`` selects the item indexed by ``k`` from the mapping + ``a``; this can be used in expressions and as the target of assignments or + :keyword:`del` statements. The built-in function :func:`len` returns the number + of items in a mapping. + + There is currently a single intrinsic mapping type: + + Dictionaries + .. index:: object: dictionary + + These represent finite sets of objects indexed by nearly arbitrary values. The + only types of values not acceptable as keys are values containing lists or + dictionaries or other mutable types that are compared by value rather than by + object identity, the reason being that the efficient implementation of + dictionaries requires a key's hash value to remain constant. Numeric types used + for keys obey the normal rules for numeric comparison: if two numbers compare + equal (e.g., ``1`` and ``1.0``) then they can be used interchangeably to index + the same dictionary entry. + + Dictionaries are mutable; they can be created by the ``{...}`` notation (see + section :ref:`dict`). + + .. index:: + module: dbm + module: gdbm + module: bsddb + + The extension modules :mod:`dbm`, :mod:`gdbm`, and :mod:`bsddb` provide + additional examples of mapping types. + + .. % Mapping types + +Callable types + .. index:: + object: callable + pair: function; call + single: invocation + pair: function; argument + + These are the types to which the function call operation (see section + :ref:`calls`) can be applied: + + User-defined functions + .. index:: + pair: user-defined; function + object: function + object: user-defined function + + A user-defined function object is created by a function definition (see + section :ref:`function`). It should be called with an argument list + containing the same number of items as the function's formal parameter + list. + + Special attributes: + + +-------------------------+-------------------------------+-----------+ + | Attribute | Meaning | | + +=========================+===============================+===========+ + | :attr:`__doc__` | The function's documentation | Writable | + | | string, or ``None`` if | | + | | unavailable | | + +-------------------------+-------------------------------+-----------+ + | :attr:`__name__` | The function's name | Writable | + +-------------------------+-------------------------------+-----------+ + | :attr:`__module__` | The name of the module the | Writable | + | | function was defined in, or | | + | | ``None`` if unavailable. | | + +-------------------------+-------------------------------+-----------+ + | :attr:`__defaults__` | A tuple containing default | Writable | + | | argument values for those | | + | | arguments that have defaults, | | + | | or ``None`` if no arguments | | + | | have a default value | | + +-------------------------+-------------------------------+-----------+ + | :attr:`__code__` | The code object representing | Writable | + | | the compiled function body. | | + +-------------------------+-------------------------------+-----------+ + | :attr:`__globals__` | A reference to the dictionary | Read-only | + | | that holds the function's | | + | | global variables --- the | | + | | global namespace of the | | + | | module in which the function | | + | | was defined. | | + +-------------------------+-------------------------------+-----------+ + | :attr:`__dict__` | The namespace supporting | Writable | + | | arbitrary function | | + | | attributes. | | + +-------------------------+-------------------------------+-----------+ + | :attr:`__closure__` | ``None`` or a tuple of cells | Read-only | + | | that contain bindings for the | | + | | function's free variables. | | + +-------------------------+-------------------------------+-----------+ + | :attr:`__annotations__` | A dict containing annotations | Writable | + | | of parameters. The keys of | | + | | the dict are the parameter | | + | | names, or ``'return'`` for | | + | | the return annotation, if | | + | | provided. | | + +-------------------------+-------------------------------+-----------+ + | :attr:`__kwdefaults__` | A dict containing defaults | Writable | + | | for keyword-only parameters. | | + +-------------------------+-------------------------------+-----------+ + + Most of the attributes labelled "Writable" check the type of the assigned value. + + .. versionchanged:: 2.4 + ``__name__`` is now writable. + + Function objects also support getting and setting arbitrary attributes, which + can be used, for example, to attach metadata to functions. Regular attribute + dot-notation is used to get and set such attributes. *Note that the current + implementation only supports function attributes on user-defined functions. + Function attributes on built-in functions may be supported in the future.* + + Additional information about a function's definition can be retrieved from its + code object; see the description of internal types below. + + .. index:: + single: __doc__ (function attribute) + single: __name__ (function attribute) + single: __module__ (function attribute) + single: __dict__ (function attribute) + single: __defaults__ (function attribute) + single: __closure__ (function attribute) + single: __code__ (function attribute) + single: __globals__ (function attribute) + single: __annotations__ (function attribute) + single: __kwdefaults__ (function attribute) + pair: global; namespace + + User-defined methods + .. index:: + object: method + object: user-defined method + pair: user-defined; method + + A user-defined method object combines a class, a class instance (or ``None``) + and any callable object (normally a user-defined function). + + Special read-only attributes: :attr:`im_self` is the class instance object, + :attr:`im_func` is the function object; :attr:`im_class` is the class of + :attr:`im_self` for bound methods or the class that asked for the method for + unbound methods; :attr:`__doc__` is the method's documentation (same as + ``im_func.__doc__``); :attr:`__name__` is the method name (same as + ``im_func.__name__``); :attr:`__module__` is the name of the module the method + was defined in, or ``None`` if unavailable. + + .. versionchanged:: 2.2 + :attr:`im_self` used to refer to the class that defined the method. + + .. index:: + single: __doc__ (method attribute) + single: __name__ (method attribute) + single: __module__ (method attribute) + single: im_func (method attribute) + single: im_self (method attribute) + + Methods also support accessing (but not setting) the arbitrary function + attributes on the underlying function object. + + User-defined method objects may be created when getting an attribute of a class + (perhaps via an instance of that class), if that attribute is a user-defined + function object, an unbound user-defined method object, or a class method + object. When the attribute is a user-defined method object, a new method object + is only created if the class from which it is being retrieved is the same as, or + a derived class of, the class stored in the original method object; otherwise, + the original method object is used as it is. + + .. index:: + single: im_class (method attribute) + single: im_func (method attribute) + single: im_self (method attribute) + + When a user-defined method object is created by retrieving a user-defined + function object from a class, its :attr:`im_self` attribute is ``None`` + and the method object is said to be unbound. When one is created by + retrieving a user-defined function object from a class via one of its + instances, its :attr:`im_self` attribute is the instance, and the method + object is said to be bound. In either case, the new method's + :attr:`im_class` attribute is the class from which the retrieval takes + place, and its :attr:`im_func` attribute is the original function object. + + .. index:: single: im_func (method attribute) + + When a user-defined method object is created by retrieving another method object + from a class or instance, the behaviour is the same as for a function object, + except that the :attr:`im_func` attribute of the new instance is not the + original method object but its :attr:`im_func` attribute. + + .. index:: + single: im_class (method attribute) + single: im_func (method attribute) + single: im_self (method attribute) + + When a user-defined method object is created by retrieving a class method object + from a class or instance, its :attr:`im_self` attribute is the class itself (the + same as the :attr:`im_class` attribute), and its :attr:`im_func` attribute is + the function object underlying the class method. + + When an unbound user-defined method object is called, the underlying function + (:attr:`im_func`) is called, with the restriction that the first argument must + be an instance of the proper class (:attr:`im_class`) or of a derived class + thereof. + + When a bound user-defined method object is called, the underlying function + (:attr:`im_func`) is called, inserting the class instance (:attr:`im_self`) in + front of the argument list. For instance, when :class:`C` is a class which + contains a definition for a function :meth:`f`, and ``x`` is an instance of + :class:`C`, calling ``x.f(1)`` is equivalent to calling ``C.f(x, 1)``. + + When a user-defined method object is derived from a class method object, the + "class instance" stored in :attr:`im_self` will actually be the class itself, so + that calling either ``x.f(1)`` or ``C.f(1)`` is equivalent to calling ``f(C,1)`` + where ``f`` is the underlying function. + + Note that the transformation from function object to (unbound or bound) method + object happens each time the attribute is retrieved from the class or instance. + In some cases, a fruitful optimization is to assign the attribute to a local + variable and call that local variable. Also notice that this transformation only + happens for user-defined functions; other callable objects (and all non-callable + objects) are retrieved without transformation. It is also important to note + that user-defined functions which are attributes of a class instance are not + converted to bound methods; this *only* happens when the function is an + attribute of the class. + + Generator functions + .. index:: + single: generator; function + single: generator; iterator + + A function or method which uses the :keyword:`yield` statement (see section + :ref:`yield`) is called a :dfn:`generator + function`. Such a function, when called, always returns an iterator object + which can be used to execute the body of the function: calling the iterator's + :meth:`__next__` method will cause the function to execute until it provides a + value using the :keyword:`yield` statement. When the function executes a + :keyword:`return` statement or falls off the end, a :exc:`StopIteration` + exception is raised and the iterator will have reached the end of the set of + values to be returned. + + Built-in functions + .. index:: + object: built-in function + object: function + pair: C; language + + A built-in function object is a wrapper around a C function. Examples of + built-in functions are :func:`len` and :func:`math.sin` (:mod:`math` is a + standard built-in module). The number and type of the arguments are + determined by the C function. Special read-only attributes: + :attr:`__doc__` is the function's documentation string, or ``None`` if + unavailable; :attr:`__name__` is the function's name; :attr:`__self__` is + set to ``None`` (but see the next item); :attr:`__module__` is the name of + the module the function was defined in or ``None`` if unavailable. + + Built-in methods + .. index:: + object: built-in method + object: method + pair: built-in; method + + This is really a different disguise of a built-in function, this time containing + an object passed to the C function as an implicit extra argument. An example of + a built-in method is ``alist.append()``, assuming *alist* is a list object. In + this case, the special read-only attribute :attr:`__self__` is set to the object + denoted by *list*. + + Class Types + Class types, or "new-style classes," are callable. These objects normally act + as factories for new instances of themselves, but variations are possible for + class types that override :meth:`__new__`. The arguments of the call are passed + to :meth:`__new__` and, in the typical case, to :meth:`__init__` to initialize + the new instance. + + Classic Classes + .. index:: + single: __init__() (object method) + object: class + object: class instance + object: instance + pair: class object; call + + Class objects are described below. When a class object is called, a new class + instance (also described below) is created and returned. This implies a call to + the class's :meth:`__init__` method if it has one. Any arguments are passed on + to the :meth:`__init__` method. If there is no :meth:`__init__` method, the + class must be called without arguments. + + Class instances + Class instances are described below. Class instances are callable only when the + class has a :meth:`__call__` method; ``x(arguments)`` is a shorthand for + ``x.__call__(arguments)``. + +Modules + .. index:: + statement: import + object: module + + Modules are imported by the :keyword:`import` statement (see section + :ref:`import`). A module object has a + namespace implemented by a dictionary object (this is the dictionary referenced + by the __globals__ attribute of functions defined in the module). Attribute + references are translated to lookups in this dictionary, e.g., ``m.x`` is + equivalent to ``m.__dict__["x"]``. A module object does not contain the code + object used to initialize the module (since it isn't needed once the + initialization is done). + + .. % + + Attribute assignment updates the module's namespace dictionary, e.g., ``m.x = + 1`` is equivalent to ``m.__dict__["x"] = 1``. + + .. index:: single: __dict__ (module attribute) + + Special read-only attribute: :attr:`__dict__` is the module's namespace as a + dictionary object. + + .. index:: + single: __name__ (module attribute) + single: __doc__ (module attribute) + single: __file__ (module attribute) + pair: module; namespace + + Predefined (writable) attributes: :attr:`__name__` is the module's name; + :attr:`__doc__` is the module's documentation string, or ``None`` if + unavailable; :attr:`__file__` is the pathname of the file from which the module + was loaded, if it was loaded from a file. The :attr:`__file__` attribute is not + present for C modules that are statically linked into the interpreter; for + extension modules loaded dynamically from a shared library, it is the pathname + of the shared library file. + +Classes + Class objects are created by class definitions (see section :ref:`class`). A + class has a namespace implemented by a dictionary object. Class attribute + references are translated to lookups in this dictionary, e.g., ``C.x`` is + translated to ``C.__dict__["x"]``. When the attribute name is not found + there, the attribute search continues in the base classes. The search is + depth-first, left-to-right in the order of occurrence in the base class list. + + .. index:: + object: class + object: class instance + object: instance + pair: class object; call + single: container + object: dictionary + pair: class; attribute + + When a class attribute reference (for class :class:`C`, say) would yield a + user-defined function object or an unbound user-defined method object whose + associated class is either :class:`C` or one of its base classes, it is + transformed into an unbound user-defined method object whose :attr:`im_class` + attribute is :class:`C`. When it would yield a class method object, it is + transformed into a bound user-defined method object whose :attr:`im_class` + and :attr:`im_self` attributes are both :class:`C`. When it would yield a + static method object, it is transformed into the object wrapped by the static + method object. See section :ref:`descriptors` for another way in which + attributes retrieved from a class may differ from those actually contained in + its :attr:`__dict__`. + + .. index:: triple: class; attribute; assignment + + Class attribute assignments update the class's dictionary, never the dictionary + of a base class. + + .. index:: pair: class object; call + + A class object can be called (see above) to yield a class instance (see below). + + .. index:: + single: __name__ (class attribute) + single: __module__ (class attribute) + single: __dict__ (class attribute) + single: __bases__ (class attribute) + single: __doc__ (class attribute) + + Special attributes: :attr:`__name__` is the class name; :attr:`__module__` is + the module name in which the class was defined; :attr:`__dict__` is the + dictionary containing the class's namespace; :attr:`__bases__` is a tuple + (possibly empty or a singleton) containing the base classes, in the order of + their occurrence in the base class list; :attr:`__doc__` is the class's + documentation string, or None if undefined. + +Class instances + .. index:: + object: class instance + object: instance + pair: class; instance + pair: class instance; attribute + + A class instance is created by calling a class object (see above). A class + instance has a namespace implemented as a dictionary which is the first place in + which attribute references are searched. When an attribute is not found there, + and the instance's class has an attribute by that name, the search continues + with the class attributes. If a class attribute is found that is a user-defined + function object or an unbound user-defined method object whose associated class + is the class (call it :class:`C`) of the instance for which the attribute + reference was initiated or one of its bases, it is transformed into a bound + user-defined method object whose :attr:`im_class` attribute is :class:`C` and + whose :attr:`im_self` attribute is the instance. Static method and class method + objects are also transformed, as if they had been retrieved from class + :class:`C`; see above under "Classes". See section :ref:`descriptors` for + another way in which attributes of a class retrieved via its instances may + differ from the objects actually stored in the class's :attr:`__dict__`. If no + class attribute is found, and the object's class has a :meth:`__getattr__` + method, that is called to satisfy the lookup. + + .. index:: triple: class instance; attribute; assignment + + Attribute assignments and deletions update the instance's dictionary, never a + class's dictionary. If the class has a :meth:`__setattr__` or + :meth:`__delattr__` method, this is called instead of updating the instance + dictionary directly. + + .. index:: + object: numeric + object: sequence + object: mapping + + Class instances can pretend to be numbers, sequences, or mappings if they have + methods with certain special names. See section :ref:`specialnames`. + + .. index:: + single: __dict__ (instance attribute) + single: __class__ (instance attribute) + + Special attributes: :attr:`__dict__` is the attribute dictionary; + :attr:`__class__` is the instance's class. + +Files + .. index:: + object: file + builtin: open + single: popen() (in module os) + single: makefile() (socket method) + single: sys.stdin + single: sys.stdout + single: sys.stderr + single: stdio + single: stdin (in module sys) + single: stdout (in module sys) + single: stderr (in module sys) + + A file object represents an open file. File objects are created by the + :func:`open` built-in function, and also by :func:`os.popen`, + :func:`os.fdopen`, and the :meth:`makefile` method of socket objects (and + perhaps by other functions or methods provided by extension modules). The + objects ``sys.stdin``, ``sys.stdout`` and ``sys.stderr`` are initialized to + file objects corresponding to the interpreter's standard input, output and + error streams. See :ref:`bltin-file-objects` for complete documentation of + file objects. + +Internal types + .. index:: + single: internal type + single: types, internal + + A few types used internally by the interpreter are exposed to the user. Their + definitions may change with future versions of the interpreter, but they are + mentioned here for completeness. + + Code objects + .. index:: + single: bytecode + object: code + + Code objects represent *byte-compiled* executable Python code, or *bytecode*. + The difference between a code object and a function object is that the function + object contains an explicit reference to the function's globals (the module in + which it was defined), while a code object contains no context; also the default + argument values are stored in the function object, not in the code object + (because they represent values calculated at run-time). Unlike function + objects, code objects are immutable and contain no references (directly or + indirectly) to mutable objects. + + Special read-only attributes: :attr:`co_name` gives the function name; + :attr:`co_argcount` is the number of positional arguments (including arguments + with default values); :attr:`co_nlocals` is the number of local variables used + by the function (including arguments); :attr:`co_varnames` is a tuple containing + the names of the local variables (starting with the argument names); + :attr:`co_cellvars` is a tuple containing the names of local variables that are + referenced by nested functions; :attr:`co_freevars` is a tuple containing the + names of free variables; :attr:`co_code` is a string representing the sequence + of bytecode instructions; :attr:`co_consts` is a tuple containing the literals + used by the bytecode; :attr:`co_names` is a tuple containing the names used by + the bytecode; :attr:`co_filename` is the filename from which the code was + compiled; :attr:`co_firstlineno` is the first line number of the function; + :attr:`co_lnotab` is a string encoding the mapping from byte code offsets to + line numbers (for details see the source code of the interpreter); + :attr:`co_stacksize` is the required stack size (including local variables); + :attr:`co_flags` is an integer encoding a number of flags for the interpreter. + + .. index:: + single: co_argcount (code object attribute) + single: co_code (code object attribute) + single: co_consts (code object attribute) + single: co_filename (code object attribute) + single: co_firstlineno (code object attribute) + single: co_flags (code object attribute) + single: co_lnotab (code object attribute) + single: co_name (code object attribute) + single: co_names (code object attribute) + single: co_nlocals (code object attribute) + single: co_stacksize (code object attribute) + single: co_varnames (code object attribute) + single: co_cellvars (code object attribute) + single: co_freevars (code object attribute) + + .. index:: object: generator + + The following flag bits are defined for :attr:`co_flags`: bit ``0x04`` is set if + the function uses the ``*arguments`` syntax to accept an arbitrary number of + positional arguments; bit ``0x08`` is set if the function uses the + ``**keywords`` syntax to accept arbitrary keyword arguments; bit ``0x20`` is set + if the function is a generator. + + Future feature declarations (``from __future__ import division``) also use bits + in :attr:`co_flags` to indicate whether a code object was compiled with a + particular feature enabled: bit ``0x2000`` is set if the function was compiled + with future division enabled; bits ``0x10`` and ``0x1000`` were used in earlier + versions of Python. + + Other bits in :attr:`co_flags` are reserved for internal use. + + .. index:: single: documentation string + + If a code object represents a function, the first item in :attr:`co_consts` is + the documentation string of the function, or ``None`` if undefined. + + Frame objects + .. index:: object: frame + + Frame objects represent execution frames. They may occur in traceback objects + (see below). + + .. index:: + single: f_back (frame attribute) + single: f_code (frame attribute) + single: f_globals (frame attribute) + single: f_locals (frame attribute) + single: f_lasti (frame attribute) + single: f_builtins (frame attribute) + + Special read-only attributes: :attr:`f_back` is to the previous stack frame + (towards the caller), or ``None`` if this is the bottom stack frame; + :attr:`f_code` is the code object being executed in this frame; :attr:`f_locals` + is the dictionary used to look up local variables; :attr:`f_globals` is used for + global variables; :attr:`f_builtins` is used for built-in (intrinsic) names; + :attr:`f_lasti` gives the precise instruction (this is an index into the + bytecode string of the code object). + + .. index:: + single: f_trace (frame attribute) + single: f_exc_type (frame attribute) + single: f_exc_value (frame attribute) + single: f_exc_traceback (frame attribute) + single: f_lineno (frame attribute) + + Special writable attributes: :attr:`f_trace`, if not ``None``, is a function + called at the start of each source code line (this is used by the debugger); + :attr:`f_exc_type`, :attr:`f_exc_value`, :attr:`f_exc_traceback` represent the + last exception raised in the parent frame provided another exception was ever + raised in the current frame (in all other cases they are None); :attr:`f_lineno` + is the current line number of the frame --- writing to this from within a trace + function jumps to the given line (only for the bottom-most frame). A debugger + can implement a Jump command (aka Set Next Statement) by writing to f_lineno. + + Traceback objects + .. index:: + object: traceback + pair: stack; trace + pair: exception; handler + pair: execution; stack + single: exc_info (in module sys) + single: exc_traceback (in module sys) + single: last_traceback (in module sys) + single: sys.exc_info + single: sys.last_traceback + + Traceback objects represent a stack trace of an exception. A traceback object + is created when an exception occurs. When the search for an exception handler + unwinds the execution stack, at each unwound level a traceback object is + inserted in front of the current traceback. When an exception handler is + entered, the stack trace is made available to the program. (See section + :ref:`try`.) It is accessible as the third item of the + tuple returned by ``sys.exc_info()``. When the program contains no suitable + handler, the stack trace is written (nicely formatted) to the standard error + stream; if the interpreter is interactive, it is also made available to the user + as ``sys.last_traceback``. + + .. index:: + single: tb_next (traceback attribute) + single: tb_frame (traceback attribute) + single: tb_lineno (traceback attribute) + single: tb_lasti (traceback attribute) + statement: try + + Special read-only attributes: :attr:`tb_next` is the next level in the stack + trace (towards the frame where the exception occurred), or ``None`` if there is + no next level; :attr:`tb_frame` points to the execution frame of the current + level; :attr:`tb_lineno` gives the line number where the exception occurred; + :attr:`tb_lasti` indicates the precise instruction. The line number and last + instruction in the traceback may differ from the line number of its frame object + if the exception occurred in a :keyword:`try` statement with no matching except + clause or with a finally clause. + + Slice objects + .. index:: builtin: slice + + Slice objects are used to represent slices when *extended slice syntax* is used. + This is a slice using two colons, or multiple slices or ellipses separated by + commas, e.g., ``a[i:j:step]``, ``a[i:j, k:l]``, or ``a[..., i:j]``. They are + also created by the built-in :func:`slice` function. + + .. index:: + single: start (slice object attribute) + single: stop (slice object attribute) + single: step (slice object attribute) + + Special read-only attributes: :attr:`start` is the lower bound; :attr:`stop` is + the upper bound; :attr:`step` is the step value; each is ``None`` if omitted. + These attributes can have any type. + + Slice objects support one method: + + + .. method:: slice.indices(self, length) + + This method takes a single integer argument *length* and computes information + about the extended slice that the slice object would describe if applied to a + sequence of *length* items. It returns a tuple of three integers; respectively + these are the *start* and *stop* indices and the *step* or stride length of the + slice. Missing or out-of-bounds indices are handled in a manner consistent with + regular slices. + + .. versionadded:: 2.3 + + Static method objects + Static method objects provide a way of defeating the transformation of function + objects to method objects described above. A static method object is a wrapper + around any other object, usually a user-defined method object. When a static + method object is retrieved from a class or a class instance, the object actually + returned is the wrapped object, which is not subject to any further + transformation. Static method objects are not themselves callable, although the + objects they wrap usually are. Static method objects are created by the built-in + :func:`staticmethod` constructor. + + Class method objects + A class method object, like a static method object, is a wrapper around another + object that alters the way in which that object is retrieved from classes and + class instances. The behaviour of class method objects upon such retrieval is + described above, under "User-defined methods". Class method objects are created + by the built-in :func:`classmethod` constructor. + + .. % Internal types + +.. % Types +.. % ========================================================================= + + +New-style and classic classes +============================= + +Classes and instances come in two flavors: old-style or classic, and new-style. + +Up to Python 2.1, old-style classes were the only flavour available to the user. +The concept of (old-style) class is unrelated to the concept of type: if *x* is +an instance of an old-style class, then ``x.__class__`` designates the class of +*x*, but ``type(x)`` is always ``<type 'instance'>``. This reflects the fact +that all old-style instances, independently of their class, are implemented with +a single built-in type, called ``instance``. + +New-style classes were introduced in Python 2.2 to unify classes and types. A +new-style class neither more nor less than a user-defined type. If *x* is an +instance of a new-style class, then ``type(x)`` is the same as ``x.__class__``. + +The major motivation for introducing new-style classes is to provide a unified +object model with a full meta-model. It also has a number of immediate +benefits, like the ability to subclass most built-in types, or the introduction +of "descriptors", which enable computed properties. + +For compatibility reasons, classes are still old-style by default. New-style +classes are created by specifying another new-style class (i.e. a type) as a +parent class, or the "top-level type" :class:`object` if no other parent is +needed. The behaviour of new-style classes differs from that of old-style +classes in a number of important details in addition to what :func:`type` +returns. Some of these changes are fundamental to the new object model, like +the way special methods are invoked. Others are "fixes" that could not be +implemented before for compatibility concerns, like the method resolution order +in case of multiple inheritance. + +This manual is not up-to-date with respect to new-style classes. For now, +please see http://www.python.org/doc/newstyle.html for more information. + +.. index:: + single: class + single: class + single: class + +The plan is to eventually drop old-style classes, leaving only the semantics of +new-style classes. This change will probably only be feasible in Python 3.0. +new-style classic old-style + +.. % ========================================================================= + + +.. _specialnames: + +Special method names +==================== + +.. index:: + pair: operator; overloading + single: __getitem__() (mapping object method) + +A class can implement certain operations that are invoked by special syntax +(such as arithmetic operations or subscripting and slicing) by defining methods +with special names. This is Python's approach to :dfn:`operator overloading`, +allowing classes to define their own behavior with respect to language +operators. For instance, if a class defines a method named :meth:`__getitem__`, +and ``x`` is an instance of this class, then ``x[i]`` is equivalent [#]_ to +``x.__getitem__(i)``. Except where mentioned, attempts to execute an operation +raise an exception when no appropriate method is defined. + +When implementing a class that emulates any built-in type, it is important that +the emulation only be implemented to the degree that it makes sense for the +object being modelled. For example, some sequences may work well with retrieval +of individual elements, but extracting a slice may not make sense. (One example +of this is the :class:`NodeList` interface in the W3C's Document Object Model.) + + +.. _customization: + +Basic customization +------------------- + + +.. method:: object.__new__(cls[, ...]) + + Called to create a new instance of class *cls*. :meth:`__new__` is a static + method (special-cased so you need not declare it as such) that takes the class + of which an instance was requested as its first argument. The remaining + arguments are those passed to the object constructor expression (the call to the + class). The return value of :meth:`__new__` should be the new object instance + (usually an instance of *cls*). + + Typical implementations create a new instance of the class by invoking the + superclass's :meth:`__new__` method using ``super(currentclass, + cls).__new__(cls[, ...])`` with appropriate arguments and then modifying the + newly-created instance as necessary before returning it. + + If :meth:`__new__` returns an instance of *cls*, then the new instance's + :meth:`__init__` method will be invoked like ``__init__(self[, ...])``, where + *self* is the new instance and the remaining arguments are the same as were + passed to :meth:`__new__`. + + If :meth:`__new__` does not return an instance of *cls*, then the new instance's + :meth:`__init__` method will not be invoked. + + :meth:`__new__` is intended mainly to allow subclasses of immutable types (like + int, str, or tuple) to customize instance creation. + + +.. method:: object.__init__(self[, ...]) + + .. index:: pair: class; constructor + + Called when the instance is created. The arguments are those passed to the + class constructor expression. If a base class has an :meth:`__init__` method, + the derived class's :meth:`__init__` method, if any, must explicitly call it to + ensure proper initialization of the base class part of the instance; for + example: ``BaseClass.__init__(self, [args...])``. As a special constraint on + constructors, no value may be returned; doing so will cause a :exc:`TypeError` + to be raised at runtime. + + +.. method:: object.__del__(self) + + .. index:: + single: destructor + statement: del + + Called when the instance is about to be destroyed. This is also called a + destructor. If a base class has a :meth:`__del__` method, the derived class's + :meth:`__del__` method, if any, must explicitly call it to ensure proper + deletion of the base class part of the instance. Note that it is possible + (though not recommended!) for the :meth:`__del__` method to postpone destruction + of the instance by creating a new reference to it. It may then be called at a + later time when this new reference is deleted. It is not guaranteed that + :meth:`__del__` methods are called for objects that still exist when the + interpreter exits. + + .. note:: + + ``del x`` doesn't directly call ``x.__del__()`` --- the former decrements + the reference count for ``x`` by one, and the latter is only called when + ``x``'s reference count reaches zero. Some common situations that may + prevent the reference count of an object from going to zero include: + circular references between objects (e.g., a doubly-linked list or a tree + data structure with parent and child pointers); a reference to the object + on the stack frame of a function that caught an exception (the traceback + stored in ``sys.exc_info()[2]`` keeps the stack frame alive); or a + reference to the object on the stack frame that raised an unhandled + exception in interactive mode (the traceback stored in + ``sys.last_traceback`` keeps the stack frame alive). The first situation + can only be remedied by explicitly breaking the cycles; the latter two + situations can be resolved by storing ``None`` in ``sys.last_traceback``. + Circular references which are garbage are detected when the option cycle + detector is enabled (it's on by default), but can only be cleaned up if + there are no Python- level :meth:`__del__` methods involved. Refer to the + documentation for the :mod:`gc` module for more information about how + :meth:`__del__` methods are handled by the cycle detector, particularly + the description of the ``garbage`` value. + + .. warning:: + + Due to the precarious circumstances under which :meth:`__del__` methods are + invoked, exceptions that occur during their execution are ignored, and a warning + is printed to ``sys.stderr`` instead. Also, when :meth:`__del__` is invoked in + response to a module being deleted (e.g., when execution of the program is + done), other globals referenced by the :meth:`__del__` method may already have + been deleted. For this reason, :meth:`__del__` methods should do the absolute + minimum needed to maintain external invariants. Starting with version 1.5, + Python guarantees that globals whose name begins with a single underscore are + deleted from their module before other globals are deleted; if no other + references to such globals exist, this may help in assuring that imported + modules are still available at the time when the :meth:`__del__` method is + called. + + +.. method:: object.__repr__(self) + + .. index:: builtin: repr + + Called by the :func:`repr` built-in function and by string conversions (reverse + quotes) to compute the "official" string representation of an object. If at all + possible, this should look like a valid Python expression that could be used to + recreate an object with the same value (given an appropriate environment). If + this is not possible, a string of the form ``<...some useful description...>`` + should be returned. The return value must be a string object. If a class + defines :meth:`__repr__` but not :meth:`__str__`, then :meth:`__repr__` is also + used when an "informal" string representation of instances of that class is + required. + + .. index:: + pair: string; conversion + pair: reverse; quotes + pair: backward; quotes + single: back-quotes + + This is typically used for debugging, so it is important that the representation + is information-rich and unambiguous. + + +.. method:: object.__str__(self) + + .. index:: + builtin: str + statement: print + + Called by the :func:`str` built-in function and by the :keyword:`print` + statement to compute the "informal" string representation of an object. This + differs from :meth:`__repr__` in that it does not have to be a valid Python + expression: a more convenient or concise representation may be used instead. + The return value must be a string object. + + +.. method:: object.__lt__(self, other) + object.__le__(self, other) + object.__eq__(self, other) + object.__ne__(self, other) + object.__gt__(self, other) + object.__ge__(self, other) + + .. versionadded:: 2.1 + + These are the so-called "rich comparison" methods, and are called for comparison + operators in preference to :meth:`__cmp__` below. The correspondence between + operator symbols and method names is as follows: ``x<y`` calls ``x.__lt__(y)``, + ``x<=y`` calls ``x.__le__(y)``, ``x==y`` calls ``x.__eq__(y)``, ``x!=y`` calls + ``x.__ne__(y)``, ``x>y`` calls ``x.__gt__(y)``, and ``x>=y`` calls + ``x.__ge__(y)``. + + A rich comparison method may return the singleton ``NotImplemented`` if it does + not implement the operation for a given pair of arguments. By convention, + ``False`` and ``True`` are returned for a successful comparison. However, these + methods can return any value, so if the comparison operator is used in a Boolean + context (e.g., in the condition of an ``if`` statement), Python will call + :func:`bool` on the value to determine if the result is true or false. + + There are no implied relationships among the comparison operators. The truth of + ``x==y`` does not imply that ``x!=y`` is false. Accordingly, when defining + :meth:`__eq__`, one should also define :meth:`__ne__` so that the operators will + behave as expected. + + There are no reflected (swapped-argument) versions of these methods (to be used + when the left argument does not support the operation but the right argument + does); rather, :meth:`__lt__` and :meth:`__gt__` are each other's reflection, + :meth:`__le__` and :meth:`__ge__` are each other's reflection, and + :meth:`__eq__` and :meth:`__ne__` are their own reflection. + + Arguments to rich comparison methods are never coerced. + + +.. method:: object.__cmp__(self, other) + + .. index:: + builtin: cmp + single: comparisons + + Called by comparison operations if rich comparison (see above) is not defined. + Should return a negative integer if ``self < other``, zero if ``self == other``, + a positive integer if ``self > other``. If no :meth:`__cmp__`, :meth:`__eq__` + or :meth:`__ne__` operation is defined, class instances are compared by object + identity ("address"). See also the description of :meth:`__hash__` for some + important notes on creating objects which support custom comparison operations + and are usable as dictionary keys. (Note: the restriction that exceptions are + not propagated by :meth:`__cmp__` has been removed since Python 1.5.) + + +.. method:: object.__rcmp__(self, other) + + .. versionchanged:: 2.1 + No longer supported. + + +.. method:: object.__hash__(self) + + .. index:: + object: dictionary + builtin: hash + + Called for the key object for dictionary operations, and by the built-in + function :func:`hash`. Should return a 32-bit integer usable as a hash value + for dictionary operations. The only required property is that objects which + compare equal have the same hash value; it is advised to somehow mix together + (e.g., using exclusive or) the hash values for the components of the object that + also play a part in comparison of objects. If a class does not define a + :meth:`__cmp__` method it should not define a :meth:`__hash__` operation either; + if it defines :meth:`__cmp__` or :meth:`__eq__` but not :meth:`__hash__`, its + instances will not be usable as dictionary keys. If a class defines mutable + objects and implements a :meth:`__cmp__` or :meth:`__eq__` method, it should not + implement :meth:`__hash__`, since the dictionary implementation requires that a + key's hash value is immutable (if the object's hash value changes, it will be in + the wrong hash bucket). + + .. versionchanged:: 2.5 + :meth:`__hash__` may now also return a long integer object; the 32-bit integer + is then derived from the hash of that object. + + .. index:: single: __cmp__() (object method) + + +.. method:: object.__bool__(self) + + .. index:: single: __len__() (mapping object method) + + Called to implement truth value testing, and the built-in operation ``bool()``; + should return ``False`` or ``True``. When this method is not defined, + :meth:`__len__` is called, if it is defined (see below) and ``True`` is returned + when the length is not zero. If a class defines neither :meth:`__len__` nor + :meth:`__bool__`, all its instances are considered true. + + +.. method:: object.__unicode__(self) + + .. index:: builtin: unicode + + Called to implement :func:`unicode` builtin; should return a Unicode object. + When this method is not defined, string conversion is attempted, and the result + of string conversion is converted to Unicode using the system default encoding. + + +.. _attribute-access: + +Customizing attribute access +---------------------------- + +The following methods can be defined to customize the meaning of attribute +access (use of, assignment to, or deletion of ``x.name``) for class instances. + + +.. method:: object.__getattr__(self, name) + + Called when an attribute lookup has not found the attribute in the usual places + (i.e. it is not an instance attribute nor is it found in the class tree for + ``self``). ``name`` is the attribute name. This method should return the + (computed) attribute value or raise an :exc:`AttributeError` exception. + + .. index:: single: __setattr__() (object method) + + Note that if the attribute is found through the normal mechanism, + :meth:`__getattr__` is not called. (This is an intentional asymmetry between + :meth:`__getattr__` and :meth:`__setattr__`.) This is done both for efficiency + reasons and because otherwise :meth:`__setattr__` would have no way to access + other attributes of the instance. Note that at least for instance variables, + you can fake total control by not inserting any values in the instance attribute + dictionary (but instead inserting them in another object). See the + :meth:`__getattribute__` method below for a way to actually get total control in + new-style classes. + + +.. method:: object.__setattr__(self, name, value) + + Called when an attribute assignment is attempted. This is called instead of the + normal mechanism (i.e. store the value in the instance dictionary). *name* is + the attribute name, *value* is the value to be assigned to it. + + .. index:: single: __dict__ (instance attribute) + + If :meth:`__setattr__` wants to assign to an instance attribute, it should not + simply execute ``self.name = value`` --- this would cause a recursive call to + itself. Instead, it should insert the value in the dictionary of instance + attributes, e.g., ``self.__dict__[name] = value``. For new-style classes, + rather than accessing the instance dictionary, it should call the base class + method with the same name, for example, ``object.__setattr__(self, name, + value)``. + + +.. method:: object.__delattr__(self, name) + + Like :meth:`__setattr__` but for attribute deletion instead of assignment. This + should only be implemented if ``del obj.name`` is meaningful for the object. + + +.. _new-style-attribute-access: + +More attribute access for new-style classes +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The following methods only apply to new-style classes. + + +.. method:: object.__getattribute__(self, name) + + Called unconditionally to implement attribute accesses for instances of the + class. If the class also defines :meth:`__getattr__`, the latter will not be + called unless :meth:`__getattribute__` either calls it explicitly or raises an + :exc:`AttributeError`. This method should return the (computed) attribute value + or raise an :exc:`AttributeError` exception. In order to avoid infinite + recursion in this method, its implementation should always call the base class + method with the same name to access any attributes it needs, for example, + ``object.__getattribute__(self, name)``. + + +.. _descriptors: + +Implementing Descriptors +^^^^^^^^^^^^^^^^^^^^^^^^ + +The following methods only apply when an instance of the class containing the +method (a so-called *descriptor* class) appears in the class dictionary of +another new-style class, known as the *owner* class. In the examples below, "the +attribute" refers to the attribute whose name is the key of the property in the +owner class' ``__dict__``. Descriptors can only be implemented as new-style +classes themselves. + + +.. method:: object.__get__(self, instance, owner) + + Called to get the attribute of the owner class (class attribute access) or of an + instance of that class (instance attribute access). *owner* is always the owner + class, while *instance* is the instance that the attribute was accessed through, + or ``None`` when the attribute is accessed through the *owner*. This method + should return the (computed) attribute value or raise an :exc:`AttributeError` + exception. + + +.. method:: object.__set__(self, instance, value) + + Called to set the attribute on an instance *instance* of the owner class to a + new value, *value*. + + +.. method:: object.__delete__(self, instance) + + Called to delete the attribute on an instance *instance* of the owner class. + + +.. _descriptor-invocation: + +Invoking Descriptors +^^^^^^^^^^^^^^^^^^^^ + +In general, a descriptor is an object attribute with "binding behavior", one +whose attribute access has been overridden by methods in the descriptor +protocol: :meth:`__get__`, :meth:`__set__`, and :meth:`__delete__`. If any of +those methods are defined for an object, it is said to be a descriptor. + +The default behavior for attribute access is to get, set, or delete the +attribute from an object's dictionary. For instance, ``a.x`` has a lookup chain +starting with ``a.__dict__['x']``, then ``type(a).__dict__['x']``, and +continuing through the base classes of ``type(a)`` excluding metaclasses. + +However, if the looked-up value is an object defining one of the descriptor +methods, then Python may override the default behavior and invoke the descriptor +method instead. Where this occurs in the precedence chain depends on which +descriptor methods were defined and how they were called. Note that descriptors +are only invoked for new style objects or classes (ones that subclass +:class:`object()` or :class:`type()`). + +The starting point for descriptor invocation is a binding, ``a.x``. How the +arguments are assembled depends on ``a``: + +Direct Call + The simplest and least common call is when user code directly invokes a + descriptor method: ``x.__get__(a)``. + +Instance Binding + If binding to a new-style object instance, ``a.x`` is transformed into the call: + ``type(a).__dict__['x'].__get__(a, type(a))``. + +Class Binding + If binding to a new-style class, ``A.x`` is transformed into the call: + ``A.__dict__['x'].__get__(None, A)``. + +Super Binding + If ``a`` is an instance of :class:`super`, then the binding ``super(B, + obj).m()`` searches ``obj.__class__.__mro__`` for the base class ``A`` + immediately preceding ``B`` and then invokes the descriptor with the call: + ``A.__dict__['m'].__get__(obj, A)``. + +For instance bindings, the precedence of descriptor invocation depends on the +which descriptor methods are defined. Data descriptors define both +:meth:`__get__` and :meth:`__set__`. Non-data descriptors have just the +:meth:`__get__` method. Data descriptors always override a redefinition in an +instance dictionary. In contrast, non-data descriptors can be overridden by +instances. + +Python methods (including :func:`staticmethod` and :func:`classmethod`) are +implemented as non-data descriptors. Accordingly, instances can redefine and +override methods. This allows individual instances to acquire behaviors that +differ from other instances of the same class. + +The :func:`property` function is implemented as a data descriptor. Accordingly, +instances cannot override the behavior of a property. + + +.. _slots: + +__slots__ +^^^^^^^^^ + +By default, instances of both old and new-style classes have a dictionary for +attribute storage. This wastes space for objects having very few instance +variables. The space consumption can become acute when creating large numbers +of instances. + +The default can be overridden by defining *__slots__* in a new-style class +definition. The *__slots__* declaration takes a sequence of instance variables +and reserves just enough space in each instance to hold a value for each +variable. Space is saved because *__dict__* is not created for each instance. + + +.. data:: __slots__ + + This class variable can be assigned a string, iterable, or sequence of strings + with variable names used by instances. If defined in a new-style class, + *__slots__* reserves space for the declared variables and prevents the automatic + creation of *__dict__* and *__weakref__* for each instance. + + .. versionadded:: 2.2 + +Notes on using *__slots__* + +* Without a *__dict__* variable, instances cannot be assigned new variables not + listed in the *__slots__* definition. Attempts to assign to an unlisted + variable name raises :exc:`AttributeError`. If dynamic assignment of new + variables is desired, then add ``'__dict__'`` to the sequence of strings in the + *__slots__* declaration. + + .. versionchanged:: 2.3 + Previously, adding ``'__dict__'`` to the *__slots__* declaration would not + enable the assignment of new attributes not specifically listed in the sequence + of instance variable names. + +* Without a *__weakref__* variable for each instance, classes defining + *__slots__* do not support weak references to its instances. If weak reference + support is needed, then add ``'__weakref__'`` to the sequence of strings in the + *__slots__* declaration. + + .. versionchanged:: 2.3 + Previously, adding ``'__weakref__'`` to the *__slots__* declaration would not + enable support for weak references. + +* *__slots__* are implemented at the class level by creating descriptors + (:ref:`descriptors`) for each variable name. As a result, class attributes + cannot be used to set default values for instance variables defined by + *__slots__*; otherwise, the class attribute would overwrite the descriptor + assignment. + +* If a class defines a slot also defined in a base class, the instance variable + defined by the base class slot is inaccessible (except by retrieving its + descriptor directly from the base class). This renders the meaning of the + program undefined. In the future, a check may be added to prevent this. + +* The action of a *__slots__* declaration is limited to the class where it is + defined. As a result, subclasses will have a *__dict__* unless they also define + *__slots__*. + +* *__slots__* do not work for classes derived from "variable-length" built-in + types such as :class:`long`, :class:`str` and :class:`tuple`. + +* Any non-string iterable may be assigned to *__slots__*. Mappings may also be + used; however, in the future, special meaning may be assigned to the values + corresponding to each key. + +* *__class__* assignment works only if both classes have the same *__slots__*. + + .. versionchanged:: 2.6 + Previously, *__class__* assignment raised an error if either new or old class + had *__slots__*. + + +.. _metaclasses: + +Customizing class creation +-------------------------- + +By default, new-style classes are constructed using :func:`type`. A class +definition is read into a separate namespace and the value of class name is +bound to the result of ``type(name, bases, dict)``. + +When the class definition is read, if *__metaclass__* is defined then the +callable assigned to it will be called instead of :func:`type`. The allows +classes or functions to be written which monitor or alter the class creation +process: + +* Modifying the class dictionary prior to the class being created. + +* Returning an instance of another class -- essentially performing the role of a + factory function. + + +.. data:: __metaclass__ + + This variable can be any callable accepting arguments for ``name``, ``bases``, + and ``dict``. Upon class creation, the callable is used instead of the built-in + :func:`type`. + + .. versionadded:: 2.2 + +The appropriate metaclass is determined by the following precedence rules: + +* If ``dict['__metaclass__']`` exists, it is used. + +* Otherwise, if there is at least one base class, its metaclass is used (this + looks for a *__class__* attribute first and if not found, uses its type). + +* Otherwise, if a global variable named __metaclass__ exists, it is used. + +* Otherwise, the old-style, classic metaclass (types.ClassType) is used. + +The potential uses for metaclasses are boundless. Some ideas that have been +explored including logging, interface checking, automatic delegation, automatic +property creation, proxies, frameworks, and automatic resource +locking/synchronization. + + +.. _callable-types: + +Emulating callable objects +-------------------------- + + +.. method:: object.__call__(self[, args...]) + + .. index:: pair: call; instance + + Called when the instance is "called" as a function; if this method is defined, + ``x(arg1, arg2, ...)`` is a shorthand for ``x.__call__(arg1, arg2, ...)``. + + +.. _sequence-types: + +Emulating container types +------------------------- + +The following methods can be defined to implement container objects. Containers +usually are sequences (such as lists or tuples) or mappings (like dictionaries), +but can represent other containers as well. The first set of methods is used +either to emulate a sequence or to emulate a mapping; the difference is that for +a sequence, the allowable keys should be the integers *k* for which ``0 <= k < +N`` where *N* is the length of the sequence, or slice objects, which define a +range of items. (For backwards compatibility, the method :meth:`__getslice__` +(see below) can also be defined to handle simple, but not extended slices.) It +is also recommended that mappings provide the methods :meth:`keys`, +:meth:`values`, :meth:`items`, :meth:`has_key`, :meth:`get`, :meth:`clear`, +:meth:`setdefault`, :meth:`iterkeys`, :meth:`itervalues`, :meth:`iteritems`, +:meth:`pop`, :meth:`popitem`, :meth:`copy`, and :meth:`update` behaving similar +to those for Python's standard dictionary objects. The :mod:`UserDict` module +provides a :class:`DictMixin` class to help create those methods from a base set +of :meth:`__getitem__`, :meth:`__setitem__`, :meth:`__delitem__`, and +:meth:`keys`. Mutable sequences should provide methods :meth:`append`, +:meth:`count`, :meth:`index`, :meth:`extend`, :meth:`insert`, :meth:`pop`, +:meth:`remove`, :meth:`reverse` and :meth:`sort`, like Python standard list +objects. Finally, sequence types should implement addition (meaning +concatenation) and multiplication (meaning repetition) by defining the methods +:meth:`__add__`, :meth:`__radd__`, :meth:`__iadd__`, :meth:`__mul__`, +:meth:`__rmul__` and :meth:`__imul__` described below; they should not define +other numerical operators. It is recommended that both mappings and sequences +implement the :meth:`__contains__` method to allow efficient use of the ``in`` +operator; for mappings, ``in`` should be equivalent of :meth:`has_key`; for +sequences, it should search through the values. It is further recommended that +both mappings and sequences implement the :meth:`__iter__` method to allow +efficient iteration through the container; for mappings, :meth:`__iter__` should +be the same as :meth:`iterkeys`; for sequences, it should iterate through the +values. + + +.. method:: object.__len__(self) + + .. index:: + builtin: len + single: __bool__() (object method) + + Called to implement the built-in function :func:`len`. Should return the length + of the object, an integer ``>=`` 0. Also, an object that doesn't define a + :meth:`__bool__` method and whose :meth:`__len__` method returns zero is + considered to be false in a Boolean context. + + +.. method:: object.__getitem__(self, key) + + .. index:: object: slice + + Called to implement evaluation of ``self[key]``. For sequence types, the + accepted keys should be integers and slice objects. Note that the special + interpretation of negative indexes (if the class wishes to emulate a sequence + type) is up to the :meth:`__getitem__` method. If *key* is of an inappropriate + type, :exc:`TypeError` may be raised; if of a value outside the set of indexes + for the sequence (after any special interpretation of negative values), + :exc:`IndexError` should be raised. For mapping types, if *key* is missing (not + in the container), :exc:`KeyError` should be raised. + + .. note:: + + :keyword:`for` loops expect that an :exc:`IndexError` will be raised for illegal + indexes to allow proper detection of the end of the sequence. + + +.. method:: object.__setitem__(self, key, value) + + Called to implement assignment to ``self[key]``. Same note as for + :meth:`__getitem__`. This should only be implemented for mappings if the + objects support changes to the values for keys, or if new keys can be added, or + for sequences if elements can be replaced. The same exceptions should be raised + for improper *key* values as for the :meth:`__getitem__` method. + + +.. method:: object.__delitem__(self, key) + + Called to implement deletion of ``self[key]``. Same note as for + :meth:`__getitem__`. This should only be implemented for mappings if the + objects support removal of keys, or for sequences if elements can be removed + from the sequence. The same exceptions should be raised for improper *key* + values as for the :meth:`__getitem__` method. + + +.. method:: object.__iter__(self) + + This method is called when an iterator is required for a container. This method + should return a new iterator object that can iterate over all the objects in the + container. For mappings, it should iterate over the keys of the container, and + should also be made available as the method :meth:`iterkeys`. + + Iterator objects also need to implement this method; they are required to return + themselves. For more information on iterator objects, see :ref:`typeiter`. + +The membership test operators (:keyword:`in` and :keyword:`not in`) are normally +implemented as an iteration through a sequence. However, container objects can +supply the following special method with a more efficient implementation, which +also does not require the object be a sequence. + + +.. method:: object.__contains__(self, item) + + Called to implement membership test operators. Should return true if *item* is + in *self*, false otherwise. For mapping objects, this should consider the keys + of the mapping rather than the values or the key-item pairs. + + +.. _sequence-methods: + +Additional methods for emulation of sequence types +-------------------------------------------------- + +The following optional methods can be defined to further emulate sequence +objects. Immutable sequences methods should at most only define +:meth:`__getslice__`; mutable sequences might define all three methods. + + +.. method:: object.__getslice__(self, i, j) + + .. deprecated:: 2.0 + Support slice objects as parameters to the :meth:`__getitem__` method. + + Called to implement evaluation of ``self[i:j]``. The returned object should be + of the same type as *self*. Note that missing *i* or *j* in the slice + expression are replaced by zero or ``sys.maxint``, respectively. If negative + indexes are used in the slice, the length of the sequence is added to that + index. If the instance does not implement the :meth:`__len__` method, an + :exc:`AttributeError` is raised. No guarantee is made that indexes adjusted this + way are not still negative. Indexes which are greater than the length of the + sequence are not modified. If no :meth:`__getslice__` is found, a slice object + is created instead, and passed to :meth:`__getitem__` instead. + + +.. method:: object.__setslice__(self, i, j, sequence) + + Called to implement assignment to ``self[i:j]``. Same notes for *i* and *j* as + for :meth:`__getslice__`. + + This method is deprecated. If no :meth:`__setslice__` is found, or for extended + slicing of the form ``self[i:j:k]``, a slice object is created, and passed to + :meth:`__setitem__`, instead of :meth:`__setslice__` being called. + + +.. method:: object.__delslice__(self, i, j) + + Called to implement deletion of ``self[i:j]``. Same notes for *i* and *j* as for + :meth:`__getslice__`. This method is deprecated. If no :meth:`__delslice__` is + found, or for extended slicing of the form ``self[i:j:k]``, a slice object is + created, and passed to :meth:`__delitem__`, instead of :meth:`__delslice__` + being called. + +Notice that these methods are only invoked when a single slice with a single +colon is used, and the slice method is available. For slice operations +involving extended slice notation, or in absence of the slice methods, +:meth:`__getitem__`, :meth:`__setitem__` or :meth:`__delitem__` is called with a +slice object as argument. + +The following example demonstrate how to make your program or module compatible +with earlier versions of Python (assuming that methods :meth:`__getitem__`, +:meth:`__setitem__` and :meth:`__delitem__` support slice objects as +arguments):: + + class MyClass: + ... + def __getitem__(self, index): + ... + def __setitem__(self, index, value): + ... + def __delitem__(self, index): + ... + + if sys.version_info < (2, 0): + # They won't be defined if version is at least 2.0 final + + def __getslice__(self, i, j): + return self[max(0, i):max(0, j):] + def __setslice__(self, i, j, seq): + self[max(0, i):max(0, j):] = seq + def __delslice__(self, i, j): + del self[max(0, i):max(0, j):] + ... + +Note the calls to :func:`max`; these are necessary because of the handling of +negative indices before the :meth:`__\*slice__` methods are called. When +negative indexes are used, the :meth:`__\*item__` methods receive them as +provided, but the :meth:`__\*slice__` methods get a "cooked" form of the index +values. For each negative index value, the length of the sequence is added to +the index before calling the method (which may still result in a negative +index); this is the customary handling of negative indexes by the built-in +sequence types, and the :meth:`__\*item__` methods are expected to do this as +well. However, since they should already be doing that, negative indexes cannot +be passed in; they must be constrained to the bounds of the sequence before +being passed to the :meth:`__\*item__` methods. Calling ``max(0, i)`` +conveniently returns the proper value. + + +.. _numeric-types: + +Emulating numeric types +----------------------- + +The following methods can be defined to emulate numeric objects. Methods +corresponding to operations that are not supported by the particular kind of +number implemented (e.g., bitwise operations for non-integral numbers) should be +left undefined. + + +.. method:: object.__add__(self, other) + object.__sub__(self, other) + object.__mul__(self, other) + object.__floordiv__(self, other) + object.__mod__(self, other) + object.__divmod__(self, other) + object.__pow__(self, other[, modulo]) + object.__lshift__(self, other) + object.__rshift__(self, other) + object.__and__(self, other) + object.__xor__(self, other) + object.__or__(self, other) + + .. index:: + builtin: divmod + builtin: pow + builtin: pow + + These methods are called to implement the binary arithmetic operations (``+``, + ``-``, ``*``, ``//``, ``%``, :func:`divmod`, :func:`pow`, ``**``, ``<<``, + ``>>``, ``&``, ``^``, ``|``). For instance, to evaluate the expression + *x*``+``*y*, where *x* is an instance of a class that has an :meth:`__add__` + method, ``x.__add__(y)`` is called. The :meth:`__divmod__` method should be the + equivalent to using :meth:`__floordiv__` and :meth:`__mod__`; it should not be + related to :meth:`__truediv__` (described below). Note that :meth:`__pow__` + should be defined to accept an optional third argument if the ternary version of + the built-in :func:`pow` function is to be supported. + + If one of those methods does not support the operation with the supplied + arguments, it should return ``NotImplemented``. + + +.. method:: object.__div__(self, other) + object.__truediv__(self, other) + + The division operator (``/``) is implemented by these methods. The + :meth:`__truediv__` method is used when ``__future__.division`` is in effect, + otherwise :meth:`__div__` is used. If only one of these two methods is defined, + the object will not support division in the alternate context; :exc:`TypeError` + will be raised instead. + + +.. method:: object.__radd__(self, other) + object.__rsub__(self, other) + object.__rmul__(self, other) + object.__rdiv__(self, other) + object.__rtruediv__(self, other) + object.__rfloordiv__(self, other) + object.__rmod__(self, other) + object.__rdivmod__(self, other) + object.__rpow__(self, other) + object.__rlshift__(self, other) + object.__rrshift__(self, other) + object.__rand__(self, other) + object.__rxor__(self, other) + object.__ror__(self, other) + + .. index:: + builtin: divmod + builtin: pow + + These methods are called to implement the binary arithmetic operations (``+``, + ``-``, ``*``, ``/``, ``%``, :func:`divmod`, :func:`pow`, ``**``, ``<<``, ``>>``, + ``&``, ``^``, ``|``) with reflected (swapped) operands. These functions are + only called if the left operand does not support the corresponding operation and + the operands are of different types. [#]_ For instance, to evaluate the + expression *x*``-``*y*, where *y* is an instance of a class that has an + :meth:`__rsub__` method, ``y.__rsub__(x)`` is called if ``x.__sub__(y)`` returns + *NotImplemented*. + + .. index:: builtin: pow + + Note that ternary :func:`pow` will not try calling :meth:`__rpow__` (the + coercion rules would become too complicated). + + .. note:: + + If the right operand's type is a subclass of the left operand's type and that + subclass provides the reflected method for the operation, this method will be + called before the left operand's non-reflected method. This behavior allows + subclasses to override their ancestors' operations. + + +.. method:: object.__iadd__(self, other) + object.__isub__(self, other) + object.__imul__(self, other) + object.__idiv__(self, other) + object.__itruediv__(self, other) + object.__ifloordiv__(self, other) + object.__imod__(self, other) + object.__ipow__(self, other[, modulo]) + object.__ilshift__(self, other) + object.__irshift__(self, other) + object.__iand__(self, other) + object.__ixor__(self, other) + object.__ior__(self, other) + + These methods are called to implement the augmented arithmetic operations + (``+=``, ``-=``, ``*=``, ``/=``, ``//=``, ``%=``, ``**=``, ``<<=``, ``>>=``, + ``&=``, ``^=``, ``|=``). These methods should attempt to do the operation + in-place (modifying *self*) and return the result (which could be, but does + not have to be, *self*). If a specific method is not defined, the augmented + operation falls back to the normal methods. For instance, to evaluate the + expression *x*``+=``*y*, where *x* is an instance of a class that has an + :meth:`__iadd__` method, ``x.__iadd__(y)`` is called. If *x* is an instance + of a class that does not define a :meth:`__iadd__` method, ``x.__add__(y)`` + and ``y.__radd__(x)`` are considered, as with the evaluation of *x*``+``*y*. + + +.. method:: object.__neg__(self) + object.__pos__(self) + object.__abs__(self) + object.__invert__(self) + + .. index:: builtin: abs + + Called to implement the unary arithmetic operations (``-``, ``+``, :func:`abs` + and ``~``). + + +.. method:: object.__complex__(self) + object.__int__(self) + object.__long__(self) + object.__float__(self) + + .. index:: + builtin: complex + builtin: int + builtin: long + builtin: float + + Called to implement the built-in functions :func:`complex`, :func:`int`, + :func:`long`, and :func:`float`. Should return a value of the appropriate type. + + +.. method:: object.__index__(self) + + Called to implement :func:`operator.index`. Also called whenever Python needs + an integer object (such as in slicing, or in the built-in :func:`bin`, + :func:`hex` and :func:`oct` functions). Must return an integer (int or long). + + .. versionadded:: 2.5 + + +.. _context-managers: + +With Statement Context Managers +------------------------------- + +.. versionadded:: 2.5 + +A :dfn:`context manager` is an object that defines the runtime context to be +established when executing a :keyword:`with` statement. The context manager +handles the entry into, and the exit from, the desired runtime context for the +execution of the block of code. Context managers are normally invoked using the +:keyword:`with` statement (described in section :ref:`with`), but can also be +used by directly invoking their methods. + +.. index:: + statement: with + single: context manager + +Typical uses of context managers include saving and restoring various kinds of +global state, locking and unlocking resources, closing opened files, etc. + +For more information on context managers, see :ref:`typecontextmanager`. + + +.. method:: object.__enter__(self) + + Enter the runtime context related to this object. The :keyword:`with` statement + will bind this method's return value to the target(s) specified in the + :keyword:`as` clause of the statement, if any. + + +.. method:: object.__exit__(self, exc_type, exc_value, traceback) + + Exit the runtime context related to this object. The parameters describe the + exception that caused the context to be exited. If the context was exited + without an exception, all three arguments will be :const:`None`. + + If an exception is supplied, and the method wishes to suppress the exception + (i.e., prevent it from being propagated), it should return a true value. + Otherwise, the exception will be processed normally upon exit from this method. + + Note that :meth:`__exit__` methods should not reraise the passed-in exception; + this is the caller's responsibility. + + +.. seealso:: + + :pep:`0343` - The "with" statement + The specification, background, and examples for the Python :keyword:`with` + statement. + +.. rubric:: Footnotes + +.. [#] Since Python 2.2, a gradual merging of types and classes has been started that + makes this and a few other assertions made in this manual not 100% accurate and + complete: for example, it *is* now possible in some cases to change an object's + type, under certain controlled conditions. Until this manual undergoes + extensive revision, it must now be taken as authoritative only regarding + "classic classes", that are still the default, for compatibility purposes, in + Python 2.2 and 2.3. For more information, see + http://www.python.org/doc/newstyle.html. + +.. [#] This, and other statements, are only roughly true for instances of new-style + classes. + +.. [#] For operands of the same type, it is assumed that if the non-reflected method + (such as :meth:`__add__`) fails the operation is not supported, which is why the + reflected method is not called. + diff --git a/Doc/reference/executionmodel.rst b/Doc/reference/executionmodel.rst new file mode 100644 index 0000000000..27802c8b85 --- /dev/null +++ b/Doc/reference/executionmodel.rst @@ -0,0 +1,232 @@ + +.. _execmodel: + +*************** +Execution model +*************** + +.. index:: single: execution model + + +.. _naming: + +Naming and binding +================== + +.. index:: + pair: code; block + single: namespace + single: scope + +.. index:: + single: name + pair: binding; name + +:dfn:`Names` refer to objects. Names are introduced by name binding operations. +Each occurrence of a name in the program text refers to the :dfn:`binding` of +that name established in the innermost function block containing the use. + +.. index:: single: block + +A :dfn:`block` is a piece of Python program text that is executed as a unit. +The following are blocks: a module, a function body, and a class definition. +Each command typed interactively is a block. A script file (a file given as +standard input to the interpreter or specified on the interpreter command line +the first argument) is a code block. A script command (a command specified on +the interpreter command line with the '**-c**' option) is a code block. The string +argument passed to the built-in functions :func:`eval` and :func:`exec` is a +code block. The expression read and evaluated by the built-in function +:func:`input` is a code block. + +.. index:: pair: execution; frame + +A code block is executed in an :dfn:`execution frame`. A frame contains some +administrative information (used for debugging) and determines where and how +execution continues after the code block's execution has completed. + +.. index:: single: scope + +A :dfn:`scope` defines the visibility of a name within a block. If a local +variable is defined in a block, its scope includes that block. If the +definition occurs in a function block, the scope extends to any blocks contained +within the defining one, unless a contained block introduces a different binding +for the name. The scope of names defined in a class block is limited to the +class block; it does not extend to the code blocks of methods. + +.. index:: single: environment + +When a name is used in a code block, it is resolved using the nearest enclosing +scope. The set of all such scopes visible to a code block is called the block's +:dfn:`environment`. + +.. index:: pair: free; variable + +If a name is bound in a block, it is a local variable of that block. If a name +is bound at the module level, it is a global variable. (The variables of the +module code block are local and global.) If a variable is used in a code block +but not defined there, it is a :dfn:`free variable`. + +.. index:: + single: NameError (built-in exception) + single: UnboundLocalError + +When a name is not found at all, a :exc:`NameError` exception is raised. If the +name refers to a local variable that has not been bound, a +:exc:`UnboundLocalError` exception is raised. :exc:`UnboundLocalError` is a +subclass of :exc:`NameError`. + +.. index:: statement: from + +The following constructs bind names: formal parameters to functions, +:keyword:`import` statements, class and function definitions (these bind the +class or function name in the defining block), and targets that are identifiers +if occurring in an assignment, :keyword:`for` loop header, or in the second +position of an :keyword:`except` clause header. The :keyword:`import` statement +of the form "``from ...import *``" binds all names defined in the imported +module, except those beginning with an underscore. This form may only be used +at the module level. + +A target occurring in a :keyword:`del` statement is also considered bound for +this purpose (though the actual semantics are to unbind the name). It is +illegal to unbind a name that is referenced by an enclosing scope; the compiler +will report a :exc:`SyntaxError`. + +Each assignment or import statement occurs within a block defined by a class or +function definition or at the module level (the top-level code block). + +If a name binding operation occurs anywhere within a code block, all uses of the +name within the block are treated as references to the current block. This can +lead to errors when a name is used within a block before it is bound. This rule +is subtle. Python lacks declarations and allows name binding operations to +occur anywhere within a code block. The local variables of a code block can be +determined by scanning the entire text of the block for name binding operations. + +If the global statement occurs within a block, all uses of the name specified in +the statement refer to the binding of that name in the top-level namespace. +Names are resolved in the top-level namespace by searching the global namespace, +i.e. the namespace of the module containing the code block, and the builtin +namespace, the namespace of the module :mod:`__builtin__`. The global namespace +is searched first. If the name is not found there, the builtin namespace is +searched. The global statement must precede all uses of the name. + +.. index:: pair: restricted; execution + +The built-in namespace associated with the execution of a code block is actually +found by looking up the name ``__builtins__`` in its global namespace; this +should be a dictionary or a module (in the latter case the module's dictionary +is used). By default, when in the :mod:`__main__` module, ``__builtins__`` is +the built-in module :mod:`__builtin__` (note: no 's'); when in any other module, +``__builtins__`` is an alias for the dictionary of the :mod:`__builtin__` module +itself. ``__builtins__`` can be set to a user-created dictionary to create a +weak form of restricted execution. + +.. note:: + + Users should not touch ``__builtins__``; it is strictly an implementation + detail. Users wanting to override values in the built-in namespace should + :keyword:`import` the :mod:`__builtin__` (no 's') module and modify its + attributes appropriately. + +.. index:: module: __main__ + +The namespace for a module is automatically created the first time a module is +imported. The main module for a script is always called :mod:`__main__`. + +The global statement has the same scope as a name binding operation in the same +block. If the nearest enclosing scope for a free variable contains a global +statement, the free variable is treated as a global. + +A class definition is an executable statement that may use and define names. +These references follow the normal rules for name resolution. The namespace of +the class definition becomes the attribute dictionary of the class. Names +defined at the class scope are not visible in methods. + + +.. _dynamic-features: + +Interaction with dynamic features +--------------------------------- + +There are several cases where Python statements are illegal when used in +conjunction with nested scopes that contain free variables. + +If a variable is referenced in an enclosing scope, it is illegal to delete the +name. An error will be reported at compile time. + +If the wild card form of import --- ``import *`` --- is used in a function and +the function contains or is a nested block with free variables, the compiler +will raise a :exc:`SyntaxError`. + +The :func:`eval` and :func:`exec` functions do +not have access to the full environment for resolving names. Names may be +resolved in the local and global namespaces of the caller. Free variables are +not resolved in the nearest enclosing namespace, but in the global namespace. +[#]_ The :func:`exec` and :func:`eval` functions have optional +arguments to override the global and local namespace. If only one namespace is +specified, it is used for both. + + +.. _exceptions: + +Exceptions +========== + +.. index:: single: exception + +.. index:: + single: raise an exception + single: handle an exception + single: exception handler + single: errors + single: error handling + +Exceptions are a means of breaking out of the normal flow of control of a code +block in order to handle errors or other exceptional conditions. An exception +is *raised* at the point where the error is detected; it may be *handled* by the +surrounding code block or by any code block that directly or indirectly invoked +the code block where the error occurred. + +The Python interpreter raises an exception when it detects a run-time error +(such as division by zero). A Python program can also explicitly raise an +exception with the :keyword:`raise` statement. Exception handlers are specified +with the :keyword:`try` ... :keyword:`except` statement. The :keyword:`try` ... +:keyword:`finally` statement specifies cleanup code which does not handle the +exception, but is executed whether an exception occurred or not in the preceding +code. + +.. index:: single: termination model + +Python uses the "termination" model of error handling: an exception handler can +find out what happened and continue execution at an outer level, but it cannot +repair the cause of the error and retry the failing operation (except by +re-entering the offending piece of code from the top). + +.. index:: single: SystemExit (built-in exception) + +When an exception is not handled at all, the interpreter terminates execution of +the program, or returns to its interactive main loop. In either case, it prints +a stack backtrace, except when the exception is :exc:`SystemExit`. + +Exceptions are identified by class instances. The :keyword:`except` clause is +selected depending on the class of the instance: it must reference the class of +the instance or a base class thereof. The instance can be received by the +handler and can carry additional information about the exceptional condition. + +Exceptions can also be identified by strings, in which case the +:keyword:`except` clause is selected by object identity. An arbitrary value can +be raised along with the identifying string which can be passed to the handler. + +.. warning:: + + Messages to exceptions are not part of the Python API. Their contents may + change from one version of Python to the next without warning and should not be + relied on by code which will run under multiple versions of the interpreter. + +See also the description of the :keyword:`try` statement in section :ref:`try` +and :keyword:`raise` statement in section :ref:`raise`. + +.. rubric:: Footnotes + +.. [#] This limitation occurs because the code that is executed by these operations is + not available at the time the module is compiled. + diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst new file mode 100644 index 0000000000..28c1406eeb --- /dev/null +++ b/Doc/reference/expressions.rst @@ -0,0 +1,1283 @@ + +.. _expressions: + +*********** +Expressions +*********** + +.. index:: single: expression + +This chapter explains the meaning of the elements of expressions in Python. + +.. index:: single: BNF + +**Syntax Notes:** In this and the following chapters, extended BNF notation will +be used to describe syntax, not lexical analysis. When (one alternative of) a +syntax rule has the form + +.. productionlist:: * + name: `othername` + +.. index:: single: syntax + +and no semantics are given, the semantics of this form of ``name`` are the same +as for ``othername``. + + +.. _conversions: + +Arithmetic conversions +====================== + +.. index:: pair: arithmetic; conversion + +.. XXX no coercion rules are documented anymore + +When a description of an arithmetic operator below uses the phrase "the numeric +arguments are converted to a common type," the arguments are coerced using the +coercion rules. If both arguments are standard +numeric types, the following coercions are applied: + +* If either argument is a complex number, the other is converted to complex; + +* otherwise, if either argument is a floating point number, the other is + converted to floating point; + +* otherwise, if either argument is a long integer, the other is converted to + long integer; + +* otherwise, both must be plain integers and no conversion is necessary. + +Some additional rules apply for certain operators (e.g., a string left argument +to the '%' operator). Extensions can define their own coercions. + + +.. _atoms: + +Atoms +===== + +.. index:: single: atom + +Atoms are the most basic elements of expressions. The simplest atoms are +identifiers or literals. Forms enclosed in reverse quotes or in parentheses, +brackets or braces are also categorized syntactically as atoms. The syntax for +atoms is: + +.. productionlist:: + atom: `identifier` | `literal` | `enclosure` + enclosure: `parenth_form` | `list_display` + : | `generator_expression` | `dict_display` + : | `string_conversion` | `yield_atom` + + +.. _atom-identifiers: + +Identifiers (Names) +------------------- + +.. index:: + single: name + single: identifier + +An identifier occurring as an atom is a name. See section :ref:`identifiers` +for lexical definition and section :ref:`naming` for documentation of naming and +binding. + +.. index:: exception: NameError + +When the name is bound to an object, evaluation of the atom yields that object. +When a name is not bound, an attempt to evaluate it raises a :exc:`NameError` +exception. + +.. index:: + pair: name; mangling + pair: private; names + +**Private name mangling:** When an identifier that textually occurs in a class +definition begins with two or more underscore characters and does not end in two +or more underscores, it is considered a :dfn:`private name` of that class. +Private names are transformed to a longer form before code is generated for +them. The transformation inserts the class name in front of the name, with +leading underscores removed, and a single underscore inserted in front of the +class name. For example, the identifier ``__spam`` occurring in a class named +``Ham`` will be transformed to ``_Ham__spam``. This transformation is +independent of the syntactical context in which the identifier is used. If the +transformed name is extremely long (longer than 255 characters), implementation +defined truncation may happen. If the class name consists only of underscores, +no transformation is done. + +.. % +.. % + + +.. _atom-literals: + +Literals +-------- + +.. index:: single: literal + +Python supports string literals and various numeric literals: + +.. productionlist:: + literal: `stringliteral` | `integer` | `longinteger` + : | `floatnumber` | `imagnumber` + +Evaluation of a literal yields an object of the given type (string, integer, +long integer, floating point number, complex number) with the given value. The +value may be approximated in the case of floating point and imaginary (complex) +literals. See section :ref:`literals` for details. + +.. index:: + triple: immutable; data; type + pair: immutable; object + +All literals correspond to immutable data types, and hence the object's identity +is less important than its value. Multiple evaluations of literals with the +same value (either the same occurrence in the program text or a different +occurrence) may obtain the same object or a different object with the same +value. + + +.. _parenthesized: + +Parenthesized forms +------------------- + +.. index:: single: parenthesized form + +A parenthesized form is an optional expression list enclosed in parentheses: + +.. productionlist:: + parenth_form: "(" [`expression_list`] ")" + +A parenthesized expression list yields whatever that expression list yields: if +the list contains at least one comma, it yields a tuple; otherwise, it yields +the single expression that makes up the expression list. + +.. index:: pair: empty; tuple + +An empty pair of parentheses yields an empty tuple object. Since tuples are +immutable, the rules for literals apply (i.e., two occurrences of the empty +tuple may or may not yield the same object). + +.. index:: + single: comma + pair: tuple; display + +Note that tuples are not formed by the parentheses, but rather by use of the +comma operator. The exception is the empty tuple, for which parentheses *are* +required --- allowing unparenthesized "nothing" in expressions would cause +ambiguities and allow common typos to pass uncaught. + + +.. _lists: + +List displays +------------- + +.. index:: + pair: list; display + pair: list; comprehensions + +A list display is a possibly empty series of expressions enclosed in square +brackets: + +.. productionlist:: + list_display: "[" [`expression_list` | `list_comprehension`] "]" + list_comprehension: `expression` `list_for` + list_for: "for" `target_list` "in" `old_expression_list` [`list_iter`] + old_expression_list: `old_expression` [("," `old_expression`)+ [","]] + list_iter: `list_for` | `list_if` + list_if: "if" `old_expression` [`list_iter`] + +.. index:: + pair: list; comprehensions + object: list + pair: empty; list + +A list display yields a new list object. Its contents are specified by +providing either a list of expressions or a list comprehension. When a +comma-separated list of expressions is supplied, its elements are evaluated from +left to right and placed into the list object in that order. When a list +comprehension is supplied, it consists of a single expression followed by at +least one :keyword:`for` clause and zero or more :keyword:`for` or :keyword:`if` +clauses. In this case, the elements of the new list are those that would be +produced by considering each of the :keyword:`for` or :keyword:`if` clauses a +block, nesting from left to right, and evaluating the expression to produce a +list element each time the innermost block is reached [#]_. + + +.. _genexpr: + +Generator expressions +--------------------- + +.. index:: pair: generator; expression + +A generator expression is a compact generator notation in parentheses: + +.. productionlist:: + generator_expression: "(" `expression` `genexpr_for` ")" + genexpr_for: "for" `target_list` "in" `or_test` [`genexpr_iter`] + genexpr_iter: `genexpr_for` | `genexpr_if` + genexpr_if: "if" `old_expression` [`genexpr_iter`] + +.. index:: object: generator + +A generator expression yields a new generator object. It consists of a single +expression followed by at least one :keyword:`for` clause and zero or more +:keyword:`for` or :keyword:`if` clauses. The iterating values of the new +generator are those that would be produced by considering each of the +:keyword:`for` or :keyword:`if` clauses a block, nesting from left to right, and +evaluating the expression to yield a value that is reached the innermost block +for each iteration. + +Variables used in the generator expression are evaluated lazily when the +:meth:`__next__` method is called for generator object (in the same fashion as +normal generators). However, the leftmost :keyword:`for` clause is immediately +evaluated so that error produced by it can be seen before any other possible +error in the code that handles the generator expression. Subsequent +:keyword:`for` clauses cannot be evaluated immediately since they may depend on +the previous :keyword:`for` loop. For example: ``(x*y for x in range(10) for y +in bar(x))``. + +The parentheses can be omitted on calls with only one argument. See section +:ref:`calls` for the detail. + + +.. _dict: + +Dictionary displays +------------------- + +.. index:: pair: dictionary; display + +.. index:: + single: key + single: datum + single: key/datum pair + +A dictionary display is a possibly empty series of key/datum pairs enclosed in +curly braces: + +.. productionlist:: + dict_display: "{" [`key_datum_list`] "}" + key_datum_list: `key_datum` ("," `key_datum`)* [","] + key_datum: `expression` ":" `expression` + +.. index:: object: dictionary + +A dictionary display yields a new dictionary object. + +The key/datum pairs are evaluated from left to right to define the entries of +the dictionary: each key object is used as a key into the dictionary to store +the corresponding datum. + +.. index:: pair: immutable; object + +Restrictions on the types of the key values are listed earlier in section +:ref:`types`. (To summarize, the key type should be hashable, which excludes +all mutable objects.) Clashes between duplicate keys are not detected; the last +datum (textually rightmost in the display) stored for a given key value +prevails. + + +.. _yieldexpr: + +Yield expressions +----------------- + +.. index:: + keyword: yield + pair: yield; expression + pair: generator; function + +.. productionlist:: + yield_atom: "(" `yield_expression` ")" + yield_expression: "yield" [`expression_list`] + +.. versionadded:: 2.5 + +The :keyword:`yield` expression is only used when defining a generator function, +and can only be used in the body of a function definition. Using a +:keyword:`yield` expression in a function definition is sufficient to cause that +definition to create a generator function instead of a normal function. + +When a generator function is called, it returns an iterator known as a +generator. That generator then controls the execution of a generator function. +The execution starts when one of the generator's methods is called. At that +time, the execution proceeds to the first :keyword:`yield` expression, where it +is suspended again, returning the value of :token:`expression_list` to +generator's caller. By suspended we mean that all local state is retained, +including the current bindings of local variables, the instruction pointer, and +the internal evaluation stack. When the execution is resumed by calling one of +the generator's methods, the function can proceed exactly as if the +:keyword:`yield` expression was just another external call. The value of the +:keyword:`yield` expression after resuming depends on the method which resumed +the execution. + +.. index:: single: coroutine + +All of this makes generator functions quite similar to coroutines; they yield +multiple times, they have more than one entry point and their execution can be +suspended. The only difference is that a generator function cannot control +where should the execution continue after it yields; the control is always +transfered to the generator's caller. + +.. index:: object: generator + +The following generator's methods can be used to control the execution of a +generator function: + +.. index:: exception: StopIteration + + +.. method:: generator.next() + + Starts the execution of a generator function or resumes it at the last executed + :keyword:`yield` expression. When a generator function is resumed with a + :meth:`next` method, the current :keyword:`yield` expression always evaluates to + :const:`None`. The execution then continues to the next :keyword:`yield` + expression, where the generator is suspended again, and the value of the + :token:`expression_list` is returned to :meth:`next`'s caller. If the generator + exits without yielding another value, a :exc:`StopIteration` exception is + raised. + + +.. method:: generator.send(value) + + Resumes the execution and "sends" a value into the generator function. The + ``value`` argument becomes the result of the current :keyword:`yield` + expression. The :meth:`send` method returns the next value yielded by the + generator, or raises :exc:`StopIteration` if the generator exits without + yielding another value. When :meth:`send` is called to start the generator, it + must be called with :const:`None` as the argument, because there is no + :keyword:`yield` expression that could receieve the value. + + +.. method:: generator.throw(type[, value[, traceback]]) + + Raises an exception of type ``type`` at the point where generator was paused, + and returns the next value yielded by the generator function. If the generator + exits without yielding another value, a :exc:`StopIteration` exception is + raised. If the generator function does not catch the passed-in exception, or + raises a different exception, then that exception propagates to the caller. + +.. index:: exception: GeneratorExit + + +.. method:: generator.close() + + Raises a :exc:`GeneratorExit` at the point where the generator function was + paused. If the generator function then raises :exc:`StopIteration` (by exiting + normally, or due to already being closed) or :exc:`GeneratorExit` (by not + catching the exception), close returns to its caller. If the generator yields a + value, a :exc:`RuntimeError` is raised. If the generator raises any other + exception, it is propagated to the caller. :meth:`close` does nothing if the + generator has already exited due to an exception or normal exit. + +Here is a simple example that demonstrates the behavior of generators and +generator functions:: + + >>> def echo(value=None): + ... print "Execution starts when 'next()' is called for the first time." + ... try: + ... while True: + ... try: + ... value = (yield value) + ... except GeneratorExit: + ... # never catch GeneratorExit + ... raise + ... except Exception, e: + ... value = e + ... finally: + ... print "Don't forget to clean up when 'close()' is called." + ... + >>> generator = echo(1) + >>> print generator.next() + Execution starts when 'next()' is called for the first time. + 1 + >>> print generator.next() + None + >>> print generator.send(2) + 2 + >>> generator.throw(TypeError, "spam") + TypeError('spam',) + >>> generator.close() + Don't forget to clean up when 'close()' is called. + + +.. seealso:: + + :pep:`0342` - Coroutines via Enhanced Generators + The proposal to enhance the API and syntax of generators, making them usable as + simple coroutines. + + +.. _primaries: + +Primaries +========= + +.. index:: single: primary + +Primaries represent the most tightly bound operations of the language. Their +syntax is: + +.. productionlist:: + primary: `atom` | `attributeref` | `subscription` | `slicing` | `call` + + +.. _attribute-references: + +Attribute references +-------------------- + +.. index:: pair: attribute; reference + +An attribute reference is a primary followed by a period and a name: + +.. productionlist:: + attributeref: `primary` "." `identifier` + +.. index:: + exception: AttributeError + object: module + object: list + +The primary must evaluate to an object of a type that supports attribute +references, e.g., a module, list, or an instance. This object is then asked to +produce the attribute whose name is the identifier. If this attribute is not +available, the exception :exc:`AttributeError` is raised. Otherwise, the type +and value of the object produced is determined by the object. Multiple +evaluations of the same attribute reference may yield different objects. + + +.. _subscriptions: + +Subscriptions +------------- + +.. index:: single: subscription + +.. index:: + object: sequence + object: mapping + object: string + object: tuple + object: list + object: dictionary + pair: sequence; item + +A subscription selects an item of a sequence (string, tuple or list) or mapping +(dictionary) object: + +.. productionlist:: + subscription: `primary` "[" `expression_list` "]" + +The primary must evaluate to an object of a sequence or mapping type. + +If the primary is a mapping, the expression list must evaluate to an object +whose value is one of the keys of the mapping, and the subscription selects the +value in the mapping that corresponds to that key. (The expression list is a +tuple except if it has exactly one item.) + +If the primary is a sequence, the expression (list) must evaluate to a plain +integer. If this value is negative, the length of the sequence is added to it +(so that, e.g., ``x[-1]`` selects the last item of ``x``.) The resulting value +must be a nonnegative integer less than the number of items in the sequence, and +the subscription selects the item whose index is that value (counting from +zero). + +.. index:: + single: character + pair: string; item + +A string's items are characters. A character is not a separate data type but a +string of exactly one character. + + +.. _slicings: + +Slicings +-------- + +.. index:: + single: slicing + single: slice + +.. index:: + object: sequence + object: string + object: tuple + object: list + +A slicing selects a range of items in a sequence object (e.g., a string, tuple +or list). Slicings may be used as expressions or as targets in assignment or +:keyword:`del` statements. The syntax for a slicing: + +.. productionlist:: + slicing: `simple_slicing` | `extended_slicing` + simple_slicing: `primary` "[" `short_slice` "]" + extended_slicing: `primary` "[" `slice_list` "]" + slice_list: `slice_item` ("," `slice_item`)* [","] + slice_item: `expression` | `proper_slice` | `ellipsis` + proper_slice: `short_slice` | `long_slice` + short_slice: [`lower_bound`] ":" [`upper_bound`] + long_slice: `short_slice` ":" [`stride`] + lower_bound: `expression` + upper_bound: `expression` + stride: `expression` + ellipsis: "..." + +.. index:: pair: extended; slicing + +There is ambiguity in the formal syntax here: anything that looks like an +expression list also looks like a slice list, so any subscription can be +interpreted as a slicing. Rather than further complicating the syntax, this is +disambiguated by defining that in this case the interpretation as a subscription +takes priority over the interpretation as a slicing (this is the case if the +slice list contains no proper slice nor ellipses). Similarly, when the slice +list has exactly one short slice and no trailing comma, the interpretation as a +simple slicing takes priority over that as an extended slicing. + +The semantics for a simple slicing are as follows. The primary must evaluate to +a sequence object. The lower and upper bound expressions, if present, must +evaluate to plain integers; defaults are zero and the ``sys.maxint``, +respectively. If either bound is negative, the sequence's length is added to +it. The slicing now selects all items with index *k* such that ``i <= k < j`` +where *i* and *j* are the specified lower and upper bounds. This may be an +empty sequence. It is not an error if *i* or *j* lie outside the range of valid +indexes (such items don't exist so they aren't selected). + +.. index:: + single: start (slice object attribute) + single: stop (slice object attribute) + single: step (slice object attribute) + +The semantics for an extended slicing are as follows. The primary must evaluate +to a mapping object, and it is indexed with a key that is constructed from the +slice list, as follows. If the slice list contains at least one comma, the key +is a tuple containing the conversion of the slice items; otherwise, the +conversion of the lone slice item is the key. The conversion of a slice item +that is an expression is that expression. The conversion of a proper slice is a +slice object (see section :ref:`types`) whose :attr:`start`, :attr:`stop` and +:attr:`step` attributes are the values of the expressions given as lower bound, +upper bound and stride, respectively, substituting ``None`` for missing +expressions. + + +.. _calls: + +Calls +----- + +.. index:: single: call + +.. index:: object: callable + +A call calls a callable object (e.g., a function) with a possibly empty series +of arguments: + +.. productionlist:: + call: `primary` "(" [`argument_list` [","] + : | `expression` `genexpr_for`] ")" + argument_list: `positional_arguments` ["," `keyword_arguments`] + : ["," "*" `expression`] + : ["," "**" `expression`] + : | `keyword_arguments` ["," "*" `expression`] + : ["," "**" `expression`] + : | "*" `expression` ["," "**" `expression`] + : | "**" `expression` + positional_arguments: `expression` ("," `expression`)* + keyword_arguments: `keyword_item` ("," `keyword_item`)* + keyword_item: `identifier` "=" `expression` + +A trailing comma may be present after the positional and keyword arguments but +does not affect the semantics. + +The primary must evaluate to a callable object (user-defined functions, built-in +functions, methods of built-in objects, class objects, methods of class +instances, and certain class instances themselves are callable; extensions may +define additional callable object types). All argument expressions are +evaluated before the call is attempted. Please refer to section :ref:`function` +for the syntax of formal parameter lists. + +If keyword arguments are present, they are first converted to positional +arguments, as follows. First, a list of unfilled slots is created for the +formal parameters. If there are N positional arguments, they are placed in the +first N slots. Next, for each keyword argument, the identifier is used to +determine the corresponding slot (if the identifier is the same as the first +formal parameter name, the first slot is used, and so on). If the slot is +already filled, a :exc:`TypeError` exception is raised. Otherwise, the value of +the argument is placed in the slot, filling it (even if the expression is +``None``, it fills the slot). When all arguments have been processed, the slots +that are still unfilled are filled with the corresponding default value from the +function definition. (Default values are calculated, once, when the function is +defined; thus, a mutable object such as a list or dictionary used as default +value will be shared by all calls that don't specify an argument value for the +corresponding slot; this should usually be avoided.) If there are any unfilled +slots for which no default value is specified, a :exc:`TypeError` exception is +raised. Otherwise, the list of filled slots is used as the argument list for +the call. + +If there are more positional arguments than there are formal parameter slots, a +:exc:`TypeError` exception is raised, unless a formal parameter using the syntax +``*identifier`` is present; in this case, that formal parameter receives a tuple +containing the excess positional arguments (or an empty tuple if there were no +excess positional arguments). + +If any keyword argument does not correspond to a formal parameter name, a +:exc:`TypeError` exception is raised, unless a formal parameter using the syntax +``**identifier`` is present; in this case, that formal parameter receives a +dictionary containing the excess keyword arguments (using the keywords as keys +and the argument values as corresponding values), or a (new) empty dictionary if +there were no excess keyword arguments. + +If the syntax ``*expression`` appears in the function call, ``expression`` must +evaluate to a sequence. Elements from this sequence are treated as if they were +additional positional arguments; if there are postional arguments *x1*,...,*xN* +, and ``expression`` evaluates to a sequence *y1*,...,*yM*, this is equivalent +to a call with M+N positional arguments *x1*,...,*xN*,*y1*,...,*yM*. + +A consequence of this is that although the ``*expression`` syntax appears +*after* any keyword arguments, it is processed *before* the keyword arguments +(and the ``**expression`` argument, if any -- see below). So:: + + >>> def f(a, b): + ... print a, b + ... + >>> f(b=1, *(2,)) + 2 1 + >>> f(a=1, *(2,)) + Traceback (most recent call last): + File "<stdin>", line 1, in ? + TypeError: f() got multiple values for keyword argument 'a' + >>> f(1, *(2,)) + 1 2 + +It is unusual for both keyword arguments and the ``*expression`` syntax to be +used in the same call, so in practice this confusion does not arise. + +If the syntax ``**expression`` appears in the function call, ``expression`` must +evaluate to a mapping, the contents of which are treated as additional keyword +arguments. In the case of a keyword appearing in both ``expression`` and as an +explicit keyword argument, a :exc:`TypeError` exception is raised. + +Formal parameters using the syntax ``*identifier`` or ``**identifier`` cannot be +used as positional argument slots or as keyword argument names. + +A call always returns some value, possibly ``None``, unless it raises an +exception. How this value is computed depends on the type of the callable +object. + +If it is--- + +a user-defined function: + .. index:: + pair: function; call + triple: user-defined; function; call + object: user-defined function + object: function + + The code block for the function is executed, passing it the argument list. The + first thing the code block will do is bind the formal parameters to the + arguments; this is described in section :ref:`function`. When the code block + executes a :keyword:`return` statement, this specifies the return value of the + function call. + +a built-in function or method: + .. index:: + pair: function; call + pair: built-in function; call + pair: method; call + pair: built-in method; call + object: built-in method + object: built-in function + object: method + object: function + + The result is up to the interpreter; see :ref:`built-in-funcs` for the + descriptions of built-in functions and methods. + +a class object: + .. index:: + object: class + pair: class object; call + + A new instance of that class is returned. + +a class instance method: + .. index:: + object: class instance + object: instance + pair: class instance; call + + The corresponding user-defined function is called, with an argument list that is + one longer than the argument list of the call: the instance becomes the first + argument. + +a class instance: + .. index:: + pair: instance; call + single: __call__() (object method) + + The class must define a :meth:`__call__` method; the effect is then the same as + if that method was called. + + +.. _power: + +The power operator +================== + +The power operator binds more tightly than unary operators on its left; it binds +less tightly than unary operators on its right. The syntax is: + +.. productionlist:: + power: `primary` ["**" `u_expr`] + +Thus, in an unparenthesized sequence of power and unary operators, the operators +are evaluated from right to left (this does not constrain the evaluation order +for the operands). + +The power operator has the same semantics as the built-in :func:`pow` function, +when called with two arguments: it yields its left argument raised to the power +of its right argument. The numeric arguments are first converted to a common +type. The result type is that of the arguments after coercion. + +With mixed operand types, the coercion rules for binary arithmetic operators +apply. For int and long int operands, the result has the same type as the +operands (after coercion) unless the second argument is negative; in that case, +all arguments are converted to float and a float result is delivered. For +example, ``10**2`` returns ``100``, but ``10**-2`` returns ``0.01``. (This last +feature was added in Python 2.2. In Python 2.1 and before, if both arguments +were of integer types and the second argument was negative, an exception was +raised). + +Raising ``0.0`` to a negative power results in a :exc:`ZeroDivisionError`. +Raising a negative number to a fractional power results in a :exc:`ValueError`. + + +.. _unary: + +Unary arithmetic operations +=========================== + +.. index:: + triple: unary; arithmetic; operation + triple: unary; bit-wise; operation + +All unary arithmetic (and bit-wise) operations have the same priority: + +.. productionlist:: + u_expr: `power` | "-" `u_expr` | "+" `u_expr` | "~" `u_expr` + +.. index:: + single: negation + single: minus + +The unary ``-`` (minus) operator yields the negation of its numeric argument. + +.. index:: single: plus + +The unary ``+`` (plus) operator yields its numeric argument unchanged. + +.. index:: single: inversion + +The unary ``~`` (invert) operator yields the bit-wise inversion of its plain or +long integer argument. The bit-wise inversion of ``x`` is defined as +``-(x+1)``. It only applies to integral numbers. + +.. index:: exception: TypeError + +In all three cases, if the argument does not have the proper type, a +:exc:`TypeError` exception is raised. + + +.. _binary: + +Binary arithmetic operations +============================ + +.. index:: triple: binary; arithmetic; operation + +The binary arithmetic operations have the conventional priority levels. Note +that some of these operations also apply to certain non-numeric types. Apart +from the power operator, there are only two levels, one for multiplicative +operators and one for additive operators: + +.. productionlist:: + m_expr: `u_expr` | `m_expr` "*" `u_expr` | `m_expr` "//" `u_expr` | `m_expr` "/" `u_expr` + : | `m_expr` "%" `u_expr` + a_expr: `m_expr` | `a_expr` "+" `m_expr` | `a_expr` "-" `m_expr` + +.. index:: single: multiplication + +The ``*`` (multiplication) operator yields the product of its arguments. The +arguments must either both be numbers, or one argument must be an integer (plain +or long) and the other must be a sequence. In the former case, the numbers are +converted to a common type and then multiplied together. In the latter case, +sequence repetition is performed; a negative repetition factor yields an empty +sequence. + +.. index:: + exception: ZeroDivisionError + single: division + +The ``/`` (division) and ``//`` (floor division) operators yield the quotient of +their arguments. The numeric arguments are first converted to a common type. +Plain or long integer division yields an integer of the same type; the result is +that of mathematical division with the 'floor' function applied to the result. +Division by zero raises the :exc:`ZeroDivisionError` exception. + +.. index:: single: modulo + +The ``%`` (modulo) operator yields the remainder from the division of the first +argument by the second. The numeric arguments are first converted to a common +type. A zero right argument raises the :exc:`ZeroDivisionError` exception. The +arguments may be floating point numbers, e.g., ``3.14%0.7`` equals ``0.34`` +(since ``3.14`` equals ``4*0.7 + 0.34``.) The modulo operator always yields a +result with the same sign as its second operand (or zero); the absolute value of +the result is strictly smaller than the absolute value of the second operand +[#]_. + +The integer division and modulo operators are connected by the following +identity: ``x == (x/y)*y + (x%y)``. Integer division and modulo are also +connected with the built-in function :func:`divmod`: ``divmod(x, y) == (x/y, +x%y)``. These identities don't hold for floating point numbers; there similar +identities hold approximately where ``x/y`` is replaced by ``floor(x/y)`` or +``floor(x/y) - 1`` [#]_. + +In addition to performing the modulo operation on numbers, the ``%`` operator is +also overloaded by string and unicode objects to perform string formatting (also +known as interpolation). The syntax for string formatting is described in the +Python Library Reference, section :ref:`string-formatting`. + +The floor division operator, the modulo operator, and the :func:`divmod` +function are not defined for complex numbers. Instead, convert to a +floating point number using the :func:`abs` function if appropriate. + +.. index:: single: addition + +The ``+`` (addition) operator yields the sum of its arguments. The arguments +must either both be numbers or both sequences of the same type. In the former +case, the numbers are converted to a common type and then added together. In +the latter case, the sequences are concatenated. + +.. index:: single: subtraction + +The ``-`` (subtraction) operator yields the difference of its arguments. The +numeric arguments are first converted to a common type. + + +.. _shifting: + +Shifting operations +=================== + +.. index:: pair: shifting; operation + +The shifting operations have lower priority than the arithmetic operations: + +.. productionlist:: + shift_expr: `a_expr` | `shift_expr` ( "<<" | ">>" ) `a_expr` + +These operators accept plain or long integers as arguments. The arguments are +converted to a common type. They shift the first argument to the left or right +by the number of bits given by the second argument. + +.. index:: exception: ValueError + +A right shift by *n* bits is defined as division by ``pow(2,n)``. A left shift +by *n* bits is defined as multiplication with ``pow(2,n)``; for plain integers +there is no overflow check so in that case the operation drops bits and flips +the sign if the result is not less than ``pow(2,31)`` in absolute value. +Negative shift counts raise a :exc:`ValueError` exception. + + +.. _bitwise: + +Binary bit-wise operations +========================== + +.. index:: triple: binary; bit-wise; operation + +Each of the three bitwise operations has a different priority level: + +.. productionlist:: + and_expr: `shift_expr` | `and_expr` "&" `shift_expr` + xor_expr: `and_expr` | `xor_expr` "^" `and_expr` + or_expr: `xor_expr` | `or_expr` "|" `xor_expr` + +.. index:: pair: bit-wise; and + +The ``&`` operator yields the bitwise AND of its arguments, which must be plain +or long integers. The arguments are converted to a common type. + +.. index:: + pair: bit-wise; xor + pair: exclusive; or + +The ``^`` operator yields the bitwise XOR (exclusive OR) of its arguments, which +must be plain or long integers. The arguments are converted to a common type. + +.. index:: + pair: bit-wise; or + pair: inclusive; or + +The ``|`` operator yields the bitwise (inclusive) OR of its arguments, which +must be plain or long integers. The arguments are converted to a common type. + + +.. _comparisons: + +Comparisons +=========== + +.. index:: single: comparison + +.. index:: pair: C; language + +Unlike C, all comparison operations in Python have the same priority, which is +lower than that of any arithmetic, shifting or bitwise operation. Also unlike +C, expressions like ``a < b < c`` have the interpretation that is conventional +in mathematics: + +.. productionlist:: + comparison: `or_expr` ( `comp_operator` `or_expr` )* + comp_operator: "<" | ">" | "==" | ">=" | "<=" | "!=" + : | "is" ["not"] | ["not"] "in" + +Comparisons yield boolean values: ``True`` or ``False``. + +.. index:: pair: chaining; comparisons + +Comparisons can be chained arbitrarily, e.g., ``x < y <= z`` is equivalent to +``x < y and y <= z``, except that ``y`` is evaluated only once (but in both +cases ``z`` is not evaluated at all when ``x < y`` is found to be false). + +Formally, if *a*, *b*, *c*, ..., *y*, *z* are expressions and *opa*, *opb*, ..., +*opy* are comparison operators, then *a opa b opb c* ...*y opy z* is equivalent +to *a opa b* :keyword:`and` *b opb c* :keyword:`and` ... *y opy z*, except that +each expression is evaluated at most once. + +Note that *a opa b opb c* doesn't imply any kind of comparison between *a* and +*c*, so that, e.g., ``x < y > z`` is perfectly legal (though perhaps not +pretty). + +The operators ``<``, ``>``, ``==``, ``>=``, ``<=``, and ``!=`` compare the +values of two objects. The objects need not have the same type. If both are +numbers, they are converted to a common type. Otherwise, objects of different +types *always* compare unequal, and are ordered consistently but arbitrarily. +You can control comparison behavior of objects of non-builtin types by defining +a ``__cmp__`` method or rich comparison methods like ``__gt__``, described in +section :ref:`specialnames`. + +(This unusual definition of comparison was used to simplify the definition of +operations like sorting and the :keyword:`in` and :keyword:`not in` operators. +In the future, the comparison rules for objects of different types are likely to +change.) + +Comparison of objects of the same type depends on the type: + +* Numbers are compared arithmetically. + +* Strings are compared lexicographically using the numeric equivalents (the + result of the built-in function :func:`ord`) of their characters. Unicode and + 8-bit strings are fully interoperable in this behavior. + +* Tuples and lists are compared lexicographically using comparison of + corresponding elements. This means that to compare equal, each element must + compare equal and the two sequences must be of the same type and have the same + length. + + If not equal, the sequences are ordered the same as their first differing + elements. For example, ``cmp([1,2,x], [1,2,y])`` returns the same as + ``cmp(x,y)``. If the corresponding element does not exist, the shorter sequence + is ordered first (for example, ``[1,2] < [1,2,3]``). + +* Mappings (dictionaries) compare equal if and only if their sorted (key, value) + lists compare equal. [#]_ Outcomes other than equality are resolved + consistently, but are not otherwise defined. [#]_ + +* Most other objects of builtin types compare unequal unless they are the same + object; the choice whether one object is considered smaller or larger than + another one is made arbitrarily but consistently within one execution of a + program. + +The operators :keyword:`in` and :keyword:`not in` test for set membership. ``x +in s`` evaluates to true if *x* is a member of the set *s*, and false otherwise. +``x not in s`` returns the negation of ``x in s``. The set membership test has +traditionally been bound to sequences; an object is a member of a set if the set +is a sequence and contains an element equal to that object. However, it is +possible for an object to support membership tests without being a sequence. In +particular, dictionaries support membership testing as a nicer way of spelling +``key in dict``; other mapping types may follow suit. + +For the list and tuple types, ``x in y`` is true if and only if there exists an +index *i* such that ``x == y[i]`` is true. + +For the Unicode and string types, ``x in y`` is true if and only if *x* is a +substring of *y*. An equivalent test is ``y.find(x) != -1``. Note, *x* and *y* +need not be the same type; consequently, ``u'ab' in 'abc'`` will return +``True``. Empty strings are always considered to be a substring of any other +string, so ``"" in "abc"`` will return ``True``. + +.. versionchanged:: 2.3 + Previously, *x* was required to be a string of length ``1``. + +For user-defined classes which define the :meth:`__contains__` method, ``x in +y`` is true if and only if ``y.__contains__(x)`` is true. + +For user-defined classes which do not define :meth:`__contains__` and do define +:meth:`__getitem__`, ``x in y`` is true if and only if there is a non-negative +integer index *i* such that ``x == y[i]``, and all lower integer indices do not +raise :exc:`IndexError` exception. (If any other exception is raised, it is as +if :keyword:`in` raised that exception). + +.. index:: + operator: in + operator: not in + pair: membership; test + object: sequence + +The operator :keyword:`not in` is defined to have the inverse true value of +:keyword:`in`. + +.. index:: + operator: is + operator: is not + pair: identity; test + +The operators :keyword:`is` and :keyword:`is not` test for object identity: ``x +is y`` is true if and only if *x* and *y* are the same object. ``x is not y`` +yields the inverse truth value. + + +.. _booleans: + +Boolean operations +================== + +.. index:: + pair: Conditional; expression + pair: Boolean; operation + +Boolean operations have the lowest priority of all Python operations: + +.. productionlist:: + expression: `conditional_expression` | `lambda_form` + old_expression: `or_test` | `old_lambda_form` + conditional_expression: `or_test` ["if" `or_test` "else" `expression`] + or_test: `and_test` | `or_test` "or" `and_test` + and_test: `not_test` | `and_test` "and" `not_test` + not_test: `comparison` | "not" `not_test` + +In the context of Boolean operations, and also when expressions are used by +control flow statements, the following values are interpreted as false: +``False``, ``None``, numeric zero of all types, and empty strings and containers +(including strings, tuples, lists, dictionaries, sets and frozensets). All +other values are interpreted as true. + +.. index:: operator: not + +The operator :keyword:`not` yields ``True`` if its argument is false, ``False`` +otherwise. + +The expression ``x if C else y`` first evaluates *C* (*not* *x*); if *C* is +true, *x* is evaluated and its value is returned; otherwise, *y* is evaluated +and its value is returned. + +.. versionadded:: 2.5 + +.. index:: operator: and + +The expression ``x and y`` first evaluates *x*; if *x* is false, its value is +returned; otherwise, *y* is evaluated and the resulting value is returned. + +.. index:: operator: or + +The expression ``x or y`` first evaluates *x*; if *x* is true, its value is +returned; otherwise, *y* is evaluated and the resulting value is returned. + +(Note that neither :keyword:`and` nor :keyword:`or` restrict the value and type +they return to ``False`` and ``True``, but rather return the last evaluated +argument. This is sometimes useful, e.g., if ``s`` is a string that should be +replaced by a default value if it is empty, the expression ``s or 'foo'`` yields +the desired value. Because :keyword:`not` has to invent a value anyway, it does +not bother to return a value of the same type as its argument, so e.g., ``not +'foo'`` yields ``False``, not ``''``.) + + +.. _lambdas: + +Lambdas +======= + +.. index:: + pair: lambda; expression + pair: lambda; form + pair: anonymous; function + +.. productionlist:: + lambda_form: "lambda" [`parameter_list`]: `expression` + old_lambda_form: "lambda" [`parameter_list`]: `old_expression` + +Lambda forms (lambda expressions) have the same syntactic position as +expressions. They are a shorthand to create anonymous functions; the expression +``lambda arguments: expression`` yields a function object. The unnamed object +behaves like a function object defined with :: + + def name(arguments): + return expression + +See section :ref:`function` for the syntax of parameter lists. Note that +functions created with lambda forms cannot contain statements or annotations. + +.. _lambda: + + +.. _exprlists: + +Expression lists +================ + +.. index:: pair: expression; list + +.. productionlist:: + expression_list: `expression` ( "," `expression` )* [","] + +.. index:: object: tuple + +An expression list containing at least one comma yields a tuple. The length of +the tuple is the number of expressions in the list. The expressions are +evaluated from left to right. + +.. index:: pair: trailing; comma + +The trailing comma is required only to create a single tuple (a.k.a. a +*singleton*); it is optional in all other cases. A single expression without a +trailing comma doesn't create a tuple, but rather yields the value of that +expression. (To create an empty tuple, use an empty pair of parentheses: +``()``.) + + +.. _evalorder: + +Evaluation order +================ + +.. index:: pair: evaluation; order + +Python evaluates expressions from left to right. Notice that while evaluating an +assignment, the right-hand side is evaluated before the left-hand side. + +In the following lines, expressions will be evaluated in the arithmetic order of +their suffixes:: + + expr1, expr2, expr3, expr4 + (expr1, expr2, expr3, expr4) + {expr1: expr2, expr3: expr4} + expr1 + expr2 * (expr3 - expr4) + func(expr1, expr2, *expr3, **expr4) + expr3, expr4 = expr1, expr2 + + +.. _operator-summary: + +Summary +======= + +.. index:: pair: operator; precedence + +The following table summarizes the operator precedences in Python, from lowest +precedence (least binding) to highest precedence (most binding). Operators in +the same box have the same precedence. Unless the syntax is explicitly given, +operators are binary. Operators in the same box group left to right (except for +comparisons, including tests, which all have the same precedence and chain from +left to right --- see section :ref:`comparisons` --- and exponentiation, which +groups from right to left). + ++----------------------------------------------+-------------------------------------+ +| Operator | Description | ++==============================================+=====================================+ +| :keyword:`lambda` | Lambda expression | ++----------------------------------------------+-------------------------------------+ +| :keyword:`or` | Boolean OR | ++----------------------------------------------+-------------------------------------+ +| :keyword:`and` | Boolean AND | ++----------------------------------------------+-------------------------------------+ +| :keyword:`not` *x* | Boolean NOT | ++----------------------------------------------+-------------------------------------+ +| :keyword:`in`, :keyword:`not` :keyword:`in` | Membership tests | ++----------------------------------------------+-------------------------------------+ +| :keyword:`is`, :keyword:`is not` | Identity tests | ++----------------------------------------------+-------------------------------------+ +| ``<``, ``<=``, ``>``, ``>=``, ``!=``, ``==`` | Comparisons | ++----------------------------------------------+-------------------------------------+ +| ``|`` | Bitwise OR | ++----------------------------------------------+-------------------------------------+ +| ``^`` | Bitwise XOR | ++----------------------------------------------+-------------------------------------+ +| ``&`` | Bitwise AND | ++----------------------------------------------+-------------------------------------+ +| ``<<``, ``>>`` | Shifts | ++----------------------------------------------+-------------------------------------+ +| ``+``, ``-`` | Addition and subtraction | ++----------------------------------------------+-------------------------------------+ +| ``*``, ``/``, ``%`` | Multiplication, division, remainder | ++----------------------------------------------+-------------------------------------+ +| ``+x``, ``-x`` | Positive, negative | ++----------------------------------------------+-------------------------------------+ +| ``~x`` | Bitwise not | ++----------------------------------------------+-------------------------------------+ +| ``**`` | Exponentiation | ++----------------------------------------------+-------------------------------------+ +| ``x.attribute`` | Attribute reference | ++----------------------------------------------+-------------------------------------+ +| ``x[index]`` | Subscription | ++----------------------------------------------+-------------------------------------+ +| ``x[index:index]`` | Slicing | ++----------------------------------------------+-------------------------------------+ +| ``f(arguments...)`` | Function call | ++----------------------------------------------+-------------------------------------+ +| ``(expressions...)`` | Binding or tuple display | ++----------------------------------------------+-------------------------------------+ +| ``[expressions...]`` | List display | ++----------------------------------------------+-------------------------------------+ +| ``{key:datum...}`` | Dictionary display | ++----------------------------------------------+-------------------------------------+ + +.. rubric:: Footnotes + +.. [#] In Python 2.3, a list comprehension "leaks" the control variables of each + ``for`` it contains into the containing scope. However, this behavior is + deprecated, and relying on it will not work once this bug is fixed in a future + release + +.. [#] While ``abs(x%y) < abs(y)`` is true mathematically, for floats it may not be + true numerically due to roundoff. For example, and assuming a platform on which + a Python float is an IEEE 754 double-precision number, in order that ``-1e-100 % + 1e100`` have the same sign as ``1e100``, the computed result is ``-1e-100 + + 1e100``, which is numerically exactly equal to ``1e100``. Function :func:`fmod` + in the :mod:`math` module returns a result whose sign matches the sign of the + first argument instead, and so returns ``-1e-100`` in this case. Which approach + is more appropriate depends on the application. + +.. [#] If x is very close to an exact integer multiple of y, it's possible for + ``floor(x/y)`` to be one larger than ``(x-x%y)/y`` due to rounding. In such + cases, Python returns the latter result, in order to preserve that + ``divmod(x,y)[0] * y + x % y`` be very close to ``x``. + +.. [#] The implementation computes this efficiently, without constructing lists or + sorting. + +.. [#] Earlier versions of Python used lexicographic comparison of the sorted (key, + value) lists, but this was very expensive for the common case of comparing for + equality. An even earlier version of Python compared dictionaries by identity + only, but this caused surprises because people expected to be able to test a + dictionary for emptiness by comparing it to ``{}``. + diff --git a/Doc/reference/index.rst b/Doc/reference/index.rst new file mode 100644 index 0000000000..18bf0537a9 --- /dev/null +++ b/Doc/reference/index.rst @@ -0,0 +1,30 @@ +.. _reference-index: + +################################# + The Python language reference +################################# + +:Release: |version| +:Date: |today| + +This reference manual describes the syntax and "core semantics" of the +language. It is terse, but attempts to be exact and complete. The semantics of +non-essential built-in object types and of the built-in functions and modules +are described in :ref:`library-index`. For an informal introduction to the +language, see :ref:`tutorial-index`. For C or C++ programmers, two additional +manuals exist: :ref:`extending-index` describes the high-level picture of how to +write a Python extension module, and the :ref:`c-api-index` describes the +interfaces available to C/C++ programmers in detail. + +.. toctree:: + :maxdepth: 2 + + introduction.rst + lexical_analysis.rst + datamodel.rst + executionmodel.rst + expressions.rst + simple_stmts.rst + compound_stmts.rst + toplevel_components.rst + diff --git a/Doc/reference/introduction.rst b/Doc/reference/introduction.rst new file mode 100644 index 0000000000..0d53719f27 --- /dev/null +++ b/Doc/reference/introduction.rst @@ -0,0 +1,138 @@ + +.. _introduction: + +************ +Introduction +************ + +This reference manual describes the Python programming language. It is not +intended as a tutorial. + +While I am trying to be as precise as possible, I chose to use English rather +than formal specifications for everything except syntax and lexical analysis. +This should make the document more understandable to the average reader, but +will leave room for ambiguities. Consequently, if you were coming from Mars and +tried to re-implement Python from this document alone, you might have to guess +things and in fact you would probably end up implementing quite a different +language. On the other hand, if you are using Python and wonder what the precise +rules about a particular area of the language are, you should definitely be able +to find them here. If you would like to see a more formal definition of the +language, maybe you could volunteer your time --- or invent a cloning machine +:-). + +It is dangerous to add too many implementation details to a language reference +document --- the implementation may change, and other implementations of the +same language may work differently. On the other hand, there is currently only +one Python implementation in widespread use (although alternate implementations +exist), and its particular quirks are sometimes worth being mentioned, +especially where the implementation imposes additional limitations. Therefore, +you'll find short "implementation notes" sprinkled throughout the text. + +Every Python implementation comes with a number of built-in and standard +modules. These are documented in :ref:`library-index`. A few built-in modules +are mentioned when they interact in a significant way with the language +definition. + + +.. _implementations: + +Alternate Implementations +========================= + +Though there is one Python implementation which is by far the most popular, +there are some alternate implementations which are of particular interest to +different audiences. + +Known implementations include: + +CPython + This is the original and most-maintained implementation of Python, written in C. + New language features generally appear here first. + +Jython + Python implemented in Java. This implementation can be used as a scripting + language for Java applications, or can be used to create applications using the + Java class libraries. It is also often used to create tests for Java libraries. + More information can be found at `the Jython website <http://www.jython.org/>`_. + +Python for .NET + This implementation actually uses the CPython implementation, but is a managed + .NET application and makes .NET libraries available. This was created by Brian + Lloyd. For more information, see the `Python for .NET home page + <http://www.zope.org/Members/Brian/PythonNet>`_. + +IronPython + An alternate Python for .NET. Unlike Python.NET, this is a complete Python + implementation that generates IL, and compiles Python code directly to .NET + assemblies. It was created by Jim Hugunin, the original creator of Jython. For + more information, see `the IronPython website + <http://workspaces.gotdotnet.com/ironpython>`_. + +PyPy + An implementation of Python written in Python; even the bytecode interpreter is + written in Python. This is executed using CPython as the underlying + interpreter. One of the goals of the project is to encourage experimentation + with the language itself by making it easier to modify the interpreter (since it + is written in Python). Additional information is available on `the PyPy + project's home page <http://codespeak.net/pypy/>`_. + +Each of these implementations varies in some way from the language as documented +in this manual, or introduces specific information beyond what's covered in the +standard Python documentation. Please refer to the implementation-specific +documentation to determine what else you need to know about the specific +implementation you're using. + + +.. _notation: + +Notation +======== + +.. index:: + single: BNF + single: grammar + single: syntax + single: notation + +The descriptions of lexical analysis and syntax use a modified BNF grammar +notation. This uses the following style of definition: + +.. productionlist:: * + name: `lc_letter` (`lc_letter` | "_")* + lc_letter: "a"..."z" + +The first line says that a ``name`` is an ``lc_letter`` followed by a sequence +of zero or more ``lc_letter``\ s and underscores. An ``lc_letter`` in turn is +any of the single characters ``'a'`` through ``'z'``. (This rule is actually +adhered to for the names defined in lexical and grammar rules in this document.) + +Each rule begins with a name (which is the name defined by the rule) and +``::=``. A vertical bar (``|``) is used to separate alternatives; it is the +least binding operator in this notation. A star (``*``) means zero or more +repetitions of the preceding item; likewise, a plus (``+``) means one or more +repetitions, and a phrase enclosed in square brackets (``[ ]``) means zero or +one occurrences (in other words, the enclosed phrase is optional). The ``*`` +and ``+`` operators bind as tightly as possible; parentheses are used for +grouping. Literal strings are enclosed in quotes. White space is only +meaningful to separate tokens. Rules are normally contained on a single line; +rules with many alternatives may be formatted alternatively with each line after +the first beginning with a vertical bar. + +.. index:: + single: lexical definitions + single: ASCII@ASCII + +In lexical definitions (as the example above), two more conventions are used: +Two literal characters separated by three dots mean a choice of any single +character in the given (inclusive) range of ASCII characters. A phrase between +angular brackets (``<...>``) gives an informal description of the symbol +defined; e.g., this could be used to describe the notion of 'control character' +if needed. + +Even though the notation used is almost the same, there is a big difference +between the meaning of lexical and syntactic definitions: a lexical definition +operates on the individual characters of the input source, while a syntax +definition operates on the stream of tokens generated by the lexical analysis. +All uses of BNF in the next chapter ("Lexical Analysis") are lexical +definitions; uses in subsequent chapters are syntactic definitions. + diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst new file mode 100644 index 0000000000..35e92cf9cb --- /dev/null +++ b/Doc/reference/lexical_analysis.rst @@ -0,0 +1,758 @@ + +.. _lexical: + +**************** +Lexical analysis +**************** + +.. index:: + single: lexical analysis + single: parser + single: token + +A Python program is read by a *parser*. Input to the parser is a stream of +*tokens*, generated by the *lexical analyzer*. This chapter describes how the +lexical analyzer breaks a file into tokens. + +Python uses the 7-bit ASCII character set for program text. + +.. versionadded:: 2.3 + An encoding declaration can be used to indicate that string literals and + comments use an encoding different from ASCII. + +For compatibility with older versions, Python only warns if it finds 8-bit +characters; those warnings should be corrected by either declaring an explicit +encoding, or using escape sequences if those bytes are binary data, instead of +characters. + +The run-time character set depends on the I/O devices connected to the program +but is generally a superset of ASCII. + +**Future compatibility note:** It may be tempting to assume that the character +set for 8-bit characters is ISO Latin-1 (an ASCII superset that covers most +western languages that use the Latin alphabet), but it is possible that in the +future Unicode text editors will become common. These generally use the UTF-8 +encoding, which is also an ASCII superset, but with very different use for the +characters with ordinals 128-255. While there is no consensus on this subject +yet, it is unwise to assume either Latin-1 or UTF-8, even though the current +implementation appears to favor Latin-1. This applies both to the source +character set and the run-time character set. + + +.. _line-structure: + +Line structure +============== + +.. index:: single: line structure + +A Python program is divided into a number of *logical lines*. + + +.. _logical: + +Logical lines +------------- + +.. index:: + single: logical line + single: physical line + single: line joining + single: NEWLINE token + +The end of a logical line is represented by the token NEWLINE. Statements +cannot cross logical line boundaries except where NEWLINE is allowed by the +syntax (e.g., between statements in compound statements). A logical line is +constructed from one or more *physical lines* by following the explicit or +implicit *line joining* rules. + + +.. _physical: + +Physical lines +-------------- + +A physical line is a sequence of characters terminated by an end-of-line +sequence. In source files, any of the standard platform line termination +sequences can be used - the Unix form using ASCII LF (linefeed), the Windows +form using the ASCII sequence CR LF (return followed by linefeed), or the +Macintosh form using the ASCII CR (return) character. All of these forms can be +used equally, regardless of platform. + +When embedding Python, source code strings should be passed to Python APIs using +the standard C conventions for newline characters (the ``\n`` character, +representing ASCII LF, is the line terminator). + + +.. _comments: + +Comments +-------- + +.. index:: + single: comment + single: hash character + +A comment starts with a hash character (``#``) that is not part of a string +literal, and ends at the end of the physical line. A comment signifies the end +of the logical line unless the implicit line joining rules are invoked. Comments +are ignored by the syntax; they are not tokens. + + +.. _encodings: + +Encoding declarations +--------------------- + +.. index:: + single: source character set + single: encodings + +If a comment in the first or second line of the Python script matches the +regular expression ``coding[=:]\s*([-\w.]+)``, this comment is processed as an +encoding declaration; the first group of this expression names the encoding of +the source code file. The recommended forms of this expression are :: + + # -*- coding: <encoding-name> -*- + +which is recognized also by GNU Emacs, and :: + + # vim:fileencoding=<encoding-name> + +which is recognized by Bram Moolenaar's VIM. In addition, if the first bytes of +the file are the UTF-8 byte-order mark (``'\xef\xbb\xbf'``), the declared file +encoding is UTF-8 (this is supported, among others, by Microsoft's +:program:`notepad`). + +If an encoding is declared, the encoding name must be recognized by Python. The +encoding is used for all lexical analysis, in particular to find the end of a +string, and to interpret the contents of Unicode literals. String literals are +converted to Unicode for syntactical analysis, then converted back to their +original encoding before interpretation starts. The encoding declaration must +appear on a line of its own. + +.. % XXX there should be a list of supported encodings. + + +.. _explicit-joining: + +Explicit line joining +--------------------- + +.. index:: + single: physical line + single: line joining + single: line continuation + single: backslash character + +Two or more physical lines may be joined into logical lines using backslash +characters (``\``), as follows: when a physical line ends in a backslash that is +not part of a string literal or comment, it is joined with the following forming +a single logical line, deleting the backslash and the following end-of-line +character. For example: + +.. % + +:: + + if 1900 < year < 2100 and 1 <= month <= 12 \ + and 1 <= day <= 31 and 0 <= hour < 24 \ + and 0 <= minute < 60 and 0 <= second < 60: # Looks like a valid date + return 1 + +A line ending in a backslash cannot carry a comment. A backslash does not +continue a comment. A backslash does not continue a token except for string +literals (i.e., tokens other than string literals cannot be split across +physical lines using a backslash). A backslash is illegal elsewhere on a line +outside a string literal. + + +.. _implicit-joining: + +Implicit line joining +--------------------- + +Expressions in parentheses, square brackets or curly braces can be split over +more than one physical line without using backslashes. For example:: + + month_names = ['Januari', 'Februari', 'Maart', # These are the + 'April', 'Mei', 'Juni', # Dutch names + 'Juli', 'Augustus', 'September', # for the months + 'Oktober', 'November', 'December'] # of the year + +Implicitly continued lines can carry comments. The indentation of the +continuation lines is not important. Blank continuation lines are allowed. +There is no NEWLINE token between implicit continuation lines. Implicitly +continued lines can also occur within triple-quoted strings (see below); in that +case they cannot carry comments. + + +.. _blank-lines: + +Blank lines +----------- + +.. index:: single: blank line + +A logical line that contains only spaces, tabs, formfeeds and possibly a +comment, is ignored (i.e., no NEWLINE token is generated). During interactive +input of statements, handling of a blank line may differ depending on the +implementation of the read-eval-print loop. In the standard implementation, an +entirely blank logical line (i.e. one containing not even whitespace or a +comment) terminates a multi-line statement. + + +.. _indentation: + +Indentation +----------- + +.. index:: + single: indentation + single: whitespace + single: leading whitespace + single: space + single: tab + single: grouping + single: statement grouping + +Leading whitespace (spaces and tabs) at the beginning of a logical line is used +to compute the indentation level of the line, which in turn is used to determine +the grouping of statements. + +First, tabs are replaced (from left to right) by one to eight spaces such that +the total number of characters up to and including the replacement is a multiple +of eight (this is intended to be the same rule as used by Unix). The total +number of spaces preceding the first non-blank character then determines the +line's indentation. Indentation cannot be split over multiple physical lines +using backslashes; the whitespace up to the first backslash determines the +indentation. + +**Cross-platform compatibility note:** because of the nature of text editors on +non-UNIX platforms, it is unwise to use a mixture of spaces and tabs for the +indentation in a single source file. It should also be noted that different +platforms may explicitly limit the maximum indentation level. + +A formfeed character may be present at the start of the line; it will be ignored +for the indentation calculations above. Formfeed characters occurring elsewhere +in the leading whitespace have an undefined effect (for instance, they may reset +the space count to zero). + +.. index:: + single: INDENT token + single: DEDENT token + +The indentation levels of consecutive lines are used to generate INDENT and +DEDENT tokens, using a stack, as follows. + +Before the first line of the file is read, a single zero is pushed on the stack; +this will never be popped off again. The numbers pushed on the stack will +always be strictly increasing from bottom to top. At the beginning of each +logical line, the line's indentation level is compared to the top of the stack. +If it is equal, nothing happens. If it is larger, it is pushed on the stack, and +one INDENT token is generated. If it is smaller, it *must* be one of the +numbers occurring on the stack; all numbers on the stack that are larger are +popped off, and for each number popped off a DEDENT token is generated. At the +end of the file, a DEDENT token is generated for each number remaining on the +stack that is larger than zero. + +Here is an example of a correctly (though confusingly) indented piece of Python +code:: + + def perm(l): + # Compute the list of all permutations of l + if len(l) <= 1: + return [l] + r = [] + for i in range(len(l)): + s = l[:i] + l[i+1:] + p = perm(s) + for x in p: + r.append(l[i:i+1] + x) + return r + +The following example shows various indentation errors:: + + def perm(l): # error: first line indented + for i in range(len(l)): # error: not indented + s = l[:i] + l[i+1:] + p = perm(l[:i] + l[i+1:]) # error: unexpected indent + for x in p: + r.append(l[i:i+1] + x) + return r # error: inconsistent dedent + +(Actually, the first three errors are detected by the parser; only the last +error is found by the lexical analyzer --- the indentation of ``return r`` does +not match a level popped off the stack.) + + +.. _whitespace: + +Whitespace between tokens +------------------------- + +Except at the beginning of a logical line or in string literals, the whitespace +characters space, tab and formfeed can be used interchangeably to separate +tokens. Whitespace is needed between two tokens only if their concatenation +could otherwise be interpreted as a different token (e.g., ab is one token, but +a b is two tokens). + + +.. _other-tokens: + +Other tokens +============ + +Besides NEWLINE, INDENT and DEDENT, the following categories of tokens exist: +*identifiers*, *keywords*, *literals*, *operators*, and *delimiters*. Whitespace +characters (other than line terminators, discussed earlier) are not tokens, but +serve to delimit tokens. Where ambiguity exists, a token comprises the longest +possible string that forms a legal token, when read from left to right. + + +.. _identifiers: + +Identifiers and keywords +======================== + +.. index:: + single: identifier + single: name + +Identifiers (also referred to as *names*) are described by the following lexical +definitions: + +.. productionlist:: + identifier: (`letter`|"_") (`letter` | `digit` | "_")* + letter: `lowercase` | `uppercase` + lowercase: "a"..."z" + uppercase: "A"..."Z" + digit: "0"..."9" + +Identifiers are unlimited in length. Case is significant. + + +.. _keywords: + +Keywords +-------- + +.. index:: + single: keyword + single: reserved word + +The following identifiers are used as reserved words, or *keywords* of the +language, and cannot be used as ordinary identifiers. They must be spelled +exactly as written here:: + + and def for is raise + as del from lambda return + assert elif global not try + break else if or while + class except import pass with + continue finally in print yield + +.. versionchanged:: 2.4 + :const:`None` became a constant and is now recognized by the compiler as a name + for the built-in object :const:`None`. Although it is not a keyword, you cannot + assign a different object to it. + +.. versionchanged:: 2.5 + Both :keyword:`as` and :keyword:`with` are only recognized when the + ``with_statement`` future feature has been enabled. It will always be enabled in + Python 2.6. See section :ref:`with` for details. Note that using :keyword:`as` + and :keyword:`with` as identifiers will always issue a warning, even when the + ``with_statement`` future directive is not in effect. + + +.. _id-classes: + +Reserved classes of identifiers +------------------------------- + +Certain classes of identifiers (besides keywords) have special meanings. These +classes are identified by the patterns of leading and trailing underscore +characters: + +``_*`` + Not imported by ``from module import *``. The special identifier ``_`` is used + in the interactive interpreter to store the result of the last evaluation; it is + stored in the :mod:`__builtin__` module. When not in interactive mode, ``_`` + has no special meaning and is not defined. See section :ref:`import`. + + .. note:: + + The name ``_`` is often used in conjunction with internationalization; + refer to the documentation for the :mod:`gettext` module for more + information on this convention. + +``__*__`` + System-defined names. These names are defined by the interpreter and its + implementation (including the standard library); applications should not expect + to define additional names using this convention. The set of names of this + class defined by Python may be extended in future versions. See section + :ref:`specialnames`. + +``__*`` + Class-private names. Names in this category, when used within the context of a + class definition, are re-written to use a mangled form to help avoid name + clashes between "private" attributes of base and derived classes. See section + :ref:`atom-identifiers`. + + +.. _literals: + +Literals +======== + +.. index:: + single: literal + single: constant + +Literals are notations for constant values of some built-in types. + + +.. _strings: + +String literals +--------------- + +.. index:: single: string literal + +String literals are described by the following lexical definitions: + +.. index:: single: ASCII@ASCII + +.. productionlist:: + stringliteral: [`stringprefix`](`shortstring` | `longstring`) + stringprefix: "r" | "u" | "ur" | "R" | "U" | "UR" | "Ur" | "uR" + shortstring: "'" `shortstringitem`* "'" | '"' `shortstringitem`* '"' + longstring: ""'" `longstringitem`* ""'" + : | '"""' `longstringitem`* '"""' + shortstringitem: `shortstringchar` | `escapeseq` + longstringitem: `longstringchar` | `escapeseq` + shortstringchar: <any source character except "\" or newline or the quote> + longstringchar: <any source character except "\"> + escapeseq: "\" <any ASCII character> + +One syntactic restriction not indicated by these productions is that whitespace +is not allowed between the :token:`stringprefix` and the rest of the string +literal. The source character set is defined by the encoding declaration; it is +ASCII if no encoding declaration is given in the source file; see section +:ref:`encodings`. + +.. index:: + single: triple-quoted string + single: Unicode Consortium + single: string; Unicode + single: raw string + +In plain English: String literals can be enclosed in matching single quotes +(``'``) or double quotes (``"``). They can also be enclosed in matching groups +of three single or double quotes (these are generally referred to as +*triple-quoted strings*). The backslash (``\``) character is used to escape +characters that otherwise have a special meaning, such as newline, backslash +itself, or the quote character. String literals may optionally be prefixed with +a letter ``'r'`` or ``'R'``; such strings are called :dfn:`raw strings` and use +different rules for interpreting backslash escape sequences. A prefix of +``'u'`` or ``'U'`` makes the string a Unicode string. Unicode strings use the +Unicode character set as defined by the Unicode Consortium and ISO 10646. Some +additional escape sequences, described below, are available in Unicode strings. +The two prefix characters may be combined; in this case, ``'u'`` must appear +before ``'r'``. + +In triple-quoted strings, unescaped newlines and quotes are allowed (and are +retained), except that three unescaped quotes in a row terminate the string. (A +"quote" is the character used to open the string, i.e. either ``'`` or ``"``.) + +.. index:: + single: physical line + single: escape sequence + single: Standard C + single: C + +Unless an ``'r'`` or ``'R'`` prefix is present, escape sequences in strings are +interpreted according to rules similar to those used by Standard C. The +recognized escape sequences are: + ++-----------------+---------------------------------+-------+ +| Escape Sequence | Meaning | Notes | ++=================+=================================+=======+ +| ``\newline`` | Ignored | | ++-----------------+---------------------------------+-------+ +| ``\\`` | Backslash (``\``) | | ++-----------------+---------------------------------+-------+ +| ``\'`` | Single quote (``'``) | | ++-----------------+---------------------------------+-------+ +| ``\"`` | Double quote (``"``) | | ++-----------------+---------------------------------+-------+ +| ``\a`` | ASCII Bell (BEL) | | ++-----------------+---------------------------------+-------+ +| ``\b`` | ASCII Backspace (BS) | | ++-----------------+---------------------------------+-------+ +| ``\f`` | ASCII Formfeed (FF) | | ++-----------------+---------------------------------+-------+ +| ``\n`` | ASCII Linefeed (LF) | | ++-----------------+---------------------------------+-------+ +| ``\N{name}`` | Character named *name* in the | | +| | Unicode database (Unicode only) | | ++-----------------+---------------------------------+-------+ +| ``\r`` | ASCII Carriage Return (CR) | | ++-----------------+---------------------------------+-------+ +| ``\t`` | ASCII Horizontal Tab (TAB) | | ++-----------------+---------------------------------+-------+ +| ``\uxxxx`` | Character with 16-bit hex value | \(1) | +| | *xxxx* (Unicode only) | | ++-----------------+---------------------------------+-------+ +| ``\Uxxxxxxxx`` | Character with 32-bit hex value | \(2) | +| | *xxxxxxxx* (Unicode only) | | ++-----------------+---------------------------------+-------+ +| ``\v`` | ASCII Vertical Tab (VT) | | ++-----------------+---------------------------------+-------+ +| ``\ooo`` | Character with octal value | (3,5) | +| | *ooo* | | ++-----------------+---------------------------------+-------+ +| ``\xhh`` | Character with hex value *hh* | (4,5) | ++-----------------+---------------------------------+-------+ + +.. index:: single: ASCII@ASCII + +Notes: + +(1) + Individual code units which form parts of a surrogate pair can be encoded using + this escape sequence. + +(2) + Any Unicode character can be encoded this way, but characters outside the Basic + Multilingual Plane (BMP) will be encoded using a surrogate pair if Python is + compiled to use 16-bit code units (the default). Individual code units which + form parts of a surrogate pair can be encoded using this escape sequence. + +(3) + As in Standard C, up to three octal digits are accepted. + +(4) + Unlike in Standard C, at most two hex digits are accepted. + +(5) + In a string literal, hexadecimal and octal escapes denote the byte with the + given value; it is not necessary that the byte encodes a character in the source + character set. In a Unicode literal, these escapes denote a Unicode character + with the given value. + +.. index:: single: unrecognized escape sequence + +Unlike Standard C, all unrecognized escape sequences are left in the string +unchanged, i.e., *the backslash is left in the string*. (This behavior is +useful when debugging: if an escape sequence is mistyped, the resulting output +is more easily recognized as broken.) It is also important to note that the +escape sequences marked as "(Unicode only)" in the table above fall into the +category of unrecognized escapes for non-Unicode string literals. + +When an ``'r'`` or ``'R'`` prefix is present, a character following a backslash +is included in the string without change, and *all backslashes are left in the +string*. For example, the string literal ``r"\n"`` consists of two characters: +a backslash and a lowercase ``'n'``. String quotes can be escaped with a +backslash, but the backslash remains in the string; for example, ``r"\""`` is a +valid string literal consisting of two characters: a backslash and a double +quote; ``r"\"`` is not a valid string literal (even a raw string cannot end in +an odd number of backslashes). Specifically, *a raw string cannot end in a +single backslash* (since the backslash would escape the following quote +character). Note also that a single backslash followed by a newline is +interpreted as those two characters as part of the string, *not* as a line +continuation. + +When an ``'r'`` or ``'R'`` prefix is used in conjunction with a ``'u'`` or +``'U'`` prefix, then the ``\uXXXX`` and ``\UXXXXXXXX`` escape sequences are +processed while *all other backslashes are left in the string*. For example, +the string literal ``ur"\u0062\n"`` consists of three Unicode characters: 'LATIN +SMALL LETTER B', 'REVERSE SOLIDUS', and 'LATIN SMALL LETTER N'. Backslashes can +be escaped with a preceding backslash; however, both remain in the string. As a +result, ``\uXXXX`` escape sequences are only recognized when there are an odd +number of backslashes. + + +.. _string-catenation: + +String literal concatenation +---------------------------- + +Multiple adjacent string literals (delimited by whitespace), possibly using +different quoting conventions, are allowed, and their meaning is the same as +their concatenation. Thus, ``"hello" 'world'`` is equivalent to +``"helloworld"``. This feature can be used to reduce the number of backslashes +needed, to split long strings conveniently across long lines, or even to add +comments to parts of strings, for example:: + + re.compile("[A-Za-z_]" # letter or underscore + "[A-Za-z0-9_]*" # letter, digit or underscore + ) + +Note that this feature is defined at the syntactical level, but implemented at +compile time. The '+' operator must be used to concatenate string expressions +at run time. Also note that literal concatenation can use different quoting +styles for each component (even mixing raw strings and triple quoted strings). + + +.. _numbers: + +Numeric literals +---------------- + +.. index:: + single: number + single: numeric literal + single: integer literal + single: plain integer literal + single: long integer literal + single: floating point literal + single: hexadecimal literal + single: octal literal + single: binary literal + single: decimal literal + single: imaginary literal + single: complex; literal + +There are four types of numeric literals: plain integers, long integers, +floating point numbers, and imaginary numbers. There are no complex literals +(complex numbers can be formed by adding a real number and an imaginary number). + +Note that numeric literals do not include a sign; a phrase like ``-1`` is +actually an expression composed of the unary operator '``-``' and the literal +``1``. + + +.. _integers: + +Integer literals +---------------- + +Integer literals are described by the following lexical definitions: + +.. productionlist:: + integer: `decimalinteger` | `octinteger` | `hexinteger` + decimalinteger: `nonzerodigit` `digit`* | "0"+ + octinteger: "0" ("o" | "O") `octdigit`+ + hexinteger: "0" ("x" | "X") `hexdigit`+ + bininteger: "0" ("b" | "B") `bindigit`+ + nonzerodigit: "1"..."9" + octdigit: "0"..."7" + hexdigit: `digit` | "a"..."f" | "A"..."F" + bindigit: "0"..."1" + +Plain integer literals that are above the largest representable plain integer +(e.g., 2147483647 when using 32-bit arithmetic) are accepted as if they were +long integers instead. [#]_ There is no limit for long integer literals apart +from what can be stored in available memory. + +Note that leading zeros in a non-zero decimal number are not allowed. This is +for disambiguation with C-style octal literals, which Python used before version +3.0. + +Some examples of integer literals:: + + 7 2147483647 0o177 0b100110111 + 3 79228162514264337593543950336 0o377 0x100000000 + 79228162514264337593543950336 0xdeadbeef + + +.. _floating: + +Floating point literals +----------------------- + +Floating point literals are described by the following lexical definitions: + +.. productionlist:: + floatnumber: `pointfloat` | `exponentfloat` + pointfloat: [`intpart`] `fraction` | `intpart` "." + exponentfloat: (`intpart` | `pointfloat`) `exponent` + intpart: `digit`+ + fraction: "." `digit`+ + exponent: ("e" | "E") ["+" | "-"] `digit`+ + +Note that the integer and exponent parts are always interpreted using radix 10. +For example, ``077e010`` is legal, and denotes the same number as ``77e10``. The +allowed range of floating point literals is implementation-dependent. Some +examples of floating point literals:: + + 3.14 10. .001 1e100 3.14e-10 0e0 + +Note that numeric literals do not include a sign; a phrase like ``-1`` is +actually an expression composed of the unary operator ``-`` and the literal +``1``. + + +.. _imaginary: + +Imaginary literals +------------------ + +Imaginary literals are described by the following lexical definitions: + +.. productionlist:: + imagnumber: (`floatnumber` | `intpart`) ("j" | "J") + +An imaginary literal yields a complex number with a real part of 0.0. Complex +numbers are represented as a pair of floating point numbers and have the same +restrictions on their range. To create a complex number with a nonzero real +part, add a floating point number to it, e.g., ``(3+4j)``. Some examples of +imaginary literals:: + + 3.14j 10.j 10j .001j 1e100j 3.14e-10j + + +.. _operators: + +Operators +========= + +.. index:: single: operators + +The following tokens are operators:: + + + - * ** / // % + << >> & | ^ ~ + < > <= >= == != + + +.. _delimiters: + +Delimiters +========== + +.. index:: single: delimiters + +The following tokens serve as delimiters in the grammar:: + + ( ) [ ] { } @ + , : . ` = ; + += -= *= /= //= %= + &= |= ^= >>= <<= **= + +The period can also occur in floating-point and imaginary literals. A sequence +of three periods has a special meaning as an ellipsis in slices. The second half +of the list, the augmented assignment operators, serve lexically as delimiters, +but also perform an operation. + +The following printing ASCII characters have special meaning as part of other +tokens or are otherwise significant to the lexical analyzer:: + + ' " # \ + +.. index:: single: ASCII@ASCII + +The following printing ASCII characters are not used in Python. Their +occurrence outside string literals and comments is an unconditional error:: + + $ ? + +.. rubric:: Footnotes + +.. [#] In versions of Python prior to 2.4, octal and hexadecimal literals in the range + just above the largest representable plain integer but below the largest + unsigned 32-bit number (on a machine using 32-bit arithmetic), 4294967296, were + taken as the negative plain integer obtained by subtracting 4294967296 from + their unsigned value. + diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst new file mode 100644 index 0000000000..fbc626f7e3 --- /dev/null +++ b/Doc/reference/simple_stmts.rst @@ -0,0 +1,825 @@ + +.. _simple: + +***************** +Simple statements +***************** + +.. index:: pair: simple; statement + +Simple statements are comprised within a single logical line. Several simple +statements may occur on a single line separated by semicolons. The syntax for +simple statements is: + +.. productionlist:: + simple_stmt: `expression_stmt` + : | `assert_stmt` + : | `assignment_stmt` + : | `augmented_assignment_stmt` + : | `pass_stmt` + : | `del_stmt` + : | `return_stmt` + : | `yield_stmt` + : | `raise_stmt` + : | `break_stmt` + : | `continue_stmt` + : | `import_stmt` + : | `global_stmt` + + +.. _exprstmts: + +Expression statements +===================== + +.. index:: pair: expression; statement + +Expression statements are used (mostly interactively) to compute and write a +value, or (usually) to call a procedure (a function that returns no meaningful +result; in Python, procedures return the value ``None``). Other uses of +expression statements are allowed and occasionally useful. The syntax for an +expression statement is: + +.. productionlist:: + expression_stmt: `expression_list` + +.. index:: pair: expression; list + +An expression statement evaluates the expression list (which may be a single +expression). + +.. index:: + builtin: repr + object: None + pair: string; conversion + single: output + pair: standard; output + pair: writing; values + pair: procedure; call + +In interactive mode, if the value is not ``None``, it is converted to a string +using the built-in :func:`repr` function and the resulting string is written to +standard output (see :func:`print`) on a line by itself. (Expression +statements yielding ``None`` are not written, so that procedure calls do not +cause any output.) + + +.. _assert: + +Assert statements +================= + +.. index:: + statement: assert + pair: debugging; assertions + +Assert statements are a convenient way to insert debugging assertions into a +program: + +.. productionlist:: + assert_stmt: "assert" `expression` ["," `expression`] + +The simple form, ``assert expression``, is equivalent to :: + + if __debug__: + if not expression: raise AssertionError + +The extended form, ``assert expression1, expression2``, is equivalent to :: + + if __debug__: + if not expression1: raise AssertionError, expression2 + +.. index:: + single: __debug__ + exception: AssertionError + +These equivalences assume that ``__debug__`` and :exc:`AssertionError` refer to +the built-in variables with those names. In the current implementation, the +built-in variable ``__debug__`` is ``True`` under normal circumstances, +``False`` when optimization is requested (command line option -O). The current +code generator emits no code for an assert statement when optimization is +requested at compile time. Note that it is unnecessary to include the source +code for the expression that failed in the error message; it will be displayed +as part of the stack trace. + +Assignments to ``__debug__`` are illegal. The value for the built-in variable +is determined when the interpreter starts. + + +.. _assignment: + +Assignment statements +===================== + +.. index:: + pair: assignment; statement + pair: binding; name + pair: rebinding; name + object: mutable + pair: attribute; assignment + +Assignment statements are used to (re)bind names to values and to modify +attributes or items of mutable objects: + +.. productionlist:: + assignment_stmt: (`target_list` "=")+ (`expression_list` | `yield_expression`) + target_list: `target` ("," `target`)* [","] + target: `identifier` + : | "(" `target_list` ")" + : | "[" `target_list` "]" + : | `attributeref` + : | `subscription` + : | `slicing` + +(See section :ref:`primaries` for the syntax definitions for the last three +symbols.) + +.. index:: pair: expression; list + +An assignment statement evaluates the expression list (remember that this can be +a single expression or a comma-separated list, the latter yielding a tuple) and +assigns the single resulting object to each of the target lists, from left to +right. + +.. index:: + single: target + pair: target; list + +Assignment is defined recursively depending on the form of the target (list). +When a target is part of a mutable object (an attribute reference, subscription +or slicing), the mutable object must ultimately perform the assignment and +decide about its validity, and may raise an exception if the assignment is +unacceptable. The rules observed by various types and the exceptions raised are +given with the definition of the object types (see section :ref:`types`). + +.. index:: triple: target; list; assignment + +Assignment of an object to a target list is recursively defined as follows. + +* If the target list is a single target: The object is assigned to that target. + +* If the target list is a comma-separated list of targets: The object must be a + sequence with the same number of items as there are targets in the target list, + and the items are assigned, from left to right, to the corresponding targets. + (This rule is relaxed as of Python 1.5; in earlier versions, the object had to + be a tuple. Since strings are sequences, an assignment like ``a, b = "xy"`` is + now legal as long as the string has the right length.) + +Assignment of an object to a single target is recursively defined as follows. + +* If the target is an identifier (name): + + .. index:: statement: global + +* If the name does not occur in a :keyword:`global` statement in the current + code block: the name is bound to the object in the current local namespace. + +* Otherwise: the name is bound to the object in the current global namespace. + + .. index:: single: destructor + + The name is rebound if it was already bound. This may cause the reference count + for the object previously bound to the name to reach zero, causing the object to + be deallocated and its destructor (if it has one) to be called. + + .. % nested + +* If the target is a target list enclosed in parentheses or in square brackets: + The object must be a sequence with the same number of items as there are targets + in the target list, and its items are assigned, from left to right, to the + corresponding targets. + + .. index:: pair: attribute; assignment + +* If the target is an attribute reference: The primary expression in the + reference is evaluated. It should yield an object with assignable attributes; + if this is not the case, :exc:`TypeError` is raised. That object is then asked + to assign the assigned object to the given attribute; if it cannot perform the + assignment, it raises an exception (usually but not necessarily + :exc:`AttributeError`). + + .. index:: + pair: subscription; assignment + object: mutable + +* If the target is a subscription: The primary expression in the reference is + evaluated. It should yield either a mutable sequence object (such as a list) or + a mapping object (such as a dictionary). Next, the subscript expression is + evaluated. + + .. index:: + object: sequence + object: list + + If the primary is a mutable sequence object (such as a list), the subscript must + yield a plain integer. If it is negative, the sequence's length is added to it. + The resulting value must be a nonnegative integer less than the sequence's + length, and the sequence is asked to assign the assigned object to its item with + that index. If the index is out of range, :exc:`IndexError` is raised + (assignment to a subscripted sequence cannot add new items to a list). + + .. index:: + object: mapping + object: dictionary + + If the primary is a mapping object (such as a dictionary), the subscript must + have a type compatible with the mapping's key type, and the mapping is then + asked to create a key/datum pair which maps the subscript to the assigned + object. This can either replace an existing key/value pair with the same key + value, or insert a new key/value pair (if no key with the same value existed). + + .. index:: pair: slicing; assignment + +* If the target is a slicing: The primary expression in the reference is + evaluated. It should yield a mutable sequence object (such as a list). The + assigned object should be a sequence object of the same type. Next, the lower + and upper bound expressions are evaluated, insofar they are present; defaults + are zero and the sequence's length. The bounds should evaluate to (small) + integers. If either bound is negative, the sequence's length is added to it. + The resulting bounds are clipped to lie between zero and the sequence's length, + inclusive. Finally, the sequence object is asked to replace the slice with the + items of the assigned sequence. The length of the slice may be different from + the length of the assigned sequence, thus changing the length of the target + sequence, if the object allows it. + +(In the current implementation, the syntax for targets is taken to be the same +as for expressions, and invalid syntax is rejected during the code generation +phase, causing less detailed error messages.) + +WARNING: Although the definition of assignment implies that overlaps between the +left-hand side and the right-hand side are 'safe' (for example ``a, b = b, a`` +swaps two variables), overlaps *within* the collection of assigned-to variables +are not safe! For instance, the following program prints ``[0, 2]``:: + + x = [0, 1] + i = 0 + i, x[i] = 1, 2 + print x + + +.. _augassign: + +Augmented assignment statements +------------------------------- + +.. index:: + pair: augmented; assignment + single: statement; assignment, augmented + +Augmented assignment is the combination, in a single statement, of a binary +operation and an assignment statement: + +.. productionlist:: + augmented_assignment_stmt: `target` `augop` (`expression_list` | `yield_expression`) + augop: "+=" | "-=" | "*=" | "/=" | "%=" | "**=" + : | ">>=" | "<<=" | "&=" | "^=" | "|=" + +(See section :ref:`primaries` for the syntax definitions for the last three +symbols.) + +An augmented assignment evaluates the target (which, unlike normal assignment +statements, cannot be an unpacking) and the expression list, performs the binary +operation specific to the type of assignment on the two operands, and assigns +the result to the original target. The target is only evaluated once. + +An augmented assignment expression like ``x += 1`` can be rewritten as ``x = x + +1`` to achieve a similar, but not exactly equal effect. In the augmented +version, ``x`` is only evaluated once. Also, when possible, the actual operation +is performed *in-place*, meaning that rather than creating a new object and +assigning that to the target, the old object is modified instead. + +With the exception of assigning to tuples and multiple targets in a single +statement, the assignment done by augmented assignment statements is handled the +same way as normal assignments. Similarly, with the exception of the possible +*in-place* behavior, the binary operation performed by augmented assignment is +the same as the normal binary operations. + +For targets which are attribute references, the initial value is retrieved with +a :meth:`getattr` and the result is assigned with a :meth:`setattr`. Notice +that the two methods do not necessarily refer to the same variable. When +:meth:`getattr` refers to a class variable, :meth:`setattr` still writes to an +instance variable. For example:: + + class A: + x = 3 # class variable + a = A() + a.x += 1 # writes a.x as 4 leaving A.x as 3 + + +.. _pass: + +The :keyword:`pass` statement +============================= + +.. index:: statement: pass + +.. productionlist:: + pass_stmt: "pass" + +.. index:: pair: null; operation + +:keyword:`pass` is a null operation --- when it is executed, nothing happens. +It is useful as a placeholder when a statement is required syntactically, but no +code needs to be executed, for example:: + + def f(arg): pass # a function that does nothing (yet) + + class C: pass # a class with no methods (yet) + + +.. _del: + +The :keyword:`del` statement +============================ + +.. index:: statement: del + +.. productionlist:: + del_stmt: "del" `target_list` + +.. index:: + pair: deletion; target + triple: deletion; target; list + +Deletion is recursively defined very similar to the way assignment is defined. +Rather that spelling it out in full details, here are some hints. + +Deletion of a target list recursively deletes each target, from left to right. + +.. index:: + statement: global + pair: unbinding; name + +Deletion of a name removes the binding of that name from the local or global +namespace, depending on whether the name occurs in a :keyword:`global` statement +in the same code block. If the name is unbound, a :exc:`NameError` exception +will be raised. + +.. index:: pair: free; variable + +It is illegal to delete a name from the local namespace if it occurs as a free +variable in a nested block. + +.. index:: pair: attribute; deletion + +Deletion of attribute references, subscriptions and slicings is passed to the +primary object involved; deletion of a slicing is in general equivalent to +assignment of an empty slice of the right type (but even this is determined by +the sliced object). + + +.. _return: + +The :keyword:`return` statement +=============================== + +.. index:: statement: return + +.. productionlist:: + return_stmt: "return" [`expression_list`] + +.. index:: + pair: function; definition + pair: class; definition + +:keyword:`return` may only occur syntactically nested in a function definition, +not within a nested class definition. + +If an expression list is present, it is evaluated, else ``None`` is substituted. + +:keyword:`return` leaves the current function call with the expression list (or +``None``) as return value. + +.. index:: keyword: finally + +When :keyword:`return` passes control out of a :keyword:`try` statement with a +:keyword:`finally` clause, that :keyword:`finally` clause is executed before +really leaving the function. + +In a generator function, the :keyword:`return` statement is not allowed to +include an :token:`expression_list`. In that context, a bare :keyword:`return` +indicates that the generator is done and will cause :exc:`StopIteration` to be +raised. + + +.. _yield: + +The :keyword:`yield` statement +============================== + +.. index:: statement: yield + +.. productionlist:: + yield_stmt: `yield_expression` + +.. index:: + single: generator; function + single: generator; iterator + single: function; generator + exception: StopIteration + +The :keyword:`yield` statement is only used when defining a generator function, +and is only used in the body of the generator function. Using a :keyword:`yield` +statement in a function definition is sufficient to cause that definition to +create a generator function instead of a normal function. + +When a generator function is called, it returns an iterator known as a generator +iterator, or more commonly, a generator. The body of the generator function is +executed by calling the generator's :meth:`__next__` method repeatedly until it +raises an exception. + +When a :keyword:`yield` statement is executed, the state of the generator is +frozen and the value of :token:`expression_list` is returned to +:meth:`__next__`'s caller. By "frozen" we mean that all local state is +retained, including the current bindings of local variables, the instruction +pointer, and the internal evaluation stack: enough information is saved so that +the next time :meth:`__next__` is invoked, the function can proceed exactly as +if the :keyword:`yield` statement were just another external call. + +As of Python version 2.5, the :keyword:`yield` statement is now allowed in the +:keyword:`try` clause of a :keyword:`try` ... :keyword:`finally` construct. If +the generator is not resumed before it is finalized (by reaching a zero +reference count or by being garbage collected), the generator-iterator's +:meth:`close` method will be called, allowing any pending :keyword:`finally` +clauses to execute. + +.. note:: + + In Python 2.2, the :keyword:`yield` statement is only allowed when the + ``generators`` feature has been enabled. It will always be enabled in Python + 2.3. This ``__future__`` import statement can be used to enable the feature:: + + from __future__ import generators + + +.. seealso:: + + :pep:`0255` - Simple Generators + The proposal for adding generators and the :keyword:`yield` statement to Python. + + :pep:`0342` - Coroutines via Enhanced Generators + The proposal that, among other generator enhancements, proposed allowing + :keyword:`yield` to appear inside a :keyword:`try` ... :keyword:`finally` block. + + +.. _raise: + +The :keyword:`raise` statement +============================== + +.. index:: statement: raise + +.. productionlist:: + raise_stmt: "raise" [`expression` ["," `expression` ["," `expression`]]] + +.. index:: + single: exception + pair: raising; exception + +If no expressions are present, :keyword:`raise` re-raises the last exception +that was active in the current scope. If no exception is active in the current +scope, a :exc:`TypeError` exception is raised indicating that this is an error +(if running under IDLE, a :exc:`Queue.Empty` exception is raised instead). + +Otherwise, :keyword:`raise` evaluates the expressions to get three objects, +using ``None`` as the value of omitted expressions. The first two objects are +used to determine the *type* and *value* of the exception. + +If the first object is an instance, the type of the exception is the class of +the instance, the instance itself is the value, and the second object must be +``None``. + +If the first object is a class, it becomes the type of the exception. The second +object is used to determine the exception value: If it is an instance of the +class, the instance becomes the exception value. If the second object is a +tuple, it is used as the argument list for the class constructor; if it is +``None``, an empty argument list is used, and any other object is treated as a +single argument to the constructor. The instance so created by calling the +constructor is used as the exception value. + +.. index:: object: traceback + +If a third object is present and not ``None``, it must be a traceback object +(see section :ref:`types`), and it is substituted instead of the current +location as the place where the exception occurred. If the third object is +present and not a traceback object or ``None``, a :exc:`TypeError` exception is +raised. The three-expression form of :keyword:`raise` is useful to re-raise an +exception transparently in an except clause, but :keyword:`raise` with no +expressions should be preferred if the exception to be re-raised was the most +recently active exception in the current scope. + +Additional information on exceptions can be found in section :ref:`exceptions`, +and information about handling exceptions is in section :ref:`try`. + + +.. _break: + +The :keyword:`break` statement +============================== + +.. index:: statement: break + +.. productionlist:: + break_stmt: "break" + +.. index:: + statement: for + statement: while + pair: loop; statement + +:keyword:`break` may only occur syntactically nested in a :keyword:`for` or +:keyword:`while` loop, but not nested in a function or class definition within +that loop. + +.. index:: keyword: else + +It terminates the nearest enclosing loop, skipping the optional :keyword:`else` +clause if the loop has one. + +.. index:: pair: loop control; target + +If a :keyword:`for` loop is terminated by :keyword:`break`, the loop control +target keeps its current value. + +.. index:: keyword: finally + +When :keyword:`break` passes control out of a :keyword:`try` statement with a +:keyword:`finally` clause, that :keyword:`finally` clause is executed before +really leaving the loop. + + +.. _continue: + +The :keyword:`continue` statement +================================= + +.. index:: statement: continue + +.. productionlist:: + continue_stmt: "continue" + +.. index:: + statement: for + statement: while + pair: loop; statement + keyword: finally + +:keyword:`continue` may only occur syntactically nested in a :keyword:`for` or +:keyword:`while` loop, but not nested in a function or class definition or +:keyword:`finally` statement within that loop. [#]_ It continues with the next +cycle of the nearest enclosing loop. + + +.. _import: + +The :keyword:`import` statement +=============================== + +.. index:: + statement: import + single: module; importing + pair: name; binding + keyword: from + +.. productionlist:: + import_stmt: "import" `module` ["as" `name`] ( "," `module` ["as" `name`] )* + : | "from" `relative_module` "import" `identifier` ["as" `name`] + : ( "," `identifier` ["as" `name`] )* + : | "from" `relative_module` "import" "(" `identifier` ["as" `name`] + : ( "," `identifier` ["as" `name`] )* [","] ")" + : | "from" `module` "import" "*" + module: (`identifier` ".")* `identifier` + relative_module: "."* `module` | "."+ + name: `identifier` + +Import statements are executed in two steps: (1) find a module, and initialize +it if necessary; (2) define a name or names in the local namespace (of the scope +where the :keyword:`import` statement occurs). The first form (without +:keyword:`from`) repeats these steps for each identifier in the list. The form +with :keyword:`from` performs step (1) once, and then performs step (2) +repeatedly. + +In this context, to "initialize" a built-in or extension module means to call an +initialization function that the module must provide for the purpose (in the +reference implementation, the function's name is obtained by prepending string +"init" to the module's name); to "initialize" a Python-coded module means to +execute the module's body. + +.. index:: + single: modules (in module sys) + single: sys.modules + pair: module; name + pair: built-in; module + pair: user-defined; module + module: sys + pair: filename; extension + triple: module; search; path + +The system maintains a table of modules that have been or are being initialized, +indexed by module name. This table is accessible as ``sys.modules``. When a +module name is found in this table, step (1) is finished. If not, a search for +a module definition is started. When a module is found, it is loaded. Details +of the module searching and loading process are implementation and platform +specific. It generally involves searching for a "built-in" module with the +given name and then searching a list of locations given as ``sys.path``. + +.. index:: + pair: module; initialization + exception: ImportError + single: code block + exception: SyntaxError + +If a built-in module is found, its built-in initialization code is executed and +step (1) is finished. If no matching file is found, :exc:`ImportError` is +raised. If a file is found, it is parsed, yielding an executable code block. If +a syntax error occurs, :exc:`SyntaxError` is raised. Otherwise, an empty module +of the given name is created and inserted in the module table, and then the code +block is executed in the context of this module. Exceptions during this +execution terminate step (1). + +When step (1) finishes without raising an exception, step (2) can begin. + +The first form of :keyword:`import` statement binds the module name in the local +namespace to the module object, and then goes on to import the next identifier, +if any. If the module name is followed by :keyword:`as`, the name following +:keyword:`as` is used as the local name for the module. + +.. index:: + pair: name; binding + exception: ImportError + +The :keyword:`from` form does not bind the module name: it goes through the list +of identifiers, looks each one of them up in the module found in step (1), and +binds the name in the local namespace to the object thus found. As with the +first form of :keyword:`import`, an alternate local name can be supplied by +specifying ":keyword:`as` localname". If a name is not found, +:exc:`ImportError` is raised. If the list of identifiers is replaced by a star +(``'*'``), all public names defined in the module are bound in the local +namespace of the :keyword:`import` statement.. + +.. index:: single: __all__ (optional module attribute) + +The *public names* defined by a module are determined by checking the module's +namespace for a variable named ``__all__``; if defined, it must be a sequence of +strings which are names defined or imported by that module. The names given in +``__all__`` are all considered public and are required to exist. If ``__all__`` +is not defined, the set of public names includes all names found in the module's +namespace which do not begin with an underscore character (``'_'``). +``__all__`` should contain the entire public API. It is intended to avoid +accidentally exporting items that are not part of the API (such as library +modules which were imported and used within the module). + +The :keyword:`from` form with ``*`` may only occur in a module scope. If the +wild card form of import --- ``import *`` --- is used in a function and the +function contains or is a nested block with free variables, the compiler will +raise a :exc:`SyntaxError`. + +.. index:: + keyword: from + statement: from + +.. index:: + triple: hierarchical; module; names + single: packages + single: __init__.py + +**Hierarchical module names:** when the module names contains one or more dots, +the module search path is carried out differently. The sequence of identifiers +up to the last dot is used to find a "package"; the final identifier is then +searched inside the package. A package is generally a subdirectory of a +directory on ``sys.path`` that has a file :file:`__init__.py`. [XXX Can't be +bothered to spell this out right now; see the URL +http://www.python.org/doc/essays/packages.html for more details, also about how +the module search works from inside a package.] + +.. % + +.. index:: builtin: __import__ + +The built-in function :func:`__import__` is provided to support applications +that determine which modules need to be loaded dynamically; refer to +:ref:`built-in-funcs` for additional information. + + +.. _future: + +Future statements +----------------- + +.. index:: pair: future; statement + +A :dfn:`future statement` is a directive to the compiler that a particular +module should be compiled using syntax or semantics that will be available in a +specified future release of Python. The future statement is intended to ease +migration to future versions of Python that introduce incompatible changes to +the language. It allows use of the new features on a per-module basis before +the release in which the feature becomes standard. + +.. productionlist:: * + future_statement: "from" "__future__" "import" feature ["as" name] + : ("," feature ["as" name])* + : | "from" "__future__" "import" "(" feature ["as" name] + : ("," feature ["as" name])* [","] ")" + feature: identifier + name: identifier + +A future statement must appear near the top of the module. The only lines that +can appear before a future statement are: + +* the module docstring (if any), +* comments, +* blank lines, and +* other future statements. + +The features recognized by Python 2.5 are ``absolute_import``, ``division``, +``generators``, ``nested_scopes`` and ``with_statement``. ``generators`` and +``nested_scopes`` are redundant in Python version 2.3 and above because they +are always enabled. + +A future statement is recognized and treated specially at compile time: Changes +to the semantics of core constructs are often implemented by generating +different code. It may even be the case that a new feature introduces new +incompatible syntax (such as a new reserved word), in which case the compiler +may need to parse the module differently. Such decisions cannot be pushed off +until runtime. + +For any given release, the compiler knows which feature names have been defined, +and raises a compile-time error if a future statement contains a feature not +known to it. + +The direct runtime semantics are the same as for any import statement: there is +a standard module :mod:`__future__`, described later, and it will be imported in +the usual way at the time the future statement is executed. + +The interesting runtime semantics depend on the specific feature enabled by the +future statement. + +Note that there is nothing special about the statement:: + + import __future__ [as name] + +That is not a future statement; it's an ordinary import statement with no +special semantics or syntax restrictions. + +Code compiled by calls to the builtin functions :func:`exec` and :func:`compile` +that occur in a module :mod:`M` containing a future +statement will, by default, use the new syntax or semantics associated with the +future statement. This can, starting with Python 2.2 be controlled by optional +arguments to :func:`compile` --- see the documentation of that function +for details. + +A future statement typed at an interactive interpreter prompt will take effect +for the rest of the interpreter session. If an interpreter is started with the +:option:`-i` option, is passed a script name to execute, and the script includes +a future statement, it will be in effect in the interactive session started +after the script is executed. + + +.. _global: + +The :keyword:`global` statement +=============================== + +.. index:: statement: global + +.. productionlist:: + global_stmt: "global" `identifier` ("," `identifier`)* + +.. index:: triple: global; name; binding + +The :keyword:`global` statement is a declaration which holds for the entire +current code block. It means that the listed identifiers are to be interpreted +as globals. It would be impossible to assign to a global variable without +:keyword:`global`, although free variables may refer to globals without being +declared global. + +Names listed in a :keyword:`global` statement must not be used in the same code +block textually preceding that :keyword:`global` statement. + +Names listed in a :keyword:`global` statement must not be defined as formal +parameters or in a :keyword:`for` loop control target, :keyword:`class` +definition, function definition, or :keyword:`import` statement. + +(The current implementation does not enforce the latter two restrictions, but +programs should not abuse this freedom, as future implementations may enforce +them or silently change the meaning of the program.) + +.. index:: + builtin: exec + builtin: eval + builtin: compile + +**Programmer's note:** the :keyword:`global` is a directive to the parser. It +applies only to code parsed at the same time as the :keyword:`global` statement. +In particular, a :keyword:`global` statement contained in a string or code +object supplied to the builtin :func:`exec` function does not affect the code +block *containing* the function call, and code contained in such a string is +unaffected by :keyword:`global` statements in the code containing the function +call. The same applies to the :func:`eval` and :func:`compile` functions. + +.. rubric:: Footnotes + +.. [#] It may occur within an :keyword:`except` or :keyword:`else` clause. The + restriction on occurring in the :keyword:`try` clause is implementor's laziness + and will eventually be lifted. + diff --git a/Doc/reference/toplevel_components.rst b/Doc/reference/toplevel_components.rst new file mode 100644 index 0000000000..2149311783 --- /dev/null +++ b/Doc/reference/toplevel_components.rst @@ -0,0 +1,123 @@ + +.. _top-level: + +******************** +Top-level components +******************** + +.. index:: single: interpreter + +The Python interpreter can get its input from a number of sources: from a script +passed to it as standard input or as program argument, typed in interactively, +from a module source file, etc. This chapter gives the syntax used in these +cases. + + +.. _programs: + +Complete Python programs +======================== + +.. index:: single: program + +.. index:: + module: sys + module: __main__ + module: __builtin__ + +While a language specification need not prescribe how the language interpreter +is invoked, it is useful to have a notion of a complete Python program. A +complete Python program is executed in a minimally initialized environment: all +built-in and standard modules are available, but none have been initialized, +except for :mod:`sys` (various system services), :mod:`__builtin__` (built-in +functions, exceptions and ``None``) and :mod:`__main__`. The latter is used to +provide the local and global namespace for execution of the complete program. + +The syntax for a complete Python program is that for file input, described in +the next section. + +.. index:: + single: interactive mode + module: __main__ + +The interpreter may also be invoked in interactive mode; in this case, it does +not read and execute a complete program but reads and executes one statement +(possibly compound) at a time. The initial environment is identical to that of +a complete program; each statement is executed in the namespace of +:mod:`__main__`. + +.. index:: + single: UNIX + single: command line + single: standard input + +Under Unix, a complete program can be passed to the interpreter in three forms: +with the :option:`-c` *string* command line option, as a file passed as the +first command line argument, or as standard input. If the file or standard input +is a tty device, the interpreter enters interactive mode; otherwise, it executes +the file as a complete program. + + +.. _file-input: + +File input +========== + +All input read from non-interactive files has the same form: + +.. productionlist:: + file_input: (NEWLINE | `statement`)* + +This syntax is used in the following situations: + +* when parsing a complete Python program (from a file or from a string); + +* when parsing a module; + +* when parsing a string passed to the :func:`exec` function; + + +.. _interactive: + +Interactive input +================= + +Input in interactive mode is parsed using the following grammar: + +.. productionlist:: + interactive_input: [`stmt_list`] NEWLINE | `compound_stmt` NEWLINE + +Note that a (top-level) compound statement must be followed by a blank line in +interactive mode; this is needed to help the parser detect the end of the input. + + +.. _expression-input: + +Expression input +================ + +.. index:: single: input + +.. index:: builtin: eval + +There are two forms of expression input. Both ignore leading whitespace. The +string argument to :func:`eval` must have the following form: + +.. productionlist:: + eval_input: `expression_list` NEWLINE* + +.. index:: builtin: input + +The input line read by :func:`input` must have the following form: + +.. productionlist:: + input_input: `expression_list` NEWLINE + +.. index:: + object: file + single: input; raw + single: readline() (file method) + +Note: to read 'raw' input line without interpretation, you can use the the +:meth:`readline` method of file objects, including ``sys.stdin``. + |