diff options
104 files changed, 2528 insertions, 1775 deletions
diff --git a/Doc/bugs.rst b/Doc/bugs.rst index 8b8df81e64..9abe50cd9b 100644 --- a/Doc/bugs.rst +++ b/Doc/bugs.rst @@ -53,7 +53,7 @@ taken on the bug. Article which goes into some detail about how to create a useful bug report. This describes what kind of information is useful and why it is useful. - `Bug Writing Guidelines <http://www.mozilla.org/quality/bug-writing-guidelines.html>`_ + `Bug Writing Guidelines <http://developer.mozilla.org/en/docs/Bug_writing_guidelines>`_ Information about writing a good bug report. Some of this is specific to the Mozilla project, but describes general good practices. diff --git a/Doc/c-api/intro.rst b/Doc/c-api/intro.rst index 2742a53b1c..d165f16630 100644 --- a/Doc/c-api/intro.rst +++ b/Doc/c-api/intro.rst @@ -137,7 +137,7 @@ reference counts for other objects contained in the object if this is a compound object type, such as a list, as well as performing any additional finalization that's needed. There's no chance that the reference count can overflow; at least as many bits are used to hold the reference count as there are distinct -memory locations in virtual memory (assuming ``sizeof(long) >= sizeof(char*)``). +memory locations in virtual memory (assuming ``sizeof(Py_ssize_t) >= sizeof(void*)``). Thus, the reference count increment is a simple operation. It is not necessary to increment an object's reference count for every local diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst index 6c2be3e9f0..36684e2255 100644 --- a/Doc/distutils/apiref.rst +++ b/Doc/distutils/apiref.rst @@ -73,7 +73,7 @@ setup script). Indirectly provides the :class:`distutils.dist.Distribution` and +--------------------+--------------------------------+-------------------------------------------------------------+ | *classifiers* | A list of categories for the | The list of available | | | package | categorizations is at | - | | | http://cheeseshop.python.org/pypi?:action=list_classifiers. | + | | | http://pypi.python.org/pypi?:action=list_classifiers. | +--------------------+--------------------------------+-------------------------------------------------------------+ | *distclass* | the :class:`Distribution` | A subclass of | | | class to use | :class:`distutils.core.Distribution` | diff --git a/Doc/distutils/examples.rst b/Doc/distutils/examples.rst index 4e4adc56d2..d937b83583 100644 --- a/Doc/distutils/examples.rst +++ b/Doc/distutils/examples.rst @@ -11,7 +11,7 @@ Distutils Cookbook. .. seealso:: - `Distutils Cookbook <http://www.python.org/cgi-bin/moinmoin/DistutilsCookbook>`_ + `Distutils Cookbook <http://wiki.python.org/moin/DistutilsCookbook>`_ Collection of recipes showing how to achieve more control over distutils. diff --git a/Doc/distutils/setupscript.rst b/Doc/distutils/setupscript.rst index 7c65821d74..3cc1da92e9 100644 --- a/Doc/distutils/setupscript.rst +++ b/Doc/distutils/setupscript.rst @@ -578,7 +578,7 @@ Notes: (4) These fields should not be used if your package is to be compatible with Python versions prior to 2.2.3 or 2.3. The list is available from the `PyPI website - <http://www.python.org/pypi>`_. + <http://pypi.python.org/pypi>`_. 'short string' A single line of text, not more than 200 characters. diff --git a/Doc/distutils/sourcedist.rst b/Doc/distutils/sourcedist.rst index 9f15870ccd..960cc0ae4d 100644 --- a/Doc/distutils/sourcedist.rst +++ b/Doc/distutils/sourcedist.rst @@ -122,7 +122,8 @@ distribution: * all files in the Distutils "build" tree (default :file:`build/`) -* all files in directories named :file:`RCS`, :file:`CVS` or :file:`.svn` +* all files in directories named :file:`RCS`, :file:`CVS`, :file:`.svn`, + :file:`.hg`, :file:`.git`, :file:`.bzr` or :file:`_darcs` Now we have our complete list of files, which is written to the manifest for future reference, and then used to build the source distribution archive(s). @@ -156,8 +157,9 @@ source distribution: previous two steps, so it's important that the ``prune`` command in the manifest template comes after the ``recursive-include`` command -#. exclude the entire :file:`build` tree, and any :file:`RCS`, :file:`CVS` and - :file:`.svn` directories +#. exclude the entire :file:`build` tree, and any :file:`RCS`, :file:`CVS`, + :file:`.svn`, :file:`.hg`, :file:`.git`, :file:`.bzr` and :file:`_darcs` + directories Just like in the setup script, file and directory names in the manifest template should always be slash-separated; the Distutils will take care of converting diff --git a/Doc/documenting/style.rst b/Doc/documenting/style.rst index 5d0ccb7bed..5821bd80d5 100644 --- a/Doc/documenting/style.rst +++ b/Doc/documenting/style.rst @@ -66,5 +66,5 @@ Unix 1970s. -.. _Apple Publications Style Guide: http://developer.apple.com/documentation/UserExperience/Conceptual/APStyleGuide/AppleStyleGuide2003.pdf +.. _Apple Publications Style Guide: http://developer.apple.com/documentation/UserExperience/Conceptual/APStyleGuide/AppleStyleGuide2006.pdf diff --git a/Doc/howto/advocacy.rst b/Doc/howto/advocacy.rst index 8b5b11c2da..d71e1ec7d0 100644 --- a/Doc/howto/advocacy.rst +++ b/Doc/howto/advocacy.rst @@ -346,7 +346,7 @@ http://www.opensource.org wasn't written commercially. This site presents arguments that show how open source software can have considerable advantages over closed-source software. -http://sunsite.unc.edu/LDP/HOWTO/mini/Advocacy.html +http://www.faqs.org/docs/Linux-mini/Advocacy.html The Linux Advocacy mini-HOWTO was the inspiration for this document, and is also well worth reading for general suggestions on winning acceptance for a new technology, such as Linux or Python. In general, you won't make much progress diff --git a/Doc/howto/curses.rst b/Doc/howto/curses.rst index 841a030f99..b56b2c80d7 100644 --- a/Doc/howto/curses.rst +++ b/Doc/howto/curses.rst @@ -52,7 +52,7 @@ everything, though. No one has made a Windows port of the curses module. On a Windows platform, try the Console module written by Fredrik Lundh. The Console module provides cursor-addressable text output, plus full support for mouse and keyboard input, -and is available from http://effbot.org/efflib/console. +and is available from http://effbot.org/zone/console-index.htm. The Python curses module @@ -432,5 +432,5 @@ ncurses; feel free to add that. If you write an interesting little program, feel free to contribute it as another demo. We can always use more of them! -The ncurses FAQ: http://dickey.his.com/ncurses/ncurses.faq.html +The ncurses FAQ: http://invisible-island.net/ncurses/ncurses.faq.html diff --git a/Doc/howto/regex.rst b/Doc/howto/regex.rst index 40f5fdb5fa..d83665f274 100644 --- a/Doc/howto/regex.rst +++ b/Doc/howto/regex.rst @@ -367,8 +367,8 @@ module. If you have Tkinter available, you may also want to look at Python distribution. It allows you to enter REs and strings, and displays whether the RE matches or fails. :file:`redemo.py` can be quite useful when trying to debug a complicated RE. Phil Schwartz's `Kodos -<http://www.phil-schwartz.com/kodos.spy>`_ is also an interactive tool for -developing and testing RE patterns. +<http://kodos.sourceforge.net/>`_ is also an interactive tool for developing and +testing RE patterns. This HOWTO uses the standard Python interpreter for its examples. First, run the Python interpreter, import the :mod:`re` module, and compile a RE:: diff --git a/Doc/howto/unicode.rst b/Doc/howto/unicode.rst index 67aa2b2947..454d25e7a2 100644 --- a/Doc/howto/unicode.rst +++ b/Doc/howto/unicode.rst @@ -216,10 +216,6 @@ To help understand the standard, Jukka Korpela has written an introductory guide to reading the Unicode character tables, available at <http://www.cs.tut.fi/~jkorpela/unicode/guide.html>. -Roman Czyborra wrote another explanation of Unicode's basic principles; it's at -<http://czyborra.com/unicode/characters.html>. Czyborra has written a number of -other Unicode-related documentation, available from <http://www.cyzborra.com>. - Two other good introductory articles were written by Joel Spolsky <http://www.joelonsoftware.com/articles/Unicode.html> and Jason Orendorff <http://www.jorendorff.com/articles/unicode/>. If this introduction didn't make @@ -429,7 +425,7 @@ The documentation for the :mod:`codecs` module. Marc-André Lemburg gave a presentation at EuroPython 2002 titled "Python and Unicode". A PDF version of his slides is available at -<http://www.egenix.com/files/python/Unicode-EPC2002-Talk.pdf>, and is an +<http://downloads.egenix.com/python/Unicode-EPC2002-Talk.pdf>, and is an excellent overview of the design of Python's Unicode features. @@ -616,7 +612,7 @@ References The PDF slides for Marc-André Lemburg's presentation "Writing Unicode-aware Applications in Python" are available at -<http://www.egenix.com/files/python/LSM2005-Developing-Unicode-aware-applications-in-Python.pdf> +<http://downloads.egenix.com/python/LSM2005-Developing-Unicode-aware-applications-in-Python.pdf> and discuss questions of character encodings as well as how to internationalize and localize an application. diff --git a/Doc/howto/urllib2.rst b/Doc/howto/urllib2.rst index 05588b9dcf..4ba3932db0 100644 --- a/Doc/howto/urllib2.rst +++ b/Doc/howto/urllib2.rst @@ -8,7 +8,7 @@ There is an French translation of an earlier revision of this HOWTO, available at `urllib2 - Le Manuel manquant - <http://www.voidspace/python/articles/urllib2_francais.shtml>`_. + <http://www.voidspace.org.uk/python/articles/urllib2_francais.shtml>`_. diff --git a/Doc/install/index.rst b/Doc/install/index.rst index c830c8de5d..d83e3f2418 100644 --- a/Doc/install/index.rst +++ b/Doc/install/index.rst @@ -869,10 +869,10 @@ Using non-Microsoft compilers on Windows -Borland C++ -^^^^^^^^^^^ +Borland/CodeGear C++ +^^^^^^^^^^^^^^^^^^^^ -This subsection describes the necessary steps to use Distutils with the Borland +This subsection describes the necessary steps to use Distutils with the Borland C++ compiler version 5.5. First you have to know that Borland's object file format (OMF) is different from the format used by the Python version you can download from the Python or ActiveState Web site. (Python is built with @@ -912,7 +912,7 @@ section :ref:`inst-config-files`.) .. seealso:: - `C++Builder Compiler <http://www.borland.com/bcppbuilder/freecompiler/>`_ + `C++Builder Compiler <http://www.codegear.com/downloads/free/cppbuilder>`_ Information about the free C++ compiler from Borland, including links to the download pages. @@ -935,9 +935,7 @@ following steps. These compilers require some special libraries. This task is more complex than for Borland's C++, because there is no program to convert the library. First you have to create a list of symbols which the Python DLL exports. (You can find -a good program for this task at -http://starship.python.net/crew/kernr/mingw32/Notes.html, see at PExports 0.42h -there.) +a good program for this task at http://www.emmestech.com/software/cygwin/pexports-0.43/download_pexports.html) .. I don't understand what the next line means. --amk .. (inclusive the references on data structures.) @@ -981,9 +979,6 @@ Distutils (see section :ref:`inst-config-files`.) `Building Python modules on MS Windows platform with MinGW <http://www.zope.org/Members/als/tips/win32_mingw_modules>`_ Information about building the required libraries for the MinGW environment. - http://pyopengl.sourceforge.net/ftp/win32-stuff/ - Converted import libraries in Cygwin/MinGW and Borland format, and a script to - create the registry entries needed for Distutils to locate the built Python. .. rubric:: Footnotes diff --git a/Doc/library/aepack.rst b/Doc/library/aepack.rst index daaa9b2a97..d8aef6b3c3 100644 --- a/Doc/library/aepack.rst +++ b/Doc/library/aepack.rst @@ -84,7 +84,3 @@ The :mod:`aepack` module defines the following functions: Module :mod:`aetypes` Python definitions of codes for Apple Event descriptor types. - - `Inside Macintosh: Interapplication Communication <http://developer.apple.com/techpubs/mac/IAC/IAC-2.html>`_ - Information about inter-process communications on the Macintosh. - diff --git a/Doc/library/cookielib.rst b/Doc/library/cookielib.rst index bc14051a58..f6008f1104 100644 --- a/Doc/library/cookielib.rst +++ b/Doc/library/cookielib.rst @@ -117,7 +117,7 @@ The following classes are provided: Extensions to this module, including a class for reading Microsoft Internet Explorer cookies on Windows. - http://www.netscape.com/newsref/std/cookie_spec.html + http://wp.netscape.com/newsref/std/cookie_spec.html The specification of the original Netscape cookie protocol. Though this is still the dominant protocol, the 'Netscape cookie protocol' implemented by all the major browsers (and :mod:`cookielib`) only bears a passing resemblance to diff --git a/Doc/library/datetime.rst b/Doc/library/datetime.rst index 3fad2fb583..8db1cb50bc 100644 --- a/Doc/library/datetime.rst +++ b/Doc/library/datetime.rst @@ -1486,9 +1486,31 @@ For :class:`time` objects, the format codes for year, month, and day should not be used, as time objects have no such values. If they're used anyway, ``1900`` is substituted for the year, and ``0`` for the month and day. -For :class:`date` objects, the format codes for hours, minutes, and seconds -should not be used, as :class:`date` objects have no such values. If they're -used anyway, ``0`` is substituted for them. +For :class:`date` objects, the format codes for hours, minutes, seconds, and +microseconds should not be used, as :class:`date` objects have no such +values. If they're used anyway, ``0`` is substituted for them. + +:class:`time` and :class:`datetime` objects support a ``%f`` format code +which expands to the number of microseconds in the object, zero-padded on +the left to six places. + +.. versionadded:: 2.6 + +For a naive object, the ``%z`` and ``%Z`` format codes are replaced by empty +strings. + +For an aware object: + +``%z`` + :meth:`utcoffset` is transformed into a 5-character string of the form +HHMM or + -HHMM, where HH is a 2-digit string giving the number of UTC offset hours, and + MM is a 2-digit string giving the number of UTC offset minutes. For example, if + :meth:`utcoffset` returns ``timedelta(hours=-3, minutes=-30)``, ``%z`` is + replaced with the string ``'-0330'``. + +``%Z`` + If :meth:`tzname` returns ``None``, ``%Z`` is replaced by an empty string. + Otherwise ``%Z`` is replaced by the returned value, which must be a string. The full set of format codes supported varies across platforms, because Python calls the platform C library's :func:`strftime` function, and platform @@ -1521,6 +1543,10 @@ platforms. Regardless of platform, years before 1900 cannot be used. | ``%d`` | Day of the month as a decimal | | | | number [01,31]. | | +-----------+--------------------------------+-------+ +| ``%f`` | Microsecond as a decimal | \(1) | +| | number [0,999999], zero-padded | | +| | on the left | | ++-----------+--------------------------------+-------+ | ``%H`` | Hour (24-hour clock) as a | | | | decimal number [00,23]. | | +-----------+--------------------------------+-------+ @@ -1536,13 +1562,13 @@ platforms. Regardless of platform, years before 1900 cannot be used. | ``%M`` | Minute as a decimal number | | | | [00,59]. | | +-----------+--------------------------------+-------+ -| ``%p`` | Locale's equivalent of either | \(1) | +| ``%p`` | Locale's equivalent of either | \(2) | | | AM or PM. | | +-----------+--------------------------------+-------+ -| ``%S`` | Second as a decimal number | \(2) | +| ``%S`` | Second as a decimal number | \(3) | | | [00,61]. | | +-----------+--------------------------------+-------+ -| ``%U`` | Week number of the year | \(3) | +| ``%U`` | Week number of the year | \(4) | | | (Sunday as the first day of | | | | the week) as a decimal number | | | | [00,53]. All days in a new | | @@ -1553,7 +1579,7 @@ platforms. Regardless of platform, years before 1900 cannot be used. | ``%w`` | Weekday as a decimal number | | | | [0(Sunday),6]. | | +-----------+--------------------------------+-------+ -| ``%W`` | Week number of the year | \(3) | +| ``%W`` | Week number of the year | \(4) | | | (Monday as the first day of | | | | the week) as a decimal number | | | | [00,53]. All days in a new | | @@ -1573,7 +1599,7 @@ platforms. Regardless of platform, years before 1900 cannot be used. | ``%Y`` | Year with century as a decimal | | | | number. | | +-----------+--------------------------------+-------+ -| ``%z`` | UTC offset in the form +HHMM | \(4) | +| ``%z`` | UTC offset in the form +HHMM | \(5) | | | or -HHMM (empty string if the | | | | the object is naive). | | +-----------+--------------------------------+-------+ @@ -1586,17 +1612,22 @@ platforms. Regardless of platform, years before 1900 cannot be used. Notes: (1) + When used with the :func:`strptime` function, the ``%f`` directive + accepts from one to six digits and zero pads on the right. ``%f`` is + an extension to the set of format characters in the C standard. + +(2) When used with the :func:`strptime` function, the ``%p`` directive only affects the output hour field if the ``%I`` directive is used to parse the hour. -(2) +(3) The range really is ``0`` to ``61``; this accounts for leap seconds and the (very rare) double leap seconds. -(3) +(4) When used with the :func:`strptime` function, ``%U`` and ``%W`` are only used in calculations when the day of the week and the year are specified. -(4) +(5) For example, if :meth:`utcoffset` returns ``timedelta(hours=-3, minutes=-30)``, ``%z`` is replaced with the string ``'-0330'``. diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 5b0984788c..3af9250ebc 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -412,21 +412,24 @@ Miscellaneous opcodes. .. opcode:: WITH_CLEANUP () - Cleans up the stack when a :keyword:`with` statement block exits. TOS is the - context manager's :meth:`__exit__` bound method. Below that are 1--3 values - indicating how/why the finally clause was entered: + Cleans up the stack when a :keyword:`with` statement block exits. On top of + the stack are 1--3 values indicating how/why the finally clause was entered: - * SECOND = ``None`` - * (SECOND, THIRD) = (``WHY_{RETURN,CONTINUE}``), retval - * SECOND = ``WHY_*``; no retval below it - * (SECOND, THIRD, FOURTH) = exc_info() + * TOP = ``None`` + * (TOP, SECOND) = (``WHY_{RETURN,CONTINUE}``), retval + * TOP = ``WHY_*``; no retval below it + * (TOP, SECOND, THIRD) = exc_info() - In the last case, ``TOS(SECOND, THIRD, FOURTH)`` is called, otherwise - ``TOS(None, None, None)``. + Under them is EXIT, the context manager's :meth:`__exit__` bound method. - In addition, if the stack represents an exception, *and* the function call - returns a 'true' value, this information is "zapped", to prevent ``END_FINALLY`` - from re-raising the exception. (But non-local gotos should still be resumed.) + In the last case, ``EXIT(TOP, SECOND, THIRD)`` is called, otherwise + ``EXIT(None, None, None)``. + + EXIT is removed from the stack, leaving the values above it in the same + order. In addition, if the stack represents an exception, *and* the function + call returns a 'true' value, this information is "zapped", to prevent + ``END_FINALLY`` from re-raising the exception. (But non-local gotos should + still be resumed.) .. XXX explain the WHY stuff! diff --git a/Doc/library/email.message.rst b/Doc/library/email.message.rst index 7f3cf6f71d..2b0df3497a 100644 --- a/Doc/library/email.message.rst +++ b/Doc/library/email.message.rst @@ -38,7 +38,7 @@ Here are the methods of the :class:`Message` class: .. method:: Message.as_string([unixfrom]) - Return the entire message flatten as a string. When optional *unixfrom* is + Return the entire message flattened as a string. When optional *unixfrom* is ``True``, the envelope header is included in the returned string. *unixfrom* defaults to ``False``. diff --git a/Doc/library/heapq.rst b/Doc/library/heapq.rst index f26d29b9d6..1530144918 100644 --- a/Doc/library/heapq.rst +++ b/Doc/library/heapq.rst @@ -43,6 +43,13 @@ The following functions are provided: Pop and return the smallest item from the *heap*, maintaining the heap invariant. If the heap is empty, :exc:`IndexError` is raised. +.. function:: heappushpop(heap, item) + + Push *item* on the heap, then pop and return the smallest item from the + *heap*. The combined action runs more efficiently than :func:`heappush` + followed by a separate call to :func:`heappop`. + + .. versionadded:: 2.6 .. function:: heapify(x) diff --git a/Doc/library/imaplib.rst b/Doc/library/imaplib.rst index 10f3a42cfa..01249b2c20 100644 --- a/Doc/library/imaplib.rst +++ b/Doc/library/imaplib.rst @@ -116,7 +116,7 @@ example of usage. Documents describing the protocol, and sources and binaries for servers implementing it, can all be found at the University of Washington's *IMAP - Information Center* (http://www.cac.washington.edu/imap/). + Information Center* (http://www.washington.edu/imap/). .. _imap4-objects: diff --git a/Doc/library/mailbox.rst b/Doc/library/mailbox.rst index e05f44a1ec..b5773f4e0d 100644 --- a/Doc/library/mailbox.rst +++ b/Doc/library/mailbox.rst @@ -403,7 +403,7 @@ remarks: Notes on Maildir by its inventor. Includes an updated name-creation scheme and details on "info" semantics. - `maildir man page from Courier <http://www.courier-mta.org/?maildir.html>`_ + `maildir man page from Courier <http://www.courier-mta.org/maildir.html>`_ Another specification of the format. Describes a common extension for supporting folders. @@ -460,7 +460,7 @@ remarks: `mbox man page from tin <http://www.tin.org/bin/man.cgi?section=5&topic=mbox>`_ Another specification of the format, with details on locking. - `Configuring Netscape Mail on Unix: Why The Content-Length Format is Bad <http://home.netscape.com/eng/mozilla/2.0/relnotes/demo/content-length.html>`_ + `Configuring Netscape Mail on Unix: Why The Content-Length Format is Bad <http://www.jwz.org/doc/content-length.html>`_ An argument for using the original mbox format rather than a variation. `"mbox" is a family of several mutually incompatible mailbox formats <http://homepages.tesco.net./~J.deBoynePollard/FGA/mail-mbox-formats.html>`_ @@ -664,7 +664,7 @@ remarks: `Format of Version 5 Babyl Files <http://quimby.gnus.org/notes/BABYL>`_ A specification of the Babyl format. - `Reading Mail with Rmail <http://www.gnu.org/software/emacs/manual/html_node/Rmail.html>`_ + `Reading Mail with Rmail <http://www.gnu.org/software/emacs/manual/html_node/emacs/Rmail.html>`_ The Rmail manual, with some information on Babyl semantics. @@ -1540,10 +1540,6 @@ counterparts are as follows: :class:`UnixMailbox` except that individual messages are separated by only ``From`` lines. - For more information, see `Configuring Netscape Mail on Unix: Why the - Content-Length Format is Bad - <http://home.netscape.com/eng/mozilla/2.0/relnotes/demo/content-length.html>`_. - .. class:: PortableUnixMailbox(fp[, factory]) @@ -1629,7 +1625,7 @@ format-specific information that can be converted:: destination = mailbox.MH('~/Mail') destination.lock() for message in mailbox.Babyl('~/RMAIL'): - destination.add(MHMessage(message)) + destination.add(mailbox.MHMessage(message)) destination.flush() destination.unlock() diff --git a/Doc/library/mimetools.rst b/Doc/library/mimetools.rst index 8da385e0ce..7813867476 100644 --- a/Doc/library/mimetools.rst +++ b/Doc/library/mimetools.rst @@ -73,7 +73,7 @@ It defines the following items: Module :mod:`multifile` Support for reading files which contain distinct parts, such as MIME data. - http://www.cs.uu.nl/wais/html/na-dir/mail/mime-faq/.html + http://faqs.cs.uu.nl/na-dir/mail/mime-faq/.html The MIME Frequently Asked Questions document. For an overview of MIME, see the answer to question 1.1 in Part 1 of this document. diff --git a/Doc/library/mimetypes.rst b/Doc/library/mimetypes.rst index f59cd829a9..cc11dfcae7 100644 --- a/Doc/library/mimetypes.rst +++ b/Doc/library/mimetypes.rst @@ -41,7 +41,7 @@ the information :func:`init` sets up. Optional *strict* is a flag specifying whether the list of known MIME types is limited to only the official types `registered with IANA - <http://www.isi.edu/in-notes/iana/assignments/media-types>`_ are recognized. + <http://www.iana.org/assignments/media-types/>`_ are recognized. When *strict* is true (the default), only the IANA types are supported; when *strict* is false, some additional non-standard but commonly used MIME types are also recognized. diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst index ca056aba8d..590098bc37 100644 --- a/Doc/library/operator.rst +++ b/Doc/library/operator.rst @@ -428,21 +428,43 @@ expect a function argument. .. function:: itemgetter(item[, args...]) - Return a callable object that fetches *item* from its operand. If more than one - item is requested, returns a tuple of items. After, ``f=itemgetter(2)``, the - call ``f(b)`` returns ``b[2]``. After, ``f=itemgetter(2,5,3)``, the call - ``f(b)`` returns ``(b[2], b[5], b[3])``. - - -Examples:: - - >>> from operator import itemgetter - >>> inventory = [('apple', 3), ('banana', 2), ('pear', 5), ('orange', 1)] - >>> getcount = itemgetter(1) - >>> map(getcount, inventory) - [3, 2, 5, 1] - >>> sorted(inventory, key=getcount) - [('orange', 1), ('banana', 2), ('apple', 3), ('pear', 5)] + Return a callable object that fetches *item* from its operand using the + operand's :meth:`__getitem__` method. If multiple items are specified, + returns a tuple of lookup values. Equivalent to:: + + def itemgetter(*items): + if len(items) == 1: + item = items[0] + def g(obj): + return obj[item] + else: + def g(obj): + return tuple(obj[item] for item in items) + return g + + The items can be any type accepted by the operand's :meth:`__getitem__` + method. Dictionaries accept any hashable value. Lists, tuples, and + strings accept an index or a slice:: + + >>> itemgetter(1)('ABCDEFG') + 'B' + >>> itemgetter(1,3,5)('ABCDEFG') + ('B', 'D', 'F') + >>> itemgetter(slice(2,None))('ABCDEFG') + 'CDEFG' + + .. versionadded:: 2.4 + + Example of using :func:`itemgetter` to retrieve specific fields from a + tuple record:: + + >>> from operator import itemgetter + >>> inventory = [('apple', 3), ('banana', 2), ('pear', 5), ('orange', 1)] + >>> getcount = itemgetter(1) + >>> map(getcount, inventory) + [3, 2, 5, 1] + >>> sorted(inventory, key=getcount) + [('orange', 1), ('banana', 2), ('apple', 3), ('pear', 5)] .. function:: methodcaller(name[, args...]) diff --git a/Doc/library/os.rst b/Doc/library/os.rst index ef81304bfc..ec35c3b8e6 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -1387,7 +1387,8 @@ written in Python, such as a mail server's external command delivery program. .. function:: fork() Fork a child process. Return ``0`` in the child and the child's process id in the - parent. Availability: Macintosh, Unix. + parent. If an error occurs :exc:`OSError` is raised. + Availability: Macintosh, Unix. .. function:: forkpty() @@ -1396,7 +1397,8 @@ written in Python, such as a mail server's external command delivery program. terminal. Return a pair of ``(pid, fd)``, where *pid* is ``0`` in the child, the new child's process id in the parent, and *fd* is the file descriptor of the master end of the pseudo-terminal. For a more portable approach, use the - :mod:`pty` module. Availability: Macintosh, some flavors of Unix. + :mod:`pty` module. If an error occurs :exc:`OSError` is raised. + Availability: Macintosh, some flavors of Unix. .. function:: kill(pid, sig) diff --git a/Doc/library/othergui.rst b/Doc/library/othergui.rst index b36568d835..ab4efdb3b5 100644 --- a/Doc/library/othergui.rst +++ b/Doc/library/othergui.rst @@ -36,14 +36,12 @@ also available for Python: `PyGTK <http://www.pygtk.org/>`_ is a set of bindings for the `GTK <http://www.gtk.org/>`_ widget set. It - provides an object oriented interface that is slightly higher level than the C - one. It comes with many more widgets than Tkinter provides, and - has good Python-specific reference documentation. There are also `bindings - <http://www.daa.com.au/~james/gnome/>`_ to `GNOME <http://www.gnome.org>`_. - One well known PyGTK application is - `PythonCAD <http://www.pythoncad.org/>`_. An - online `tutorial <http://www.pygtk.org/pygtk2tutorial/index.html>`_ is - available. + provides an object oriented interface that is slightly higher level than + the C one. It comes with many more widgets than Tkinter provides, and has + good Python-specific reference documentation. There are also bindings to + `GNOME <http://www.gnome.org>`_. One well known PyGTK application is + `PythonCAD <http://www.pythoncad.org/>`_. An online `tutorial + <http://www.pygtk.org/pygtk2tutorial/index.html>`_ is available. `PyQt <http://www.riverbankcomputing.co.uk/pyqt/index.php>`_ PyQt is a :program:`sip`\ -wrapped binding to the Qt toolkit. Qt is an diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst index 017fbd6ea6..df8cf6c47a 100644 --- a/Doc/library/pdb.rst +++ b/Doc/library/pdb.rst @@ -103,9 +103,12 @@ slightly different way: being debugged (e.g. when an assertion fails). -.. function:: post_mortem(traceback) +.. function:: post_mortem([traceback]) - Enter post-mortem debugging of the given *traceback* object. + Enter post-mortem debugging of the given *traceback* object. If no + *traceback* is given, it uses the one of the exception that is currently + being handled (an exception must be being handled if the default is to be + used). .. function:: pm() diff --git a/Doc/library/py_compile.rst b/Doc/library/py_compile.rst index de9a80e2a6..77ed8cf4a0 100644 --- a/Doc/library/py_compile.rst +++ b/Doc/library/py_compile.rst @@ -42,7 +42,12 @@ byte-code cache files in the directory containing the source code. structure to locate source files; it only compiles files named explicitly. When this module is run as a script, the :func:`main` is used to compile all the -files named on the command line. +files named on the command line. The exit status is nonzero if one of the files +could not be compiled. + +.. versionchanged:: 2.6 + + Added the nonzero exit status. .. seealso:: diff --git a/Doc/library/re.rst b/Doc/library/re.rst index c3ec777890..d7eb6f6fbf 100644 --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -539,14 +539,26 @@ form. >>> re.split('\W+', 'Words, words, words.', 1) ['Words', 'words, words.'] + If there are capturing groups in the separator and it matches at the start of + the string, the result will start with an empty string. The same holds for + the end of the string:: + + >>> re.split('(\W+)', '...words, words...') + ['', '...', 'words', ', ', 'words', '...', ''] + + That way, separator components are always found at the same relative + indices within the result list (e.g., if there's one capturing group + in the separator, the 0th, the 2nd and so forth). + Note that *split* will never split a string on an empty pattern match. - For example :: + For example:: >>> re.split('x*', 'foo') ['foo'] >>> re.split("(?m)^$", "foo\n\nbar\n") ['foo\n\nbar\n'] + .. function:: findall(pattern, string[, flags]) Return all non-overlapping matches of *pattern* in *string*, as a list of diff --git a/Doc/library/robotparser.rst b/Doc/library/robotparser.rst index 2451799208..6cc7df898e 100644 --- a/Doc/library/robotparser.rst +++ b/Doc/library/robotparser.rst @@ -15,9 +15,8 @@ This module provides a single class, :class:`RobotFileParser`, which answers questions about whether or not a particular user agent can fetch a URL on the -Web site that published the :file:`robots.txt` file. For more details on the -structure of :file:`robots.txt` files, see -http://www.robotstxt.org/wc/norobots.html. +Web site that published the :file:`robots.txt` file. For more details on the +structure of :file:`robots.txt` files, see http://www.robotstxt.org/orig.html. .. class:: RobotFileParser() diff --git a/Doc/library/socketserver.rst b/Doc/library/socketserver.rst index 2c85c863a2..a8eb9532bd 100644 --- a/Doc/library/socketserver.rst +++ b/Doc/library/socketserver.rst @@ -113,7 +113,8 @@ or inappropriate for the service) is to maintain an explicit table of partially finished requests and to use :func:`select` to decide which request to work on next (or whether to handle a new incoming request). This is particularly important for stream services where each client can potentially be connected for -a long time (if threads or subprocesses cannot be used). +a long time (if threads or subprocesses cannot be used). See :mod:`asyncore` for +another way to manage this. .. XXX should data and methods be intermingled, or separate? how should the distinction between class and instance variables be drawn? @@ -132,16 +133,24 @@ Server Objects .. function:: handle_request() - Process a single request. This function calls the following methods in order: - :meth:`get_request`, :meth:`verify_request`, and :meth:`process_request`. If - the user-provided :meth:`handle` method of the handler class raises an - exception, the server's :meth:`handle_error` method will be called. + Process a single request. This function calls the following methods in + order: :meth:`get_request`, :meth:`verify_request`, and + :meth:`process_request`. If the user-provided :meth:`handle` method of the + handler class raises an exception, the server's :meth:`handle_error` method + will be called. If no request is received within :attr:`self.timeout` + seconds, :meth:`handle_timeout` will be called and :meth:`handle_request` + will return. -.. function:: serve_forever() +.. function:: serve_forever(poll_interval=0.5) - Handle an infinite number of requests. This simply calls :meth:`handle_request` - inside an infinite loop. + Handle requests until an explicit :meth:`shutdown` request. Polls for + shutdown every *poll_interval* seconds. + + +.. function:: shutdown() + + Tells the :meth:`serve_forever` loop to stop and waits until it does. .. data:: address_family @@ -195,10 +204,9 @@ The server classes support the following class variables: .. data:: timeout - Timeout duration, measured in seconds, or :const:`None` if no timeout is desired. - If no incoming requests are received within the timeout period, - the :meth:`handle_timeout` method is called and then the server resumes waiting for - requests. + Timeout duration, measured in seconds, or :const:`None` if no timeout is + desired. If :meth:`handle_request` receives no incoming requests within the + timeout period, the :meth:`handle_timeout` method is called. There are various server methods that can be overridden by subclasses of base server classes like :class:`TCPServer`; these methods aren't useful to external diff --git a/Doc/library/struct.rst b/Doc/library/struct.rst index ee568e6241..924cb606ea 100644 --- a/Doc/library/struct.rst +++ b/Doc/library/struct.rst @@ -73,7 +73,7 @@ Python values should be obvious given their types: +--------+-------------------------+--------------------+-------+ | ``B`` | :ctype:`unsigned char` | integer | | +--------+-------------------------+--------------------+-------+ -| ``t`` | :ctype:`_Bool` | bool | \(1) | +| ``?`` | :ctype:`_Bool` | bool | \(1) | +--------+-------------------------+--------------------+-------+ | ``h`` | :ctype:`short` | integer | | +--------+-------------------------+--------------------+-------+ @@ -106,7 +106,7 @@ Python values should be obvious given their types: Notes: (1) - The ``'t'`` conversion code corresponds to the :ctype:`_Bool` type defined by + The ``'?'`` conversion code corresponds to the :ctype:`_Bool` type defined by C99. If this type is not available, it is simulated using a :ctype:`char`. In standard mode, it is always represented by one byte. @@ -141,7 +141,7 @@ bytes, but that the string returned can never contain more than 255 characters. -For the ``'t'`` format character, the return value is either :const:`True` or +For the ``'?'`` format character, the return value is either :const:`True` or :const:`False`. When packing, the truth value of the argument object is used. Either 0 or 1 in the native or standard bool representation will be packed, and any non-zero value will be True when unpacking. diff --git a/Doc/library/threading.rst b/Doc/library/threading.rst index 6f3e95b4c0..3f62e6947d 100644 --- a/Doc/library/threading.rst +++ b/Doc/library/threading.rst @@ -724,3 +724,26 @@ Currently, :class:`Lock`, :class:`RLock`, :class:`Condition`, with some_rlock: print("some_rlock is locked while this executes") + +.. _threaded-imports: + +Importing in threaded code +-------------------------- + +While the import machinery is thread safe, there are two key +restrictions on threaded imports due to inherent limitations in the way +that thread safety is provided: + +* Firstly, other than in the main module, an import should not have the + side effect of spawning a new thread and then waiting for that thread in + any way. Failing to abide by this restriction can lead to a deadlock if + the spawned thread directly or indirectly attempts to import a module. +* Secondly, all import attempts must be completed before the interpreter + starts shutting itself down. This can be most easily achieved by only + performing imports from non-daemon threads created through the threading + module. Daemon threads and threads created directly with the thread + module will require some other form of synchronization to ensure they do + not attempt imports after system shutdown has commenced. Failure to + abide by this restriction will lead to intermittent exceptions and + crashes during interpreter shutdown (as the late imports attempt to + access machinery which is no longer in a valid state). diff --git a/Doc/library/tix.rst b/Doc/library/tix.rst index f4bb20f871..ad14cc4ae2 100644 --- a/Doc/library/tix.rst +++ b/Doc/library/tix.rst @@ -35,7 +35,7 @@ special needs of your application and users. `Tix Programming Guide <http://tix.sourceforge.net/dist/current/docs/tix-book/tix.book.html>`_ On-line version of the programmer's reference material. - `Tix Development Applications <http://tix.sourceforge.net/Tide/>`_ + `Tix Development Applications <http://tix.sourceforge.net/Tixapps/src/Tide.html>`_ Tix applications for development of Tix and Tkinter programs. Tide applications work under Tk or Tkinter, and include :program:`TixInspect`, an inspector to remotely modify and debug Tix/Tk/Tkinter applications. diff --git a/Doc/library/tkinter.rst b/Doc/library/tkinter.rst index 4d5cce8643..e03d959b1b 100644 --- a/Doc/library/tkinter.rst +++ b/Doc/library/tkinter.rst @@ -21,7 +21,7 @@ of Python; it is maintained at ActiveState.) `An Introduction to Tkinter <http://www.pythonware.com/library/an-introduction-to-tkinter.htm>`_ Fredrik Lundh's on-line reference material. - `Tkinter reference: a GUI for Python <http://www.nmt.edu/tcc/help/pubs/lang.html>`_ + `Tkinter reference: a GUI for Python <http://infohost.nmt.edu/tcc/help/pubs/lang.html>`_ On-line reference material. `Tkinter for JPython <http://jtkinter.sourceforge.net>`_ diff --git a/Doc/library/unittest.rst b/Doc/library/unittest.rst index 7703ad9337..8188e7044a 100644 --- a/Doc/library/unittest.rst +++ b/Doc/library/unittest.rst @@ -562,6 +562,7 @@ failures. .. method:: TestCase.assert_(expr[, msg]) TestCase.failUnless(expr[, msg]) + TestCase.assertTrue(expr[, msg]) Signal a test failure if *expr* is false; the explanation for the error will be *msg* if given, otherwise it will be :const:`None`. @@ -618,6 +619,7 @@ failures. .. method:: TestCase.failIf(expr[, msg]) + TestCase.assertFalse(expr[, msg]) The inverse of the :meth:`failUnless` method is the :meth:`failIf` method. This signals a test failure if *expr* is true, with *msg* or :const:`None` for the diff --git a/Doc/library/urllib.rst b/Doc/library/urllib.rst index 63fb53e0f8..e976cb1911 100644 --- a/Doc/library/urllib.rst +++ b/Doc/library/urllib.rst @@ -107,10 +107,6 @@ High-level interface filehandle = urllib.urlopen(some_url, proxies=None) filehandle = urllib.urlopen(some_url) - The :func:`urlopen` function does not support explicit proxy specification. If - you need to override environmental proxy settings, use :class:`URLopener`, or a - subclass such as :class:`FancyURLopener`. - Proxies which require authentication for use are not currently supported; this is considered an implementation limitation. diff --git a/Doc/library/xml.dom.rst b/Doc/library/xml.dom.rst index 768c9c8b8d..2a0c788abb 100644 --- a/Doc/library/xml.dom.rst +++ b/Doc/library/xml.dom.rst @@ -506,7 +506,7 @@ Document Objects ^^^^^^^^^^^^^^^^ A :class:`Document` represents an entire XML document, including its constituent -elements, attributes, processing instructions, comments etc. Remeber that it +elements, attributes, processing instructions, comments etc. Remember that it inherits properties from :class:`Node`. diff --git a/Doc/library/zipfile.rst b/Doc/library/zipfile.rst index c90f946a51..b1c903864a 100644 --- a/Doc/library/zipfile.rst +++ b/Doc/library/zipfile.rst @@ -11,7 +11,7 @@ The ZIP file format is a common archive and compression standard. This module provides tools to create, read, write, append, and list a ZIP file. Any advanced use of this module will require an understanding of the format, as defined in `PKZIP Application Note -<http://www.pkware.com/business_and_developers/developer/appnote/>`_. +<http://www.pkware.com/documents/casestudies/APPNOTE.TXT>`_. This module does not currently handle multi-disk ZIP files, or ZIP files which have appended comments (although it correctly handles comments @@ -81,7 +81,7 @@ The module defines the following items: .. seealso:: - `PKZIP Application Note <http://www.pkware.com/business_and_developers/developer/appnote/>`_ + `PKZIP Application Note <http://www.pkware.com/documents/casestudies/APPNOTE.TXT>`_ Documentation on the ZIP file format by Phil Katz, the creator of the format and algorithms used. @@ -361,7 +361,7 @@ Instances have the following attributes: .. attribute:: ZipInfo.extra Expansion field data. The `PKZIP Application Note - <http://www.pkware.com/business_and_developers/developer/appnote/>`_ contains + <http://www.pkware.com/documents/casestudies/APPNOTE.TXT>`_ contains some comments on the internal structure of the data contained in this string. diff --git a/Doc/library/zipimport.rst b/Doc/library/zipimport.rst index ed9c631469..f8d0b1d18b 100644 --- a/Doc/library/zipimport.rst +++ b/Doc/library/zipimport.rst @@ -29,7 +29,7 @@ doesn't contain :file:`.pyc` files, importing may be rather slow. .. seealso:: - `PKZIP Application Note <http://www.pkware.com/business_and_developers/developer/appnote/>`_ + `PKZIP Application Note <http://www.pkware.com/documents/casestudies/APPNOTE.TXT>`_ Documentation on the ZIP file format by Phil Katz, the creator of the format and algorithms used. diff --git a/Doc/license.rst b/Doc/license.rst index 6ffa65f71a..0226ec5311 100644 --- a/Doc/license.rst +++ b/Doc/license.rst @@ -347,7 +347,7 @@ Sockets The :mod:`socket` module uses the functions, :func:`getaddrinfo`, and :func:`getnameinfo`, which are coded in separate source files from the WIDE -Project, http://www.wide.ad.jp/about/index.html. :: +Project, http://www.wide.ad.jp/. :: Copyright (C) 1995, 1996, 1997, and 1998 WIDE Project. All rights reserved. diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 74833dfdf7..5b590ce0ce 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -52,6 +52,7 @@ Summarizing: : | `with_stmt` : | `funcdef` : | `classdef` + : | `decorated` suite: `stmt_list` NEWLINE | NEWLINE INDENT `statement`+ DEDENT statement: `stmt_list` NEWLINE | `compound_stmt` stmt_list: `simple_stmt` (";" `simple_stmt`)* [";"] @@ -424,6 +425,7 @@ A function definition defines a user-defined function object (see section funcdef: [`decorators`] "def" `funcname` "(" [`parameter_list`] ")" ["->" `expression`]? ":" `suite` decorators: `decorator`+ decorator: "@" `dotted_name` ["(" [`argument_list` [","]] ")"] NEWLINE + funcdef: "def" `funcname` "(" [`parameter_list`] ")" ":" `suite` dotted_name: `identifier` ("." `identifier`)* parameter_list: (`defparameter` ",")* : ( "*" [`parameter`] ("," `defparameter`)* @@ -585,6 +587,10 @@ implementation details. :pep:`3129` - Class Decorators +Class definitions, like function definitions, may be wrapped by one or +more :term:`decorator` expressions. The evaluation rules for the +decorator expressions are the same as for functions. The result must +be a class object, which is then bound to the class name. .. rubric:: Footnotes diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index 1dc49f3331..e149710a5a 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -539,9 +539,13 @@ The :keyword:`continue` statement :keyword:`continue` may only occur syntactically nested in a :keyword:`for` or :keyword:`while` loop, but not nested in a function or class definition or -:keyword:`finally` statement within that loop. [#]_ It continues with the next +:keyword:`finally` clause within that loop. It continues with the next cycle of the nearest enclosing loop. +When :keyword:`continue` passes control out of a :keyword:`try` statement with a +:keyword:`finally` clause, that :keyword:`finally` clause is executed before +really starting the next loop cycle. + .. _import: .. _from: @@ -832,4 +836,3 @@ pre-existing bindings in the local scope. .. [#] It may occur within an :keyword:`except` or :keyword:`else` clause. The restriction on occurring in the :keyword:`try` clause is implementor's laziness and will eventually be lifted. - diff --git a/Doc/tutorial/classes.rst b/Doc/tutorial/classes.rst index d9e2f460ad..b9db87b7e6 100644 --- a/Doc/tutorial/classes.rst +++ b/Doc/tutorial/classes.rst @@ -124,6 +124,8 @@ found outside of the innermost scope are read-only (an attempt to write to such a variable will simply create a *new* local variable in the innermost scope, leaving the identically named outer variable unchanged). +.. XXX mention nonlocal + Usually, the local scope references the local names of the (textually) current function. Outside functions, the local scope references the same namespace as the global scope: the module's namespace. Class definitions place yet another @@ -137,12 +139,15 @@ language definition is evolving towards static name resolution, at "compile" time, so don't rely on dynamic name resolution! (In fact, local variables are already determined statically.) -A special quirk of Python is that assignments normally go into the innermost -scope. Assignments do not copy data --- they just bind names to objects. The -same is true for deletions: the statement ``del x`` removes the binding of ``x`` -from the namespace referenced by the local scope. In fact, all operations that -introduce new names use the local scope: in particular, import statements and -function definitions bind the module or function name in the local scope. +A special quirk of Python is that -- if no :keyword:`global` or +:keyword:`nonlocal` statement is in effect -- assignments to names always go +into the innermost scope. Assignments do not copy data --- they just bind names +to objects. The same is true for deletions: the statement ``del x`` removes the +binding of ``x`` from the namespace referenced by the local scope. In fact, all +operations that introduce new names use the local scope: in particular, import +statements and function definitions bind the module or function name in the +local scope. (The :keyword:`global` statement can be used to indicate that +particular variables live in the global scope.) The :keyword:`global` statement can be used to indicate that particular variables live in the global scope and should be rebound there; the @@ -466,6 +471,9 @@ classes defined in it. Usually, the class containing the method is itself defined in this global scope, and in the next section we'll find some good reasons why a method would want to reference its own class! +Each value is an object, and therefore has a *class* (also called its *type*). +It is stored as ``object.__class__``. + .. _tut-inheritance: @@ -515,6 +523,19 @@ arguments)``. This is occasionally useful to clients as well. (Note that this only works if the base class is defined or imported directly in the global scope.) +Python has two builtin functions that work with inheritance: + +* Use :func:`isinstance` to check an object's type: ``isinstance(obj, int)`` + will be ``True`` only if ``obj.__class__`` is :class:`int` or some class + derived from :class:`int`. + +* Use :func:`issubclass` to check class inheritance: ``issubclass(bool, int)`` + is ``True`` since :class:`bool` is a subclass of :class:`int`. However, + ``issubclass(unicode, str)`` is ``False`` since :class:`unicode` is not a + subclass of :class:`str` (they only share a common ancestor, + :class:`basestring`). + + .. _tut-multiple: diff --git a/Doc/tutorial/whatnow.rst b/Doc/tutorial/whatnow.rst index 25d42a8604..b950cbc2c4 100644 --- a/Doc/tutorial/whatnow.rst +++ b/Doc/tutorial/whatnow.rst @@ -38,9 +38,9 @@ More Python resources: * http://docs.python.org: Fast access to Python's documentation. -* http://cheeseshop.python.org: The Python Package Index, nicknamed the Cheese - Shop, is an index of user-created Python modules that are available for - download. Once you begin releasing code, you can register it here so that +* http://pypi.python.org: The Python Package Index, previously also nicknamed + the Cheese Shop, is an index of user-created Python modules that are available + for download. Once you begin releasing code, you can register it here so that others can find it. * http://aspn.activestate.com/ASPN/Python/Cookbook/: The Python Cookbook is a diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst index 3dcfc7e292..a94c3e71c7 100644 --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -311,7 +311,8 @@ These environment variables influence Python's behavior. Augment the default search path for module files. The format is the same as the shell's :envvar:`PATH`: one or more directory pathnames separated by - colons. Non-existent directories are silently ignored. + :data:`os.pathsep` (e.g. colons on Unix or semicolons on Windows). + Non-existent directories are silently ignored. The default search path is installation dependent, but generally begins with :file:`{prefix}/lib/python{version}`` (see :envvar:`PYTHONHOME` above). It diff --git a/Doc/whatsnew/2.6.rst b/Doc/whatsnew/2.6.rst index bba46c990a..20f17c7fa3 100644 --- a/Doc/whatsnew/2.6.rst +++ b/Doc/whatsnew/2.6.rst @@ -117,8 +117,12 @@ LaTeX to reStructured Text. New Issue Tracker: Roundup -------------------------------------------------- -XXX write this. +XXX write this -- this section is currently just brief notes. +The developers were growing increasingly annoyed by SourceForge's +bug tracker. (Discuss problems in a sentence or two.) + +Hosting provided by XXX. New Documentation Format: ReStructured Text -------------------------------------------------- @@ -455,7 +459,46 @@ can now be used in scripts running from inside a package. PEP 3101: Advanced String Formatting ===================================================== -XXX write this +XXX write this -- this section is currently just brief notes. + +8-bit and Unicode strings have a .format() method that takes the arguments +to be formatted. + +.format() uses curly brackets ({, }) as special characters: + + format("User ID: {0}", "root") -> "User ID: root" + format("Empty dict: {{}}") -> "Empty dict: {}" + 0.name + 0[name] + +Format specifiers: + + 0:8 -> left-align, pad + 0:>8 -> right-align, pad + +Format data types:: + + ... take table from PEP 3101 + +Classes and types can define a __format__ method to control how it's +formatted. It receives a single argument, the format specifier:: + + def __format__(self, format_spec): + if isinstance(format_spec, unicode): + return unicode(str(self)) + else: + return str(self) + +There's also a format() built-in that will format a single value. It calls +the type's :meth:`__format__` method with the provided specifier:: + + >>> format(75.6564, '.2f') + '75.66' + +.. seealso:: + + :pep:`3101` - Advanced String Formatting + PEP written by Talin. .. ====================================================================== @@ -509,12 +552,30 @@ work. .. ====================================================================== +.. _pep-3112: + +PEP 3112: Byte Literals +===================================================== + +Python 3.0 adopts Unicode as the language's fundamental string type, and +denotes 8-bit literals differently, either as ``b'string'`` +or using a :class:`bytes` constructor. For future compatibility, +Python 2.6 adds :class:`bytes` as a synonym for the :class:`str` type, +and it also supports the ``b''`` notation. + +.. seealso:: + + :pep:`3112` - Bytes literals in Python 3000 + PEP written by Jason Orendorff; backported to 2.6 by Christian Heimes. + +.. ====================================================================== + .. _pep-3119: PEP 3119: Abstract Base Classes ===================================================== -XXX +XXX write this -- this section is currently just brief notes. How to identify a file object? @@ -558,16 +619,23 @@ an abstract method. PEP 3127: Integer Literal Support and Syntax ===================================================== -XXX write this +XXX write this -- this section is currently just brief notes. Python 3.0 changes the syntax for octal integer literals, and adds supports for binary integers: 0o instad of 0, and 0b for binary. Python 2.6 doesn't support this, but a bin() -builtin was added, and +builtin was added. + +XXX changes to the hex/oct builtins New bin() built-in returns the binary form of a number. +.. seealso:: + + :pep:`3127` - Integer Literal Support and Syntax + PEP written by Patrick Maupin. + .. ====================================================================== .. _pep-3129: @@ -575,7 +643,30 @@ New bin() built-in returns the binary form of a number. PEP 3129: Class Decorators ===================================================== -XXX write this. +XXX write this -- this section is currently just brief notes. + +Class decorators are analogous to function decorators. After defining a class, +it's passed through the specified series of decorator functions +and the ultimate return value is recorded as the class. + +:: + + class A: + pass + A = foo(bar(A)) + + + @foo + @bar + class A: + pass + +XXX need to find a good motivating example. + +.. seealso:: + + :pep:`3129` - Class Decorators + PEP written by Collin Winter. .. ====================================================================== @@ -631,11 +722,14 @@ one, :func:`math.trunc`, that's been backported to Python 2.6. .. seealso:: + :pep:`3141` - A Type Hierarchy for Numbers + PEP written by Jeffrey Yasskin. + XXX link: Discusses Scheme's numeric tower. -The Fraction Module +The :mod:`fractions` Module -------------------------------------------------- To fill out the hierarchy of numeric types, a rational-number class @@ -657,11 +751,27 @@ that will be the numerator and denominator of the resulting fraction. :: >>> a/b Fraction(5, 3) +To help in converting floating-point numbers to rationals, +the float type now has a :meth:`as_integer_ratio()` method that returns +the numerator and denominator for a fraction that evaluates to the same +floating-point value:: + + >>> (2.5) .as_integer_ratio() + (5, 2) + >>> (3.1415) .as_integer_ratio() + (7074029114692207L, 2251799813685248L) + >>> (1./3) .as_integer_ratio() + (6004799503160661L, 18014398509481984L) + +Note that values that can only be approximated by floating-point +numbers, such as 1./3, are not simplified to the number being +approximated; the fraction attempts to match the floating-point value +**exactly**. + The :mod:`fractions` module is based upon an implementation by Sjoerd Mullender that was in Python's :file:`Demo/classes/` directory for a long time. This implementation was significantly updated by Jeffrey -Yaskin. - +Yasskin. Other Language Changes ====================== @@ -740,7 +850,7 @@ Here are all of the changes that Python 2.6 makes to the core Python language. positive or negative infinity. This works on any platform with IEEE 754 semantics. (Contributed by Christian Heimes.) - .. Patch 1635. + .. Patch 1635 Other functions in the :mod:`math` module, :func:`isinf` and :func:`isnan`, return true if their floating-point argument is @@ -767,6 +877,12 @@ Here are all of the changes that Python 2.6 makes to the core Python language. .. Patch #1537 +* Generator objects now have a :attr:`gi_code` attribute that refers to + the original code object backing the generator. + (Contributed by Collin Winter.) + + .. Patch #1473257 + * The :func:`compile` built-in function now accepts keyword arguments as well as positional parameters. (Contributed by Thomas Wouters.) @@ -816,7 +932,7 @@ Optimizations (Original optimization implemented by Armin Rigo, updated for Python 2.6 by Kevin Jacobs.) - .. % Patch 1700288 + .. Patch 1700288 * All of the functions in the :mod:`struct` module have been rewritten in C, thanks to work at the Need For Speed sprint. @@ -1041,7 +1157,7 @@ complete list of changes, or look through the CVS logs for all the details. (2, 3, 1, 3), (2, 3, 1, 4), (2, 3, 2, 3), (2, 3, 2, 4), (2, 4, 1, 3), (2, 4, 1, 4), (2, 4, 2, 3), (2, 4, 2, 4)] - ``combinations(iter, r)`` returns combinations of length *r* from + ``combinations(iter, r)`` returns sub-sequences of length *r* from the elements of *iterable*. :: itertools.combinations('123', 2) -> @@ -1054,8 +1170,18 @@ complete list of changes, or look through the CVS logs for all the details. [('1', '2', '3'), ('1', '2', '4'), ('1', '3', '4'), ('2', '3', '4')] + ``permutations(iter[, r])`` returns all the permutations of length *r* of + the iterable's elements. If *r* is not specified, it will default to the + number of elements produced by the iterable. + + itertools.permutations([1,2,3,4], 2) -> + [(1, 2), (1, 3), (1, 4), + (2, 1), (2, 3), (2, 4), + (3, 1), (3, 2), (3, 4), + (4, 1), (4, 2), (4, 3)] + ``itertools.chain(*iterables)` is an existing function in - :mod:`itertools` that gained a new constructor. + :mod:`itertools` that gained a new constructor in Python 2.6. ``itertools.chain.from_iterable(iterable)`` takes a single iterable that should return other iterables. :func:`chain` will then return all the elements of the first iterable, then @@ -1066,6 +1192,13 @@ complete list of changes, or look through the CVS logs for all the details. (All contributed by Raymond Hettinger.) +* The :mod:`logging` module's :class:`FileHandler` class + and its subclasses :class:`WatchedFileHandler`, :class:`RotatingFileHandler`, + and :class:`TimedRotatingFileHandler` now + have an optional *delay* parameter to its constructor. If *delay* + is true, opening of the log file is deferred until the first + :meth:`emit` call is made. (Contributed by Vinay Sajip.) + * The :mod:`macfs` module has been removed. This in turn required the :func:`macostools.touched` function to be removed because it depended on the :mod:`macfs` module. @@ -1171,6 +1304,13 @@ complete list of changes, or look through the CVS logs for all the details. changed and :const:`UF_APPEND` to indicate that data can only be appended to the file. (Contributed by M. Levinson.) + ``os.closerange(*low*, *high*)`` efficiently closes all file descriptors + from *low* to *high*, ignoring any errors and not including *high* itself. + This function is now used by the :mod:`subprocess` module to make starting + processes faster. (Contributed by Georg Brandl.) + + .. Patch #1663329 + * The :mod:`pyexpat` module's :class:`Parser` objects now allow setting their :attr:`buffer_size` attribute to change the size of the buffer used to hold character data. @@ -1199,10 +1339,18 @@ complete list of changes, or look through the CVS logs for all the details. long searches can now be interrupted. (Contributed by Josh Hoyt and Ralf Schmitt.) - .. % Patch 846388 + .. Patch 846388 * The :mod:`rgbimg` module has been removed. +* The :mod:`sched` module's :class:`scheduler` instances now + have a read-only :attr:`queue` attribute that returns the + contents of the scheduler's queue, represented as a list of + named tuples with the fields ``(time, priority, action, argument)``. + (Contributed by Raymond Hettinger XXX check.) + + .. Patch 1861 + * The :mod:`sets` module has been deprecated; it's better to use the built-in :class:`set` and :class:`frozenset` types. @@ -1223,9 +1371,9 @@ complete list of changes, or look through the CVS logs for all the details. On receiving a signal, a byte will be written and the main event loop will be woken up, without the need to poll. - Contributed by Adam Olsen. + (Contributed by Adam Olsen.) - .. % Patch 1583 + .. Patch 1583 The :func:`siginterrupt` function is now available from Python code, and allows changing whether signals can interrupt system calls or not. @@ -1250,7 +1398,7 @@ complete list of changes, or look through the CVS logs for all the details. * In the :mod:`smtplib` module, SMTP.starttls() now complies with :rfc:`3207` and forgets any knowledge obtained from the server not obtained from - the TLS negotiation itself. Patch contributed by Bill Fenner. + the TLS negotiation itself. (Patch contributed by Bill Fenner.) .. Issue 829951 @@ -1267,6 +1415,10 @@ complete list of changes, or look through the CVS logs for all the details. by Michael Pomraning.) .. Patch #742598 + +* The :mod:`struct` module now supports the C99 :ctype:`_Bool` type, + using the format character ``'?'``. + (Contributed by David Remahl.) * A new variable in the :mod:`sys` module, :attr:`float_info`, is an object @@ -1297,6 +1449,12 @@ complete list of changes, or look through the CVS logs for all the details. These attributes are all read-only. (Contributed by Christian Heimes.) + It's now possible to determine the current profiler and tracer functions + by calling :func:`sys.getprofile` and :func:`sys.gettrace`. + (Contributed by Georg Brandl.) + + .. Patch #1648 + * The :mod:`tarfile` module now supports POSIX.1-2001 (pax) and POSIX.1-1988 (ustar) format tarfiles, in addition to the GNU tar format that was already supported. The default format @@ -1547,11 +1705,13 @@ Changes to Python's build process and to the C API include: .. Issue 1635 -* Some macros were renamed to make it clearer that they are macros, +* Some macros were renamed in both 3.0 and 2.6 to make it clearer that + they are macros, not functions. :cmacro:`Py_Size()` became :cmacro:`Py_SIZE()`, :cmacro:`Py_Type()` became :cmacro:`Py_TYPE()`, and - :cmacro:`Py_Refcnt()` became :cmacro:`Py_REFCNT()`. Macros for backward - compatibility are still available for Python 2.6. + :cmacro:`Py_Refcnt()` became :cmacro:`Py_REFCNT()`. + The mixed-case macros are still available + in Python 2.6 for backward compatibility. .. Issue 1629 diff --git a/Lib/SocketServer.py b/Lib/SocketServer.py index 47d87285f6..2a9ec8aced 100644 --- a/Lib/SocketServer.py +++ b/Lib/SocketServer.py @@ -130,8 +130,13 @@ __version__ = "0.4" import socket +import select import sys import os +try: + import threading +except ImportError: + import dummy_threading as threading __all__ = ["TCPServer","UDPServer","ForkingUDPServer","ForkingTCPServer", "ThreadingUDPServer","ThreadingTCPServer","BaseRequestHandler", @@ -149,7 +154,8 @@ class BaseServer: Methods for the caller: - __init__(server_address, RequestHandlerClass) - - serve_forever() + - serve_forever(poll_interval=0.5) + - shutdown() - handle_request() # if you do not use serve_forever() - fileno() -> int # for select() @@ -190,6 +196,8 @@ class BaseServer: """Constructor. May be extended, do not override.""" self.server_address = server_address self.RequestHandlerClass = RequestHandlerClass + self.__is_shut_down = threading.Event() + self.__serving = False def server_activate(self): """Called by constructor to activate the server. @@ -199,27 +207,73 @@ class BaseServer: """ pass - def serve_forever(self): - """Handle one request at a time until doomsday.""" - while 1: - self.handle_request() + def serve_forever(self, poll_interval=0.5): + """Handle one request at a time until shutdown. + + Polls for shutdown every poll_interval seconds. Ignores + self.timeout. If you need to do periodic tasks, do them in + another thread. + """ + self.__serving = True + self.__is_shut_down.clear() + while self.__serving: + # XXX: Consider using another file descriptor or + # connecting to the socket to wake this up instead of + # polling. Polling reduces our responsiveness to a + # shutdown request and wastes cpu at all other times. + r, w, e = select.select([self], [], [], poll_interval) + if r: + self._handle_request_noblock() + self.__is_shut_down.set() + + def shutdown(self): + """Stops the serve_forever loop. + + Blocks until the loop has finished. This must be called while + serve_forever() is running in another thread, or it will + deadlock. + """ + self.__serving = False + self.__is_shut_down.wait() # The distinction between handling, getting, processing and # finishing a request is fairly arbitrary. Remember: # # - handle_request() is the top-level call. It calls - # await_request(), verify_request() and process_request() - # - get_request(), called by await_request(), is different for - # stream or datagram sockets + # select, get_request(), verify_request() and process_request() + # - get_request() is different for stream or datagram sockets # - process_request() is the place that may fork a new process # or create a new thread to finish the request # - finish_request() instantiates the request handler class; # this constructor will handle the request all by itself def handle_request(self): - """Handle one request, possibly blocking.""" + """Handle one request, possibly blocking. + + Respects self.timeout. + """ + # Support people who used socket.settimeout() to escape + # handle_request before self.timeout was available. + timeout = self.socket.gettimeout() + if timeout is None: + timeout = self.timeout + elif self.timeout is not None: + timeout = min(timeout, self.timeout) + fd_sets = select.select([self], [], [], timeout) + if not fd_sets[0]: + self.handle_timeout() + return + self._handle_request_noblock() + + def _handle_request_noblock(self): + """Handle one request, without blocking. + + I assume that select.select has returned that the socket is + readable before this function was called, so there should be + no risk of blocking in get_request(). + """ try: - request, client_address = self.await_request() + request, client_address = self.get_request() except socket.error: return if self.verify_request(request, client_address): @@ -229,21 +283,6 @@ class BaseServer: self.handle_error(request, client_address) self.close_request(request) - def await_request(self): - """Call get_request or handle_timeout, observing self.timeout. - - Returns value from get_request() or raises socket.timeout exception if - timeout was exceeded. - """ - if self.timeout is not None: - # If timeout == 0, you're responsible for your own fd magic. - import select - fd_sets = select.select([self], [], [], self.timeout) - if not fd_sets[0]: - self.handle_timeout() - raise socket.timeout("Listening timed out") - return self.get_request() - def handle_timeout(self): """Called if no new request arrives within self.timeout. @@ -307,7 +346,8 @@ class TCPServer(BaseServer): Methods for the caller: - __init__(server_address, RequestHandlerClass, bind_and_activate=True) - - serve_forever() + - serve_forever(poll_interval=0.5) + - shutdown() - handle_request() # if you don't use serve_forever() - fileno() -> int # for select() @@ -523,7 +563,6 @@ class ThreadingMixIn: def process_request(self, request, client_address): """Start a new thread to process the request.""" - import threading t = threading.Thread(target = self.process_request_thread, args = (request, client_address)) if self.daemon_threads: diff --git a/Lib/_strptime.py b/Lib/_strptime.py index 28ccac77b9..b323e8fc4c 100644 --- a/Lib/_strptime.py +++ b/Lib/_strptime.py @@ -22,7 +22,7 @@ try: except: from dummy_thread import allocate_lock as _thread_allocate_lock -__all__ = ['strptime'] +__all__ = [] def _getlang(): # Figure out what the current language is set to. @@ -190,6 +190,7 @@ class TimeRE(dict): base.__init__({ # The " \d" part of the regex is to make %c from ANSI C work 'd': r"(?P<d>3[0-1]|[1-2]\d|0[1-9]|[1-9]| [1-9])", + 'f': r"(?P<f>[0-9]{1,6})", 'H': r"(?P<H>2[0-3]|[0-1]\d|\d)", 'I': r"(?P<I>1[0-2]|0[1-9]|[1-9])", 'j': r"(?P<j>36[0-6]|3[0-5]\d|[1-2]\d\d|0[1-9]\d|00[1-9]|[1-9]\d|0[1-9]|[1-9])", @@ -291,7 +292,7 @@ def _calc_julian_from_U_or_W(year, week_of_year, day_of_week, week_starts_Mon): return 1 + days_to_week + day_of_week -def strptime(data_string, format="%a %b %d %H:%M:%S %Y"): +def _strptime(data_string, format="%a %b %d %H:%M:%S %Y"): """Return a time struct based on the input string and the format string.""" global _TimeRE_cache, _regex_cache with _cache_lock: @@ -327,7 +328,7 @@ def strptime(data_string, format="%a %b %d %H:%M:%S %Y"): data_string[found.end():]) year = 1900 month = day = 1 - hour = minute = second = 0 + hour = minute = second = fraction = 0 tz = -1 # Default to -1 to signify that values not known; not critical to have, # though @@ -384,6 +385,11 @@ def strptime(data_string, format="%a %b %d %H:%M:%S %Y"): minute = int(found_dict['M']) elif group_key == 'S': second = int(found_dict['S']) + elif group_key == 'f': + s = found_dict['f'] + # Pad to always return microseconds. + s += "0" * (6 - len(s)) + fraction = int(s) elif group_key == 'A': weekday = locale_time.f_weekday.index(found_dict['A'].lower()) elif group_key == 'a': @@ -440,6 +446,9 @@ def strptime(data_string, format="%a %b %d %H:%M:%S %Y"): day = datetime_result.day if weekday == -1: weekday = datetime_date(year, month, day).weekday() - return time.struct_time((year, month, day, - hour, minute, second, - weekday, julian, tz)) + return (time.struct_time((year, month, day, + hour, minute, second, + weekday, julian, tz)), fraction) + +def _strptime_time(data_string, format="%a %b %d %H:%M:%S %Y"): + return _strptime(data_string, format)[0] diff --git a/Lib/bsddb/test/test_dbshelve.py b/Lib/bsddb/test/test_dbshelve.py index 99d0c49716..a597154f58 100644 --- a/Lib/bsddb/test/test_dbshelve.py +++ b/Lib/bsddb/test/test_dbshelve.py @@ -50,10 +50,7 @@ class DBShelveTestCase(unittest.TestCase): def tearDown(self): self.do_close() - try: - os.remove(self.filename) - except os.error: - pass + test_support.unlink(self.filename) def mk(self, key): """Turn key into an appropriate key type for this db""" @@ -284,8 +281,8 @@ class BasicEnvShelveTestCase(DBShelveTestCase): def tearDown(self): - test_support.rmtree(self.homeDir) self.do_close() + test_support.rmtree(self.homeDir) class EnvBTreeShelveTestCase(BasicEnvShelveTestCase): diff --git a/Lib/bsddb/test/test_thread.py b/Lib/bsddb/test/test_thread.py index 359c10d50b..51ed0f52fb 100644 --- a/Lib/bsddb/test/test_thread.py +++ b/Lib/bsddb/test/test_thread.py @@ -68,9 +68,9 @@ class BaseThreadedTestCase(unittest.TestCase): self.d.open(self.filename, self.dbtype, self.dbopenflags|db.DB_CREATE) def tearDown(self): - test_support.rmtree(self.homeDir) self.d.close() self.env.close() + test_support.rmtree(self.homeDir) def setEnvOpts(self): pass diff --git a/Lib/ctypes/__init__.py b/Lib/ctypes/__init__.py index 8dd8e0a028..b6e598ffed 100644 --- a/Lib/ctypes/__init__.py +++ b/Lib/ctypes/__init__.py @@ -237,7 +237,7 @@ c_voidp = c_void_p # backwards compatibility (to a bug) _check_size(c_void_p) class c_bool(_SimpleCData): - _type_ = "t" + _type_ = "?" # This cache maps types to pointers to them. _pointer_type_cache = {} diff --git a/Lib/distutils/command/sdist.py b/Lib/distutils/command/sdist.py index 8289925785..b29fc64eca 100644 --- a/Lib/distutils/command/sdist.py +++ b/Lib/distutils/command/sdist.py @@ -325,14 +325,14 @@ class sdist (Command): * the build tree (typically "build") * the release tree itself (only an issue if we ran "sdist" previously with --keep-temp, or it aborted) - * any RCS, CVS and .svn directories + * any RCS, CVS, .svn, .hg, .git, .bzr, _darcs directories """ build = self.get_finalized_command('build') base_dir = self.distribution.get_fullname() self.filelist.exclude_pattern(None, prefix=build.build_base) self.filelist.exclude_pattern(None, prefix=base_dir) - self.filelist.exclude_pattern(r'/(RCS|CVS|\.svn)/.*', is_regex=1) + self.filelist.exclude_pattern(r'(^|/)(RCS|CVS|\.svn|\.hg|\.git|\.bzr|_darcs)/.*', is_regex=1) def write_manifest(self): """Write the file list in 'self.filelist' (presumably as filled in diff --git a/Lib/heapq.py b/Lib/heapq.py index 47d246ec56..71a12e7bf4 100644 --- a/Lib/heapq.py +++ b/Lib/heapq.py @@ -127,7 +127,7 @@ From all times, sorting has always been a Great Art! :-) """ __all__ = ['heappush', 'heappop', 'heapify', 'heapreplace', 'merge', - 'nlargest', 'nsmallest'] + 'nlargest', 'nsmallest', 'heappushpop'] from itertools import islice, repeat, count, tee from operator import itemgetter, neg @@ -165,6 +165,13 @@ def heapreplace(heap, item): _siftup(heap, 0) return returnitem +def heappushpop(heap, item): + """Fast version of a heappush followed by a heappop.""" + if heap and item > heap[0]: + item, heap[0] = heap[0], item + _siftup(heap, 0) + return item + def heapify(x): """Transform list into a heap, in-place, in O(len(heap)) time.""" n = len(x) @@ -186,13 +193,9 @@ def nlargest(n, iterable): if not result: return result heapify(result) - _heapreplace = heapreplace - sol = result[0] # sol --> smallest of the nlargest + _heappushpop = heappushpop for elem in it: - if elem <= sol: - continue - _heapreplace(result, elem) - sol = result[0] + heappushpop(result, elem) result.sort(reverse=True) return result @@ -304,7 +307,7 @@ def _siftup(heap, pos): # If available, use C implementation try: - from _heapq import heappush, heappop, heapify, heapreplace, nlargest, nsmallest + from _heapq import heappush, heappop, heapify, heapreplace, nlargest, nsmallest, heappushpop except ImportError: pass diff --git a/Lib/pdb.py b/Lib/pdb.py index 57a2b45bae..29746d7aab 100755 --- a/Lib/pdb.py +++ b/Lib/pdb.py @@ -1193,7 +1193,16 @@ def set_trace(): # Post-Mortem interface -def post_mortem(t): +def post_mortem(t=None): + # handling the default + if t is None: + # sys.exc_info() returns (type, value, traceback) if an exception is + # being handled, otherwise it returns None + t = sys.exc_info()[2] + if t is None: + raise ValueError("A valid traceback must be passed if no " + "exception is being handled") + p = Pdb() p.reset() while t.tb_next is not None: diff --git a/Lib/platform.py b/Lib/platform.py index 1fc10329ac..c2f34b573a 100755 --- a/Lib/platform.py +++ b/Lib/platform.py @@ -238,9 +238,10 @@ _release_version = re.compile(r'([^0-9]+)' # and http://data.linux-ntfs.org/rpm/whichrpm # and http://www.die.net/doc/linux/man/man1/lsb_release.1.html -_supported_dists = ('SuSE', 'debian', 'fedora', 'redhat', 'centos', - 'mandrake', 'rocks', 'slackware', 'yellowdog', - 'gentoo', 'UnitedLinux', 'turbolinux') +_supported_dists = ( + 'SuSE', 'debian', 'fedora', 'redhat', 'centos', + 'mandrake', 'mandriva', 'rocks', 'slackware', 'yellowdog', 'gentoo', + 'UnitedLinux', 'turbolinux') def _parse_release_file(firstline): diff --git a/Lib/py_compile.py b/Lib/py_compile.py index 982d19d142..8ef3662707 100644 --- a/Lib/py_compile.py +++ b/Lib/py_compile.py @@ -171,11 +171,15 @@ def main(args=None): """ if args is None: args = sys.argv[1:] + rv = 0 for filename in args: try: compile(filename, doraise=True) except PyCompileError as err: + # return value to indicate at least one failure + rv = 1 sys.stderr.write(err.msg) + return rv if __name__ == "__main__": - main() + sys.exit(main()) diff --git a/Lib/rlcompleter.py b/Lib/rlcompleter.py index 74b2e47a81..c605c7d254 100644 --- a/Lib/rlcompleter.py +++ b/Lib/rlcompleter.py @@ -119,7 +119,7 @@ class Completer: import re m = re.match(r"(\w+(\.\w+)*)\.(\w*)", text) if not m: - return + return [] expr, attr = m.group(1, 3) object = eval(expr, self.namespace) words = dir(object) diff --git a/Lib/sqlite3/test/transactions.py b/Lib/sqlite3/test/transactions.py index 1f0b19aa9b..9edc4ac3ba 100644 --- a/Lib/sqlite3/test/transactions.py +++ b/Lib/sqlite3/test/transactions.py @@ -31,7 +31,7 @@ class TransactionTests(unittest.TestCase): def setUp(self): try: os.remove(get_db_path()) - except: + except OSError: pass self.con1 = sqlite.connect(get_db_path(), timeout=0.1) @@ -47,7 +47,10 @@ class TransactionTests(unittest.TestCase): self.cur2.close() self.con2.close() - os.unlink(get_db_path()) + try: + os.unlink(get_db_path()) + except OSError: + pass def CheckDMLdoesAutoCommitBefore(self): self.cur1.execute("create table test(i)") diff --git a/Lib/test/output/test_thread b/Lib/test/output/test_thread deleted file mode 100644 index 68c6a92dab..0000000000 --- a/Lib/test/output/test_thread +++ /dev/null @@ -1,18 +0,0 @@ -test_thread -waiting for all tasks to complete -all tasks done - -*** Barrier Test *** -all tasks done - -*** Changing thread stack size *** -caught expected ValueError setting stack_size(4096) -successfully set stack_size(262144) -successfully set stack_size(1048576) -successfully set stack_size(0) -trying stack_size = 262144 -waiting for all tasks to complete -all tasks done -trying stack_size = 1048576 -waiting for all tasks to complete -all tasks done diff --git a/Lib/test/output/test_tokenize b/Lib/test/output/test_tokenize deleted file mode 100644 index 670809717d..0000000000 --- a/Lib/test/output/test_tokenize +++ /dev/null @@ -1,715 +0,0 @@ -test_tokenize -1,0-1,34: COMMENT "# Tests for the 'tokenize' module." -1,34-1,35: NL '\n' -2,0-2,42: COMMENT '# Large bits stolen from test_grammar.py. ' -2,42-2,43: NL '\n' -3,0-3,1: NL '\n' -4,0-4,10: COMMENT '# Comments' -4,10-4,11: NL '\n' -5,0-5,3: STRING '"#"' -5,3-5,4: NEWLINE '\n' -6,0-6,2: COMMENT "#'" -6,2-6,3: NL '\n' -7,0-7,2: COMMENT '#"' -7,2-7,3: NL '\n' -8,0-8,2: COMMENT '#\\' -8,2-8,3: NL '\n' -9,7-9,8: COMMENT '#' -9,8-9,9: NL '\n' -10,4-10,9: COMMENT '# abc' -10,9-10,10: NL '\n' -11,0-12,4: STRING "'''#\n#'''" -12,4-12,5: NEWLINE '\n' -13,0-13,1: NL '\n' -14,0-14,1: NAME 'x' -14,2-14,3: OP '=' -14,4-14,5: NUMBER '1' -14,7-14,8: COMMENT '#' -14,8-14,9: NEWLINE '\n' -15,0-15,1: NL '\n' -16,0-16,24: COMMENT '# Balancing continuation' -16,24-16,25: NL '\n' -17,0-17,1: NL '\n' -18,0-18,1: NAME 'a' -18,2-18,3: OP '=' -18,4-18,5: OP '(' -18,5-18,6: NUMBER '3' -18,6-18,7: OP ',' -18,8-18,9: NUMBER '4' -18,9-18,10: OP ',' -18,10-18,11: NL '\n' -19,2-19,3: NUMBER '5' -19,3-19,4: OP ',' -19,5-19,6: NUMBER '6' -19,6-19,7: OP ')' -19,7-19,8: NEWLINE '\n' -20,0-20,1: NAME 'y' -20,2-20,3: OP '=' -20,4-20,5: OP '[' -20,5-20,6: NUMBER '3' -20,6-20,7: OP ',' -20,8-20,9: NUMBER '4' -20,9-20,10: OP ',' -20,10-20,11: NL '\n' -21,2-21,3: NUMBER '5' -21,3-21,4: OP ']' -21,4-21,5: NEWLINE '\n' -22,0-22,1: NAME 'z' -22,2-22,3: OP '=' -22,4-22,5: OP '{' -22,5-22,8: STRING "'a'" -22,8-22,9: OP ':' -22,9-22,10: NUMBER '5' -22,10-22,11: OP ',' -22,11-22,12: NL '\n' -23,2-23,5: STRING "'b'" -23,5-23,6: OP ':' -23,6-23,7: NUMBER '6' -23,7-23,8: OP '}' -23,8-23,9: NEWLINE '\n' -24,0-24,1: NAME 'x' -24,2-24,3: OP '=' -24,4-24,5: OP '(' -24,5-24,8: NAME 'len' -24,8-24,9: OP '(' -24,9-24,13: NAME 'repr' -24,13-24,14: OP '(' -24,14-24,15: NAME 'y' -24,15-24,16: OP ')' -24,16-24,17: OP ')' -24,18-24,19: OP '+' -24,20-24,21: NUMBER '5' -24,21-24,22: OP '*' -24,22-24,23: NAME 'x' -24,24-24,25: OP '-' -24,26-24,27: NAME 'a' -24,27-24,28: OP '[' -24,28-24,29: NL '\n' -25,3-25,4: NUMBER '3' -25,5-25,6: OP ']' -25,6-25,7: NL '\n' -26,3-26,4: OP '-' -26,5-26,6: NAME 'x' -26,7-26,8: OP '+' -26,9-26,12: NAME 'len' -26,12-26,13: OP '(' -26,13-26,14: OP '{' -26,14-26,15: NL '\n' -27,3-27,4: OP '}' -27,4-27,5: NL '\n' -28,4-28,5: OP ')' -28,5-28,6: NL '\n' -29,2-29,3: OP ')' -29,3-29,4: NEWLINE '\n' -30,0-30,1: NL '\n' -31,0-31,36: COMMENT '# Backslash means line continuation:' -31,36-31,37: NL '\n' -32,0-32,1: NAME 'x' -32,2-32,3: OP '=' -32,4-32,5: NUMBER '1' -33,0-33,1: OP '+' -33,2-33,3: NUMBER '1' -33,3-33,4: NEWLINE '\n' -34,0-34,1: NL '\n' -35,0-35,54: COMMENT '# Backslash does not means continuation in comments :\\' -35,54-35,55: NL '\n' -36,0-36,1: NAME 'x' -36,2-36,3: OP '=' -36,4-36,5: NUMBER '0' -36,5-36,6: NEWLINE '\n' -37,0-37,1: NL '\n' -38,0-38,19: COMMENT '# Ordinary integers' -38,19-38,20: NL '\n' -39,0-39,4: NUMBER '0xff' -39,5-39,7: OP '!=' -39,8-39,11: NUMBER '255' -39,11-39,12: NEWLINE '\n' -40,0-40,5: NUMBER '0o377' -40,6-40,8: OP '!=' -40,9-40,12: NUMBER '255' -40,12-40,13: NEWLINE '\n' -41,0-41,10: NUMBER '2147483647' -41,13-41,15: OP '!=' -41,16-41,29: NUMBER '0o17777777777' -41,29-41,30: NEWLINE '\n' -42,0-42,1: OP '-' -42,1-42,11: NUMBER '2147483647' -42,11-42,12: OP '-' -42,12-42,13: NUMBER '1' -42,14-42,16: OP '!=' -42,17-42,30: NUMBER '0o20000000000' -42,30-42,31: NEWLINE '\n' -43,0-43,13: NUMBER '0o37777777777' -43,14-43,16: OP '!=' -43,17-43,18: OP '-' -43,18-43,19: NUMBER '1' -43,19-43,20: NEWLINE '\n' -44,0-44,10: NUMBER '0xffffffff' -44,11-44,13: OP '!=' -44,14-44,15: OP '-' -44,15-44,16: NUMBER '1' -44,16-44,17: OP ';' -44,18-44,31: NUMBER '0o37777777777' -44,32-44,34: OP '!=' -44,35-44,36: OP '-' -44,36-44,37: NUMBER '1' -44,37-44,38: OP ';' -44,39-44,40: OP '-' -44,40-44,49: NUMBER '0o1234567' -44,50-44,52: OP '==' -44,53-44,64: NUMBER '0O001234567' -44,64-44,65: OP ';' -44,66-44,73: NUMBER '0b10101' -44,74-44,76: OP '==' -44,77-44,87: NUMBER '0B00010101' -44,87-44,88: NEWLINE '\n' -45,0-45,1: NL '\n' -46,0-46,15: COMMENT '# Long integers' -46,15-46,16: NL '\n' -47,0-47,1: NAME 'x' -47,2-47,3: OP '=' -47,4-47,5: NUMBER '0' -47,5-47,6: NEWLINE '\n' -48,0-48,1: NAME 'x' -48,2-48,3: OP '=' -48,4-48,5: NUMBER '0' -48,5-48,6: NEWLINE '\n' -49,0-49,1: NAME 'x' -49,2-49,3: OP '=' -49,4-49,22: NUMBER '0xffffffffffffffff' -49,22-49,23: NEWLINE '\n' -50,0-50,1: NAME 'x' -50,2-50,3: OP '=' -50,4-50,22: NUMBER '0xffffffffffffffff' -50,22-50,23: NEWLINE '\n' -51,0-51,1: NAME 'x' -51,2-51,3: OP '=' -51,4-51,23: NUMBER '0o77777777777777777' -51,23-51,24: NEWLINE '\n' -52,0-52,1: NAME 'x' -52,2-52,3: OP '=' -52,4-52,23: NUMBER '0B11101010111111111' -52,23-52,24: NEWLINE '\n' -53,0-53,1: NAME 'x' -53,2-53,3: OP '=' -53,4-53,34: NUMBER '123456789012345678901234567890' -53,34-53,35: NEWLINE '\n' -54,0-54,1: NAME 'x' -54,2-54,3: OP '=' -54,4-54,34: NUMBER '123456789012345678901234567890' -54,34-54,35: NEWLINE '\n' -55,0-55,1: NL '\n' -56,0-56,24: COMMENT '# Floating-point numbers' -56,24-56,25: NL '\n' -57,0-57,1: NAME 'x' -57,2-57,3: OP '=' -57,4-57,8: NUMBER '3.14' -57,8-57,9: NEWLINE '\n' -58,0-58,1: NAME 'x' -58,2-58,3: OP '=' -58,4-58,8: NUMBER '314.' -58,8-58,9: NEWLINE '\n' -59,0-59,1: NAME 'x' -59,2-59,3: OP '=' -59,4-59,9: NUMBER '0.314' -59,9-59,10: NEWLINE '\n' -60,0-60,17: COMMENT '# XXX x = 000.314' -60,17-60,18: NL '\n' -61,0-61,1: NAME 'x' -61,2-61,3: OP '=' -61,4-61,8: NUMBER '.314' -61,8-61,9: NEWLINE '\n' -62,0-62,1: NAME 'x' -62,2-62,3: OP '=' -62,4-62,8: NUMBER '3e14' -62,8-62,9: NEWLINE '\n' -63,0-63,1: NAME 'x' -63,2-63,3: OP '=' -63,4-63,8: NUMBER '3E14' -63,8-63,9: NEWLINE '\n' -64,0-64,1: NAME 'x' -64,2-64,3: OP '=' -64,4-64,9: NUMBER '3e-14' -64,9-64,10: NEWLINE '\n' -65,0-65,1: NAME 'x' -65,2-65,3: OP '=' -65,4-65,9: NUMBER '3e+14' -65,9-65,10: NEWLINE '\n' -66,0-66,1: NAME 'x' -66,2-66,3: OP '=' -66,4-66,9: NUMBER '3.e14' -66,9-66,10: NEWLINE '\n' -67,0-67,1: NAME 'x' -67,2-67,3: OP '=' -67,4-67,9: NUMBER '.3e14' -67,9-67,10: NEWLINE '\n' -68,0-68,1: NAME 'x' -68,2-68,3: OP '=' -68,4-68,9: NUMBER '3.1e4' -68,9-68,10: NEWLINE '\n' -69,0-69,1: NL '\n' -70,0-70,17: COMMENT '# String literals' -70,17-70,18: NL '\n' -71,0-71,1: NAME 'x' -71,2-71,3: OP '=' -71,4-71,6: STRING "''" -71,6-71,7: OP ';' -71,8-71,9: NAME 'y' -71,10-71,11: OP '=' -71,12-71,14: STRING '""' -71,14-71,15: OP ';' -71,15-71,16: NEWLINE '\n' -72,0-72,1: NAME 'x' -72,2-72,3: OP '=' -72,4-72,8: STRING "'\\''" -72,8-72,9: OP ';' -72,10-72,11: NAME 'y' -72,12-72,13: OP '=' -72,14-72,17: STRING '"\'"' -72,17-72,18: OP ';' -72,18-72,19: NEWLINE '\n' -73,0-73,1: NAME 'x' -73,2-73,3: OP '=' -73,4-73,7: STRING '\'"\'' -73,7-73,8: OP ';' -73,9-73,10: NAME 'y' -73,11-73,12: OP '=' -73,13-73,17: STRING '"\\""' -73,17-73,18: OP ';' -73,18-73,19: NEWLINE '\n' -74,0-74,1: NAME 'x' -74,2-74,3: OP '=' -74,4-74,32: STRING '"doesn\'t \\"shrink\\" does it"' -74,32-74,33: NEWLINE '\n' -75,0-75,1: NAME 'y' -75,2-75,3: OP '=' -75,4-75,31: STRING '\'doesn\\\'t "shrink" does it\'' -75,31-75,32: NEWLINE '\n' -76,0-76,1: NAME 'x' -76,2-76,3: OP '=' -76,4-76,32: STRING '"does \\"shrink\\" doesn\'t it"' -76,32-76,33: NEWLINE '\n' -77,0-77,1: NAME 'y' -77,2-77,3: OP '=' -77,4-77,31: STRING '\'does "shrink" doesn\\\'t it\'' -77,31-77,32: NEWLINE '\n' -78,0-78,1: NAME 'x' -78,2-78,3: OP '=' -78,4-83,3: STRING '"""\nThe "quick"\nbrown fox\njumps over\nthe \'lazy\' dog.\n"""' -83,3-83,4: NEWLINE '\n' -84,0-84,1: NAME 'y' -84,2-84,3: OP '=' -84,4-84,63: STRING '\'\\nThe "quick"\\nbrown fox\\njumps over\\nthe \\\'lazy\\\' dog.\\n\'' -84,63-84,64: NEWLINE '\n' -85,0-85,1: NAME 'y' -85,2-85,3: OP '=' -85,4-90,3: STRING '\'\'\'\nThe "quick"\nbrown fox\njumps over\nthe \'lazy\' dog.\n\'\'\'' -90,3-90,4: OP ';' -90,4-90,5: NEWLINE '\n' -91,0-91,1: NAME 'y' -91,2-91,3: OP '=' -91,4-96,1: STRING '"\\n\\\nThe \\"quick\\"\\n\\\nbrown fox\\n\\\njumps over\\n\\\nthe \'lazy\' dog.\\n\\\n"' -96,1-96,2: OP ';' -96,2-96,3: NEWLINE '\n' -97,0-97,1: NAME 'y' -97,2-97,3: OP '=' -97,4-102,1: STRING '\'\\n\\\nThe \\"quick\\"\\n\\\nbrown fox\\n\\\njumps over\\n\\\nthe \\\'lazy\\\' dog.\\n\\\n\'' -102,1-102,2: OP ';' -102,2-102,3: NEWLINE '\n' -103,0-103,1: NAME 'x' -103,2-103,3: OP '=' -103,4-103,9: STRING "r'\\\\'" -103,10-103,11: OP '+' -103,12-103,17: STRING "R'\\\\'" -103,17-103,18: NEWLINE '\n' -104,0-104,1: NAME 'x' -104,2-104,3: OP '=' -104,4-104,9: STRING "r'\\''" -104,10-104,11: OP '+' -104,12-104,14: STRING "''" -104,14-104,15: NEWLINE '\n' -105,0-105,1: NAME 'y' -105,2-105,3: OP '=' -105,4-107,6: STRING "r'''\nfoo bar \\\\\nbaz'''" -107,7-107,8: OP '+' -107,9-108,6: STRING "R'''\nfoo'''" -108,6-108,7: NEWLINE '\n' -109,0-109,1: NAME 'y' -109,2-109,3: OP '=' -109,4-111,3: STRING 'r"""foo\nbar \\\\ baz\n"""' -111,4-111,5: OP '+' -111,6-112,3: STRING "R'''spam\n'''" -112,3-112,4: NEWLINE '\n' -113,0-113,1: NAME 'x' -113,2-113,3: OP '=' -113,4-113,10: STRING "b'abc'" -113,11-113,12: OP '+' -113,13-113,19: STRING "B'ABC'" -113,19-113,20: NEWLINE '\n' -114,0-114,1: NAME 'y' -114,2-114,3: OP '=' -114,4-114,10: STRING 'b"abc"' -114,11-114,12: OP '+' -114,13-114,19: STRING 'B"ABC"' -114,19-114,20: NEWLINE '\n' -115,0-115,1: NAME 'x' -115,2-115,3: OP '=' -115,4-115,11: STRING "br'abc'" -115,12-115,13: OP '+' -115,14-115,21: STRING "Br'ABC'" -115,22-115,23: OP '+' -115,24-115,31: STRING "bR'ABC'" -115,32-115,33: OP '+' -115,34-115,41: STRING "BR'ABC'" -115,41-115,42: NEWLINE '\n' -116,0-116,1: NAME 'y' -116,2-116,3: OP '=' -116,4-116,11: STRING 'br"abc"' -116,12-116,13: OP '+' -116,14-116,21: STRING 'Br"ABC"' -116,22-116,23: OP '+' -116,24-116,31: STRING 'bR"ABC"' -116,32-116,33: OP '+' -116,34-116,41: STRING 'BR"ABC"' -116,41-116,42: NEWLINE '\n' -117,0-117,1: NAME 'x' -117,2-117,3: OP '=' -117,4-117,10: STRING "br'\\\\'" -117,11-117,12: OP '+' -117,13-117,19: STRING "BR'\\\\'" -117,19-117,20: NEWLINE '\n' -118,0-118,1: NAME 'x' -118,2-118,3: OP '=' -118,4-118,10: STRING "br'\\''" -118,11-118,12: OP '+' -118,13-118,15: STRING "''" -118,15-118,16: NEWLINE '\n' -119,0-119,1: NAME 'y' -119,2-119,3: OP '=' -119,4-121,6: STRING "br'''\nfoo bar \\\\\nbaz'''" -121,7-121,8: OP '+' -121,9-122,6: STRING "BR'''\nfoo'''" -122,6-122,7: NEWLINE '\n' -123,0-123,1: NAME 'y' -123,2-123,3: OP '=' -123,4-125,3: STRING 'Br"""foo\nbar \\\\ baz\n"""' -125,4-125,5: OP '+' -125,6-126,3: STRING "bR'''spam\n'''" -126,3-126,4: NEWLINE '\n' -127,0-127,1: NL '\n' -128,0-128,13: COMMENT '# Indentation' -128,13-128,14: NL '\n' -129,0-129,2: NAME 'if' -129,3-129,4: NUMBER '1' -129,4-129,5: OP ':' -129,5-129,6: NEWLINE '\n' -130,0-130,4: INDENT ' ' -130,4-130,5: NAME 'x' -130,6-130,7: OP '=' -130,8-130,9: NUMBER '2' -130,9-130,10: NEWLINE '\n' -131,0-131,0: DEDENT '' -131,0-131,2: NAME 'if' -131,3-131,4: NUMBER '1' -131,4-131,5: OP ':' -131,5-131,6: NEWLINE '\n' -132,0-132,8: INDENT ' ' -132,8-132,9: NAME 'x' -132,10-132,11: OP '=' -132,12-132,13: NUMBER '2' -132,13-132,14: NEWLINE '\n' -133,0-133,0: DEDENT '' -133,0-133,2: NAME 'if' -133,3-133,4: NUMBER '1' -133,4-133,5: OP ':' -133,5-133,6: NEWLINE '\n' -134,0-134,4: INDENT ' ' -134,4-134,9: NAME 'while' -134,10-134,11: NUMBER '0' -134,11-134,12: OP ':' -134,12-134,13: NEWLINE '\n' -135,0-135,5: INDENT ' ' -135,5-135,7: NAME 'if' -135,8-135,9: NUMBER '0' -135,9-135,10: OP ':' -135,10-135,11: NEWLINE '\n' -136,0-136,11: INDENT ' ' -136,11-136,12: NAME 'x' -136,13-136,14: OP '=' -136,15-136,16: NUMBER '2' -136,16-136,17: NEWLINE '\n' -137,5-137,5: DEDENT '' -137,5-137,6: NAME 'x' -137,7-137,8: OP '=' -137,9-137,10: NUMBER '2' -137,10-137,11: NEWLINE '\n' -138,0-138,0: DEDENT '' -138,0-138,0: DEDENT '' -138,0-138,2: NAME 'if' -138,3-138,4: NUMBER '0' -138,4-138,5: OP ':' -138,5-138,6: NEWLINE '\n' -139,0-139,2: INDENT ' ' -139,2-139,4: NAME 'if' -139,5-139,6: NUMBER '2' -139,6-139,7: OP ':' -139,7-139,8: NEWLINE '\n' -140,0-140,3: INDENT ' ' -140,3-140,8: NAME 'while' -140,9-140,10: NUMBER '0' -140,10-140,11: OP ':' -140,11-140,12: NEWLINE '\n' -141,0-141,8: INDENT ' ' -141,8-141,10: NAME 'if' -141,11-141,12: NUMBER '1' -141,12-141,13: OP ':' -141,13-141,14: NEWLINE '\n' -142,0-142,10: INDENT ' ' -142,10-142,11: NAME 'x' -142,12-142,13: OP '=' -142,14-142,15: NUMBER '2' -142,15-142,16: NEWLINE '\n' -143,0-143,1: NL '\n' -144,0-144,11: COMMENT '# Operators' -144,11-144,12: NL '\n' -145,0-145,1: NL '\n' -146,0-146,0: DEDENT '' -146,0-146,0: DEDENT '' -146,0-146,0: DEDENT '' -146,0-146,0: DEDENT '' -146,0-146,3: NAME 'def' -146,4-146,7: NAME 'd22' -146,7-146,8: OP '(' -146,8-146,9: NAME 'a' -146,9-146,10: OP ',' -146,11-146,12: NAME 'b' -146,12-146,13: OP ',' -146,14-146,15: NAME 'c' -146,15-146,16: OP '=' -146,16-146,17: NUMBER '1' -146,17-146,18: OP ',' -146,19-146,20: NAME 'd' -146,20-146,21: OP '=' -146,21-146,22: NUMBER '2' -146,22-146,23: OP ')' -146,23-146,24: OP ':' -146,25-146,29: NAME 'pass' -146,29-146,30: NEWLINE '\n' -147,0-147,3: NAME 'def' -147,4-147,8: NAME 'd01v' -147,8-147,9: OP '(' -147,9-147,10: NAME 'a' -147,10-147,11: OP '=' -147,11-147,12: NUMBER '1' -147,12-147,13: OP ',' -147,14-147,15: OP '*' -147,15-147,20: NAME 'restt' -147,20-147,21: OP ',' -147,22-147,24: OP '**' -147,24-147,29: NAME 'restd' -147,29-147,30: OP ')' -147,30-147,31: OP ':' -147,32-147,36: NAME 'pass' -147,36-147,37: NEWLINE '\n' -148,0-148,1: NL '\n' -149,0-149,1: OP '(' -149,1-149,2: NAME 'x' -149,2-149,3: OP ',' -149,4-149,5: NAME 'y' -149,5-149,6: OP ')' -149,7-149,9: OP '!=' -149,10-149,11: OP '(' -149,11-149,12: OP '{' -149,12-149,15: STRING "'a'" -149,15-149,16: OP ':' -149,16-149,17: NUMBER '1' -149,17-149,18: OP '}' -149,18-149,19: OP ',' -149,20-149,21: OP '{' -149,21-149,24: STRING "'b'" -149,24-149,25: OP ':' -149,25-149,26: NUMBER '2' -149,26-149,27: OP '}' -149,27-149,28: OP ')' -149,28-149,29: NEWLINE '\n' -150,0-150,1: NL '\n' -151,0-151,12: COMMENT '# comparison' -151,12-151,13: NL '\n' -152,0-152,2: NAME 'if' -152,3-152,4: NUMBER '1' -152,5-152,6: OP '<' -152,7-152,8: NUMBER '1' -152,9-152,10: OP '>' -152,11-152,12: NUMBER '1' -152,13-152,15: OP '==' -152,16-152,17: NUMBER '1' -152,18-152,20: OP '>=' -152,21-152,22: NUMBER '1' -152,23-152,25: OP '<=' -152,26-152,27: NUMBER '1' -152,28-152,30: OP '!=' -152,31-152,32: NUMBER '1' -152,33-152,35: OP '!=' -152,36-152,37: NUMBER '1' -152,38-152,40: NAME 'in' -152,41-152,42: NUMBER '1' -152,43-152,46: NAME 'not' -152,47-152,49: NAME 'in' -152,50-152,51: NUMBER '1' -152,52-152,54: NAME 'is' -152,55-152,56: NUMBER '1' -152,57-152,59: NAME 'is' -152,60-152,63: NAME 'not' -152,64-152,65: NUMBER '1' -152,65-152,66: OP ':' -152,67-152,71: NAME 'pass' -152,71-152,72: NEWLINE '\n' -153,0-153,1: NL '\n' -154,0-154,8: COMMENT '# binary' -154,8-154,9: NL '\n' -155,0-155,1: NAME 'x' -155,2-155,3: OP '=' -155,4-155,5: NUMBER '1' -155,6-155,7: OP '&' -155,8-155,9: NUMBER '1' -155,9-155,10: NEWLINE '\n' -156,0-156,1: NAME 'x' -156,2-156,3: OP '=' -156,4-156,5: NUMBER '1' -156,6-156,7: OP '^' -156,8-156,9: NUMBER '1' -156,9-156,10: NEWLINE '\n' -157,0-157,1: NAME 'x' -157,2-157,3: OP '=' -157,4-157,5: NUMBER '1' -157,6-157,7: OP '|' -157,8-157,9: NUMBER '1' -157,9-157,10: NEWLINE '\n' -158,0-158,1: NL '\n' -159,0-159,7: COMMENT '# shift' -159,7-159,8: NL '\n' -160,0-160,1: NAME 'x' -160,2-160,3: OP '=' -160,4-160,5: NUMBER '1' -160,6-160,8: OP '<<' -160,9-160,10: NUMBER '1' -160,11-160,13: OP '>>' -160,14-160,15: NUMBER '1' -160,15-160,16: NEWLINE '\n' -161,0-161,1: NL '\n' -162,0-162,10: COMMENT '# additive' -162,10-162,11: NL '\n' -163,0-163,1: NAME 'x' -163,2-163,3: OP '=' -163,4-163,5: NUMBER '1' -163,6-163,7: OP '-' -163,8-163,9: NUMBER '1' -163,10-163,11: OP '+' -163,12-163,13: NUMBER '1' -163,14-163,15: OP '-' -163,16-163,17: NUMBER '1' -163,18-163,19: OP '+' -163,20-163,21: NUMBER '1' -163,21-163,22: NEWLINE '\n' -164,0-164,1: NL '\n' -165,0-165,16: COMMENT '# multiplicative' -165,16-165,17: NL '\n' -166,0-166,1: NAME 'x' -166,2-166,3: OP '=' -166,4-166,5: NUMBER '1' -166,6-166,7: OP '/' -166,8-166,9: NUMBER '1' -166,10-166,11: OP '*' -166,12-166,13: NUMBER '1' -166,14-166,15: OP '%' -166,16-166,17: NUMBER '1' -166,17-166,18: NEWLINE '\n' -167,0-167,1: NL '\n' -168,0-168,7: COMMENT '# unary' -168,7-168,8: NL '\n' -169,0-169,1: NAME 'x' -169,2-169,3: OP '=' -169,4-169,5: OP '~' -169,5-169,6: NUMBER '1' -169,7-169,8: OP '^' -169,9-169,10: NUMBER '1' -169,11-169,12: OP '&' -169,13-169,14: NUMBER '1' -169,15-169,16: OP '|' -169,17-169,18: NUMBER '1' -169,19-169,20: OP '&' -169,21-169,22: NUMBER '1' -169,23-169,24: OP '^' -169,25-169,26: OP '-' -169,26-169,27: NUMBER '1' -169,27-169,28: NEWLINE '\n' -170,0-170,1: NAME 'x' -170,2-170,3: OP '=' -170,4-170,5: OP '-' -170,5-170,6: NUMBER '1' -170,6-170,7: OP '*' -170,7-170,8: NUMBER '1' -170,8-170,9: OP '/' -170,9-170,10: NUMBER '1' -170,11-170,12: OP '+' -170,13-170,14: NUMBER '1' -170,14-170,15: OP '*' -170,15-170,16: NUMBER '1' -170,17-170,18: OP '-' -170,19-170,20: OP '-' -170,20-170,21: OP '-' -170,21-170,22: OP '-' -170,22-170,23: NUMBER '1' -170,23-170,24: OP '*' -170,24-170,25: NUMBER '1' -170,25-170,26: NEWLINE '\n' -171,0-171,1: NL '\n' -172,0-172,10: COMMENT '# selector' -172,10-172,11: NL '\n' -173,0-173,6: NAME 'import' -173,7-173,10: NAME 'sys' -173,10-173,11: OP ',' -173,12-173,16: NAME 'time' -173,16-173,17: NEWLINE '\n' -174,0-174,1: NAME 'x' -174,2-174,3: OP '=' -174,4-174,7: NAME 'sys' -174,7-174,8: OP '.' -174,8-174,15: NAME 'modules' -174,15-174,16: OP '[' -174,16-174,22: STRING "'time'" -174,22-174,23: OP ']' -174,23-174,24: OP '.' -174,24-174,28: NAME 'time' -174,28-174,29: OP '(' -174,29-174,30: OP ')' -174,30-174,31: NEWLINE '\n' -175,0-175,1: NL '\n' -176,0-176,1: OP '@' -176,1-176,13: NAME 'staticmethod' -176,13-176,14: NEWLINE '\n' -177,0-177,3: NAME 'def' -177,4-177,7: NAME 'foo' -177,7-177,8: OP '(' -177,8-177,9: OP ')' -177,9-177,10: OP ':' -177,11-177,15: NAME 'pass' -177,15-177,16: NEWLINE '\n' -178,0-178,1: NL '\n' -179,0-179,1: OP '@' -179,1-179,13: NAME 'staticmethod' -179,13-179,14: NEWLINE '\n' -180,0-180,3: NAME 'def' -180,4-180,7: NAME 'foo' -180,7-180,8: OP '(' -180,8-180,9: NAME 'x' -180,9-180,10: OP ':' -180,10-180,11: NUMBER '1' -180,11-180,12: OP ')' -180,12-180,14: OP '->' -180,14-180,15: NUMBER '1' -180,15-180,16: OP ':' -180,17-180,21: NAME 'pass' -180,21-180,22: NEWLINE '\n' -181,0-181,1: NL '\n' -182,0-182,0: ENDMARKER '' diff --git a/Lib/test/test_crypt.py b/Lib/test/test_crypt.py index a9c28cd6bd..aa07f495db 100755 --- a/Lib/test/test_crypt.py +++ b/Lib/test/test_crypt.py @@ -1,11 +1,16 @@ -#! /usr/bin/env python -"""Simple test script for cryptmodule.c - Roger E. Masse -""" - -from test.test_support import verbose +from test import test_support +import unittest import crypt -c = crypt.crypt('mypassword', 'ab') -if verbose: - print('Test encryption: ', c) +class CryptTestCase(unittest.TestCase): + + def test_crypt(self): + c = crypt.crypt('mypassword', 'ab') + if test_support.verbose: + print('Test encryption: ', c) + +def test_main(): + test_support.run_unittest(CryptTestCase) + +if __name__ == "__main__": + test_main() diff --git a/Lib/test/test_datetime.py b/Lib/test/test_datetime.py index e768a9c7b4..e83048ec78 100644 --- a/Lib/test/test_datetime.py +++ b/Lib/test/test_datetime.py @@ -1521,11 +1521,12 @@ class TestDateTime(TestDate): self.failUnless(abs(from_timestamp - from_now) <= tolerance) def test_strptime(self): - import time + import _strptime - string = '2004-12-01 13:02:47' - format = '%Y-%m-%d %H:%M:%S' - expected = self.theclass(*(time.strptime(string, format)[0:6])) + string = '2004-12-01 13:02:47.197' + format = '%Y-%m-%d %H:%M:%S.%f' + result, frac = _strptime._strptime(string, format) + expected = self.theclass(*(result[0:6]+(frac,))) got = self.theclass.strptime(string, format) self.assertEqual(expected, got) @@ -1553,9 +1554,9 @@ class TestDateTime(TestDate): def test_more_strftime(self): # This tests fields beyond those tested by the TestDate.test_strftime. - t = self.theclass(2004, 12, 31, 6, 22, 33) - self.assertEqual(t.strftime("%m %d %y %S %M %H %j"), - "12 31 04 33 22 06 366") + t = self.theclass(2004, 12, 31, 6, 22, 33, 47) + self.assertEqual(t.strftime("%m %d %y %f %S %M %H %j"), + "12 31 04 000047 33 22 06 366") def test_extract(self): dt = self.theclass(2002, 3, 4, 18, 45, 3, 1234) @@ -1828,7 +1829,7 @@ class TestTime(HarmlessMixedComparison, unittest.TestCase): def test_strftime(self): t = self.theclass(1, 2, 3, 4) - self.assertEqual(t.strftime('%H %M %S'), "01 02 03") + self.assertEqual(t.strftime('%H %M %S %f'), "01 02 03 000004") # A naive object replaces %z and %Z with empty strings. self.assertEqual(t.strftime("'%z' '%Z'"), "'' ''") diff --git a/Lib/test/test_fcntl.py b/Lib/test/test_fcntl.py index 870876d076..665f5b3bc1 100755 --- a/Lib/test/test_fcntl.py +++ b/Lib/test/test_fcntl.py @@ -1,69 +1,88 @@ -#! /usr/bin/env python """Test program for the fcntl C module. - OS/2+EMX doesn't support the file locking operations. - Roger E. Masse + +OS/2+EMX doesn't support the file locking operations. + """ import struct import fcntl import os, sys -from test.test_support import verbose, TESTFN +import unittest +from test.test_support import verbose, TESTFN, unlink, run_unittest -filename = TESTFN +# TODO - Write tests for ioctl(), flock() and lockf(). -try: - os.O_LARGEFILE -except AttributeError: - start_len = "ll" -else: - start_len = "qq" -if sys.platform.startswith('atheos'): - start_len = "qq" +def get_lockdata(): + if sys.platform.startswith('atheos'): + start_len = "qq" + else: + try: + os.O_LARGEFILE + except AttributeError: + start_len = "ll" + else: + start_len = "qq" -if sys.platform in ('netbsd1', 'netbsd2', 'netbsd3', - 'Darwin1.2', 'darwin', - 'freebsd2', 'freebsd3', 'freebsd4', 'freebsd5', - 'freebsd6', 'freebsd7', 'freebsd8', - 'bsdos2', 'bsdos3', 'bsdos4', - 'openbsd', 'openbsd2', 'openbsd3', 'openbsd4'): - if struct.calcsize('l') == 8: - off_t = 'l' - pid_t = 'i' + if sys.platform in ('netbsd1', 'netbsd2', 'netbsd3', + 'Darwin1.2', 'darwin', + 'freebsd2', 'freebsd3', 'freebsd4', 'freebsd5', + 'freebsd6', 'freebsd7', 'freebsd8', + 'bsdos2', 'bsdos3', 'bsdos4', + 'openbsd', 'openbsd2', 'openbsd3', 'openbsd4'): + if struct.calcsize('l') == 8: + off_t = 'l' + pid_t = 'i' + else: + off_t = 'lxxxx' + pid_t = 'l' + lockdata = struct.pack(off_t + off_t + pid_t + 'hh', 0, 0, 0, + fcntl.F_WRLCK, 0) + elif sys.platform in ['aix3', 'aix4', 'hp-uxB', 'unixware7']: + lockdata = struct.pack('hhlllii', fcntl.F_WRLCK, 0, 0, 0, 0, 0, 0) + elif sys.platform in ['os2emx']: + lockdata = None else: - off_t = 'lxxxx' - pid_t = 'l' - lockdata = struct.pack(off_t+off_t+pid_t+'hh', 0, 0, 0, fcntl.F_WRLCK, 0) -elif sys.platform in ['aix3', 'aix4', 'hp-uxB', 'unixware7']: - lockdata = struct.pack('hhlllii', fcntl.F_WRLCK, 0, 0, 0, 0, 0, 0) -elif sys.platform in ['os2emx']: - lockdata = None -else: - lockdata = struct.pack('hh'+start_len+'hh', fcntl.F_WRLCK, 0, 0, 0, 0, 0) -if lockdata: - if verbose: - print('struct.pack: ', repr(lockdata)) + lockdata = struct.pack('hh'+start_len+'hh', fcntl.F_WRLCK, 0, 0, 0, 0, 0) + if lockdata: + if verbose: + print('struct.pack: ', repr(lockdata)) + return lockdata + +lockdata = get_lockdata() + +class TestFcntl(unittest.TestCase): -# the example from the library docs -f = open(filename, 'w') -rv = fcntl.fcntl(f.fileno(), fcntl.F_SETFL, os.O_NONBLOCK) -if verbose: - print('Status from fcntl with O_NONBLOCK: ', rv) + def setUp(self): + self.f = None -if sys.platform not in ['os2emx']: - rv = fcntl.fcntl(f.fileno(), fcntl.F_SETLKW, lockdata) - if verbose: - print('String from fcntl with F_SETLKW: ', repr(rv)) + def tearDown(self): + if not self.f.closed: + self.f.close() + unlink(TESTFN) -f.close() -os.unlink(filename) + def test_fcntl_fileno(self): + # the example from the library docs + self.f = open(TESTFN, 'w') + rv = fcntl.fcntl(self.f.fileno(), fcntl.F_SETFL, os.O_NONBLOCK) + if verbose: + print('Status from fcntl with O_NONBLOCK: ', rv) + if sys.platform not in ['os2emx']: + rv = fcntl.fcntl(self.f.fileno(), fcntl.F_SETLKW, lockdata) + if verbose: + print('String from fcntl with F_SETLKW: ', repr(rv)) + self.f.close() + def test_fcntl_file_descriptor(self): + # again, but pass the file rather than numeric descriptor + self.f = open(TESTFN, 'w') + rv = fcntl.fcntl(self.f, fcntl.F_SETFL, os.O_NONBLOCK) + if sys.platform not in ['os2emx']: + rv = fcntl.fcntl(self.f, fcntl.F_SETLKW, lockdata) + self.f.close() -# Again, but pass the file rather than numeric descriptor: -f = open(filename, 'w') -rv = fcntl.fcntl(f, fcntl.F_SETFL, os.O_NONBLOCK) -if sys.platform not in ['os2emx']: - rv = fcntl.fcntl(f, fcntl.F_SETLKW, lockdata) +def test_main(): + run_unittest(TestFcntl) -f.close() -os.unlink(filename) +if __name__ == '__main__': + test_main() diff --git a/Lib/test/test_gdbm.py b/Lib/test/test_gdbm.py index ee6827717e..17a7f632cf 100755 --- a/Lib/test/test_gdbm.py +++ b/Lib/test/test_gdbm.py @@ -1,46 +1,81 @@ -#! /usr/bin/env python -"""Test script for the gdbm module - Roger E. Masse -""" - import gdbm -from gdbm import error -from test.test_support import verbose, verify, TestFailed, TESTFN +import unittest +import os +from test.test_support import verbose, TESTFN, run_unittest, unlink + filename = TESTFN -g = gdbm.open(filename, 'c') -verify(g.keys() == []) -g[b'a'] = b'b' -g[b'12345678910'] = b'019237410982340912840198242' -a = g.keys() -if verbose: - print('Test gdbm file keys: ', a) - -b'a' in g -g.close() -try: - g['a'] -except error: - pass -else: - raise TestFailed("expected gdbm.error accessing closed database") -g = gdbm.open(filename, 'r') -g.close() -g = gdbm.open(filename, 'w') -g.close() -g = gdbm.open(filename, 'n') -g.close() -try: - g = gdbm.open(filename, 'rx') - g.close() -except error: - pass -else: - raise TestFailed("expected gdbm.error when passing invalid open flags") - -try: - import os - os.unlink(filename) -except: - pass +class TestGdbm(unittest.TestCase): + def setUp(self): + self.g = None + + def tearDown(self): + if self.g is not None: + self.g.close() + unlink(filename) + + def test_key_methods(self): + self.g = gdbm.open(filename, 'c') + self.assertEqual(self.g.keys(), []) + self.g['a'] = 'b' + self.g['12345678910'] = '019237410982340912840198242' + key_set = set(self.g.keys()) + self.assertEqual(key_set, set([b'a', b'12345678910'])) + self.assert_(b'a' in self.g) + key = self.g.firstkey() + while key: + self.assert_(key in key_set) + key_set.remove(key) + key = self.g.nextkey(key) + self.assertRaises(KeyError, lambda: self.g['xxx']) + + def test_error_conditions(self): + # Try to open a non-existent database. + unlink(filename) + self.assertRaises(gdbm.error, gdbm.open, filename, 'r') + # Try to access a closed database. + self.g = gdbm.open(filename, 'c') + self.g.close() + self.assertRaises(gdbm.error, lambda: self.g['a']) + # try pass an invalid open flag + self.assertRaises(gdbm.error, lambda: gdbm.open(filename, 'rx').close()) + + def test_flags(self): + # Test the flag parameter open() by trying all supported flag modes. + all = set(gdbm.open_flags) + # Test standard flags (presumably "crwn"). + modes = all - set('fsu') + for mode in modes: + self.g = gdbm.open(filename, mode) + self.g.close() + + # Test additional flags (presumably "fsu"). + flags = all - set('crwn') + for mode in modes: + for flag in flags: + self.g = gdbm.open(filename, mode + flag) + self.g.close() + + def test_reorganize(self): + self.g = gdbm.open(filename, 'c') + size0 = os.path.getsize(filename) + + self.g['x'] = 'x' * 10000 + size1 = os.path.getsize(filename) + self.assert_(size0 < size1) + + del self.g['x'] + # 'size' is supposed to be the same even after deleting an entry. + self.assertEqual(os.path.getsize(filename), size1) + + self.g.reorganize() + size2 = os.path.getsize(filename) + self.assert_(size1 > size2 >= size0) + + +def test_main(): + run_unittest(TestGdbm) + +if __name__ == '__main__': + test_main() diff --git a/Lib/test/test_grammar.py b/Lib/test/test_grammar.py index 04aedd536e..4c0eda53d3 100644 --- a/Lib/test/test_grammar.py +++ b/Lib/test/test_grammar.py @@ -310,6 +310,10 @@ class GrammarTests(unittest.TestCase): def f(*, k=1): return closure def f() -> int: return closure + # Check ast errors in *args and *kwargs + check_syntax_error(self, "f(*g(1=2))") + check_syntax_error(self, "f(**g(1=2))") + def testLambdef(self): ### lambdef: 'lambda' [varargslist] ':' test l1 = lambda : 0 diff --git a/Lib/test/test_heapq.py b/Lib/test/test_heapq.py index 1e21736b7a..24d2992479 100644 --- a/Lib/test/test_heapq.py +++ b/Lib/test/test_heapq.py @@ -107,6 +107,34 @@ class TestHeap(unittest.TestCase): self.assertRaises(TypeError, self.module.heapreplace, None, None) self.assertRaises(IndexError, self.module.heapreplace, [], None) + def test_nbest_with_pushpop(self): + data = [random.randrange(2000) for i in range(1000)] + heap = data[:10] + self.module.heapify(heap) + for item in data[10:]: + self.module.heappushpop(heap, item) + self.assertEqual(list(self.heapiter(heap)), sorted(data)[-10:]) + self.assertEqual(self.module.heappushpop([], 'x'), 'x') + + def test_heappushpop(self): + h = [] + x = self.module.heappushpop(h, 10) + self.assertEqual((h, x), ([], 10)) + + h = [10] + x = self.module.heappushpop(h, 10.0) + self.assertEqual((h, x), ([10], 10.0)) + self.assertEqual(type(h[0]), int) + self.assertEqual(type(x), float) + + h = [10]; + x = self.module.heappushpop(h, 9) + self.assertEqual((h, x), ([10], 9)) + + h = [10]; + x = self.module.heappushpop(h, 11) + self.assertEqual((h, x), ([11], 10)) + def test_heapsort(self): # Exercise everything with repeated heapsort checks for trial in range(100): diff --git a/Lib/test/test_itertools.py b/Lib/test/test_itertools.py index d44235bc77..335e47ddc0 100644 --- a/Lib/test/test_itertools.py +++ b/Lib/test/test_itertools.py @@ -51,22 +51,21 @@ def fact(n): 'Factorial' return prod(range(1, n+1)) -def permutations(iterable, r=None): - # XXX use this until real permutations code is added - pool = tuple(iterable) - n = len(pool) - r = n if r is None else r - for indices in product(range(n), repeat=r): - if len(set(indices)) == r: - yield tuple(pool[i] for i in indices) - class TestBasicOps(unittest.TestCase): def test_chain(self): - self.assertEqual(list(chain('abc', 'def')), list('abcdef')) - self.assertEqual(list(chain('abc')), list('abc')) - self.assertEqual(list(chain('')), []) - self.assertEqual(take(4, chain('abc', 'def')), list('abcd')) - self.assertRaises(TypeError, list,chain(2, 3)) + + def chain2(*iterables): + 'Pure python version in the docs' + for it in iterables: + for element in it: + yield element + + for c in (chain, chain2): + self.assertEqual(list(c('abc', 'def')), list('abcdef')) + self.assertEqual(list(c('abc')), list('abc')) + self.assertEqual(list(c('')), []) + self.assertEqual(take(4, c('abc', 'def')), list('abcd')) + self.assertRaises(TypeError, list,c(2, 3)) def test_chain_from_iterable(self): self.assertEqual(list(chain.from_iterable(['abc', 'def'])), list('abcdef')) @@ -121,6 +120,8 @@ class TestBasicOps(unittest.TestCase): self.assertEqual(len(set(c)), r) # no duplicate elements self.assertEqual(list(c), sorted(c)) # keep original ordering self.assert_(all(e in values for e in c)) # elements taken from input iterable + self.assertEqual(list(c), + [e for e in values if e in c]) # comb is a subsequence of the input iterable self.assertEqual(result, list(combinations1(values, r))) # matches first pure python version self.assertEqual(result, list(combinations2(values, r))) # matches first pure python version @@ -131,9 +132,10 @@ class TestBasicOps(unittest.TestCase): def test_permutations(self): self.assertRaises(TypeError, permutations) # too few arguments self.assertRaises(TypeError, permutations, 'abc', 2, 1) # too many arguments -## self.assertRaises(TypeError, permutations, None) # pool is not iterable -## self.assertRaises(ValueError, permutations, 'abc', -2) # r is negative -## self.assertRaises(ValueError, permutations, 'abc', 32) # r is too big + self.assertRaises(TypeError, permutations, None) # pool is not iterable + self.assertRaises(ValueError, permutations, 'abc', -2) # r is negative + self.assertRaises(ValueError, permutations, 'abc', 32) # r is too big + self.assertRaises(TypeError, permutations, 'abc', 's') # r is not an int or None self.assertEqual(list(permutations(range(3), 2)), [(0,1), (0,2), (1,0), (1,2), (2,0), (2,1)]) @@ -186,7 +188,7 @@ class TestBasicOps(unittest.TestCase): self.assertEqual(result, list(permutations(values))) # test default r # Test implementation detail: tuple re-use -## self.assertEqual(len(set(map(id, permutations('abcde', 3)))), 1) + self.assertEqual(len(set(map(id, permutations('abcde', 3)))), 1) self.assertNotEqual(len(set(map(id, list(permutations('abcde', 3))))), 1) def test_count(self): @@ -416,12 +418,46 @@ class TestBasicOps(unittest.TestCase): list(product(*args, **dict(repeat=r)))) self.assertEqual(len(list(product(*[range(7)]*6))), 7**6) self.assertRaises(TypeError, product, range(6), None) + + def product1(*args, **kwds): + pools = list(map(tuple, args)) * kwds.get('repeat', 1) + n = len(pools) + if n == 0: + yield () + return + if any(len(pool) == 0 for pool in pools): + return + indices = [0] * n + yield tuple(pool[i] for pool, i in zip(pools, indices)) + while 1: + for i in reversed(range(n)): # right to left + if indices[i] == len(pools[i]) - 1: + continue + indices[i] += 1 + for j in range(i+1, n): + indices[j] = 0 + yield tuple(pool[i] for pool, i in zip(pools, indices)) + break + else: + return + + def product2(*args, **kwds): + 'Pure python version used in docs' + pools = list(map(tuple, args)) * kwds.get('repeat', 1) + result = [[]] + for pool in pools: + result = [x+[y] for x in result for y in pool] + for prod in result: + yield tuple(prod) + argtypes = ['', 'abc', '', range(0), range(4), dict(a=1, b=2, c=3), set('abcdefg'), range(11), tuple(range(13))] for i in range(100): args = [random.choice(argtypes) for j in range(random.randrange(5))] expected_len = prod(map(len, args)) self.assertEqual(len(list(product(*args))), expected_len) + self.assertEqual(list(product(*args)), list(product1(*args))) + self.assertEqual(list(product(*args)), list(product2(*args))) args = map(iter, args) self.assertEqual(len(list(product(*args))), expected_len) @@ -661,6 +697,81 @@ class TestBasicOps(unittest.TestCase): self.assertRaises(StopIteration, next, f(lambda x:x, [])) self.assertRaises(StopIteration, next, f(lambda x:x, StopNow())) +class TestExamples(unittest.TestCase): + + def test_chain(self): + self.assertEqual(''.join(chain('ABC', 'DEF')), 'ABCDEF') + + def test_chain_from_iterable(self): + self.assertEqual(''.join(chain.from_iterable(['ABC', 'DEF'])), 'ABCDEF') + + def test_combinations(self): + self.assertEqual(list(combinations('ABCD', 2)), + [('A','B'), ('A','C'), ('A','D'), ('B','C'), ('B','D'), ('C','D')]) + self.assertEqual(list(combinations(range(4), 3)), + [(0,1,2), (0,1,3), (0,2,3), (1,2,3)]) + + def test_count(self): + self.assertEqual(list(islice(count(10), 5)), [10, 11, 12, 13, 14]) + + def test_cycle(self): + self.assertEqual(list(islice(cycle('ABCD'), 12)), list('ABCDABCDABCD')) + + def test_dropwhile(self): + self.assertEqual(list(dropwhile(lambda x: x<5, [1,4,6,4,1])), [6,4,1]) + + def test_groupby(self): + self.assertEqual([k for k, g in groupby('AAAABBBCCDAABBB')], + list('ABCDAB')) + self.assertEqual([(list(g)) for k, g in groupby('AAAABBBCCD')], + [list('AAAA'), list('BBB'), list('CC'), list('D')]) + + def test_filter(self): + self.assertEqual(list(filter(lambda x: x%2, range(10))), [1,3,5,7,9]) + + def test_filterfalse(self): + self.assertEqual(list(filterfalse(lambda x: x%2, range(10))), [0,2,4,6,8]) + + def test_map(self): + self.assertEqual(list(map(pow, (2,3,10), (5,2,3))), [32, 9, 1000]) + + def test_islice(self): + self.assertEqual(list(islice('ABCDEFG', 2)), list('AB')) + self.assertEqual(list(islice('ABCDEFG', 2, 4)), list('CD')) + self.assertEqual(list(islice('ABCDEFG', 2, None)), list('CDEFG')) + self.assertEqual(list(islice('ABCDEFG', 0, None, 2)), list('ACEG')) + + def test_zip(self): + self.assertEqual(list(zip('ABCD', 'xy')), [('A', 'x'), ('B', 'y')]) + + def test_zip_longest(self): + self.assertEqual(list(zip_longest('ABCD', 'xy', fillvalue='-')), + [('A', 'x'), ('B', 'y'), ('C', '-'), ('D', '-')]) + + def test_permutations(self): + self.assertEqual(list(permutations('ABCD', 2)), + list(map(tuple, 'AB AC AD BA BC BD CA CB CD DA DB DC'.split()))) + self.assertEqual(list(permutations(range(3))), + [(0,1,2), (0,2,1), (1,0,2), (1,2,0), (2,0,1), (2,1,0)]) + + def test_product(self): + self.assertEqual(list(product('ABCD', 'xy')), + list(map(tuple, 'Ax Ay Bx By Cx Cy Dx Dy'.split()))) + self.assertEqual(list(product(range(2), repeat=3)), + [(0,0,0), (0,0,1), (0,1,0), (0,1,1), + (1,0,0), (1,0,1), (1,1,0), (1,1,1)]) + + def test_repeat(self): + self.assertEqual(list(repeat(10, 3)), [10, 10, 10]) + + def test_stapmap(self): + self.assertEqual(list(starmap(pow, [(2,5), (3,2), (10,3)])), + [32, 9, 1000]) + + def test_takewhile(self): + self.assertEqual(list(takewhile(lambda x: x<5, [1,4,6,4,1])), [1,4]) + + class TestGC(unittest.TestCase): def makecycle(self, iterator, container): @@ -672,6 +783,14 @@ class TestGC(unittest.TestCase): a = [] self.makecycle(chain(a), a) + def test_chain_from_iterable(self): + a = [] + self.makecycle(chain.from_iterable([a]), a) + + def test_combinations(self): + a = [] + self.makecycle(combinations([1,2,a,3], 3), a) + def test_cycle(self): a = [] self.makecycle(cycle([a]*2), a) @@ -684,6 +803,13 @@ class TestGC(unittest.TestCase): a = [] self.makecycle(groupby([a]*2, lambda x:x), a) + def test_issue2246(self): + # Issue 2246 -- the _grouper iterator was not included in GC + n = 10 + keyfunc = lambda x: x + for i, j in groupby(range(n), key=keyfunc): + keyfunc.__dict__.setdefault('x',[]).append(j) + def test_filter(self): a = [] self.makecycle(filter(lambda x:True, [a]*2), a) @@ -696,6 +822,12 @@ class TestGC(unittest.TestCase): a = [] self.makecycle(zip([a]*2, [a]*3), a) + def test_zip_longest(self): + a = [] + self.makecycle(zip_longest([a]*2, [a]*3), a) + b = [a, None] + self.makecycle(zip_longest([a]*2, [a]*3, fillvalue=b), a) + def test_map(self): a = [] self.makecycle(map(lambda x:x, [a]*2), a) @@ -704,6 +836,14 @@ class TestGC(unittest.TestCase): a = [] self.makecycle(islice([a]*2, None), a) + def test_permutations(self): + a = [] + self.makecycle(permutations([1,2,a,3], 3), a) + + def test_product(self): + a = [] + self.makecycle(product([1,2,a,3], repeat=3), a) + def test_repeat(self): a = [] self.makecycle(repeat(a), a) @@ -1115,7 +1255,7 @@ Samuele ... return sum(map(operator.mul, vec1, vec2)) >>> def flatten(listOfLists): -... return list(chain(*listOfLists)) +... return list(chain.from_iterable(listOfLists)) >>> def repeatfunc(func, times=None, *args): ... "Repeat calls to func with specified arguments." @@ -1134,6 +1274,38 @@ Samuele ... pass ... return zip(a, b) +>>> def grouper(n, iterable, fillvalue=None): +... "grouper(3, 'abcdefg', 'x') --> ('a','b','c'), ('d','e','f'), ('g','x','x')" +... args = [iter(iterable)] * n +... kwds = dict(fillvalue=fillvalue) +... return zip_longest(*args, **kwds) + +>>> def roundrobin(*iterables): +... "roundrobin('abc', 'd', 'ef') --> 'a', 'd', 'e', 'b', 'f', 'c'" +... # Recipe credited to George Sakkis +... pending = len(iterables) +... nexts = cycle(iter(it).__next__ for it in iterables) +... while pending: +... try: +... for next in nexts: +... yield next() +... except StopIteration: +... pending -= 1 +... nexts = cycle(islice(nexts, pending)) + +>>> def powerset(iterable): +... "powerset('ab') --> set([]), set(['a']), set(['b']), set(['a', 'b'])" +... # Recipe credited to Eric Raymond +... pairs = [(2**i, x) for i, x in enumerate(iterable)] +... for n in range(2**len(pairs)): +... yield set(x for m, x in pairs if m&n) + +>>> def compress(data, selectors): +... "compress('abcdef', [1,0,1,0,1,1]) --> a c e f" +... for d, s in zip(data, selectors): +... if s: +... yield d + This is not part of the examples but it tests to make sure the definitions perform as purported. @@ -1199,6 +1371,18 @@ False >>> dotproduct([1,2,3], [4,5,6]) 32 +>>> list(grouper(3, 'abcdefg', 'x')) +[('a', 'b', 'c'), ('d', 'e', 'f'), ('g', 'x', 'x')] + +>>> list(roundrobin('abc', 'd', 'ef')) +['a', 'd', 'e', 'b', 'f', 'c'] + +>>> list(map(sorted, powerset('ab'))) +[[], ['a'], ['b'], ['a', 'b']] + +>>> list(compress('abcdef', [1,0,1,0,1,1])) +['a', 'c', 'e', 'f'] + """ __test__ = {'libreftest' : libreftest} @@ -1206,7 +1390,7 @@ __test__ = {'libreftest' : libreftest} def test_main(verbose=None): test_classes = (TestBasicOps, TestVariousIteratorArgs, TestGC, RegressionTests, LengthTransparency, - SubclassWithKwargsTest) + SubclassWithKwargsTest, TestExamples) test_support.run_unittest(*test_classes) # verify reference counting diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py index 6cabb82f55..5f233366a2 100644 --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -26,6 +26,114 @@ class FileTests(unittest.TestCase): os.closerange(f, f+2) self.assertRaises(OSError, os.write, f, "a") +class TemporaryFileTests(unittest.TestCase): + def setUp(self): + self.files = [] + os.mkdir(test_support.TESTFN) + + def tearDown(self): + for name in self.files: + os.unlink(name) + os.rmdir(test_support.TESTFN) + + def check_tempfile(self, name): + # make sure it doesn't already exist: + self.failIf(os.path.exists(name), + "file already exists for temporary file") + # make sure we can create the file + open(name, "w") + self.files.append(name) + + def test_tempnam(self): + if not hasattr(os, "tempnam"): + return + warnings.filterwarnings("ignore", "tempnam", RuntimeWarning, + r"test_os$") + self.check_tempfile(os.tempnam()) + + name = os.tempnam(test_support.TESTFN) + self.check_tempfile(name) + + name = os.tempnam(test_support.TESTFN, "pfx") + self.assert_(os.path.basename(name)[:3] == "pfx") + self.check_tempfile(name) + + def test_tmpfile(self): + if not hasattr(os, "tmpfile"): + return + # As with test_tmpnam() below, the Windows implementation of tmpfile() + # attempts to create a file in the root directory of the current drive. + # On Vista and Server 2008, this test will always fail for normal users + # as writing to the root directory requires elevated privileges. With + # XP and below, the semantics of tmpfile() are the same, but the user + # running the test is more likely to have administrative privileges on + # their account already. If that's the case, then os.tmpfile() should + # work. In order to make this test as useful as possible, rather than + # trying to detect Windows versions or whether or not the user has the + # right permissions, just try and create a file in the root directory + # and see if it raises a 'Permission denied' OSError. If it does, then + # test that a subsequent call to os.tmpfile() raises the same error. If + # it doesn't, assume we're on XP or below and the user running the test + # has administrative privileges, and proceed with the test as normal. + if sys.platform == 'win32': + name = '\\python_test_os_test_tmpfile.txt' + if os.path.exists(name): + os.remove(name) + try: + fp = open(name, 'w') + except IOError as first: + # open() failed, assert tmpfile() fails in the same way. + # Although open() raises an IOError and os.tmpfile() raises an + # OSError(), 'args' will be (13, 'Permission denied') in both + # cases. + try: + fp = os.tmpfile() + except OSError as second: + self.assertEqual(first.args, second.args) + else: + self.fail("expected os.tmpfile() to raise OSError") + return + else: + # open() worked, therefore, tmpfile() should work. Close our + # dummy file and proceed with the test as normal. + fp.close() + os.remove(name) + + fp = os.tmpfile() + fp.write("foobar") + fp.seek(0,0) + s = fp.read() + fp.close() + self.assert_(s == "foobar") + + def test_tmpnam(self): + import sys + if not hasattr(os, "tmpnam"): + return + warnings.filterwarnings("ignore", "tmpnam", RuntimeWarning, + r"test_os$") + name = os.tmpnam() + if sys.platform in ("win32",): + # The Windows tmpnam() seems useless. From the MS docs: + # + # The character string that tmpnam creates consists of + # the path prefix, defined by the entry P_tmpdir in the + # file STDIO.H, followed by a sequence consisting of the + # digit characters '0' through '9'; the numerical value + # of this string is in the range 1 - 65,535. Changing the + # definitions of L_tmpnam or P_tmpdir in STDIO.H does not + # change the operation of tmpnam. + # + # The really bizarre part is that, at least under MSVC6, + # P_tmpdir is "\\". That is, the path returned refers to + # the root of the current drive. That's a terrible place to + # put temp files, and, depending on privileges, the user + # may not even be able to open a file in the root directory. + self.failIf(os.path.exists(name), + "file already exists for temporary file") + else: + self.check_tempfile(name) + # Test attributes on return values from os.*stat* family. class StatAttributeTests(unittest.TestCase): def setUp(self): diff --git a/Lib/test/test_select.py b/Lib/test/test_select.py index b45ec7f8ed..f4032d1c0c 100644 --- a/Lib/test/test_select.py +++ b/Lib/test/test_select.py @@ -1,70 +1,52 @@ -# Testing select module -from test.test_support import verbose, reap_children +from test import test_support +import unittest import select import os - -# test some known error conditions -try: - rfd, wfd, xfd = select.select(1, 2, 3) -except TypeError: - pass -else: - print('expected TypeError exception not raised') - -class Nope: - pass - -class Almost: - def fileno(self): - return 'fileno' - -try: - rfd, wfd, xfd = select.select([Nope()], [], []) -except TypeError: - pass -else: - print('expected TypeError exception not raised') - -try: - rfd, wfd, xfd = select.select([Almost()], [], []) -except TypeError: - pass -else: - print('expected TypeError exception not raised') - -try: - rfd, wfd, xfd = select.select([], [], [], 'not a number') -except TypeError: - pass -else: - print('expected TypeError exception not raised') - - -def test(): - import sys - if sys.platform[:3] in ('win', 'mac', 'os2'): - if verbose: - print("Can't test select easily on", sys.platform) - return - cmd = 'for i in 0 1 2 3 4 5 6 7 8 9; do echo testing...; sleep 1; done' - p = os.popen(cmd, 'r') - for tout in (0, 1, 2, 4, 8, 16) + (None,)*10: - if verbose: - print('timeout =', tout) - rfd, wfd, xfd = select.select([p], [], [], tout) - if (rfd, wfd, xfd) == ([], [], []): - continue - if (rfd, wfd, xfd) == ([p], [], []): - line = p.readline() - if verbose: - print(repr(line)) - if not line: - if verbose: - print('EOF') - break - continue - print('Unexpected return values from select():', rfd, wfd, xfd) - p.close() - reap_children() - -test() +import sys + +class SelectTestCase(unittest.TestCase): + + class Nope: + pass + + class Almost: + def fileno(self): + return 'fileno' + + def test_error_conditions(self): + self.assertRaises(TypeError, select.select, 1, 2, 3) + self.assertRaises(TypeError, select.select, [self.Nope()], [], []) + self.assertRaises(TypeError, select.select, [self.Almost()], [], []) + self.assertRaises(TypeError, select.select, [], [], [], "not a number") + + def test_select(self): + if sys.platform[:3] in ('win', 'mac', 'os2', 'riscos'): + if test_support.verbose: + print("Can't test select easily on", sys.platform) + return + cmd = 'for i in 0 1 2 3 4 5 6 7 8 9; do echo testing...; sleep 1; done' + p = os.popen(cmd, 'r') + for tout in (0, 1, 2, 4, 8, 16) + (None,)*10: + if test_support.verbose: + print('timeout =', tout) + rfd, wfd, xfd = select.select([p], [], [], tout) + if (rfd, wfd, xfd) == ([], [], []): + continue + if (rfd, wfd, xfd) == ([p], [], []): + line = p.readline() + if test_support.verbose: + print(repr(line)) + if not line: + if test_support.verbose: + print('EOF') + break + continue + self.fail('Unexpected return values from select():', rfd, wfd, xfd) + p.close() + +def test_main(): + test_support.run_unittest(SelectTestCase) + test_support.reap_children() + +if __name__ == "__main__": + test_main() diff --git a/Lib/test/test_smtplib.py b/Lib/test/test_smtplib.py index 5694bb5cda..a9d5672396 100644 --- a/Lib/test/test_smtplib.py +++ b/Lib/test/test_smtplib.py @@ -19,7 +19,7 @@ PORT = None def server(evt, buf): serv = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - serv.settimeout(1) + serv.settimeout(15) serv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) serv.bind(("", 0)) global PORT diff --git a/Lib/test/test_socketserver.py b/Lib/test/test_socketserver.py index 658056c968..7fe746d2f0 100644 --- a/Lib/test/test_socketserver.py +++ b/Lib/test/test_socketserver.py @@ -21,13 +21,16 @@ from test.test_support import TESTFN as TEST_FILE test.test_support.requires("network") -NREQ = 3 TEST_STR = b"hello world\n" HOST = "localhost" HAVE_UNIX_SOCKETS = hasattr(socket, "AF_UNIX") HAVE_FORKING = hasattr(os, "fork") and os.name != "os2" +def signal_alarm(n): + """Call signal.alarm when it exists (i.e. not on Windows).""" + if hasattr(signal, 'alarm'): + signal.alarm(n) def receive(sock, n, timeout=20): r, w, x = select.select([sock], [], [], timeout) @@ -46,70 +49,6 @@ if HAVE_UNIX_SOCKETS: pass -class MyMixinServer: - def serve_a_few(self): - for i in range(NREQ): - self.handle_request() - - def handle_error(self, request, client_address): - self.close_request(request) - self.server_close() - raise - -def receive(sock, n, timeout=20): - r, w, x = select.select([sock], [], [], timeout) - if sock in r: - return sock.recv(n) - else: - raise RuntimeError("timed out on %r" % (sock,)) - -def testdgram(proto, addr): - s = socket.socket(proto, socket.SOCK_DGRAM) - s.sendto(teststring, addr) - buf = data = receive(s, 100) - while data and b'\n' not in buf: - data = receive(s, 100) - buf += data - verify(buf == teststring) - s.close() - -def teststream(proto, addr): - s = socket.socket(proto, socket.SOCK_STREAM) - s.connect(addr) - s.sendall(teststring) - buf = data = receive(s, 100) - while data and b'\n' not in buf: - data = receive(s, 100) - buf += data - verify(buf == teststring) - s.close() - -class ServerThread(threading.Thread): - def __init__(self, addr, svrcls, hdlrcls): - threading.Thread.__init__(self) - self.__addr = addr - self.__svrcls = svrcls - self.__hdlrcls = hdlrcls - self.ready = threading.Event() - - def run(self): - class svrcls(MyMixinServer, self.__svrcls): - pass - if verbose: print("thread: creating server") - svr = svrcls(self.__addr, self.__hdlrcls) - # We had the OS pick a port, so pull the real address out of - # the server. - self.addr = svr.server_address - self.port = self.addr[1] - if self.addr != svr.socket.getsockname(): - raise RuntimeError('server_address was %s, expected %s' % - (self.addr, svr.socket.getsockname())) - self.ready.set() - if verbose: print("thread: serving three times") - svr.serve_a_few() - if verbose: print("thread: done") - - @contextlib.contextmanager def simple_subprocess(testcase): pid = os.fork() @@ -126,7 +65,7 @@ class SocketServerTest(unittest.TestCase): """Test all socket servers.""" def setUp(self): - signal.alarm(20) # Kill deadlocks after 20 seconds. + signal_alarm(20) # Kill deadlocks after 20 seconds. self.port_seed = 0 self.test_files = [] @@ -139,7 +78,7 @@ class SocketServerTest(unittest.TestCase): except os.error: pass self.test_files[:] = [] - signal.alarm(0) # Didn't deadlock. + signal_alarm(0) # Didn't deadlock. def pickaddr(self, proto): if proto == socket.AF_INET: @@ -166,29 +105,48 @@ class SocketServerTest(unittest.TestCase): self.test_files.append(fn) return fn + def make_server(self, addr, svrcls, hdlrbase): + class MyServer(svrcls): + def handle_error(self, request, client_address): + self.close_request(request) + self.server_close() + raise - def run_server(self, svrcls, hdlrbase, testfunc): class MyHandler(hdlrbase): def handle(self): line = self.rfile.readline() self.wfile.write(line) - addr = self.pickaddr(svrcls.address_family) + if verbose: print("creating server") + server = MyServer(addr, MyHandler) + self.assertEquals(server.server_address, server.socket.getsockname()) + return server + + def run_server(self, svrcls, hdlrbase, testfunc): + server = self.make_server(self.pickaddr(svrcls.address_family), + svrcls, hdlrbase) + # We had the OS pick a port, so pull the real address out of + # the server. + addr = server.server_address if verbose: print("ADDR =", addr) print("CLASS =", svrcls) - t = ServerThread(addr, svrcls, MyHandler) - if verbose: print("server created") + + t = threading.Thread( + name='%s serving' % svrcls, + target=server.serve_forever, + # Short poll interval to make the test finish quickly. + # Time between requests is short enough that we won't wake + # up spuriously too many times. + kwargs={'poll_interval':0.01}) + t.setDaemon(True) # In case this function raises. t.start() if verbose: print("server running") - t.ready.wait(10) - self.assert_(t.ready.isSet(), - "%s not ready within a reasonable time" % svrcls) - addr = t.addr - for i in range(NREQ): + for i in range(3): if verbose: print("test client", i) testfunc(svrcls.address_family, addr) if verbose: print("waiting for server") + server.shutdown() t.join() if verbose: print("done") @@ -295,4 +253,4 @@ def test_main(): if __name__ == "__main__": test_main() - signal.alarm(3) # Shutdown shouldn't take more than 3 seconds. + signal_alarm(3) # Shutdown shouldn't take more than 3 seconds. diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py index 34bb31a4f7..4e9bfb36b1 100644 --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -368,6 +368,7 @@ else: # we assume the certfile contains both private key and certificate self.certfile = certfile self.active = False + self.active_lock = threading.Lock() self.allow_reuse_address = True def __str__(self): @@ -398,23 +399,32 @@ else: # We want this to run in a thread, so we use a slightly # modified version of "forever". self.active = True - while self.active: + while 1: try: - self.handle_request() + # We need to lock while handling the request. + # Another thread can close the socket after self.active + # has been checked and before the request is handled. + # This causes an exception when using the closed socket. + with self.active_lock: + if not self.active: + break + self.handle_request() except socket.timeout: pass except KeyboardInterrupt: self.server_close() return except: - sys.stdout.write(''.join(traceback.format_exception(*sys.exc_info()))); + sys.stdout.write(''.join(traceback.format_exception(*sys.exc_info()))) + break def server_close(self): # Again, we want this to run in a thread, so we need to override # close to clear the "active" flag, so that serve_forever() will # terminate. - HTTPServer.server_close(self) - self.active = False + with self.active_lock: + HTTPServer.server_close(self) + self.active = False class RootedHTTPRequestHandler(SimpleHTTPRequestHandler): @@ -749,7 +759,7 @@ else: not in cert['subject']): raise test_support.TestFailed( "Missing or invalid 'organizationName' field in certificate subject; " - "should be 'Python Software Foundation'."); + "should be 'Python Software Foundation'.") s.close() finally: server.stop() diff --git a/Lib/test/test_strptime.py b/Lib/test/test_strptime.py index 81f83927eb..ac8905a299 100644 --- a/Lib/test/test_strptime.py +++ b/Lib/test/test_strptime.py @@ -208,11 +208,11 @@ class StrptimeTests(unittest.TestCase): def test_ValueError(self): # Make sure ValueError is raised when match fails or format is bad - self.assertRaises(ValueError, _strptime.strptime, data_string="%d", + self.assertRaises(ValueError, _strptime._strptime_time, data_string="%d", format="%A") for bad_format in ("%", "% ", "%e"): try: - _strptime.strptime("2005", bad_format) + _strptime._strptime_time("2005", bad_format) except ValueError: continue except Exception as err: @@ -223,12 +223,12 @@ class StrptimeTests(unittest.TestCase): def test_unconverteddata(self): # Check ValueError is raised when there is unconverted data - self.assertRaises(ValueError, _strptime.strptime, "10 12", "%m") + self.assertRaises(ValueError, _strptime._strptime_time, "10 12", "%m") def helper(self, directive, position): """Helper fxn in testing.""" strf_output = time.strftime("%" + directive, self.time_tuple) - strp_output = _strptime.strptime(strf_output, "%" + directive) + strp_output = _strptime._strptime_time(strf_output, "%" + directive) self.failUnless(strp_output[position] == self.time_tuple[position], "testing of '%s' directive failed; '%s' -> %s != %s" % (directive, strf_output, strp_output[position], @@ -241,7 +241,7 @@ class StrptimeTests(unittest.TestCase): # Must also make sure %y values are correct for bounds set by Open Group for century, bounds in ((1900, ('69', '99')), (2000, ('00', '68'))): for bound in bounds: - strp_output = _strptime.strptime(bound, '%y') + strp_output = _strptime._strptime_time(bound, '%y') expected_result = century + int(bound) self.failUnless(strp_output[0] == expected_result, "'y' test failed; passed in '%s' " @@ -260,7 +260,7 @@ class StrptimeTests(unittest.TestCase): # Test hour directives self.helper('H', 3) strf_output = time.strftime("%I %p", self.time_tuple) - strp_output = _strptime.strptime(strf_output, "%I %p") + strp_output = _strptime._strptime_time(strf_output, "%I %p") self.failUnless(strp_output[3] == self.time_tuple[3], "testing of '%%I %%p' directive failed; '%s' -> %s != %s" % (strf_output, strp_output[3], self.time_tuple[3])) @@ -273,6 +273,12 @@ class StrptimeTests(unittest.TestCase): # Test second directives self.helper('S', 5) + def test_fraction(self): + import datetime + now = datetime.datetime.now() + tup, frac = _strptime._strptime(str(now), format="%Y-%m-%d %H:%M:%S.%f") + self.assertEqual(frac, now.microsecond) + def test_weekday(self): # Test weekday directives for directive in ('A', 'a', 'w'): @@ -287,16 +293,16 @@ class StrptimeTests(unittest.TestCase): # When gmtime() is used with %Z, entire result of strftime() is empty. # Check for equal timezone names deals with bad locale info when this # occurs; first found in FreeBSD 4.4. - strp_output = _strptime.strptime("UTC", "%Z") + strp_output = _strptime._strptime_time("UTC", "%Z") self.failUnlessEqual(strp_output.tm_isdst, 0) - strp_output = _strptime.strptime("GMT", "%Z") + strp_output = _strptime._strptime_time("GMT", "%Z") self.failUnlessEqual(strp_output.tm_isdst, 0) if sys.platform == "mac": # Timezones don't really work on MacOS9 return time_tuple = time.localtime() strf_output = time.strftime("%Z") #UTC does not have a timezone - strp_output = _strptime.strptime(strf_output, "%Z") + strp_output = _strptime._strptime_time(strf_output, "%Z") locale_time = _strptime.LocaleTime() if time.tzname[0] != time.tzname[1] or not time.daylight: self.failUnless(strp_output[8] == time_tuple[8], @@ -320,7 +326,7 @@ class StrptimeTests(unittest.TestCase): original_daylight = time.daylight time.tzname = (tz_name, tz_name) time.daylight = 1 - tz_value = _strptime.strptime(tz_name, "%Z")[8] + tz_value = _strptime._strptime_time(tz_name, "%Z")[8] self.failUnlessEqual(tz_value, -1, "%s lead to a timezone value of %s instead of -1 when " "time.daylight set to %s and passing in %s" % @@ -347,7 +353,7 @@ class StrptimeTests(unittest.TestCase): def test_percent(self): # Make sure % signs are handled properly strf_output = time.strftime("%m %% %Y", self.time_tuple) - strp_output = _strptime.strptime(strf_output, "%m %% %Y") + strp_output = _strptime._strptime_time(strf_output, "%m %% %Y") self.failUnless(strp_output[0] == self.time_tuple[0] and strp_output[1] == self.time_tuple[1], "handling of percent sign failed") @@ -355,17 +361,17 @@ class StrptimeTests(unittest.TestCase): def test_caseinsensitive(self): # Should handle names case-insensitively. strf_output = time.strftime("%B", self.time_tuple) - self.failUnless(_strptime.strptime(strf_output.upper(), "%B"), + self.failUnless(_strptime._strptime_time(strf_output.upper(), "%B"), "strptime does not handle ALL-CAPS names properly") - self.failUnless(_strptime.strptime(strf_output.lower(), "%B"), + self.failUnless(_strptime._strptime_time(strf_output.lower(), "%B"), "strptime does not handle lowercase names properly") - self.failUnless(_strptime.strptime(strf_output.capitalize(), "%B"), + self.failUnless(_strptime._strptime_time(strf_output.capitalize(), "%B"), "strptime does not handle capword names properly") def test_defaults(self): # Default return value should be (1900, 1, 1, 0, 0, 0, 0, 1, 0) defaults = (1900, 1, 1, 0, 0, 0, 0, 1, -1) - strp_output = _strptime.strptime('1', '%m') + strp_output = _strptime._strptime_time('1', '%m') self.failUnless(strp_output == defaults, "Default values for strptime() are incorrect;" " %s != %s" % (strp_output, defaults)) @@ -377,7 +383,7 @@ class StrptimeTests(unittest.TestCase): # escaped. # Test instigated by bug #796149 . need_escaping = ".^$*+?{}\[]|)(" - self.failUnless(_strptime.strptime(need_escaping, need_escaping)) + self.failUnless(_strptime._strptime_time(need_escaping, need_escaping)) class Strptime12AMPMTests(unittest.TestCase): """Test a _strptime regression in '%I %p' at 12 noon (12 PM)""" @@ -386,8 +392,8 @@ class Strptime12AMPMTests(unittest.TestCase): eq = self.assertEqual eq(time.strptime('12 PM', '%I %p')[3], 12) eq(time.strptime('12 AM', '%I %p')[3], 0) - eq(_strptime.strptime('12 PM', '%I %p')[3], 12) - eq(_strptime.strptime('12 AM', '%I %p')[3], 0) + eq(_strptime._strptime_time('12 PM', '%I %p')[3], 12) + eq(_strptime._strptime_time('12 AM', '%I %p')[3], 0) class JulianTests(unittest.TestCase): @@ -397,7 +403,7 @@ class JulianTests(unittest.TestCase): eq = self.assertEqual for i in range(1, 367): # use 2004, since it is a leap year, we have 366 days - eq(_strptime.strptime('%d 2004' % i, '%j %Y')[7], i) + eq(_strptime._strptime_time('%d 2004' % i, '%j %Y')[7], i) class CalculationTests(unittest.TestCase): """Test that strptime() fills in missing info correctly""" @@ -408,7 +414,7 @@ class CalculationTests(unittest.TestCase): def test_julian_calculation(self): # Make sure that when Julian is missing that it is calculated format_string = "%Y %m %d %H %M %S %w %Z" - result = _strptime.strptime(time.strftime(format_string, self.time_tuple), + result = _strptime._strptime_time(time.strftime(format_string, self.time_tuple), format_string) self.failUnless(result.tm_yday == self.time_tuple.tm_yday, "Calculation of tm_yday failed; %s != %s" % @@ -417,7 +423,7 @@ class CalculationTests(unittest.TestCase): def test_gregorian_calculation(self): # Test that Gregorian date can be calculated from Julian day format_string = "%Y %H %M %S %w %j %Z" - result = _strptime.strptime(time.strftime(format_string, self.time_tuple), + result = _strptime._strptime_time(time.strftime(format_string, self.time_tuple), format_string) self.failUnless(result.tm_year == self.time_tuple.tm_year and result.tm_mon == self.time_tuple.tm_mon and @@ -431,7 +437,7 @@ class CalculationTests(unittest.TestCase): def test_day_of_week_calculation(self): # Test that the day of the week is calculated as needed format_string = "%Y %m %d %H %S %j %Z" - result = _strptime.strptime(time.strftime(format_string, self.time_tuple), + result = _strptime._strptime_time(time.strftime(format_string, self.time_tuple), format_string) self.failUnless(result.tm_wday == self.time_tuple.tm_wday, "Calculation of day of the week failed;" @@ -445,7 +451,7 @@ class CalculationTests(unittest.TestCase): format_string = "%%Y %%%s %%w" % directive dt_date = datetime_date(*ymd_tuple) strp_input = dt_date.strftime(format_string) - strp_output = _strptime.strptime(strp_input, format_string) + strp_output = _strptime._strptime_time(strp_input, format_string) self.failUnless(strp_output[:3] == ymd_tuple, "%s(%s) test failed w/ '%s': %s != %s (%s != %s)" % (test_reason, directive, strp_input, @@ -484,11 +490,11 @@ class CacheTests(unittest.TestCase): def test_time_re_recreation(self): # Make sure cache is recreated when current locale does not match what # cached object was created with. - _strptime.strptime("10", "%d") - _strptime.strptime("2005", "%Y") + _strptime._strptime_time("10", "%d") + _strptime._strptime_time("2005", "%Y") _strptime._TimeRE_cache.locale_time.lang = "Ni" original_time_re = id(_strptime._TimeRE_cache) - _strptime.strptime("10", "%d") + _strptime._strptime_time("10", "%d") self.failIfEqual(original_time_re, id(_strptime._TimeRE_cache)) self.failUnlessEqual(len(_strptime._regex_cache), 1) @@ -502,7 +508,7 @@ class CacheTests(unittest.TestCase): while len(_strptime._regex_cache) <= _strptime._CACHE_MAX_SIZE: _strptime._regex_cache[bogus_key] = None bogus_key += 1 - _strptime.strptime("10", "%d") + _strptime._strptime_time("10", "%d") self.failUnlessEqual(len(_strptime._regex_cache), 1) def test_new_localetime(self): @@ -510,7 +516,7 @@ class CacheTests(unittest.TestCase): # is created. locale_time_id = id(_strptime._TimeRE_cache.locale_time) _strptime._TimeRE_cache.locale_time.lang = "Ni" - _strptime.strptime("10", "%d") + _strptime._strptime_time("10", "%d") self.failIfEqual(locale_time_id, id(_strptime._TimeRE_cache.locale_time)) @@ -522,13 +528,13 @@ class CacheTests(unittest.TestCase): except locale.Error: return try: - _strptime.strptime('10', '%d') + _strptime._strptime_time('10', '%d') # Get id of current cache object. first_time_re_id = id(_strptime._TimeRE_cache) try: # Change the locale and force a recreation of the cache. locale.setlocale(locale.LC_TIME, ('de_DE', 'UTF8')) - _strptime.strptime('10', '%d') + _strptime._strptime_time('10', '%d') # Get the new cache object's id. second_time_re_id = id(_strptime._TimeRE_cache) # They should not be equal. diff --git a/Lib/test/test_struct.py b/Lib/test/test_struct.py index 23358be383..a81555585b 100644 --- a/Lib/test/test_struct.py +++ b/Lib/test/test_struct.py @@ -84,8 +84,8 @@ sz = struct.calcsize('i') if sz * 3 != struct.calcsize('iii'): raise TestFailed('inconsistent sizes') -fmt = 'cbxxxxxxhhhhiillffdt' -fmt3 = '3c3b18x12h6i6l6f3d3t' +fmt = 'cbxxxxxxhhhhiillffd?' +fmt3 = '3c3b18x12h6i6l6f3d3?' sz = struct.calcsize(fmt) sz3 = struct.calcsize(fmt3) if sz * 3 != sz3: @@ -111,7 +111,7 @@ d = 3.1415 t = True for prefix in ('', '@', '<', '>', '=', '!'): - for format in ('xcbhilfdt', 'xcBHILfdt'): + for format in ('xcbhilfd?', 'xcBHILfd?'): format = prefix + format if verbose: print("trying:", format) @@ -160,11 +160,11 @@ tests = [ ('f', -2.0, '\300\000\000\000', '\000\000\000\300', 0), ('d', -2.0, '\300\000\000\000\000\000\000\000', '\000\000\000\000\000\000\000\300', 0), - ('t', 0, '\0', '\0', 0), - ('t', 3, '\1', '\1', 1), - ('t', True, '\1', '\1', 0), - ('t', [], '\0', '\0', 1), - ('t', (1,), '\1', '\1', 1), + ('?', 0, '\0', '\0', 0), + ('?', 3, '\1', '\1', 1), + ('?', True, '\1', '\1', 0), + ('?', [], '\0', '\0', 1), + ('?', (1,), '\1', '\1', 1), ] for fmt, arg, big, lil, asy in tests: @@ -490,7 +490,7 @@ def test_705836(): except OverflowError: pass else: - TestFailed("expected OverflowError") + raise TestFailed("expected OverflowError") test_705836() @@ -646,13 +646,13 @@ def test_bool(): false = (), [], [], '', 0 true = [1], 'test', 5, -1, 0xffffffff+1, 0xffffffff/2 - falseFormat = prefix + 't' * len(false) + falseFormat = prefix + '?' * len(false) if verbose: print('trying bool pack/unpack on', false, 'using format', falseFormat) packedFalse = struct.pack(falseFormat, *false) unpackedFalse = struct.unpack(falseFormat, packedFalse) - trueFormat = prefix + 't' * len(true) + trueFormat = prefix + '?' * len(true) if verbose: print('trying bool pack/unpack on', true, 'using format', trueFormat) packedTrue = struct.pack(trueFormat, *true) @@ -671,10 +671,10 @@ def test_bool(): raise TestFailed('%r did not unpack as false' % t) if prefix and verbose: - print('trying size of bool with format %r' % (prefix+'t')) - packed = struct.pack(prefix+'t', 1) + print('trying size of bool with format %r' % (prefix+'?')) + packed = struct.pack(prefix+'?', 1) - if len(packed) != struct.calcsize(prefix+'t'): + if len(packed) != struct.calcsize(prefix+'?'): raise TestFailed('packed length is not equal to calculated size') if len(packed) != 1 and prefix: @@ -683,7 +683,7 @@ def test_bool(): print('size of bool in native format is %i' % (len(packed))) for c in b'\x01\x7f\xff\x0f\xf0': - if struct.unpack('>t', bytes([c]))[0] is not True: + if struct.unpack('>?', c)[0] is not True: raise TestFailed('%c did not unpack as True' % c) test_bool() diff --git a/Lib/test/test_thread.py b/Lib/test/test_thread.py index 577d4cbc9a..c89c5a147e 100644 --- a/Lib/test/test_thread.py +++ b/Lib/test/test_thread.py @@ -1,160 +1,161 @@ -# Very rudimentary test of thread module - -# Create a bunch of threads, let each do some work, wait until all are done - -from test.test_support import verbose +import os +import unittest import random +from test import test_support import thread import time -mutex = thread.allocate_lock() -rmutex = thread.allocate_lock() # for calls to random -running = 0 -done = thread.allocate_lock() -done.acquire() - -numtasks = 10 - -def task(ident): - global running - rmutex.acquire() - delay = random.random() * numtasks - rmutex.release() - if verbose: - print('task', ident, 'will run for', round(delay, 1), 'sec') - time.sleep(delay) - if verbose: - print('task', ident, 'done') - mutex.acquire() - running = running - 1 - if running == 0: - done.release() - mutex.release() - -next_ident = 0 -def newtask(): - global next_ident, running - mutex.acquire() - next_ident = next_ident + 1 - if verbose: - print('creating task', next_ident) - thread.start_new_thread(task, (next_ident,)) - running = running + 1 - mutex.release() - -for i in range(numtasks): - newtask() - -print('waiting for all tasks to complete') -done.acquire() -print('all tasks done') - -class barrier: - def __init__(self, n): - self.n = n + +NUMTASKS = 10 +NUMTRIPS = 3 + +def verbose_print(arg): + """Helper function for printing out debugging output.""" + if test_support.verbose: + print(arg) + +class BasicThreadTest(unittest.TestCase): + + def setUp(self): + self.done_mutex = thread.allocate_lock() + self.done_mutex.acquire() + self.running_mutex = thread.allocate_lock() + self.random_mutex = thread.allocate_lock() + self.running = 0 + self.next_ident = 0 + + +class ThreadRunningTests(BasicThreadTest): + + def newtask(self): + with self.running_mutex: + self.next_ident += 1 + verbose_print("creating task %s" % self.next_ident) + thread.start_new_thread(self.task, (self.next_ident,)) + self.running += 1 + + def task(self, ident): + with self.random_mutex: + delay = random.random() * NUMTASKS + verbose_print("task %s will run for %s" % (ident, round(delay, 1))) + time.sleep(delay) + verbose_print("task %s done" % ident) + with self.running_mutex: + self.running -= 1 + if self.running == 0: + self.done_mutex.release() + + def test_starting_threads(self): + # Basic test for thread creation. + for i in range(NUMTASKS): + self.newtask() + verbose_print("waiting for tasks to complete...") + self.done_mutex.acquire() + verbose_print("all tasks done") + + def test_stack_size(self): + # Various stack size tests. + self.assertEquals(thread.stack_size(), 0, "intial stack size is not 0") + + thread.stack_size(0) + self.assertEquals(thread.stack_size(), 0, "stack_size not reset to default") + + if os.name not in ("nt", "os2", "posix"): + return + + tss_supported = True + try: + thread.stack_size(4096) + except ValueError: + verbose_print("caught expected ValueError setting " + "stack_size(4096)") + except thread.error: + tss_supported = False + verbose_print("platform does not support changing thread stack " + "size") + + if tss_supported: + fail_msg = "stack_size(%d) failed - should succeed" + for tss in (262144, 0x100000, 0): + thread.stack_size(tss) + self.assertEquals(thread.stack_size(), tss, fail_msg % tss) + verbose_print("successfully set stack_size(%d)" % tss) + + for tss in (262144, 0x100000): + verbose_print("trying stack_size = (%d)" % tss) + self.next_ident = 0 + for i in range(NUMTASKS): + self.newtask() + + verbose_print("waiting for all tasks to complete") + self.done_mutex.acquire() + verbose_print("all tasks done") + + thread.stack_size(0) + + +class Barrier: + def __init__(self, num_threads): + self.num_threads = num_threads self.waiting = 0 - self.checkin = thread.allocate_lock() - self.checkout = thread.allocate_lock() - self.checkout.acquire() + self.checkin_mutex = thread.allocate_lock() + self.checkout_mutex = thread.allocate_lock() + self.checkout_mutex.acquire() def enter(self): - checkin, checkout = self.checkin, self.checkout - - checkin.acquire() + self.checkin_mutex.acquire() self.waiting = self.waiting + 1 - if self.waiting == self.n: - self.waiting = self.n - 1 - checkout.release() + if self.waiting == self.num_threads: + self.waiting = self.num_threads - 1 + self.checkout_mutex.release() return - checkin.release() + self.checkin_mutex.release() - checkout.acquire() + self.checkout_mutex.acquire() self.waiting = self.waiting - 1 if self.waiting == 0: - checkin.release() + self.checkin_mutex.release() return - checkout.release() - -numtrips = 3 -def task2(ident): - global running - for i in range(numtrips): - if ident == 0: - # give it a good chance to enter the next - # barrier before the others are all out - # of the current one - delay = 0.001 - else: - rmutex.acquire() - delay = random.random() * numtasks - rmutex.release() - if verbose: - print('task', ident, 'will run for', round(delay, 1), 'sec') - time.sleep(delay) - if verbose: - print('task', ident, 'entering barrier', i) - bar.enter() - if verbose: - print('task', ident, 'leaving barrier', i) - mutex.acquire() - running -= 1 - # Must release mutex before releasing done, else the main thread can - # exit and set mutex to None as part of global teardown; then - # mutex.release() raises AttributeError. - finished = running == 0 - mutex.release() - if finished: - done.release() - -print('\n*** Barrier Test ***') -if done.acquire(0): - raise ValueError("'done' should have remained acquired") -bar = barrier(numtasks) -running = numtasks -for i in range(numtasks): - thread.start_new_thread(task2, (i,)) -done.acquire() -print('all tasks done') - -# not all platforms support changing thread stack size -print('\n*** Changing thread stack size ***') -if thread.stack_size() != 0: - raise ValueError("initial stack_size not 0") - -thread.stack_size(0) -if thread.stack_size() != 0: - raise ValueError("stack_size not reset to default") - -from os import name as os_name -if os_name in ("nt", "os2", "posix"): - - tss_supported = 1 - try: - thread.stack_size(4096) - except ValueError: - print('caught expected ValueError setting stack_size(4096)') - except thread.error: - tss_supported = 0 - print('platform does not support changing thread stack size') - - if tss_supported: - failed = lambda s, e: s != e - fail_msg = "stack_size(%d) failed - should succeed" - for tss in (262144, 0x100000, 0): - thread.stack_size(tss) - if failed(thread.stack_size(), tss): - raise ValueError(fail_msg % tss) - print('successfully set stack_size(%d)' % tss) - - for tss in (262144, 0x100000): - print('trying stack_size = %d' % tss) - next_ident = 0 - for i in range(numtasks): - newtask() - - print('waiting for all tasks to complete') - done.acquire() - print('all tasks done') - - # reset stack size to default - thread.stack_size(0) + self.checkout_mutex.release() + + +class BarrierTest(BasicThreadTest): + + def test_barrier(self): + self.bar = Barrier(NUMTASKS) + self.running = NUMTASKS + for i in range(NUMTASKS): + thread.start_new_thread(self.task2, (i,)) + verbose_print("waiting for tasks to end") + self.done_mutex.acquire() + verbose_print("tasks done") + + def task2(self, ident): + for i in range(NUMTRIPS): + if ident == 0: + # give it a good chance to enter the next + # barrier before the others are all out + # of the current one + delay = 0.001 + else: + with self.random_mutex: + delay = random.random() * NUMTASKS + verbose_print("task %s will run for %s" % (ident, round(delay, 1))) + time.sleep(delay) + verbose_print("task %s entering %s" % (ident, i)) + self.bar.enter() + verbose_print("task %s leaving barrier" % ident) + with self.running_mutex: + self.running -= 1 + # Must release mutex before releasing done, else the main thread can + # exit and set mutex to None as part of global teardown; then + # mutex.release() raises AttributeError. + finished = self.running == 0 + if finished: + self.done_mutex.release() + +def test_main(): + test_support.run_unittest(ThreadRunningTests, BarrierTest) + +if __name__ == "__main__": + test_main() diff --git a/Lib/test/test_tokenize.py b/Lib/test/test_tokenize.py index e59d9c672c..2ed723b87e 100644 --- a/Lib/test/test_tokenize.py +++ b/Lib/test/test_tokenize.py @@ -1,126 +1,501 @@ -"""Tests for the tokenize module. +doctests = """ +Tests for the tokenize module. -The tests were originally written in the old Python style, where the -test output was compared to a golden file. This docstring represents -the first steps towards rewriting the entire test as a doctest. + >>> import glob, random, sys -The tests can be really simple. Given a small fragment of source -code, print out a table with the tokens. The ENDMARK is omitted for +The tests can be really simple. Given a small fragment of source +code, print out a table with thokens. The ENDMARK is omitted for brevity. ->>> dump_tokens("1 + 1") -NUMBER '1' (1, 0) (1, 1) -OP '+' (1, 2) (1, 3) -NUMBER '1' (1, 4) (1, 5) - -A comment generates a token here, unlike in the parser module. The -comment token is followed by an NL or a NEWLINE token, depending on -whether the line contains the completion of a statement. - ->>> dump_tokens("if False:\\n" -... " # NL\\n" -... " a = False # NEWLINE\\n") -NAME 'if' (1, 0) (1, 2) -NAME 'False' (1, 3) (1, 8) -OP ':' (1, 8) (1, 9) -NEWLINE '\\n' (1, 9) (1, 10) -COMMENT '# NL' (2, 4) (2, 8) -NL '\\n' (2, 8) (2, 9) -INDENT ' ' (3, 0) (3, 4) -NAME 'a' (3, 4) (3, 5) -OP '=' (3, 9) (3, 10) -NAME 'False' (3, 11) (3, 16) -COMMENT '# NEWLINE' (3, 17) (3, 26) -NEWLINE '\\n' (3, 26) (3, 27) -DEDENT '' (4, 0) (4, 0) - -' # Emacs hint - -There will be a bunch more tests of specific source patterns. - -The tokenize module also defines an untokenize function that should -regenerate the original program text from the tokens. - -There are some standard formatting practices that are easy to get right. - ->>> roundtrip("if x == 1:\\n" -... " print(x)\\n") -if x == 1: - print(x) + >>> dump_tokens("1 + 1") + NUMBER '1' (1, 0) (1, 1) + OP '+' (1, 2) (1, 3) + NUMBER '1' (1, 4) (1, 5) + + >>> dump_tokens("if False:\\n" + ... " # NL\\n" + ... " True = False # NEWLINE\\n") + NAME 'if' (1, 0) (1, 2) + NAME 'False' (1, 3) (1, 8) + OP ':' (1, 8) (1, 9) + NEWLINE '\\n' (1, 9) (1, 10) + COMMENT '# NL' (2, 4) (2, 8) + NL '\\n' (2, 8) (2, 9) + INDENT ' ' (3, 0) (3, 4) + NAME 'True' (3, 4) (3, 8) + OP '=' (3, 9) (3, 10) + NAME 'False' (3, 11) (3, 16) + COMMENT '# NEWLINE' (3, 17) (3, 26) + NEWLINE '\\n' (3, 26) (3, 27) + DEDENT '' (4, 0) (4, 0) + + >>> indent_error_file = \""" + ... def k(x): + ... x += 2 + ... x += 5 + ... \""" + >>> for tok in generate_tokens(StringIO(indent_error_file).readline): pass + Traceback (most recent call last): + ... + IndentationError: unindent does not match any outer indentation level + +Test roundtrip for `untokenize`. `f` is an open file or a string. The source +code in f is tokenized, converted back to source code via tokenize.untokenize(), +and tokenized again from the latter. The test fails if the second tokenization +doesn't match the first. + + >>> def roundtrip(f): + ... if isinstance(f, str): f = StringIO(f) + ... token_list = list(generate_tokens(f.readline)) + ... f.close() + ... tokens1 = [tok[:2] for tok in token_list] + ... new_text = untokenize(tokens1) + ... readline = iter(new_text.splitlines(1)).__next__ + ... tokens2 = [tok[:2] for tok in generate_tokens(readline)] + ... return tokens1 == tokens2 + ... + +There are some standard formattig practises that are easy to get right. + + >>> roundtrip("if x == 1:\\n" + ... " print(x)\\n") + True + + >>> roundtrip("# This is a comment\\n# This also") + True Some people use different formatting conventions, which makes -untokenize a little trickier. Note that this test involves trailing -whitespace after the colon. Note that we use hex escapes to make the -two trailing blanks apparent in the expected output. - ->>> roundtrip("if x == 1 : \\n" -... " print(x)\\n") -if x == 1 :\x20\x20 - print(x) - -Comments need to go in the right place. - ->>> roundtrip("if x == 1:\\n" -... " # A comment by itself.\\n" -... " print(x) # Comment here, too.\\n" -... " # Another comment.\\n" -... "after_if = True\\n") -if x == 1: - # A comment by itself. - print(x) # Comment here, too. - # Another comment. -after_if = True - ->>> roundtrip("if (x # The comments need to go in the right place\\n" -... " == 1):\\n" -... " print('x == 1')\\n") -if (x # The comments need to go in the right place - == 1): - print('x == 1') - +untokenize a little trickier. Note that this test involves trailing +whitespace after the colon. Note that we use hex escapes to make the +two trailing blanks apperant in the expected output. + + >>> roundtrip("if x == 1 : \\n" + ... " print(x)\\n") + True + + >>> f = test_support.findfile("tokenize_tests.txt") + >>> roundtrip(open(f)) + True + + >>> roundtrip("if x == 1:\\n" + ... " # A comment by itself.\\n" + ... " print(x) # Comment here, too.\\n" + ... " # Another comment.\\n" + ... "after_if = True\\n") + True + + >>> roundtrip("if (x # The comments need to go in the right place\\n" + ... " == 1):\\n" + ... " print('x==1')\\n") + True + + >>> roundtrip("class Test: # A comment here\\n" + ... " # A comment with weird indent\\n" + ... " after_com = 5\\n" + ... " def x(m): return m*5 # a one liner\\n" + ... " def y(m): # A whitespace after the colon\\n" + ... " return y*4 # 3-space indent\\n") + True + +Some error-handling code + + >>> roundtrip("try: import somemodule\\n" + ... "except ImportError: # comment\\n" + ... " print 'Can not import' # comment2\\n" + ... "else: print 'Loaded'\\n") + True + +Balancing contunuation + + >>> roundtrip("a = (3,4, \\n" + ... "5,6)\\n" + ... "y = [3, 4,\\n" + ... "5]\\n" + ... "z = {'a': 5,\\n" + ... "'b':15, 'c':True}\\n" + ... "x = len(y) + 5 - a[\\n" + ... "3] - a[2]\\n" + ... "+ len(z) - z[\\n" + ... "'b']\\n") + True + +Ordinary integers and binary operators + + >>> dump_tokens("0xff <= 255") + NUMBER '0xff' (1, 0) (1, 4) + OP '<=' (1, 5) (1, 7) + NUMBER '255' (1, 8) (1, 11) + >>> dump_tokens("01234567 > ~0x15") + NUMBER '01234567' (1, 0) (1, 8) + OP '>' (1, 9) (1, 10) + OP '~' (1, 11) (1, 12) + NUMBER '0x15' (1, 12) (1, 16) + >>> dump_tokens("2134568 != 01231515") + NUMBER '2134568' (1, 0) (1, 7) + OP '!=' (1, 8) (1, 10) + NUMBER '01231515' (1, 11) (1, 19) + >>> dump_tokens("(-124561-1) & 0200000000") + OP '(' (1, 0) (1, 1) + OP '-' (1, 1) (1, 2) + NUMBER '124561' (1, 2) (1, 8) + OP '-' (1, 8) (1, 9) + NUMBER '1' (1, 9) (1, 10) + OP ')' (1, 10) (1, 11) + OP '&' (1, 12) (1, 13) + NUMBER '0200000000' (1, 14) (1, 24) + >>> dump_tokens("0xdeadbeef != -1") + NUMBER '0xdeadbeef' (1, 0) (1, 10) + OP '!=' (1, 11) (1, 13) + OP '-' (1, 14) (1, 15) + NUMBER '1' (1, 15) (1, 16) + >>> dump_tokens("0xdeadc0de & 012345") + NUMBER '0xdeadc0de' (1, 0) (1, 10) + OP '&' (1, 11) (1, 12) + NUMBER '012345' (1, 13) (1, 19) + >>> dump_tokens("0xFF & 0x15 | 1234") + NUMBER '0xFF' (1, 0) (1, 4) + OP '&' (1, 5) (1, 6) + NUMBER '0x15' (1, 7) (1, 11) + OP '|' (1, 12) (1, 13) + NUMBER '1234' (1, 14) (1, 18) + +Long integers + + >>> dump_tokens("x = 0L") + NAME 'x' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + NUMBER '0L' (1, 4) (1, 6) + >>> dump_tokens("x = 0xfffffffffff") + NAME 'x' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + NUMBER '0xffffffffff (1, 4) (1, 17) + >>> dump_tokens("x = 123141242151251616110l") + NAME 'x' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + NUMBER '123141242151 (1, 4) (1, 26) + >>> dump_tokens("x = -15921590215012591L") + NAME 'x' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + OP '-' (1, 4) (1, 5) + NUMBER '159215902150 (1, 5) (1, 23) + +Floating point numbers + + >>> dump_tokens("x = 3.14159") + NAME 'x' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + NUMBER '3.14159' (1, 4) (1, 11) + >>> dump_tokens("x = 314159.") + NAME 'x' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + NUMBER '314159.' (1, 4) (1, 11) + >>> dump_tokens("x = .314159") + NAME 'x' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + NUMBER '.314159' (1, 4) (1, 11) + >>> dump_tokens("x = 3e14159") + NAME 'x' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + NUMBER '3e14159' (1, 4) (1, 11) + >>> dump_tokens("x = 3E123") + NAME 'x' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + NUMBER '3E123' (1, 4) (1, 9) + >>> dump_tokens("x+y = 3e-1230") + NAME 'x' (1, 0) (1, 1) + OP '+' (1, 1) (1, 2) + NAME 'y' (1, 2) (1, 3) + OP '=' (1, 4) (1, 5) + NUMBER '3e-1230' (1, 6) (1, 13) + >>> dump_tokens("x = 3.14e159") + NAME 'x' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + NUMBER '3.14e159' (1, 4) (1, 12) + +String literals + + >>> dump_tokens("x = ''; y = \\\"\\\"") + NAME 'x' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + STRING "''" (1, 4) (1, 6) + OP ';' (1, 6) (1, 7) + NAME 'y' (1, 8) (1, 9) + OP '=' (1, 10) (1, 11) + STRING '""' (1, 12) (1, 14) + >>> dump_tokens("x = '\\\"'; y = \\\"'\\\"") + NAME 'x' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + STRING '\\'"\\'' (1, 4) (1, 7) + OP ';' (1, 7) (1, 8) + NAME 'y' (1, 9) (1, 10) + OP '=' (1, 11) (1, 12) + STRING '"\\'"' (1, 13) (1, 16) + >>> dump_tokens("x = \\\"doesn't \\\"shrink\\\", does it\\\"") + NAME 'x' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + STRING '"doesn\\'t "' (1, 4) (1, 14) + NAME 'shrink' (1, 14) (1, 20) + STRING '", does it"' (1, 20) (1, 31) + >>> dump_tokens("x = u'abc' + U'ABC'") + NAME 'x' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + STRING "u'abc'" (1, 4) (1, 10) + OP '+' (1, 11) (1, 12) + STRING "U'ABC'" (1, 13) (1, 19) + >>> dump_tokens('y = "ABC" + "ABC"') + NAME 'y' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + STRING '"ABC"' (1, 4) (1, 10) + OP '+' (1, 11) (1, 12) + STRING '"ABC"' (1, 13) (1, 19) + >>> dump_tokens("x = r'abc' + r'ABC' + R'ABC' + R'ABC'") + NAME 'x' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + STRING "r'abc'" (1, 4) (1, 11) + OP '+' (1, 12) (1, 13) + STRING "r'ABC'" (1, 14) (1, 21) + OP '+' (1, 22) (1, 23) + STRING "R'ABC'" (1, 24) (1, 31) + OP '+' (1, 32) (1, 33) + STRING "R'ABC'" (1, 34) (1, 41) + >>> dump_tokens('y = r"abc" + r"ABC" + R"ABC" + R"ABC"') + NAME 'y' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + STRING 'r"abc"' (1, 4) (1, 11) + OP '+' (1, 12) (1, 13) + STRING 'r"ABC"' (1, 14) (1, 21) + OP '+' (1, 22) (1, 23) + STRING 'R"ABC"' (1, 24) (1, 31) + OP '+' (1, 32) (1, 33) + STRING 'R"ABC"' (1, 34) (1, 41) + +Operators + + >>> dump_tokens("def d22(a, b, c=2, d=2, *k): pass") + NAME 'def' (1, 0) (1, 3) + NAME 'd22' (1, 4) (1, 7) + OP '(' (1, 7) (1, 8) + NAME 'a' (1, 8) (1, 9) + OP ',' (1, 9) (1, 10) + NAME 'b' (1, 11) (1, 12) + OP ',' (1, 12) (1, 13) + NAME 'c' (1, 14) (1, 15) + OP '=' (1, 15) (1, 16) + NUMBER '2' (1, 16) (1, 17) + OP ',' (1, 17) (1, 18) + NAME 'd' (1, 19) (1, 20) + OP '=' (1, 20) (1, 21) + NUMBER '2' (1, 21) (1, 22) + OP ',' (1, 22) (1, 23) + OP '*' (1, 24) (1, 25) + NAME 'k' (1, 25) (1, 26) + OP ')' (1, 26) (1, 27) + OP ':' (1, 27) (1, 28) + NAME 'pass' (1, 29) (1, 33) + >>> dump_tokens("def d01v_(a=1, *k, **w): pass") + NAME 'def' (1, 0) (1, 3) + NAME 'd01v_' (1, 4) (1, 9) + OP '(' (1, 9) (1, 10) + NAME 'a' (1, 10) (1, 11) + OP '=' (1, 11) (1, 12) + NUMBER '1' (1, 12) (1, 13) + OP ',' (1, 13) (1, 14) + OP '*' (1, 15) (1, 16) + NAME 'k' (1, 16) (1, 17) + OP ',' (1, 17) (1, 18) + OP '**' (1, 19) (1, 21) + NAME 'w' (1, 21) (1, 22) + OP ')' (1, 22) (1, 23) + OP ':' (1, 23) (1, 24) + NAME 'pass' (1, 25) (1, 29) + +Comparison + + >>> dump_tokens("if 1 < 1 > 1 == 1 >= 5 <= 0x15 <= 0x12 != " + + ... "1 and 5 in 1 not in 1 is 1 or 5 is not 1: pass") + NAME 'if' (1, 0) (1, 2) + NUMBER '1' (1, 3) (1, 4) + OP '<' (1, 5) (1, 6) + NUMBER '1' (1, 7) (1, 8) + OP '>' (1, 9) (1, 10) + NUMBER '1' (1, 11) (1, 12) + OP '==' (1, 13) (1, 15) + NUMBER '1' (1, 16) (1, 17) + OP '>=' (1, 18) (1, 20) + NUMBER '5' (1, 21) (1, 22) + OP '<=' (1, 23) (1, 25) + NUMBER '0x15' (1, 26) (1, 30) + OP '<=' (1, 31) (1, 33) + NUMBER '0x12' (1, 34) (1, 38) + OP '!=' (1, 39) (1, 41) + NUMBER '1' (1, 42) (1, 43) + NAME 'and' (1, 44) (1, 47) + NUMBER '5' (1, 48) (1, 49) + NAME 'in' (1, 50) (1, 52) + NUMBER '1' (1, 53) (1, 54) + NAME 'not' (1, 55) (1, 58) + NAME 'in' (1, 59) (1, 61) + NUMBER '1' (1, 62) (1, 63) + NAME 'is' (1, 64) (1, 66) + NUMBER '1' (1, 67) (1, 68) + NAME 'or' (1, 69) (1, 71) + NUMBER '5' (1, 72) (1, 73) + NAME 'is' (1, 74) (1, 76) + NAME 'not' (1, 77) (1, 80) + NUMBER '1' (1, 81) (1, 82) + OP ':' (1, 82) (1, 83) + NAME 'pass' (1, 84) (1, 88) + +Shift + + >>> dump_tokens("x = 1 << 1 >> 5") + NAME 'x' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + NUMBER '1' (1, 4) (1, 5) + OP '<<' (1, 6) (1, 8) + NUMBER '1' (1, 9) (1, 10) + OP '>>' (1, 11) (1, 13) + NUMBER '5' (1, 14) (1, 15) + +Additive + + >>> dump_tokens("x = 1 - y + 15 - 01 + 0x124 + z + a[5]") + NAME 'x' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + NUMBER '1' (1, 4) (1, 5) + OP '-' (1, 6) (1, 7) + NAME 'y' (1, 8) (1, 9) + OP '+' (1, 10) (1, 11) + NUMBER '15' (1, 12) (1, 14) + OP '-' (1, 15) (1, 16) + NUMBER '01' (1, 17) (1, 19) + OP '+' (1, 20) (1, 21) + NUMBER '0x124' (1, 22) (1, 27) + OP '+' (1, 28) (1, 29) + NAME 'z' (1, 30) (1, 31) + OP '+' (1, 32) (1, 33) + NAME 'a' (1, 34) (1, 35) + OP '[' (1, 35) (1, 36) + NUMBER '5' (1, 36) (1, 37) + OP ']' (1, 37) (1, 38) + +Multiplicative + + >>> dump_tokens("x = 1//1*1/5*12%0x12") + NAME 'x' (1, 0) (1, 1) + OP '=' (1, 2) (1, 3) + NUMBER '1' (1, 4) (1, 5) + OP '//' (1, 5) (1, 7) + NUMBER '1' (1, 7) (1, 8) + OP '*' (1, 8) (1, 9) + NUMBER '1' (1, 9) (1, 10) + OP '/' (1, 10) (1, 11) + NUMBER '5' (1, 11) (1, 12) + OP '*' (1, 12) (1, 13) + NUMBER '12' (1, 13) (1, 15) + OP '%' (1, 15) (1, 16) + NUMBER '0x12' (1, 16) (1, 20) + +Unary + + >>> dump_tokens("~1 ^ 1 & 1 |1 ^ -1") + OP '~' (1, 0) (1, 1) + NUMBER '1' (1, 1) (1, 2) + OP '^' (1, 3) (1, 4) + NUMBER '1' (1, 5) (1, 6) + OP '&' (1, 7) (1, 8) + NUMBER '1' (1, 9) (1, 10) + OP '|' (1, 11) (1, 12) + NUMBER '1' (1, 12) (1, 13) + OP '^' (1, 14) (1, 15) + OP '-' (1, 16) (1, 17) + NUMBER '1' (1, 17) (1, 18) + >>> dump_tokens("-1*1/1+1*1//1 - ---1**1") + OP '-' (1, 0) (1, 1) + NUMBER '1' (1, 1) (1, 2) + OP '*' (1, 2) (1, 3) + NUMBER '1' (1, 3) (1, 4) + OP '/' (1, 4) (1, 5) + NUMBER '1' (1, 5) (1, 6) + OP '+' (1, 6) (1, 7) + NUMBER '1' (1, 7) (1, 8) + OP '*' (1, 8) (1, 9) + NUMBER '1' (1, 9) (1, 10) + OP '//' (1, 10) (1, 12) + NUMBER '1' (1, 12) (1, 13) + OP '-' (1, 14) (1, 15) + OP '-' (1, 16) (1, 17) + OP '-' (1, 17) (1, 18) + OP '-' (1, 18) (1, 19) + NUMBER '1' (1, 19) (1, 20) + OP '**' (1, 20) (1, 22) + NUMBER '1' (1, 22) (1, 23) + +Selector + + >>> dump_tokens("import sys, time\\nx = sys.modules['time'].time()") + NAME 'import' (1, 0) (1, 6) + NAME 'sys' (1, 7) (1, 10) + OP ',' (1, 10) (1, 11) + NAME 'time' (1, 12) (1, 16) + NEWLINE '\\n' (1, 16) (1, 17) + NAME 'x' (2, 0) (2, 1) + OP '=' (2, 2) (2, 3) + NAME 'sys' (2, 4) (2, 7) + OP '.' (2, 7) (2, 8) + NAME 'modules' (2, 8) (2, 15) + OP '[' (2, 15) (2, 16) + STRING "'time'" (2, 16) (2, 22) + OP ']' (2, 22) (2, 23) + OP '.' (2, 23) (2, 24) + NAME 'time' (2, 24) (2, 28) + OP '(' (2, 28) (2, 29) + OP ')' (2, 29) (2, 30) + +Methods + + >>> dump_tokens("@staticmethod\\ndef foo(x,y): pass") + OP '@' (1, 0) (1, 1) + NAME 'staticmethod (1, 1) (1, 13) + NEWLINE '\\n' (1, 13) (1, 14) + NAME 'def' (2, 0) (2, 3) + NAME 'foo' (2, 4) (2, 7) + OP '(' (2, 7) (2, 8) + NAME 'x' (2, 8) (2, 9) + OP ',' (2, 9) (2, 10) + NAME 'y' (2, 10) (2, 11) + OP ')' (2, 11) (2, 12) + OP ':' (2, 12) (2, 13) + NAME 'pass' (2, 14) (2, 18) + +Backslash means line continuation, except for comments + + >>> roundtrip("x=1+\\\\n" + ... "1\\n" + ... "# This is a comment\\\\n" + ... "# This also\\n") + True + >>> roundtrip("# Comment \\\\nx = 0") + True + + >>> + >>> tempdir = os.path.dirname(f) or os.curdir + >>> testfiles = glob.glob(os.path.join(tempdir, "test*.py")) + >>> if not test_support.is_resource_enabled("compiler"): + ... testfiles = random.sample(testfiles, 10) + ... + >>> for testfile in testfiles: + ... if not roundtrip(open(testfile)): break + ... else: True + True """ -# ' Emacs hint -import os, glob, random, time, sys -import re +from test import test_support +from tokenize import (tokenize, untokenize, generate_tokens, NUMBER, NAME, OP, + STRING, ENDMARKER, tok_name) from io import StringIO -from test.test_support import (verbose, findfile, is_resource_enabled, - TestFailed) -from tokenize import (tokenize, generate_tokens, untokenize, tok_name, - ENDMARKER, NUMBER, NAME, OP, STRING, COMMENT) - -# How much time in seconds can pass before we print a 'Still working' message. -_PRINT_WORKING_MSG_INTERVAL = 5 * 60 - -# Test roundtrip for `untokenize`. `f` is a file path. The source code in f -# is tokenized, converted back to source code via tokenize.untokenize(), -# and tokenized again from the latter. The test fails if the second -# tokenization doesn't match the first. -def test_roundtrip(f): - ## print('Testing:', f) - # Get the encoding first - fobj = open(f, encoding="latin-1") - first2lines = fobj.readline() + fobj.readline() - fobj.close() - m = re.search(r"coding:\s*(\S+)", first2lines) - if m: - encoding = m.group(1) - ## print(" coding:", encoding) - else: - encoding = "utf-8" - fobj = open(f, encoding=encoding) - try: - fulltok = list(generate_tokens(fobj.readline)) - finally: - fobj.close() - - t1 = [tok[:2] for tok in fulltok] - newtext = untokenize(t1) - readline = iter(newtext.splitlines(1)).__next__ - t2 = [tok[:2] for tok in generate_tokens(readline)] - if t1 != t2: - raise TestFailed("untokenize() roundtrip failed for %r" % f) +import os def dump_tokens(s): """Print out the tokens in s in a table format. @@ -132,7 +507,7 @@ def dump_tokens(s): if type == ENDMARKER: break type = tok_name[type] - print("%(type)-10.10s %(token)-13.13r %(start)s %(end)s" % locals()) + print("%(type)-10.10s %(token)-13.13r %(start)s %(end)s" % locals()) def roundtrip(s): f = StringIO(s) @@ -177,82 +552,12 @@ def decistmt(s): result.append((toknum, tokval)) return untokenize(result) -def test_main(): - if verbose: - print('starting...') - - next_time = time.time() + _PRINT_WORKING_MSG_INTERVAL - - # Validate the tokenize_tests.txt file. - # This makes sure it compiles, and displays any errors in it. - f = open(findfile('tokenize_tests.txt')) - sf = f.read() - f.close() - cf = compile(sf, 'tokenize_tests.txt', 'exec') - - # This displays the tokenization of tokenize_tests.py to stdout, and - # regrtest.py checks that this equals the expected output (in the - # test/output/ directory). - f = open(findfile('tokenize_tests.txt')) - tokenize(f.readline) - f.close() - - # Now run test_roundtrip() over test_tokenize.py too, and over all - # (if the "compiler" resource is enabled) or a small random sample (if - # "compiler" is not enabled) of the test*.py files. - f = findfile('test_tokenize.py') - if verbose: - print(' round trip: ', f, file=sys.__stdout__) - test_roundtrip(f) - - testdir = os.path.dirname(f) or os.curdir - testfiles = glob.glob(testdir + os.sep + 'test*.py') - if not is_resource_enabled('compiler'): - testfiles = random.sample(testfiles, 10) - - for f in testfiles: - # Print still working message since this test can be really slow - if verbose: - print(' round trip: ', f, file=sys.__stdout__) - if next_time <= time.time(): - next_time = time.time() + _PRINT_WORKING_MSG_INTERVAL - print(' test_main still working, be patient...', file=sys.__stdout__) - sys.__stdout__.flush() - - test_roundtrip(f) - - # Test detecton of IndentationError. - sampleBadText = """\ -def foo(): - bar - baz -""" - - try: - for tok in generate_tokens(StringIO(sampleBadText).readline): - pass - except IndentationError: - pass - else: - raise TestFailed("Did not detect IndentationError:") - - # Run the doctests in this module. - from test import test_tokenize # i.e., this module - from test.test_support import run_doctest - run_doctest(test_tokenize, verbose) - if verbose: - print('finished') +__test__ = {"doctests" : doctests, 'decistmt': decistmt} -def test_rarrow(): - """ - This function exists solely to test the tokenization of the RARROW - operator. - - >>> tokenize(iter(['->']).__next__) #doctest: +NORMALIZE_WHITESPACE - 1,0-1,2:\tOP\t'->' - 2,0-2,0:\tENDMARKER\t'' - """ +def test_main(): + from test import test_tokenize + test_support.run_doctest(test_tokenize, True) if __name__ == "__main__": test_main() diff --git a/Lib/test/test_winsound.py b/Lib/test/test_winsound.py index 32b49d3081..0d6ddf9ca0 100644 --- a/Lib/test/test_winsound.py +++ b/Lib/test/test_winsound.py @@ -9,6 +9,13 @@ import subprocess class BeepTest(unittest.TestCase): + # As with PlaySoundTest, incorporate the _have_soundcard() check + # into our test methods. If there's no audio device present, + # winsound.Beep returns 0 and GetLastError() returns 127, which + # is: ERROR_PROC_NOT_FOUND ("The specified procedure could not + # be found"). (FWIW, virtual/Hyper-V systems fall under this + # scenario as they have no sound devices whatsoever (not even + # a legacy Beep device).) def test_errors(self): self.assertRaises(TypeError, winsound.Beep) @@ -16,12 +23,17 @@ class BeepTest(unittest.TestCase): self.assertRaises(ValueError, winsound.Beep, 32768, 75) def test_extremes(self): - winsound.Beep(37, 75) - winsound.Beep(32767, 75) + if _have_soundcard(): + winsound.Beep(37, 75) + winsound.Beep(32767, 75) + else: + self.assertRaises(RuntimeError, winsound.Beep, 37, 75) + self.assertRaises(RuntimeError, winsound.Beep, 32767, 75) def test_increasingfrequency(self): - for i in range(100, 2000, 100): - winsound.Beep(i, 75) + if _have_soundcard(): + for i in range(100, 2000, 100): + winsound.Beep(i, 75) class MessageBeepTest(unittest.TestCase): @@ -179,6 +179,7 @@ Luke Dunstan Andy Dustman Gary Duzan Eugene Dvurechenski +Josip Dzolonga Maxim Dzumanenko Hans Eckardt Grant Edwards @@ -430,6 +431,7 @@ Steve Majewski Grzegorz Makarewicz Ken Manheimer Vladimir Marangozov +David Marek Doug Marien Alex Martelli Anthony Martin @@ -561,6 +563,7 @@ Mark Roberts Andy Robinson Jim Robinson Kevin Rodgers +Giampaolo Rodola Mike Romberg Case Roole Timothy Roscoe diff --git a/Misc/build.sh b/Misc/build.sh index 70034c461c..3a0e982829 100755 --- a/Misc/build.sh +++ b/Misc/build.sh @@ -67,7 +67,7 @@ REFLOG="build/reflog.txt.out" # Note: test_XXX (none currently) really leak, but are disabled # so we don't send spam. Any test which really leaks should only # be listed here if there are also test cases under Lib/test/leakers. -LEAKY_TESTS="test_(asynchat|cmd_line|popen2|socket|sys|threadsignals|urllib2_localnet)" +LEAKY_TESTS="test_(asynchat|cmd_line|popen2|socket|smtplib|sys|threadsignals|urllib2_localnet)" # These tests always fail, so skip them so we don't get false positives. _ALWAYS_SKIP="" @@ -95,7 +95,7 @@ update_status() { place_summary_first() { testf=$1 sed -n '/^[0-9][0-9]* tests OK\./,$p' < $testf \ - | egrep -v '\[[0-9]+ refs\]' > $testf.tmp + | egrep -v '\[[0-9]+ refs\]' > $testf.tmp echo "" >> $testf.tmp cat $testf >> $testf.tmp mv $testf.tmp $testf @@ -105,7 +105,7 @@ count_failures () { testf=$1 n=`grep -ic " failed:" $testf` if [ $n -eq 1 ] ; then - n=`grep " failed:" $testf | sed -e 's/ .*//'` + n=`grep " failed:" $testf | sed -e 's/ .*//'` fi echo $n } @@ -117,17 +117,17 @@ mail_on_failure() { if [ "$FAILURE_CC" != "" ]; then dest="$dest -c $FAILURE_CC" fi - if [ "x$3" != "x" ] ; then - (echo "More important issues:" - echo "----------------------" - egrep -v "$3" < $2 - echo "" - echo "Less important issues:" - echo "----------------------" - egrep "$3" < $2) + if [ "x$3" != "x" ] ; then + (echo "More important issues:" + echo "----------------------" + egrep -v "$3" < $2 + echo "" + echo "Less important issues:" + echo "----------------------" + egrep "$3" < $2) else - cat $2 - fi | mutt -s "$FAILURE_SUBJECT $1 ($NUM_FAILURES)" $dest + cat $2 + fi | mutt -s "$FAILURE_SUBJECT $1 ($NUM_FAILURES)" $dest fi } @@ -204,7 +204,7 @@ if [ $err = 0 -a "$BUILD_DISABLED" != "yes" ]; then ## make and run basic tests F=make-test.out start=`current_time` - $PYTHON $REGRTEST_ARGS $ALWAYS_SKIP >& build/$F + $PYTHON $REGRTEST_ARGS -u urlfetch >& build/$F NUM_FAILURES=`count_failures build/$F` place_summary_first build/$F update_status "Testing basics ($NUM_FAILURES failures)" "$F" $start @@ -212,7 +212,7 @@ if [ $err = 0 -a "$BUILD_DISABLED" != "yes" ]; then F=make-test-opt.out start=`current_time` - $PYTHON -O $REGRTEST_ARGS $ALWAYS_SKIP >& build/$F + $PYTHON -O $REGRTEST_ARGS -u urlfetch >& build/$F NUM_FAILURES=`count_failures build/$F` place_summary_first build/$F update_status "Testing opt ($NUM_FAILURES failures)" "$F" $start @@ -223,8 +223,8 @@ if [ $err = 0 -a "$BUILD_DISABLED" != "yes" ]; then start=`current_time` ## ensure that the reflog exists so the grep doesn't fail touch $REFLOG - $PYTHON $REGRTEST_ARGS -R 4:3:$REFLOG -u network $LEAKY_SKIPS >& build/$F - LEAK_PAT="($LEAKY_TESTS|sum=0)" + $PYTHON $REGRTEST_ARGS -R 4:3:$REFLOG -u network,urlfetch $LEAKY_SKIPS >& build/$F + LEAK_PAT="($LEAKY_TESTS|sum=0)" NUM_FAILURES=`egrep -vc "$LEAK_PAT" $REFLOG` place_summary_first build/$F update_status "Testing refleaks ($NUM_FAILURES failures)" "$F" $start diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c index e4fa612a06..d39cf4b4ff 100644 --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -203,7 +203,7 @@ static PyTypeObject DictRemover_Type = { 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ - PyType_GenericNew, /* tp_new */ + 0, /* tp_new */ 0, /* tp_free */ }; @@ -1242,7 +1242,7 @@ _type_ attribute. */ -static char *SIMPLE_TYPE_CHARS = "cbBhHiIlLdfuzZqQPXOvtg"; +static char *SIMPLE_TYPE_CHARS = "cbBhHiIlLdfuzZqQPXOv?g"; static PyObject * c_wchar_p_from_param(PyObject *type, PyObject *value) diff --git a/Modules/_ctypes/_ctypes_test.c b/Modules/_ctypes/_ctypes_test.c index 63fb580fe1..ce83df7548 100644 --- a/Modules/_ctypes/_ctypes_test.c +++ b/Modules/_ctypes/_ctypes_test.c @@ -12,6 +12,15 @@ /* some functions handy for testing */ +EXPORT(void)testfunc_array(int values[4]) +{ + printf("testfunc_array %d %d %d %d\n", + values[0], + values[1], + values[2], + values[3]); +} + EXPORT(long double)testfunc_Ddd(double a, double b) { long double result = (long double)(a * b); diff --git a/Modules/_ctypes/cfield.c b/Modules/_ctypes/cfield.c index f7c78f3546..4e025635b3 100644 --- a/Modules/_ctypes/cfield.c +++ b/Modules/_ctypes/cfield.c @@ -716,7 +716,7 @@ vBOOL_get(void *ptr, Py_ssize_t size) #endif static PyObject * -t_set(void *ptr, PyObject *value, Py_ssize_t size) +bool_set(void *ptr, PyObject *value, Py_ssize_t size) { switch (PyObject_IsTrue(value)) { case -1: @@ -731,7 +731,7 @@ t_set(void *ptr, PyObject *value, Py_ssize_t size) } static PyObject * -t_get(void *ptr, Py_ssize_t size) +bool_get(void *ptr, Py_ssize_t size) { return PyBool_FromLong((long)*(BOOL_TYPE *)ptr); } @@ -1665,15 +1665,15 @@ static struct fielddesc formattable[] = { { 'v', vBOOL_set, vBOOL_get, &ffi_type_sshort}, #endif #if SIZEOF__BOOL == 1 - { 't', t_set, t_get, &ffi_type_uchar}, /* Also fallback for no native _Bool support */ + { '?', bool_set, bool_get, &ffi_type_uchar}, /* Also fallback for no native _Bool support */ #elif SIZEOF__BOOL == SIZEOF_SHORT - { 't', t_set, t_get, &ffi_type_ushort}, + { '?', bool_set, bool_get, &ffi_type_ushort}, #elif SIZEOF__BOOL == SIZEOF_INT - { 't', t_set, t_get, &ffi_type_uint, I_set_sw, I_get_sw}, + { '?', bool_set, bool_get, &ffi_type_uint, I_set_sw, I_get_sw}, #elif SIZEOF__BOOL == SIZEOF_LONG - { 't', t_set, t_get, &ffi_type_ulong, L_set_sw, L_get_sw}, + { '?', bool_set, bool_get, &ffi_type_ulong, L_set_sw, L_get_sw}, #elif SIZEOF__BOOL == SIZEOF_LONG_LONG - { 't', t_set, t_get, &ffi_type_ulong, Q_set_sw, Q_get_sw}, + { '?', bool_set, bool_get, &ffi_type_ulong, Q_set_sw, Q_get_sw}, #endif /* SIZEOF__BOOL */ { 'O', O_set, O_get, &ffi_type_pointer}, { 0, NULL, NULL, NULL}, diff --git a/Modules/_heapqmodule.c b/Modules/_heapqmodule.c index 2ae1f03910..97ccd86e4d 100644 --- a/Modules/_heapqmodule.c +++ b/Modules/_heapqmodule.c @@ -196,6 +196,48 @@ this routine unless written as part of a conditional replacement:\n\n\ item = heapreplace(heap, item)\n"); static PyObject * +heappushpop(PyObject *self, PyObject *args) +{ + PyObject *heap, *item, *returnitem; + int cmp; + + if (!PyArg_UnpackTuple(args, "heappushpop", 2, 2, &heap, &item)) + return NULL; + + if (!PyList_Check(heap)) { + PyErr_SetString(PyExc_TypeError, "heap argument must be a list"); + return NULL; + } + + if (PyList_GET_SIZE(heap) < 1) { + Py_INCREF(item); + return item; + } + + cmp = PyObject_RichCompareBool(item, PyList_GET_ITEM(heap, 0), Py_LE); + if (cmp == -1) + return NULL; + if (cmp == 1) { + Py_INCREF(item); + return item; + } + + returnitem = PyList_GET_ITEM(heap, 0); + Py_INCREF(item); + PyList_SET_ITEM(heap, 0, item); + if (_siftup((PyListObject *)heap, 0) == -1) { + Py_DECREF(returnitem); + return NULL; + } + return returnitem; +} + +PyDoc_STRVAR(heappushpop_doc, +"Push item on the heap, then pop and return the smallest item\n\ +from the heap. The combined action runs more efficiently than\n\ +heappush() followed by a separate call to heappop()."); + +static PyObject * heapify(PyObject *self, PyObject *heap) { Py_ssize_t i, n; @@ -468,6 +510,8 @@ Equivalent to: sorted(iterable)[:n]\n"); static PyMethodDef heapq_methods[] = { {"heappush", (PyCFunction)heappush, METH_VARARGS, heappush_doc}, + {"heappushpop", (PyCFunction)heappushpop, + METH_VARARGS, heappushpop_doc}, {"heappop", (PyCFunction)heappop, METH_O, heappop_doc}, {"heapreplace", (PyCFunction)heapreplace, diff --git a/Modules/_struct.c b/Modules/_struct.c index 28f67e97c4..13ffee7e4a 100644 --- a/Modules/_struct.c +++ b/Modules/_struct.c @@ -797,7 +797,7 @@ static formatdef native_table[] = { {'q', sizeof(PY_LONG_LONG), LONG_LONG_ALIGN, nu_longlong, np_longlong}, {'Q', sizeof(PY_LONG_LONG), LONG_LONG_ALIGN, nu_ulonglong,np_ulonglong}, #endif - {'t', sizeof(BOOL_TYPE), BOOL_ALIGN, nu_bool, np_bool}, + {'?', sizeof(BOOL_TYPE), BOOL_ALIGN, nu_bool, np_bool}, {'f', sizeof(float), FLOAT_ALIGN, nu_float, np_float}, {'d', sizeof(double), DOUBLE_ALIGN, nu_double, np_double}, {'P', sizeof(void *), VOID_P_ALIGN, nu_void_p, np_void_p}, @@ -1034,7 +1034,7 @@ static formatdef bigendian_table[] = { {'L', 4, 0, bu_uint, bp_uint}, {'q', 8, 0, bu_longlong, bp_longlong}, {'Q', 8, 0, bu_ulonglong, bp_ulonglong}, - {'t', 1, 0, bu_bool, bp_bool}, + {'?', 1, 0, bu_bool, bp_bool}, {'f', 4, 0, bu_float, bp_float}, {'d', 8, 0, bu_double, bp_double}, {0} @@ -1253,7 +1253,7 @@ static formatdef lilendian_table[] = { {'L', 4, 0, lu_uint, lp_uint}, {'q', 8, 0, lu_longlong, lp_longlong}, {'Q', 8, 0, lu_ulonglong, lp_ulonglong}, - {'t', 1, 0, bu_bool, bp_bool}, /* Std rep not endian dep, + {'?', 1, 0, bu_bool, bp_bool}, /* Std rep not endian dep, but potentially different from native rep -- reuse bx_bool funcs. */ {'f', 4, 0, lu_float, lp_float}, {'d', 8, 0, lu_double, lp_double}, diff --git a/Modules/datetimemodule.c b/Modules/datetimemodule.c index 27c404f7d9..798865d6bc 100644 --- a/Modules/datetimemodule.c +++ b/Modules/datetimemodule.c @@ -1170,10 +1170,24 @@ make_Zreplacement(PyObject *object, PyObject *tzinfoarg) return NULL; } +static PyObject * +make_freplacement(PyObject *object) +{ + char freplacement[7]; + if (PyTime_Check(object)) + sprintf(freplacement, "%06d", TIME_GET_MICROSECOND(object)); + else if (PyDateTime_Check(object)) + sprintf(freplacement, "%06d", DATE_GET_MICROSECOND(object)); + else + sprintf(freplacement, "%06d", 0); + + return PyString_FromStringAndSize(freplacement, strlen(freplacement)); +} + /* I sure don't want to reproduce the strftime code from the time module, * so this imports the module and calls it. All the hair is due to - * giving special meanings to the %z and %Z format codes via a preprocessing - * step on the format string. + * giving special meanings to the %z, %Z and %f format codes via a + * preprocessing step on the format string. * tzinfoarg is the argument to pass to the object's tzinfo method, if * needed. */ @@ -1185,6 +1199,7 @@ wrap_strftime(PyObject *object, PyObject *format, PyObject *timetuple, PyObject *zreplacement = NULL; /* py string, replacement for %z */ PyObject *Zreplacement = NULL; /* py string, replacement for %Z */ + PyObject *freplacement = NULL; /* py string, replacement for %f */ const char *pin;/* pointer to next char in input format */ Py_ssize_t flen;/* length of input format */ @@ -1232,7 +1247,7 @@ wrap_strftime(PyObject *object, PyObject *format, PyObject *timetuple, } } - /* Scan the input format, looking for %z and %Z escapes, building + /* Scan the input format, looking for %z/%Z/%f escapes, building * a new format. Since computing the replacements for those codes * is expensive, don't unless they're actually used. */ @@ -1295,6 +1310,18 @@ wrap_strftime(PyObject *object, PyObject *format, PyObject *timetuple, &ntoappend); ntoappend = Py_SIZE(Zreplacement); } + else if (ch == 'f') { + /* format microseconds */ + if (freplacement == NULL) { + freplacement = make_freplacement(object); + if (freplacement == NULL) + goto Done; + } + assert(freplacement != NULL); + assert(PyString_Check(freplacement)); + ptoappend = PyString_AS_STRING(freplacement); + ntoappend = PyString_GET_SIZE(freplacement); + } else { /* percent followed by neither z nor Z */ ptoappend = pin - 2; @@ -1341,6 +1368,7 @@ wrap_strftime(PyObject *object, PyObject *format, PyObject *timetuple, Py_DECREF(time); } Done: + Py_XDECREF(freplacement); Py_XDECREF(zreplacement); Py_XDECREF(Zreplacement); Py_XDECREF(newfmt); @@ -3800,28 +3828,47 @@ datetime_utcfromtimestamp(PyObject *cls, PyObject *args) static PyObject * datetime_strptime(PyObject *cls, PyObject *args) { - PyObject *result = NULL, *obj, *module; + static PyObject *module = NULL; + PyObject *result = NULL, *obj, *st = NULL, *frac = NULL; const Py_UNICODE *string, *format; if (!PyArg_ParseTuple(args, "uu:strptime", &string, &format)) return NULL; - if ((module = PyImport_ImportModuleNoBlock("time")) == NULL) + if (module == NULL && + (module = PyImport_ImportModuleNoBlock("_strptime")) == NULL) return NULL; - obj = PyObject_CallMethod(module, "strptime", "uu", string, format); - Py_DECREF(module); + /* _strptime._strptime returns a two-element tuple. The first + element is a time.struct_time object. The second is the + microseconds (which are not defined for time.struct_time). */ + obj = PyObject_CallMethod(module, "_strptime", "ss", string, format); if (obj != NULL) { - int i, good_timetuple = 1, overflow; - long int ia[6]; - if (PySequence_Check(obj) && PySequence_Size(obj) >= 6) - for (i=0; i < 6; i++) { - PyObject *p = PySequence_GetItem(obj, i); - if (p == NULL) { - Py_DECREF(obj); - return NULL; + int i, good_timetuple = 1; + long int ia[7]; + if (PySequence_Check(obj) && PySequence_Size(obj) == 2) { + st = PySequence_GetItem(obj, 0); + frac = PySequence_GetItem(obj, 1); + if (st == NULL || frac == NULL) + good_timetuple = 0; + /* copy y/m/d/h/m/s values out of the + time.struct_time */ + if (good_timetuple && + PySequence_Check(st) && + PySequence_Size(st) >= 6) { + for (i=0; i < 6; i++) { + PyObject *p = PySequence_GetItem(st, i); + if (p == NULL) { + good_timetuple = 0; + break; + } + if (PyLong_Check(p)) + ia[i] = PyLong_AsLong(p); + else + good_timetuple = 0; + Py_DECREF(p); } - if (PyLong_CheckExact(p)) { +/* if (PyLong_CheckExact(p)) { ia[i] = PyLong_AsLongAndOverflow(p, &overflow); if (overflow) good_timetuple = 0; @@ -3829,17 +3876,29 @@ datetime_strptime(PyObject *cls, PyObject *args) else good_timetuple = 0; Py_DECREF(p); - } +*/ } + else + good_timetuple = 0; + /* follow that up with a little dose of microseconds */ + if (PyLong_Check(frac)) + ia[6] = PyLong_AsLong(frac); + else + good_timetuple = 0; + } else good_timetuple = 0; if (good_timetuple) - result = PyObject_CallFunction(cls, "iiiiii", - ia[0], ia[1], ia[2], ia[3], ia[4], ia[5]); + result = PyObject_CallFunction(cls, "iiiiiii", + ia[0], ia[1], ia[2], + ia[3], ia[4], ia[5], + ia[6]); else PyErr_SetString(PyExc_ValueError, - "unexpected value from time.strptime"); - Py_DECREF(obj); + "unexpected value from _strptime._strptime"); } + Py_XDECREF(obj); + Py_XDECREF(st); + Py_XDECREF(frac); return result; } diff --git a/Modules/gdbmmodule.c b/Modules/gdbmmodule.c index cf197f57dd..a8abbd37cc 100644 --- a/Modules/gdbmmodule.c +++ b/Modules/gdbmmodule.c @@ -130,7 +130,7 @@ dbm_subscript(dbmobject *dp, register PyObject *key) PyErr_SetObject(PyExc_KeyError, key); return NULL; } - v = PyBytes_FromStringAndSize(drec.dptr, drec.dsize); + v = PyString_FromStringAndSize(drec.dptr, drec.dsize); free(drec.dptr); return v; } @@ -220,7 +220,7 @@ dbm_keys(register dbmobject *dp, PyObject *unused) key = gdbm_firstkey(dp->di_dbm); while (key.dptr) { - item = PyBytes_FromStringAndSize(key.dptr, key.dsize); + item = PyString_FromStringAndSize(key.dptr, key.dsize); if (item == NULL) { free(key.dptr); Py_DECREF(v); @@ -291,7 +291,7 @@ dbm_firstkey(register dbmobject *dp, PyObject *unused) check_dbmobject_open(dp); key = gdbm_firstkey(dp->di_dbm); if (key.dptr) { - v = PyBytes_FromStringAndSize(key.dptr, key.dsize); + v = PyString_FromStringAndSize(key.dptr, key.dsize); free(key.dptr); return v; } @@ -323,7 +323,7 @@ dbm_nextkey(register dbmobject *dp, PyObject *args) check_dbmobject_open(dp); nextkey = gdbm_nextkey(dp->di_dbm, key); if (nextkey.dptr) { - v = PyBytes_FromStringAndSize(nextkey.dptr, nextkey.dsize); + v = PyString_FromStringAndSize(nextkey.dptr, nextkey.dsize); free(nextkey.dptr); return v; } diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c index b52bea8413..61128d2aa8 100644 --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -198,7 +198,7 @@ _grouper_create(groupbyobject *parent, PyObject *tgtkey) { _grouperobject *igo; - igo = PyObject_New(_grouperobject, &_grouper_type); + igo = PyObject_GC_New(_grouperobject, &_grouper_type); if (igo == NULL) return NULL; igo->parent = (PyObject *)parent; @@ -206,15 +206,25 @@ _grouper_create(groupbyobject *parent, PyObject *tgtkey) igo->tgtkey = tgtkey; Py_INCREF(tgtkey); + PyObject_GC_Track(igo); return (PyObject *)igo; } static void _grouper_dealloc(_grouperobject *igo) { + PyObject_GC_UnTrack(igo); Py_DECREF(igo->parent); Py_DECREF(igo->tgtkey); - PyObject_Del(igo); + PyObject_GC_Del(igo); +} + +static int +_grouper_traverse(_grouperobject *igo, visitproc visit, void *arg) +{ + Py_VISIT(igo->parent); + Py_VISIT(igo->tgtkey); + return 0; } static PyObject * @@ -280,9 +290,9 @@ static PyTypeObject _grouper_type = { PyObject_GenericGetAttr, /* tp_getattro */ 0, /* tp_setattro */ 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT, /* tp_flags */ + Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC, /* tp_flags */ 0, /* tp_doc */ - 0, /* tp_traverse */ + (traverseproc)_grouper_traverse,/* tp_traverse */ 0, /* tp_clear */ 0, /* tp_richcompare */ 0, /* tp_weaklistoffset */ @@ -299,7 +309,7 @@ static PyTypeObject _grouper_type = { 0, /* tp_init */ 0, /* tp_alloc */ 0, /* tp_new */ - PyObject_Del, /* tp_free */ + PyObject_GC_Del, /* tp_free */ }; @@ -2059,6 +2069,281 @@ static PyTypeObject combinations_type = { }; +/* permutations object ************************************************************ + +def permutations(iterable, r=None): + 'permutations(range(3), 2) --> (0,1) (0,2) (1,0) (1,2) (2,0) (2,1)' + pool = tuple(iterable) + n = len(pool) + r = n if r is None else r + indices = range(n) + cycles = range(n-r+1, n+1)[::-1] + yield tuple(pool[i] for i in indices[:r]) + while n: + for i in reversed(range(r)): + cycles[i] -= 1 + if cycles[i] == 0: + indices[i:] = indices[i+1:] + indices[i:i+1] + cycles[i] = n - i + else: + j = cycles[i] + indices[i], indices[-j] = indices[-j], indices[i] + yield tuple(pool[i] for i in indices[:r]) + break + else: + return +*/ + +typedef struct { + PyObject_HEAD + PyObject *pool; /* input converted to a tuple */ + Py_ssize_t *indices; /* one index per element in the pool */ + Py_ssize_t *cycles; /* one rollover counter per element in the result */ + PyObject *result; /* most recently returned result tuple */ + Py_ssize_t r; /* size of result tuple */ + int stopped; /* set to 1 when the permutations iterator is exhausted */ +} permutationsobject; + +static PyTypeObject permutations_type; + +static PyObject * +permutations_new(PyTypeObject *type, PyObject *args, PyObject *kwds) +{ + permutationsobject *po; + Py_ssize_t n; + Py_ssize_t r; + PyObject *robj = Py_None; + PyObject *pool = NULL; + PyObject *iterable = NULL; + Py_ssize_t *indices = NULL; + Py_ssize_t *cycles = NULL; + Py_ssize_t i; + static char *kwargs[] = {"iterable", "r", NULL}; + + if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|O:permutations", kwargs, + &iterable, &robj)) + return NULL; + + pool = PySequence_Tuple(iterable); + if (pool == NULL) + goto error; + n = PyTuple_GET_SIZE(pool); + + r = n; + if (robj != Py_None) { + if (!PyLong_Check(robj)) { + PyErr_SetString(PyExc_TypeError, "Expected int as r"); + return NULL; + } + r = PyLong_AsSsize_t(robj); + if (r == -1 && PyErr_Occurred()) + goto error; + } + if (r < 0) { + PyErr_SetString(PyExc_ValueError, "r must be non-negative"); + goto error; + } + if (r > n) { + PyErr_SetString(PyExc_ValueError, "r cannot be bigger than the iterable"); + goto error; + } + + indices = PyMem_Malloc(n * sizeof(Py_ssize_t)); + cycles = PyMem_Malloc(r * sizeof(Py_ssize_t)); + if (indices == NULL || cycles == NULL) { + PyErr_NoMemory(); + goto error; + } + + for (i=0 ; i<n ; i++) + indices[i] = i; + for (i=0 ; i<r ; i++) + cycles[i] = n - i; + + /* create permutationsobject structure */ + po = (permutationsobject *)type->tp_alloc(type, 0); + if (po == NULL) + goto error; + + po->pool = pool; + po->indices = indices; + po->cycles = cycles; + po->result = NULL; + po->r = r; + po->stopped = 0; + + return (PyObject *)po; + +error: + if (indices != NULL) + PyMem_Free(indices); + if (cycles != NULL) + PyMem_Free(cycles); + Py_XDECREF(pool); + return NULL; +} + +static void +permutations_dealloc(permutationsobject *po) +{ + PyObject_GC_UnTrack(po); + Py_XDECREF(po->pool); + Py_XDECREF(po->result); + PyMem_Free(po->indices); + PyMem_Free(po->cycles); + Py_TYPE(po)->tp_free(po); +} + +static int +permutations_traverse(permutationsobject *po, visitproc visit, void *arg) +{ + Py_VISIT(po->pool); + Py_VISIT(po->result); + return 0; +} + +static PyObject * +permutations_next(permutationsobject *po) +{ + PyObject *elem; + PyObject *oldelem; + PyObject *pool = po->pool; + Py_ssize_t *indices = po->indices; + Py_ssize_t *cycles = po->cycles; + PyObject *result = po->result; + Py_ssize_t n = PyTuple_GET_SIZE(pool); + Py_ssize_t r = po->r; + Py_ssize_t i, j, k, index; + + if (po->stopped) + return NULL; + + if (result == NULL) { + /* On the first pass, initialize result tuple using the indices */ + result = PyTuple_New(r); + if (result == NULL) + goto empty; + po->result = result; + for (i=0; i<r ; i++) { + index = indices[i]; + elem = PyTuple_GET_ITEM(pool, index); + Py_INCREF(elem); + PyTuple_SET_ITEM(result, i, elem); + } + } else { + if (n == 0) + goto empty; + + /* Copy the previous result tuple or re-use it if available */ + if (Py_REFCNT(result) > 1) { + PyObject *old_result = result; + result = PyTuple_New(r); + if (result == NULL) + goto empty; + po->result = result; + for (i=0; i<r ; i++) { + elem = PyTuple_GET_ITEM(old_result, i); + Py_INCREF(elem); + PyTuple_SET_ITEM(result, i, elem); + } + Py_DECREF(old_result); + } + /* Now, we've got the only copy so we can update it in-place */ + assert(r == 0 || Py_REFCNT(result) == 1); + + /* Decrement rightmost cycle, moving leftward upon zero rollover */ + for (i=r-1 ; i>=0 ; i--) { + cycles[i] -= 1; + if (cycles[i] == 0) { + /* rotatation: indices[i:] = indices[i+1:] + indices[i:i+1] */ + index = indices[i]; + for (j=i ; j<n-1 ; j++) + indices[j] = indices[j+1]; + indices[n-1] = index; + cycles[i] = n - i; + } else { + j = cycles[i]; + index = indices[i]; + indices[i] = indices[n-j]; + indices[n-j] = index; + + for (k=i; k<r ; k++) { + /* start with i, the leftmost element that changed */ + /* yield tuple(pool[k] for k in indices[:r]) */ + index = indices[k]; + elem = PyTuple_GET_ITEM(pool, index); + Py_INCREF(elem); + oldelem = PyTuple_GET_ITEM(result, k); + PyTuple_SET_ITEM(result, k, elem); + Py_DECREF(oldelem); + } + break; + } + } + /* If i is negative, then the cycles have all + rolled-over and we're done. */ + if (i < 0) + goto empty; + } + Py_INCREF(result); + return result; + +empty: + po->stopped = 1; + return NULL; +} + +PyDoc_STRVAR(permutations_doc, +"permutations(iterables[, r]) --> permutations object\n\ +\n\ +Return successive r-length permutations of elements in the iterable.\n\n\ +permutations(range(4), 3) --> (0,1,2), (0,1,3), (0,2,3), (1,2,3)"); + +static PyTypeObject permutations_type = { + PyVarObject_HEAD_INIT(NULL, 0) + "itertools.permutations", /* tp_name */ + sizeof(permutationsobject), /* tp_basicsize */ + 0, /* tp_itemsize */ + /* methods */ + (destructor)permutations_dealloc, /* tp_dealloc */ + 0, /* tp_print */ + 0, /* tp_getattr */ + 0, /* tp_setattr */ + 0, /* tp_compare */ + 0, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + PyObject_GenericGetAttr, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ + Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | + Py_TPFLAGS_BASETYPE, /* tp_flags */ + permutations_doc, /* tp_doc */ + (traverseproc)permutations_traverse, /* tp_traverse */ + 0, /* tp_clear */ + 0, /* tp_richcompare */ + 0, /* tp_weaklistoffset */ + PyObject_SelfIter, /* tp_iter */ + (iternextfunc)permutations_next, /* tp_iternext */ + 0, /* tp_methods */ + 0, /* tp_members */ + 0, /* tp_getset */ + 0, /* tp_base */ + 0, /* tp_dict */ + 0, /* tp_descr_get */ + 0, /* tp_descr_set */ + 0, /* tp_dictoffset */ + 0, /* tp_init */ + 0, /* tp_alloc */ + permutations_new, /* tp_new */ + PyObject_GC_Del, /* tp_free */ +}; + + /* filterfalse object ************************************************************/ typedef struct { @@ -2762,6 +3047,7 @@ inititertools(void) &filterfalse_type, &count_type, &ziplongest_type, + &permutations_type, &product_type, &repeat_type, &groupby_type, diff --git a/Modules/timemodule.c b/Modules/timemodule.c index de8c99c1f1..bd0d063e57 100644 --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -561,7 +561,7 @@ time_strptime(PyObject *self, PyObject *args) if (!strptime_module) return NULL; - strptime_result = PyObject_CallMethod(strptime_module, "strptime", "O", args); + strptime_result = PyObject_CallMethod(strptime_module, "_strptime_time", "O", args); Py_DECREF(strptime_module); return strptime_result; } diff --git a/Objects/floatobject.c b/Objects/floatobject.c index e074a3704c..353a21d010 100644 --- a/Objects/floatobject.c +++ b/Objects/floatobject.c @@ -1639,9 +1639,6 @@ PyFloat_Fini(void) /*---------------------------------------------------------------------------- * _PyFloat_{Pack,Unpack}{4,8}. See floatobject.h. - * - * TODO: On platforms that use the standard IEEE-754 single and double - * formats natively, these routines could simply copy the bytes. */ int _PyFloat_Pack4(double x, unsigned char *p, int le) @@ -1721,28 +1718,31 @@ _PyFloat_Pack4(double x, unsigned char *p, int le) /* Done */ return 0; - Overflow: - PyErr_SetString(PyExc_OverflowError, - "float too large to pack with f format"); - return -1; } else { float y = (float)x; const char *s = (char*)&y; int i, incr = 1; + if (Py_IS_INFINITY(y) && !Py_IS_INFINITY(x)) + goto Overflow; + if ((float_format == ieee_little_endian_format && !le) || (float_format == ieee_big_endian_format && le)) { p += 3; incr = -1; } - + for (i = 0; i < 4; i++) { *p = *s++; p += incr; } return 0; } + Overflow: + PyErr_SetString(PyExc_OverflowError, + "float too large to pack with f format"); + return -1; } int diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index 73598213bb..dcad80896a 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -6590,7 +6590,8 @@ unicode_expandtabs(PyUnicodeObject *self, PyObject *args) Py_UNICODE *e; Py_UNICODE *p; Py_UNICODE *q; - Py_ssize_t i, j, old_j; + Py_UNICODE *qe; + Py_ssize_t i, j, incr; PyUnicodeObject *u; int tabsize = 8; @@ -6598,63 +6599,70 @@ unicode_expandtabs(PyUnicodeObject *self, PyObject *args) return NULL; /* First pass: determine size of output string */ - i = j = old_j = 0; - e = self->str + self->length; + i = 0; /* chars up to and including most recent \n or \r */ + j = 0; /* chars since most recent \n or \r (use in tab calculations) */ + e = self->str + self->length; /* end of input */ for (p = self->str; p < e; p++) if (*p == '\t') { if (tabsize > 0) { - j += tabsize - (j % tabsize); - if (old_j > j) { - PyErr_SetString(PyExc_OverflowError, - "new string is too long"); - return NULL; - } - old_j = j; - } + incr = tabsize - (j % tabsize); /* cannot overflow */ + if (j > PY_SSIZE_T_MAX - incr) + goto overflow1; + j += incr; + } } else { + if (j > PY_SSIZE_T_MAX - 1) + goto overflow1; j++; if (*p == '\n' || *p == '\r') { + if (i > PY_SSIZE_T_MAX - j) + goto overflow1; i += j; - old_j = j = 0; - if (i < 0) { - PyErr_SetString(PyExc_OverflowError, - "new string is too long"); - return NULL; - } + j = 0; } } - if ((i + j) < 0) { - PyErr_SetString(PyExc_OverflowError, "new string is too long"); - return NULL; - } + if (i > PY_SSIZE_T_MAX - j) + goto overflow1; /* Second pass: create output string and fill it */ u = _PyUnicode_New(i + j); if (!u) return NULL; - j = 0; - q = u->str; + j = 0; /* same as in first pass */ + q = u->str; /* next output char */ + qe = u->str + u->length; /* end of output */ for (p = self->str; p < e; p++) if (*p == '\t') { if (tabsize > 0) { i = tabsize - (j % tabsize); j += i; - while (i--) + while (i--) { + if (q >= qe) + goto overflow2; *q++ = ' '; + } } } else { - j++; + if (q >= qe) + goto overflow2; *q++ = *p; + j++; if (*p == '\n' || *p == '\r') j = 0; } return (PyObject*) u; + + overflow2: + Py_DECREF(u); + overflow1: + PyErr_SetString(PyExc_OverflowError, "new string is too long"); + return NULL; } PyDoc_STRVAR(find__doc__, diff --git a/PCbuild/_bsddb.vcproj b/PCbuild/_bsddb.vcproj index a59c7032ae..3efdcce0c5 100644 --- a/PCbuild/_bsddb.vcproj +++ b/PCbuild/_bsddb.vcproj @@ -115,11 +115,11 @@ /> <Tool Name="VCPreLinkEventTool" - CommandLine="cd $(bsddbDir)
if exist Debug_AMD64\libdb44sd.lib exit 0
vcbuild /useenv db_static.vcproj "Debug AMD64|Win32"
" + CommandLine="cd $(bsddbDir)
if exist "x64\Debug AMD64\db_static.lib" exit 0
vcbuild /useenv db_static.vcproj "Debug AMD64|x64"
" /> <Tool Name="VCLinkerTool" - AdditionalDependencies="$(bsddbDir)\Debug_AMD64\libdb44sd.lib" + AdditionalDependencies=""$(bsddbDir)x64\Debug AMD64\db_static.lib"" BaseAddress="0x1e180000" /> <Tool @@ -242,11 +242,11 @@ /> <Tool Name="VCPreLinkEventTool" - CommandLine="cd $(bsddbDir)
if exist Release_AMD64\libdb44s.lib exit 0
vcbuild /useenv db_static.vcproj "Release AMD64|Win32"
" + CommandLine="cd $(bsddbDir)
if exist x64\Release AMD64\db_static.lib exit 0
vcbuild /useenv db_static.vcproj "Release AMD64|Win32"
" /> <Tool Name="VCLinkerTool" - AdditionalDependencies="$(bsddbDir)\Release_AMD64\libdb44s.lib" + AdditionalDependencies="$(bsddbDir)x64\Release AMD64\db_static.lib" BaseAddress="0x1e180000" /> <Tool @@ -369,11 +369,11 @@ /> <Tool Name="VCPreLinkEventTool" - CommandLine="cd $(bsddbDir)
if exist Release_AMD64\libdb44s.lib exit 0
vcbuild /useenv db_static.vcproj "Release AMD64|Win32"
" + CommandLine="cd $(bsddbDir)
if exist "x64\Release AMD64\db_static.lib" exit 0
vcbuild /useenv db_static.vcproj "Release AMD64|Win32"
" /> <Tool Name="VCLinkerTool" - AdditionalDependencies="$(bsddbDir)\Release_AMD64\libdb44s.lib" + AdditionalDependencies="$(bsddbDir)x64\Release AMD64\db_static.lib" BaseAddress="0x1e180000" TargetMachine="17" /> @@ -497,11 +497,11 @@ /> <Tool Name="VCPreLinkEventTool" - CommandLine="cd $(bsddbDir)
if exist Release_AMD64\libdb44s.lib exit 0
vcbuild /useenv db_static.vcproj "Release AMD64|Win32"
" + CommandLine="cd $(bsddbDir)
if exist x64\Release AMD64\db_static.lib exit 0
vcbuild /useenv db_static.vcproj "Release AMD64|Win32"
" /> <Tool Name="VCLinkerTool" - AdditionalDependencies="$(bsddbDir)\Release_AMD64\libdb44s.lib" + AdditionalDependencies="$(bsddbDir)x64\Release AMD64\db_static.lib" BaseAddress="0x1e180000" TargetMachine="17" /> diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt index 4d2f9b6173..6b04b92c8c 100644 --- a/PCbuild/readme.txt +++ b/PCbuild/readme.txt @@ -202,7 +202,9 @@ _bsddb The _bsddb subprojects depends only on the db_static project of Berkeley DB. You have to choose either "Release", "Release AMD64", "Debug" - or "Debug AMD64" as configuration. + or "Debug AMD64" as configuration. For the AND64 builds, you need to + create the "x64" platform first (in Solution Platforms\Configuration + Manager...) Alternatively, if you want to start with the original sources, go to Sleepycat's download page: diff --git a/PCbuild/rt.bat b/PCbuild/rt.bat index 8e386c5856..583d4f3339 100644 --- a/PCbuild/rt.bat +++ b/PCbuild/rt.bat @@ -1,11 +1,13 @@ @echo off rem Run Tests. Run the regression test suite. -rem Usage: rt [-d] [-O] [-q] regrtest_args +rem Usage: rt [-d] [-O] [-q] [-x64] regrtest_args rem -d Run Debug build (python_d.exe). Else release build. rem -O Run python.exe or python_d.exe (see -d) with -O. rem -q "quick" -- normally the tests are run twice, the first time rem after deleting all the .py[co] files reachable from Lib/. rem -q runs the tests just once, and without deleting .py[co] files. +rem -x64 Run the 64-bit build of python (or python_d if -d was specified) +rem from the 'amd64' dir instead of the 32-bit build in this dir. rem All leading instances of these switches are shifted off, and rem whatever remains is passed to regrtest.py. For example, rem rt -O -d -x test_thread @@ -24,16 +26,20 @@ rem rt -u "network,largefile" setlocal -set exe=python +set prefix=.\ +set suffix= set qmode= set dashO= -PATH %PATH%;..\..\tcltk\bin +set tcltk= :CheckOpts if "%1"=="-O" (set dashO=-O) & shift & goto CheckOpts if "%1"=="-q" (set qmode=yes) & shift & goto CheckOpts -if "%1"=="-d" (set exe=python_d) & shift & goto CheckOpts +if "%1"=="-d" (set suffix=_d) & shift & goto CheckOpts +if "%1"=="-x64" (set prefix=amd64) & (set tcltk=tcltk64) & shift & goto CheckOpts +PATH %PATH%;..\..\%tcltk%\bin +set exe=%prefix%\python%suffix% set cmd=%exe% %dashO% -E -tt ../lib/test/regrtest.py %1 %2 %3 %4 %5 %6 %7 %8 %9 if defined qmode goto Qmode diff --git a/Python/ast.c b/Python/ast.c index 8b68182582..d5d84f74b8 100644 --- a/Python/ast.c +++ b/Python/ast.c @@ -1994,10 +1994,14 @@ ast_for_call(struct compiling *c, const node *n, expr_ty func) } else if (TYPE(ch) == STAR) { vararg = ast_for_expr(c, CHILD(n, i+1)); + if (!vararg) + return NULL; i++; } else if (TYPE(ch) == DOUBLESTAR) { kwarg = ast_for_expr(c, CHILD(n, i+1)); + if (!kwarg) + return NULL; i++; } } diff --git a/Python/ceval.c b/Python/ceval.c index ec6d4e32d4..bac8278126 100644 --- a/Python/ceval.c +++ b/Python/ceval.c @@ -2021,17 +2021,20 @@ PyEval_EvalFrameEx(PyFrameObject *f, int throwflag) case WITH_CLEANUP: { - /* TOP is the context.__exit__ bound method. - Below that are 1-3 values indicating how/why - we entered the finally clause: - - SECOND = None - - (SECOND, THIRD) = (WHY_{RETURN,CONTINUE}), retval - - SECOND = WHY_*; no retval below it - - (SECOND, THIRD, FOURTH) = exc_info() + /* At the top of the stack are 1-3 values indicating + how/why we entered the finally clause: + - TOP = None + - (TOP, SECOND) = (WHY_{RETURN,CONTINUE}), retval + - TOP = WHY_*; no retval below it + - (TOP, SECOND, THIRD) = exc_info() + Below them is EXIT, the context.__exit__ bound method. In the last case, we must call - TOP(SECOND, THIRD, FOURTH) + EXIT(TOP, SECOND, THIRD) otherwise we must call - TOP(None, None, None) + EXIT(None, None, None) + + In all cases, we remove EXIT from the stack, leaving + the rest in the same order. In addition, if the stack represents an exception, *and* the function call returns a 'true' value, we @@ -2040,36 +2043,59 @@ PyEval_EvalFrameEx(PyFrameObject *f, int throwflag) should still be resumed.) */ - x = TOP(); - u = SECOND(); - if (PyLong_Check(u) || u == Py_None) { + PyObject *exit_func; + + u = POP(); + if (u == Py_None) { + exit_func = TOP(); + SET_TOP(u); + v = w = Py_None; + } + else if (PyLong_Check(u)) { + switch(PyLong_AS_LONG(u)) { + case WHY_RETURN: + case WHY_CONTINUE: + /* Retval in TOP. */ + exit_func = SECOND(); + SET_SECOND(TOP()); + SET_TOP(u); + break; + default: + exit_func = TOP(); + SET_TOP(u); + break; + } u = v = w = Py_None; } else { - v = THIRD(); - w = FOURTH(); + v = TOP(); + w = SECOND(); + exit_func = THIRD(); + SET_TOP(u); + SET_SECOND(v); + SET_THIRD(w); } /* XXX Not the fastest way to call it... */ - x = PyObject_CallFunctionObjArgs(x, u, v, w, NULL); - if (x == NULL) + x = PyObject_CallFunctionObjArgs(exit_func, u, v, w, + NULL); + if (x == NULL) { + Py_DECREF(exit_func); break; /* Go to error exit */ + } if (u != Py_None && PyObject_IsTrue(x)) { /* There was an exception and a true return */ - Py_DECREF(x); - x = TOP(); /* Again */ - STACKADJ(-3); + STACKADJ(-2); Py_INCREF(Py_None); SET_TOP(Py_None); - Py_DECREF(x); Py_DECREF(u); Py_DECREF(v); Py_DECREF(w); } else { - /* Let END_FINALLY do its thing */ - Py_DECREF(x); - x = POP(); - Py_DECREF(x); + /* The stack was rearranged to remove EXIT + above. Let END_FINALLY do its thing */ } + Py_DECREF(x); + Py_DECREF(exit_func); PREDICT(END_FINALLY); break; } diff --git a/Python/compile.c b/Python/compile.c index f415519855..6ad3822a98 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -3092,7 +3092,7 @@ compiler_with(struct compiler *c, stmt_ty s) { static identifier enter_attr, exit_attr; basicblock *block, *finally; - identifier tmpexit, tmpvalue = NULL; + identifier tmpvalue = NULL; assert(s->kind == With_kind); @@ -3112,12 +3112,6 @@ compiler_with(struct compiler *c, stmt_ty s) if (!block || !finally) return 0; - /* Create a temporary variable to hold context.__exit__ */ - tmpexit = compiler_new_tmpname(c); - if (tmpexit == NULL) - return 0; - PyArena_AddPyObject(c->c_arena, tmpexit); - if (s->v.With.optional_vars) { /* Create a temporary variable to hold context.__enter__(). We need to do this rather than preserving it on the stack @@ -3137,11 +3131,10 @@ compiler_with(struct compiler *c, stmt_ty s) /* Evaluate EXPR */ VISIT(c, expr, s->v.With.context_expr); - /* Squirrel away context.__exit__ */ + /* Squirrel away context.__exit__ by stuffing it under context */ ADDOP(c, DUP_TOP); ADDOP_O(c, LOAD_ATTR, exit_attr, names); - if (!compiler_nameop(c, tmpexit, Store)) - return 0; + ADDOP(c, ROT_TWO); /* Call context.__enter__() */ ADDOP_O(c, LOAD_ATTR, enter_attr, names); @@ -3185,10 +3178,9 @@ compiler_with(struct compiler *c, stmt_ty s) if (!compiler_push_fblock(c, FINALLY_END, finally)) return 0; - /* Finally block starts; push tmpexit and issue our magic opcode. */ - if (!compiler_nameop(c, tmpexit, Load) || - !compiler_nameop(c, tmpexit, Del)) - return 0; + /* Finally block starts; context.__exit__ is on the stack under + the exception or return information. Just issue our magic + opcode. */ ADDOP(c, WITH_CLEANUP); /* Finally block ends. */ diff --git a/Python/import.c b/Python/import.c index 7dab46ea8e..8f8341038b 100644 --- a/Python/import.c +++ b/Python/import.c @@ -72,6 +72,7 @@ extern time_t PyOS_GetLastModificationTime(char *, FILE *); storing constants that should have been removed) Python 2.5c2: 62131 (fix wrong code: for x, in ... in listcomp/genexp) Python 2.6a0: 62151 (peephole optimizations and STORE_MAP opcode) + Python 2.6a1: 62161 (WITH_CLEANUP optimization) Python 3000: 3000 3010 (removed UNARY_CONVERT) 3020 (added BUILD_SET) @@ -84,9 +85,9 @@ extern time_t PyOS_GetLastModificationTime(char *, FILE *); 3090 (kill str8 interning) 3100 (merge from 2.6a0, see 62151) 3102 (__file__ points to source file) -. + Python 3.0a4: 3110 (WITH_CLEANUP optimization). */ -#define MAGIC (3102 | ((long)'\r'<<16) | ((long)'\n'<<24)) +#define MAGIC (3110 | ((long)'\r'<<16) | ((long)'\n'<<24)) /* Magic word as global; note that _PyImport_Init() can change the value of this global to accommodate for alterations of how the @@ -61,6 +61,9 @@ the "What's New in Python 3.0" document, found at Please help write it! +If you want to install multiple versions of Python see the section below +entitled "Installing multiple versions". + What's New Since 3.0a1 ---------------------- @@ -83,6 +86,29 @@ code that needs to be changed, such as optional warnings when deprecated features are used, and backported versions of certain key Python 3000 features. + +Installing multiple versions +---------------------------- + +On Unix and Mac systems if you intend to install multiple versions of Python +using the same installation prefix (--prefix argument to the configure +script) you must take care that your primary python executable is not +overwritten by the installation of a different versio. All files and +directories installed using "make altinstall" contain the major and minor +version and can thus live side-by-side. "make install" also creates +${prefix}/bin/python which refers to ${prefix}/bin/pythonX.Y. If you intend +to install multiple versions using the same prefix you must decide which +version (if any) is your "primary" version. Install that version using +"make install". Install all other versions using "make altinstall". + +For example, if you want to install Python 2.5, 2.6 and 3.0 with 2.6 being +the primary version, you would execute "make install" in your 2.6 build +directory and "make altinstall" in the others. + + +Configuration options and variables +----------------------------------- + A source-to-source translation tool, "2to3", can take care of the mundane task of converting large amounts of source code. It is not a complete solution but is complemented by the deprecation warnings in diff --git a/Tools/buildbot/clean.bat b/Tools/buildbot/clean.bat index 15d7365068..ec71804da0 100644 --- a/Tools/buildbot/clean.bat +++ b/Tools/buildbot/clean.bat @@ -1,7 +1,7 @@ @rem Used by the buildbot "clean" step. call "%VS90COMNTOOLS%vsvars32.bat" -cd PCbuild @echo Deleting .pyc/.pyo files ... del /s Lib\*.pyc Lib\*.pyo +cd PCbuild vcbuild /clean pcbuild.sln "Release|Win32" vcbuild /clean pcbuild.sln "Debug|Win32" diff --git a/Tools/buildbot/external-amd64.bat b/Tools/buildbot/external-amd64.bat index 3e5c8595ab..0ea4d64160 100644 --- a/Tools/buildbot/external-amd64.bat +++ b/Tools/buildbot/external-amd64.bat @@ -10,10 +10,15 @@ call "%VS90COMNTOOLS%vsvars32.bat" if not exist bzip2-1.0.3 svn export http://svn.python.org/projects/external/bzip2-1.0.3 @rem Sleepycat db -if not exist db-4.4.20 svn export http://svn.python.org/projects/external/db-4.4.20 -@REM if not exist db-4.4.20\build_win32\debug\libdb44sd.lib ( -@REM vcbuild db-4.4.20\build_win32\Berkeley_DB.sln /build Debug /project db_static -@REM ) +@rem Remove VS 2003 builds +if exist db-4.4.20 if not exist db-4.4.20\build_win32\this_is_for_vs9 ( + echo Removing old build + rd /s/q db-4.4.20 +) +if not exist db-4.4.20 svn export http://svn.python.org/projects/external/db-4.4.20-vs9 db-4.4.20 +if not exist db-4.4.20\build_win32\debug\libdb44sd.lib ( + vcbuild db-4.4.20\build_win32\db_static.vcproj "Debug AMD64|x64" +) @rem OpenSSL if not exist openssl-0.9.8g ( diff --git a/Tools/buildbot/test-amd64.bat b/Tools/buildbot/test-amd64.bat index 9cb6968ecc..d178c92fb5 100644 --- a/Tools/buildbot/test-amd64.bat +++ b/Tools/buildbot/test-amd64.bat @@ -1,3 +1,3 @@ @rem Used by the buildbot "test" step. -cd PC\VS7.1 -call rt.bat -q -uall -rw +cd PCbuild +call rt.bat -q -d -x64 -uall -rw diff --git a/Tools/pybench/Setup.py b/Tools/pybench/Setup.py index f1417e6c3f..21e654afba 100644 --- a/Tools/pybench/Setup.py +++ b/Tools/pybench/Setup.py @@ -30,6 +30,10 @@ from Lists import * from Tuples import * from Dict import * from Exceptions import * +try: + from With import * +except SyntaxError: + pass from Imports import * from Strings import * from Numbers import * diff --git a/Tools/pybench/pybench.py b/Tools/pybench/pybench.py index dd398f5cba..624347f50e 100755 --- a/Tools/pybench/pybench.py +++ b/Tools/pybench/pybench.py @@ -121,7 +121,8 @@ def get_machine_details(): 'platform': platform.platform(), 'processor': platform.processor(), 'executable': sys.executable, - 'implementation': platform.python_implementation(), + 'implementation': getattr(platform, 'python_implementation', + lambda:'n/a')(), 'python': platform.python_version(), 'compiler': platform.python_compiler(), 'buildno': buildno, @@ -833,7 +834,7 @@ python pybench.py -s p25.pybench -c p21.pybench print('PYBENCH %s' % __version__) print('-' * LINE) print('* using %s %s' % ( - platform.python_implementation(), + getattr(platform, 'python_implementation', lambda:'Python')(), ' '.join(sys.version.split()))) # Switch off garbage collection |