more documentation work

10 files changed, 216 insertions, 86 deletions

diff --git a/docs/lib/passlib.hash.apr_md5_crypt.rst b/docs/lib/passlib.hash.apr_md5_crypt.rst
index 0aefe1d..017ff31 100644
--- a/docs/lib/passlib.hash.apr_md5_crypt.rst
+++ b/docs/lib/passlib.hash.apr_md5_crypt.rst
@@ -5,20 +5,22 @@
 .. module:: passlib.hash.apr_md5_crypt
     :synopsis: Apache MD5-Crypt variant
 
-Stats: 96 bit checksum, 48 bit salt, :ref:`modular-crypt-format` compatible.
-
 This format is a variation of :mod:`~passlib.hash.md5_crypt`,
 primarily used by the Apache webserver in ``htpasswd`` files.
+It contains only minor changes to md5-crypt, and should
+be considered just as strong / weak as md5-crypt itself.
 
+Format & Algorithm
+==================
 This format is identical to md5-crypt, except for two things:
 it uses ``$apr1$`` as a prefix where md5-crypt uses ``$1$``,
 and inserts ``$apr1$`` where md5-crypt inserts ``$1$`` into
-it's internal hash calculation. Thus, this algorithm is just
-as strong as md5-crypt, but the formats (and their contained checksums)
-are in no way compatible with eachother.
+it's internal hash calculation. Thus, hashes generated
+by this and md5-crypt are in no way compatible with eachother
+(they will not even have the same checksum for the same salt).
+
+For details about usage & algorithm, see :mod:`~passlib.hash.md5_crypt`.
 
-Implementation
-==============
-PassLib contains a builtin pure-python implementation of apr-md5-crypt,
-based of the specification at `http://httpd.apache.org/docs/2.2/misc/password_encryptions.html`,
-but code shared with :mod:`~passlib.hash.md5_crypt`.
+References
+==========
+* `<http://httpd.apache.org/docs/2.2/misc/password_encryptions.html>`_
diff --git a/docs/lib/passlib.hash.bcrypt.rst b/docs/lib/passlib.hash.bcrypt.rst
index 5c68a22..32842fc 100644
--- a/docs/lib/passlib.hash.bcrypt.rst
+++ b/docs/lib/passlib.hash.bcrypt.rst
@@ -1,10 +1,66 @@
 ==================================================================
-:mod:`passlib.hash.bcrypt` - BCrypt password hash
+:mod:`passlib.hash.bcrypt` - BCrypt
 ==================================================================
 
 .. module:: passlib.hash.bcrypt
     :synopsis: BCrypt
 
+BCrypt was developed to replace :mod:`~passlib.hash.md5_crypt` for BSD systems.
+It uses a modified version of the Blowfish stream cipher. Featuring
+a large salt and variable number of rounds, it's currently the default
+password hash for many systems (notably BSD), and has no known weaknesses.
+
+.. note::
+
+    It is strongly recommended to install PyBcrypt if this algorithm
+    is going to be used.
+
+Usage
+=====
+
 .. todo::
 
-    write documentation
+    write usage instructions
+
+Functions
+=========
+.. autofunction:: genconfig
+.. autofunction:: genhash
+.. autofunction:: encrypt
+.. autofunction:: identify
+.. autofunction:: verify
+
+Format & Algorithm
+==================
+Bcrypt is compatible with the :ref:`modular-crypt-format`, and uses ``$2$`` and ``$2a$`` as the identifying prefix
+for all it's strings (``$2$`` is seen only for legacy hashes which used an older version of Bcrypt).
+An example hash (of ``password``) is ``$2a$12$GhvMmNVjRW29ulnudl.LbuAnUtN/LRfe1JsBm1Xu6LE3059z5Tr8m``.
+Bcrypt hashes have the format ``$2a${cost}${salt}{checksum}``, where:
+
+* ``{cost}`` is the cost parameter, encoded as 2 zero-padded decimal digits,
+  which determines the number of rounds used via ``rounds=2**cost`` (cost is 12 in the example).
+* ``{salt}`` is the 22 character salt string, using the characters ``[./A-Za-z0-9]`` (``GhvMmNVjRW29ulnudl.Lbu`` in the example).
+* ``{checksum}`` is the 31 character checksum, using the same characters as the salt (``AnUtN/LRfe1JsBm1Xu6LE3059z5Tr8m`` in the example).
+
+BCrypt's algorithm is described in detail in it's specification document,
+listed below.
+
+Deviations
+==========
+This implementation of bcrypt differs from others in a few ways:
+
+* The bcrypt specification (and implementations) have no predefined
+  or predictable behavior when passed a salt containing characters
+  outside of the base64 range. To avoid this situtation,
+  PassLib will simply throw an error if invalid characters
+  are provided for the salt.
+
+* Before generating a hash, PassLib encodes unicode passwords using UTF-8.
+  While the algorithm accepts passwords containing any 8-bit value
+  except for ``\x00``, it specifies no preference for encodings,
+  or for handling unicode strings.
+
+References
+==========
+* `<http://www.usenix.org/event/usenix99/provos/provos_html/>`_ - the bcrypt format specification
+* `<http://www.mindrot.org/projects/jBCrypt/>`_ - java implementation used as reference for PassLib
diff --git a/docs/lib/passlib.hash.des_crypt.rst b/docs/lib/passlib.hash.des_crypt.rst
index 0253487..350fbac 100644
--- a/docs/lib/passlib.hash.des_crypt.rst
+++ b/docs/lib/passlib.hash.des_crypt.rst
@@ -47,7 +47,7 @@ Functions
 
 Format
 ======
-A des-crypt hash string consists of 13 characters, drawn from ``[0-9a-zA-Z./]``.
+A des-crypt hash string consists of 13 characters, drawn from ``[./0-9A-Za-z]``.
 The first 2 characters form a :mod:`hash64 <passlib.utils.h64>`-encoded
 12 bit integer used as the salt, with the remaining characters
 forming a hash64-encoded 64-bit integer checksum.
@@ -75,7 +75,7 @@ Deviations
 ==========
 This implementation of des-crypt differs from others in a few ways:
 
-* Unicode strings are encoded using UTF-8 before being passed into the algorithm.
+* Before generating a hash, PassLib encodes unicode passwords using UTF-8.
   The original des-crypt was designed for 7-bit us-ascii, so this should not
   conflict with most existing hashes. As of this writing, the authors
   know of no specification defining the official behavior that should be used
diff --git a/docs/lib/passlib.hash.ext_des_crypt.rst b/docs/lib/passlib.hash.ext_des_crypt.rst
index c166cdb..7b3911e 100644
--- a/docs/lib/passlib.hash.ext_des_crypt.rst
+++ b/docs/lib/passlib.hash.ext_des_crypt.rst
@@ -6,8 +6,9 @@
     :synopsis: BSDi Extended Unix (DES) Crypt
 
 This algorithm was developed by BSDi for their BSD/OS distribution.
-It's based on :mod:`~passlib.hash.des_crypt`, contains many modern improvements.
-Nonetheless, since it's based on DES, and still shared many of des-crypt's flaws,
+It's based on :mod:`~passlib.hash.des_crypt`, and contains a larger
+salt and a variable number of rounds.  Nonetheless, since it's based on DES,
+and still shares many of des-crypt's other flaws,
 it should not be used in new applications.
 
 Usage
@@ -15,10 +16,6 @@ Usage
 Aside from differences in format and salt size,
 ext-des-crypt usage is exactly the same as :mod:`~passlib.hash.des_crypt`.
 
-.. todo::
-
-    show examples when specifying number of rounds
-
 Functions
 =========
 .. autofunction:: genconfig
@@ -29,32 +26,30 @@ Functions
 
 Format
 ======
-A ext-des-crypt hash string consists of an underscore ``_``, followed
-by 20 characters, drawn from ``[0-9a-zA-Z./]``.
-The first 4 characters after the underscore form a :mod:`hash64 <passlib.utils.h64>`-encoded
-24 bit integer determining the number of rounds.
-The next 4 characters form a hash64-encoded
-24 bit integer determining the salt.
-The final 11 characters form a hash64-encoded 64-bit integer checksum.
+An example hash (of ``password``) is ``_EQ0.jzhSVeUyoSqLupI``.
+An ext_des_crypt hash string consists of a 21 character string of the form ``_{rounds}{salt}{checksum}``.
+All characters except the underscore prefix are drawn from ``[./0-9A-Za-z]``.
 
-A ext-des-crypt configuration string is also accepted by this module,
-and has the same format as the hash string, but with the final 11 checksum characters.
+* ``_`` - the underscore is used to distinguish this scheme from others, such as des-crypt.
+* ``{rounds>`` is the number of rounds, stored as a 4 character :mod:`hash64 <passlib.utils.h64>`-encoded 24-bit integer (``EQ0.`` in the example).
+* ``{salt}`` is the salt, stored as as a 4 character hash64-encoded 24-bit integer (``jzhS`` in the example).
+* ``{checksum}`` is the checksum, stored as an 11 character hash64-encoded 64-bit integer (``VeUyoSqLupI`` in the example).
 
-An example hash (of ``password``) is ``_EQ0.jzhSVeUyoSqLupI``, where the rounds
-are encoded in ``EQ0.`` (=10000), the salt is ``jzhS``, and the checksum
-is ``VeUyoSqLupI``.
+A ext_des_crypt configuration string is also accepted by this module;
+and has the same format as the hash string, but with the checksum portion omitted.
 
 Algorithm
 =========
 The checksum is formed by a modified version of the DES cipher in encrypt mode:
 
 * First, the lower 7 bits of the first 8 characters of the password are used
-  to form a 56-bit DES key, as in des-crypt.
+  to form a 56-bit DES key, in the same manner as des-crypt.
 
 * Unlike des-crypt, the remainder of the password is also used. For every additional
   8 characters in the password, the key is encrypted using a single round of DES,
   with itself as the input block. It is then xor'ed against the lower 7 bits
-  of the next 8 characters in the password.
+  of the next 8 characters in the password. This is repeated until the password
+  is used up.
 
 * The checksum is then generated by recursively performing a variable number rounds of DES encryption
   starting with a null input block. The 24 bits of salt are used to mutate
@@ -65,9 +60,9 @@ The checksum is formed by a modified version of the DES cipher in encrypt mode:
 
 Deviations
 ==========
-This implementation of ext-des-crypt differs from others in a few ways:
+This implementation of ext-des-crypt differs from others in one way:
 
-* Unicode strings are encoded using UTF-8 before being passed into the algorithm.
+* Before generating a hash, PassLib encodes unicode passwords using UTF-8.
   The original ext-des-crypt was designed for 7-bit us-ascii, so this should not
   conflict with most existing hashes. As of this writing, the authors
   know of no specification defining the official behavior that should be used
diff --git a/docs/lib/passlib.hash.md5_crypt.rst b/docs/lib/passlib.hash.md5_crypt.rst
index 4fb9db9..333a6e1 100644
--- a/docs/lib/passlib.hash.md5_crypt.rst
+++ b/docs/lib/passlib.hash.md5_crypt.rst
@@ -7,15 +7,13 @@
 
 This algorithm was developed to replace the aging des-crypt crypt.
 It is supported by a wide variety of unix flavors, and is found
-in other contexts as well.
-
-Security-wise, MD5-Crypt lacks newer features,
+in other contexts as well. Security-wise, MD5-Crypt lacks newer features,
 such as a variable number of rounds. Futhermore, the MD5 message digest
 algorithm which it's based around is considered broken,
 though pre-image attacks are currently only theoretical.
 Despite this, MD5-Crypt itself is not considered broken,
 and is still considered ok to use, though new applications
-should use a strong scheme if feasible.
+should use a stronger scheme (eg :mod:`~passlib.hash.sha512_crypt`) if possible.
 
 Usage
 =====
@@ -33,21 +31,21 @@ Functions
 
 Format
 ======
-This algorithm was created in parallel with
-the :ref:`modular-crypt-format`, and so it uses
-the identifier ``$1$`` for all of it's hashes.
-
-An md5-crypt hash string has length 26-34, with the format ``$1$<salt>$<checksum>``;
-where ``<salt>`` is 0-8 characters drawn from ``[0-9a-zA-Z./]``,
-and ``<checksum>`` is 22 characters drawn from the same set.
-
 An example hash (of ``password``) is ``$1$5pZSV9va$azfrPr6af3Fc7dLblQXVa0``.
+An md5-crypt hash string has the format ``$1${salt}${checksum}``, where:
+
+* ``$1$`` is the prefix used to identify md5_crypt hashes,
+  following the :ref:`modular-crypt-format`
+* ``{salt}`` is 0-8 characters drawn from ``[./0-9A-Za-z]``,
+  providing a 48-bit salt (``5pZSV9va`` in the example).
+* ``{checksum}`` is 22 characters drawn from the same set,
+  encoding a 128-bit checksum (``azfrPr6af3Fc7dLblQXVa0`` in the example).
 
 Algorithm
 =========
 The algorithm used by MD5-Crypt is convoluted,
 and is best described by examining the BSD implementation
-linked to below.
+linked to below, or the source code to this module.
 
 It uses the MD5 message digest algorithm to generate
 various intermediate digests based on combinations
@@ -69,14 +67,13 @@ This implementation of md5-crypt differs from others in a few ways:
   unix ``/etc/shadow`` files. Futhermore, most unix systems
   will only generate salts using the standard 64 characters listed above.
   This implementation follows along with that, by strictly limiting
-  salt strings to the known-good set, until counter-examples are found.
+  salt strings to the least common denominator, ``[./0-9A-Za-z]``.
 
-* Unicode strings are encoded using UTF-8 before being passed into the algorithm.
+* Before generating a hash, PassLib encodes unicode passwords using UTF-8.
   While the algorithm accepts passwords containing any 8-bit value
-  except for ``\x00``, as of this writing, the authors
-  know of no specification defining the official behavior that should be used
-  for unicode strings.
+  except for ``\x00``, it specifies no preference for encodings,
+  or for handling unicode strings.
 
 References
 ==========
-* `<http://www.freebsd.org/cgi/cvsweb.cgi/~checkout~/src/lib/libcrypt/crypt.c?rev=1.2>` - primary reference used for information & implementation
+* `<http://www.freebsd.org/cgi/cvsweb.cgi/~checkout~/src/lib/libcrypt/crypt.c?rev=1.2>`_ - primary reference used for information & implementation
diff --git a/docs/lib/passlib.hash.nthash.rst b/docs/lib/passlib.hash.nthash.rst
index bc6a6b1..bb35415 100644
--- a/docs/lib/passlib.hash.nthash.rst
+++ b/docs/lib/passlib.hash.nthash.rst
@@ -15,10 +15,6 @@ This module implements the Windows NT-HASH algorithm,
 encoded in a manner compatible with the :ref:`modular-crypt-format`.
 It is found on some unix systems where the administrator has decided
 to store user passwords in a manner compatible with the SMB/CIFS protocol.
-
-It supports two identifiers, ``$3$`` and ``$NT$``, though this
-implementation defaults to ``$3$``.
-
 It has no salt, or variable rounds.
 
 Usage
@@ -46,9 +42,9 @@ the following method:
 
 Format & Algorithm
 ==================
-A nthash encoded for crypt consists of ``$3$$<checksum>`` or
-``$NT$<checksum>``; where ``checksum`` is 32 hexidecimal digits
+A nthash encoded for crypt consists of ``$3$${checksum}`` or
+``$NT${checksum}``; where ``{checksum}`` is 32 hexidecimal digits
 encoding the checksum. An example hash (of ``password``) is ``$3$$8846f7eaee8fb117ad06bdd830b7586c``.
 
 The checksum is simply the :mod:`~passlib.utils.md4` digest
-of the secret using the ``UTF16-LE`` encoding.
+of the secret using the ``UTF16-LE`` encoding, encoded in hexidecimal
diff --git a/docs/lib/passlib.hash.phpass.rst b/docs/lib/passlib.hash.phpass.rst
index 0bdc24a..678746b 100644
--- a/docs/lib/passlib.hash.phpass.rst
+++ b/docs/lib/passlib.hash.phpass.rst
@@ -9,9 +9,7 @@ This algorithm is used primarily by PHP software
 which uses the `PHPass <http://www.openwall.com/phpass/>`_ library,
 a PHP library similar to PassLib. The PHPass Portable Hash
 is a custom password hash used by PHPass as a fallback
-when none of it's other hashes are available. It's hashes
-can be identified by the :ref:`modular-crypt-format` prefix
-``$P$`` (or ``$H$`` in phpBB3 databases).
+when none of it's other hashes are available.
 Due to it's reliance on MD5, and the simplistic implementation,
 other hash algorithms should be used if possible.
 
@@ -29,26 +27,37 @@ Functions
 .. autofunction:: identify
 .. autofunction:: verify
 
-Format & Algorithm
+Format
 ==================
-An phpass portable hash string has length 34, with the format ``$P$<rounds><salt><checksum>``;
-where ``<rounds>`` is a single character encoding a 6-bit integer,
-``<salt>`` is an eight-character salt, and ``<checksum>`` is an encoding
-of the 128 bit checksum. All values are encoded using :mod:`hash64 <passlib.utils.h64>`.
+An example hash (of ``password``) is ``$P$8ohUJ.1sdFw09/bMaAQPTGDNi2BIUt1``.
+A phpass portable hash string has the format ``$P${cost}{salt}{checksum}``, where:
 
-An example hash (of ``password``) is ``$P$8ohUJ.1sdFw09/bMaAQPTGDNi2BIUt1``;
-the rounds are encoded in ``8``, the salt is ``ohUJ.1sd``,
-and the checksum is ``Fw09/bMaAQPTGDNi2BIUt1``.
+* ``$P$`` is the prefix used to identify phpass hashes,
+  following the :ref:`modular-crypt-format`.
+  Note that phpBB3 databases uses the alternate prefix ``$H$``, both prefixes
+  are recognized by this module, and the checksums are the same.
 
+* ``{cost}``  is a single character encoding a 6-bit integer
+  encoding the cost, which affects the number of rounds
+  used via the formula ``rounds=2**cost``. (cost is ``8`` or 13 in the example).
+
+* ``{salt}`` is eight characters drawn from ``[./0-9A-Za-z]``,
+  providing a 48-bit salt (``ohUJ.1sd`` in the example).
+
+* ``{checksum}`` is 22 characters drawn from the same set,
+  encoding the 128-bit checksum (``Fw09/bMaAQPTGDNi2BIUt1`` in the example).
+
+Algorithm
+=========
 PHPass uses a straightforward algorithm to calculate the checksum:
 
 * an initial result is generated from the MD5 digest of the salt string + the secret.
-* for ``2**rounds`` repetitions, a new result is created from the MD5 digest of the last result + the secret.
+* for ``2**cost`` rounds, a new result is created from the MD5 digest of the last result + the secret.
 * the last result is then encoded according to the format described above.
 
 Deviations
 ==========
-This implementation of phpass differs from the specification:
+This implementation of phpass differs from the specification in one way:
 
 * Unicode strings are encoded using UTF-8 before being passed into the algorithm.
   While the original code accepts passwords containing any 8-bit value,
diff --git a/docs/lib/passlib.hash.sha256_crypt.rst b/docs/lib/passlib.hash.sha256_crypt.rst
index d725d47..73ae43f 100644
--- a/docs/lib/passlib.hash.sha256_crypt.rst
+++ b/docs/lib/passlib.hash.sha256_crypt.rst
@@ -5,6 +5,12 @@
 .. module:: passlib.hash.sha526_crypt
     :synopsis: SHA-256 Crypt
 
-.. todo::
+This scheme is identical to :mod:`~passlib.hash.sha512_crypt` in almost every way,
+they are defined by the same specification and have the same design and structure,
+except the following differences:
 
-    write documentation
+* it uses the prefix ``$5$`` where the SHA-512-Crypt uses ``$6$``.
+* it uses SHA-256 as it's internal hash function instead of SHA-512.
+* it's output hash is correspondingly smaller.
+
+For details about this module, see :mod:`~passlib.hash.sha512_crypt`.
diff --git a/docs/lib/passlib.hash.sha512_crypt.rst b/docs/lib/passlib.hash.sha512_crypt.rst
index e9c9b51..3e115eb 100644
--- a/docs/lib/passlib.hash.sha512_crypt.rst
+++ b/docs/lib/passlib.hash.sha512_crypt.rst
@@ -5,12 +5,81 @@
 .. module:: passlib.hash.sha512_crypt
     :synopsis: SHA-512 Crypt
 
-This scheme is identical to :mod:`~passlib.hash.sha256_crypt` in almost every way,
-they are defined by the same specification and have the same design and structure,
-except the following differences:
+SHA-512 Crypt and SHA-256 Crypt were developed as a response
+to :mod:`~passlib.hash.bcrypt`. They are descendants of :mod:`~passlib.hash.md5_crypt`,
+and incorporate many changes: replaced MD5 with newer message digest algorithms,
+some internal cleanups in MD5-Crypt's rounds algorithm,
+and the introduction of a variable rounds parameter.
+SHA-512 Crypt is currently the default password hash for many systems
+(notably Linux), and has no known weaknesses.
 
-* it uses the prefix ``$6$`` where the SHA-256 Crypt uses ``$5$``.
-* it uses SHA-512 as it's internal hash function instead of SHA-256.
-* it's output hash is correspondingly larger.
+Usage
+=====
 
-For details about this module, see :mod:`~passlib.hash.sha256_crypt`.
+.. todo::
+
+    write usage instructions
+
+Functions
+=========
+.. autofunction:: genconfig
+.. autofunction:: genhash
+.. autofunction:: encrypt
+.. autofunction:: identify
+.. autofunction:: verify
+
+Format & Algorithm
+==================
+An example hash (of ``password``) is:
+
+    ``$6$rounds=40000$JvTuqzqw9bQ8iBl6$SxklIkW4gz00LvuOsKRCfNEllLciOqY/FSAwODHon45YTJEozmy.QAWiyVpuiq7XMTUMWbIWWEuQytdHkigcN/``.
+
+An sha512-crypt hash string has the format ``$6$rounds={rounds}${salt}${checksum}``, where:
+
+* ``$6$`` is the prefix used to identify sha512-crypt hashes,
+  following the :ref:`modular-crypt-format`
+
+* ``{rounds}`` is the decimal number of rounds to use (40000 in the example).
+
+* ``{salt}`` is 0-16 characters drawn from ``[./0-9A-Za-z]``, providing a
+  96-bit salt (``JvTuqzqw9bQ8iBl6`` in the example).
+
+* ``{checksum}`` is 86 characters drawn from the same set, encoding a 512-bit
+  checksum.
+
+  (``SxklIkW4gz00LvuOsKRCfNEllLciOqY/FSAwODHon45YTJEozmy.QAWiyVpuiq7XMTUMWbIWWEuQytdHkigcN/`` in the example).
+
+There is also an alternate format ``$6${salt}${checksum}``,
+which can be used when the rounds parameter is equal to 5000.
+
+The algorithm used by SHA512-Crypt is laid out in detail
+in the specification document linked to below.
+
+Deviations
+==========
+This implementation of sha512-crypt differs from the specification,
+and other implementations, in a few ways:
+
+* The specification does not specify how to deal with zero-padding
+  within the rounds portion of the hash. No existing examples
+  or test vectors have zero padding, and allowing it would
+  result in multiple encodings for the same configuration / hash.
+  To prevent this situation, PassLib will throw an error if the rounds in a hash
+  have leading zeros.
+
+* While the underlying algorithm technically allows salt strings
+  to contain any possible byte value besides ``\x00`` and ``$``,
+  this would conflict with many uses of sha512-crypt, such as within
+  unix ``/etc/shadow`` files. Futhermore, most unix systems
+  will only generate salts using the standard 64 characters listed above.
+  This implementation follows along with that, by strictly limiting
+  salt strings to the least common denominator, ``[./0-9A-Za-z]``.
+
+* Before generating a hash, PassLib encodes unicode passwords using UTF-8.
+  While the algorithm accepts passwords containing any 8-bit value
+  except for ``\x00``, it specifies no preference for encodings,
+  or for handling unicode strings.
+
+References
+==========
+* `sha-crypt specification <http://www.akkadia.org/drepper/sha-crypt.html>`_ - Ulrich Drepper's SHA-256/512-Crypt specification, reference implementation, and test vectors
diff --git a/docs/lib/passlib.utils.h64.rst b/docs/lib/passlib.utils.h64.rst
index 548720c..0475f34 100644
--- a/docs/lib/passlib.utils.h64.rst
+++ b/docs/lib/passlib.utils.h64.rst
@@ -19,8 +19,8 @@ and decoding strings in that format.
 
 .. note::
     It may *look* like bcrypt uses this scheme,
-    when in fact bcrypt uses the standard base64 encoding scheme,
-    but with ``+`` replaced with ``.``.
+    when in fact bcrypt uses yet another ordering,
+    which does not match hash64 or other base64 schemes.
 
 Constants
 =========