summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorEli Collins <elic@assurancetechnologies.com>2012-01-18 15:35:29 -0500
committerEli Collins <elic@assurancetechnologies.com>2012-01-18 15:35:29 -0500
commitd4ebb05e010b435c62eeb425059bc9d78e090660 (patch)
tree799ce222914000f13c9423090576df249a947f15
parentfc22ab711426e37816d862db5928892d718de53a (diff)
downloadpasslib-d4ebb05e010b435c62eeb425059bc9d78e090660.tar.gz
replaced utils.h64 module with utils.Base64Engine instance
* new utils.Base64Engine() provides flexible class for encoding arbitrary base64 charmaps. it should also be a bit faster than the old h64 module. * predefined 'h64' instance has mostly the same methods as the old h64 module which it takes the place off - so imports should be unaffected. (the only the exception of the xxx_dc_xxx methods, which now use the 'h64big' instance) * replaced utils._blowfish base64 encoding with custom Base64Engine instance to reduce code duplication. * more through unittests for Base64Engine.
-rw-r--r--CHANGES11
-rw-r--r--docs/lib/passlib.hash.bigcrypt.rst8
-rw-r--r--docs/lib/passlib.hash.bsdi_crypt.rst12
-rw-r--r--docs/lib/passlib.hash.crypt16.rst21
-rw-r--r--docs/lib/passlib.hash.des_crypt.rst12
-rw-r--r--docs/lib/passlib.hash.dlitz_pbkdf2_sha1.rst2
-rw-r--r--docs/lib/passlib.hash.md5_crypt.rst6
-rw-r--r--docs/lib/passlib.hash.pbkdf2_digest.rst6
-rw-r--r--docs/lib/passlib.hash.sha1_crypt.rst2
-rw-r--r--docs/lib/passlib.hash.sha512_crypt.rst2
-rw-r--r--docs/lib/passlib.utils.rst67
-rw-r--r--docs/modular_crypt_format.rst2
-rw-r--r--docs/password_hash_api.rst2
-rw-r--r--passlib/handlers/bcrypt.py7
-rw-r--r--passlib/handlers/pbkdf2.py12
-rw-r--r--passlib/handlers/scram.py10
-rw-r--r--passlib/tests/test_utils.py387
-rw-r--r--passlib/utils/__init__.py521
-rw-r--r--passlib/utils/_blowfish/__init__.py111
-rw-r--r--passlib/utils/h64.py286
20 files changed, 942 insertions, 545 deletions
diff --git a/CHANGES b/CHANGES
index 86f7e06..4958ebe 100644
--- a/CHANGES
+++ b/CHANGES
@@ -13,7 +13,7 @@ Release History
a user against the SCRAM protocol (:rfc:`5802`). It can also
be used to verify users in the same way as any other password
hash in Passlib, though it offers no particular advantages
- outside of this special case.
+ outside of this special case.
CryptContext
@@ -49,6 +49,13 @@ Release History
This affects all schemes in :mod:`passlib.hash` which support
multiple backends.
+ .. currentmodule:: passlib.utils
+
+ * :mod:`!passlib.utils.h64` has been replaced by an instance of the
+ new :class:`~passlib.utils.Base64Engine` class. This instance is
+ imported under the same name, and has (mostly) the same interface;
+ but should be faster, more flexible, and better unit-tested.
+
Other
* Passlib is now source-compatible with Python 2.5+ and Python 3,
@@ -144,7 +151,7 @@ Release History
.. currentmodule:: passlib.hash
* *bugfix:* :class:`django_des_crypt` now accepts all
- :mod:`Hash64 <passlib.utils.h64>` characters in it's salts;
+ :data:`hash64 <passlib.utils.h64>` characters in it's salts;
previously it accepted only lower-case hexidecimal characters [issue 22].
* Additional unittests added for all
diff --git a/docs/lib/passlib.hash.bigcrypt.rst b/docs/lib/passlib.hash.bigcrypt.rst
index 4106a71..e16c261 100644
--- a/docs/lib/passlib.hash.bigcrypt.rst
+++ b/docs/lib/passlib.hash.bigcrypt.rst
@@ -27,11 +27,11 @@ Format
An example hash (of the string ``passphrase``) is ``S/8NbAAlzbYO66hAa9XZyWy2``.
A bigcrypt hash string has the format :samp:`{salt}{checksum_1}{checksum_2...}{checksum_n}` for some integer :samp:`{n}>0`, where:
-* :samp:`{salt}` is the salt, stored as a 2 character :func:`hash64 <passlib.utils.h64.encode_int12>`-encoded
+* :samp:`{salt}` is the salt, stored as a 2 character :data:`hash64 <passlib.utils.h64>`-encoded
12-bit integer (``S/`` in the example).
* each :samp:`{checksum_i}` is a separate checksum, stored as an 11 character
- :func:`hash64 <passlib.utils.h64.encode_dc_int64>`-encoded 64-bit integer (``8NbAAlzbYO6`` and ``6hAa9XZyWy2``
+ :data:`hash64-big <passlib.utils.h64big>`-encoded 64-bit integer (``8NbAAlzbYO6`` and ``6hAa9XZyWy2``
in the example).
* the integer :samp:`n` (the number of checksums) is determined by the formula
@@ -54,7 +54,7 @@ The bigcrypt algorithm is designed to re-use the original des-crypt algorithm:
4. The 2 character salt string is decoded to a 12-bit integer salt value;
The salt string uses little-endian
- :func:`hash64 <passlib.utils.h64.decode_int12>` encoding.
+ :data:`hash64 <passlib.utils.h64>` encoding.
5. 25 repeated rounds of modified DES encryption are performed;
starting with a null input block,
@@ -68,7 +68,7 @@ The bigcrypt algorithm is designed to re-use the original des-crypt algorithm:
lsb-padded with 2 zero bits.
7. The resulting 66-bit integer is encoded in big-endian order
- using the :func:`hash 64 <passlib.utils.h64.encode_int>` format.
+ using the :data:`hash64-big <passlib.utils.h64big>` format.
This forms the first checksum segment.
8. For each additional block of 8 bytes in the padded password (from step 2),
diff --git a/docs/lib/passlib.hash.bsdi_crypt.rst b/docs/lib/passlib.hash.bsdi_crypt.rst
index 464cd67..6a60f79 100644
--- a/docs/lib/passlib.hash.bsdi_crypt.rst
+++ b/docs/lib/passlib.hash.bsdi_crypt.rst
@@ -44,7 +44,7 @@ A bsdi_crypt hash string consists of a 21 character string of the form :samp:`_{
All characters except the underscore prefix are drawn from ``[./0-9A-Za-z]``.
* ``_`` - the underscore is used to distinguish this scheme from others, such as des-crypt.
-* :samp:`{rounds}` is the number of rounds, stored as a 4 character :mod:`hash64 <passlib.utils.h64>`-encoded 24-bit integer (``EQ0.`` in the example).
+* :samp:`{rounds}` is the number of rounds, stored as a 4 character :data:`hash64 <passlib.utils.h64>`-encoded 24-bit integer (``EQ0.`` in the example).
* :samp:`{salt}` is the salt, stored as as a 4 character hash64-encoded 24-bit integer (``jzhS`` in the example).
* :samp:`{checksum}` is the checksum, stored as an 11 character hash64-encoded 64-bit integer (``VeUyoSqLupI`` in the example).
@@ -60,12 +60,12 @@ The checksum is formed by a modified version of the DES cipher in encrypt mode:
1. Given a password string, a salt string, and rounds string.
2. The 4 character rounds string is decoded to a 24-bit integer rounds value;
- The rounds string uses little-endian
- :func:`hash64 <passlib.utils.h64.decode_int24>` encoding.
+ The rounds string uses little-endian :data:`hash64 <passlib.utils.h64>`
+ encoding.
3. The 4 character salt string is decoded to a 24-bit integer salt value;
- The salt string uses little-endian
- :func:`hash64 <passlib.utils.h64.decode_int24>` encoding.
+ The salt string uses little-endian :data:`hash64 <passlib.utils.h64>`
+ encoding.
4. The password is NULL-padded on the end to the smallest non-zero multiple of 8 bytes.
@@ -95,7 +95,7 @@ The checksum is formed by a modified version of the DES cipher in encrypt mode:
lsb-padded with 2 zero bits.
9. The resulting 66-bit integer is encoded in big-endian order
- using the :func:`hash 64 <passlib.utils.h64.encode_int>` format.
+ using the :data:`hash64-big <passlib.utils.h64big>` format.
Security Issues
===============
diff --git a/docs/lib/passlib.hash.crypt16.rst b/docs/lib/passlib.hash.crypt16.rst
index 9fd2a21..2cdd5eb 100644
--- a/docs/lib/passlib.hash.crypt16.rst
+++ b/docs/lib/passlib.hash.crypt16.rst
@@ -32,12 +32,13 @@ Format
An example hash (of the string ``passphrase``) is ``aaX/UmCcBrceQ0kQGGWKTbuE``.
A crypt16 hash string has the format :samp:`{salt}{checksum_1}{checksum_2}`, where:
-* :samp:`{salt}` is the salt, stored as a 2 character :func:`hash64 <passlib.utils.h64.encode_int12>`-encoded
- 12-bit integer (``aa`` in the example).
+* :samp:`{salt}` is the salt, stored as a 2 character
+ :data:`hash64 <passlib.utils.h64>`-encoded 12-bit integer (``aa`` in the
+ example).
* each :samp:`{checksum_i}` is a separate checksum, stored as an 11 character
- :func:`hash64 <passlib.utils.h64.encode_dc_int64>`-encoded 64-bit integer (``X/UmCcBrceQ`` and ``0kQGGWKTbuE``
- in the example).
+ :data:`hash64-big <passlib.utils.h64big>`-encoded 64-bit integer
+ (``X/UmCcBrceQ`` and ``0kQGGWKTbuE`` in the example).
.. rst-class:: html-toggle
@@ -48,8 +49,8 @@ The crypt16 algorithm uses a weakened version of the des-crypt algorithm:
1. Given a password string and a salt string.
2. The 2 character salt string is decoded to a 12-bit integer salt value;
- The salt string uses little-endian
- :func:`hash64 <passlib.utils.h64.decode_int12>` encoding.
+ The salt string uses little-endian :data:`hash64 <passlib.utils.h64>`
+ encoding.
3. If the password is larger than 16 bytes, the end is truncated to 16 bytes.
If the password is smaller than 16 bytes, the end is NULL padded to 16 bytes.
@@ -63,16 +64,16 @@ The crypt16 algorithm uses a weakened version of the des-crypt algorithm:
starting with a null input block,
and using the 56-bit integer from step 4 as the DES key.
- The salt value from step 2 is used to to mutate the normal
- DES encrypt operation by swapping bits :samp:`{i}` and :samp:`{i}+24`
- in the DES E-Box output if and only if bit :samp:`{i}` is set in
+ The salt value from step 2 is used to to mutate the normal
+ DES encrypt operation by swapping bits :samp:`{i}` and :samp:`{i}+24`
+ in the DES E-Box output if and only if bit :samp:`{i}` is set in
the salt value.
6. The 64-bit result of the last round of step 5 is then
lsb-padded with 2 zero bits.
7. The resulting 66-bit integer is encoded in big-endian order
- using the :func:`hash 64 <passlib.utils.h64.encode_int>` format.
+ using the :data:`hash64-big <passlib.utils.h64big>` format.
This is the first checksum segment.
8. The second checksum segment is created by repeating
diff --git a/docs/lib/passlib.hash.des_crypt.rst b/docs/lib/passlib.hash.des_crypt.rst
index 06d4f85..ba5202e 100644
--- a/docs/lib/passlib.hash.des_crypt.rst
+++ b/docs/lib/passlib.hash.des_crypt.rst
@@ -39,7 +39,7 @@ Interface
Format
======
A des-crypt hash string consists of 13 characters, drawn from ``[./0-9A-Za-z]``.
-The first 2 characters form a :mod:`hash64 <passlib.utils.h64>`-encoded
+The first 2 characters form a :data:`hash64 <passlib.utils.h64>`-encoded
12 bit integer used as the salt, with the remaining characters
forming a hash64-encoded 64-bit integer checksum.
@@ -57,8 +57,8 @@ The checksum is formed by a modified version of the DES cipher in encrypt mode:
1. Given a password string and a salt string.
2. The 2 character salt string is decoded to a 12-bit integer salt value;
- The salt string uses little-endian
- :func:`hash64 <passlib.utils.h64.decode_int12>` encoding.
+ The salt string uses little-endian :data:`hash64 <passlib.utils.h64>`
+ encoding.
3. If the password is less than 8 bytes, it's NULL padded at the end to 8 bytes.
@@ -83,8 +83,8 @@ The checksum is formed by a modified version of the DES cipher in encrypt mode:
6. The 64-bit result of the last round of step 5 is then
lsb-padded with 2 zero bits.
-7. The resulting 66-bit integer is encoded in big-endian order
- using the :func:`hash 64 <passlib.utils.h64.encode_int>` format.
+7. The resulting 66-bit integer is encoded in big-endian order using the
+ :data:`hash64-big <passlib.utils.h64big>` format.
Security Issues
===============
@@ -115,7 +115,7 @@ This implementation of des-crypt differs from others in a few ways:
* Restricted salt string character set:
The underlying algorithm expects salt strings to use the
- :mod:`hash64 <passlib.utils.h64>` character set to encode
+ :data:`hash64 <passlib.utils.HASH64_CHARS>` character set to encode
a 12-bit integer. Many implementations of des-crypt will
accept a salt containing other characters, but
vary wildly in how they are handled, including errors and implementation-specific value mappings.
diff --git a/docs/lib/passlib.hash.dlitz_pbkdf2_sha1.rst b/docs/lib/passlib.hash.dlitz_pbkdf2_sha1.rst
index 794e506..1dfd832 100644
--- a/docs/lib/passlib.hash.dlitz_pbkdf2_sha1.rst
+++ b/docs/lib/passlib.hash.dlitz_pbkdf2_sha1.rst
@@ -51,7 +51,7 @@ where:
(``.pPqsEwHD7MiECU0`` in the example).
* :samp:`{checksum}` is 32 characters, which encode
- the resulting 24-byte PBKDF2 derived key using :func:`~passlib.utils.adapted_b64_encode`
+ the resulting 24-byte PBKDF2 derived key using :func:`~passlib.utils.ab64_encode`
(``b8TQ5AMQemtlaSgegw5Je.JBE3QQhLbO`` in the example).
In order to generate the checksum, the password is first encoded into UTF-8 if it's unicode.
diff --git a/docs/lib/passlib.hash.md5_crypt.rst b/docs/lib/passlib.hash.md5_crypt.rst
index d033723..a75187f 100644
--- a/docs/lib/passlib.hash.md5_crypt.rst
+++ b/docs/lib/passlib.hash.md5_crypt.rst
@@ -81,7 +81,7 @@ The MD5-Crypt algorithm [#f1]_ calculates a checksum as follows:
9. Add the password to digest A.
-10. Add the constant string ``$1$`` to digest A.
+10. Add the constant string ``$1$`` to digest A.
(The Apache variant of MD5-Crypt uses ``$apr1$`` instead,
this is the only change made by this variant).
@@ -122,7 +122,7 @@ The MD5-Crypt algorithm [#f1]_ calculates a checksum as follows:
following order: ``12,6,0,13,7,1,14,8,2,15,9,3,5,10,4,11``.
18. Encode the resulting 16 byte string into a 22 character
- :mod:`hash 64 <passlib.utils.h64.encode_bytes>`-encoded string
+ :data:`hash64 <passlib.utils.h64>`-encoded string
(the 2 msb bits encoded by the last hash64 character are used as 0 padding).
This results in the portion of the md5 crypt hash string referred to as :samp:`{checksum}` in the format section.
@@ -151,7 +151,7 @@ PassLib's implementation of md5-crypt differs from the reference implementation
The underlying algorithm can unambigously handle salt strings
which contain any possible byte value besides ``\x00`` and ``$``.
However, PassLib strictly limits salts to the
- :mod:`hash 64 <passlib.utils.h64>` character set,
+ :data:`hash64 <passlib.utils.HASH64_CHARS>` character set,
as nearly all implementations of md5-crypt generate
and expect salts containing those characters,
but may have unexpected behaviors for other character values.
diff --git a/docs/lib/passlib.hash.pbkdf2_digest.rst b/docs/lib/passlib.hash.pbkdf2_digest.rst
index 6cd3ea6..e732c5d 100644
--- a/docs/lib/passlib.hash.pbkdf2_digest.rst
+++ b/docs/lib/passlib.hash.pbkdf2_digest.rst
@@ -89,10 +89,10 @@ follow the same format, :samp:`$pbkdf2-{digest}${rounds}${salt}${checksum}`.
this is encoded as a positive decimal number with no zero-padding
(``6400`` in the example).
-* :samp:`{salt}` - this is the :func:`adapted base64 encoding <passlib.utils.adapted_b64_encode>`
+* :samp:`{salt}` - this is the :func:`adapted base64 encoding <passlib.utils.ab64_encode>`
of the raw salt bytes passed into the PBKDF2 function.
-* :samp:`{checksum}` - this is the :func:`adapted base64 encoding <passlib.utils.adapted_b64_encode>`
+* :samp:`{checksum}` - this is the :func:`adapted base64 encoding <passlib.utils.ab64_encode>`
of the raw derived key bytes returned from the PBKDF2 function.
Each scheme uses output size of it's specific :samp:`{digest}`
as the size of the raw derived key. This is enlarged
@@ -104,7 +104,7 @@ The password is encoded into UTF-8 if not already encoded,
and passed through :func:`~passlib.utils.pbkdf2.pbkdf2`
along with the decoded salt, the number of rounds,
and a prf built from HMAC + the respective message digest.
-The result is then encoded using :func:`~passlib.utils.adapted_b64_encode`.
+The result is then encoded using :func:`~passlib.utils.ab64_encode`.
.. rubric:: Footnotes
diff --git a/docs/lib/passlib.hash.sha1_crypt.rst b/docs/lib/passlib.hash.sha1_crypt.rst
index cae9c42..fa80a6b 100644
--- a/docs/lib/passlib.hash.sha1_crypt.rst
+++ b/docs/lib/passlib.hash.sha1_crypt.rst
@@ -80,7 +80,7 @@ in a few ways:
The underlying algorithm can unambigously handle salt strings
which contain any possible byte value besides ``\x00`` and ``$``.
However, PassLib strictly limits salts to the
- :mod:`hash 64 <passlib.utils.h64>` character set,
+ :data:`hash64 <passlib.utils.HASH64_CHARS>` character set,
as nearly all implementations of sha1-crypt generate
and expect salts containing those characters.
diff --git a/docs/lib/passlib.hash.sha512_crypt.rst b/docs/lib/passlib.hash.sha512_crypt.rst
index d9aa1ea..f9e438c 100644
--- a/docs/lib/passlib.hash.sha512_crypt.rst
+++ b/docs/lib/passlib.hash.sha512_crypt.rst
@@ -91,7 +91,7 @@ and other implementations, in a few ways:
The underlying algorithm can unambigously handle salt strings
which contain any possible byte value besides ``\x00`` and ``$``.
However, PassLib strictly limits salts to the
- :mod:`hash 64 <passlib.utils.h64>` character set,
+ :data:`hash64 <passlib.utils.HASH64_CHARS>` character set,
as nearly all implementations of sha512-crypt generate
and expect salts containing those characters,
but may have unexpected behaviors for other character values.
diff --git a/docs/lib/passlib.utils.rst b/docs/lib/passlib.utils.rst
index 580c830..b052f5a 100644
--- a/docs/lib/passlib.utils.rst
+++ b/docs/lib/passlib.utils.rst
@@ -50,6 +50,72 @@ Bytes Manipulation
.. autofunction:: xor_bytes
.. autofunction:: consteq
+Base64 Encoding
+===============
+
+Base64Engine Class
+------------------
+Passlib has to deal with a number of different Base64 encodings,
+with varying endianness, as well as wildly different value <-> character
+mappings. This is all encapsulated in the :class:`Base64Engine` class,
+which provides common encoding actions for an arbitrary base64-style encoding
+scheme. There are also a couple of predefined instances which are commonly
+used by the hashes in Passlib.
+
+.. autoclass:: Base64Engine
+
+Common Character Maps
+---------------------
+.. data:: BASE64_CHARS
+
+ Character map used by standard MIME-compatible Base64 encoding scheme.
+
+.. data:: HASH64_CHARS
+
+ Base64 character map used by a number of hash formats;
+ the ordering is wildly different from the standard base64 character map.
+ (see :data:`h64` for details).
+
+.. data:: BCRYPT_CHARS
+
+ Base64 character map used by :class:`~passlib.hash.bcrypt`.
+ The ordering is wildly different from both the standard base64 character map,
+ and the common hash64 character map.
+
+Predefined Instances
+--------------------
+.. data:: h64
+ Predefined instance of :class:`Base64Engine` which uses
+ the :data:`HASH64_CHARS` character map and little-endian encoding.
+
+ This encoding system appears to have originated with
+ :class:`~passlib.hash.des_crypt`, but is used by
+ :class:`~passlib.hash.md5-crypt`, `~passlib.hash.sha256_crypt`,
+ and others. Within Passlib, this encoding is referred as ``hash64`` encoding
+ to distinguish it from normal base64 and other encodings.
+
+.. data:: h64big
+ Predefined variant of :data:`h64` which uses big-endian encoding.
+ This is mainly used by :class:`~passlib.hash.des_crypt`.
+
+.. note::
+
+ *changed in Passlib 1.6:* the :mod:`passlib.utils.h64` module used by
+ Passlib <= 1.5 has been replaced by the the ``h64`` and ``h64big``
+ instances; but the interface remains mostly unchanged.
+
+..
+
+ .. data:: AB64_CHARS
+
+ Variant of standard Base64 character map used by some
+ custom Passlib hashes (see :func:`ab64_encode`).
+
+ Other
+ -----
+ .. autofunction:: ab64_encode
+ .. autofunction:: ab64_decode
+
Randomness
==========
.. data:: rng
@@ -83,7 +149,6 @@ There are also a few sub modules which provide additional utility functions:
:maxdepth: 1
passlib.utils.des
- passlib.utils.h64
passlib.utils.md4
passlib.utils.pbkdf2
passlib.utils.handlers
diff --git a/docs/modular_crypt_format.rst b/docs/modular_crypt_format.rst
index bc96ffe..f40ceee 100644
--- a/docs/modular_crypt_format.rst
+++ b/docs/modular_crypt_format.rst
@@ -84,7 +84,7 @@ by the modular crypt format hashes found in passlib:
use ascii letters, numbers, ``.``, and ``/``
to provide base64 encoding of their raw data,
though the exact character value assignments vary between hashes
- (see :mod:`passlib.utils.h64`).
+ (see :data:`passlib.utils.h64`).
4. Hash schemes should put their "checksum" portion
at the end of the hash, preferrably separated
diff --git a/docs/password_hash_api.rst b/docs/password_hash_api.rst
index 572c8a9..a015e73 100644
--- a/docs/password_hash_api.rst
+++ b/docs/password_hash_api.rst
@@ -455,7 +455,7 @@ the following attributes are usually exposed.
string containing list of all characters which are allowed
to be specified in salt parameter.
for most :ref:`MCF <modular-crypt-format>` hashes,
- this is equal to :data:`passlib.utils.h64.CHARS`.
+ this is equal to :data:`passlib.utils.HASH64_CHARS`.
this must be a :class:`!unicode` string if the salt is encoded,
or (rarely) :class:`!bytes` if the salt is manipulating as unencoded raw bytes.
diff --git a/passlib/handlers/bcrypt.py b/passlib/handlers/bcrypt.py
index 62d875e..f651dd0 100644
--- a/passlib/handlers/bcrypt.py
+++ b/passlib/handlers/bcrypt.py
@@ -28,7 +28,7 @@ except ImportError: #pragma: no cover - though should run whole suite w/o bcrypt
bcryptor_engine = None
#libs
from passlib.utils import safe_os_crypt, classproperty, handlers as uh, \
- h64, to_native_str, rng, getrandstr, bytes
+ h64, to_native_str, rng, getrandstr, bytes, BCRYPT_CHARS as BCHARS
from passlib.utils.compat import unicode
#pkg
@@ -44,9 +44,8 @@ def _load_builtin():
if _builtin_bcrypt is None:
from passlib.utils._blowfish import raw_bcrypt as _builtin_bcrypt
-# base64 character->value mapping used by bcrypt.
-# this is same as as H64_CHARS, but the positions are different.
-BCHARS = u("./ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789")
+# BCHARS imported from passlib.utils
+# BCHARS = u("./ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789")
# last bcrypt salt char should have 4 padding bits set to 0.
# thus, only the following chars are allowed:
diff --git a/passlib/handlers/pbkdf2.py b/passlib/handlers/pbkdf2.py
index 76d9966..3fbf1dc 100644
--- a/passlib/handlers/pbkdf2.py
+++ b/passlib/handlers/pbkdf2.py
@@ -10,7 +10,7 @@ import logging; log = logging.getLogger(__name__)
from warnings import warn
#site
#libs
-from passlib.utils import adapted_b64_encode, adapted_b64_decode, \
+from passlib.utils import ab64_encode, ab64_decode, \
handlers as uh, to_native_str, to_unicode, bytes, b
from passlib.utils.compat import unicode
from passlib.utils.pbkdf2 import pbkdf2
@@ -72,8 +72,8 @@ class Pbkdf2DigestHandler(uh.HasRounds, uh.HasRawSalt, uh.HasRawChecksum, uh.Gen
int_rounds = int(rounds)
if rounds != unicode(int_rounds): #forbid zero padding, etc.
raise ValueError("invalid %s hash" % (cls.name,))
- raw_salt = adapted_b64_decode(salt.encode("ascii"))
- raw_chk = adapted_b64_decode(chk.encode("ascii")) if chk else None
+ raw_salt = ab64_decode(salt.encode("ascii"))
+ raw_chk = ab64_decode(chk.encode("ascii")) if chk else None
return cls(
rounds=int_rounds,
salt=raw_salt,
@@ -82,9 +82,9 @@ class Pbkdf2DigestHandler(uh.HasRounds, uh.HasRawSalt, uh.HasRawChecksum, uh.Gen
)
def to_string(self, withchk=True):
- salt = adapted_b64_encode(self.salt).decode("ascii")
+ salt = ab64_encode(self.salt).decode("ascii")
if withchk and self.checksum:
- chk = adapted_b64_encode(self.checksum).decode("ascii")
+ chk = ab64_encode(self.checksum).decode("ascii")
hash = u('%s%d$%s$%s') % (self.ident, self.rounds, salt, chk)
else:
hash = u('%s%d$%s') % (self.ident, self.rounds, salt)
@@ -331,7 +331,7 @@ class dlitz_pbkdf2_sha1(uh.HasRounds, uh.HasSalt, uh.GenericHandler):
secret = secret.encode("utf-8")
salt = self.to_string(withchk=False, native=False).encode("ascii")
result = pbkdf2(secret, salt, self.rounds, 24, "hmac-sha1")
- return adapted_b64_encode(result).decode("ascii")
+ return ab64_encode(result).decode("ascii")
#=========================================================
#eoc
diff --git a/passlib/handlers/scram.py b/passlib/handlers/scram.py
index a0c03f0..f35a967 100644
--- a/passlib/handlers/scram.py
+++ b/passlib/handlers/scram.py
@@ -23,7 +23,7 @@ import logging; log = logging.getLogger(__name__)
from warnings import warn
#site
#libs
-from passlib.utils import adapted_b64_encode, adapted_b64_decode, xor_bytes, \
+from passlib.utils import ab64_encode, ab64_decode, xor_bytes, \
handlers as uh, to_native_str, to_unicode, consteq, saslprep
from passlib.utils.compat import unicode, bytes, u, b, iteritems, itervalues, \
PY2, PY3
@@ -331,7 +331,7 @@ class scram(uh.HasRounds, uh.HasRawSalt, uh.HasRawChecksum, uh.GenericHandler):
# decode salt
try:
- salt = adapted_b64_decode(salt_str.encode("ascii"))
+ salt = ab64_decode(salt_str.encode("ascii"))
except TypeError:
raise ValueError("malformed scram hash")
@@ -346,7 +346,7 @@ class scram(uh.HasRounds, uh.HasRawSalt, uh.HasRawChecksum, uh.GenericHandler):
for pair in chk_str.split(","):
alg, digest = pair.split("=")
try:
- chkmap[alg] = adapted_b64_decode(digest.encode("ascii"))
+ chkmap[alg] = ab64_decode(digest.encode("ascii"))
except TypeError:
raise ValueError("malformed scram hash")
else:
@@ -364,13 +364,13 @@ class scram(uh.HasRounds, uh.HasRawSalt, uh.HasRawChecksum, uh.GenericHandler):
)
def to_string(self, withchk=True):
- salt = adapted_b64_encode(self.salt)
+ salt = ab64_encode(self.salt)
if PY3:
salt = salt.decode("ascii")
chkmap = self.checksum
if withchk and chkmap:
chk_str = ",".join(
- "%s=%s" % (alg, to_native_str(adapted_b64_encode(chkmap[alg])))
+ "%s=%s" % (alg, to_native_str(ab64_encode(chkmap[alg])))
for alg in self.algs
)
else:
diff --git a/passlib/tests/test_utils.py b/passlib/tests/test_utils.py
index 5396a89..ca6cf90 100644
--- a/passlib/tests/test_utils.py
+++ b/passlib/tests/test_utils.py
@@ -504,129 +504,338 @@ b(""" 0000000000000000 0000000000000000 8CA64DE9C1B123A7
# though des-crypt builtin backend test should thump it well enough
#=========================================================
-#hash64
+# base64engine
#=========================================================
-class H64_Test(TestCase):
- "test H64 codec functions"
- case_prefix = "H64 codec"
-
+class _Base64Test(TestCase):
+ "common tests for all Base64Engine instances"
#=========================================================
- #test basic encode/decode
+ # class attrs
#=========================================================
- encoded_bytes = [
- #test lengths 0..6 to ensure tail is encoded properly
- (b(""),b("")),
- (b("\x55"),b("J/")),
- (b("\x55\xaa"),b("Jd8")),
- (b("\x55\xaa\x55"),b("JdOJ")),
- (b("\x55\xaa\x55\xaa"),b("JdOJe0")),
- (b("\x55\xaa\x55\xaa\x55"),b("JdOJeK3")),
- (b("\x55\xaa\x55\xaa\x55\xaa"),b("JdOJeKZe")),
- #test padding bits are null
- (b("\x55\xaa\x55\xaf"),b("JdOJj0")), # len = 1 mod 3
- (b("\x55\xaa\x55\xaa\x5f"),b("JdOJey3")), # len = 2 mod 3
- ]
+ # Base64Engine instance to test
+ engine = None
- decode_padding_bytes = [
- #len = 2 mod 4 -> 2 msb of last digit is padding
- (b(".."), b("\x00")), # . = h64.CHARS[0b000000]
- (b(".0"), b("\x80")), # 0 = h64.CHARS[0b000010]
- (b(".2"), b("\x00")), # 2 = h64.CHARS[0b000100]
- (b(".U"), b("\x00")), # U = h64.CHARS[0b100000]
-
- #len = 3 mod 4 -> 4 msb of last digit is padding
- (b("..."), b("\x00\x00")),
- (b("..6"), b("\x00\x80")), # 6 = h64.CHARS[0b001000]
- (b("..E"), b("\x00\x00")), # E = h64.CHARS[0b010000]
- (b("..U"), b("\x00\x00")),
- ]
+ # pairs of (raw, encoded) bytes to test - should encode/decode correctly
+ encoded_data = None
- def test_encode_bytes(self):
- for source, result in self.encoded_bytes:
- out = h64.encode_bytes(source)
- self.assertEqual(out, result)
+ # tuples of (encoded, value, bits) for known integer encodings
+ encoded_ints = None
- def test_decode_bytes(self):
- for result, source in self.encoded_bytes:
- out = h64.decode_bytes(source)
- self.assertEqual(out, result)
-
- #wrong size (1 % 4)
- self.assertRaises(ValueError, h64.decode_bytes, b('abcde'))
-
- self.assertRaises(TypeError, h64.decode_bytes, u('abcd'))
+ # invalid encoded byte
+ bad_byte = b("?")
- def test_encode_int(self):
- self.assertEqual(h64.encode_int(63, 11, True), b('..........z'))
- self.assertEqual(h64.encode_int(63, 11), b('z..........'))
+ # helper to generate bytemap-specific strings
+ def m(self, *offsets):
+ "generate byte string from offsets"
+ return b("").join(self.engine.bytemap[o:o+1] for o in offsets)
- self.assertRaises(ValueError, h64.encode_int64, -1)
-
- def test_decode_int(self):
- self.assertEqual(h64.decode_int64(b('...........')), 0)
-
- self.assertRaises(ValueError, h64.decode_int12, b('a?'))
- self.assertRaises(ValueError, h64.decode_int24, b('aaa?'))
- self.assertRaises(ValueError, h64.decode_int64, b('aaa?aaa?aaa'))
- self.assertRaises(ValueError, h64.decode_dc_int64, b('aaa?aaa?aaa'))
+ #=========================================================
+ # test encode_bytes
+ #=========================================================
+ def test_encode_bytes(self):
+ "test encode_bytes() against reference inputs"
+ engine = self.engine
+ encode = engine.encode_bytes
+ for raw, encoded in self.encoded_data:
+ result = encode(raw)
+ self.assertEqual(result, encoded, "encode %r:" % (raw,))
+
+ def test_encode_bytes_bad(self):
+ "test encode_bytes() with bad input"
+ engine = self.engine
+ encode = engine.encode_bytes
+ self.assertRaises(TypeError, encode, u('\x00'))
+ self.assertRaises(TypeError, encode, None)
- self.assertRaises(TypeError, h64.decode_int12, u('a')*2)
- self.assertRaises(TypeError, h64.decode_int24, u('a')*4)
- self.assertRaises(TypeError, h64.decode_int64, u('a')*11)
- self.assertRaises(TypeError, h64.decode_dc_int64, u('a')*11)
+ #=========================================================
+ # test decode_bytes
+ #=========================================================
+ def test_decode_bytes(self):
+ "test decode_bytes() against reference inputs"
+ engine = self.engine
+ decode = engine.decode_bytes
+ for raw, encoded in self.encoded_data:
+ result = decode(encoded)
+ self.assertEqual(result, raw, "decode %r:" % (encoded,))
def test_decode_bytes_padding(self):
- for source, result in self.decode_padding_bytes:
- out = h64.decode_bytes(source)
- self.assertEqual(out, result)
- self.assertRaises(TypeError, h64.decode_bytes, u('..'))
-
- def test_decode_int6(self):
- self.assertEqual(h64.decode_int6(b('.')),0)
- self.assertEqual(h64.decode_int6(b('z')),63)
- self.assertRaises(ValueError, h64.decode_int6, b('?'))
- self.assertRaises(TypeError, h64.decode_int6, u('?'))
+ "test decode_bytes() ignores padding bits"
+ bchr = (lambda v: bytes([v])) if PY3 else chr
+ engine = self.engine
+ m = self.m
+ decode = engine.decode_bytes
+ BNULL = b("\x00")
+
+ # length == 2 mod 4: 4 bits of padding
+ self.assertEqual(decode(m(0,0)), BNULL)
+ for i in range(0,6):
+ if engine.big: # 4 lsb padding
+ correct = BNULL if i < 4 else bchr(1<<(i-4))
+ else: # 4 msb padding
+ correct = bchr(1<<(i+6)) if i < 2 else BNULL
+ self.assertEqual(decode(m(0,1<<i)), correct, "%d/4 bits:" % i)
+
+ # length == 3 mod 4: 2 bits of padding
+ self.assertEqual(decode(m(0,0,0)), BNULL*2)
+ for i in range(0,6):
+ if engine.big: # 2 lsb are padding
+ correct = BNULL if i < 2 else bchr(1<<(i-2))
+ else: # 2 msg are padding
+ correct = bchr(1<<(i+4)) if i < 4 else BNULL
+ self.assertEqual(decode(m(0,0,1<<i)), BNULL + correct,
+ "%d/2 bits:" % i)
+
+ def test_decode_bytes_bad(self):
+ "test decode_bytes() with bad input"
+ engine = self.engine
+ decode = engine.decode_bytes
+
+ # wrong size (1 % 4)
+ self.assertRaises(ValueError, decode, engine.bytemap[:5])
+
+ # wrong char
+ self.assertTrue(self.bad_byte not in engine.bytemap)
+ self.assertRaises(ValueError, decode, self.bad_byte*4)
+
+ # wrong type
+ self.assertRaises(TypeError, decode, engine.charmap[:4])
+ self.assertRaises(TypeError, decode, None)
- def test_encode_int6(self):
- self.assertEqual(h64.encode_int6(0),b('.'))
- self.assertEqual(h64.encode_int6(63),b('z'))
- self.assertRaises(ValueError, h64.encode_int6, -1)
- self.assertRaises(ValueError, h64.encode_int6, 64)
+ #=========================================================
+ # encode_bytes+decode_bytes
+ #=========================================================
+ def test_codec(self):
+ "test encode_bytes/decode_bytes against random data"
+ engine = self.engine
+ from passlib.utils import getrandbytes, getrandstr
+ saw_zero = False
+ for i in irange(500):
+ #
+ # test raw -> encode() -> decode() -> raw
+ #
+
+ # generate some random bytes
+ size = random.randint(1 if saw_zero else 0, 12)
+ if not size:
+ saw_zero = True
+ enc_size = (4*size+2)//3
+ raw = getrandbytes(random, size)
+
+ # encode them, check invariants
+ encoded = engine.encode_bytes(raw)
+ self.assertEqual(len(encoded), enc_size)
+
+ # make sure decode returns original
+ result = engine.decode_bytes(encoded)
+ self.assertEqual(result, raw)
+
+ #
+ # test encoded -> decode() -> encode() -> encoded
+ #
+
+ # generate some random encoded data
+ if size % 4 == 1:
+ size += random.choice([-1,1,2])
+ raw_size = 3*size//4
+ encoded = getrandstr(random, engine.bytemap, size)
+
+ # decode them, check invariants
+ raw = engine.decode_bytes(encoded)
+ self.assertEqual(len(raw), raw_size, "encoded %d:" % size)
+
+ # make sure encode returns original (barring padding bits)
+ result = engine.encode_bytes(raw)
+ if size % 4:
+ self.assertEqual(result[:-1], encoded[:-1])
+ else:
+ self.assertEqual(result, encoded)
#=========================================================
- #test transposed encode/decode
+ # test transposed encode/decode - encoding independant
#=========================================================
- encode_transposed = [
+ # NOTE: these tests assume normal encode/decode has been tested elsewhere.
+
+ transposed = [
+ # orig, result, transpose map
(b("\x33\x22\x11"), b("\x11\x22\x33"),[2,1,0]),
(b("\x22\x33\x11"), b("\x11\x22\x33"),[1,2,0]),
]
- encode_transposed_dups = [
+ transposed_dups = [
+ # orig, result, transpose projection
(b("\x11\x11\x22"), b("\x11\x22\x33"),[0,0,1]),
]
def test_encode_transposed_bytes(self):
- for result, input, offsets in self.encode_transposed + self.encode_transposed_dups:
- tmp = h64.encode_transposed_bytes(input, offsets)
- out = h64.decode_bytes(tmp)
+ "test encode_transposed_bytes()"
+ engine = self.engine
+ for result, input, offsets in self.transposed + self.transposed_dups:
+ tmp = engine.encode_transposed_bytes(input, offsets)
+ out = engine.decode_bytes(tmp)
self.assertEqual(out, result)
def test_decode_transposed_bytes(self):
- for input, result, offsets in self.encode_transposed:
- tmp = h64.encode_bytes(input)
- out = h64.decode_transposed_bytes(tmp, offsets)
+ "test decode_transposed_bytes()"
+ engine = self.engine
+ for input, result, offsets in self.transposed:
+ tmp = engine.encode_bytes(input)
+ out = engine.decode_transposed_bytes(tmp, offsets)
self.assertEqual(out, result)
def test_decode_transposed_bytes_bad(self):
- for input, _, offsets in self.encode_transposed_dups:
- tmp = h64.encode_bytes(input)
- self.assertRaises(TypeError, h64.decode_transposed_bytes, tmp, offsets)
+ "test decode_transposed_bytes() fails if map is a one-way"
+ engine = self.engine
+ for input, _, offsets in self.transposed_dups:
+ tmp = engine.encode_bytes(input)
+ self.assertRaises(TypeError, engine.decode_transposed_bytes, tmp,
+ offsets)
+
+ #=========================================================
+ # test 6bit handling
+ #=========================================================
+ def check_int_pair(self, bits, encoded_pairs):
+ "helper to check encode_intXX & decode_intXX functions"
+ engine = self.engine
+ encode = getattr(engine, "encode_int%s" % bits)
+ decode = getattr(engine, "decode_int%s" % bits)
+ pad = -bits % 6
+ chars = (bits+pad)/6
+ upper = 1<<bits
+
+ # test encode func
+ for value, encoded in encoded_pairs:
+ self.assertEqual(encode(value), encoded)
+ self.assertRaises(ValueError, encode, -1)
+ self.assertRaises(ValueError, encode, upper)
+
+ # test decode func
+ for value, encoded in encoded_pairs:
+ self.assertEqual(decode(encoded), value, "encoded %r:" % (encoded,))
+ m = self.m
+ self.assertRaises(ValueError, decode, m(0)*(chars+1))
+ self.assertRaises(ValueError, decode, m(0)*(chars-1))
+ self.assertRaises(ValueError, decode, self.bad_byte*chars)
+ self.assertRaises(TypeError, decode, engine.charmap[0])
+ self.assertRaises(TypeError, decode, None)
+
+ # do random testing.
+ from passlib.utils import getrandbytes, getrandstr
+ for i in irange(100):
+ # generate random value, encode, and then decode
+ value = random.randint(0, upper-1)
+ encoded = encode(value)
+ self.assertEqual(len(encoded), chars)
+ self.assertEqual(decode(encoded), value)
+
+ # generate some random encoded data, decode, then encode.
+ encoded = getrandstr(random, engine.bytemap, chars)
+ value = decode(encoded)
+ self.assertGreaterEqual(value, 0, "decode %r out of bounds:" % encoded)
+ self.assertLess(value, upper, "decode %r out of bounds:" % encoded)
+ result = encode(value)
+ if pad:
+ self.assertEqual(result[:-2], encoded[:-2])
+ else:
+ self.assertEqual(result, encoded)
+
+ def test_int6(self):
+ engine = self.engine
+ m = self.m
+ self.check_int_pair(6, [(0, m(0)), (63, m(63))])
+
+ def test_int12(self):
+ engine = self.engine
+ m = self.m
+ self.check_int_pair(12,[(0, m(0,0)),
+ (63, m(0,63) if engine.big else m(63,0)), (0xFFF, m(63,63))])
+
+ def test_int24(self):
+ engine = self.engine
+ m = self.m
+ self.check_int_pair(24,[(0, m(0,0,0,0)),
+ (63, m(0,0,0,63) if engine.big else m(63,0,0,0)),
+ (0xFFFFFF, m(63,63,63,63))])
+
+ def test_int64(self):
+ # NOTE: this isn't multiple of 6, it has 2 padding bits appended
+ # before encoding.
+ engine = self.engine
+ m = self.m
+ self.check_int_pair(64, [(0, m(0,0,0,0, 0,0,0,0, 0,0,0)),
+ (63, m(0,0,0,0, 0,0,0,0, 0,3,60) if engine.big else
+ m(63,0,0,0, 0,0,0,0, 0,0,0)),
+ ((1<<64)-1, m(63,63,63,63, 63,63,63,63, 63,63,60) if engine.big
+ else m(63,63,63,63, 63,63,63,63, 63,63,15))])
+
+ def test_encoded_ints(self):
+ "test against reference integer encodings"
+ if not self.encoded_ints:
+ raise self.skipTests("none defined for class")
+ engine = self.engine
+ for encoded, value, bits in self.encoded_ints:
+ encode = getattr(engine, "encode_int%d" % bits)
+ decode = getattr(engine, "decode_int%d" % bits)
+ self.assertEqual(encode(value), encoded)
+ self.assertEqual(decode(encoded), value)
#=========================================================
- #TODO: test other h64 methods
+ # eoc
#=========================================================
+# NOTE: testing H64 & H64Big should be sufficient to verify
+# that Base64Engine() works in general.
+from passlib.utils import h64, h64big
+
+class H64_Test(_Base64Test):
+ "test H64 codec functions"
+ engine = h64
+ case_prefix = "h64 codec"
+
+ encoded_data = [
+ #test lengths 0..6 to ensure tail is encoded properly
+ (b(""),b("")),
+ (b("\x55"),b("J/")),
+ (b("\x55\xaa"),b("Jd8")),
+ (b("\x55\xaa\x55"),b("JdOJ")),
+ (b("\x55\xaa\x55\xaa"),b("JdOJe0")),
+ (b("\x55\xaa\x55\xaa\x55"),b("JdOJeK3")),
+ (b("\x55\xaa\x55\xaa\x55\xaa"),b("JdOJeKZe")),
+
+ #test padding bits are null
+ (b("\x55\xaa\x55\xaf"),b("JdOJj0")), # len = 1 mod 3
+ (b("\x55\xaa\x55\xaa\x5f"),b("JdOJey3")), # len = 2 mod 3
+ ]
+
+ encoded_ints = [
+ ("z.", 63, 12),
+ (".z", 4032, 12),
+ ]
+
+class H64Big_Test(_Base64Test):
+ "test H64Big codec functions"
+ engine = h64big
+ case_prefix = "h64big codec"
+
+ encoded_data = [
+ #test lengths 0..6 to ensure tail is encoded properly
+ (b(""),b("")),
+ (b("\x55"),b("JE")),
+ (b("\x55\xaa"),b("JOc")),
+ (b("\x55\xaa\x55"),b("JOdJ")),
+ (b("\x55\xaa\x55\xaa"),b("JOdJeU")),
+ (b("\x55\xaa\x55\xaa\x55"),b("JOdJeZI")),
+ (b("\x55\xaa\x55\xaa\x55\xaa"),b("JOdJeZKe")),
+
+ #test padding bits are null
+ (b("\x55\xaa\x55\xaf"),b("JOdJfk")), # len = 1 mod 3
+ (b("\x55\xaa\x55\xaa\x5f"),b("JOdJeZw")), # len = 2 mod 3
+ ]
+
+ encoded_ints = [
+ (".z", 63, 12),
+ ("z.", 4032, 12),
+ ]
+
#=========================================================
#test md4
#=========================================================
diff --git a/passlib/utils/__init__.py b/passlib/utils/__init__.py
index be6ce48..d2bb85c 100644
--- a/passlib/utils/__init__.py
+++ b/passlib/utils/__init__.py
@@ -50,6 +50,11 @@ __all__ = [
#byte manipulation
"xor_bytes",
+ # base64 helpers
+ "BASE64_CHARS", "HASH64_CHARS", "BCRYPT_CHARS", "AB64_CHARS",
+ "Base64Engine", "h64", "h64big",
+ "ab64_encode", "ab64_decode",
+
#random
'tick',
'rng',
@@ -839,42 +844,540 @@ else:
return bjoin(chr(ord(l) ^ ord(r)) for l, r in zip(left, right))
#=================================================================================
-#alt base64 encoding
+# base64-variant encoding
#=================================================================================
+
+class Base64Engine(object):
+ """provides routines for encoding/decoding base64 data using
+ arbitrary character mappings, selectable endianness, etc.
+
+ Raw Bytes <-> Encoded Bytes
+ ===========================
+ .. automethod:: encode_bytes
+ .. automethod:: decode_bytes
+ .. automethod:: encode_transposed_bytes
+ .. automethod:: decode_transposed_bytes
+
+ Integers <-> Encoded Bytes
+ ==========================
+ .. automethod:: encode_int6
+ .. automethod:: decode_int6
+
+ .. automethod:: encode_int12
+ .. automethod:: decode_int12
+
+ .. automethod:: encode_int24
+ .. automethod:: decode_int24
+
+ .. automethod:: encode_int64
+ .. automethod:: decode_int64
+
+ Informational Attributes
+ ========================
+ .. attribute:: charmap
+ unicode string containing list of characters used in encoding;
+ position in string matches 6bit value of character.
+
+ .. attribute:: bytemap
+ bytes version of :attr:`charmap`
+
+ .. attribute:: big
+ boolean flag indicating this using big-endian encoding.
+ """
+
+ #=============================================================
+ # instance attrs
+ #=============================================================
+ # public config
+ bytemap = None # charmap as bytes
+ big = None # little or big endian
+
+ # filled in by init based on charmap.
+ # encode: maps 6bit value -> byte_elem, decode: the reverse.
+ # byte_elem is 1-byte under py2, and 0-255 int under py3.
+ _encode64 = None
+ _decode64 = None
+
+ # helpers filled in by init based on endianness
+ _encode_bytes = None # throws IndexError if bad value (shouldn't happen)
+ _decode_bytes = None # throws KeyError if bad char.
+
+ #=============================================================
+ # init
+ #=============================================================
+ def __init__(self, charmap, big=False):
+ # validate charmap, generate encode64/decode64 helper functions.
+ if isinstance(charmap, unicode):
+ charmap = charmap.encode("latin-1")
+ elif not isinstance(charmap, bytes):
+ raise TypeError("charmap must be unicode/bytes string")
+ if len(charmap) != 64:
+ raise ValueError("charmap must be 64 characters in length")
+ if len(set(charmap)) != 64:
+ raise ValueError("charmap must not contain duplicate characters")
+ self.bytemap = charmap
+ self._encode64 = charmap.__getitem__
+ lookup = dict((value, idx) for idx, value in enumerate(charmap))
+ self._decode64 = lookup.__getitem__
+
+ # validate big, set appropriate helper functions.
+ self.big = big
+ if big:
+ self._encode_bytes = self._encode_bytes_big
+ self._decode_bytes = self._decode_bytes_big
+ else:
+ self._encode_bytes = self._encode_bytes_little
+ self._decode_bytes = self._decode_bytes_little
+
+ # TODO: support padding character
+ ##if padding is not None:
+ ## if isinstance(padding, unicode):
+ ## padding = padding.encode("latin-1")
+ ## elif not isinstance(padding, bytes):
+ ## raise TypeError("padding char must be unicode or bytes")
+ ## if len(padding) != 1:
+ ## raise ValueError("padding must be single character")
+ ##self.padding = padding
+
+ @property
+ def charmap(self):
+ "charmap as unicode"
+ return self.bytemap.decode("latin-1")
+
+ #=============================================================
+ # encoding byte strings
+ #=============================================================
+ def encode_bytes(self, source):
+ """encode bytes to engine's specific base64 variant.
+ :arg source: byte string to encode.
+ :returns: byte string containing encoded data.
+ """
+ if not isinstance(source, bytes):
+ raise TypeError("source must be bytes, not %s" % (type(source),))
+ chunks, tail = divmod(len(source), 3)
+ if PY3:
+ next_value = iter(source).__next__
+ else:
+ next_value = (ord(elem) for elem in source).next
+ gen = self._encode_bytes(next_value, chunks, tail)
+ out = bjoin_elems(imap(self._encode64, gen))
+ ##if tail:
+ ## padding = self.padding
+ ## if padding:
+ ## out += padding * (3-tail)
+ return out
+
+ def _encode_bytes_little(self, next_value, chunks, tail):
+ "helper used by encode_bytes() to handle little-endian encoding"
+ #
+ # output bit layout:
+ #
+ # first byte: v1 543210
+ #
+ # second byte: v1 ....76
+ # +v2 3210..
+ #
+ # third byte: v2 ..7654
+ # +v3 10....
+ #
+ # fourth byte: v3 765432
+ #
+ idx = 0
+ while idx < chunks:
+ v1 = next_value()
+ v2 = next_value()
+ v3 = next_value()
+ yield v1 & 0x3f
+ yield ((v2 & 0x0f)<<2)|(v1>>6)
+ yield ((v3 & 0x03)<<4)|(v2>>4)
+ yield v3>>2
+ idx += 1
+ if tail:
+ v1 = next_value()
+ if tail == 1:
+ # note: 4 msb of last byte are padding
+ yield v1 & 0x3f
+ yield v1>>6
+ else:
+ assert tail == 2
+ # note: 2 msb of last byte are padding
+ v2 = next_value()
+ yield v1 & 0x3f
+ yield ((v2 & 0x0f)<<2)|(v1>>6)
+ yield v2>>4
+
+ def _encode_bytes_big(self, next_value, chunks, tail):
+ "helper used by encode_bytes() to handle big-endian encoding"
+ #
+ # output bit layout:
+ #
+ # first byte: v1 765432
+ #
+ # second byte: v1 10....
+ # +v2 ..7654
+ #
+ # third byte: v2 3210..
+ # +v3 ....76
+ #
+ # fourth byte: v3 543210
+ #
+ idx = 0
+ while idx < chunks:
+ v1 = next_value()
+ v2 = next_value()
+ v3 = next_value()
+ yield v1>>2
+ yield ((v1&0x03)<<4)|(v2>>4)
+ yield ((v2&0x0f)<<2)|(v3>>6)
+ yield v3 & 0x3f
+ idx += 1
+ if tail:
+ v1 = next_value()
+ if tail == 1:
+ # note: 4 lsb of last byte are padding
+ yield v1>>2
+ yield (v1&0x03)<<4
+ else:
+ assert tail == 2
+ # note: 2 lsb of last byte are padding
+ v2 = next_value()
+ yield v1>>2
+ yield ((v1&0x03)<<4)|(v2>>4)
+ yield ((v2&0x0f)<<2)
+
+ #=============================================================
+ # decoding byte strings
+ #=============================================================
+
+ def decode_bytes(self, source):
+ """decode bytes from engine's specific base64 variant.
+ :arg source: byte string to decode.
+ :returns: byte string containing decoded data.
+ """
+ if not isinstance(source, bytes):
+ raise TypeError("source must be bytes, not %s" % (type(source),))
+ ##padding = self.padding
+ ##if padding:
+ ## # TODO: add padding size check?
+ ## source = source.rstrip(padding)
+ chunks, tail = divmod(len(source), 4)
+ if tail == 1:
+ #only 6 bits left, can't encode a whole byte!
+ raise ValueError("input string length cannot be == 1 mod 4")
+ if PY3:
+ next_value = imap(self._decode64, source).__next__
+ else:
+ next_value = imap(self._decode64, source).next
+ try:
+ return bjoin_ints(self._decode_bytes(next_value, chunks, tail))
+ except KeyError:
+ err = exc_err()
+ raise ValueError("invalid character: %r" % (err.args[0],))
+
+ def _decode_bytes_little(self, next_value, chunks, tail):
+ "helper used by decode_bytes() to handle little-endian encoding"
+ #
+ # input bit layout:
+ #
+ # first byte: v1 ..543210
+ # +v2 10......
+ #
+ # second byte: v2 ....5432
+ # +v3 3210....
+ #
+ # third byte: v3 ......54
+ # +v4 543210..
+ #
+ idx = 0
+ while idx < chunks:
+ v1 = next_value()
+ v2 = next_value()
+ v3 = next_value()
+ v4 = next_value()
+ yield v1 | ((v2 & 0x3) << 6)
+ yield (v2>>2) | ((v3 & 0xF) << 4)
+ yield (v3>>4) | (v4<<2)
+ idx += 1
+ if tail:
+ # tail is 2 or 3
+ v1 = next_value()
+ v2 = next_value()
+ yield v1 | ((v2 & 0x3) << 6)
+ #NOTE: if tail == 2, 4 msb of v2 are ignored (should be 0)
+ if tail == 3:
+ #NOTE: 2 msb of v3 are ignored (should be 0)
+ v3 = next_value()
+ yield (v2>>2) | ((v3 & 0xF) << 4)
+
+ def _decode_bytes_big(self, next_value, chunks, tail):
+ "helper used by decode_bytes() to handle big-endian encoding"
+ #
+ # input bit layout:
+ #
+ # first byte: v1 543210..
+ # +v2 ......54
+ #
+ # second byte: v2 3210....
+ # +v3 ....5432
+ #
+ # third byte: v3 10......
+ # +v4 ..543210
+ #
+ idx = 0
+ while idx < chunks:
+ v1 = next_value()
+ v2 = next_value()
+ v3 = next_value()
+ v4 = next_value()
+ yield (v1<<2) | (v2>>4)
+ yield ((v2&0xF)<<4) | (v3>>2)
+ yield ((v3&0x3)<<6) | v4
+ idx += 1
+ if tail:
+ # tail is 2 or 3
+ v1 = next_value()
+ v2 = next_value()
+ yield (v1<<2) | (v2>>4)
+ #NOTE: if tail == 2, 4 lsb of v2 are ignored (should be 0)
+ if tail == 3:
+ #NOTE: 2 lsb of v3 are ignored (should be 0)
+ v3 = next_value()
+ yield ((v2&0xF)<<4) | (v3>>2)
+
+ #=============================================================
+ # transposed encoding/decoding
+ #=============================================================
+ def encode_transposed_bytes(self, source, offsets):
+ "encode byte string, first transposing source using offset list"
+ if not isinstance(source, bytes):
+ raise TypeError("source must be bytes, not %s" % (type(source),))
+ tmp = bjoin_elems(source[off] for off in offsets)
+ return self.encode_bytes(tmp)
+
+ def decode_transposed_bytes(self, source, offsets):
+ "decode byte string, then reverse transposition described by offset list"
+ # NOTE: if transposition does not use all bytes of source,
+ # the original can't be recovered... and bjoin_elems() will throw
+ # an error because 1+ values in <buf> will be None.
+ tmp = self.decode_bytes(source)
+ buf = [None] * len(offsets)
+ for off, char in zip(offsets, tmp):
+ buf[off] = char
+ return bjoin_elems(buf)
+
+ #=============================================================
+ # integer decoding helpers - mainly used by des_crypt family
+ #=============================================================
+ def _decode_int(self, source, bits):
+ """decode hash64 string -> integer
+
+ :arg source: base64 string to decode.
+ :arg bits: number of bits in resulting integer.
+
+ :raises ValueError:
+ * if the string contains invalid base64 characters.
+ * if the string is not long enough - it must be at least
+ ``int(ceil(bits/6))`` in length.
+
+ :returns:
+ a integer in the range ``0 <= n < 2**bits``
+ """
+ if not isinstance(source, bytes):
+ raise TypeError("source must be bytes, not %s" % (type(source),))
+ big = self.big
+ pad = -bits % 6
+ chars = (bits+pad)/6
+ if len(source) != chars:
+ raise ValueError("source must be %d chars" % (chars,))
+ decode = self._decode64
+ out = 0
+ try:
+ for c in source if big else reversed(source):
+ out = (out<<6) + decode(c)
+ except KeyError:
+ raise ValueError("invalid character in string: %r" % (c,))
+ if pad:
+ # strip padding bits
+ if big:
+ out >>= pad
+ else:
+ out &= (1<<bits)-1
+ return out
+
+ #---------------------------------------------
+ # optimized versions for common integer sizes
+ #---------------------------------------------
+
+ def decode_int6(self, source):
+ "decode single character -> 6 bit integer"
+ if not isinstance(source, bytes):
+ raise TypeError("source must be bytes, not %s" % (type(source),))
+ if len(source) != 1:
+ raise ValueError("source must be exactly 1 byte")
+ try:
+ return self._decode64(source)
+ except KeyError:
+ raise ValueError("invalid character")
+
+ def decode_int12(self, source):
+ "decodes 2 char string -> 12-bit integer (little-endian order)"
+ if not isinstance(source, bytes):
+ raise TypeError("source must be bytes, not %s" % (type(source),))
+ if len(source) != 2:
+ raise ValueError("source must be exactly 2 bytes")
+ decode = self._decode64
+ try:
+ if self.big:
+ return decode(source[1]) + (decode(source[0])<<6)
+ else:
+ return decode(source[0]) + (decode(source[1])<<6)
+ except KeyError:
+ raise ValueError("invalid character")
+
+ def decode_int24(self, source):
+ "decodes 4 char string -> 24-bit integer (little-endian order)"
+ if not isinstance(source, bytes):
+ raise TypeError("source must be bytes, not %s" % (type(source),))
+ if len(source) != 4:
+ raise ValueError("source must be exactly 4 bytes")
+ decode = self._decode64
+ try:
+ if self.big:
+ return decode(source[3]) + (decode(source[2])<<6)+ \
+ (decode(source[1])<<12) + (decode(source[0])<<18)
+ else:
+ return decode(source[0]) + (decode(source[1])<<6)+ \
+ (decode(source[2])<<12) + (decode(source[3])<<18)
+ except KeyError:
+ raise ValueError("invalid character")
+
+ def decode_int64(self, source):
+ """decode 11 char base64 string -> 64-bit integer
+
+ this format is used primarily by des-crypt & variants to encode
+ the DES output value used as a checksum.
+ """
+ return self._decode_int(source, 64)
+
+ #=============================================================
+ # integer encoding helpers - mainly used by des_crypt family
+ #=============================================================
+ def _encode_int(self, value, bits):
+ """encode integer into base64 format
+
+ :arg value: non-negative integer to encode
+ :arg bits: number of bits to encode
+
+ :returns:
+ a string of length ``int(ceil(bits/6.0))``.
+ """
+ if value < 0:
+ raise ValueError("value cannot be negative")
+ pad = -bits % 6
+ bits += pad
+ if self.big:
+ itr = irange(bits-6, -6, -6)
+ # shift to add lsb padding.
+ value <<= pad
+ else:
+ itr = irange(0, bits, 6)
+ # padding is msb, so no change needed.
+ return bjoin_elems(imap(self._encode64,
+ ((value>>off) & 0x3f for off in itr)))
+
+ #---------------------------------------------
+ # optimized versions for common integer sizes
+ #---------------------------------------------
+
+ def encode_int6(self, value):
+ "encodes 6-bit integer -> single hash64 character"
+ if value < 0 or value > 63:
+ raise ValueError("value out of range")
+ if PY3:
+ return self.bytemap[value:value+1]
+ else:
+ return self._encode64(value)
+
+ def encode_int12(self, value):
+ "encodes 12-bit integer -> 2 char string"
+ if value < 0 or value > 0xFFF:
+ raise ValueError("value out of range")
+ raw = [value & 0x3f, (value>>6) & 0x3f]
+ if self.big:
+ raw = reversed(raw)
+ return bjoin_elems(imap(self._encode64, raw))
+
+ def encode_int24(self, value):
+ "encodes 24-bit integer -> 4 char string"
+ if value < 0 or value > 0xFFFFFF:
+ raise ValueError("value out of range")
+ raw = [value & 0x3f, (value>>6) & 0x3f,
+ (value>>12) & 0x3f, (value>>18) & 0x3f]
+ if self.big:
+ raw = reversed(raw)
+ return bjoin_elems(imap(self._encode64, raw))
+
+ def encode_int64(self, value):
+ """encode 64-bit integer -> 11 char hash64 string
+
+ this format is used primarily by des-crypt & variants to encode
+ the DES output value used as a checksum.
+ """
+ if value < 0 or value > 0xffffffffffffffff:
+ raise ValueError("value out of range")
+ return self._encode_int(value, 64)
+
+ #=============================================================
+ # eof
+ #=============================================================
+
+# common charmaps
+BASE64_CHARS = u("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/")
+AB64_CHARS = u("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789./")
+HASH64_CHARS = u("./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz")
+BCRYPT_CHARS = u("./ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789")
+
+# common variants
+h64 = Base64Engine(HASH64_CHARS)
+h64big = Base64Engine(HASH64_CHARS, big=True)
+
+#=============================================================================
+# adapted-base64 encoding
+#=============================================================================
_A64_ALTCHARS = b("./")
_A64_STRIP = b("=\n")
_A64_PAD1 = b("=")
_A64_PAD2 = b("==")
-def adapted_b64_encode(data):
+def ab64_encode(data):
"""encode using variant of base64
the output of this function is identical to b64_encode,
except that it uses ``.`` instead of ``+``,
and omits trailing padding ``=`` and whitepsace.
- it is primarily used for by passlib's custom pbkdf2 hashes.
+ it is primarily used by Passlib's custom pbkdf2 hashes.
"""
return b64encode(data, _A64_ALTCHARS).strip(_A64_STRIP)
-def adapted_b64_decode(data, sixthree="."):
+def ab64_decode(data):
"""decode using variant of base64
the input of this function is identical to b64_decode,
except that it uses ``.`` instead of ``+``,
and should not include trailing padding ``=`` or whitespace.
- it is primarily used for by passlib's custom pbkdf2 hashes.
+ it is primarily used by Passlib's custom pbkdf2 hashes.
"""
- off = len(data) % 4
+ off = len(data) & 3
if off == 0:
return b64decode(data, _A64_ALTCHARS)
- elif off == 1:
- raise ValueError("invalid bas64 input")
elif off == 2:
return b64decode(data + _A64_PAD2, _A64_ALTCHARS)
- else:
+ elif off == 3:
return b64decode(data + _A64_PAD1, _A64_ALTCHARS)
+ else: # off == 1
+ raise ValueError("invalid base64 input")
#=================================================================================
#randomness
diff --git a/passlib/utils/_blowfish/__init__.py b/passlib/utils/_blowfish/__init__.py
index f0a0923..41914dd 100644
--- a/passlib/utils/_blowfish/__init__.py
+++ b/passlib/utils/_blowfish/__init__.py
@@ -54,7 +54,7 @@ released under the BSD license::
from itertools import chain
import struct
#pkg
-from passlib.utils import rng, getrandbytes, bytes, bord
+from passlib.utils import Base64Engine, BCRYPT_CHARS, rng, getrandbytes, bytes, bord
from passlib.utils.compat import b, unicode, u
from passlib.utils.compat.aliases import BytesIO
from passlib.utils._blowfish.unrolled import BlowfishEngine
@@ -77,109 +77,8 @@ BCRYPT_CDATA = [
# struct used to encode ciphertext as digest (last output byte discarded)
digest_struct = struct.Struct(">6I")
-#=========================================================
-#base64 encoding
-#=========================================================
-
-# Table for Base64 encoding
-CHARS = b("./ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789")
-CHARSIDX = dict( (c, i) for i, c in enumerate(CHARS))
-
-def encode_base64(d):
- """Encode a byte array using bcrypt's slightly-modified base64 encoding scheme.
-
- Note that this is *not* compatible with the standard MIME-base64 encoding.
-
- :param d:
- the bytes to encode
- :returns:
- the bytes as encoded using bcrypt's base64
- """
- if isinstance(d, unicode):
- d = d.encode("utf-8")
- #ensure ord() returns something w/in 0..255
-
- rs = BytesIO()
- write = rs.write
- dlen = len(d)
- didx = 0
-
- while True:
- #encode first byte -> 1 byte (6 bits) w/ 2 bits left over
- if didx >= dlen:
- break
- c1 = ord(d[didx])
- write(CHARS[(c1 >> 2) & 0x3f])
- c1 = (c1 & 0x03) << 4
- didx += 1
-
- #encode 2 bits + second byte -> 1 byte (6 bits) w/ 4 bits left over
- if didx >= dlen:
- write(CHARS[c1])
- break
- c2 = ord(d[didx])
- write(CHARS[c1 | (c2 >> 4) ])
- c2 = (c2 & 0x0f) << 2
- didx += 1
-
- #encode 4 bits left over + third byte -> 1 byte (6 bits) w/ 2 bits left over
- if didx >= dlen:
- write(CHARS[c2])
- break
- c3 = ord(d[didx])
- write(CHARS[c2 | (c3 >> 6)])
- write(CHARS[c3 & 0x3f])
- didx += 1
-
- return rs.getvalue()
-
-def decode_base64(s):
- """Decode bytes encoded using bcrypt's base64 scheme.
-
- :param s:
- string of bcrypt-base64 encoded bytes
-
- :returns:
- string of decoded bytes
-
- :raises ValueError:
- if invalid values are passed in
- """
- rs = BytesIO()
- write = rs.write
- slen = len(s)
- sidx = 0
-
- def char64(c):
- "look up 6 bit value in table"
- try:
- return CHARSIDX[c]
- except KeyError:
- raise ValueError("invalid chars in base64 string")
-
- while True:
-
- #decode byte 1 + byte 2 -> 1 byte + 4 bits left over
- if sidx >= slen-1:
- break
- c2 = char64(s[sidx+1])
- write(chr((char64(s[sidx]) << 2) | (c2 >> 4)))
- sidx += 2
-
- #decode 4 bits left over + 3rd byte -> 1 byte + 2 bits left over
- if sidx >= slen:
- break
- c3 = char64(s[sidx])
- write(chr(((c2 & 0x0f) << 4) | (c3 >> 2)))
- sidx += 1
-
- #decode 2 bits left over + 4th byte -> 1 byte
- if sidx >= slen:
- break
- write(chr(((c3 & 0x03) << 6) | char64(s[sidx])))
- sidx += 1
-
- return rs.getvalue()
+# base64 variant used by bcrypt
+bcrypt64 = Base64Engine(BCRYPT_CHARS, big=True)
#=========================================================
#base bcrypt helper
@@ -213,7 +112,7 @@ def raw_bcrypt(password, ident, salt, log_rounds):
# decode & validate salt
assert isinstance(salt, bytes)
- salt = decode_base64(salt)
+ salt = bcrypt64.decode_bytes(salt)
if len(salt) < 16:
raise ValueError("Missing salt bytes")
elif len(salt) > 16:
@@ -260,7 +159,7 @@ def raw_bcrypt(password, ident, salt, log_rounds):
data[i], data[i+1] = engine.repeat_encipher(data[i], data[i+1], 64)
i += 2
raw = digest_struct.pack(*data)[:-1]
- return encode_base64(raw)
+ return bcrypt64.encode_bytes(raw)
#=========================================================
#eof
diff --git a/passlib/utils/h64.py b/passlib/utils/h64.py
deleted file mode 100644
index cf889d3..0000000
--- a/passlib/utils/h64.py
+++ /dev/null
@@ -1,286 +0,0 @@
-"""passlib.utils.h64 - hash64 encoding helpers"""
-#=================================================================================
-#imports
-#=================================================================================
-#core
-import logging; log = logging.getLogger(__name__)
-#site
-#pkg
-from passlib.utils import bytes, bjoin, bord, bjoin_elems, bjoin_ints
-from passlib.utils.compat import irange, u, PY3
-#local
-__all__ = [
- "CHARS",
-
- "decode_bytes", "encode_bytes",
- "decode_transposed_bytes", "encode_transposed_bytes",
-
- "decode_int6", "encode_int6",
- "decode_int12", "encode_int12"
- "decode_int18", "encode_int18"
- "decode_int24", "encode_int24",
- "decode_int64", "encode_int64",
- "decode_int", "encode_int",
-]
-
-#=================================================================================
-#6 bit value <-> char mapping, and other internal helpers
-#=================================================================================
-
-#: hash64 char sequence
-CHARS = u("./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz")
-BCHARS = CHARS.encode("ascii")
-
-#: encode int -> hash64 char as efficiently as possible, w/ minimal checking
-if PY3:
- _encode_6bit = lambda v: BCHARS[v:v+1]
-else:
- _encode_6bit = BCHARS.__getitem__
-
-#: decode hash64 char -> int as efficiently as possible, w/ minimal checking
-_CHARIDX = dict((_encode_6bit(i),i) for i in irange(64))
-_decode_6bit = _CHARIDX.__getitem__ # char -> int
-
-#for py3, enhance _CHARIDX to also support int value of bytes
-if PY3:
- _CHARIDX.update((v,i) for i,v in enumerate(BCHARS))
-
-#=================================================================================
-#encode offsets from buffer - used by md5_crypt, sha_crypt, et al
-#=================================================================================
-
-def _encode_bytes_helper(source):
- #FIXME: do something much more efficient here.
- # can't quite just use base64 and then translate chars,
- # since this scheme is little-endian.
- end = len(source)
- tail = end % 3
- end -= tail
- idx = 0
- while idx < end:
- v1 = bord(source[idx])
- v2 = bord(source[idx+1])
- v3 = bord(source[idx+2])
- yield encode_int24(v1 + (v2<<8) + (v3<<16))
- idx += 3
- if tail:
- v1 = bord(source[idx])
- if tail == 1:
- #NOTE: 4 msb of int are always 0
- yield encode_int12(v1)
- else:
- #NOTE: 2 msb of int are always 0
- v2 = bord(source[idx+1])
- yield encode_int18(v1 + (v2<<8))
-
-def encode_bytes(source):
- "encode byte string to h64 format"
- if not isinstance(source, bytes):
- raise TypeError("source must be bytes, not %s" % (type(source),))
- return bjoin(_encode_bytes_helper(source))
-
-def _decode_bytes_helper(source):
- end = len(source)
- tail = end % 4
- if tail == 1:
- #only 6 bits left, can't encode a whole byte!
- raise ValueError("input string length cannot be == 1 mod 4")
- end -= tail
- idx = 0
- while idx < end:
- v = decode_int24(source[idx:idx+4])
- yield bjoin_ints([v&0xff, (v>>8)&0xff, v>>16])
- idx += 4
- if tail:
- if tail == 2:
- #NOTE: 2 msb of int are ignored (should be 0)
- v = decode_int12(source[idx:idx+2])
- yield bjoin_ints([v&0xff])
- else:
- #NOTE: 4 msb of int are ignored (should be 0)
- v = decode_int18(source[idx:idx+3])
- yield bjoin_ints([v&0xff, (v>>8)&0xff])
-
-def decode_bytes(source):
- "decode h64 format into byte string"
- if not isinstance(source, bytes):
- raise TypeError("source must be bytes, not %s" % (type(source),))
- return bjoin(_decode_bytes_helper(source))
-
-def encode_transposed_bytes(source, offsets):
- "encode byte string to h64 format, using offset list to transpose elements"
- if not isinstance(source, bytes):
- raise TypeError("source must be bytes, not %s" % (type(source),))
- #XXX: could make this a dup of encode_bytes(), which directly accesses source[offsets[idx]],
- # but speed isn't *that* critical for this function
- tmp = bjoin_elems(source[off] for off in offsets)
- return encode_bytes(tmp)
-
-def decode_transposed_bytes(source, offsets):
- "decode h64 format into byte string, then undoing specified transposition; inverse of :func:`encode_transposed_bytes`"
- #NOTE: if transposition does not use all bytes of source, original can't be recovered
- tmp = decode_bytes(source)
- buf = [None] * len(offsets)
- for off, char in zip(offsets, tmp):
- buf[off] = char
- return bjoin_elems(buf)
-
-#=================================================================================
-# int <-> b64 string, used by des_crypt, bsdi_crypt
-#=================================================================================
-
-def decode_int6(source):
- "decodes single hash64 character -> 6-bit integer"
- if not isinstance(source, bytes):
- raise TypeError("source must be bytes, not %s" % (type(source),))
- try:
- return _decode_6bit(source)
- except KeyError:
- raise ValueError("invalid character")
-
-def encode_int6(value):
- "encodes 6-bit integer -> single hash64 character"
- if value < 0 or value > 63:
- raise ValueError("value out of range")
- return _encode_6bit(value)
-
-#---------------------------------------------------------------------
-
-def decode_int12(source):
- "decodes 2 char hash64 string -> 12-bit integer (little-endian order)"
- #NOTE: this is optimized form of decode_int(value) for 4 chars
- if not isinstance(source, bytes):
- raise TypeError("source must be bytes, not %s" % (type(source),))
- try:
- return (_decode_6bit(source[1])<<6)+_decode_6bit(source[0])
- except KeyError:
- raise ValueError("invalid character")
-
-def encode_int12(value):
- "encodes 12-bit integer -> 2 char hash64 string (little-endian order)"
- #NOTE: this is optimized form of encode_int(value,2)
- return _encode_6bit(value & 0x3f) + _encode_6bit((value>>6) & 0x3f)
-
-#---------------------------------------------------------------------
-def decode_int18(source):
- "decodes 3 char hash64 string -> 18-bit integer (little-endian order)"
- #NOTE: this is optimized form of decode_int(value) for 3 chars
- if not isinstance(source, bytes):
- raise TypeError("source must be bytes, not %s" % (type(source),))
- return (
- _decode_6bit(source[0]) +
- (_decode_6bit(source[1])<<6) +
- (_decode_6bit(source[2])<<12)
- )
-
-def encode_int18(value):
- "encodes 18-bit integer -> 3 char hash64 string (little-endian order)"
- #NOTE: this is optimized form of encode_int(value,3)
- return (
- _encode_6bit(value & 0x3f) +
- _encode_6bit((value>>6) & 0x3f) +
- _encode_6bit((value>>12) & 0x3f)
- )
-
-#---------------------------------------------------------------------
-
-def decode_int24(source):
- "decodes 4 char hash64 string -> 24-bit integer (little-endian order)"
- #NOTE: this is optimized form of decode_int(source) for 4 chars
- if not isinstance(source, bytes):
- raise TypeError("source must be bytes, not %s" % (type(source),))
- try:
- return _decode_6bit(source[0]) +\
- (_decode_6bit(source[1])<<6)+\
- (_decode_6bit(source[2])<<12)+\
- (_decode_6bit(source[3])<<18)
- except KeyError:
- raise ValueError("invalid character")
-
-def encode_int24(value):
- "encodes 24-bit integer -> 4 char hash64 string (little-endian order)"
- #NOTE: this is optimized form of encode_int(value,4)
- return _encode_6bit(value & 0x3f) + \
- _encode_6bit((value>>6) & 0x3f) + \
- _encode_6bit((value>>12) & 0x3f) + \
- _encode_6bit((value>>18) & 0x3f)
-
-#---------------------------------------------------------------------
-
-def decode_int64(source):
- "decodes 11 char hash64 string -> 64-bit integer (little-endian order; 2 msb assumed to be padding)"
- return decode_int(source)
-
-def encode_int64(value):
- "encodes 64-bit integer -> 11 char hash64 string (little-endian order; 2 msb of 0's added as padding)"
- return encode_int(value, 11)
-
-def decode_dc_int64(source):
- """decode 11 char hash64 string -> 64-bit integer (big-endian order; 2 lsb assumed to be padding)
-
- this format is used primarily by des-crypt & variants to encode the DES output value
- used as a checksum.
- """
- return decode_int(source, True)>>2
-
-def encode_dc_int64(value):
- """encode 64-bit integer -> 11 char hash64 string (big-endian order; 2 lsb added as padding)
-
- this format is used primarily by des-crypt & variants to encode the DES output value
- used as a checksum.
- """
- #NOTE: insert 2 padding bits as lsb, to make 66 bits total
- return encode_int(value<<2,11,True)
-
-#---------------------------------------------------------------------
-
-def decode_int(source, big=False):
- """decode hash64 string -> integer
-
- :arg source: hash64 string of any length
- :arg big: if ``True``, big-endian encoding is used instead of little-endian (the default).
-
- :raises ValueError: if the string contains invalid hash64 characters.
-
- :returns:
- a integer whose value is in ``range(0,2**(6*len(source)))``
- """
- if not isinstance(source, bytes):
- raise TypeError("source must be bytes, not %s" % (type(source),))
- #FORMAT: little-endian, each char contributes 6 bits,
- # char value = index in H64_CHARS string
- if not big:
- source = reversed(source)
- try:
- out = 0
- for c in source:
- #NOTE: under py3, 'c' is int, relying on _CHARIDX to support this.
- out = (out<<6) + _decode_6bit(c)
- return out
- except KeyError:
- raise ValueError("invalid character in string")
-
-def encode_int(value, count, big=False):
- """encode integer into hash-64 format
-
- :arg value: non-negative integer to encode
- :arg count: number of output characters / 6 bit chunks to encode
- :arg big: if ``True``, big-endian encoding is used instead of little-endian (the default).
-
- :returns:
- a hash64 string of length ``count``.
- """
- if value < 0:
- raise ValueError("value cannot be negative")
- if big:
- itr = irange(6*count-6, -6, -6)
- else:
- itr = irange(0, 6*count, 6)
- return bjoin(
- _encode_6bit((value>>off) & 0x3f)
- for off in itr
- )
-
-#=================================================================================
-#eof
-#=================================================================================