Bug #35087: Inserting duplicate values at one time with DES_ENCRYPT leads

  to wrong results
      
3 problems found with DES_ENCRYPT/DES_DECRYPT :

1. The max length was not calculated properly. Fixed in fix_length_and_dec()
2. DES_ENCRYPT had a side effect of sometimes reallocating and changing 
the value of its argument. Fixed by explicitly pre-allocating the necessary
space to pad the argument with trailing '*' (stars) when calculating the 
DES digest.
3. in DES_ENCRYPT the string buffer for the result value was not 
reallocated to the correct size and only string length was assigned to it. 
Fixed by making sure there's enough space to hold the result.

1 files changed, 28 insertions, 0 deletions

diff --git a/mysql-test/t/func_des_encrypt.test b/mysql-test/t/func_des_encrypt.test
index b757a632adf..2c364a40090 100644
--- a/mysql-test/t/func_des_encrypt.test
+++ b/mysql-test/t/func_des_encrypt.test
@@ -9,3 +9,31 @@
 select des_encrypt('hello');
 
 # End of 4.1 tests
+
+--echo #
+--echo # Bug #11643: des_encrypt() causes server to die
+--echo #
+
+CREATE TABLE t1 (des VARBINARY(200) NOT NULL DEFAULT '') ENGINE=MyISAM;
+
+INSERT INTO t1 VALUES ('1234'), ('12345'), ('123456'), ('1234567');
+
+UPDATE t1 SET des=DES_ENCRYPT('1234');
+
+SELECT LENGTH(des) FROM t1;
+SELECT DES_DECRYPT(des) FROM t1;
+
+SELECT 
+ LENGTH(DES_ENCRYPT('1234')), 
+ LENGTH(DES_ENCRYPT('12345')), 
+ LENGTH(DES_ENCRYPT('123456')), 
+ LENGTH(DES_ENCRYPT('1234567'));
+SELECT 
+ DES_DECRYPT(DES_ENCRYPT('1234')), 
+ DES_DECRYPT(DES_ENCRYPT('12345')), 
+ DES_DECRYPT(DES_ENCRYPT('123456')), 
+ DES_DECRYPT(DES_ENCRYPT('1234567'));
+
+DROP TABLE t1;
+
+--Echo End of 5.0 tests