Fix for bug #27643 "query failed : 1114 (The table '' is full)

Problem:

HASH indexes on VARCHAR columns with binary collations did not ignore trailing spaces from strings before comparisons. This could result in duplicate records being successfully inserted into a MEMORY table with unique key constraints.

As a direct consequence of the above, internal MEMORY tables used for GROUP BY calculation in testcases for bug #27643 contained duplicate rows which resulted in duplicate key errors when converting those temporary tables to MyISAM. Additionally, that error was incorrectly converted to the 'table is full' error.

Solution:

- ignore trailing spaces in VARCHAR fields with binary collations when calculating hashes.
- return a proper error from create_myisam_from_heap() when conversion fails.


mysql-test/r/ctype_ucs2_def.result:
  Added a testcase for bug #27643.
mysql-test/r/heap_hash.result:
  Added a testcase for bug #27643.
mysql-test/t/ctype_ucs2_def.test:
  Added a testcase for bug #27643.
mysql-test/t/heap_hash.test:
  Added a testcase for bug #27643.
sql/sql_select.cc:
  Return an appropriate error instead of 'table is full' when conversion from MEMORY to MyISAM fails.
strings/ctype-bin.c:
  Added my_hash_sort_8bit_bin() which ignores trailing spaces when calculating hashes, and is now used for VARCHAR columns instead of my_hash_sort_bin().
strings/ctype-mb.c:
  Ignore trailing spaces when calculating a string hash in my_hash_sort_mb_bin().
strings/ctype-ucs2.c:
  Ignore trailing spaces when calculating a string hash in my_hash_sort_ucs2_bin().

1 files changed, 7 insertions, 0 deletions

diff --git a/strings/ctype-mb.c b/strings/ctype-mb.c
index 40cec669766..c12426b555f 100644
--- a/strings/ctype-mb.c
+++ b/strings/ctype-mb.c
@@ -467,6 +467,13 @@ static void my_hash_sort_mb_bin(CHARSET_INFO *cs __attribute__((unused)),
   
   key+= len;
   
+  /*
+     Remove trailing spaces. We have to do this to be able to compare
+    'A ' and 'A' as identical
+  */
+  while (key > pos && key[-1] == ' ')
+    key--;
+  
   for (; pos < (uchar*) key ; pos++)
   {
     nr1[0]^=(ulong) ((((uint) nr1[0] & 63)+nr2[0]) *