Bug# 30946: mysqldump silently ignores --default-character-set

            when used with --tab

1) New syntax: added CHARACTER SET clause to the
  SELECT ... INTO OUTFILE (to complement the same clause in
  LOAD DATA INFILE).
  mysqldump is updated to use this in --tab mode.

2) ESCAPED BY/ENCLOSED BY field parameters are documented as
   accepting CHAR argument, however SELECT .. INTO OUTFILE
   silently ignored rests of multisymbol arguments.
   For the symmetrical behavior with LOAD DATA INFILE the
   server has been modified to fail with the same error:

     ERROR 42000: Field separator argument is not what is
                  expected; check the manual

3) Current LOAD DATA INFILE recognizes field/line separators
   "as is" without converting from client charset to data
   file charset. So, it is supposed, that input file of
   LOAD DATA INFILE consists of data in one charset and
   separators in other charset. For the compatibility with
   that [buggy] behaviour SELECT INTO OUTFILE implementation
   has been saved "as is" too, but the new warning message
   has been added:

     Non-ASCII separator arguments are not fully supported

   This message warns on field/line separators that contain
   non-ASCII symbols.


client/mysqldump.c:
  mysqldump has been updated to call SELECT ... INTO OUTFILE
  statement with a charset from the --default-charset command
  line parameter.
mysql-test/r/mysqldump.result:
  Added test case for bug #30946.
mysql-test/r/outfile_loaddata.result:
  Added test case for bug #30946.
mysql-test/t/mysqldump.test:
  Added test case for bug #30946.
mysql-test/t/outfile_loaddata.test:
  Added test case for bug #30946.
sql/field.cc:
  String conversion code has been moved from check_string_copy_error()
  to convert_to_printable() for reuse.
sql/share/errmsg.txt:
  New WARN_NON_ASCII_SEPARATOR_NOT_IMPLEMENTED message has been added.
sql/sql_class.cc:
  The select_export::prepare() method has been modified to:
  
    1) raise the ER_WRONG_FIELD_TERMINATORS error on multisymbol
       ENCLOSED BY/ESCAPED BY field arguments like LOAD DATA INFILE;
  
    2) warn with a new WARN_NON_ASCII_SEPARATOR_NOT_IMPLEMENTED
       message on non-ASCII field or line separators.
  
  The select_export::send_data() merhod has been modified to
  convert item data to output charset (see new SELECT INTO OUTFILE
  syntax). By default the BINARY charset is used for backward
  compatibility.
sql/sql_class.h:
  The select_export::write_cs field added to keep output
  charset.
sql/sql_load.cc:
  mysql_load has been modified to warn on non-ASCII field or
  line separators with a new WARN_NON_ASCII_SEPARATOR_NOT_IMPLEMENTED
  message.
sql/sql_string.cc:
  New global function has been added: convert_to_printable()
  (common code has been moved from check_string_copy_error()).
sql/sql_string.h:
  New String::is_ascii() method and new global convert_to_printable()
  function have been added.
sql/sql_yacc.yy:
  New syntax: added CHARACTER SET clause to the
  SELECT ... INTO OUTFILE (to complement the same clause in
  LOAD DATA INFILE). By default the BINARY charset is used for
  backward compatibility.

1 files changed, 73 insertions, 0 deletions

diff --git a/sql/sql_string.cc b/sql/sql_string.cc
index 7759985ba85..7c9793b273b 100644
--- a/sql/sql_string.cc
+++ b/sql/sql_string.cc
@@ -1119,3 +1119,76 @@ void String::swap(String &s)
   swap_variables(bool, alloced, s.alloced);
   swap_variables(CHARSET_INFO*, str_charset, s.str_charset);
 }
+
+
+/**
+  Convert string to printable ASCII string
+
+  @details This function converts input string "from" replacing non-ASCII bytes
+  with hexadecimal sequences ("\xXX") optionally appending "..." to the end of
+  the resulting string.
+  This function used in the ER_TRUNCATED_WRONG_VALUE_FOR_FIELD error messages,
+  e.g. when a string cannot be converted to a result charset.
+
+
+  @param    to          output buffer
+  @param    to_len      size of the output buffer (8 bytes or greater)
+  @param    from        input string
+  @param    from_len    size of the input string
+  @param    from_cs     input charset
+  @param    nbytes      maximal number of bytes to convert (from_len if 0)
+
+  @return   number of bytes in the output string
+*/
+
+uint convert_to_printable(char *to, size_t to_len,
+                          const char *from, size_t from_len,
+                          CHARSET_INFO *from_cs, size_t nbytes /*= 0*/)
+{
+  /* needs at least 8 bytes for '\xXX...' and zero byte */
+  DBUG_ASSERT(to_len >= 8);
+
+  char *t= to;
+  char *t_end= to + to_len - 1; // '- 1' is for the '\0' at the end
+  const char *f= from;
+  const char *f_end= from + (nbytes ? min(from_len, nbytes) : from_len);
+  char *dots= to; // last safe place to append '...'
+
+  if (!f || t == t_end)
+    return 0;
+
+  for (; t < t_end && f < f_end; f++)
+  {
+    /*
+      If the source string is ASCII compatible (mbminlen==1)
+      and the source character is in ASCII printable range (0x20..0x7F),
+      then display the character as is.
+      
+      Otherwise, if the source string is not ASCII compatible (e.g. UCS2),
+      or the source character is not in the printable range,
+      then print the character using HEX notation.
+    */
+    if (((unsigned char) *f) >= 0x20 &&
+        ((unsigned char) *f) <= 0x7F &&
+        from_cs->mbminlen == 1)
+    {
+      *t++= *f;
+    }
+    else
+    {
+      if (t_end - t < 4) // \xXX
+        break;
+      *t++= '\\';
+      *t++= 'x';
+      *t++= _dig_vec_upper[((unsigned char) *f) >> 4];
+      *t++= _dig_vec_upper[((unsigned char) *f) & 0x0F];
+    }
+    if (t_end - t >= 3) // '...'
+      dots= t;
+  }
+  if (f < from + from_len)
+    memcpy(dots, STRING_WITH_LEN("...\0"));
+  else
+    *t= '\0';
+  return t - to;
+}