Bug# 30946: mysqldump silently ignores --default-character-set

            when used with --tab

1) New syntax: added CHARACTER SET clause to the
  SELECT ... INTO OUTFILE (to complement the same clause in
  LOAD DATA INFILE).
  mysqldump is updated to use this in --tab mode.

2) ESCAPED BY/ENCLOSED BY field parameters are documented as
   accepting CHAR argument, however SELECT .. INTO OUTFILE
   silently ignored rests of multisymbol arguments.
   For the symmetrical behavior with LOAD DATA INFILE the
   server has been modified to fail with the same error:

     ERROR 42000: Field separator argument is not what is
                  expected; check the manual

3) Current LOAD DATA INFILE recognizes field/line separators
   "as is" without converting from client charset to data
   file charset. So, it is supposed, that input file of
   LOAD DATA INFILE consists of data in one charset and
   separators in other charset. For the compatibility with
   that [buggy] behaviour SELECT INTO OUTFILE implementation
   has been saved "as is" too, but the new warning message
   has been added:

     Non-ASCII separator arguments are not fully supported

   This message warns on field/line separators that contain
   non-ASCII symbols.


client/mysqldump.c:
  mysqldump has been updated to call SELECT ... INTO OUTFILE
  statement with a charset from the --default-charset command
  line parameter.
mysql-test/r/mysqldump.result:
  Added test case for bug #30946.
mysql-test/r/outfile_loaddata.result:
  Added test case for bug #30946.
mysql-test/t/mysqldump.test:
  Added test case for bug #30946.
mysql-test/t/outfile_loaddata.test:
  Added test case for bug #30946.
sql/field.cc:
  String conversion code has been moved from check_string_copy_error()
  to convert_to_printable() for reuse.
sql/share/errmsg.txt:
  New WARN_NON_ASCII_SEPARATOR_NOT_IMPLEMENTED message has been added.
sql/sql_class.cc:
  The select_export::prepare() method has been modified to:
  
    1) raise the ER_WRONG_FIELD_TERMINATORS error on multisymbol
       ENCLOSED BY/ESCAPED BY field arguments like LOAD DATA INFILE;
  
    2) warn with a new WARN_NON_ASCII_SEPARATOR_NOT_IMPLEMENTED
       message on non-ASCII field or line separators.
  
  The select_export::send_data() merhod has been modified to
  convert item data to output charset (see new SELECT INTO OUTFILE
  syntax). By default the BINARY charset is used for backward
  compatibility.
sql/sql_class.h:
  The select_export::write_cs field added to keep output
  charset.
sql/sql_load.cc:
  mysql_load has been modified to warn on non-ASCII field or
  line separators with a new WARN_NON_ASCII_SEPARATOR_NOT_IMPLEMENTED
  message.
sql/sql_string.cc:
  New global function has been added: convert_to_printable()
  (common code has been moved from check_string_copy_error()).
sql/sql_string.h:
  New String::is_ascii() method and new global convert_to_printable()
  function have been added.
sql/sql_yacc.yy:
  New syntax: added CHARACTER SET clause to the
  SELECT ... INTO OUTFILE (to complement the same clause in
  LOAD DATA INFILE). By default the BINARY charset is used for
  backward compatibility.

1 files changed, 139 insertions, 0 deletions

diff --git a/mysql-test/r/outfile_loaddata.result b/mysql-test/r/outfile_loaddata.result
index 4a9bdcf412d..453e3adb54c 100644
--- a/mysql-test/r/outfile_loaddata.result
+++ b/mysql-test/r/outfile_loaddata.result
@@ -91,13 +91,152 @@ SELECT HEX(c1) FROM t1;
 HEX(c1)
 C3
 SELECT * INTO OUTFILE 'MYSQLTEST_VARDIR/tmp/bug32533.txt' FIELDS ENCLOSED BY 0xC3 FROM t1;
+Warnings:
+Warning	1638	Non-ASCII separator arguments are not fully supported
 TRUNCATE t1;
 SELECT HEX(LOAD_FILE('MYSQLTEST_VARDIR/tmp/bug32533.txt'));
 HEX(LOAD_FILE('MYSQLTEST_VARDIR/tmp/bug32533.txt'))
 C35CC3C30A
 LOAD DATA INFILE 'MYSQLTEST_VARDIR/tmp/bug32533.txt' INTO TABLE t1 FIELDS ENCLOSED BY 0xC3;
+Warnings:
+Warning	1638	Non-ASCII separator arguments are not fully supported
 SELECT HEX(c1) FROM t1;
 HEX(c1)
 C3
 DROP TABLE t1;
 # End of 5.0 tests.
+#
+# Bug #30946: mysqldump silently ignores --default-character-set
+#             when used with --tab
+#
+# Also see mysqldump.test
+#
+SET NAMES utf8;
+CREATE TABLE t1 (a INT, b CHAR(10) CHARSET koi8r, c CHAR(10) CHARSET latin1);
+CREATE TABLE t2 LIKE t1;
+INSERT INTO t1 VALUES (1, 'ABC-АБВ', 'DEF-ÂÃÄ'), (2, NULL, NULL);
+# Error on multi-character ENCLOSED/ESCAPED BY
+SELECT * FROM t1 INTO OUTFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' FIELDS ENCLOSED BY '12345';
+ERROR 42000: Field separator argument is not what is expected; check the manual
+SELECT * FROM t1 INTO OUTFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' FIELDS ESCAPED BY '12345';
+ERROR 42000: Field separator argument is not what is expected; check the manual
+# "Not implemented" warning on multibyte  ENCLOSED/ESCAPED BY character,
+# LOAD DATA rises error or has unpredictable result -- to be fixed later
+SELECT * FROM t1 INTO OUTFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' FIELDS ENCLOSED BY 'ъ';
+Warnings:
+Warning	1638	Non-ASCII separator arguments are not fully supported
+LOAD DATA INFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' INTO TABLE t2 CHARACTER SET binary FIELDS ENCLOSED BY 'ъ';
+ERROR 42000: Field separator argument is not what is expected; check the manual
+SELECT * FROM t1 INTO OUTFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' FIELDS ESCAPED BY 'ъ';
+Warnings:
+Warning	1638	Non-ASCII separator arguments are not fully supported
+LOAD DATA INFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' INTO TABLE t2 CHARACTER SET binary FIELDS ESCAPED BY 'ъ';
+ERROR 42000: Field separator argument is not what is expected; check the manual
+SELECT * FROM t1 INTO OUTFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' FIELDS TERMINATED BY 'ъ';
+Warnings:
+Warning	1638	Non-ASCII separator arguments are not fully supported
+##################################################
+1ъABC-���ъDEF-���
+2ъ\Nъ\N
+##################################################
+TRUNCATE t2;
+LOAD DATA INFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' INTO TABLE t2 CHARACTER SET binary FIELDS TERMINATED BY 'ъ';
+Warnings:
+Warning	1638	Non-ASCII separator arguments are not fully supported
+Warning	1265	Data truncated for column 'a' at row 1
+Warning	1261	Row 1 doesn't contain data for all columns
+Warning	1261	Row 1 doesn't contain data for all columns
+Warning	1265	Data truncated for column 'a' at row 2
+Warning	1261	Row 2 doesn't contain data for all columns
+Warning	1261	Row 2 doesn't contain data for all columns
+SELECT * FROM t1 UNION SELECT * FROM t2 ORDER BY a, b, c;
+a	b	c
+1	NULL	NULL
+1	ABC-АБВ	DEF-ÂÃÄ
+2	NULL	NULL
+SELECT * FROM t1 INTO OUTFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' LINES STARTING BY 'ъ';
+Warnings:
+Warning	1638	Non-ASCII separator arguments are not fully supported
+##################################################
+ъ1	ABC-���	DEF-���
+ъ2	\N	\N
+##################################################
+TRUNCATE t2;
+LOAD DATA INFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' INTO TABLE t2 CHARACTER SET binary LINES STARTING BY 'ъ';
+Warnings:
+Warning	1638	Non-ASCII separator arguments are not fully supported
+SELECT * FROM t1 UNION SELECT * FROM t2 ORDER BY a, b, c;
+a	b	c
+1	ABC-АБВ	DEF-ÂÃÄ
+2	NULL	NULL
+SELECT * FROM t1 INTO OUTFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' LINES TERMINATED BY 'ъ';
+Warnings:
+Warning	1638	Non-ASCII separator arguments are not fully supported
+##################################################
+1	ABC-���	DEF-���ъ2	\N	\Nъ##################################################
+TRUNCATE t2;
+LOAD DATA INFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' INTO TABLE t2 CHARACTER SET binary LINES TERMINATED BY 'ъ';
+Warnings:
+Warning	1638	Non-ASCII separator arguments are not fully supported
+SELECT * FROM t1 UNION SELECT * FROM t2 ORDER BY a, b, c;
+a	b	c
+1	ABC-АБВ	DEF-ÂÃÄ
+1	ABC-АБВ	DEF-ÂÃÄÑŠ2
+2	NULL	NULL
+# Default (binary) charset:
+SELECT * INTO OUTFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' FROM t1;
+##################################################
+1	ABC-���	DEF-���
+2	\N	\N
+##################################################
+TRUNCATE t2;
+LOAD DATA INFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' INTO TABLE t2 CHARACTER SET binary;
+SELECT * FROM t1 UNION SELECT * FROM t2 ORDER BY a, b, c;
+a	b	c
+1	ABC-АБВ	DEF-ÂÃÄ
+2	NULL	NULL
+# latin1 charset (INTO OUTFILE warning is expected):
+SELECT * INTO OUTFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' CHARACTER SET latin1 FROM t1;
+Warnings:
+Warning	1366	Incorrect string value: '\xE1\xE2\xF7' for column 'b' at row 1
+##################################################
+1	ABC-???	DEF-���
+2	\N	\N
+##################################################
+TRUNCATE t2;
+LOAD DATA INFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' INTO TABLE t2 CHARACTER SET latin1 ;
+SELECT * FROM t1 UNION SELECT * FROM t2 ORDER BY a, b, c;
+a	b	c
+1	ABC-???	DEF-ÂÃÄ
+1	ABC-АБВ	DEF-ÂÃÄ
+2	NULL	NULL
+# KOI8-R charset (INTO OUTFILE warning is expected):
+SELECT * INTO OUTFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' CHARACTER SET koi8r FROM t1;
+Warnings:
+Warning	1366	Incorrect string value: '\xC2\xC3\xC4' for column 'c' at row 1
+##################################################
+1	ABC-���	DEF-???
+2	\N	\N
+##################################################
+TRUNCATE t2;
+LOAD DATA INFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' INTO TABLE t2 CHARACTER SET koi8r;
+SELECT * FROM t1 UNION SELECT * FROM t2 ORDER BY a, b, c;
+a	b	c
+1	ABC-АБВ	DEF-???
+1	ABC-АБВ	DEF-ÂÃÄ
+2	NULL	NULL
+# UTF-8 charset:
+SELECT * INTO OUTFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' CHARACTER SET utf8 FROM t1;
+##################################################
+1	ABC-АБВ	DEF-ÂÃÄ
+2	\N	\N
+##################################################
+TRUNCATE t2;
+LOAD DATA INFILE 'MYSQLTEST_VARDIR/tmp/t1.txt' INTO TABLE t2 CHARACTER SET utf8;
+SELECT * FROM t1 UNION SELECT * FROM t2 ORDER BY a, b, c;
+a	b	c
+1	ABC-АБВ	DEF-ÂÃÄ
+2	NULL	NULL
+SET NAMES default;
+DROP TABLE t1, t2;
+# End of 5.1 tests.