summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorJan Lindström <jan.lindstrom@mariadb.com>2018-03-14 16:13:14 +0200
committerJan Lindström <jan.lindstrom@mariadb.com>2018-03-14 16:13:14 +0200
commitbc2c0e1e27b1d1003429dc5afad29281a0b63ccb (patch)
tree0ad47be9c29fa1975cbbab129f23593ae97ec4a9
parentb4026efaacbfc935d3ce8fc0fdb0a88920c4f426 (diff)
downloadmariadb-git-bb-10.1-MDEV-13103-MDEV-15032.tar.gz
MDEV-13103: InnoDB crash recovery fails to decompress a page in buf_dblwr_process()bb-10.1-MDEV-13103-MDEV-15032
There were several problems. Firstly, page decompression code did not handle possible decompression errors correctly. Secondly, not all compression methods tolerate corrupted input (e.g. lz4 did not tolerate input that was compressed using snappy method). Finally, if page is actually also encrypted we can't decompress page. Solutions: Add proper error handling to decompression code and add post compression checksum to page. As whole page including page checksum is compressed we can reuse the original checksum field for post compression checksum. With post compression checksum we can detect most of the corruptions. If no corruption is detected hopefully decompression code can detect remaining problems. Doublewrite buffer page recovery for page compressed pages require that post compression checksum matches. For pages from old releases supporting page compression checksum must be BUF_NO_CHECKSUM_MAGIC. Upgrade from older versions is supported as post compression checksum check accepts the BUF_NO_CHECKSUM_MAGIC that they stored in checksum filed. Downgrade to older versions is not supported (assuming that there were some changes to compressed tables) as page compression code would not tolerate any other checksum except BUF_NO_CHECKSUM_MAGIC. innochecksum.cc is_page_corrupted: If page is compressed verify post compression checksum buf_page_decrypt_after_read Return DB_PAGE_CORRUPTED if page is found to be corrupted after post compression checksum check. buf_page_io_complete If page is found corrupted after buf_page_decrypt_after_read there is no need to continue page check. buf_page_decrypt_after_read Verify post compression checksum before decompression and if it does not match mark page corrupted. Note that old compressed pages do not really have post compression checksum so they are treated as not corrupted and then we need to hope that decompression code can handle the possible corruptions by returning error. buf_calc_compressed_crc32 New function to calculate post compression checksum so that necessary compression metadata fields are included. buf_dblwr_decompress New function that handles post compression checksum check and page decompression if it is ok. buf_dblwr_process Verify post compression checksum before trying to decompress page. fil_space_verify_crypt_checksum Remove incorrect code as compressed and encrypted pages do have post encryption checksum. fil_compress_page Calculate and store post compression checksum to FIL_SPACE_OR_CHKSUM field as original value is stored on compressed image. fil_decompress_page Add error handling if decompression fails. fil_verify_compression_checksum New function to verify post compression checksum. Compressed tablespaces before this change have BUF_NO_CHECKSUM_MAGIC in checksum field and they must be treated as not corrupted. fil_iterate Add post compression checksum check before actual decompression. convert_error_code_to_mysql Handle also page corruptions DB_PAGE_CORRUPTED as HA_ERR_CRASHED. Note that there are cases when we do not know for certain is page corrupted, corrupted and compressed, or still encrypted after failed decrypt, thus tablespace could be marked just corrupted. Tests modified innodb-page_compression_[zip, lz4, lzma, lzo, bzip2, snappy] to use innodb-page-compression.inc innodb-page-compression.inc add innochecksum and intentional tablespace corruption tests. innodb-force-corrupt, innodb_bug14147491 add new error messages to mtr suppression and new error codes. New tests encryption/innodb-corrupted.test test intentionally corrupted tablespaces containing encryption and compression. doublewrite-compressed test doublewrite recovery for page compressed tables innodb-import import files from both big_endian and little_endian machine
-rw-r--r--extra/CMakeLists.txt1
-rw-r--r--extra/innochecksum.cc32
-rw-r--r--mysql-test/include/innodb-page-compression.inc288
-rw-r--r--mysql-test/std_data/t3_be.ibdbin0 -> 98304 bytes
-rw-r--r--mysql-test/std_data/t3_le.ibdbin0 -> 98304 bytes
-rw-r--r--mysql-test/std_data/t4_be.ibdbin0 -> 98304 bytes
-rw-r--r--mysql-test/std_data/t4_le.ibdbin0 -> 98304 bytes
-rw-r--r--mysql-test/suite/encryption/r/innodb-bad-key-change.result10
-rw-r--r--mysql-test/suite/encryption/r/innodb-bad-key-change2.result4
-rw-r--r--mysql-test/suite/encryption/r/innodb-bad-key-change4.result2
-rw-r--r--mysql-test/suite/encryption/r/innodb-compressed-blob.result2
-rw-r--r--mysql-test/suite/encryption/r/innodb-corrupted.result146
-rw-r--r--mysql-test/suite/encryption/r/innodb-encryption-disable.result3
-rw-r--r--mysql-test/suite/encryption/r/innodb-force-corrupt.result8
-rw-r--r--mysql-test/suite/encryption/r/innodb-import.result39
-rw-r--r--mysql-test/suite/encryption/r/innodb-missing-key.result10
-rw-r--r--mysql-test/suite/encryption/r/innodb_encryption-page-compression.result331
-rw-r--r--mysql-test/suite/encryption/t/innodb-bad-key-change.test10
-rw-r--r--mysql-test/suite/encryption/t/innodb-bad-key-change2.test4
-rw-r--r--mysql-test/suite/encryption/t/innodb-bad-key-change4.test2
-rw-r--r--mysql-test/suite/encryption/t/innodb-compressed-blob.test2
-rw-r--r--mysql-test/suite/encryption/t/innodb-corrupted.test25
-rw-r--r--mysql-test/suite/encryption/t/innodb-encryption-disable.test3
-rw-r--r--mysql-test/suite/encryption/t/innodb-force-corrupt.test8
-rw-r--r--mysql-test/suite/encryption/t/innodb-import.test74
-rw-r--r--mysql-test/suite/encryption/t/innodb-missing-key.test12
-rw-r--r--mysql-test/suite/encryption/t/innodb_encryption-page-compression.test149
-rw-r--r--mysql-test/suite/innodb/include/innodb-page-compression.inc290
-rw-r--r--mysql-test/suite/innodb/r/doublewrite-compressed.result253
-rw-r--r--mysql-test/suite/innodb/r/innodb-page_compression_bzip2.result227
-rw-r--r--mysql-test/suite/innodb/r/innodb-page_compression_default.result223
-rw-r--r--mysql-test/suite/innodb/r/innodb-page_compression_lz4.result227
-rw-r--r--mysql-test/suite/innodb/r/innodb-page_compression_lzma.result227
-rw-r--r--mysql-test/suite/innodb/r/innodb-page_compression_lzo.result227
-rw-r--r--mysql-test/suite/innodb/r/innodb-page_compression_snappy.result227
-rw-r--r--mysql-test/suite/innodb/r/innodb-page_compression_zip.result227
-rw-r--r--mysql-test/suite/innodb/r/innodb_bug14147491.result2
-rw-r--r--mysql-test/suite/innodb/t/doublewrite-compressed.test439
-rw-r--r--mysql-test/suite/innodb/t/innodb-page_compression_bzip2.test9
-rw-r--r--mysql-test/suite/innodb/t/innodb-page_compression_lz4.test8
-rw-r--r--mysql-test/suite/innodb/t/innodb-page_compression_lzma.test10
-rw-r--r--mysql-test/suite/innodb/t/innodb-page_compression_lzo.test9
-rw-r--r--mysql-test/suite/innodb/t/innodb-page_compression_snappy.test9
-rw-r--r--mysql-test/suite/innodb/t/innodb-page_compression_zip.test9
-rw-r--r--mysql-test/suite/innodb/t/innodb_bug14147491.test4
-rw-r--r--storage/innobase/buf/buf0buf.cc93
-rw-r--r--storage/innobase/buf/buf0checksum.cc45
-rw-r--r--storage/innobase/buf/buf0dblwr.cc76
-rw-r--r--storage/innobase/dict/dict0dict.cc6
-rw-r--r--storage/innobase/fil/fil0crypt.cc17
-rw-r--r--storage/innobase/fil/fil0fil.cc28
-rw-r--r--storage/innobase/fil/fil0pagecompress.cc272
-rw-r--r--storage/innobase/handler/ha_innodb.cc3
-rw-r--r--storage/innobase/include/buf0checksum.h13
-rw-r--r--storage/innobase/include/fil0pagecompress.h58
-rw-r--r--storage/innobase/include/fsp0pagecompress.h4
-rw-r--r--storage/innobase/include/fsp0pagecompress.ic2
-rw-r--r--storage/xtradb/buf/buf0buf.cc81
-rw-r--r--storage/xtradb/buf/buf0checksum.cc45
-rw-r--r--storage/xtradb/buf/buf0dblwr.cc76
-rw-r--r--storage/xtradb/dict/dict0dict.cc6
-rw-r--r--storage/xtradb/fil/fil0crypt.cc17
-rw-r--r--storage/xtradb/fil/fil0fil.cc28
-rw-r--r--storage/xtradb/fil/fil0pagecompress.cc272
-rw-r--r--storage/xtradb/handler/ha_innodb.cc3
-rw-r--r--storage/xtradb/include/buf0checksum.h13
-rw-r--r--storage/xtradb/include/fil0pagecompress.h58
-rw-r--r--storage/xtradb/include/fsp0pagecompress.h4
-rw-r--r--storage/xtradb/include/fsp0pagecompress.ic2
69 files changed, 3413 insertions, 1601 deletions
diff --git a/extra/CMakeLists.txt b/extra/CMakeLists.txt
index c8e15dd4fb4..7ed46935675 100644
--- a/extra/CMakeLists.txt
+++ b/extra/CMakeLists.txt
@@ -80,6 +80,7 @@ IF(WITH_INNOBASE_STORAGE_ENGINE OR WITH_XTRADB_STORAGE_ENGINE)
../storage/innobase/buf/buf0buf.cc
../storage/innobase/page/page0zip.cc
../storage/innobase/fil/fil0crypt.cc
+ ../storage/innobase/fil/fil0pagecompress.cc
)
IF(CMAKE_SYSTEM_PROCESSOR MATCHES "ppc64le")
diff --git a/extra/innochecksum.cc b/extra/innochecksum.cc
index 7ad71aa8159..8e88241ab95 100644
--- a/extra/innochecksum.cc
+++ b/extra/innochecksum.cc
@@ -1,6 +1,6 @@
/*
Copyright (c) 2005, 2016, Oracle and/or its affiliates. All rights reserved.
- Copyright (c) 2014, 2017, MariaDB Corporation.
+ Copyright (c) 2014, 2018, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@@ -68,6 +68,7 @@ typedef void fil_space_t;
#include "fsp0pagecompress.h" /* fil_get_compression_alg_name */
#include "fil0crypt.h" /* fil_space_verify_crypt_checksum */
#include "page0size.h"
+#include "fil0pagecompress.h" /* fil_verify_compression_checksum */
#include <string.h>
@@ -478,13 +479,9 @@ is_page_corrupted(
buf + FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID);
/* We can't trust only a page type, thus we take account
- also fsp_flags or crypt_data on page 0 */
- if ((page_type == FIL_PAGE_PAGE_COMPRESSED && is_compressed) ||
- (page_type == FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED &&
- is_compressed && is_encrypted)) {
- /* Page compressed tables do not contain post compression
- checksum. */
- return (false);
+ also fsp_flags. */
+ if (page_type == FIL_PAGE_PAGE_COMPRESSED && is_compressed) {
+ return(!fil_verify_compression_checksum(buf, space_id, cur_page_num));
}
if (page_size.is_compressed()) {
@@ -523,11 +520,13 @@ is_page_corrupted(
if (is_encrypted && key_version != 0) {
is_corrupted = !fil_space_verify_crypt_checksum(buf,
page_size.is_compressed() ? page_size.physical() : 0, NULL, cur_page_num);
- } else {
- is_corrupted = true;
- }
- if (is_corrupted) {
+ if (is_corrupted) {
+ is_corrupted = buf_page_is_corrupted(
+ true, buf, page_size.is_compressed() ? page_size.physical() : 0, NULL);
+ }
+
+ } else {
is_corrupted = buf_page_is_corrupted(
true, buf, page_size.is_compressed() ? page_size.physical() : 0, NULL);
}
@@ -1894,15 +1893,6 @@ int main(
skip_page = false;
}
- ulint cur_page_type = mach_read_from_2(buf+FIL_PAGE_TYPE);
-
- /* FIXME: Page compressed or Page compressed and encrypted
- pages do not contain checksum. */
- if (cur_page_type == FIL_PAGE_PAGE_COMPRESSED ||
- cur_page_type == FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED) {
- skip_page = true;
- }
-
/* If no-check is enabled, skip the
checksum verification.*/
if (!no_check
diff --git a/mysql-test/include/innodb-page-compression.inc b/mysql-test/include/innodb-page-compression.inc
new file mode 100644
index 00000000000..7a9c9651e7a
--- /dev/null
+++ b/mysql-test/include/innodb-page-compression.inc
@@ -0,0 +1,288 @@
+if (!$INNOCHECKSUM) {
+ --echo Need innochecksum binary
+ --die Need innochecksum binary
+}
+
+--disable_warnings
+set global innodb_file_format = `Barracuda`;
+set global innodb_file_per_table = on;
+--enable_warnings
+
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t1 page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t2 page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t3 page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t4 page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Page \\[page id: space=[0-9]+, page number= [0-9]+\\] in file ./test/t[0-9]+.ibd may be corrupted. Post compression checksum [0-9]+ stored [0-9]+ compression_method [ZLIB|SNAPPY|LZ4|LZO|LZMA|BZIP2]");
+call mtr.add_suppression("InnoDB: Background Page read failed to read or decrypt \\[page id: space=[0-9]+, page number=[0-9]+\\]");
+call mtr.add_suppression("mysqld: Index for table 't[0-9]+' is corrupt; try to repair it");
+
+create table t0 (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
+create table t1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
+create table t2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
+create table t3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
+create table t4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
+create table t5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
+create table t6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
+create table t7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
+create table t8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
+create table t9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
+
+--disable_query_log
+begin;
+let $i = 10;
+while ($i)
+{
+ insert into t0(b) values(REPEAT('Aa',50));
+ insert into t0(b) values(REPEAT('a',100));
+ insert into t0(b) values(REPEAT('b',100));
+ insert into t0(b) values(REPEAT('0',100));
+ insert into t0(b) values(REPEAT('1',100));
+ dec $i;
+}
+
+insert into t1 select * from t0;
+insert into t2 select * from t0;
+insert into t3 select * from t0;
+insert into t4 select * from t0;
+insert into t5 select * from t0;
+insert into t6 select * from t0;
+insert into t7 select * from t0;
+insert into t8 select * from t0;
+insert into t9 select * from t0;
+commit;
+--enable_query_log
+
+select count(*) from t1;
+select count(*) from t3;
+select count(*) from t4;
+select count(*) from t5;
+select count(*) from t6;
+select count(*) from t6;
+select count(*) from t7;
+select count(*) from t8;
+select count(*) from t9;
+
+#
+# Wait until pages are really compressed
+#
+let $wait_condition= select variable_value > 0 from information_schema.global_status where variable_name = 'INNODB_NUM_PAGES_PAGE_COMPRESSED';
+--source include/wait_condition.inc
+
+--let $MYSQLD_DATADIR=`select @@datadir`
+
+# shutdown before grep
+
+--source include/shutdown_mysqld.inc
+
+--let t1_IBD = $MYSQLD_DATADIR/test/t0.ibd
+--let SEARCH_RANGE = 10000000
+--let SEARCH_PATTERN=AaAaAaAa
+--echo # t0 expected FOUND
+-- let SEARCH_FILE=$t1_IBD
+-- source include/search_pattern_in_file.inc
+--let t1_IBD = $MYSQLD_DATADIR/test/t1.ibd
+--echo # t1 page compressed expected NOT FOUND
+-- let SEARCH_FILE=$t1_IBD
+-- source include/search_pattern_in_file.inc
+--let t1_IBD = $MYSQLD_DATADIR/test/t2.ibd
+--echo # t2 page compressed expected NOT FOUND
+-- let SEARCH_FILE=$t1_IBD
+-- source include/search_pattern_in_file.inc
+--let t1_IBD = $MYSQLD_DATADIR/test/t3.ibd
+--echo # t3 page compressed expected NOT FOUND
+-- let SEARCH_FILE=$t1_IBD
+-- source include/search_pattern_in_file.inc
+--let t1_IBD = $MYSQLD_DATADIR/test/t4.ibd
+--echo # t4 page compressed expected NOT FOUND
+-- let SEARCH_FILE=$t1_IBD
+-- source include/search_pattern_in_file.inc
+--let t1_IBD = $MYSQLD_DATADIR/test/t5.ibd
+--echo # t5 page compressed expected NOT FOUND
+-- let SEARCH_FILE=$t1_IBD
+-- source include/search_pattern_in_file.inc
+--let t1_IBD = $MYSQLD_DATADIR/test/t6.ibd
+--echo # t6 page compressed expected NOT FOUND
+-- let SEARCH_FILE=$t1_IBD
+-- source include/search_pattern_in_file.inc
+--let t1_IBD = $MYSQLD_DATADIR/test/t7.ibd
+--echo # t7 page compressed expected NOT FOUND
+-- let SEARCH_FILE=$t1_IBD
+-- source include/search_pattern_in_file.inc
+--let t1_IBD = $MYSQLD_DATADIR/test/t8.ibd
+--echo # t8 page compressed expected NOT FOUND
+-- let SEARCH_FILE=$t1_IBD
+-- source include/search_pattern_in_file.inc
+--let t1_IBD = $MYSQLD_DATADIR/test/t9.ibd
+--echo # t9 page compressed expected NOT FOUND
+-- let SEARCH_FILE=$t1_IBD
+-- source include/search_pattern_in_file.inc
+
+#
+# Run innochecksum to all tables, all tables should be ok
+#
+let t_IBD = $MYSQLD_DATADIR/test/t0.ibd;
+--exec $INNOCHECKSUM $t_IBD
+let $i=9;
+while $($i > 0) {
+--echo # Run innochecksum on t$i
+let t_IBD = $MYSQLD_DATADIR/test/t$i.ibd;
+--exec $INNOCHECKSUM $t_IBD
+dec $i;
+}
+
+-- source include/start_mysqld.inc
+
+select count(*) from t0;
+select count(*) from t1;
+select count(*) from t3;
+select count(*) from t4;
+select count(*) from t5;
+select count(*) from t6;
+select count(*) from t7;
+select count(*) from t8;
+select count(*) from t9;
+
+let $wait_condition= select variable_value > 0 from information_schema.global_status where variable_name = 'INNODB_NUM_PAGES_PAGE_DECOMPRESSED';
+--source include/wait_condition.inc
+
+let INNODB_PAGE_SIZE=`select @@innodb_page_size`;
+let MYSQLD_DATADIR=`select @@datadir`;
+
+--echo # Restart server
+--source include/shutdown_mysqld.inc
+
+--echo # Corrupting tablespaces...
+
+#
+# Now we corrupt page compressed pages as follows:
+# (1) compression method
+# (2) payload size
+# (3) data
+# (4) checksum
+#
+# Page 0 is not compressed or encrypted
+#
+perl;
+sub ib_write_value {
+ my($fh, $file, $pos, $len, $val) = @_;
+ seek($fh, $pos, SEEK_SET) or die "$0: seek $file to pos $pos: $!";;
+ syswrite($fh, $val, $len) == $len or die "$0: write to $file val $val len $len: $!";;
+}
+sub ib_corrupt_table {
+ my($file, $pos, $val, $len) = @_;
+ open(my $fh, "+<", $file) or die "$0: open $file: $!";
+ binmode $fh;
+ ib_write_value($fh, $file, $pos, $len, $val);
+ close $fh or die "$0: close $file: $!";
+}
+sub ib_read_value {
+ my($fh, $file, $pos, $len) = @_;
+ seek($fh, $pos, SEEK_SET) or die "$0: seek $file to pos $pos: $!";
+ sysread($fh, $buf, $len) == $len or die "$0: read $file : $!";
+ return $buf;
+}
+
+my($pos) = $ENV{'INNODB_PAGE_SIZE'} * 3;
+my($file) = "$ENV{MYSQLD_DATADIR}/test/t1.ibd";
+open($fh, "+<", $file) or die "$0: open $file : $!";
+binmode $fh;
+
+# Find out the page type, compression method location is based on that
+# Note that FIL_HEADER is not encrypted
+my($ptype) = unpack("n", ib_read_value($fh, $file, $pos+24, 2));
+if ($ptype == 34354) {
+ $pos += 32; # FLUSH_LSN_OR_KEY_VERSION + 6
+} else {
+ $pos += 42; # FIL_PAGE_DATA + 2
+}
+
+# Corrupt compression method by decreasing it by one, if zero set 6
+# Note that compression method is not encrypted
+my($cmethod) = unpack("n", ib_read_value($fh, $file, $pos, 2));
+$cmethod--;
+if ($cmethod == 0) {
+ $cmethod = 6;
+}
+
+ib_write_value($fh, $file, $pos, 2, pack("n", $cmethod));
+close $fh or die $!;
+
+# (2) corrupt payload size by decreasing size by 50 and if 0 set it to 20
+# note that payload size is not encrypted
+my($pos) = $ENV{'INNODB_PAGE_SIZE'} * 3 + 40;
+my($file) = "$ENV{MYSQLD_DATADIR}/test/t2.ibd";
+open($fh, "+<", $file) or die "$0: open $file : $!";
+my($size) = unpack("n", ib_read_value($fh, $file, $pos, 2));
+$size -= 50;
+if ($size <= 0) {
+ $size = 20;
+}
+ib_write_value($fh, $file, $pos, 2, pack("n", $size));
+close $fh or die $!;
+
+# (3) Corrupt data
+
+ib_corrupt_table("$ENV{MYSQLD_DATADIR}/test/t3.ibd", $ENV{'INNODB_PAGE_SIZE'} * 3 + 42,
+ "deadaaaaffffbbbb",14);
+
+# (4) Corrupt checksum
+my($pos) = $ENV{'INNODB_PAGE_SIZE'} * 3;
+my($file) = "$ENV{MYSQLD_DATADIR}/test/t4.ibd";
+open($fh, "+<", $file) or die "$0: open $file : $!";
+
+# Find out the page type, checksum location is based on that
+# Note that FIL_HEADER is not encrypted
+my($ptype) = unpack("n", ib_read_value($fh, $file, $pos + 24, 2));
+if ($ptype == 37401) {
+ $pos += 30;
+}
+close $fh or die $!;
+ib_corrupt_table($file, $pos, pack("n", 0xdead), 2);
+EOF
+
+--echo # Corruption done
+
+#
+# Run innochecksum to page compressed (and maybe encrypted) tables
+# now we should detect corruptions on compressed cases.
+# In compressed and encrypted cases innochecksum can't decrypt
+# or decompress so we do not detect all corruptions e.g. corruption on
+# compression method.
+#
+--echo # Run innochecksum on t$i
+let t_IBD = $MYSQLD_DATADIR/test/t$i.ibd;
+--error 0,1
+--exec $INNOCHECKSUM $t_IBD
+
+let $i=4;
+while $($i > 1) {
+--echo # Run innochecksum on t$i
+let t_IBD = $MYSQLD_DATADIR/test/t$i.ibd;
+--error 1
+--exec $INNOCHECKSUM $t_IBD
+dec $i;
+}
+
+--echo # Start server again
+--source include/start_mysqld.inc
+
+#
+# Server should not crash on corrupted tables
+#
+--error ER_NOT_KEYFILE, ER_GET_ERRMSG
+select * from t1;
+--error ER_NOT_KEYFILE, ER_GET_ERRMSG
+select * from t2;
+--error ER_NOT_KEYFILE, ER_GET_ERRMSG
+select * from t3;
+--error ER_NOT_KEYFILE, ER_GET_ERRMSG
+select * from t4;
+select count(*) from t5;
+select count(*) from t6;
+select count(*) from t7;
+select count(*) from t8;
+select count(*) from t9;
+
+# We should be able to drop even corrupted tables
+
+drop table t0, t1, t2, t3, t4, t5, t6, t7, t8, t9;
diff --git a/mysql-test/std_data/t3_be.ibd b/mysql-test/std_data/t3_be.ibd
new file mode 100644
index 00000000000..7cb320f10e2
--- /dev/null
+++ b/mysql-test/std_data/t3_be.ibd
Binary files differ
diff --git a/mysql-test/std_data/t3_le.ibd b/mysql-test/std_data/t3_le.ibd
new file mode 100644
index 00000000000..7cb320f10e2
--- /dev/null
+++ b/mysql-test/std_data/t3_le.ibd
Binary files differ
diff --git a/mysql-test/std_data/t4_be.ibd b/mysql-test/std_data/t4_be.ibd
new file mode 100644
index 00000000000..68b74fd31c1
--- /dev/null
+++ b/mysql-test/std_data/t4_be.ibd
Binary files differ
diff --git a/mysql-test/std_data/t4_le.ibd b/mysql-test/std_data/t4_le.ibd
new file mode 100644
index 00000000000..66692ca5992
--- /dev/null
+++ b/mysql-test/std_data/t4_le.ibd
Binary files differ
diff --git a/mysql-test/suite/encryption/r/innodb-bad-key-change.result b/mysql-test/suite/encryption/r/innodb-bad-key-change.result
index d581b98513d..baf2a98307a 100644
--- a/mysql-test/suite/encryption/r/innodb-bad-key-change.result
+++ b/mysql-test/suite/encryption/r/innodb-bad-key-change.result
@@ -1,7 +1,9 @@
-call mtr.add_suppression("InnoDB: The page .*");
-call mtr.add_suppression("InnoDB: Cannot calculate statistics for table .* ");
-call mtr.add_suppression("Plugin 'file_key_management' .*");
-call mtr.add_suppression("mysqld: File .*");
+call mtr.add_suppression("Plugin 'file_key_management' init function returned error");
+call mtr.add_suppression("Plugin 'file_key_management' registration.*failed");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t1 page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[0-9]+, page number=[0-9]+\\] in file ./test/t1.ibd cannot be decrypted.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[0-9]+, page number=[0-9]+\\] in file ./test/t2.ibd cannot be decrypted.");
+call mtr.add_suppression("mysqld: File .*keysbad3.txt' not found .*");
# Start server with keys2.txt
SET GLOBAL innodb_file_format = `Barracuda`;
diff --git a/mysql-test/suite/encryption/r/innodb-bad-key-change2.result b/mysql-test/suite/encryption/r/innodb-bad-key-change2.result
index 94ed922a0ec..dbf5f24f876 100644
--- a/mysql-test/suite/encryption/r/innodb-bad-key-change2.result
+++ b/mysql-test/suite/encryption/r/innodb-bad-key-change2.result
@@ -1,5 +1,5 @@
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1new cannot be decrypted.");
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[0-9]+, page number=[0-9]+\\] in file ./test/t1new.ibd cannot be decrypted.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[0-9]+, page number=[0-9]+\\] in file ./test/t1.ibd cannot be decrypted.");
call mtr.add_suppression("Couldn't load plugins from 'file_key_management.*");
call mtr.add_suppression("InnoDB: Table \'\"test\".\"t1\"\' tablespace is set as discarded.");
SET GLOBAL innodb_file_format = `Barracuda`;
diff --git a/mysql-test/suite/encryption/r/innodb-bad-key-change4.result b/mysql-test/suite/encryption/r/innodb-bad-key-change4.result
index 227b8cc3deb..b80788d9eb1 100644
--- a/mysql-test/suite/encryption/r/innodb-bad-key-change4.result
+++ b/mysql-test/suite/encryption/r/innodb-bad-key-change4.result
@@ -1,4 +1,4 @@
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[0-9]+, page number=[0-9]+\\] in file ./test/t1.ibd cannot be decrypted.");
call mtr.add_suppression("InnoDB: Cannot open table .*");
call mtr.add_suppression("InnoDB: .ibd file is missing for table test/.*");
call mtr.add_suppression("Couldn't load plugins from 'file_key_management.*");
diff --git a/mysql-test/suite/encryption/r/innodb-compressed-blob.result b/mysql-test/suite/encryption/r/innodb-compressed-blob.result
index 15c2a410948..b9889df00e9 100644
--- a/mysql-test/suite/encryption/r/innodb-compressed-blob.result
+++ b/mysql-test/suite/encryption/r/innodb-compressed-blob.result
@@ -1,5 +1,5 @@
call mtr.add_suppression("InnoDB: However key management plugin or used key_version .*");
-call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[0-9]+\\] in file test/t[1-3] cannot be decrypted.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[0-9]+, page number=[0-9]+\\] in file ./test/t[1-3].ibd cannot be decrypted.");
call mtr.add_suppression("Unable to decompress space ..test.t[1-3].ibd \\[[1-9][0-9]*:[0-9]+\\]");
# Restart mysqld --file-key-management-filename=keys2.txt
SET GLOBAL innodb_file_format = `Barracuda`;
diff --git a/mysql-test/suite/encryption/r/innodb-corrupted.result b/mysql-test/suite/encryption/r/innodb-corrupted.result
new file mode 100644
index 00000000000..be745777c04
--- /dev/null
+++ b/mysql-test/suite/encryption/r/innodb-corrupted.result
@@ -0,0 +1,146 @@
+set global innodb_encrypt_tables='FORCE';
+select @@innodb_encrypt_tables;
+@@innodb_encrypt_tables
+FORCE
+set global innodb_default_encryption_key_id=4;
+select @@innodb_default_encryption_key_id;
+@@innodb_default_encryption_key_id
+1
+set global innodb_compression_algorithm=zlib;
+select @@innodb_compression_algorithm;
+@@innodb_compression_algorithm
+zlib
+set global innodb_file_format = `Barracuda`;
+set global innodb_file_per_table = on;
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t1 page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t2 page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t3 page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t4 page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Page \\[page id: space=[0-9]+, page number= [0-9]+\\] in file ./test/t[0-9]+.ibd may be corrupted. Post compression checksum [0-9]+ stored [0-9]+ compression_method [ZLIB|SNAPPY|LZ4|LZO|LZMA|BZIP2]");
+call mtr.add_suppression("InnoDB: Background Page read failed to read or decrypt \\[page id: space=[0-9]+, page number=[0-9]+\\]");
+call mtr.add_suppression("mysqld: Index for table 't[0-9]+' is corrupt; try to repair it");
+create table t0 (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
+create table t1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
+create table t2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
+create table t3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
+create table t4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
+create table t5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
+create table t6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
+create table t7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
+create table t8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
+create table t9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
+select count(*) from t1;
+count(*)
+50
+select count(*) from t3;
+count(*)
+50
+select count(*) from t4;
+count(*)
+50
+select count(*) from t5;
+count(*)
+50
+select count(*) from t6;
+count(*)
+50
+select count(*) from t6;
+count(*)
+50
+select count(*) from t7;
+count(*)
+50
+select count(*) from t8;
+count(*)
+50
+select count(*) from t9;
+count(*)
+50
+# t0 expected FOUND
+NOT FOUND /AaAaAaAa/ in t0.ibd
+# t1 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t1.ibd
+# t2 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t2.ibd
+# t3 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t3.ibd
+# t4 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t4.ibd
+# t5 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t5.ibd
+# t6 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t6.ibd
+# t7 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t7.ibd
+# t8 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t8.ibd
+# t9 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t9.ibd
+# Run innochecksum on t9
+# Run innochecksum on t8
+# Run innochecksum on t7
+# Run innochecksum on t6
+# Run innochecksum on t5
+# Run innochecksum on t4
+# Run innochecksum on t3
+# Run innochecksum on t2
+# Run innochecksum on t1
+select count(*) from t0;
+count(*)
+50
+select count(*) from t1;
+count(*)
+50
+select count(*) from t3;
+count(*)
+50
+select count(*) from t4;
+count(*)
+50
+select count(*) from t5;
+count(*)
+50
+select count(*) from t6;
+count(*)
+50
+select count(*) from t7;
+count(*)
+50
+select count(*) from t8;
+count(*)
+50
+select count(*) from t9;
+count(*)
+50
+# Restart server
+# Corrupting tablespaces...
+# Corruption done
+# Run innochecksum on t0
+# Run innochecksum on t4
+# Run innochecksum on t3
+# Run innochecksum on t2
+# Start server again
+select * from t1;
+Got one of the listed errors
+select * from t2;
+Got one of the listed errors
+select * from t3;
+Got one of the listed errors
+select * from t4;
+Got one of the listed errors
+select count(*) from t5;
+count(*)
+50
+select count(*) from t6;
+count(*)
+50
+select count(*) from t7;
+count(*)
+50
+select count(*) from t8;
+count(*)
+50
+select count(*) from t9;
+count(*)
+50
+drop table t0, t1, t2, t3, t4, t5, t6, t7, t8, t9;
diff --git a/mysql-test/suite/encryption/r/innodb-encryption-disable.result b/mysql-test/suite/encryption/r/innodb-encryption-disable.result
index dd2b025553e..58cbe6db632 100644
--- a/mysql-test/suite/encryption/r/innodb-encryption-disable.result
+++ b/mysql-test/suite/encryption/r/innodb-encryption-disable.result
@@ -1,5 +1,4 @@
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t5 cannot be decrypted.");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t[1-9] page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
call mtr.add_suppression("Couldn't load plugins from 'file_key_management*");
create table t5 (
`intcol1` int(32) DEFAULT NULL,
diff --git a/mysql-test/suite/encryption/r/innodb-force-corrupt.result b/mysql-test/suite/encryption/r/innodb-force-corrupt.result
index 3f3a2afb02d..7d63f47d17e 100644
--- a/mysql-test/suite/encryption/r/innodb-force-corrupt.result
+++ b/mysql-test/suite/encryption/r/innodb-force-corrupt.result
@@ -1,9 +1,5 @@
-CALL mtr.add_suppression("InnoDB: Database page corruption on disk or a failed .*");
-CALL mtr.add_suppression("InnoDB: Corruption: Block in space_id .*");
-CALL mtr.add_suppression("InnoDB: However key management plugin or used key_version .*");
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t2 cannot be decrypted.");
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t3 cannot be decrypted.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t[123]\\.ibd' cannot be decrypted\\.");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t[0-9]+ page \[page id: space=[0-9]+, page number=[0-9]+\]. You may have to recover from a backup.");
SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_file_per_table = ON;
set global innodb_compression_algorithm = 1;
diff --git a/mysql-test/suite/encryption/r/innodb-import.result b/mysql-test/suite/encryption/r/innodb-import.result
index 1b4ec053d21..372798bf423 100644
--- a/mysql-test/suite/encryption/r/innodb-import.result
+++ b/mysql-test/suite/encryption/r/innodb-import.result
@@ -1,25 +1,37 @@
-call mtr.add_suppression("InnoDB: Table \'\"test\".\"t[1-2]_[l|b]e\"\' tablespace is set as discarded.");
+call mtr.add_suppression("InnoDB: Table \'\"test\".\"t[1-4]_[l|b]e\"\' tablespace is set as discarded.");
SET GLOBAL innodb_file_format = `Barracuda`;
create table t1_le(a int) engine=innodb encrypted=yes;
create table t2_le(a int) engine=innodb row_format=compressed encrypted=yes;
+create table t3_le(a int) engine=innodb page_compressed=yes;
+create table t4_le(a int) engine=innodb page_compressed=yes encrypted=yes;
create table t1_be(a int) engine=innodb encrypted=yes;
create table t2_be(a int) engine=innodb row_format=compressed encrypted=yes;
+create table t3_be(a int) engine=innodb page_compressed=yes;
+create table t4_be(a int) engine=innodb page_compressed=yes encrypted=yes;
ALTER TABLE t1_le DISCARD TABLESPACE;
ALTER TABLE t2_le DISCARD TABLESPACE;
+ALTER TABLE t3_le DISCARD TABLESPACE;
+ALTER TABLE t4_le DISCARD TABLESPACE;
ALTER TABLE t1_be DISCARD TABLESPACE;
ALTER TABLE t2_be DISCARD TABLESPACE;
+ALTER TABLE t3_be DISCARD TABLESPACE;
+ALTER TABLE t4_be DISCARD TABLESPACE;
+# innochecksum t1_le
+# innochecksum t2_le
+# innochecksum t3_le
+# innochecksum t4_le
+# innochecksum t1_be
+# innochecksum t2_be
+# innochecksum t3_be
+# innochecksum t3_be
ALTER TABLE t1_le IMPORT TABLESPACE;
-Warnings:
-Warning 1810 IO Read error: (2, No such file or directory) Error opening './test/t1_le.cfg', will attempt to import without schema verification
ALTER TABLE t1_be IMPORT TABLESPACE;
-Warnings:
-Warning 1810 IO Read error: (2, No such file or directory) Error opening './test/t1_be.cfg', will attempt to import without schema verification
ALTER TABLE t2_le IMPORT TABLESPACE;
-Warnings:
-Warning 1810 IO Read error: (2, No such file or directory) Error opening './test/t2_le.cfg', will attempt to import without schema verification
ALTER TABLE t2_be IMPORT TABLESPACE;
-Warnings:
-Warning 1810 IO Read error: (2, No such file or directory) Error opening './test/t2_be.cfg', will attempt to import without schema verification
+ALTER TABLE t3_le IMPORT TABLESPACE;
+ALTER TABLE t3_be IMPORT TABLESPACE;
+ALTER TABLE t4_le IMPORT TABLESPACE;
+ALTER TABLE t4_be IMPORT TABLESPACE;
select * from t1_le;
a
select * from t1_be;
@@ -28,5 +40,14 @@ select * from t2_le;
a
select * from t2_be;
a
+select * from t3_le;
+a
+select * from t3_be;
+a
+select * from t4_le;
+a
+select * from t4_be;
+a
drop table t1_le, t1_be, t2_le, t2_be;
+drop table t3_le, t3_be, t4_le, t4_be;
SET GLOBAL innodb_file_format = Default;
diff --git a/mysql-test/suite/encryption/r/innodb-missing-key.result b/mysql-test/suite/encryption/r/innodb-missing-key.result
index 2f2cc025973..cd01f4c4004 100644
--- a/mysql-test/suite/encryption/r/innodb-missing-key.result
+++ b/mysql-test/suite/encryption/r/innodb-missing-key.result
@@ -1,6 +1,6 @@
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t2 cannot be decrypted.");
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t3 cannot be decrypted.");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t1 page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Table \"test\".\"t1\" is corrupted. Please drop the table and recreate.");
+call mtr.add_suppression("InnoDB: Cannot open table test/t1 from the internal data dictionary of InnoDB though the .frm file for the table exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.");
# Start server with keys2.txt
CREATE TABLE t1(a int not null primary key auto_increment, b varchar(128)) engine=innodb ENCRYPTED=YES ENCRYPTION_KEY_ID=19;
@@ -37,9 +37,9 @@ COUNT(1)
SELECT COUNT(1) FROM t2,t1 where t2.a = t1.a;
ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
SELECT COUNT(1) FROM t1 where b = 'ab';
-ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
+Got one of the listed errors
SELECT COUNT(1) FROM t1;
-ERROR HY000: Got error 192 'Table encrypted but decryption failed. This could be because correct encryption management plugin is not loaded, used encryption key is not available or encryption method does not match.' from InnoDB
+Got one of the listed errors
# Start server with keys2.txt
SELECT COUNT(1) FROM t1;
diff --git a/mysql-test/suite/encryption/r/innodb_encryption-page-compression.result b/mysql-test/suite/encryption/r/innodb_encryption-page-compression.result
index 14933e526c4..4acfca801ab 100644
--- a/mysql-test/suite/encryption/r/innodb_encryption-page-compression.result
+++ b/mysql-test/suite/encryption/r/innodb_encryption-page-compression.result
@@ -1,189 +1,146 @@
-SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_encryption_threads = 4;
+select @@innodb_encryption_threads;
+@@innodb_encryption_threads
+4
SET GLOBAL innodb_encrypt_tables = on;
-set global innodb_compression_algorithm = 1;
-create table innodb_normal (c1 int, b char(20)) engine=innodb;
-show warnings;
-Level Code Message
-create table innodb_page_compressed1 (c1 int, b char(20)) engine=innodb page_compressed=1 page_compression_level=1;
-show warnings;
-Level Code Message
-show create table innodb_page_compressed1;
-Table Create Table
-innodb_page_compressed1 CREATE TABLE `innodb_page_compressed1` (
- `c1` int(11) DEFAULT NULL,
- `b` char(20) DEFAULT NULL
-) ENGINE=InnoDB DEFAULT CHARSET=latin1 `page_compressed`=1 `page_compression_level`=1
-create table innodb_page_compressed2 (c1 int, b char(20)) engine=innodb page_compressed=1 page_compression_level=2;
-show warnings;
-Level Code Message
-show create table innodb_page_compressed2;
-Table Create Table
-innodb_page_compressed2 CREATE TABLE `innodb_page_compressed2` (
- `c1` int(11) DEFAULT NULL,
- `b` char(20) DEFAULT NULL
-) ENGINE=InnoDB DEFAULT CHARSET=latin1 `page_compressed`=1 `page_compression_level`=2
-create table innodb_page_compressed3 (c1 int, b char(20)) engine=innodb page_compressed=1 page_compression_level=3;
-show warnings;
-Level Code Message
-show create table innodb_page_compressed3;
-Table Create Table
-innodb_page_compressed3 CREATE TABLE `innodb_page_compressed3` (
- `c1` int(11) DEFAULT NULL,
- `b` char(20) DEFAULT NULL
-) ENGINE=InnoDB DEFAULT CHARSET=latin1 `page_compressed`=1 `page_compression_level`=3
-create table innodb_page_compressed4 (c1 int, b char(20)) engine=innodb page_compressed=1 page_compression_level=4;
-show warnings;
-Level Code Message
-show create table innodb_page_compressed4;
-Table Create Table
-innodb_page_compressed4 CREATE TABLE `innodb_page_compressed4` (
- `c1` int(11) DEFAULT NULL,
- `b` char(20) DEFAULT NULL
-) ENGINE=InnoDB DEFAULT CHARSET=latin1 `page_compressed`=1 `page_compression_level`=4
-create table innodb_page_compressed5 (c1 int, b char(20)) engine=innodb page_compressed=1 page_compression_level=5;
-show warnings;
-Level Code Message
-show create table innodb_page_compressed5;
-Table Create Table
-innodb_page_compressed5 CREATE TABLE `innodb_page_compressed5` (
- `c1` int(11) DEFAULT NULL,
- `b` char(20) DEFAULT NULL
-) ENGINE=InnoDB DEFAULT CHARSET=latin1 `page_compressed`=1 `page_compression_level`=5
-create table innodb_page_compressed6 (c1 int, b char(20)) engine=innodb page_compressed=1 page_compression_level=6;
-show warnings;
-Level Code Message
-show create table innodb_page_compressed6;
-Table Create Table
-innodb_page_compressed6 CREATE TABLE `innodb_page_compressed6` (
- `c1` int(11) DEFAULT NULL,
- `b` char(20) DEFAULT NULL
-) ENGINE=InnoDB DEFAULT CHARSET=latin1 `page_compressed`=1 `page_compression_level`=6
-create table innodb_page_compressed7 (c1 int, b char(20)) engine=innodb page_compressed=1 page_compression_level=7;
-show warnings;
-Level Code Message
-show create table innodb_page_compressed7;
-Table Create Table
-innodb_page_compressed7 CREATE TABLE `innodb_page_compressed7` (
- `c1` int(11) DEFAULT NULL,
- `b` char(20) DEFAULT NULL
-) ENGINE=InnoDB DEFAULT CHARSET=latin1 `page_compressed`=1 `page_compression_level`=7
-create table innodb_page_compressed8 (c1 int, b char(20)) engine=innodb page_compressed=1 page_compression_level=8;
-show warnings;
-Level Code Message
-show create table innodb_page_compressed8;
-Table Create Table
-innodb_page_compressed8 CREATE TABLE `innodb_page_compressed8` (
- `c1` int(11) DEFAULT NULL,
- `b` char(20) DEFAULT NULL
-) ENGINE=InnoDB DEFAULT CHARSET=latin1 `page_compressed`=1 `page_compression_level`=8
-create table innodb_page_compressed9 (c1 int, b char(20)) engine=innodb page_compressed=1 page_compression_level=9;
-show warnings;
-Level Code Message
-show create table innodb_page_compressed9;
-Table Create Table
-innodb_page_compressed9 CREATE TABLE `innodb_page_compressed9` (
- `c1` int(11) DEFAULT NULL,
- `b` char(20) DEFAULT NULL
-) ENGINE=InnoDB DEFAULT CHARSET=latin1 `page_compressed`=1 `page_compression_level`=9
-create procedure innodb_insert_proc (repeat_count int)
-begin
-declare current_num int;
-set current_num = 0;
-while current_num < repeat_count do
-insert into innodb_normal values(current_num,'testing..');
-set current_num = current_num + 1;
-end while;
-end//
-commit;
-begin;
-call innodb_insert_proc(2000);
-insert into innodb_page_compressed1 select * from innodb_normal;
-insert into innodb_page_compressed2 select * from innodb_normal;
-insert into innodb_page_compressed3 select * from innodb_normal;
-insert into innodb_page_compressed4 select * from innodb_normal;
-insert into innodb_page_compressed5 select * from innodb_normal;
-insert into innodb_page_compressed6 select * from innodb_normal;
-insert into innodb_page_compressed7 select * from innodb_normal;
-insert into innodb_page_compressed8 select * from innodb_normal;
-insert into innodb_page_compressed9 select * from innodb_normal;
-commit;
-select count(*) from innodb_page_compressed1 where c1 < 500000;
-count(*)
-2000
-select count(*) from innodb_page_compressed2 where c1 < 500000;
-count(*)
-2000
-select count(*) from innodb_page_compressed3 where c1 < 500000;
-count(*)
-2000
-select count(*) from innodb_page_compressed4 where c1 < 500000;
-count(*)
-2000
-select count(*) from innodb_page_compressed5 where c1 < 500000;
-count(*)
-2000
-select count(*) from innodb_page_compressed6 where c1 < 500000;
-count(*)
-2000
-select count(*) from innodb_page_compressed7 where c1 < 500000;
-count(*)
-2000
-select count(*) from innodb_page_compressed8 where c1 < 500000;
-count(*)
-2000
-select count(*) from innodb_page_compressed9 where c1 < 500000;
-count(*)
-2000
-flush tables innodb_page_compressed1, innodb_page_compressed2,
-innodb_page_compressed3, innodb_page_compressed4,
-innodb_page_compressed5, innodb_page_compressed6,
-innodb_page_compressed7, innodb_page_compressed8,
-innodb_page_compressed9 for export;
-unlock tables;
-# Wait until dirty pages are compressed and encrypted
-SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
-variable_value > 0
-1
-SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_compressed';
-variable_value > 0
-1
-SET GLOBAL innodb_encryption_threads = 4;
-SET GLOBAL innodb_encrypt_tables = off;
-update innodb_page_compressed1 set c1 = c1 + 1;
-update innodb_page_compressed2 set c1 = c1 + 1;
-update innodb_page_compressed3 set c1 = c1 + 1;
-update innodb_page_compressed4 set c1 = c1 + 1;
-update innodb_page_compressed5 set c1 = c1 + 1;
-update innodb_page_compressed6 set c1 = c1 + 1;
-update innodb_page_compressed7 set c1 = c1 + 1;
-update innodb_page_compressed8 set c1 = c1 + 1;
-update innodb_page_compressed9 set c1 = c1 + 1;
-flush tables innodb_page_compressed1, innodb_page_compressed2,
-innodb_page_compressed3, innodb_page_compressed4,
-innodb_page_compressed5, innodb_page_compressed6,
-innodb_page_compressed7, innodb_page_compressed8,
-innodb_page_compressed9 for export;
-unlock tables;
-# Wait until dirty pages are compressed and encrypted 2
-unlock tables;
-SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
-variable_value > 0
-1
-SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_compressed';
-variable_value > 0
-1
-SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_decompressed';
-variable_value > 0
-1
-drop procedure innodb_insert_proc;
-drop table innodb_normal;
-drop table innodb_page_compressed1;
-drop table innodb_page_compressed2;
-drop table innodb_page_compressed3;
-drop table innodb_page_compressed4;
-drop table innodb_page_compressed5;
-drop table innodb_page_compressed6;
-drop table innodb_page_compressed7;
-drop table innodb_page_compressed8;
-drop table innodb_page_compressed9;
+select @@innodb_encrypt_tables;
+@@innodb_encrypt_tables
+ON
+set global innodb_compression_algorithm = zlib;
+select @@innodb_compression_algorithm;
+@@innodb_compression_algorithm
+zlib
+set global innodb_file_format = `Barracuda`;
+set global innodb_file_per_table = on;
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t1 page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t2 page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t3 page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t4 page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Page \\[page id: space=[0-9]+, page number= [0-9]+\\] in file ./test/t[0-9]+.ibd may be corrupted. Post compression checksum [0-9]+ stored [0-9]+ compression_method [ZLIB|SNAPPY|LZ4|LZO|LZMA|BZIP2]");
+call mtr.add_suppression("InnoDB: Background Page read failed to read or decrypt \\[page id: space=[0-9]+, page number=[0-9]+\\]");
+call mtr.add_suppression("mysqld: Index for table 't[0-9]+' is corrupt; try to repair it");
+create table t0 (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
+create table t1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
+create table t2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
+create table t3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
+create table t4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
+create table t5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
+create table t6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
+create table t7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
+create table t8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
+create table t9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
+select count(*) from t1;
+count(*)
+50
+select count(*) from t3;
+count(*)
+50
+select count(*) from t4;
+count(*)
+50
+select count(*) from t5;
+count(*)
+50
+select count(*) from t6;
+count(*)
+50
+select count(*) from t6;
+count(*)
+50
+select count(*) from t7;
+count(*)
+50
+select count(*) from t8;
+count(*)
+50
+select count(*) from t9;
+count(*)
+50
+# t0 expected FOUND
+NOT FOUND /AaAaAaAa/ in t0.ibd
+# t1 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t1.ibd
+# t2 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t2.ibd
+# t3 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t3.ibd
+# t4 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t4.ibd
+# t5 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t5.ibd
+# t6 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t6.ibd
+# t7 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t7.ibd
+# t8 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t8.ibd
+# t9 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t9.ibd
+# Run innochecksum on t9
+# Run innochecksum on t8
+# Run innochecksum on t7
+# Run innochecksum on t6
+# Run innochecksum on t5
+# Run innochecksum on t4
+# Run innochecksum on t3
+# Run innochecksum on t2
+# Run innochecksum on t1
+select count(*) from t0;
+count(*)
+50
+select count(*) from t1;
+count(*)
+50
+select count(*) from t3;
+count(*)
+50
+select count(*) from t4;
+count(*)
+50
+select count(*) from t5;
+count(*)
+50
+select count(*) from t6;
+count(*)
+50
+select count(*) from t7;
+count(*)
+50
+select count(*) from t8;
+count(*)
+50
+select count(*) from t9;
+count(*)
+50
+# Restart server
+# Corrupting tablespaces...
+# Corruption done
+# Run innochecksum on t0
+# Run innochecksum on t4
+# Run innochecksum on t3
+# Run innochecksum on t2
+# Start server again
+select * from t1;
+Got one of the listed errors
+select * from t2;
+Got one of the listed errors
+select * from t3;
+Got one of the listed errors
+select * from t4;
+Got one of the listed errors
+select count(*) from t5;
+count(*)
+50
+select count(*) from t6;
+count(*)
+50
+select count(*) from t7;
+count(*)
+50
+select count(*) from t8;
+count(*)
+50
+select count(*) from t9;
+count(*)
+50
+drop table t0, t1, t2, t3, t4, t5, t6, t7, t8, t9;
diff --git a/mysql-test/suite/encryption/t/innodb-bad-key-change.test b/mysql-test/suite/encryption/t/innodb-bad-key-change.test
index 6fa5fc9847f..5416d3f1273 100644
--- a/mysql-test/suite/encryption/t/innodb-bad-key-change.test
+++ b/mysql-test/suite/encryption/t/innodb-bad-key-change.test
@@ -8,10 +8,12 @@
# table exists and encryption service is not available.
#
-call mtr.add_suppression("InnoDB: The page .*");
-call mtr.add_suppression("InnoDB: Cannot calculate statistics for table .* ");
-call mtr.add_suppression("Plugin 'file_key_management' .*");
-call mtr.add_suppression("mysqld: File .*");
+call mtr.add_suppression("Plugin 'file_key_management' init function returned error");
+call mtr.add_suppression("Plugin 'file_key_management' registration.*failed");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t1 page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[0-9]+, page number=[0-9]+\\] in file ./test/t1.ibd cannot be decrypted.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[0-9]+, page number=[0-9]+\\] in file ./test/t2.ibd cannot be decrypted.");
+call mtr.add_suppression("mysqld: File .*keysbad3.txt' not found .*");
--echo
--echo # Start server with keys2.txt
diff --git a/mysql-test/suite/encryption/t/innodb-bad-key-change2.test b/mysql-test/suite/encryption/t/innodb-bad-key-change2.test
index 52ee442c725..c2459534f17 100644
--- a/mysql-test/suite/encryption/t/innodb-bad-key-change2.test
+++ b/mysql-test/suite/encryption/t/innodb-bad-key-change2.test
@@ -8,8 +8,8 @@
# MDEV-8768: Server crash at file btr0btr.ic line 122 when checking encrypted table using incorrect keys
# MDEV-8727: Server/InnoDB hangs on shutdown after trying to read an encrypted table with a wrong key
#
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1new cannot be decrypted.");
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[0-9]+, page number=[0-9]+\\] in file ./test/t1new.ibd cannot be decrypted.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[0-9]+, page number=[0-9]+\\] in file ./test/t1.ibd cannot be decrypted.");
# Suppression for builds where file_key_management plugin is linked statically
call mtr.add_suppression("Couldn't load plugins from 'file_key_management.*");
call mtr.add_suppression("InnoDB: Table \'\"test\".\"t1\"\' tablespace is set as discarded.");
diff --git a/mysql-test/suite/encryption/t/innodb-bad-key-change4.test b/mysql-test/suite/encryption/t/innodb-bad-key-change4.test
index 14d88614f55..d38e962bd79 100644
--- a/mysql-test/suite/encryption/t/innodb-bad-key-change4.test
+++ b/mysql-test/suite/encryption/t/innodb-bad-key-change4.test
@@ -7,7 +7,7 @@
# MDEV-8768: Server crash at file btr0btr.ic line 122 when checking encrypted table using incorrect keys
#
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[0-9]+, page number=[0-9]+\\] in file ./test/t1.ibd cannot be decrypted.");
call mtr.add_suppression("InnoDB: Cannot open table .*");
call mtr.add_suppression("InnoDB: .ibd file is missing for table test/.*");
# Suppression for builds where file_key_management plugin is linked statically
diff --git a/mysql-test/suite/encryption/t/innodb-compressed-blob.test b/mysql-test/suite/encryption/t/innodb-compressed-blob.test
index cb69f22f745..8af2363aed8 100644
--- a/mysql-test/suite/encryption/t/innodb-compressed-blob.test
+++ b/mysql-test/suite/encryption/t/innodb-compressed-blob.test
@@ -5,7 +5,7 @@
-- source include/not_embedded.inc
call mtr.add_suppression("InnoDB: However key management plugin or used key_version .*");
-call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[0-9]+\\] in file test/t[1-3] cannot be decrypted.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[0-9]+, page number=[0-9]+\\] in file ./test/t[1-3].ibd cannot be decrypted.");
call mtr.add_suppression("Unable to decompress space ..test.t[1-3].ibd \\[[1-9][0-9]*:[0-9]+\\]");
--echo # Restart mysqld --file-key-management-filename=keys2.txt
diff --git a/mysql-test/suite/encryption/t/innodb-corrupted.test b/mysql-test/suite/encryption/t/innodb-corrupted.test
new file mode 100644
index 00000000000..10a958b2468
--- /dev/null
+++ b/mysql-test/suite/encryption/t/innodb-corrupted.test
@@ -0,0 +1,25 @@
+# Don't test under embedded as we restart server
+-- source include/not_embedded.inc
+# Require InnoDB
+-- source include/have_innodb.inc
+-- source include/have_file_key_management_plugin.inc
+
+let $encrypt=`SELECT @@innodb_compression_algorithm`;
+let $encrypt=`SELECT @@innodb_default_encryption_key_id`;
+
+set global innodb_encrypt_tables='FORCE';
+select @@innodb_encrypt_tables;
+set global innodb_default_encryption_key_id=4;
+select @@innodb_default_encryption_key_id;
+set global innodb_compression_algorithm=zlib;
+select @@innodb_compression_algorithm;
+
+--source include/innodb-page-compression.inc
+
+--disable_query_log
+set global innodb_encrypt_tables=default;
+set global innodb_default_encryption_key_id=default;
+set global innodb_compression_algorithm=default;
+--enable_query_log
+
+
diff --git a/mysql-test/suite/encryption/t/innodb-encryption-disable.test b/mysql-test/suite/encryption/t/innodb-encryption-disable.test
index 38f36076c73..2aa4a89c8d4 100644
--- a/mysql-test/suite/encryption/t/innodb-encryption-disable.test
+++ b/mysql-test/suite/encryption/t/innodb-encryption-disable.test
@@ -7,8 +7,7 @@
# MDEV-9559: Server without encryption configs crashes if selecting from an implicitly encrypted table
#
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t5 cannot be decrypted.");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t[1-9] page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
# Suppression for builds where file_key_management plugin is linked statically
call mtr.add_suppression("Couldn't load plugins from 'file_key_management*");
diff --git a/mysql-test/suite/encryption/t/innodb-force-corrupt.test b/mysql-test/suite/encryption/t/innodb-force-corrupt.test
index dd4ee51b1eb..cb6440127cc 100644
--- a/mysql-test/suite/encryption/t/innodb-force-corrupt.test
+++ b/mysql-test/suite/encryption/t/innodb-force-corrupt.test
@@ -7,12 +7,8 @@
# Don't test under embedded
-- source include/not_embedded.inc
-CALL mtr.add_suppression("InnoDB: Database page corruption on disk or a failed .*");
-CALL mtr.add_suppression("InnoDB: Corruption: Block in space_id .*");
-CALL mtr.add_suppression("InnoDB: However key management plugin or used key_version .*");
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t2 cannot be decrypted.");
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t3 cannot be decrypted.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t[123]\\.ibd' cannot be decrypted\\.");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t[0-9]+ page \[page id: space=[0-9]+, page number=[0-9]+\]. You may have to recover from a backup.");
--disable_warnings
SET GLOBAL innodb_file_format = `Barracuda`;
diff --git a/mysql-test/suite/encryption/t/innodb-import.test b/mysql-test/suite/encryption/t/innodb-import.test
index ab4c20f3ea5..d96910b29d5 100644
--- a/mysql-test/suite/encryption/t/innodb-import.test
+++ b/mysql-test/suite/encryption/t/innodb-import.test
@@ -1,11 +1,17 @@
--source include/have_innodb.inc
--source include/have_file_key_management_plugin.inc
+if (!$INNOCHECKSUM) {
+ --echo Need innochecksum binary
+ --die Need innochecksum binary
+}
+
#
-# MDEV-15032 Encrypted database created on big endian machine cannot be used on little endian machine
+# MDEV-15032: Encrypted database created on big endian machine cannot be used on little endian machine
+# MDEV-13103: InnoDB crash recovery fails to decompress a page in buf_dblwr_process()
#
-call mtr.add_suppression("InnoDB: Table \'\"test\".\"t[1-2]_[l|b]e\"\' tablespace is set as discarded.");
+call mtr.add_suppression("InnoDB: Table \'\"test\".\"t[1-4]_[l|b]e\"\' tablespace is set as discarded.");
--disable_warnings
SET GLOBAL innodb_file_format = `Barracuda`;
@@ -13,41 +19,87 @@ SET GLOBAL innodb_file_format = `Barracuda`;
create table t1_le(a int) engine=innodb encrypted=yes;
create table t2_le(a int) engine=innodb row_format=compressed encrypted=yes;
+create table t3_le(a int) engine=innodb page_compressed=yes;
+create table t4_le(a int) engine=innodb page_compressed=yes encrypted=yes;
create table t1_be(a int) engine=innodb encrypted=yes;
create table t2_be(a int) engine=innodb row_format=compressed encrypted=yes;
-
-#
-# TODO: In MDEV-13103 InnoDB crash recovery fails to decompress a page in buf_dblwr_process()
-# add page compressed tables.
-#
+create table t3_be(a int) engine=innodb page_compressed=yes;
+create table t4_be(a int) engine=innodb page_compressed=yes encrypted=yes;
ALTER TABLE t1_le DISCARD TABLESPACE;
ALTER TABLE t2_le DISCARD TABLESPACE;
+ALTER TABLE t3_le DISCARD TABLESPACE;
+ALTER TABLE t4_le DISCARD TABLESPACE;
ALTER TABLE t1_be DISCARD TABLESPACE;
ALTER TABLE t2_be DISCARD TABLESPACE;
+ALTER TABLE t3_be DISCARD TABLESPACE;
+ALTER TABLE t4_be DISCARD TABLESPACE;
let $MYSQLD_DATADIR =`SELECT @@datadir`;
#
+# Test innochecksum
+#
+
+--disable_result_log
+
+--echo # innochecksum t1_le
+--exec $INNOCHECKSUM $MYSQL_TEST_DIR/std_data/t1_le.ibd
+--echo # innochecksum t2_le
+--exec $INNOCHECKSUM $MYSQL_TEST_DIR/std_data/t2_le.ibd
+--echo # innochecksum t3_le
+--exec $INNOCHECKSUM $MYSQL_TEST_DIR/std_data/t3_le.ibd
+--echo # innochecksum t4_le
+--exec $INNOCHECKSUM $MYSQL_TEST_DIR/std_data/t4_le.ibd
+--echo # innochecksum t1_be
+--exec $INNOCHECKSUM $MYSQL_TEST_DIR/std_data/t1_be.ibd
+--echo # innochecksum t2_be
+--exec $INNOCHECKSUM $MYSQL_TEST_DIR/std_data/t2_be.ibd
+--echo # innochecksum t3_be
+--exec $INNOCHECKSUM $MYSQL_TEST_DIR/std_data/t3_be.ibd
+--echo # innochecksum t3_be
+--exec $INNOCHECKSUM $MYSQL_TEST_DIR/std_data/t4_be.ibd
+
+--enable_result_log
+
+#
# Copy exported datafiles
#
---copy_file std_data/t1_le.ibd $MYSQLD_DATADIR/test/t1_le.ibd
---copy_file std_data/t1_be.ibd $MYSQLD_DATADIR/test/t1_be.ibd
---copy_file std_data/t2_le.ibd $MYSQLD_DATADIR/test/t2_le.ibd
---copy_file std_data/t2_be.ibd $MYSQLD_DATADIR/test/t2_be.ibd
+--copy_file $MYSQL_TEST_DIR/std_data/t1_le.ibd $MYSQLD_DATADIR/test/t1_le.ibd
+--copy_file $MYSQL_TEST_DIR/std_data/t1_be.ibd $MYSQLD_DATADIR/test/t1_be.ibd
+--copy_file $MYSQL_TEST_DIR/std_data/t2_le.ibd $MYSQLD_DATADIR/test/t2_le.ibd
+--copy_file $MYSQL_TEST_DIR/std_data/t2_be.ibd $MYSQLD_DATADIR/test/t2_be.ibd
+--copy_file $MYSQL_TEST_DIR/std_data/t3_le.ibd $MYSQLD_DATADIR/test/t3_le.ibd
+--copy_file $MYSQL_TEST_DIR/std_data/t3_be.ibd $MYSQLD_DATADIR/test/t3_be.ibd
+--copy_file $MYSQL_TEST_DIR/std_data/t4_le.ibd $MYSQLD_DATADIR/test/t4_le.ibd
+--copy_file $MYSQL_TEST_DIR/std_data/t4_be.ibd $MYSQLD_DATADIR/test/t4_be.ibd
+#
+# We do not use .cfg so disable warnings
+#
+--disable_warnings
ALTER TABLE t1_le IMPORT TABLESPACE;
ALTER TABLE t1_be IMPORT TABLESPACE;
ALTER TABLE t2_le IMPORT TABLESPACE;
ALTER TABLE t2_be IMPORT TABLESPACE;
+ALTER TABLE t3_le IMPORT TABLESPACE;
+ALTER TABLE t3_be IMPORT TABLESPACE;
+ALTER TABLE t4_le IMPORT TABLESPACE;
+ALTER TABLE t4_be IMPORT TABLESPACE;
+--enable_warnings
select * from t1_le;
select * from t1_be;
select * from t2_le;
select * from t2_be;
+select * from t3_le;
+select * from t3_be;
+select * from t4_le;
+select * from t4_be;
drop table t1_le, t1_be, t2_le, t2_be;
+drop table t3_le, t3_be, t4_le, t4_be;
--disable_warnings
SET GLOBAL innodb_file_format = Default;
diff --git a/mysql-test/suite/encryption/t/innodb-missing-key.test b/mysql-test/suite/encryption/t/innodb-missing-key.test
index 07a2b16211c..6fbf8b35882 100644
--- a/mysql-test/suite/encryption/t/innodb-missing-key.test
+++ b/mysql-test/suite/encryption/t/innodb-missing-key.test
@@ -7,9 +7,9 @@
# MDEV-11004: Unable to start (Segfault or os error 2) when encryption key missing
#
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t1 cannot be decrypted.");
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t2 cannot be decrypted.");
-call mtr.add_suppression("InnoDB: The page \[page id: space=[0-9]+, page number=[0-9]+\] in file test/t3 cannot be decrypted.");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t1 page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Table \"test\".\"t1\" is corrupted. Please drop the table and recreate.");
+call mtr.add_suppression("InnoDB: Cannot open table test/t1 from the internal data dictionary of InnoDB though the .frm file for the table exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.");
--echo
--echo # Start server with keys2.txt
@@ -45,11 +45,11 @@ CREATE TABLE t4(a int not null primary key auto_increment, b varchar(128)) engin
SELECT SLEEP(5);
SELECT COUNT(1) FROM t3;
SELECT COUNT(1) FROM t2;
---error 1296
+--error ER_GET_ERRMSG
SELECT COUNT(1) FROM t2,t1 where t2.a = t1.a;
---error 1296
+--error ER_GET_ERRMSG,ER_NO_SUCH_TABLE_IN_ENGINE
SELECT COUNT(1) FROM t1 where b = 'ab';
---error 1296
+--error ER_GET_ERRMSG,ER_NO_SUCH_TABLE_IN_ENGINE
SELECT COUNT(1) FROM t1;
--echo
diff --git a/mysql-test/suite/encryption/t/innodb_encryption-page-compression.test b/mysql-test/suite/encryption/t/innodb_encryption-page-compression.test
index fe132c0c59b..8b0d7ca8c15 100644
--- a/mysql-test/suite/encryption/t/innodb_encryption-page-compression.test
+++ b/mysql-test/suite/encryption/t/innodb_encryption-page-compression.test
@@ -2,150 +2,11 @@
-- source include/have_example_key_management_plugin.inc
-- source include/not_embedded.inc
---disable_query_log
-let $innodb_encrypt_tables_orig = `SELECT @@innodb_encrypt_tables`;
-let $innodb_encryption_threads_orig = `SELECT @@innodb_encryption_threads`;
---enable_query_log
-
---disable_warnings
-SET GLOBAL innodb_file_format = `Barracuda`;
SET GLOBAL innodb_encryption_threads = 4;
+select @@innodb_encryption_threads;
SET GLOBAL innodb_encrypt_tables = on;
---enable_warnings
-
-# zlib
-set global innodb_compression_algorithm = 1;
-
-create table innodb_normal (c1 int, b char(20)) engine=innodb;
-show warnings;
-create table innodb_page_compressed1 (c1 int, b char(20)) engine=innodb page_compressed=1 page_compression_level=1;
-show warnings;
-show create table innodb_page_compressed1;
-create table innodb_page_compressed2 (c1 int, b char(20)) engine=innodb page_compressed=1 page_compression_level=2;
-show warnings;
-show create table innodb_page_compressed2;
-create table innodb_page_compressed3 (c1 int, b char(20)) engine=innodb page_compressed=1 page_compression_level=3;
-show warnings;
-show create table innodb_page_compressed3;
-create table innodb_page_compressed4 (c1 int, b char(20)) engine=innodb page_compressed=1 page_compression_level=4;
-show warnings;
-show create table innodb_page_compressed4;
-create table innodb_page_compressed5 (c1 int, b char(20)) engine=innodb page_compressed=1 page_compression_level=5;
-show warnings;
-show create table innodb_page_compressed5;
-create table innodb_page_compressed6 (c1 int, b char(20)) engine=innodb page_compressed=1 page_compression_level=6;
-show warnings;
-show create table innodb_page_compressed6;
-create table innodb_page_compressed7 (c1 int, b char(20)) engine=innodb page_compressed=1 page_compression_level=7;
-show warnings;
-show create table innodb_page_compressed7;
-create table innodb_page_compressed8 (c1 int, b char(20)) engine=innodb page_compressed=1 page_compression_level=8;
-show warnings;
-show create table innodb_page_compressed8;
-create table innodb_page_compressed9 (c1 int, b char(20)) engine=innodb page_compressed=1 page_compression_level=9;
-show warnings;
-show create table innodb_page_compressed9;
-delimiter //;
-create procedure innodb_insert_proc (repeat_count int)
-begin
- declare current_num int;
- set current_num = 0;
- while current_num < repeat_count do
- insert into innodb_normal values(current_num,'testing..');
- set current_num = current_num + 1;
- end while;
-end//
-delimiter ;//
-commit;
-
-begin;
-call innodb_insert_proc(2000);
-insert into innodb_page_compressed1 select * from innodb_normal;
-insert into innodb_page_compressed2 select * from innodb_normal;
-insert into innodb_page_compressed3 select * from innodb_normal;
-insert into innodb_page_compressed4 select * from innodb_normal;
-insert into innodb_page_compressed5 select * from innodb_normal;
-insert into innodb_page_compressed6 select * from innodb_normal;
-insert into innodb_page_compressed7 select * from innodb_normal;
-insert into innodb_page_compressed8 select * from innodb_normal;
-insert into innodb_page_compressed9 select * from innodb_normal;
-commit;
-
-select count(*) from innodb_page_compressed1 where c1 < 500000;
-select count(*) from innodb_page_compressed2 where c1 < 500000;
-select count(*) from innodb_page_compressed3 where c1 < 500000;
-select count(*) from innodb_page_compressed4 where c1 < 500000;
-select count(*) from innodb_page_compressed5 where c1 < 500000;
-select count(*) from innodb_page_compressed6 where c1 < 500000;
-select count(*) from innodb_page_compressed7 where c1 < 500000;
-select count(*) from innodb_page_compressed8 where c1 < 500000;
-select count(*) from innodb_page_compressed9 where c1 < 500000;
-
-flush tables innodb_page_compressed1, innodb_page_compressed2,
-innodb_page_compressed3, innodb_page_compressed4,
-innodb_page_compressed5, innodb_page_compressed6,
-innodb_page_compressed7, innodb_page_compressed8,
-innodb_page_compressed9 for export;
-
-unlock tables;
-
---echo # Wait until dirty pages are compressed and encrypted
-let $wait_condition= select variable_value > 0 from information_schema.global_status where variable_name = 'INNODB_NUM_PAGES_PAGE_COMPRESSED';
---source include/wait_condition.inc
-let $wait_condition= select variable_value > 0 from information_schema.global_status where variable_name = 'INNODB_NUM_PAGES_ENCRYPTED';
---source include/wait_condition.inc
-
-SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_encrypted';
-SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_compressed';
-
---source include/restart_mysqld.inc
-
-SET GLOBAL innodb_encryption_threads = 4;
-SET GLOBAL innodb_encrypt_tables = off;
-
-update innodb_page_compressed1 set c1 = c1 + 1;
-update innodb_page_compressed2 set c1 = c1 + 1;
-update innodb_page_compressed3 set c1 = c1 + 1;
-update innodb_page_compressed4 set c1 = c1 + 1;
-update innodb_page_compressed5 set c1 = c1 + 1;
-update innodb_page_compressed6 set c1 = c1 + 1;
-update innodb_page_compressed7 set c1 = c1 + 1;
-update innodb_page_compressed8 set c1 = c1 + 1;
-update innodb_page_compressed9 set c1 = c1 + 1;
-
-flush tables innodb_page_compressed1, innodb_page_compressed2,
-innodb_page_compressed3, innodb_page_compressed4,
-innodb_page_compressed5, innodb_page_compressed6,
-innodb_page_compressed7, innodb_page_compressed8,
-innodb_page_compressed9 for export;
-unlock tables;
-
---echo # Wait until dirty pages are compressed and encrypted 2
-let $wait_condition= select variable_value > 0 from information_schema.global_status where variable_name = 'INNODB_NUM_PAGES_PAGE_COMPRESSED';
---source include/wait_condition.inc
-unlock tables;
-let $wait_condition= select variable_value > 0 from information_schema.global_status where variable_name = 'INNODB_NUM_PAGES_DECRYPTED';
---source include/wait_condition.inc
-
-SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_decrypted';
-SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_compressed';
-SELECT variable_value > 0 FROM information_schema.global_status WHERE variable_name = 'innodb_num_pages_page_decompressed';
-
-drop procedure innodb_insert_proc;
-drop table innodb_normal;
-drop table innodb_page_compressed1;
-drop table innodb_page_compressed2;
-drop table innodb_page_compressed3;
-drop table innodb_page_compressed4;
-drop table innodb_page_compressed5;
-drop table innodb_page_compressed6;
-drop table innodb_page_compressed7;
-drop table innodb_page_compressed8;
-drop table innodb_page_compressed9;
-
-# reset system
---disable_query_log
-EVAL SET GLOBAL innodb_encrypt_tables = $innodb_encrypt_tables_orig;
-EVAL SET GLOBAL innodb_encryption_threads = $innodb_encryption_threads_orig;
---enable_query_log
+select @@innodb_encrypt_tables;
+set global innodb_compression_algorithm = zlib;
+select @@innodb_compression_algorithm;
+--source include/innodb-page-compression.inc
diff --git a/mysql-test/suite/innodb/include/innodb-page-compression.inc b/mysql-test/suite/innodb/include/innodb-page-compression.inc
index 3acbeaf0988..8a0036d61e0 100644
--- a/mysql-test/suite/innodb/include/innodb-page-compression.inc
+++ b/mysql-test/suite/innodb/include/innodb-page-compression.inc
@@ -1,53 +1,63 @@
+if (!$INNOCHECKSUM) {
+ --echo Need innochecksum binary
+ --die Need innochecksum binary
+}
+
--disable_warnings
set global innodb_file_format = `Barracuda`;
set global innodb_file_per_table = on;
--enable_warnings
-create table innodb_normal (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
-create table innodb_page_compressed1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
-create table innodb_page_compressed2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
-create table innodb_page_compressed3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
-create table innodb_page_compressed4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
-create table innodb_page_compressed5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
-create table innodb_page_compressed6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
-create table innodb_page_compressed7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
-create table innodb_page_compressed8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
-create table innodb_page_compressed9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t[0-9]+ page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Page \\[page id: space=[0-9]+, page number= [0-9]+\\] in file ./test/t[0-9]+.ibd may be corrupted. Post compression checksum [0-9]+ stored [0-9]+ compression_method [ZLIB|SNAPPY|LZ4|LZO|LZMA|BZIP2]");
+call mtr.add_suppression("InnoDB: Background Page read failed to read or decrypt \\[page id: space=[0-9]+, page number=[0-9]+\\]");
+call mtr.add_suppression("mysqld: Index for table 't[0-9]+' is corrupt; try to repair it");
+
+create table t0 (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
+create table t1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
+create table t2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
+create table t3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
+create table t4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
+create table t5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
+create table t6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
+create table t7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
+create table t8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
+create table t9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
--disable_query_log
begin;
-let $i = 2000;
+let $i = 100;
while ($i)
{
- insert into innodb_normal(b) values(REPEAT('Aa',50));
- insert into innodb_normal(b) values(REPEAT('a',100));
- insert into innodb_normal(b) values(REPEAT('b',100));
- insert into innodb_normal(b) values(REPEAT('0',100));
- insert into innodb_normal(b) values(REPEAT('1',100));
+ insert into t0(b) values(REPEAT('Aa',50));
+ insert into t0(b) values(REPEAT('a',100));
+ insert into t0(b) values(REPEAT('b',100));
+ insert into t0(b) values(REPEAT('0',100));
+ insert into t0(b) values(REPEAT('1',100));
dec $i;
}
-insert into innodb_page_compressed1 select * from innodb_normal;
-insert into innodb_page_compressed2 select * from innodb_normal;
-insert into innodb_page_compressed3 select * from innodb_normal;
-insert into innodb_page_compressed4 select * from innodb_normal;
-insert into innodb_page_compressed5 select * from innodb_normal;
-insert into innodb_page_compressed6 select * from innodb_normal;
-insert into innodb_page_compressed7 select * from innodb_normal;
-insert into innodb_page_compressed8 select * from innodb_normal;
-insert into innodb_page_compressed9 select * from innodb_normal;
+insert into t1 select * from t0;
+insert into t2 select * from t0;
+insert into t3 select * from t0;
+insert into t4 select * from t0;
+insert into t5 select * from t0;
+insert into t6 select * from t0;
+insert into t7 select * from t0;
+insert into t8 select * from t0;
+insert into t9 select * from t0;
commit;
--enable_query_log
-select count(*) from innodb_page_compressed1;
-select count(*) from innodb_page_compressed3;
-select count(*) from innodb_page_compressed4;
-select count(*) from innodb_page_compressed5;
-select count(*) from innodb_page_compressed6;
-select count(*) from innodb_page_compressed6;
-select count(*) from innodb_page_compressed7;
-select count(*) from innodb_page_compressed8;
-select count(*) from innodb_page_compressed9;
+select count(*) from t1;
+select count(*) from t3;
+select count(*) from t4;
+select count(*) from t5;
+select count(*) from t6;
+select count(*) from t6;
+select count(*) from t7;
+select count(*) from t8;
+select count(*) from t9;
#
# Wait until pages are really compressed
@@ -61,71 +71,205 @@ let $wait_condition= select variable_value > 0 from information_schema.global_st
--source include/shutdown_mysqld.inc
---let t1_IBD = $MYSQLD_DATADIR/test/innodb_normal.ibd
+--let t1_IBD = $MYSQLD_DATADIR/test/t0.ibd
--let SEARCH_RANGE = 10000000
--let SEARCH_PATTERN=AaAaAaAa
---echo # innodb_normal expected FOUND
+--echo # t0 expected FOUND
-- let SEARCH_FILE=$t1_IBD
-- source include/search_pattern_in_file.inc
---let t1_IBD = $MYSQLD_DATADIR/test/innodb_page_compressed1.ibd
---echo # innodb_page_compressed1 page compressed expected NOT FOUND
+--let t1_IBD = $MYSQLD_DATADIR/test/t1.ibd
+--echo # t1 page compressed expected NOT FOUND
-- let SEARCH_FILE=$t1_IBD
-- source include/search_pattern_in_file.inc
---let t1_IBD = $MYSQLD_DATADIR/test/innodb_page_compressed2.ibd
---echo # innodb_page_compressed2 page compressed expected NOT FOUND
+--let t1_IBD = $MYSQLD_DATADIR/test/t2.ibd
+--echo # t2 page compressed expected NOT FOUND
-- let SEARCH_FILE=$t1_IBD
-- source include/search_pattern_in_file.inc
---let t1_IBD = $MYSQLD_DATADIR/test/innodb_page_compressed3.ibd
---echo # innodb_page_compressed3 page compressed expected NOT FOUND
+--let t1_IBD = $MYSQLD_DATADIR/test/t3.ibd
+--echo # t3 page compressed expected NOT FOUND
-- let SEARCH_FILE=$t1_IBD
-- source include/search_pattern_in_file.inc
---let t1_IBD = $MYSQLD_DATADIR/test/innodb_page_compressed4.ibd
---echo # innodb_page_compressed4 page compressed expected NOT FOUND
+--let t1_IBD = $MYSQLD_DATADIR/test/t4.ibd
+--echo # t4 page compressed expected NOT FOUND
-- let SEARCH_FILE=$t1_IBD
-- source include/search_pattern_in_file.inc
---let t1_IBD = $MYSQLD_DATADIR/test/innodb_page_compressed5.ibd
---echo # innodb_page_compressed5 page compressed expected NOT FOUND
+--let t1_IBD = $MYSQLD_DATADIR/test/t5.ibd
+--echo # t5 page compressed expected NOT FOUND
-- let SEARCH_FILE=$t1_IBD
-- source include/search_pattern_in_file.inc
---let t1_IBD = $MYSQLD_DATADIR/test/innodb_page_compressed6.ibd
---echo # innodb_page_compressed6 page compressed expected NOT FOUND
+--let t1_IBD = $MYSQLD_DATADIR/test/t6.ibd
+--echo # t6 page compressed expected NOT FOUND
-- let SEARCH_FILE=$t1_IBD
-- source include/search_pattern_in_file.inc
---let t1_IBD = $MYSQLD_DATADIR/test/innodb_page_compressed7.ibd
---echo # innodb_page_compressed7 page compressed expected NOT FOUND
+--let t1_IBD = $MYSQLD_DATADIR/test/t7.ibd
+--echo # t7 page compressed expected NOT FOUND
-- let SEARCH_FILE=$t1_IBD
-- source include/search_pattern_in_file.inc
---let t1_IBD = $MYSQLD_DATADIR/test/innodb_page_compressed8.ibd
---echo # innodb_page_compressed8 page compressed expected NOT FOUND
+--let t1_IBD = $MYSQLD_DATADIR/test/t8.ibd
+--echo # t8 page compressed expected NOT FOUND
-- let SEARCH_FILE=$t1_IBD
-- source include/search_pattern_in_file.inc
---let t1_IBD = $MYSQLD_DATADIR/test/innodb_page_compressed9.ibd
---echo # innodb_page_compressed9 page compressed expected NOT FOUND
+--let t1_IBD = $MYSQLD_DATADIR/test/t9.ibd
+--echo # t9 page compressed expected NOT FOUND
-- let SEARCH_FILE=$t1_IBD
-- source include/search_pattern_in_file.inc
+#
+# Run innochecksum to all tables, all tables should be ok
+#
+let t_IBD = $MYSQLD_DATADIR/test/t0.ibd;
+--exec $INNOCHECKSUM $t_IBD
+let $i=9;
+while $($i > 0) {
+--echo # Run innochecksum on t$i
+let t_IBD = $MYSQLD_DATADIR/test/t$i.ibd;
+--exec $INNOCHECKSUM $t_IBD
+dec $i;
+}
+
-- source include/start_mysqld.inc
-select count(*) from innodb_page_compressed1;
-select count(*) from innodb_page_compressed3;
-select count(*) from innodb_page_compressed4;
-select count(*) from innodb_page_compressed5;
-select count(*) from innodb_page_compressed6;
-select count(*) from innodb_page_compressed6;
-select count(*) from innodb_page_compressed7;
-select count(*) from innodb_page_compressed8;
-select count(*) from innodb_page_compressed9;
+select count(*) from t0;
+select count(*) from t1;
+select count(*) from t3;
+select count(*) from t4;
+select count(*) from t5;
+select count(*) from t6;
+select count(*) from t7;
+select count(*) from t8;
+select count(*) from t9;
let $wait_condition= select variable_value > 0 from information_schema.global_status where variable_name = 'INNODB_NUM_PAGES_PAGE_DECOMPRESSED';
--source include/wait_condition.inc
-drop table innodb_normal;
-drop table innodb_page_compressed1;
-drop table innodb_page_compressed2;
-drop table innodb_page_compressed3;
-drop table innodb_page_compressed4;
-drop table innodb_page_compressed5;
-drop table innodb_page_compressed6;
-drop table innodb_page_compressed7;
-drop table innodb_page_compressed8;
-drop table innodb_page_compressed9;
+let INNODB_PAGE_SIZE=`select @@innodb_page_size`;
+let MYSQLD_DATADIR=`select @@datadir`;
+
+--echo # Restart server
+--source include/shutdown_mysqld.inc
+
+--echo # Corrupting tablespaces...
+
+#
+# Now we corrupt page compressed pages as follows:
+# (1) compression method
+# (2) payload size
+# (3) data
+# (4) checksum
+#
+# Page 0 is not compressed or encrypted
+#
+perl;
+sub ib_write_value {
+ my($fh, $file, $pos, $len, $val) = @_;
+ seek($fh, $pos, SEEK_SET) or die "$0: seek $file to pos $pos: $!";;
+ syswrite($fh, $val, $len) == $len or die "$0: write to $file val $val len $len: $!";;
+}
+sub ib_corrupt_table {
+ my($file, $pos, $val, $len) = @_;
+ open(my $fh, "+<", $file) or die "$0: open $file: $!";
+ binmode $fh;
+ ib_write_value($fh, $file, $pos, $len, $val);
+ close $fh or die "$0: close $file: $!";
+}
+sub ib_read_value {
+ my($fh, $file, $pos, $len) = @_;
+ seek($fh, $pos, SEEK_SET) or die "$0: seek $file to pos $pos: $!";
+ sysread($fh, $buf, $len) == $len or die "$0: read $file : $!";
+ return $buf;
+}
+
+my($pos) = $ENV{'INNODB_PAGE_SIZE'} * 3;
+my($file) = "$ENV{MYSQLD_DATADIR}/test/t1.ibd";
+open($fh, "+<", $file) or die "$0: open $file : $!";
+binmode $fh;
+
+# Find out the page type, compression method location is based on that
+my($ptype) = unpack("n", ib_read_value($fh, $file, $pos+24, 2));
+if ($ptype == 34354) {
+ $pos += 32; # FLUSH_LSN_OR_KEY_VERSION + 6
+} else {
+ $pos += 40; # FIL_PAGE_DATA + 2
+}
+
+# Corrupt compression method by decreasing it by one, if zero set 6
+# Note that compression method is not encrypted
+my($cmethod) = unpack("n", ib_read_value($fh, $file, $pos, 2));
+$cmethod--;
+if ($cmethod == 0) {
+ $cmethod = 6;
+}
+
+ib_write_value($fh, $file, $pos, 2, pack("n", $cmethod));
+close $fh or die $!;
+
+# (2) corrupt payload size by decreasing size by 50 and if 0 set it to 20
+# note that payload size is not encrypted
+my($pos) = $ENV{'INNODB_PAGE_SIZE'} * 3 + 38;
+my($file) = "$ENV{MYSQLD_DATADIR}/test/t2.ibd";
+open($fh, "+<", $file) or die "$0: open $file : $!";
+my($size) = unpack("n", ib_read_value($fh, $file, $pos, 2));
+$size -= 50;
+if ($size <= 0) {
+ $size = 20;
+}
+ib_write_value($fh, $file, $pos, 2, pack("n", $size));
+close $fh or die $!;
+
+# (3) Corrupt data
+
+ib_corrupt_table("$ENV{MYSQLD_DATADIR}/test/t3.ibd", $ENV{'INNODB_PAGE_SIZE'} * 3 + 42,
+ "deadaaaaffffbbbb",14);
+
+# (4) Corrupt checksum
+my($pos) = $ENV{'INNODB_PAGE_SIZE'} * 3;
+my($file) = "$ENV{MYSQLD_DATADIR}/test/t4.ibd";
+open($fh, "+<", $file) or die "$0: open $file : $!";
+
+# Find out the page type, checksum location is based on that
+my($ptype) = unpack("n", ib_read_value($fh, $file, $pos + 24, 2));
+if ($ptype == 37401) {
+ $pos += 30;
+}
+ib_write_value($fh, $file, $pos, 4, pack("N", 1020102010));
+close $fh or die $!;
+EOF
+
+--echo # Corruption done
+
+#
+# Run innochecksum to page compressed (and maybe encrypted) tables
+# now we should detect corruptions
+#
+let $i=4;
+while $($i > 0) {
+--echo # Run innochecksum on t$i
+let t_IBD = $MYSQLD_DATADIR/test/t$i.ibd;
+--error 1
+--exec $INNOCHECKSUM $t_IBD
+dec $i;
+}
+
+--echo # Start server again
+--source include/start_mysqld.inc
+
+#
+# Server should not crash on corrupted tables
+#
+--error ER_NOT_KEYFILE
+select * from t1;
+--error ER_NOT_KEYFILE
+select * from t2;
+--error ER_NOT_KEYFILE
+select * from t3;
+--error ER_NOT_KEYFILE
+select * from t4;
+select count(*) from t5;
+select count(*) from t6;
+select count(*) from t7;
+select count(*) from t8;
+select count(*) from t9;
+
+# We should be able to drop even corrupted tables
+
+drop table t0, t1, t2, t3, t4, t5, t6, t7, t8, t9;
diff --git a/mysql-test/suite/innodb/r/doublewrite-compressed.result b/mysql-test/suite/innodb/r/doublewrite-compressed.result
new file mode 100644
index 00000000000..b9b0be62b5e
--- /dev/null
+++ b/mysql-test/suite/innodb/r/doublewrite-compressed.result
@@ -0,0 +1,253 @@
+#
+# Bug #17335427 INNODB CAN NOT USE THE DOUBLEWRITE BUFFER PROPERLY
+# Bug #18144349 INNODB CANNOT USE THE DOUBLEWRITE BUFFER FOR THE FIRST
+# PAGE OF SYSTEM TABLESPACE
+#
+SET GLOBAL innodb_fast_shutdown = 0;
+show variables like 'innodb_doublewrite';
+Variable_name Value
+innodb_doublewrite ON
+show variables like 'innodb_fil_make_page_dirty_debug';
+Variable_name Value
+innodb_fil_make_page_dirty_debug 0
+show variables like 'innodb_saved_page_number_debug';
+Variable_name Value
+innodb_saved_page_number_debug 0
+set global innodb_file_per_table=ON;
+set global innodb_file_format='Barracuda';
+create table t1 (f1 int primary key, f2 blob) engine=innodb page_compressed=yes;
+start transaction;
+insert into t1 values(1, repeat('#',12));
+insert into t1 values(2, repeat('+',12));
+insert into t1 values(3, repeat('/',12));
+insert into t1 values(4, repeat('-',12));
+insert into t1 values(5, repeat('.',12));
+commit work;
+# ---------------------------------------------------------------
+# Test Begin: Test if recovery works if first page of user
+# tablespace is full of zeroes.
+select space from information_schema.innodb_sys_tables
+where name = 'test/t1' into @space_id;
+# Ensure that dirty pages of table t1 is flushed.
+flush tables t1 for export;
+unlock tables;
+begin;
+insert into t1 values (6, repeat('%', 12));
+# Make the first page dirty for table t1
+set global innodb_saved_page_number_debug = 0;
+set global innodb_fil_make_page_dirty_debug = @space_id;
+# Ensure that dirty pages of table t1 are flushed.
+set global innodb_buf_flush_list_now = 1;
+# Kill the server
+# Make the first page (page_no=0) of the user tablespace
+# full of zeroes.
+#
+# MDEV-11623: Use old FSP_SPACE_FLAGS in the doublewrite buffer.
+check table t1;
+Table Op Msg_type Msg_text
+test.t1 check status OK
+select f1, f2 from t1;
+f1 f2
+1 ############
+2 ++++++++++++
+3 ////////////
+4 ------------
+5 ............
+# Test End
+# ---------------------------------------------------------------
+# Test Begin: Test if recovery works if first page of user
+# tablespace is corrupted.
+select space from information_schema.innodb_sys_tables
+where name = 'test/t1' into @space_id;
+# Ensure that dirty pages of table t1 is flushed.
+flush tables t1 for export;
+unlock tables;
+begin;
+insert into t1 values (6, repeat('%', 12));
+# Make the first page dirty for table t1
+set global innodb_saved_page_number_debug = 0;
+set global innodb_fil_make_page_dirty_debug = @space_id;
+# Ensure that dirty pages of table t1 are flushed.
+set global innodb_buf_flush_list_now = 1;
+# Kill the server
+# Corrupt the first page (page_no=0) of the user tablespace.
+check table t1;
+Table Op Msg_type Msg_text
+test.t1 check status OK
+select f1, f2 from t1;
+f1 f2
+1 ############
+2 ++++++++++++
+3 ////////////
+4 ------------
+5 ............
+# Test End
+# ---------------------------------------------------------------
+# Test Begin: Test if recovery works if 2nd page of user
+# tablespace is full of zeroes.
+select space from information_schema.innodb_sys_tables
+where name = 'test/t1' into @space_id;
+# Ensure that dirty pages of table t1 is flushed.
+flush tables t1 for export;
+unlock tables;
+begin;
+insert into t1 values (6, repeat('%', 400));
+# Make the 2nd page dirty for table t1
+set global innodb_saved_page_number_debug = 1;
+set global innodb_fil_make_page_dirty_debug = @space_id;
+# Ensure that dirty pages of table t1 are flushed.
+set global innodb_buf_flush_list_now = 1;
+# Kill the server
+# Make the 2nd page (page_no=1) of the tablespace all zeroes.
+check table t1;
+Table Op Msg_type Msg_text
+test.t1 check status OK
+select f1, f2 from t1;
+f1 f2
+1 ############
+2 ++++++++++++
+3 ////////////
+4 ------------
+5 ............
+# Test End
+# ---------------------------------------------------------------
+# Test Begin: Test if recovery works if 2nd page of user
+# tablespace is corrupted.
+select space from information_schema.innodb_sys_tables
+where name = 'test/t1' into @space_id;
+# Ensure that dirty pages of table t1 is flushed.
+flush tables t1 for export;
+unlock tables;
+begin;
+insert into t1 values (6, repeat('%', 400));
+# Make the 2nd page dirty for table t1
+set global innodb_saved_page_number_debug = 1;
+set global innodb_fil_make_page_dirty_debug = @space_id;
+# Ensure that the dirty pages of table t1 are flushed.
+set global innodb_buf_flush_list_now = 1;
+# Kill the server
+# Corrupt the 2nd page (page_no=1) of the user tablespace.
+check table t1;
+Table Op Msg_type Msg_text
+test.t1 check status OK
+select f1, f2 from t1;
+f1 f2
+1 ############
+2 ++++++++++++
+3 ////////////
+4 ------------
+5 ............
+# Test End
+# ---------------------------------------------------------------
+# Test Begin: Test if recovery works if first page of
+# system tablespace is full of zeroes.
+begin;
+insert into t1 values (6, repeat('%', 400));
+# Ensure that all dirty pages in the system are flushed.
+set global innodb_buf_flush_list_now = 1;
+# Make the first page dirty for system tablespace
+set global innodb_saved_page_number_debug = 0;
+set global innodb_fil_make_page_dirty_debug = 0;
+# Ensure that the dirty page of system tablespace is also flushed.
+set global innodb_buf_flush_list_now = 1;
+# Kill the server
+# Make the first page (page_no=0) of the system tablespace
+# all zeroes.
+check table t1;
+Table Op Msg_type Msg_text
+test.t1 check status OK
+select f1, f2 from t1;
+f1 f2
+1 ############
+2 ++++++++++++
+3 ////////////
+4 ------------
+5 ............
+# Test End
+# ---------------------------------------------------------------
+# Test Begin: Test if recovery works if first page of
+# system tablespace is corrupted.
+begin;
+insert into t1 values (6, repeat('%', 400));
+# Ensure that all dirty pages in the system are flushed.
+set global innodb_buf_flush_list_now = 1;
+# Make the first page dirty for system tablespace
+set global innodb_saved_page_number_debug = 0;
+set global innodb_fil_make_page_dirty_debug = 0;
+# Ensure that the dirty page of system tablespace is also flushed.
+set global innodb_buf_flush_list_now = 1;
+# Kill the server
+# Corrupt the first page (page_no=0) of the system tablespace.
+check table t1;
+Table Op Msg_type Msg_text
+test.t1 check status OK
+select f1, f2 from t1;
+f1 f2
+1 ############
+2 ++++++++++++
+3 ////////////
+4 ------------
+5 ............
+# Test End
+# ---------------------------------------------------------------
+# Test Begin: Test if recovery works if 2nd page of
+# system tablespace is full of zeroes.
+begin;
+insert into t1 values (6, repeat('%', 400));
+# Ensure that all dirty pages in the system are flushed.
+set global innodb_buf_flush_list_now = 1;
+# Make the second page dirty for system tablespace
+set global innodb_saved_page_number_debug = 1;
+set global innodb_fil_make_page_dirty_debug = 0;
+# Ensure that the dirty page of system tablespace is also flushed.
+set global innodb_buf_flush_list_now = 1;
+# Kill the server
+# Make the 2nd page (page_no=1) of the system tablespace
+# all zeroes.
+check table t1;
+Table Op Msg_type Msg_text
+test.t1 check status OK
+select f1, f2 from t1;
+f1 f2
+1 ############
+2 ++++++++++++
+3 ////////////
+4 ------------
+5 ............
+# Test End
+# ---------------------------------------------------------------
+# Test Begin: Test if recovery works if 2nd page of
+# system tablespace is corrupted.
+begin;
+insert into t1 values (6, repeat('%', 400));
+# Ensure that all dirty pages in the system are flushed.
+set global innodb_buf_flush_list_now = 1;
+# Make the second page dirty for system tablespace
+set global innodb_saved_page_number_debug = 1;
+set global innodb_fil_make_page_dirty_debug = 0;
+# Ensure that the dirty page of system tablespace is also flushed.
+set global innodb_buf_flush_list_now = 1;
+# Kill the server
+# Make the 2nd page (page_no=1) of the system tablespace
+# all zeroes.
+check table t1;
+Table Op Msg_type Msg_text
+test.t1 check status OK
+FOUND /\[ERROR\] InnoDB: .* in tablespace.*test.t1.*/ in mysqld.1.err
+select f1, f2 from t1;
+f1 f2
+1 ############
+2 ++++++++++++
+3 ////////////
+4 ------------
+5 ............
+drop table t1;
+#
+# MDEV-12600 crash during install_db with innodb_page_size=32K
+# and ibdata1=3M
+#
+SELECT * FROM INFORMATION_SCHEMA.ENGINES
+WHERE engine = 'innodb'
+AND support IN ('YES', 'DEFAULT', 'ENABLED');
+ENGINE SUPPORT COMMENT TRANSACTIONS XA SAVEPOINTS
+FOUND /\[ERROR\] InnoDB: Cannot create doublewrite buffer/ in mysqld.1.err
diff --git a/mysql-test/suite/innodb/r/innodb-page_compression_bzip2.result b/mysql-test/suite/innodb/r/innodb-page_compression_bzip2.result
index 3a3d6f92cac..d0530d5bef3 100644
--- a/mysql-test/suite/innodb/r/innodb-page_compression_bzip2.result
+++ b/mysql-test/suite/innodb/r/innodb-page_compression_bzip2.result
@@ -1,98 +1,135 @@
set global innodb_compression_algorithm = bzip2;
+select @@innodb_compression_algorithm;
+@@innodb_compression_algorithm
+bzip2
set global innodb_file_format = `Barracuda`;
set global innodb_file_per_table = on;
-create table innodb_normal (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
-create table innodb_page_compressed1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
-create table innodb_page_compressed2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
-create table innodb_page_compressed3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
-create table innodb_page_compressed4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
-create table innodb_page_compressed5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
-create table innodb_page_compressed6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
-create table innodb_page_compressed7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
-create table innodb_page_compressed8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
-create table innodb_page_compressed9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
-select count(*) from innodb_page_compressed1;
-count(*)
-10000
-select count(*) from innodb_page_compressed3;
-count(*)
-10000
-select count(*) from innodb_page_compressed4;
-count(*)
-10000
-select count(*) from innodb_page_compressed5;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed7;
-count(*)
-10000
-select count(*) from innodb_page_compressed8;
-count(*)
-10000
-select count(*) from innodb_page_compressed9;
-count(*)
-10000
-# innodb_normal expected FOUND
-FOUND /AaAaAaAa/ in innodb_normal.ibd
-# innodb_page_compressed1 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed1.ibd
-# innodb_page_compressed2 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed2.ibd
-# innodb_page_compressed3 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed3.ibd
-# innodb_page_compressed4 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed4.ibd
-# innodb_page_compressed5 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed5.ibd
-# innodb_page_compressed6 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed6.ibd
-# innodb_page_compressed7 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed7.ibd
-# innodb_page_compressed8 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed8.ibd
-# innodb_page_compressed9 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed9.ibd
-select count(*) from innodb_page_compressed1;
-count(*)
-10000
-select count(*) from innodb_page_compressed3;
-count(*)
-10000
-select count(*) from innodb_page_compressed4;
-count(*)
-10000
-select count(*) from innodb_page_compressed5;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed7;
-count(*)
-10000
-select count(*) from innodb_page_compressed8;
-count(*)
-10000
-select count(*) from innodb_page_compressed9;
-count(*)
-10000
-drop table innodb_normal;
-drop table innodb_page_compressed1;
-drop table innodb_page_compressed2;
-drop table innodb_page_compressed3;
-drop table innodb_page_compressed4;
-drop table innodb_page_compressed5;
-drop table innodb_page_compressed6;
-drop table innodb_page_compressed7;
-drop table innodb_page_compressed8;
-drop table innodb_page_compressed9;
-#done
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t[0-9]+ page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Page \\[page id: space=[0-9]+, page number= [0-9]+\\] in file ./test/t[0-9]+.ibd may be corrupted. Post compression checksum [0-9]+ stored [0-9]+ compression_method [ZLIB|SNAPPY|LZ4|LZO|LZMA|BZIP2]");
+call mtr.add_suppression("InnoDB: Background Page read failed to read or decrypt \\[page id: space=[0-9]+, page number=[0-9]+\\]");
+call mtr.add_suppression("mysqld: Index for table 't[0-9]+' is corrupt; try to repair it");
+create table t0 (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
+create table t1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
+create table t2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
+create table t3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
+create table t4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
+create table t5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
+create table t6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
+create table t7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
+create table t8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
+create table t9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
+select count(*) from t1;
+count(*)
+500
+select count(*) from t3;
+count(*)
+500
+select count(*) from t4;
+count(*)
+500
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+# t0 expected FOUND
+FOUND /AaAaAaAa/ in t0.ibd
+# t1 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t1.ibd
+# t2 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t2.ibd
+# t3 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t3.ibd
+# t4 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t4.ibd
+# t5 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t5.ibd
+# t6 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t6.ibd
+# t7 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t7.ibd
+# t8 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t8.ibd
+# t9 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t9.ibd
+# Run innochecksum on t9
+# Run innochecksum on t8
+# Run innochecksum on t7
+# Run innochecksum on t6
+# Run innochecksum on t5
+# Run innochecksum on t4
+# Run innochecksum on t3
+# Run innochecksum on t2
+# Run innochecksum on t1
+select count(*) from t0;
+count(*)
+500
+select count(*) from t1;
+count(*)
+500
+select count(*) from t3;
+count(*)
+500
+select count(*) from t4;
+count(*)
+500
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+# Restart server
+# Corrupting tablespaces...
+# Corruption done
+# Run innochecksum on t4
+# Run innochecksum on t3
+# Run innochecksum on t2
+# Run innochecksum on t1
+# Start server again
+select * from t1;
+ERROR HY000: Incorrect key file for table 't1'; try to repair it
+select * from t2;
+ERROR HY000: Incorrect key file for table 't2'; try to repair it
+select * from t3;
+ERROR HY000: Incorrect key file for table 't3'; try to repair it
+select * from t4;
+ERROR HY000: Incorrect key file for table 't4'; try to repair it
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+drop table t0, t1, t2, t3, t4, t5, t6, t7, t8, t9;
diff --git a/mysql-test/suite/innodb/r/innodb-page_compression_default.result b/mysql-test/suite/innodb/r/innodb-page_compression_default.result
index 18f7fb5e04a..9cf3e969ba8 100644
--- a/mysql-test/suite/innodb/r/innodb-page_compression_default.result
+++ b/mysql-test/suite/innodb/r/innodb-page_compression_default.result
@@ -1,98 +1,133 @@
call mtr.add_suppression("InnoDB: Compression failed for space [0-9]+ name test/innodb_page_compressed[0-9] len [0-9]+ err 2 write_size [0-9]+.");
set global innodb_file_format = `Barracuda`;
set global innodb_file_per_table = on;
-create table innodb_normal (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
-create table innodb_page_compressed1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
-create table innodb_page_compressed2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
-create table innodb_page_compressed3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
-create table innodb_page_compressed4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
-create table innodb_page_compressed5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
-create table innodb_page_compressed6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
-create table innodb_page_compressed7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
-create table innodb_page_compressed8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
-create table innodb_page_compressed9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
-select count(*) from innodb_page_compressed1;
-count(*)
-10000
-select count(*) from innodb_page_compressed3;
-count(*)
-10000
-select count(*) from innodb_page_compressed4;
-count(*)
-10000
-select count(*) from innodb_page_compressed5;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed7;
-count(*)
-10000
-select count(*) from innodb_page_compressed8;
-count(*)
-10000
-select count(*) from innodb_page_compressed9;
-count(*)
-10000
-# innodb_normal expected FOUND
-FOUND /AaAaAaAa/ in innodb_normal.ibd
-# innodb_page_compressed1 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed1.ibd
-# innodb_page_compressed2 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed2.ibd
-# innodb_page_compressed3 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed3.ibd
-# innodb_page_compressed4 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed4.ibd
-# innodb_page_compressed5 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed5.ibd
-# innodb_page_compressed6 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed6.ibd
-# innodb_page_compressed7 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed7.ibd
-# innodb_page_compressed8 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed8.ibd
-# innodb_page_compressed9 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed9.ibd
-select count(*) from innodb_page_compressed1;
-count(*)
-10000
-select count(*) from innodb_page_compressed3;
-count(*)
-10000
-select count(*) from innodb_page_compressed4;
-count(*)
-10000
-select count(*) from innodb_page_compressed5;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed7;
-count(*)
-10000
-select count(*) from innodb_page_compressed8;
-count(*)
-10000
-select count(*) from innodb_page_compressed9;
-count(*)
-10000
-drop table innodb_normal;
-drop table innodb_page_compressed1;
-drop table innodb_page_compressed2;
-drop table innodb_page_compressed3;
-drop table innodb_page_compressed4;
-drop table innodb_page_compressed5;
-drop table innodb_page_compressed6;
-drop table innodb_page_compressed7;
-drop table innodb_page_compressed8;
-drop table innodb_page_compressed9;
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t[0-9]+ page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Page \\[page id: space=[0-9]+, page number= [0-9]+\\] in file ./test/t[0-9]+.ibd may be corrupted. Post compression checksum [0-9]+ stored [0-9]+ compression_method [ZLIB|SNAPPY|LZ4|LZO|LZMA|BZIP2]");
+call mtr.add_suppression("InnoDB: Background Page read failed to read or decrypt \\[page id: space=[0-9]+, page number=[0-9]+\\]");
+call mtr.add_suppression("mysqld: Index for table 't[0-9]+' is corrupt; try to repair it");
+create table t0 (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
+create table t1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
+create table t2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
+create table t3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
+create table t4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
+create table t5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
+create table t6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
+create table t7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
+create table t8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
+create table t9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
+select count(*) from t1;
+count(*)
+500
+select count(*) from t3;
+count(*)
+500
+select count(*) from t4;
+count(*)
+500
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+# t0 expected FOUND
+FOUND /AaAaAaAa/ in t0.ibd
+# t1 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t1.ibd
+# t2 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t2.ibd
+# t3 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t3.ibd
+# t4 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t4.ibd
+# t5 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t5.ibd
+# t6 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t6.ibd
+# t7 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t7.ibd
+# t8 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t8.ibd
+# t9 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t9.ibd
+# Run innochecksum on t9
+# Run innochecksum on t8
+# Run innochecksum on t7
+# Run innochecksum on t6
+# Run innochecksum on t5
+# Run innochecksum on t4
+# Run innochecksum on t3
+# Run innochecksum on t2
+# Run innochecksum on t1
+select count(*) from t0;
+count(*)
+500
+select count(*) from t1;
+count(*)
+500
+select count(*) from t3;
+count(*)
+500
+select count(*) from t4;
+count(*)
+500
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+# Restart server
+# Corrupting tablespaces...
+# Corruption done
+# Run innochecksum on t4
+# Run innochecksum on t3
+# Run innochecksum on t2
+# Run innochecksum on t1
+# Start server again
+select * from t1;
+ERROR HY000: Incorrect key file for table 't1'; try to repair it
+select * from t2;
+ERROR HY000: Incorrect key file for table 't2'; try to repair it
+select * from t3;
+ERROR HY000: Incorrect key file for table 't3'; try to repair it
+select * from t4;
+ERROR HY000: Incorrect key file for table 't4'; try to repair it
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+drop table t0, t1, t2, t3, t4, t5, t6, t7, t8, t9;
#done
diff --git a/mysql-test/suite/innodb/r/innodb-page_compression_lz4.result b/mysql-test/suite/innodb/r/innodb-page_compression_lz4.result
index d60eb62ec15..2722e36c34a 100644
--- a/mysql-test/suite/innodb/r/innodb-page_compression_lz4.result
+++ b/mysql-test/suite/innodb/r/innodb-page_compression_lz4.result
@@ -1,98 +1,135 @@
set global innodb_compression_algorithm = lz4;
+select @@innodb_compression_algorithm;
+@@innodb_compression_algorithm
+lz4
set global innodb_file_format = `Barracuda`;
set global innodb_file_per_table = on;
-create table innodb_normal (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
-create table innodb_page_compressed1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
-create table innodb_page_compressed2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
-create table innodb_page_compressed3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
-create table innodb_page_compressed4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
-create table innodb_page_compressed5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
-create table innodb_page_compressed6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
-create table innodb_page_compressed7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
-create table innodb_page_compressed8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
-create table innodb_page_compressed9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
-select count(*) from innodb_page_compressed1;
-count(*)
-10000
-select count(*) from innodb_page_compressed3;
-count(*)
-10000
-select count(*) from innodb_page_compressed4;
-count(*)
-10000
-select count(*) from innodb_page_compressed5;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed7;
-count(*)
-10000
-select count(*) from innodb_page_compressed8;
-count(*)
-10000
-select count(*) from innodb_page_compressed9;
-count(*)
-10000
-# innodb_normal expected FOUND
-FOUND /AaAaAaAa/ in innodb_normal.ibd
-# innodb_page_compressed1 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed1.ibd
-# innodb_page_compressed2 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed2.ibd
-# innodb_page_compressed3 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed3.ibd
-# innodb_page_compressed4 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed4.ibd
-# innodb_page_compressed5 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed5.ibd
-# innodb_page_compressed6 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed6.ibd
-# innodb_page_compressed7 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed7.ibd
-# innodb_page_compressed8 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed8.ibd
-# innodb_page_compressed9 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed9.ibd
-select count(*) from innodb_page_compressed1;
-count(*)
-10000
-select count(*) from innodb_page_compressed3;
-count(*)
-10000
-select count(*) from innodb_page_compressed4;
-count(*)
-10000
-select count(*) from innodb_page_compressed5;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed7;
-count(*)
-10000
-select count(*) from innodb_page_compressed8;
-count(*)
-10000
-select count(*) from innodb_page_compressed9;
-count(*)
-10000
-drop table innodb_normal;
-drop table innodb_page_compressed1;
-drop table innodb_page_compressed2;
-drop table innodb_page_compressed3;
-drop table innodb_page_compressed4;
-drop table innodb_page_compressed5;
-drop table innodb_page_compressed6;
-drop table innodb_page_compressed7;
-drop table innodb_page_compressed8;
-drop table innodb_page_compressed9;
-#done
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t[0-9]+ page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Page \\[page id: space=[0-9]+, page number= [0-9]+\\] in file ./test/t[0-9]+.ibd may be corrupted. Post compression checksum [0-9]+ stored [0-9]+ compression_method [ZLIB|SNAPPY|LZ4|LZO|LZMA|BZIP2]");
+call mtr.add_suppression("InnoDB: Background Page read failed to read or decrypt \\[page id: space=[0-9]+, page number=[0-9]+\\]");
+call mtr.add_suppression("mysqld: Index for table 't[0-9]+' is corrupt; try to repair it");
+create table t0 (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
+create table t1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
+create table t2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
+create table t3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
+create table t4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
+create table t5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
+create table t6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
+create table t7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
+create table t8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
+create table t9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
+select count(*) from t1;
+count(*)
+500
+select count(*) from t3;
+count(*)
+500
+select count(*) from t4;
+count(*)
+500
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+# t0 expected FOUND
+FOUND /AaAaAaAa/ in t0.ibd
+# t1 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t1.ibd
+# t2 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t2.ibd
+# t3 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t3.ibd
+# t4 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t4.ibd
+# t5 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t5.ibd
+# t6 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t6.ibd
+# t7 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t7.ibd
+# t8 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t8.ibd
+# t9 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t9.ibd
+# Run innochecksum on t9
+# Run innochecksum on t8
+# Run innochecksum on t7
+# Run innochecksum on t6
+# Run innochecksum on t5
+# Run innochecksum on t4
+# Run innochecksum on t3
+# Run innochecksum on t2
+# Run innochecksum on t1
+select count(*) from t0;
+count(*)
+500
+select count(*) from t1;
+count(*)
+500
+select count(*) from t3;
+count(*)
+500
+select count(*) from t4;
+count(*)
+500
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+# Restart server
+# Corrupting tablespaces...
+# Corruption done
+# Run innochecksum on t4
+# Run innochecksum on t3
+# Run innochecksum on t2
+# Run innochecksum on t1
+# Start server again
+select * from t1;
+ERROR HY000: Incorrect key file for table 't1'; try to repair it
+select * from t2;
+ERROR HY000: Incorrect key file for table 't2'; try to repair it
+select * from t3;
+ERROR HY000: Incorrect key file for table 't3'; try to repair it
+select * from t4;
+ERROR HY000: Incorrect key file for table 't4'; try to repair it
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+drop table t0, t1, t2, t3, t4, t5, t6, t7, t8, t9;
diff --git a/mysql-test/suite/innodb/r/innodb-page_compression_lzma.result b/mysql-test/suite/innodb/r/innodb-page_compression_lzma.result
index b3cc9c9b627..aa1dbec22ae 100644
--- a/mysql-test/suite/innodb/r/innodb-page_compression_lzma.result
+++ b/mysql-test/suite/innodb/r/innodb-page_compression_lzma.result
@@ -1,98 +1,135 @@
set global innodb_compression_algorithm = lzma;
+select @@innodb_compression_algorithm;
+@@innodb_compression_algorithm
+lzma
set global innodb_file_format = `Barracuda`;
set global innodb_file_per_table = on;
-create table innodb_normal (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
-create table innodb_page_compressed1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
-create table innodb_page_compressed2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
-create table innodb_page_compressed3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
-create table innodb_page_compressed4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
-create table innodb_page_compressed5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
-create table innodb_page_compressed6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
-create table innodb_page_compressed7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
-create table innodb_page_compressed8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
-create table innodb_page_compressed9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
-select count(*) from innodb_page_compressed1;
-count(*)
-10000
-select count(*) from innodb_page_compressed3;
-count(*)
-10000
-select count(*) from innodb_page_compressed4;
-count(*)
-10000
-select count(*) from innodb_page_compressed5;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed7;
-count(*)
-10000
-select count(*) from innodb_page_compressed8;
-count(*)
-10000
-select count(*) from innodb_page_compressed9;
-count(*)
-10000
-# innodb_normal expected FOUND
-FOUND /AaAaAaAa/ in innodb_normal.ibd
-# innodb_page_compressed1 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed1.ibd
-# innodb_page_compressed2 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed2.ibd
-# innodb_page_compressed3 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed3.ibd
-# innodb_page_compressed4 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed4.ibd
-# innodb_page_compressed5 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed5.ibd
-# innodb_page_compressed6 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed6.ibd
-# innodb_page_compressed7 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed7.ibd
-# innodb_page_compressed8 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed8.ibd
-# innodb_page_compressed9 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed9.ibd
-select count(*) from innodb_page_compressed1;
-count(*)
-10000
-select count(*) from innodb_page_compressed3;
-count(*)
-10000
-select count(*) from innodb_page_compressed4;
-count(*)
-10000
-select count(*) from innodb_page_compressed5;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed7;
-count(*)
-10000
-select count(*) from innodb_page_compressed8;
-count(*)
-10000
-select count(*) from innodb_page_compressed9;
-count(*)
-10000
-drop table innodb_normal;
-drop table innodb_page_compressed1;
-drop table innodb_page_compressed2;
-drop table innodb_page_compressed3;
-drop table innodb_page_compressed4;
-drop table innodb_page_compressed5;
-drop table innodb_page_compressed6;
-drop table innodb_page_compressed7;
-drop table innodb_page_compressed8;
-drop table innodb_page_compressed9;
-#done
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t[0-9]+ page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Page \\[page id: space=[0-9]+, page number= [0-9]+\\] in file ./test/t[0-9]+.ibd may be corrupted. Post compression checksum [0-9]+ stored [0-9]+ compression_method [ZLIB|SNAPPY|LZ4|LZO|LZMA|BZIP2]");
+call mtr.add_suppression("InnoDB: Background Page read failed to read or decrypt \\[page id: space=[0-9]+, page number=[0-9]+\\]");
+call mtr.add_suppression("mysqld: Index for table 't[0-9]+' is corrupt; try to repair it");
+create table t0 (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
+create table t1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
+create table t2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
+create table t3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
+create table t4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
+create table t5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
+create table t6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
+create table t7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
+create table t8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
+create table t9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
+select count(*) from t1;
+count(*)
+500
+select count(*) from t3;
+count(*)
+500
+select count(*) from t4;
+count(*)
+500
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+# t0 expected FOUND
+FOUND /AaAaAaAa/ in t0.ibd
+# t1 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t1.ibd
+# t2 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t2.ibd
+# t3 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t3.ibd
+# t4 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t4.ibd
+# t5 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t5.ibd
+# t6 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t6.ibd
+# t7 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t7.ibd
+# t8 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t8.ibd
+# t9 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t9.ibd
+# Run innochecksum on t9
+# Run innochecksum on t8
+# Run innochecksum on t7
+# Run innochecksum on t6
+# Run innochecksum on t5
+# Run innochecksum on t4
+# Run innochecksum on t3
+# Run innochecksum on t2
+# Run innochecksum on t1
+select count(*) from t0;
+count(*)
+500
+select count(*) from t1;
+count(*)
+500
+select count(*) from t3;
+count(*)
+500
+select count(*) from t4;
+count(*)
+500
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+# Restart server
+# Corrupting tablespaces...
+# Corruption done
+# Run innochecksum on t4
+# Run innochecksum on t3
+# Run innochecksum on t2
+# Run innochecksum on t1
+# Start server again
+select * from t1;
+ERROR HY000: Incorrect key file for table 't1'; try to repair it
+select * from t2;
+ERROR HY000: Incorrect key file for table 't2'; try to repair it
+select * from t3;
+ERROR HY000: Incorrect key file for table 't3'; try to repair it
+select * from t4;
+ERROR HY000: Incorrect key file for table 't4'; try to repair it
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+drop table t0, t1, t2, t3, t4, t5, t6, t7, t8, t9;
diff --git a/mysql-test/suite/innodb/r/innodb-page_compression_lzo.result b/mysql-test/suite/innodb/r/innodb-page_compression_lzo.result
index b6c5bc32bb7..204ff52d3f5 100644
--- a/mysql-test/suite/innodb/r/innodb-page_compression_lzo.result
+++ b/mysql-test/suite/innodb/r/innodb-page_compression_lzo.result
@@ -1,98 +1,135 @@
set global innodb_compression_algorithm = lzo;
+select @@innodb_compression_algorithm;
+@@innodb_compression_algorithm
+lzo
set global innodb_file_format = `Barracuda`;
set global innodb_file_per_table = on;
-create table innodb_normal (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
-create table innodb_page_compressed1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
-create table innodb_page_compressed2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
-create table innodb_page_compressed3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
-create table innodb_page_compressed4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
-create table innodb_page_compressed5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
-create table innodb_page_compressed6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
-create table innodb_page_compressed7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
-create table innodb_page_compressed8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
-create table innodb_page_compressed9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
-select count(*) from innodb_page_compressed1;
-count(*)
-10000
-select count(*) from innodb_page_compressed3;
-count(*)
-10000
-select count(*) from innodb_page_compressed4;
-count(*)
-10000
-select count(*) from innodb_page_compressed5;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed7;
-count(*)
-10000
-select count(*) from innodb_page_compressed8;
-count(*)
-10000
-select count(*) from innodb_page_compressed9;
-count(*)
-10000
-# innodb_normal expected FOUND
-FOUND /AaAaAaAa/ in innodb_normal.ibd
-# innodb_page_compressed1 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed1.ibd
-# innodb_page_compressed2 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed2.ibd
-# innodb_page_compressed3 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed3.ibd
-# innodb_page_compressed4 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed4.ibd
-# innodb_page_compressed5 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed5.ibd
-# innodb_page_compressed6 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed6.ibd
-# innodb_page_compressed7 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed7.ibd
-# innodb_page_compressed8 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed8.ibd
-# innodb_page_compressed9 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed9.ibd
-select count(*) from innodb_page_compressed1;
-count(*)
-10000
-select count(*) from innodb_page_compressed3;
-count(*)
-10000
-select count(*) from innodb_page_compressed4;
-count(*)
-10000
-select count(*) from innodb_page_compressed5;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed7;
-count(*)
-10000
-select count(*) from innodb_page_compressed8;
-count(*)
-10000
-select count(*) from innodb_page_compressed9;
-count(*)
-10000
-drop table innodb_normal;
-drop table innodb_page_compressed1;
-drop table innodb_page_compressed2;
-drop table innodb_page_compressed3;
-drop table innodb_page_compressed4;
-drop table innodb_page_compressed5;
-drop table innodb_page_compressed6;
-drop table innodb_page_compressed7;
-drop table innodb_page_compressed8;
-drop table innodb_page_compressed9;
-#done
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t[0-9]+ page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Page \\[page id: space=[0-9]+, page number= [0-9]+\\] in file ./test/t[0-9]+.ibd may be corrupted. Post compression checksum [0-9]+ stored [0-9]+ compression_method [ZLIB|SNAPPY|LZ4|LZO|LZMA|BZIP2]");
+call mtr.add_suppression("InnoDB: Background Page read failed to read or decrypt \\[page id: space=[0-9]+, page number=[0-9]+\\]");
+call mtr.add_suppression("mysqld: Index for table 't[0-9]+' is corrupt; try to repair it");
+create table t0 (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
+create table t1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
+create table t2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
+create table t3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
+create table t4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
+create table t5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
+create table t6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
+create table t7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
+create table t8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
+create table t9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
+select count(*) from t1;
+count(*)
+500
+select count(*) from t3;
+count(*)
+500
+select count(*) from t4;
+count(*)
+500
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+# t0 expected FOUND
+FOUND /AaAaAaAa/ in t0.ibd
+# t1 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t1.ibd
+# t2 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t2.ibd
+# t3 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t3.ibd
+# t4 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t4.ibd
+# t5 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t5.ibd
+# t6 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t6.ibd
+# t7 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t7.ibd
+# t8 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t8.ibd
+# t9 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t9.ibd
+# Run innochecksum on t9
+# Run innochecksum on t8
+# Run innochecksum on t7
+# Run innochecksum on t6
+# Run innochecksum on t5
+# Run innochecksum on t4
+# Run innochecksum on t3
+# Run innochecksum on t2
+# Run innochecksum on t1
+select count(*) from t0;
+count(*)
+500
+select count(*) from t1;
+count(*)
+500
+select count(*) from t3;
+count(*)
+500
+select count(*) from t4;
+count(*)
+500
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+# Restart server
+# Corrupting tablespaces...
+# Corruption done
+# Run innochecksum on t4
+# Run innochecksum on t3
+# Run innochecksum on t2
+# Run innochecksum on t1
+# Start server again
+select * from t1;
+ERROR HY000: Incorrect key file for table 't1'; try to repair it
+select * from t2;
+ERROR HY000: Incorrect key file for table 't2'; try to repair it
+select * from t3;
+ERROR HY000: Incorrect key file for table 't3'; try to repair it
+select * from t4;
+ERROR HY000: Incorrect key file for table 't4'; try to repair it
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+drop table t0, t1, t2, t3, t4, t5, t6, t7, t8, t9;
diff --git a/mysql-test/suite/innodb/r/innodb-page_compression_snappy.result b/mysql-test/suite/innodb/r/innodb-page_compression_snappy.result
index 74cfa9bce3a..693e5ed8f3b 100644
--- a/mysql-test/suite/innodb/r/innodb-page_compression_snappy.result
+++ b/mysql-test/suite/innodb/r/innodb-page_compression_snappy.result
@@ -1,99 +1,136 @@
call mtr.add_suppression("InnoDB: Compression failed for space [0-9]+ name test/innodb_page_compressed[0-9] len [0-9]+ err 2 write_size [0-9]+.");
set global innodb_compression_algorithm = snappy;
+select @@innodb_compression_algorithm;
+@@innodb_compression_algorithm
+snappy
set global innodb_file_format = `Barracuda`;
set global innodb_file_per_table = on;
-create table innodb_normal (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
-create table innodb_page_compressed1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
-create table innodb_page_compressed2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
-create table innodb_page_compressed3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
-create table innodb_page_compressed4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
-create table innodb_page_compressed5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
-create table innodb_page_compressed6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
-create table innodb_page_compressed7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
-create table innodb_page_compressed8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
-create table innodb_page_compressed9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
-select count(*) from innodb_page_compressed1;
-count(*)
-10000
-select count(*) from innodb_page_compressed3;
-count(*)
-10000
-select count(*) from innodb_page_compressed4;
-count(*)
-10000
-select count(*) from innodb_page_compressed5;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed7;
-count(*)
-10000
-select count(*) from innodb_page_compressed8;
-count(*)
-10000
-select count(*) from innodb_page_compressed9;
-count(*)
-10000
-# innodb_normal expected FOUND
-FOUND /AaAaAaAa/ in innodb_normal.ibd
-# innodb_page_compressed1 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed1.ibd
-# innodb_page_compressed2 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed2.ibd
-# innodb_page_compressed3 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed3.ibd
-# innodb_page_compressed4 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed4.ibd
-# innodb_page_compressed5 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed5.ibd
-# innodb_page_compressed6 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed6.ibd
-# innodb_page_compressed7 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed7.ibd
-# innodb_page_compressed8 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed8.ibd
-# innodb_page_compressed9 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed9.ibd
-select count(*) from innodb_page_compressed1;
-count(*)
-10000
-select count(*) from innodb_page_compressed3;
-count(*)
-10000
-select count(*) from innodb_page_compressed4;
-count(*)
-10000
-select count(*) from innodb_page_compressed5;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed7;
-count(*)
-10000
-select count(*) from innodb_page_compressed8;
-count(*)
-10000
-select count(*) from innodb_page_compressed9;
-count(*)
-10000
-drop table innodb_normal;
-drop table innodb_page_compressed1;
-drop table innodb_page_compressed2;
-drop table innodb_page_compressed3;
-drop table innodb_page_compressed4;
-drop table innodb_page_compressed5;
-drop table innodb_page_compressed6;
-drop table innodb_page_compressed7;
-drop table innodb_page_compressed8;
-drop table innodb_page_compressed9;
-#done
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t[0-9]+ page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Page \\[page id: space=[0-9]+, page number= [0-9]+\\] in file ./test/t[0-9]+.ibd may be corrupted. Post compression checksum [0-9]+ stored [0-9]+ compression_method [ZLIB|SNAPPY|LZ4|LZO|LZMA|BZIP2]");
+call mtr.add_suppression("InnoDB: Background Page read failed to read or decrypt \\[page id: space=[0-9]+, page number=[0-9]+\\]");
+call mtr.add_suppression("mysqld: Index for table 't[0-9]+' is corrupt; try to repair it");
+create table t0 (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
+create table t1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
+create table t2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
+create table t3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
+create table t4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
+create table t5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
+create table t6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
+create table t7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
+create table t8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
+create table t9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
+select count(*) from t1;
+count(*)
+500
+select count(*) from t3;
+count(*)
+500
+select count(*) from t4;
+count(*)
+500
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+# t0 expected FOUND
+FOUND /AaAaAaAa/ in t0.ibd
+# t1 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t1.ibd
+# t2 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t2.ibd
+# t3 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t3.ibd
+# t4 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t4.ibd
+# t5 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t5.ibd
+# t6 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t6.ibd
+# t7 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t7.ibd
+# t8 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t8.ibd
+# t9 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t9.ibd
+# Run innochecksum on t9
+# Run innochecksum on t8
+# Run innochecksum on t7
+# Run innochecksum on t6
+# Run innochecksum on t5
+# Run innochecksum on t4
+# Run innochecksum on t3
+# Run innochecksum on t2
+# Run innochecksum on t1
+select count(*) from t0;
+count(*)
+500
+select count(*) from t1;
+count(*)
+500
+select count(*) from t3;
+count(*)
+500
+select count(*) from t4;
+count(*)
+500
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+# Restart server
+# Corrupting tablespaces...
+# Corruption done
+# Run innochecksum on t4
+# Run innochecksum on t3
+# Run innochecksum on t2
+# Run innochecksum on t1
+# Start server again
+select * from t1;
+ERROR HY000: Incorrect key file for table 't1'; try to repair it
+select * from t2;
+ERROR HY000: Incorrect key file for table 't2'; try to repair it
+select * from t3;
+ERROR HY000: Incorrect key file for table 't3'; try to repair it
+select * from t4;
+ERROR HY000: Incorrect key file for table 't4'; try to repair it
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+drop table t0, t1, t2, t3, t4, t5, t6, t7, t8, t9;
diff --git a/mysql-test/suite/innodb/r/innodb-page_compression_zip.result b/mysql-test/suite/innodb/r/innodb-page_compression_zip.result
index 3644890f921..d98b4e8e229 100644
--- a/mysql-test/suite/innodb/r/innodb-page_compression_zip.result
+++ b/mysql-test/suite/innodb/r/innodb-page_compression_zip.result
@@ -1,98 +1,135 @@
set global innodb_compression_algorithm = zlib;
+select @@innodb_compression_algorithm;
+@@innodb_compression_algorithm
+zlib
set global innodb_file_format = `Barracuda`;
set global innodb_file_per_table = on;
-create table innodb_normal (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
-create table innodb_page_compressed1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
-create table innodb_page_compressed2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
-create table innodb_page_compressed3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
-create table innodb_page_compressed4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
-create table innodb_page_compressed5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
-create table innodb_page_compressed6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
-create table innodb_page_compressed7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
-create table innodb_page_compressed8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
-create table innodb_page_compressed9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
-select count(*) from innodb_page_compressed1;
-count(*)
-10000
-select count(*) from innodb_page_compressed3;
-count(*)
-10000
-select count(*) from innodb_page_compressed4;
-count(*)
-10000
-select count(*) from innodb_page_compressed5;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed7;
-count(*)
-10000
-select count(*) from innodb_page_compressed8;
-count(*)
-10000
-select count(*) from innodb_page_compressed9;
-count(*)
-10000
-# innodb_normal expected FOUND
-FOUND /AaAaAaAa/ in innodb_normal.ibd
-# innodb_page_compressed1 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed1.ibd
-# innodb_page_compressed2 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed2.ibd
-# innodb_page_compressed3 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed3.ibd
-# innodb_page_compressed4 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed4.ibd
-# innodb_page_compressed5 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed5.ibd
-# innodb_page_compressed6 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed6.ibd
-# innodb_page_compressed7 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed7.ibd
-# innodb_page_compressed8 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed8.ibd
-# innodb_page_compressed9 page compressed expected NOT FOUND
-NOT FOUND /AaAaAaAa/ in innodb_page_compressed9.ibd
-select count(*) from innodb_page_compressed1;
-count(*)
-10000
-select count(*) from innodb_page_compressed3;
-count(*)
-10000
-select count(*) from innodb_page_compressed4;
-count(*)
-10000
-select count(*) from innodb_page_compressed5;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed6;
-count(*)
-10000
-select count(*) from innodb_page_compressed7;
-count(*)
-10000
-select count(*) from innodb_page_compressed8;
-count(*)
-10000
-select count(*) from innodb_page_compressed9;
-count(*)
-10000
-drop table innodb_normal;
-drop table innodb_page_compressed1;
-drop table innodb_page_compressed2;
-drop table innodb_page_compressed3;
-drop table innodb_page_compressed4;
-drop table innodb_page_compressed5;
-drop table innodb_page_compressed6;
-drop table innodb_page_compressed7;
-drop table innodb_page_compressed8;
-drop table innodb_page_compressed9;
-#done
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t[0-9]+ page \\[page id: space=[0-9]+, page number=[0-9]+\\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Page \\[page id: space=[0-9]+, page number= [0-9]+\\] in file ./test/t[0-9]+.ibd may be corrupted. Post compression checksum [0-9]+ stored [0-9]+ compression_method [ZLIB|SNAPPY|LZ4|LZO|LZMA|BZIP2]");
+call mtr.add_suppression("InnoDB: Background Page read failed to read or decrypt \\[page id: space=[0-9]+, page number=[0-9]+\\]");
+call mtr.add_suppression("mysqld: Index for table 't[0-9]+' is corrupt; try to repair it");
+create table t0 (c1 int not null auto_increment primary key, b char(200)) engine=innodb;
+create table t1 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=1;
+create table t2 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=2;
+create table t3 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=3;
+create table t4 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=4;
+create table t5 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=5;
+create table t6 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=6;
+create table t7 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=7;
+create table t8 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=8;
+create table t9 (c1 int not null auto_increment primary key, b char(200)) engine=innodb page_compressed=1 page_compression_level=9;
+select count(*) from t1;
+count(*)
+500
+select count(*) from t3;
+count(*)
+500
+select count(*) from t4;
+count(*)
+500
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+# t0 expected FOUND
+FOUND /AaAaAaAa/ in t0.ibd
+# t1 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t1.ibd
+# t2 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t2.ibd
+# t3 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t3.ibd
+# t4 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t4.ibd
+# t5 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t5.ibd
+# t6 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t6.ibd
+# t7 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t7.ibd
+# t8 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t8.ibd
+# t9 page compressed expected NOT FOUND
+NOT FOUND /AaAaAaAa/ in t9.ibd
+# Run innochecksum on t9
+# Run innochecksum on t8
+# Run innochecksum on t7
+# Run innochecksum on t6
+# Run innochecksum on t5
+# Run innochecksum on t4
+# Run innochecksum on t3
+# Run innochecksum on t2
+# Run innochecksum on t1
+select count(*) from t0;
+count(*)
+500
+select count(*) from t1;
+count(*)
+500
+select count(*) from t3;
+count(*)
+500
+select count(*) from t4;
+count(*)
+500
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+# Restart server
+# Corrupting tablespaces...
+# Corruption done
+# Run innochecksum on t4
+# Run innochecksum on t3
+# Run innochecksum on t2
+# Run innochecksum on t1
+# Start server again
+select * from t1;
+ERROR HY000: Incorrect key file for table 't1'; try to repair it
+select * from t2;
+ERROR HY000: Incorrect key file for table 't2'; try to repair it
+select * from t3;
+ERROR HY000: Incorrect key file for table 't3'; try to repair it
+select * from t4;
+ERROR HY000: Incorrect key file for table 't4'; try to repair it
+select count(*) from t5;
+count(*)
+500
+select count(*) from t6;
+count(*)
+500
+select count(*) from t7;
+count(*)
+500
+select count(*) from t8;
+count(*)
+500
+select count(*) from t9;
+count(*)
+500
+drop table t0, t1, t2, t3, t4, t5, t6, t7, t8, t9;
diff --git a/mysql-test/suite/innodb/r/innodb_bug14147491.result b/mysql-test/suite/innodb/r/innodb_bug14147491.result
index 989e433338e..c329333a718 100644
--- a/mysql-test/suite/innodb/r/innodb_bug14147491.result
+++ b/mysql-test/suite/innodb/r/innodb_bug14147491.result
@@ -1,6 +1,6 @@
call mtr.add_suppression("InnoDB: Table \"test\".\"t1\" is corrupted. Please drop the table and recreate.");
call mtr.add_suppression("InnoDB: Cannot open table test/t1 from the internal data dictionary of InnoDB though the .frm file for the table exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.");
-call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t1 page \[page id: space=[0-9]+, page number=[0-9]+\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t1 page \[page id: space=[0-9]+, page number=[0-9]+\]. You may have to recover from a backup.");
# Create and populate the table to be corrupted
set global innodb_file_per_table=ON;
CREATE TABLE t1 (a INT AUTO_INCREMENT PRIMARY KEY, b TEXT) ENGINE=InnoDB;
diff --git a/mysql-test/suite/innodb/t/doublewrite-compressed.test b/mysql-test/suite/innodb/t/doublewrite-compressed.test
new file mode 100644
index 00000000000..90f62576a7f
--- /dev/null
+++ b/mysql-test/suite/innodb/t/doublewrite-compressed.test
@@ -0,0 +1,439 @@
+--echo #
+--echo # Bug #17335427 INNODB CAN NOT USE THE DOUBLEWRITE BUFFER PROPERLY
+--echo # Bug #18144349 INNODB CANNOT USE THE DOUBLEWRITE BUFFER FOR THE FIRST
+--echo # PAGE OF SYSTEM TABLESPACE
+--echo #
+
+--source include/innodb_page_size.inc
+--source include/have_debug.inc
+--source include/not_embedded.inc
+
+# Slow shutdown and restart to make sure ibuf merge is finished
+SET GLOBAL innodb_fast_shutdown = 0;
+--disable_query_log
+call mtr.add_suppression("space header page consists of zero bytes.*test.t1");
+call mtr.add_suppression("checksum mismatch in tablespace.*test.t1");
+call mtr.add_suppression("Current page size .* != page size on page");
+call mtr.add_suppression("innodb-page-size mismatch in tablespace.*test.t1");
+call mtr.add_suppression("InnoDB: New log files created");
+call mtr.add_suppression("InnoDB: Cannot create doublewrite buffer: the first file in innodb_data_file_path must be at least (3|6|12)M\\.");
+call mtr.add_suppression("InnoDB: Database creation was aborted");
+call mtr.add_suppression("Plugin 'InnoDB' (init function returned error|registration as a STORAGE ENGINE failed)");
+--enable_query_log
+--source include/restart_mysqld.inc
+
+let INNODB_PAGE_SIZE=`select @@innodb_page_size`;
+let MYSQLD_DATADIR=`select @@datadir`;
+let SEARCH_FILE= $MYSQLTEST_VARDIR/log/mysqld.1.err;
+let SEARCH_RANGE= -50000;
+
+show variables like 'innodb_doublewrite';
+show variables like 'innodb_fil_make_page_dirty_debug';
+show variables like 'innodb_saved_page_number_debug';
+
+--disable_warnings
+set global innodb_file_per_table=ON;
+set global innodb_file_format='Barracuda';
+--enable_warnings
+create table t1 (f1 int primary key, f2 blob) engine=innodb page_compressed=yes;
+
+start transaction;
+insert into t1 values(1, repeat('#',12));
+insert into t1 values(2, repeat('+',12));
+insert into t1 values(3, repeat('/',12));
+insert into t1 values(4, repeat('-',12));
+insert into t1 values(5, repeat('.',12));
+commit work;
+
+--echo # ---------------------------------------------------------------
+--echo # Test Begin: Test if recovery works if first page of user
+--echo # tablespace is full of zeroes.
+
+select space from information_schema.innodb_sys_tables
+where name = 'test/t1' into @space_id;
+
+--echo # Ensure that dirty pages of table t1 is flushed.
+flush tables t1 for export;
+unlock tables;
+
+begin;
+insert into t1 values (6, repeat('%', 12));
+
+--source ../include/no_checkpoint_start.inc
+
+--echo # Make the first page dirty for table t1
+set global innodb_saved_page_number_debug = 0;
+set global innodb_fil_make_page_dirty_debug = @space_id;
+
+--echo # Ensure that dirty pages of table t1 are flushed.
+set global innodb_buf_flush_list_now = 1;
+
+--let CLEANUP_IF_CHECKPOINT=drop table t1;
+--source ../include/no_checkpoint_end.inc
+
+--echo # Make the first page (page_no=0) of the user tablespace
+--echo # full of zeroes.
+--echo #
+--echo # MDEV-11623: Use old FSP_SPACE_FLAGS in the doublewrite buffer.
+
+perl;
+use IO::Handle;
+my $fname= "$ENV{'MYSQLD_DATADIR'}test/t1.ibd";
+my $page_size = $ENV{INNODB_PAGE_SIZE};
+my $page;
+open(FILE, "+<", $fname) or die;
+sysread(FILE, $page, $page_size)==$page_size||die "Unable to read $name\n";
+sysseek(FILE, 0, 0)||die "Unable to seek $fname\n";
+die unless syswrite(FILE, chr(0) x $page_size, $page_size) == $page_size;
+close FILE;
+
+open(FILE, "+<", "$ENV{MYSQLD_DATADIR}ibdata1")||die "cannot open ibdata1\n";
+sysseek(FILE, 6 * $page_size - 190, 0)||die "Unable to seek ibdata1\n";
+sysread(FILE, $_, 12) == 12||die "Unable to read TRX_SYS\n";
+my($magic,$d1,$d2)=unpack "NNN", $_;
+die "magic=$magic, $d1, $d2\n" unless $magic == 536853855 && $d2 >= $d1 + 64;
+sysseek(FILE, $d1 * $page_size, 0)||die "Unable to seek ibdata1\n";
+# Find the page in the doublewrite buffer
+for (my $d = $d1; $d < $d2 + 64; $d++)
+{
+ sysread(FILE, $_, $page_size)==$page_size||die "Cannot read doublewrite\n";
+ next unless $_ eq $page;
+ sysseek(FILE, $d * $page_size, 0)||die "Unable to seek ibdata1\n";
+ # Write buggy MariaDB 10.1.x FSP_SPACE_FLAGS to the doublewrite buffer
+ my($flags) = unpack "x[54]N", $_;
+ my $badflags = ($flags & 0x3f);
+ my $compression_level=6;
+ $badflags |= 1<<6|$compression_level<<7 if ($flags & 1 << 16);
+ $badflags |= ($flags & 15 << 6) << 7; # PAGE_SSIZE
+
+ substr ($_, 54, 4) = pack("N", $badflags);
+ # Replace the innodb_checksum_algorithm=none checksum
+ substr ($_, 0, 4) = pack("N", 0xdeadbeef);
+ substr ($_, $page_size - 8, 4) = pack("N", 0xdeadbeef);
+ syswrite(FILE, $_, $page_size)==$page_size||die;
+ close(FILE);
+ exit 0;
+}
+die "Did not find the page in the doublewrite buffer ($d1,$d2)\n";
+EOF
+
+--source include/start_mysqld.inc
+
+check table t1;
+select f1, f2 from t1;
+
+--echo # Test End
+--echo # ---------------------------------------------------------------
+--echo # Test Begin: Test if recovery works if first page of user
+--echo # tablespace is corrupted.
+
+select space from information_schema.innodb_sys_tables
+where name = 'test/t1' into @space_id;
+
+--echo # Ensure that dirty pages of table t1 is flushed.
+flush tables t1 for export;
+unlock tables;
+
+begin;
+insert into t1 values (6, repeat('%', 12));
+
+--source ../include/no_checkpoint_start.inc
+
+--echo # Make the first page dirty for table t1
+set global innodb_saved_page_number_debug = 0;
+set global innodb_fil_make_page_dirty_debug = @space_id;
+
+--echo # Ensure that dirty pages of table t1 are flushed.
+set global innodb_buf_flush_list_now = 1;
+
+--source include/no_checkpoint_end.inc
+
+--echo # Corrupt the first page (page_no=0) of the user tablespace.
+perl;
+use IO::Handle;
+my $fname= "$ENV{'MYSQLD_DATADIR'}test/t1.ibd";
+open(FILE, "+<", $fname) or die;
+FILE->autoflush(1);
+binmode FILE;
+print FILE chr(0) x ($ENV{'INNODB_PAGE_SIZE'}/2);
+close FILE;
+EOF
+
+--source include/start_mysqld.inc
+
+check table t1;
+select f1, f2 from t1;
+
+--echo # Test End
+--echo # ---------------------------------------------------------------
+--echo # Test Begin: Test if recovery works if 2nd page of user
+--echo # tablespace is full of zeroes.
+
+select space from information_schema.innodb_sys_tables
+where name = 'test/t1' into @space_id;
+
+--echo # Ensure that dirty pages of table t1 is flushed.
+flush tables t1 for export;
+unlock tables;
+
+begin;
+insert into t1 values (6, repeat('%', 400));
+
+--source ../include/no_checkpoint_start.inc
+
+--echo # Make the 2nd page dirty for table t1
+set global innodb_saved_page_number_debug = 1;
+set global innodb_fil_make_page_dirty_debug = @space_id;
+
+--echo # Ensure that dirty pages of table t1 are flushed.
+set global innodb_buf_flush_list_now = 1;
+
+--source include/no_checkpoint_end.inc
+
+--echo # Make the 2nd page (page_no=1) of the tablespace all zeroes.
+perl;
+use IO::Handle;
+my $fname= "$ENV{'MYSQLD_DATADIR'}test/t1.ibd";
+open(FILE, "+<", $fname) or die;
+FILE->autoflush(1);
+binmode FILE;
+seek(FILE, $ENV{'INNODB_PAGE_SIZE'}, SEEK_SET);
+print FILE chr(0) x ($ENV{'INNODB_PAGE_SIZE'});
+close FILE;
+EOF
+
+--source include/start_mysqld.inc
+
+check table t1;
+select f1, f2 from t1;
+
+--echo # Test End
+--echo # ---------------------------------------------------------------
+--echo # Test Begin: Test if recovery works if 2nd page of user
+--echo # tablespace is corrupted.
+
+select space from information_schema.innodb_sys_tables
+where name = 'test/t1' into @space_id;
+
+--echo # Ensure that dirty pages of table t1 is flushed.
+flush tables t1 for export;
+unlock tables;
+
+begin;
+insert into t1 values (6, repeat('%', 400));
+
+--source ../include/no_checkpoint_start.inc
+
+--echo # Make the 2nd page dirty for table t1
+set global innodb_saved_page_number_debug = 1;
+set global innodb_fil_make_page_dirty_debug = @space_id;
+
+--echo # Ensure that the dirty pages of table t1 are flushed.
+set global innodb_buf_flush_list_now = 1;
+
+--source include/no_checkpoint_end.inc
+
+--echo # Corrupt the 2nd page (page_no=1) of the user tablespace.
+perl;
+use IO::Handle;
+my $fname= "$ENV{'MYSQLD_DATADIR'}test/t1.ibd";
+open(FILE, "+<", $fname) or die;
+FILE->autoflush(1);
+binmode FILE;
+seek(FILE, $ENV{'INNODB_PAGE_SIZE'}, SEEK_SET);
+print FILE chr(0) x ($ENV{'INNODB_PAGE_SIZE'}/2);
+close FILE;
+EOF
+
+--source include/start_mysqld.inc
+
+check table t1;
+select f1, f2 from t1;
+
+--echo # Test End
+--echo # ---------------------------------------------------------------
+--echo # Test Begin: Test if recovery works if first page of
+--echo # system tablespace is full of zeroes.
+
+begin;
+insert into t1 values (6, repeat('%', 400));
+
+--echo # Ensure that all dirty pages in the system are flushed.
+set global innodb_buf_flush_list_now = 1;
+
+--echo # Make the first page dirty for system tablespace
+set global innodb_saved_page_number_debug = 0;
+set global innodb_fil_make_page_dirty_debug = 0;
+
+--echo # Ensure that the dirty page of system tablespace is also flushed.
+# We do this after the transaction starts and all dirty pages have been flushed
+# already. So flushing of this specified dirty page will surely keep the
+# copy in doublewrite buffer, and no more writes to doublewrite buffer would
+# overwrite the copy. Thus, we can safely modify the original page when server
+# is down. So do the following testings.
+set global innodb_buf_flush_list_now = 1;
+
+--source include/kill_mysqld.inc
+
+--echo # Make the first page (page_no=0) of the system tablespace
+--echo # all zeroes.
+perl;
+use IO::Handle;
+my $fname= "$ENV{'MYSQLD_DATADIR'}ibdata1";
+open(FILE, "+<", $fname) or die;
+FILE->autoflush(1);
+binmode FILE;
+print FILE chr(0) x ($ENV{'INNODB_PAGE_SIZE'});
+close FILE;
+EOF
+
+--source include/start_mysqld.inc
+
+check table t1;
+select f1, f2 from t1;
+
+--echo # Test End
+--echo # ---------------------------------------------------------------
+--echo # Test Begin: Test if recovery works if first page of
+--echo # system tablespace is corrupted.
+
+begin;
+insert into t1 values (6, repeat('%', 400));
+
+--echo # Ensure that all dirty pages in the system are flushed.
+set global innodb_buf_flush_list_now = 1;
+
+--echo # Make the first page dirty for system tablespace
+set global innodb_saved_page_number_debug = 0;
+set global innodb_fil_make_page_dirty_debug = 0;
+
+--echo # Ensure that the dirty page of system tablespace is also flushed.
+set global innodb_buf_flush_list_now = 1;
+
+--source include/kill_mysqld.inc
+
+--echo # Corrupt the first page (page_no=0) of the system tablespace.
+perl;
+use IO::Handle;
+my $fname= "$ENV{'MYSQLD_DATADIR'}ibdata1";
+open(FILE, "+<", $fname) or die;
+FILE->autoflush(1);
+binmode FILE;
+print FILE chr(0) x ($ENV{'INNODB_PAGE_SIZE'}/2);
+close FILE;
+EOF
+
+--source include/start_mysqld.inc
+
+check table t1;
+select f1, f2 from t1;
+
+--echo # Test End
+--echo # ---------------------------------------------------------------
+--echo # Test Begin: Test if recovery works if 2nd page of
+--echo # system tablespace is full of zeroes.
+
+begin;
+insert into t1 values (6, repeat('%', 400));
+
+--echo # Ensure that all dirty pages in the system are flushed.
+set global innodb_buf_flush_list_now = 1;
+
+--echo # Make the second page dirty for system tablespace
+set global innodb_saved_page_number_debug = 1;
+set global innodb_fil_make_page_dirty_debug = 0;
+
+--echo # Ensure that the dirty page of system tablespace is also flushed.
+set global innodb_buf_flush_list_now = 1;
+
+--source include/kill_mysqld.inc
+
+--echo # Make the 2nd page (page_no=1) of the system tablespace
+--echo # all zeroes.
+perl;
+use IO::Handle;
+my $fname= "$ENV{'MYSQLD_DATADIR'}ibdata1";
+open(FILE, "+<", $fname) or die;
+FILE->autoflush(1);
+binmode FILE;
+seek(FILE, $ENV{'INNODB_PAGE_SIZE'}, SEEK_SET);
+print FILE chr(0) x ($ENV{'INNODB_PAGE_SIZE'});
+close FILE;
+EOF
+
+--source include/start_mysqld.inc
+
+check table t1;
+select f1, f2 from t1;
+
+--echo # Test End
+--echo # ---------------------------------------------------------------
+--echo # Test Begin: Test if recovery works if 2nd page of
+--echo # system tablespace is corrupted.
+
+begin;
+insert into t1 values (6, repeat('%', 400));
+
+--echo # Ensure that all dirty pages in the system are flushed.
+set global innodb_buf_flush_list_now = 1;
+
+--echo # Make the second page dirty for system tablespace
+set global innodb_saved_page_number_debug = 1;
+set global innodb_fil_make_page_dirty_debug = 0;
+
+--echo # Ensure that the dirty page of system tablespace is also flushed.
+set global innodb_buf_flush_list_now = 1;
+
+--source include/kill_mysqld.inc
+
+--echo # Make the 2nd page (page_no=1) of the system tablespace
+--echo # all zeroes.
+perl;
+use IO::Handle;
+my $fname= "$ENV{'MYSQLD_DATADIR'}ibdata1";
+open(FILE, "+<", $fname) or die;
+FILE->autoflush(1);
+binmode FILE;
+seek(FILE, $ENV{'INNODB_PAGE_SIZE'}, SEEK_SET);
+print FILE chr(0) x ($ENV{'INNODB_PAGE_SIZE'}/2);
+close FILE;
+EOF
+
+--source include/start_mysqld.inc
+
+check table t1;
+--let SEARCH_PATTERN= \[ERROR\] InnoDB: .* in tablespace.*test.t1.*
+--source include/search_pattern_in_file.inc
+
+select f1, f2 from t1;
+
+drop table t1;
+
+--echo #
+--echo # MDEV-12600 crash during install_db with innodb_page_size=32K
+--echo # and ibdata1=3M
+--echo #
+let bugdir= $MYSQLTEST_VARDIR/tmp/doublewrite;
+--mkdir $bugdir
+
+let $check_no_innodb=SELECT * FROM INFORMATION_SCHEMA.ENGINES
+WHERE engine = 'innodb'
+AND support IN ('YES', 'DEFAULT', 'ENABLED');
+
+--let $ibp=--innodb-log-group-home-dir=$bugdir --innodb-data-home-dir=$bugdir
+--let $ibd=$ibp --innodb-undo-tablespaces=0 --innodb-log-files-in-group=2
+--let $ibp=$ibp --innodb-data-file-path=ibdata1:1M;ibdata2:1M:autoextend
+
+--let $restart_parameters= $ibp
+--source include/restart_mysqld.inc
+eval $check_no_innodb;
+--let SEARCH_PATTERN= \[ERROR\] InnoDB: Cannot create doublewrite buffer
+--source include/search_pattern_in_file.inc
+--let $restart_parameters=
+--source include/restart_mysqld.inc
+
+--remove_file $bugdir/ibdata1
+--remove_file $bugdir/ibdata2
+--remove_file $bugdir/ib_logfile0
+--remove_file $bugdir/ib_logfile1
+--rmdir $bugdir
diff --git a/mysql-test/suite/innodb/t/innodb-page_compression_bzip2.test b/mysql-test/suite/innodb/t/innodb-page_compression_bzip2.test
index 8c4cee14453..aae5c51c123 100644
--- a/mysql-test/suite/innodb/t/innodb-page_compression_bzip2.test
+++ b/mysql-test/suite/innodb/t/innodb-page_compression_bzip2.test
@@ -2,10 +2,13 @@
-- source include/have_innodb_bzip2.inc
-- source include/not_embedded.inc
-# bzip2
set global innodb_compression_algorithm = bzip2;
+select @@innodb_compression_algorithm;
-# All page compression test use the same
+# All page compression tests use the same
--source include/innodb-page-compression.inc
--- echo #done
+# reset system
+--disable_query_log
+set global innodb_compression_algorithm=default;
+--enable_query_log
diff --git a/mysql-test/suite/innodb/t/innodb-page_compression_lz4.test b/mysql-test/suite/innodb/t/innodb-page_compression_lz4.test
index 9d6e2babbe7..bd0097532dd 100644
--- a/mysql-test/suite/innodb/t/innodb-page_compression_lz4.test
+++ b/mysql-test/suite/innodb/t/innodb-page_compression_lz4.test
@@ -5,8 +5,12 @@
# lz4
set global innodb_compression_algorithm = lz4;
+select @@innodb_compression_algorithm;
-# All page compression test use the same
+# All page compression tests use the same
--source include/innodb-page-compression.inc
--- echo #done
+# reset system
+--disable_query_log
+set global innodb_compression_algorithm=default;
+--enable_query_log
diff --git a/mysql-test/suite/innodb/t/innodb-page_compression_lzma.test b/mysql-test/suite/innodb/t/innodb-page_compression_lzma.test
index 0b97c1578c6..1ab2cce74f4 100644
--- a/mysql-test/suite/innodb/t/innodb-page_compression_lzma.test
+++ b/mysql-test/suite/innodb/t/innodb-page_compression_lzma.test
@@ -2,10 +2,14 @@
-- source include/have_innodb_lzma.inc
-- source include/not_embedded.inc
-# lzma
set global innodb_compression_algorithm = lzma;
+select @@innodb_compression_algorithm;
-# All page compression test use the same
+# All page compression tests use the same
--source include/innodb-page-compression.inc
--- echo #done
+# reset system
+--disable_query_log
+set global innodb_compression_algorithm=default;
+--enable_query_log
+
diff --git a/mysql-test/suite/innodb/t/innodb-page_compression_lzo.test b/mysql-test/suite/innodb/t/innodb-page_compression_lzo.test
index aec07beeca6..14438bb7ec3 100644
--- a/mysql-test/suite/innodb/t/innodb-page_compression_lzo.test
+++ b/mysql-test/suite/innodb/t/innodb-page_compression_lzo.test
@@ -2,10 +2,13 @@
-- source include/have_innodb_lzo.inc
-- source include/not_embedded.inc
-# lzo
set global innodb_compression_algorithm = lzo;
+select @@innodb_compression_algorithm;
-# All page compression test use the same
+# All page compression tests use the same
--source include/innodb-page-compression.inc
--- echo #done
+# reset system
+--disable_query_log
+set global innodb_compression_algorithm=default;
+--enable_query_log
diff --git a/mysql-test/suite/innodb/t/innodb-page_compression_snappy.test b/mysql-test/suite/innodb/t/innodb-page_compression_snappy.test
index 532ec294d28..c9e00c7f257 100644
--- a/mysql-test/suite/innodb/t/innodb-page_compression_snappy.test
+++ b/mysql-test/suite/innodb/t/innodb-page_compression_snappy.test
@@ -4,10 +4,13 @@
call mtr.add_suppression("InnoDB: Compression failed for space [0-9]+ name test/innodb_page_compressed[0-9] len [0-9]+ err 2 write_size [0-9]+.");
-# snappy
set global innodb_compression_algorithm = snappy;
+select @@innodb_compression_algorithm;
-# All page compression test use the same
+# All page compression tests use the same
--source include/innodb-page-compression.inc
--- echo #done
+# reset system
+--disable_query_log
+set global innodb_compression_algorithm=default;
+--enable_query_log
diff --git a/mysql-test/suite/innodb/t/innodb-page_compression_zip.test b/mysql-test/suite/innodb/t/innodb-page_compression_zip.test
index ed3a27ada7b..bfcda0ea747 100644
--- a/mysql-test/suite/innodb/t/innodb-page_compression_zip.test
+++ b/mysql-test/suite/innodb/t/innodb-page_compression_zip.test
@@ -1,10 +1,13 @@
--source include/have_innodb.inc
--source include/not_embedded.inc
-# lz4
set global innodb_compression_algorithm = zlib;
+select @@innodb_compression_algorithm;
-# All page compression test use the same
+# All page compression tests use the same
--source include/innodb-page-compression.inc
--- echo #done
+# reset system
+--disable_query_log
+set global innodb_compression_algorithm=default;
+--enable_query_log
diff --git a/mysql-test/suite/innodb/t/innodb_bug14147491.test b/mysql-test/suite/innodb/t/innodb_bug14147491.test
index c21515e7d05..22bc17f0bf6 100644
--- a/mysql-test/suite/innodb/t/innodb_bug14147491.test
+++ b/mysql-test/suite/innodb/t/innodb_bug14147491.test
@@ -6,7 +6,7 @@
call mtr.add_suppression("InnoDB: Table \"test\".\"t1\" is corrupted. Please drop the table and recreate.");
call mtr.add_suppression("InnoDB: Cannot open table test/t1 from the internal data dictionary of InnoDB though the .frm file for the table exists. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting.html for how you can resolve the problem.");
-call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t1 page \[page id: space=[0-9]+, page number=[0-9]+\]. You may have to recover from a backup.");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed file read of tablespace test/t1 page \[page id: space=[0-9]+, page number=[0-9]+\]. You may have to recover from a backup.");
# Don't test under embedded
source include/not_embedded.inc;
@@ -67,7 +67,7 @@ EOF
--echo # Now t1 is corrupted but we should not crash
---error 1712,1932
+--error ER_GET_ERRNO,ER_INDEX_CORRUPT,ER_NO_SUCH_TABLE_IN_ENGINE,ER_NOT_KEYFILE
SELECT * FROM t1;
--error 1034,1712,1932
diff --git a/storage/innobase/buf/buf0buf.cc b/storage/innobase/buf/buf0buf.cc
index e3c2337659e..07fee423973 100644
--- a/storage/innobase/buf/buf0buf.cc
+++ b/storage/innobase/buf/buf0buf.cc
@@ -2,7 +2,7 @@
Copyright (c) 1995, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2008, Google Inc.
-Copyright (c) 2013, 2017, MariaDB Corporation.
+Copyright (c) 2013, 2018, MariaDB Corporation.
Portions of this file contain modifications contributed and copyrighted by
Google, Inc. Those modifications are gratefully acknowledged and are described
@@ -740,25 +740,34 @@ buf_page_is_corrupted(
#ifndef UNIV_INNOCHECKSUM
ulint space_id = mach_read_from_4(read_buf + FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID);
#endif
- ulint page_type = mach_read_from_2(read_buf + FIL_PAGE_TYPE);
-
/* We can trust page type if page compression is set on tablespace
flags because page compression flag means file must have been
created with 10.1 (later than 5.5 code base). In 10.1 page
compressed tables do not contain post compression checksum and
- FIL_PAGE_END_LSN_OLD_CHKSUM field stored. Note that space can
- be null if we are in fil_check_first_page() and first page
- is not compressed or encrypted. Page checksum is verified
- after decompression (i.e. normally pages are already
- decompressed at this stage). */
+ FIL_PAGE_END_LSN_OLD_CHKSUM field stored.
+
+ Note that space can be null if we are in fil_check_first_page() but
+ first page (page 0) is not compressed or encrypted.
+
+ Note that space can be null if we are in Datafile::find_space_id()
+ and that point pages are not yet decompressed/decrypted.
+
+ Note that for innochecksum.cc we want to know if page is corrupted
+ using traditional checksum methods even when page type is
+ compressed or compressed and encrypted.
+
+ Page checksum is verified after decompression (i.e. normally pages
+ are already decompressed at this stage). */
+#ifndef UNIV_INNOCHECKSUM
+ ulint page_type = mach_read_from_2(read_buf + FIL_PAGE_TYPE);
if ((page_type == FIL_PAGE_PAGE_COMPRESSED ||
page_type == FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED)
-#ifndef UNIV_INNOCHECKSUM
- && space && FSP_FLAGS_HAS_PAGE_COMPRESSION(space->flags)
-#endif
- ) {
+ && (!space
+ || (space && FSP_FLAGS_HAS_PAGE_COMPRESSION(space->flags))))
+ {
return (false);
}
+#endif /* !UNIV_INNOCHECKSUM */
if (!zip_size
&& memcmp(read_buf + FIL_PAGE_LSN + 4,
@@ -1052,12 +1061,14 @@ buf_page_print(const byte* read_buf, ulint zip_size)
size = UNIV_PAGE_SIZE;
}
+#ifdef UNIV_BUG_DEBUG
ut_print_timestamp(stderr);
fprintf(stderr,
" InnoDB: Page dump in ascii and hex (" ULINTPF " bytes):\n",
size);
ut_print_buf(stderr, read_buf, size);
fputs("\nInnoDB: End of page dump\n", stderr);
+#endif
if (zip_size) {
/* Print compressed page. */
@@ -3214,8 +3225,8 @@ loop:
/* Try to set table as corrupted instead of
asserting. */
- if (space > TRX_SYS_SPACE &&
- dict_set_corrupted_by_space(space)) {
+ if (space > TRX_SYS_SPACE) {
+ dict_set_corrupted_by_space(space);
return (NULL);
}
@@ -4710,21 +4721,19 @@ buf_page_check_corrupt(buf_page_t* bpage, fil_space_t* space)
still be corrupted if used key does not match. */
still_encrypted = (crypt_data &&
crypt_data->type != CRYPT_SCHEME_UNENCRYPTED &&
- !bpage->encrypted &&
fil_space_verify_crypt_checksum(dst_frame, zip_size,
space, bpage->offset));
- if (!still_encrypted) {
- /* If traditional checksums match, we assume that page is
- not anymore encrypted. */
- corrupted = buf_page_is_corrupted(true, dst_frame, zip_size,
- space);
+ /* If traditional checksums match, we assume that page is
+ not anymore encrypted. */
+ corrupted = buf_page_is_corrupted(true, dst_frame, zip_size,
+ space);
- if (!corrupted) {
- bpage->encrypted = false;
- } else {
- err = DB_PAGE_CORRUPTED;
- }
+ if (!corrupted) {
+ bpage->encrypted = false;
+ still_encrypted = false;
+ } else if (!still_encrypted) {
+ err = DB_PAGE_CORRUPTED;
}
/* Pages that we think are unencrypted but do not match the checksum
@@ -4741,7 +4750,7 @@ buf_page_check_corrupt(buf_page_t* bpage, fil_space_t* space)
", page number=%u]"
" in file %s cannot be decrypted.",
bpage->space, bpage->offset,
- space->name);
+ space->chain.start->name);
ib_logf(IB_LOG_LEVEL_INFO,
"However key management plugin or used key_version " ULINTPF
@@ -4802,7 +4811,11 @@ buf_page_io_complete(buf_page_t* bpage, bool evict)
return(DB_TABLESPACE_DELETED);
}
- buf_page_decrypt_after_read(bpage, space);
+ dberr_t err = DB_SUCCESS;
+
+ if (!buf_page_decrypt_after_read(bpage, space)) {
+ err = DB_PAGE_CORRUPTED;
+ }
if (buf_page_get_zip_size(bpage)) {
frame = bpage->zip.data;
@@ -4810,6 +4823,10 @@ buf_page_io_complete(buf_page_t* bpage, bool evict)
frame = ((buf_block_t*) bpage)->frame;
}
+ if (err != DB_SUCCESS) {
+ goto database_corrupted;
+ }
+
if (buf_page_get_zip_size(bpage)) {
frame = bpage->zip.data;
os_atomic_increment_ulint(&buf_pool->n_pend_unzip, 1);
@@ -4896,7 +4913,7 @@ database_corrupted:
ib_logf(IB_LOG_LEVEL_ERROR,
"Database page corruption on disk"
" or a failed file read of tablespace %s"
- " page [page id: space=%u"
+ " page [page id: space=%u"
", page number=%u]"
". You may have to recover from "
"a backup.",
@@ -6442,10 +6459,14 @@ buf_page_decrypt_after_read(buf_page_t* bpage, fil_space_t* space)
#endif
/* decompress using comp_buf to dst_frame */
- fil_decompress_page(slot->comp_buf,
- dst_frame,
- ulong(size),
- &bpage->write_size);
+ if (!fil_verify_compression_checksum(dst_frame,
+ bpage->space, bpage->offset)
+ || !fil_decompress_page(slot->comp_buf,
+ dst_frame,
+ ulong(size),
+ &bpage->write_size)) {
+ success = false;
+ }
/* Mark this slot as free */
slot->reserved = false;
@@ -6500,10 +6521,14 @@ buf_page_decrypt_after_read(buf_page_t* bpage, fil_space_t* space)
fil_page_type_validate(dst_frame);
#endif
/* decompress using comp_buf to dst_frame */
- fil_decompress_page(slot->comp_buf,
+ if (!fil_verify_compression_checksum(dst_frame,
+ bpage->space, bpage->offset)
+ || !fil_decompress_page(slot->comp_buf,
dst_frame,
ulong(size),
- &bpage->write_size);
+ &bpage->write_size)) {
+ success = false;
+ }
ut_d(fil_page_type_validate(dst_frame));
}
diff --git a/storage/innobase/buf/buf0checksum.cc b/storage/innobase/buf/buf0checksum.cc
index 9e5f1dfe475..f3e5d9b5359 100644
--- a/storage/innobase/buf/buf0checksum.cc
+++ b/storage/innobase/buf/buf0checksum.cc
@@ -1,6 +1,7 @@
/*****************************************************************************
Copyright (c) 1995, 2015, Oracle and/or its affiliates. All Rights Reserved.
+Copyright (c) 2017, 2018, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@@ -28,6 +29,8 @@ Created Aug 11, 2011 Vasil Dimov
#include "ut0crc32.h" /* ut_crc32() */
#include "ut0rnd.h" /* ut_fold_binary() */
#include "buf0checksum.h"
+#include "mtr0types.h"
+#include "mach0data.h"
#ifndef UNIV_INNOCHECKSUM
@@ -156,3 +159,45 @@ buf_checksum_algorithm_name(
return(NULL);
}
+/** Calculates the CRC32 checksum of a page compressed page. The value is
+stored to the page when it is written to a file and also checked for
+a match when reading from the file. Checksum is calculated from
+actual payload of the compressed page and some header fields.
+
+@param[in] page buffer page (UNIV_PAGE_SIZE bytes)
+@return checksum */
+UNIV_INTERN
+uint32_t
+buf_calc_compressed_crc32(
+ const byte* page)
+{
+ /* In page compressed pages compression method is stored to field
+ FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION, thus add it to checksum.
+ In pages first compressed and then encrypted same field
+ contains key version after compression. */
+
+ ulint page_type = mach_read_from_2(page + FIL_PAGE_TYPE);
+
+ ulint header_len = page_type == FIL_PAGE_PAGE_COMPRESSED ?
+ FIL_PAGE_SPACE_ID - FIL_PAGE_OFFSET :
+ FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION - FIL_PAGE_OFFSET;
+
+ const uint32_t c1 = ut_crc32(
+ page + FIL_PAGE_OFFSET,
+ header_len);
+
+ /* Calculate checksum from actual payload including stored size
+ field. In encrypted case add also compression method field. */
+ ulint payload_len = mach_read_from_2(page+FIL_PAGE_DATA)+FIL_PAGE_COMPRESSED_SIZE;
+
+ if (page_type == FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED) {
+ payload_len += FIL_PAGE_COMPRESSION_METHOD_SIZE;
+ }
+
+ const uint32_t c2 = ut_crc32(
+ page + FIL_PAGE_DATA,
+ payload_len);
+
+ return(c1 ^ c2);
+}
+
diff --git a/storage/innobase/buf/buf0dblwr.cc b/storage/innobase/buf/buf0dblwr.cc
index 70422671190..99c7546485a 100644
--- a/storage/innobase/buf/buf0dblwr.cc
+++ b/storage/innobase/buf/buf0dblwr.cc
@@ -1,7 +1,7 @@
/*****************************************************************************
Copyright (c) 1995, 2017, Oracle and/or its affiliates. All Rights Reserved.
-Copyright (c) 2013, 2017, MariaDB Corporation.
+Copyright (c) 2013, 2018, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@@ -488,8 +488,45 @@ leave_func:
ut_free(unaligned_read_buf);
}
-/****************************************************************//**
-Process the double write buffer pages. */
+/** Verify post compression checksum and if it matches
+decompress page.
+@param[in,out] read_buf Page compressed buffer. Result of
+ decompression is copied here.
+@param[in] space_id Tablespace identifier
+@param[in] page_no Page offset
+@param[in] page_size Actual page size
+@return true if both stored post compression checksum matches calculated one
+or it is BUF_NO_CHECKSUM_MAGIC and page decompression is succeeds.*/
+static
+bool
+buf_dblwr_decompress(
+ byte * read_buf,
+ ulint space_id,
+ ulint page_no,
+ ulint page_size)
+{
+ bool success = true;
+
+ /* Page is always fist compressed and then encrypted. Here we can
+ only decompress pages if they are not encrypted.
+
+ For compressed pages verify post compression checksum and if it
+ matches decompress the page.
+
+ Note that old datafiles have checksum BUF_NO_CHECKSUM_MAGIC and
+ that value is accepted as correct post compression checksum see
+ fil_verify_compression_checksum().
+
+ In that case we hope that compression algorithm can recover from
+ corrupted input and return a error that is then handled.*/
+ if(!fil_verify_compression_checksum(read_buf, space_id, page_no)
+ || !fil_decompress_page(NULL, read_buf, page_size, NULL)) {
+ return success = false;
+ }
+
+ return success;
+}
+
void
buf_dblwr_process()
/*===============*/
@@ -556,25 +593,24 @@ buf_dblwr_process()
const bool is_all_zero = buf_page_is_zeroes(
read_buf, zip_size);
+ bool success = true;
+
if (is_all_zero) {
/* We will check if the copy in the
doublewrite buffer is valid. If not, we will
ignore this page (there should be redo log
records to initialize it). */
} else {
- if (fil_page_is_compressed_encrypted(read_buf) ||
- fil_page_is_compressed(read_buf)) {
- /* Decompress the page before
- validating the checksum. */
- fil_decompress_page(
- NULL, read_buf, srv_page_size,
- NULL, true);
+ if (fil_page_is_compressed(read_buf)) {
+ success = buf_dblwr_decompress(read_buf,
+ space_id, page_no, srv_page_size);
}
- if (fil_space_verify_crypt_checksum(
+ if (success
+ && (fil_space_verify_crypt_checksum(
read_buf, zip_size, NULL, page_no)
- || !buf_page_is_corrupted(
- true, read_buf, zip_size, space())) {
+ || !buf_page_is_corrupted(
+ true, read_buf, zip_size, space()))) {
/* The page is good; there is no need
to consult the doublewrite buffer. */
continue;
@@ -589,16 +625,14 @@ buf_dblwr_process()
}
/* Next, validate the doublewrite page. */
- if (fil_page_is_compressed_encrypted(page) ||
- fil_page_is_compressed(page)) {
- /* Decompress the page before
- validating the checksum. */
- fil_decompress_page(
- NULL, page, srv_page_size, NULL, true);
+ if (fil_page_is_compressed(page)) {
+ success = buf_dblwr_decompress(page,
+ space_id, page_no, srv_page_size);
}
- if (!fil_space_verify_crypt_checksum(page, zip_size, NULL, page_no)
- && buf_page_is_corrupted(true, page, zip_size, space)) {
+ if (!success
+ || (!fil_space_verify_crypt_checksum(page, zip_size, NULL, page_no)
+ && buf_page_is_corrupted(true, page, zip_size, space))) {
if (!is_all_zero) {
ib_logf(IB_LOG_LEVEL_WARN,
"A doublewrite copy of page "
diff --git a/storage/innobase/dict/dict0dict.cc b/storage/innobase/dict/dict0dict.cc
index 7ec4364becc..d13d5a51a77 100644
--- a/storage/innobase/dict/dict0dict.cc
+++ b/storage/innobase/dict/dict0dict.cc
@@ -6113,6 +6113,12 @@ dict_set_corrupted_by_space(
table = dict_find_table_by_space(space_id);
if (!table) {
+#ifdef UNIV_DEBUG
+ ib_logf(IB_LOG_LEVEL_INFO,
+ "Can't set space " ULINTPF
+ " corrupted as table is not found dict_sys->table_LRU.", space_id);
+
+#endif
return(FALSE);
}
diff --git a/storage/innobase/fil/fil0crypt.cc b/storage/innobase/fil/fil0crypt.cc
index 3095503cfc5..fa0eaabf4dc 100644
--- a/storage/innobase/fil/fil0crypt.cc
+++ b/storage/innobase/fil/fil0crypt.cc
@@ -1,6 +1,6 @@
/*****************************************************************************
Copyright (C) 2013, 2015, Google Inc. All Rights Reserved.
-Copyright (c) 2014, 2017, MariaDB Corporation. All Rights Reserved.
+Copyright (c) 2014, 2018, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@@ -719,8 +719,8 @@ fil_space_encrypt(
comp_mem = (byte *)malloc(UNIV_PAGE_SIZE);
uncomp_mem = (byte *)malloc(UNIV_PAGE_SIZE);
memcpy(comp_mem, src_frame, UNIV_PAGE_SIZE);
- fil_decompress_page(uncomp_mem, comp_mem,
- srv_page_size, NULL);
+ ut_ad(fil_decompress_page(uncomp_mem, comp_mem,
+ srv_page_size, NULL));
src = uncomp_mem;
}
@@ -730,8 +730,8 @@ fil_space_encrypt(
/* Need to decompress the page if it was also compressed */
if (page_compressed_encrypted) {
memcpy(comp_mem, tmp_mem, UNIV_PAGE_SIZE);
- fil_decompress_page(tmp_mem, comp_mem,
- srv_page_size, NULL);
+ ut_ad(fil_decompress_page(tmp_mem, comp_mem,
+ srv_page_size, NULL));
}
bool corrupted = buf_page_is_corrupted(true, tmp_mem, zip_size, space);
@@ -2605,13 +2605,6 @@ fil_space_verify_crypt_checksum(
return(true);
}
- /* Compressed and encrypted pages do not have checksum. Assume not
- corrupted. Page verification happens after decompression in
- buf_page_io_complete() using buf_page_is_corrupted(). */
- if (mach_read_from_2(page+FIL_PAGE_TYPE) == FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED) {
- return (true);
- }
-
ib_uint32_t cchecksum1 = 0;
ib_uint32_t cchecksum2 = 0;
diff --git a/storage/innobase/fil/fil0fil.cc b/storage/innobase/fil/fil0fil.cc
index 8c7762c82c3..c993d188aa1 100644
--- a/storage/innobase/fil/fil0fil.cc
+++ b/storage/innobase/fil/fil0fil.cc
@@ -6516,8 +6516,32 @@ fil_iterate(
/* If the original page is page_compressed, we need
to decompress page before we can update it. */
if (page_compressed) {
- fil_decompress_page(NULL, dst, ulong(size),
- NULL);
+ /* Check post compression checksum before
+ doing actual decompression. */
+ if (fil_verify_compression_checksum(dst,
+ space_id, page_no)) {
+ if (!fil_decompress_page(NULL, dst, ulong(size),
+ NULL)) {
+ err = DB_CORRUPTION;
+ }
+ } else {
+ err = DB_CORRUPTION;
+ }
+
+ if (err != DB_SUCCESS) {
+ ib_logf(IB_LOG_LEVEL_ERROR,
+ "Imported page [page id: space=" ULINTPF
+ ", page number=" ULINTPF "]"
+ " is corrupted. Post compression"
+ " checksum " ULINTPF
+ " does not match calculated %u.",
+ space_id, page_no,
+ mach_read_from_4(dst+FIL_PAGE_SPACE_OR_CHKSUM),
+ buf_calc_compressed_crc32(dst));
+
+ return(err);
+ }
+
updated = true;
}
diff --git a/storage/innobase/fil/fil0pagecompress.cc b/storage/innobase/fil/fil0pagecompress.cc
index edc932f36f5..1ff04919121 100644
--- a/storage/innobase/fil/fil0pagecompress.cc
+++ b/storage/innobase/fil/fil0pagecompress.cc
@@ -1,6 +1,6 @@
/*****************************************************************************
-Copyright (C) 2013, 2017, MariaDB Corporation. All Rights Reserved.
+Copyright (C) 2013, 2018, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@@ -26,15 +26,19 @@ Updated 14/02/2015
#include "fil0fil.h"
#include "fil0pagecompress.h"
+#include "mtr0types.h"
+#include "mach0data.h"
+#include "buf0buf.h"
+#include "buf0checksum.h"
+#include "fsp0pagecompress.h"
+#ifndef UNIV_INNOCHECKSUM
#include <debug_sync.h>
#include <my_dbug.h>
#include "mem0mem.h"
#include "hash0hash.h"
#include "os0file.h"
-#include "mach0data.h"
-#include "buf0buf.h"
#include "buf0flu.h"
#include "log0recv.h"
#include "fsp0fsp.h"
@@ -364,8 +368,6 @@ fil_compress_page(
/* Set up the page header */
memcpy(out_buf, buf, FIL_PAGE_DATA);
- /* Set up the checksum */
- mach_write_to_4(out_buf+FIL_PAGE_SPACE_OR_CHKSUM, BUF_NO_CHECKSUM_MAGIC);
/* Set up the compression algorithm */
mach_write_to_8(out_buf+FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION, comp_method);
@@ -382,10 +384,13 @@ fil_compress_page(
/* Set up the actual payload lenght */
mach_write_to_2(out_buf+FIL_PAGE_DATA, write_size);
+ /* Set up the checksum */
+ mach_write_to_4(out_buf+FIL_PAGE_SPACE_OR_CHKSUM, buf_calc_compressed_crc32(out_buf));
+
#ifdef UNIV_DEBUG
/* Verify */
ut_ad(fil_page_is_compressed(out_buf) || fil_page_is_compressed_encrypted(out_buf));
- ut_ad(mach_read_from_4(out_buf+FIL_PAGE_SPACE_OR_CHKSUM) == BUF_NO_CHECKSUM_MAGIC);
+ ut_ad(mach_read_from_4(out_buf+FIL_PAGE_SPACE_OR_CHKSUM) == buf_calc_compressed_crc32(out_buf));
ut_ad(mach_read_from_2(out_buf+FIL_PAGE_DATA) == write_size);
ut_ad(mach_read_from_8(out_buf+FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION) == (ulint)comp_method ||
mach_read_from_2(out_buf+FIL_PAGE_DATA+FIL_PAGE_COMPRESSED_SIZE) == (ulint)comp_method);
@@ -399,7 +404,7 @@ fil_compress_page(
uncomp_page = static_cast<byte *>(ut_malloc(UNIV_PAGE_SIZE));
memcpy(comp_page, out_buf, UNIV_PAGE_SIZE);
- fil_decompress_page(uncomp_page, comp_page, ulong(len), NULL);
+ ut_ad(fil_decompress_page(uncomp_page, comp_page, ulong(len), NULL));
if (buf_page_is_corrupted(false, uncomp_page, 0, space)) {
buf_page_print(uncomp_page, 0);
@@ -466,22 +471,25 @@ err_exit:
}
-/****************************************************************//**
+/**
For page compressed pages decompress the page after actual read
-operation. */
+operation.
+@param[in,out] page_buf Preallocated temporal buffer where
+ compression is done and then copied
+ to the buffer.
+@param[in,out] buf Compressed page and after suggesful
+ decompression operation uncompressed page
+ is copied here.
+@param[in] len Length of output buffer.
+@param[out] write_size Actual payload size of the compressed data.
+@return true when operation succeeded or false when failed */
UNIV_INTERN
-void
+bool
fil_decompress_page(
-/*================*/
- byte* page_buf, /*!< in: preallocated buffer or NULL */
- byte* buf, /*!< out: buffer from which to read; in aio
- this must be appropriately aligned */
- ulong len, /*!< in: length of output buffer.*/
- ulint* write_size, /*!< in/out: Actual payload size of
- the compressed data. */
- bool return_error) /*!< in: true if only an error should
- be produced when decompression fails.
- By default this parameter is false. */
+ byte* page_buf,
+ byte* buf,
+ ulong len,
+ ulint* write_size)
{
int err = 0;
ulint actual_size = 0;
@@ -503,7 +511,7 @@ fil_decompress_page(
if (ptype != FIL_PAGE_PAGE_COMPRESSED &&
ptype != FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED &&
ptype != FIL_PAGE_TYPE_COMPRESSED) {
- return;
+ ut_error;
}
// If no buffer was given, we need to allocate temporal buffer
@@ -514,24 +522,6 @@ fil_decompress_page(
in_buf = page_buf;
}
- /* Before actual decompress, make sure that page type is correct */
-
- if (mach_read_from_4(buf+FIL_PAGE_SPACE_OR_CHKSUM) != BUF_NO_CHECKSUM_MAGIC ||
- (ptype != FIL_PAGE_PAGE_COMPRESSED &&
- ptype != FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED)) {
- ib_logf(IB_LOG_LEVEL_ERROR,
- "Corruption: We try to uncompress corrupted page"
- " CRC " ULINTPF " type " ULINTPF " len " ULINTPF ".",
- mach_read_from_4(buf+FIL_PAGE_SPACE_OR_CHKSUM),
- mach_read_from_2(buf+FIL_PAGE_TYPE), len);
-
- fflush(stderr);
- if (return_error) {
- goto error_return;
- }
- ut_error;
- }
-
/* Get compression algorithm */
if (ptype == FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED) {
compression_alg = mach_read_from_2(buf+FIL_PAGE_DATA+FIL_PAGE_COMPRESSED_SIZE);
@@ -541,17 +531,10 @@ fil_decompress_page(
/* Get the actual size of compressed page */
actual_size = mach_read_from_2(buf+FIL_PAGE_DATA);
+
/* Check if payload size is corrupted */
if (actual_size == 0 || actual_size > UNIV_PAGE_SIZE) {
- ib_logf(IB_LOG_LEVEL_ERROR,
- "Corruption: We try to uncompress corrupted page"
- " actual size " ULINTPF " compression %s.",
- actual_size, fil_get_compression_alg_name(compression_alg));
- fflush(stderr);
- if (return_error) {
- goto error_return;
- }
- ut_error;
+ goto error_return;
}
/* Store actual payload size of the compressed data. This pointer
@@ -570,19 +553,7 @@ fil_decompress_page(
/* If uncompress fails it means that page is corrupted */
if (err != Z_OK) {
-
- ib_logf(IB_LOG_LEVEL_ERROR,
- "Corruption: Page is marked as compressed"
- " but uncompress failed with error %d "
- " size " ULINTPF " len " ULINTPF ".",
- err, actual_size, len);
-
- fflush(stderr);
-
- if (return_error) {
- goto error_return;
- }
- ut_error;
+ goto error_return;
}
break;
@@ -591,18 +562,7 @@ fil_decompress_page(
err = LZ4_decompress_fast((const char *)buf+header_len, (char *)in_buf, len);
if (err != (int)actual_size) {
- ib_logf(IB_LOG_LEVEL_ERROR,
- "Corruption: Page is marked as compressed"
- " but uncompress failed with error %d "
- " size " ULINTPF " len " ULINTPF ".",
- err, actual_size, len);
-
- fflush(stderr);
-
- if (return_error) {
- goto error_return;
- }
- ut_error;
+ goto error_return;
}
break;
#endif /* HAVE_LZ4 */
@@ -616,18 +576,7 @@ fil_decompress_page(
olen = olen_lzo;
if (err != LZO_E_OK || (olen == 0 || olen > UNIV_PAGE_SIZE)) {
- ib_logf(IB_LOG_LEVEL_ERROR,
- "Corruption: Page is marked as compressed"
- " but uncompress failed with error %d "
- " size " ULINTPF " len " ULINTPF ".",
- err, actual_size, len);
-
- fflush(stderr);
-
- if (return_error) {
- goto error_return;
- }
- ut_error;
+ goto error_return;
}
break;
}
@@ -653,17 +602,7 @@ fil_decompress_page(
if (ret != LZMA_OK || (dst_pos == 0 || dst_pos > UNIV_PAGE_SIZE)) {
- ib_logf(IB_LOG_LEVEL_ERROR,
- "Corruption: Page is marked as compressed"
- " but decompression read only %ld bytes"
- " size " ULINTPF "len " ULINTPF ".",
- dst_pos, actual_size, len);
- fflush(stderr);
-
- if (return_error) {
- goto error_return;
- }
- ut_error;
+ goto error_return;
}
break;
@@ -682,17 +621,8 @@ fil_decompress_page(
0);
if (err != BZ_OK || (dst_pos == 0 || dst_pos > UNIV_PAGE_SIZE)) {
- ib_logf(IB_LOG_LEVEL_ERROR,
- "Corruption: Page is marked as compressed"
- " but decompression read only %du bytes"
- " size " ULINTPF " len " ULINTPF " err %d.",
- dst_pos, actual_size, len, err);
- fflush(stderr);
-
- if (return_error) {
- goto error_return;
- }
- ut_error;
+ len = dst_pos;
+ goto error_return;
}
break;
}
@@ -709,35 +639,17 @@ fil_decompress_page(
(char *)in_buf,
(size_t*)&olen);
- if (cstatus != SNAPPY_OK || olen != UNIV_PAGE_SIZE) {
- ib_logf(IB_LOG_LEVEL_ERROR,
- "Corruption: Page is marked as compressed"
- " but decompression read only " ULINTPF " bytes"
- " size " ULINTPF " len " ULINTPF " err %d.",
- olen, actual_size, len, (int)cstatus);
- fflush(stderr);
-
- if (return_error) {
- goto error_return;
- }
- ut_error;
+ if (cstatus != SNAPPY_OK || (olen == 0 || olen > UNIV_PAGE_SIZE)) {
+ err = (int)cstatus;
+ len = olen;
+ goto error_return;
}
break;
}
#endif /* HAVE_SNAPPY */
default:
- ib_logf(IB_LOG_LEVEL_ERROR,
- "Corruption: Page is marked as compressed"
- " but compression algorithm %s"
- " is not known."
- ,fil_get_compression_alg_name(compression_alg));
-
- fflush(stderr);
- if (return_error) {
- goto error_return;
- }
- ut_error;
+ goto error_return;
break;
}
@@ -747,8 +659,108 @@ fil_decompress_page(
really any other options. */
memcpy(buf, in_buf, len);
+ if (page_buf != in_buf) {
+ ut_free(in_buf);
+ }
+ return true;
+
error_return:
if (page_buf != in_buf) {
ut_free(in_buf);
}
+
+ /* Note that as we have found the page is corrupted, so
+ all this could be incorrect. */
+ ulint space_id = mach_read_from_4(buf+FIL_PAGE_SPACE_ID);
+ fil_space_t* space = fil_space_acquire_for_io(space_id);
+
+ ib_logf(IB_LOG_LEVEL_ERROR,
+ "Corruption: Page is marked as compressed"
+ " space : " ULINTPF " name %s "
+ " but uncompress failed with error: %d"
+ " size: " ULINTPF
+ " len: " ULINTPF
+ " compression method %s",
+ space_id,
+ (space ? space->chain.start->name : "(import)"),
+ err,
+ actual_size,
+ len,
+ fil_get_compression_alg_name(compression_alg));
+
+ if (space) {
+ fil_space_release(space);
+ }
+
+ return false;
+}
+#endif /* !UNIV_INNOCHECKSUM */
+
+/**
+Verify that stored post compression checksum matches calculated
+checksum. Note that old format did not have a checksum and
+in that case either original pre-compression page checksum will
+fail after decompression or page decompression fails.
+
+@param[in,out] page page frame
+@param[in] space_id Tablespace identifier
+@param[in] offset Page offset
+@return true if post compression checksum matches, false otherwise */
+#ifndef UNIV_INNOCHECKSUM
+UNIV_INTERN
+#endif
+bool
+fil_verify_compression_checksum(
+ const byte* page,
+ ulint space_id,
+ ulint offset)
+{
+ ut_ad(page &&
+ (mach_read_from_2(page+FIL_PAGE_TYPE) == FIL_PAGE_PAGE_COMPRESSED
+ || mach_read_from_2(page+FIL_PAGE_TYPE) == FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED));
+
+ uint32_t checksum = mach_read_from_4(page+FIL_PAGE_SPACE_OR_CHKSUM);
+
+ /* In earlier versions supporting page compression, page compression
+ stored BUF_NO_CHECKSUM_MAGIC to checksum filed. These pages are
+ treaded as not corrupted. */
+ if (checksum == BUF_NO_CHECKSUM_MAGIC) {
+ return (true);
+ }
+
+ uint32_t cchecksum = buf_calc_compressed_crc32(page);
+
+ if (checksum != cchecksum) {
+ ulint comp_method = mach_read_from_2(page+FIL_PAGE_TYPE) == FIL_PAGE_PAGE_COMPRESSED
+ ? mach_read_from_8(page+FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION)
+ : mach_read_from_2(page+FIL_PAGE_DATA+FIL_PAGE_COMPRESSED_SIZE);
+
+#ifdef UNIV_INNOCHECKSUM
+ fprintf(log_file ? log_file : stderr,
+ "Page " ULINTPF ":" ULINTPF " may be corrupted."
+ " Post compression checksum %u"
+ " stored %u compression_method %s\n",
+ space_id, offset, checksum, cchecksum,
+ fil_get_compression_alg_name(comp_method));
+#else /*! UNIV_INNOCHECKSUM */
+ FilSpace space(space_id, true);
+
+ ib_logf(IB_LOG_LEVEL_ERROR,
+ "Page [page id: space=" ULINTPF
+ ", page number= " ULINTPF "]"
+ " in file %s"
+ " may be corrupted."
+ " Post compression checksum %u"
+ " stored %u"
+ " compression_method %s",
+ space_id,
+ offset,
+ (space() ? space()->chain.start->name : "(import)"),
+ checksum,
+ cchecksum,
+ fil_get_compression_alg_name(comp_method));
+#endif
+ }
+
+ return (checksum == cchecksum);
}
diff --git a/storage/innobase/handler/ha_innodb.cc b/storage/innobase/handler/ha_innodb.cc
index 4f76c568f17..8e9a5ed2f55 100644
--- a/storage/innobase/handler/ha_innodb.cc
+++ b/storage/innobase/handler/ha_innodb.cc
@@ -1997,6 +1997,7 @@ convert_error_code_to_mysql(
code should be introduced */
case DB_CORRUPTION:
+ case DB_PAGE_CORRUPTED:
return(HA_ERR_CRASHED);
case DB_OUT_OF_FILE_SPACE:
@@ -6032,6 +6033,8 @@ table_opened:
buf, space()->chain.start->name);
ret_err = HA_ERR_DECRYPTION_FAILED;
}
+ } else if (ib_table->corrupted) {
+ ret_err = HA_ERR_CRASHED;
}
dict_table_close(ib_table, FALSE, FALSE);
diff --git a/storage/innobase/include/buf0checksum.h b/storage/innobase/include/buf0checksum.h
index 6818345f965..7834ad4d292 100644
--- a/storage/innobase/include/buf0checksum.h
+++ b/storage/innobase/include/buf0checksum.h
@@ -1,6 +1,7 @@
/*****************************************************************************
Copyright (c) 1995, 2011, Oracle and/or its affiliates. All Rights Reserved.
+Copyright (c) 2017, 2018, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@@ -80,4 +81,16 @@ buf_checksum_algorithm_name(
extern ulong srv_checksum_algorithm;
+/** Calculates the CRC32 checksum of a page compressed page. The value is
+stored to the page when it is written to a file and also checked for
+a match when reading from the file. Checksum is calculated from
+actual payload of the compressed page and some header fields.
+
+@param[in] page buffer page (UNIV_PAGE_SIZE bytes)
+@return checksum */
+UNIV_INTERN
+uint32_t
+buf_calc_compressed_crc32(
+ const byte* page);
+
#endif /* buf0checksum_h */
diff --git a/storage/innobase/include/fil0pagecompress.h b/storage/innobase/include/fil0pagecompress.h
index 03e16699ce3..bd1395b4835 100644
--- a/storage/innobase/include/fil0pagecompress.h
+++ b/storage/innobase/include/fil0pagecompress.h
@@ -1,6 +1,6 @@
/*****************************************************************************
-Copyright (C) 2013, 2017 MariaDB Corporation. All Rights Reserved.
+Copyright (C) 2013, 2018 MariaDB Corporation. All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@@ -19,6 +19,8 @@ this program; if not, write to the Free Software Foundation, Inc.,
#ifndef fil0pagecompress_h
#define fil0pagecompress_h
+#ifndef UNIV_INNOCHECKSUM
+
#include "fsp0fsp.h"
#include "fsp0pagecompress.h"
@@ -68,23 +70,26 @@ fil_compress_page(
ulint* out_len); /*!< out: actual length of compressed
page */
-/****************************************************************//**
+/**
For page compressed pages decompress the page after actual read
-operation. */
+operation.
+@param[in,out] page_buf Preallocated temporal buffer where
+ compression is done and then copied
+ to the buffer.
+@param[in,out] buf Compressed page and after suggesful
+ decompression operation uncompressed page
+ is copied here.
+@param[in] len Length of output buffer.
+@param[out] write_size Actual payload size of the compressed data.
+@return true when operation succeeded or false when failed */
UNIV_INTERN
-void
+bool
fil_decompress_page(
-/*================*/
- byte* page_buf, /*!< in: preallocated buffer or NULL */
- byte* buf, /*!< out: buffer from which to read; in aio
- this must be appropriately aligned */
- ulong len, /*!< in: length of output buffer.*/
- ulint* write_size, /*!< in/out: Actual payload size of
- the compressed data. */
- bool return_error=false);
- /*!< in: true if only an error should
- be produced when decompression fails.
- By default this parameter is false. */
+ byte* page_buf,
+ byte* buf,
+ ulong len,
+ ulint* write_size)
+ MY_ATTRIBUTE((warn_unused_result));
/****************************************************************//**
Get space id from fil node
@@ -129,4 +134,25 @@ ibool
fil_page_is_lzo_compressed(
/*=======================*/
byte* buf); /*!< in: page */
-#endif
+#endif /* !UNIV_INNOCHECKSUM */
+
+/**
+Verify that stored post compression checksum matches calculated
+checksum. Note that old format did not have a checksum and
+in that case either original pre-compression page checksum will
+fail after decompression or page decompression fails.
+
+@param[in,out] page page frame
+@param[in] space_id Tablespace identifier
+@param[in] offset Page offset
+@return true if post compression checksum matches, false otherwise */
+UNIV_INTERN
+bool
+fil_verify_compression_checksum(
+ const byte* page,
+ ulint space_id,
+ ulint offset)
+ MY_ATTRIBUTE((warn_unused_result));
+
+#endif /* fil0pagecompress_h */
+
diff --git a/storage/innobase/include/fsp0pagecompress.h b/storage/innobase/include/fsp0pagecompress.h
index c623d11c326..00645616556 100644
--- a/storage/innobase/include/fsp0pagecompress.h
+++ b/storage/innobase/include/fsp0pagecompress.h
@@ -1,6 +1,6 @@
/*****************************************************************************
-Copyright (C) 2013, 2017, MariaDB Corporation. All Rights Reserved.
+Copyright (C) 2013, 2018, MariaDB Corporation. All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@@ -38,6 +38,7 @@ Created 11/12/2013 Jan Lindström jan.lindstrom@skysql.com
#define PAGE_SNAPPY_ALGORITHM 6
#define PAGE_ALGORITHM_LAST PAGE_SNAPPY_ALGORITHM
+#ifndef UNIV_INNOCHECKSUM
/**********************************************************************//**
Reads the page compression level from the first page of a tablespace.
@return page compression level, or 0 if uncompressed */
@@ -67,6 +68,7 @@ atomic_writes_t
fsp_flags_get_atomic_writes(
/*========================*/
ulint flags); /*!< in: tablespace flags */
+#endif /* !UNIV_INNOCHECKSUM */
#ifndef UNIV_NONINL
#include "fsp0pagecompress.ic"
diff --git a/storage/innobase/include/fsp0pagecompress.ic b/storage/innobase/include/fsp0pagecompress.ic
index 14f968e319e..0e9a574eef6 100644
--- a/storage/innobase/include/fsp0pagecompress.ic
+++ b/storage/innobase/include/fsp0pagecompress.ic
@@ -25,6 +25,7 @@ Created 11/12/2013 Jan Lindström jan.lindstrom@mariadb.com
***********************************************************************/
+#ifndef UNIV_INNOCHECKSUM
/********************************************************************//**
Determine the tablespace is page compression level from dict_table_t::flags.
@return page compression level or 0 if not compressed*/
@@ -84,6 +85,7 @@ fil_page_is_compressed_encrypted(
{
return(mach_read_from_2(buf+FIL_PAGE_TYPE) == FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED);
}
+#endif /*!UNIV_INNOCHECKSUM */
/****************************************************************//**
Get the name of the compression algorithm used for page
diff --git a/storage/xtradb/buf/buf0buf.cc b/storage/xtradb/buf/buf0buf.cc
index 83e38fbcc98..3e2e2267bf4 100644
--- a/storage/xtradb/buf/buf0buf.cc
+++ b/storage/xtradb/buf/buf0buf.cc
@@ -2,7 +2,7 @@
Copyright (c) 1995, 2016, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2008, Google Inc.
-Copyright (c) 2013, 2017, MariaDB Corporation.
+Copyright (c) 2013, 2018, MariaDB Corporation.
Portions of this file contain modifications contributed and copyrighted by
Google, Inc. Those modifications are gratefully acknowledged and are described
@@ -739,22 +739,25 @@ buf_page_is_corrupted(
ulint checksum_field2;
ulint space_id = mach_read_from_4(
read_buf + FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID);
- ulint page_type = mach_read_from_2(
- read_buf + FIL_PAGE_TYPE);
/* We can trust page type if page compression is set on tablespace
flags because page compression flag means file must have been
created with 10.1 (later than 5.5 code base). In 10.1 page
compressed tables do not contain post compression checksum and
- FIL_PAGE_END_LSN_OLD_CHKSUM field stored. Note that space can
- be null if we are in fil_check_first_page() and first page
- is not compressed or encrypted. Page checksum is verified
- after decompression (i.e. normally pages are already
- decompressed at this stage). */
+ FIL_PAGE_END_LSN_OLD_CHKSUM field stored.
+
+ Note that space can be null if we are in fil_check_first_page() but
+ first page (page 0) is not compressed or encrypted.
+
+ Page checksum is verified after decompression (i.e. normally pages
+ are already decompressed at this stage). */
+ ulint page_type = mach_read_from_2(read_buf + FIL_PAGE_TYPE);
if ((page_type == FIL_PAGE_PAGE_COMPRESSED ||
page_type == FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED)
- && space && FSP_FLAGS_HAS_PAGE_COMPRESSION(space->flags)) {
- return (false);
+ && (!space
+ || (space && FSP_FLAGS_HAS_PAGE_COMPRESSION(space->flags))))
+ {
+ return(false);
}
if (!zip_size
@@ -975,12 +978,14 @@ buf_page_print(const byte* read_buf, ulint zip_size)
size = UNIV_PAGE_SIZE;
}
+#ifdef UNIV_BUG_DEBUG
ut_print_timestamp(stderr);
fprintf(stderr,
" InnoDB: Page dump in ascii and hex (%lu bytes):\n",
size);
ut_print_buf(stderr, read_buf, size);
fputs("\nInnoDB: End of page dump\n", stderr);
+#endif
if (zip_size) {
/* Print compressed page. */
@@ -3113,8 +3118,8 @@ loop:
/* Try to set table as corrupted instead of
asserting. */
- if (space > TRX_SYS_SPACE &&
- dict_set_corrupted_by_space(space)) {
+ if (space > TRX_SYS_SPACE) {
+ dict_set_corrupted_by_space(space);
return (NULL);
}
@@ -4657,21 +4662,19 @@ buf_page_check_corrupt(buf_page_t* bpage, fil_space_t* space)
still be corrupted if used key does not match. */
still_encrypted = (crypt_data &&
crypt_data->type != CRYPT_SCHEME_UNENCRYPTED &&
- !bpage->encrypted &&
fil_space_verify_crypt_checksum(dst_frame, zip_size,
space, bpage->offset));
- if (!still_encrypted) {
- /* If traditional checksums match, we assume that page is
- not anymore encrypted. */
- corrupted = buf_page_is_corrupted(true, dst_frame, zip_size,
+ /* If traditional checksums match, we assume that page is
+ not anymore encrypted. */
+ corrupted = buf_page_is_corrupted(true, dst_frame, zip_size,
space);
- if (!corrupted) {
- bpage->encrypted = false;
- } else {
- err = DB_PAGE_CORRUPTED;
- }
+ if (!corrupted) {
+ bpage->encrypted = false;
+ still_encrypted = false;
+ } else if (!still_encrypted) {
+ err = DB_PAGE_CORRUPTED;
}
/* Pages that we think are unencrypted but do not match the checksum
@@ -4688,7 +4691,7 @@ buf_page_check_corrupt(buf_page_t* bpage, fil_space_t* space)
", page number=%u]"
" in file %s cannot be decrypted.",
bpage->space, bpage->offset,
- space->name);
+ space->chain.start->name);
ib_logf(IB_LOG_LEVEL_INFO,
"However key management plugin or used key_version " ULINTPF
@@ -4748,7 +4751,11 @@ buf_page_io_complete(buf_page_t* bpage)
return(DB_TABLESPACE_DELETED);
}
- buf_page_decrypt_after_read(bpage, space);
+ dberr_t err = DB_SUCCESS;
+
+ if (!buf_page_decrypt_after_read(bpage, space)) {
+ err = DB_PAGE_CORRUPTED;
+ }
if (buf_page_get_zip_size(bpage)) {
frame = bpage->zip.data;
@@ -4756,6 +4763,10 @@ buf_page_io_complete(buf_page_t* bpage)
frame = ((buf_block_t*) bpage)->frame;
}
+ if (err != DB_SUCCESS) {
+ goto database_corrupted;
+ }
+
if (buf_page_get_zip_size(bpage)) {
frame = bpage->zip.data;
os_atomic_increment_ulint(&buf_pool->n_pend_unzip, 1);
@@ -4846,7 +4857,7 @@ database_corrupted:
ib_logf(IB_LOG_LEVEL_ERROR,
"Database page corruption on disk"
" or a failed file read of tablespace %s"
- " page [page id: space=%u"
+ " page [page id: space=%u"
", page number=%u]"
". You may have to recover from "
"a backup.",
@@ -6449,10 +6460,14 @@ buf_page_decrypt_after_read(buf_page_t* bpage, fil_space_t* space)
#endif
/* decompress using comp_buf to dst_frame */
- fil_decompress_page(slot->comp_buf,
- dst_frame,
- ulong(size),
- &bpage->write_size);
+ if (!fil_verify_compression_checksum(dst_frame,
+ bpage->space, bpage->offset)
+ || !fil_decompress_page(slot->comp_buf,
+ dst_frame,
+ ulong(size),
+ &bpage->write_size)) {
+ success = false;
+ }
/* Mark this slot as free */
slot->reserved = false;
@@ -6507,10 +6522,14 @@ buf_page_decrypt_after_read(buf_page_t* bpage, fil_space_t* space)
fil_page_type_validate(dst_frame);
#endif
/* decompress using comp_buf to dst_frame */
- fil_decompress_page(slot->comp_buf,
+ if (!fil_verify_compression_checksum(dst_frame,
+ bpage->space, bpage->offset)
+ || !fil_decompress_page(slot->comp_buf,
dst_frame,
ulong(size),
- &bpage->write_size);
+ &bpage->write_size)) {
+ success = false;
+ }
ut_d(fil_page_type_validate(dst_frame));
}
diff --git a/storage/xtradb/buf/buf0checksum.cc b/storage/xtradb/buf/buf0checksum.cc
index 01b646a78e0..98abba575ef 100644
--- a/storage/xtradb/buf/buf0checksum.cc
+++ b/storage/xtradb/buf/buf0checksum.cc
@@ -1,6 +1,7 @@
/*****************************************************************************
Copyright (c) 1995, 2015, Oracle and/or its affiliates. All Rights Reserved.
+Copyright (c) 2017, 2018, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@@ -28,6 +29,7 @@ Created Aug 11, 2011 Vasil Dimov
#include "ut0crc32.h" /* ut_crc32() */
#include "ut0rnd.h" /* ut_fold_binary() */
#include "buf0types.h"
+#include "mach0data.h"
#ifndef UNIV_INNOCHECKSUM
@@ -154,3 +156,46 @@ buf_checksum_algorithm_name(
ut_error;
return(NULL);
}
+
+/** Calculates the CRC32 checksum of a page compressed page. The value is
+stored to the page when it is written to a file and also checked for
+a match when reading from the file. Checksum is calculated from
+actual payload of the compressed page and some header fields.
+
+@param[in] page buffer page (UNIV_PAGE_SIZE bytes)
+@return checksum */
+UNIV_INTERN
+uint32_t
+buf_calc_compressed_crc32(
+ const byte* page)
+{
+ /* In page compressed pages compression method is stored to field
+ FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION, thus add it to checksum.
+ In pages first compressed and then encrypted same field
+ contains key version after compression. */
+
+ ulint page_type = mach_read_from_2(page + FIL_PAGE_TYPE);
+
+ ulint header_len = page_type == FIL_PAGE_PAGE_COMPRESSED ?
+ FIL_PAGE_SPACE_ID - FIL_PAGE_OFFSET :
+ FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION - FIL_PAGE_OFFSET;
+
+ const uint32_t c1 = ut_crc32(
+ page + FIL_PAGE_OFFSET,
+ header_len);
+
+ /* Calculate checksum from actual payload including stored size
+ field. In encrypted case add also compression method field. */
+ ulint payload_len = mach_read_from_2(page+FIL_PAGE_DATA)+FIL_PAGE_COMPRESSED_SIZE;
+
+ if (page_type == FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED) {
+ payload_len += FIL_PAGE_COMPRESSION_METHOD_SIZE;
+ }
+
+ const uint32_t c2 = ut_crc32(
+ page + FIL_PAGE_DATA,
+ payload_len);
+
+ return(c1 ^ c2);
+}
+
diff --git a/storage/xtradb/buf/buf0dblwr.cc b/storage/xtradb/buf/buf0dblwr.cc
index ee8c85446e9..369a6a3255e 100644
--- a/storage/xtradb/buf/buf0dblwr.cc
+++ b/storage/xtradb/buf/buf0dblwr.cc
@@ -1,7 +1,7 @@
/*****************************************************************************
Copyright (c) 1995, 2017, Oracle and/or its affiliates. All Rights Reserved.
-Copyright (c) 2013, 2017, MariaDB Corporation.
+Copyright (c) 2013, 2018, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@@ -488,8 +488,45 @@ leave_func:
ut_free(unaligned_read_buf);
}
-/****************************************************************//**
-Process the double write buffer pages. */
+/** Verify post compression checksum and if it matches
+decompress page.
+@param[in,out] read_buf Page compressed buffer. Result of
+ decompression is copied here.
+@param[in] space_id Tablespace identifier
+@param[in] page_no Page offset
+@param[in] page_size Actual page size
+@return true if both stored post compression checksum matches calculated one
+or it is BUF_NO_CHECKSUM_MAGIC and page decompression is succeeds.*/
+static
+bool
+buf_dblwr_decompress(
+ byte * read_buf,
+ ulint space_id,
+ ulint page_no,
+ ulint page_size)
+{
+ bool success = true;
+
+ /* Page is always fist compressed and then encrypted. Here we can
+ only decompress pages if they are not encrypted.
+
+ For compressed pages verify post compression checksum and if it
+ matches decompress the page.
+
+ Note that old datafiles have checksum BUF_NO_CHECKSUM_MAGIC and
+ that value is accepted as correct post compression checksum see
+ fil_verify_compression_checksum().
+
+ In that case we hope that compression algorithm can recover from
+ corrupted input and return a error that is then handled.*/
+ if(!fil_verify_compression_checksum(read_buf, space_id, page_no)
+ || !fil_decompress_page(NULL, read_buf, page_size, NULL)) {
+ return success = false;
+ }
+
+ return success;
+}
+
void
buf_dblwr_process()
/*===============*/
@@ -556,25 +593,24 @@ buf_dblwr_process()
const bool is_all_zero = buf_page_is_zeroes(
read_buf, zip_size);
+ bool success = true;
+
if (is_all_zero) {
/* We will check if the copy in the
doublewrite buffer is valid. If not, we will
ignore this page (there should be redo log
records to initialize it). */
} else {
- if (fil_page_is_compressed_encrypted(read_buf) ||
- fil_page_is_compressed(read_buf)) {
- /* Decompress the page before
- validating the checksum. */
- fil_decompress_page(
- NULL, read_buf, srv_page_size,
- NULL, true);
+ if (fil_page_is_compressed(read_buf)) {
+ success = buf_dblwr_decompress(read_buf,
+ space_id, page_no, srv_page_size);
}
- if (fil_space_verify_crypt_checksum(
+ if (success
+ && (fil_space_verify_crypt_checksum(
read_buf, zip_size, NULL, page_no)
- || !buf_page_is_corrupted(
- true, read_buf, zip_size, space())) {
+ || !buf_page_is_corrupted(
+ true, read_buf, zip_size, space()))) {
/* The page is good; there is no need
to consult the doublewrite buffer. */
continue;
@@ -589,16 +625,14 @@ buf_dblwr_process()
}
/* Next, validate the doublewrite page. */
- if (fil_page_is_compressed_encrypted(page) ||
- fil_page_is_compressed(page)) {
- /* Decompress the page before
- validating the checksum. */
- fil_decompress_page(
- NULL, page, srv_page_size, NULL, true);
+ if (fil_page_is_compressed(page)) {
+ success = buf_dblwr_decompress(page,
+ space_id, page_no, srv_page_size);
}
- if (!fil_space_verify_crypt_checksum(page, zip_size, NULL, page_no)
- && buf_page_is_corrupted(true, page, zip_size, space)) {
+ if (!success
+ || (!fil_space_verify_crypt_checksum(page, zip_size, NULL, page_no)
+ && buf_page_is_corrupted(true, page, zip_size, space))) {
if (!is_all_zero) {
ib_logf(IB_LOG_LEVEL_WARN,
"A doublewrite copy of page "
diff --git a/storage/xtradb/dict/dict0dict.cc b/storage/xtradb/dict/dict0dict.cc
index 9257321c7ef..cc0241e6789 100644
--- a/storage/xtradb/dict/dict0dict.cc
+++ b/storage/xtradb/dict/dict0dict.cc
@@ -6120,6 +6120,12 @@ dict_set_corrupted_by_space(
table = dict_find_table_by_space(space_id);
if (!table) {
+#ifdef UNIV_DEBUG
+ ib_logf(IB_LOG_LEVEL_INFO,
+ "Can't set space " ULINTPF
+ " corrupted as table is not found dict_sys->table_LRU.", space_id);
+
+#endif
return(FALSE);
}
diff --git a/storage/xtradb/fil/fil0crypt.cc b/storage/xtradb/fil/fil0crypt.cc
index 3095503cfc5..fa0eaabf4dc 100644
--- a/storage/xtradb/fil/fil0crypt.cc
+++ b/storage/xtradb/fil/fil0crypt.cc
@@ -1,6 +1,6 @@
/*****************************************************************************
Copyright (C) 2013, 2015, Google Inc. All Rights Reserved.
-Copyright (c) 2014, 2017, MariaDB Corporation. All Rights Reserved.
+Copyright (c) 2014, 2018, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@@ -719,8 +719,8 @@ fil_space_encrypt(
comp_mem = (byte *)malloc(UNIV_PAGE_SIZE);
uncomp_mem = (byte *)malloc(UNIV_PAGE_SIZE);
memcpy(comp_mem, src_frame, UNIV_PAGE_SIZE);
- fil_decompress_page(uncomp_mem, comp_mem,
- srv_page_size, NULL);
+ ut_ad(fil_decompress_page(uncomp_mem, comp_mem,
+ srv_page_size, NULL));
src = uncomp_mem;
}
@@ -730,8 +730,8 @@ fil_space_encrypt(
/* Need to decompress the page if it was also compressed */
if (page_compressed_encrypted) {
memcpy(comp_mem, tmp_mem, UNIV_PAGE_SIZE);
- fil_decompress_page(tmp_mem, comp_mem,
- srv_page_size, NULL);
+ ut_ad(fil_decompress_page(tmp_mem, comp_mem,
+ srv_page_size, NULL));
}
bool corrupted = buf_page_is_corrupted(true, tmp_mem, zip_size, space);
@@ -2605,13 +2605,6 @@ fil_space_verify_crypt_checksum(
return(true);
}
- /* Compressed and encrypted pages do not have checksum. Assume not
- corrupted. Page verification happens after decompression in
- buf_page_io_complete() using buf_page_is_corrupted(). */
- if (mach_read_from_2(page+FIL_PAGE_TYPE) == FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED) {
- return (true);
- }
-
ib_uint32_t cchecksum1 = 0;
ib_uint32_t cchecksum2 = 0;
diff --git a/storage/xtradb/fil/fil0fil.cc b/storage/xtradb/fil/fil0fil.cc
index 83c17bf113f..1d56450b4bd 100644
--- a/storage/xtradb/fil/fil0fil.cc
+++ b/storage/xtradb/fil/fil0fil.cc
@@ -6861,8 +6861,32 @@ fil_iterate(
/* If the original page is page_compressed, we need
to decompress page before we can update it. */
if (page_compressed) {
- fil_decompress_page(NULL, dst, ulong(size),
- NULL);
+ /* Check post compression checksum before
+ doing actual decompression. */
+ if (fil_verify_compression_checksum(dst,
+ space_id, page_no)) {
+ if (!fil_decompress_page(NULL, dst, ulong(size),
+ NULL)) {
+ err = DB_CORRUPTION;
+ }
+ } else {
+ err = DB_CORRUPTION;
+ }
+
+ if (err != DB_SUCCESS) {
+ ib_logf(IB_LOG_LEVEL_ERROR,
+ "Imported page [page id: space=" ULINTPF
+ ", page number=" ULINTPF "]"
+ " is corrupted. Post compression"
+ " checksum " ULINTPF
+ " does not match calculated %u.",
+ space_id, page_no,
+ mach_read_from_4(dst+FIL_PAGE_SPACE_OR_CHKSUM),
+ buf_calc_compressed_crc32(dst));
+
+ return(err);
+ }
+
updated = true;
}
diff --git a/storage/xtradb/fil/fil0pagecompress.cc b/storage/xtradb/fil/fil0pagecompress.cc
index edc932f36f5..1ff04919121 100644
--- a/storage/xtradb/fil/fil0pagecompress.cc
+++ b/storage/xtradb/fil/fil0pagecompress.cc
@@ -1,6 +1,6 @@
/*****************************************************************************
-Copyright (C) 2013, 2017, MariaDB Corporation. All Rights Reserved.
+Copyright (C) 2013, 2018, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@@ -26,15 +26,19 @@ Updated 14/02/2015
#include "fil0fil.h"
#include "fil0pagecompress.h"
+#include "mtr0types.h"
+#include "mach0data.h"
+#include "buf0buf.h"
+#include "buf0checksum.h"
+#include "fsp0pagecompress.h"
+#ifndef UNIV_INNOCHECKSUM
#include <debug_sync.h>
#include <my_dbug.h>
#include "mem0mem.h"
#include "hash0hash.h"
#include "os0file.h"
-#include "mach0data.h"
-#include "buf0buf.h"
#include "buf0flu.h"
#include "log0recv.h"
#include "fsp0fsp.h"
@@ -364,8 +368,6 @@ fil_compress_page(
/* Set up the page header */
memcpy(out_buf, buf, FIL_PAGE_DATA);
- /* Set up the checksum */
- mach_write_to_4(out_buf+FIL_PAGE_SPACE_OR_CHKSUM, BUF_NO_CHECKSUM_MAGIC);
/* Set up the compression algorithm */
mach_write_to_8(out_buf+FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION, comp_method);
@@ -382,10 +384,13 @@ fil_compress_page(
/* Set up the actual payload lenght */
mach_write_to_2(out_buf+FIL_PAGE_DATA, write_size);
+ /* Set up the checksum */
+ mach_write_to_4(out_buf+FIL_PAGE_SPACE_OR_CHKSUM, buf_calc_compressed_crc32(out_buf));
+
#ifdef UNIV_DEBUG
/* Verify */
ut_ad(fil_page_is_compressed(out_buf) || fil_page_is_compressed_encrypted(out_buf));
- ut_ad(mach_read_from_4(out_buf+FIL_PAGE_SPACE_OR_CHKSUM) == BUF_NO_CHECKSUM_MAGIC);
+ ut_ad(mach_read_from_4(out_buf+FIL_PAGE_SPACE_OR_CHKSUM) == buf_calc_compressed_crc32(out_buf));
ut_ad(mach_read_from_2(out_buf+FIL_PAGE_DATA) == write_size);
ut_ad(mach_read_from_8(out_buf+FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION) == (ulint)comp_method ||
mach_read_from_2(out_buf+FIL_PAGE_DATA+FIL_PAGE_COMPRESSED_SIZE) == (ulint)comp_method);
@@ -399,7 +404,7 @@ fil_compress_page(
uncomp_page = static_cast<byte *>(ut_malloc(UNIV_PAGE_SIZE));
memcpy(comp_page, out_buf, UNIV_PAGE_SIZE);
- fil_decompress_page(uncomp_page, comp_page, ulong(len), NULL);
+ ut_ad(fil_decompress_page(uncomp_page, comp_page, ulong(len), NULL));
if (buf_page_is_corrupted(false, uncomp_page, 0, space)) {
buf_page_print(uncomp_page, 0);
@@ -466,22 +471,25 @@ err_exit:
}
-/****************************************************************//**
+/**
For page compressed pages decompress the page after actual read
-operation. */
+operation.
+@param[in,out] page_buf Preallocated temporal buffer where
+ compression is done and then copied
+ to the buffer.
+@param[in,out] buf Compressed page and after suggesful
+ decompression operation uncompressed page
+ is copied here.
+@param[in] len Length of output buffer.
+@param[out] write_size Actual payload size of the compressed data.
+@return true when operation succeeded or false when failed */
UNIV_INTERN
-void
+bool
fil_decompress_page(
-/*================*/
- byte* page_buf, /*!< in: preallocated buffer or NULL */
- byte* buf, /*!< out: buffer from which to read; in aio
- this must be appropriately aligned */
- ulong len, /*!< in: length of output buffer.*/
- ulint* write_size, /*!< in/out: Actual payload size of
- the compressed data. */
- bool return_error) /*!< in: true if only an error should
- be produced when decompression fails.
- By default this parameter is false. */
+ byte* page_buf,
+ byte* buf,
+ ulong len,
+ ulint* write_size)
{
int err = 0;
ulint actual_size = 0;
@@ -503,7 +511,7 @@ fil_decompress_page(
if (ptype != FIL_PAGE_PAGE_COMPRESSED &&
ptype != FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED &&
ptype != FIL_PAGE_TYPE_COMPRESSED) {
- return;
+ ut_error;
}
// If no buffer was given, we need to allocate temporal buffer
@@ -514,24 +522,6 @@ fil_decompress_page(
in_buf = page_buf;
}
- /* Before actual decompress, make sure that page type is correct */
-
- if (mach_read_from_4(buf+FIL_PAGE_SPACE_OR_CHKSUM) != BUF_NO_CHECKSUM_MAGIC ||
- (ptype != FIL_PAGE_PAGE_COMPRESSED &&
- ptype != FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED)) {
- ib_logf(IB_LOG_LEVEL_ERROR,
- "Corruption: We try to uncompress corrupted page"
- " CRC " ULINTPF " type " ULINTPF " len " ULINTPF ".",
- mach_read_from_4(buf+FIL_PAGE_SPACE_OR_CHKSUM),
- mach_read_from_2(buf+FIL_PAGE_TYPE), len);
-
- fflush(stderr);
- if (return_error) {
- goto error_return;
- }
- ut_error;
- }
-
/* Get compression algorithm */
if (ptype == FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED) {
compression_alg = mach_read_from_2(buf+FIL_PAGE_DATA+FIL_PAGE_COMPRESSED_SIZE);
@@ -541,17 +531,10 @@ fil_decompress_page(
/* Get the actual size of compressed page */
actual_size = mach_read_from_2(buf+FIL_PAGE_DATA);
+
/* Check if payload size is corrupted */
if (actual_size == 0 || actual_size > UNIV_PAGE_SIZE) {
- ib_logf(IB_LOG_LEVEL_ERROR,
- "Corruption: We try to uncompress corrupted page"
- " actual size " ULINTPF " compression %s.",
- actual_size, fil_get_compression_alg_name(compression_alg));
- fflush(stderr);
- if (return_error) {
- goto error_return;
- }
- ut_error;
+ goto error_return;
}
/* Store actual payload size of the compressed data. This pointer
@@ -570,19 +553,7 @@ fil_decompress_page(
/* If uncompress fails it means that page is corrupted */
if (err != Z_OK) {
-
- ib_logf(IB_LOG_LEVEL_ERROR,
- "Corruption: Page is marked as compressed"
- " but uncompress failed with error %d "
- " size " ULINTPF " len " ULINTPF ".",
- err, actual_size, len);
-
- fflush(stderr);
-
- if (return_error) {
- goto error_return;
- }
- ut_error;
+ goto error_return;
}
break;
@@ -591,18 +562,7 @@ fil_decompress_page(
err = LZ4_decompress_fast((const char *)buf+header_len, (char *)in_buf, len);
if (err != (int)actual_size) {
- ib_logf(IB_LOG_LEVEL_ERROR,
- "Corruption: Page is marked as compressed"
- " but uncompress failed with error %d "
- " size " ULINTPF " len " ULINTPF ".",
- err, actual_size, len);
-
- fflush(stderr);
-
- if (return_error) {
- goto error_return;
- }
- ut_error;
+ goto error_return;
}
break;
#endif /* HAVE_LZ4 */
@@ -616,18 +576,7 @@ fil_decompress_page(
olen = olen_lzo;
if (err != LZO_E_OK || (olen == 0 || olen > UNIV_PAGE_SIZE)) {
- ib_logf(IB_LOG_LEVEL_ERROR,
- "Corruption: Page is marked as compressed"
- " but uncompress failed with error %d "
- " size " ULINTPF " len " ULINTPF ".",
- err, actual_size, len);
-
- fflush(stderr);
-
- if (return_error) {
- goto error_return;
- }
- ut_error;
+ goto error_return;
}
break;
}
@@ -653,17 +602,7 @@ fil_decompress_page(
if (ret != LZMA_OK || (dst_pos == 0 || dst_pos > UNIV_PAGE_SIZE)) {
- ib_logf(IB_LOG_LEVEL_ERROR,
- "Corruption: Page is marked as compressed"
- " but decompression read only %ld bytes"
- " size " ULINTPF "len " ULINTPF ".",
- dst_pos, actual_size, len);
- fflush(stderr);
-
- if (return_error) {
- goto error_return;
- }
- ut_error;
+ goto error_return;
}
break;
@@ -682,17 +621,8 @@ fil_decompress_page(
0);
if (err != BZ_OK || (dst_pos == 0 || dst_pos > UNIV_PAGE_SIZE)) {
- ib_logf(IB_LOG_LEVEL_ERROR,
- "Corruption: Page is marked as compressed"
- " but decompression read only %du bytes"
- " size " ULINTPF " len " ULINTPF " err %d.",
- dst_pos, actual_size, len, err);
- fflush(stderr);
-
- if (return_error) {
- goto error_return;
- }
- ut_error;
+ len = dst_pos;
+ goto error_return;
}
break;
}
@@ -709,35 +639,17 @@ fil_decompress_page(
(char *)in_buf,
(size_t*)&olen);
- if (cstatus != SNAPPY_OK || olen != UNIV_PAGE_SIZE) {
- ib_logf(IB_LOG_LEVEL_ERROR,
- "Corruption: Page is marked as compressed"
- " but decompression read only " ULINTPF " bytes"
- " size " ULINTPF " len " ULINTPF " err %d.",
- olen, actual_size, len, (int)cstatus);
- fflush(stderr);
-
- if (return_error) {
- goto error_return;
- }
- ut_error;
+ if (cstatus != SNAPPY_OK || (olen == 0 || olen > UNIV_PAGE_SIZE)) {
+ err = (int)cstatus;
+ len = olen;
+ goto error_return;
}
break;
}
#endif /* HAVE_SNAPPY */
default:
- ib_logf(IB_LOG_LEVEL_ERROR,
- "Corruption: Page is marked as compressed"
- " but compression algorithm %s"
- " is not known."
- ,fil_get_compression_alg_name(compression_alg));
-
- fflush(stderr);
- if (return_error) {
- goto error_return;
- }
- ut_error;
+ goto error_return;
break;
}
@@ -747,8 +659,108 @@ fil_decompress_page(
really any other options. */
memcpy(buf, in_buf, len);
+ if (page_buf != in_buf) {
+ ut_free(in_buf);
+ }
+ return true;
+
error_return:
if (page_buf != in_buf) {
ut_free(in_buf);
}
+
+ /* Note that as we have found the page is corrupted, so
+ all this could be incorrect. */
+ ulint space_id = mach_read_from_4(buf+FIL_PAGE_SPACE_ID);
+ fil_space_t* space = fil_space_acquire_for_io(space_id);
+
+ ib_logf(IB_LOG_LEVEL_ERROR,
+ "Corruption: Page is marked as compressed"
+ " space : " ULINTPF " name %s "
+ " but uncompress failed with error: %d"
+ " size: " ULINTPF
+ " len: " ULINTPF
+ " compression method %s",
+ space_id,
+ (space ? space->chain.start->name : "(import)"),
+ err,
+ actual_size,
+ len,
+ fil_get_compression_alg_name(compression_alg));
+
+ if (space) {
+ fil_space_release(space);
+ }
+
+ return false;
+}
+#endif /* !UNIV_INNOCHECKSUM */
+
+/**
+Verify that stored post compression checksum matches calculated
+checksum. Note that old format did not have a checksum and
+in that case either original pre-compression page checksum will
+fail after decompression or page decompression fails.
+
+@param[in,out] page page frame
+@param[in] space_id Tablespace identifier
+@param[in] offset Page offset
+@return true if post compression checksum matches, false otherwise */
+#ifndef UNIV_INNOCHECKSUM
+UNIV_INTERN
+#endif
+bool
+fil_verify_compression_checksum(
+ const byte* page,
+ ulint space_id,
+ ulint offset)
+{
+ ut_ad(page &&
+ (mach_read_from_2(page+FIL_PAGE_TYPE) == FIL_PAGE_PAGE_COMPRESSED
+ || mach_read_from_2(page+FIL_PAGE_TYPE) == FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED));
+
+ uint32_t checksum = mach_read_from_4(page+FIL_PAGE_SPACE_OR_CHKSUM);
+
+ /* In earlier versions supporting page compression, page compression
+ stored BUF_NO_CHECKSUM_MAGIC to checksum filed. These pages are
+ treaded as not corrupted. */
+ if (checksum == BUF_NO_CHECKSUM_MAGIC) {
+ return (true);
+ }
+
+ uint32_t cchecksum = buf_calc_compressed_crc32(page);
+
+ if (checksum != cchecksum) {
+ ulint comp_method = mach_read_from_2(page+FIL_PAGE_TYPE) == FIL_PAGE_PAGE_COMPRESSED
+ ? mach_read_from_8(page+FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION)
+ : mach_read_from_2(page+FIL_PAGE_DATA+FIL_PAGE_COMPRESSED_SIZE);
+
+#ifdef UNIV_INNOCHECKSUM
+ fprintf(log_file ? log_file : stderr,
+ "Page " ULINTPF ":" ULINTPF " may be corrupted."
+ " Post compression checksum %u"
+ " stored %u compression_method %s\n",
+ space_id, offset, checksum, cchecksum,
+ fil_get_compression_alg_name(comp_method));
+#else /*! UNIV_INNOCHECKSUM */
+ FilSpace space(space_id, true);
+
+ ib_logf(IB_LOG_LEVEL_ERROR,
+ "Page [page id: space=" ULINTPF
+ ", page number= " ULINTPF "]"
+ " in file %s"
+ " may be corrupted."
+ " Post compression checksum %u"
+ " stored %u"
+ " compression_method %s",
+ space_id,
+ offset,
+ (space() ? space()->chain.start->name : "(import)"),
+ checksum,
+ cchecksum,
+ fil_get_compression_alg_name(comp_method));
+#endif
+ }
+
+ return (checksum == cchecksum);
}
diff --git a/storage/xtradb/handler/ha_innodb.cc b/storage/xtradb/handler/ha_innodb.cc
index 9749bcecfe9..c467adc635f 100644
--- a/storage/xtradb/handler/ha_innodb.cc
+++ b/storage/xtradb/handler/ha_innodb.cc
@@ -2329,6 +2329,7 @@ convert_error_code_to_mysql(
code should be introduced */
case DB_CORRUPTION:
+ case DB_PAGE_CORRUPTED:
return(HA_ERR_CRASHED);
case DB_OUT_OF_FILE_SPACE:
@@ -6600,6 +6601,8 @@ table_opened:
buf, space()->chain.start->name);
ret_err = HA_ERR_DECRYPTION_FAILED;
}
+ } else if (ib_table->corrupted) {
+ ret_err = HA_ERR_CRASHED;
}
dict_table_close(ib_table, FALSE, FALSE);
diff --git a/storage/xtradb/include/buf0checksum.h b/storage/xtradb/include/buf0checksum.h
index 6818345f965..7834ad4d292 100644
--- a/storage/xtradb/include/buf0checksum.h
+++ b/storage/xtradb/include/buf0checksum.h
@@ -1,6 +1,7 @@
/*****************************************************************************
Copyright (c) 1995, 2011, Oracle and/or its affiliates. All Rights Reserved.
+Copyright (c) 2017, 2018, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@@ -80,4 +81,16 @@ buf_checksum_algorithm_name(
extern ulong srv_checksum_algorithm;
+/** Calculates the CRC32 checksum of a page compressed page. The value is
+stored to the page when it is written to a file and also checked for
+a match when reading from the file. Checksum is calculated from
+actual payload of the compressed page and some header fields.
+
+@param[in] page buffer page (UNIV_PAGE_SIZE bytes)
+@return checksum */
+UNIV_INTERN
+uint32_t
+buf_calc_compressed_crc32(
+ const byte* page);
+
#endif /* buf0checksum_h */
diff --git a/storage/xtradb/include/fil0pagecompress.h b/storage/xtradb/include/fil0pagecompress.h
index 03e16699ce3..bd1395b4835 100644
--- a/storage/xtradb/include/fil0pagecompress.h
+++ b/storage/xtradb/include/fil0pagecompress.h
@@ -1,6 +1,6 @@
/*****************************************************************************
-Copyright (C) 2013, 2017 MariaDB Corporation. All Rights Reserved.
+Copyright (C) 2013, 2018 MariaDB Corporation. All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@@ -19,6 +19,8 @@ this program; if not, write to the Free Software Foundation, Inc.,
#ifndef fil0pagecompress_h
#define fil0pagecompress_h
+#ifndef UNIV_INNOCHECKSUM
+
#include "fsp0fsp.h"
#include "fsp0pagecompress.h"
@@ -68,23 +70,26 @@ fil_compress_page(
ulint* out_len); /*!< out: actual length of compressed
page */
-/****************************************************************//**
+/**
For page compressed pages decompress the page after actual read
-operation. */
+operation.
+@param[in,out] page_buf Preallocated temporal buffer where
+ compression is done and then copied
+ to the buffer.
+@param[in,out] buf Compressed page and after suggesful
+ decompression operation uncompressed page
+ is copied here.
+@param[in] len Length of output buffer.
+@param[out] write_size Actual payload size of the compressed data.
+@return true when operation succeeded or false when failed */
UNIV_INTERN
-void
+bool
fil_decompress_page(
-/*================*/
- byte* page_buf, /*!< in: preallocated buffer or NULL */
- byte* buf, /*!< out: buffer from which to read; in aio
- this must be appropriately aligned */
- ulong len, /*!< in: length of output buffer.*/
- ulint* write_size, /*!< in/out: Actual payload size of
- the compressed data. */
- bool return_error=false);
- /*!< in: true if only an error should
- be produced when decompression fails.
- By default this parameter is false. */
+ byte* page_buf,
+ byte* buf,
+ ulong len,
+ ulint* write_size)
+ MY_ATTRIBUTE((warn_unused_result));
/****************************************************************//**
Get space id from fil node
@@ -129,4 +134,25 @@ ibool
fil_page_is_lzo_compressed(
/*=======================*/
byte* buf); /*!< in: page */
-#endif
+#endif /* !UNIV_INNOCHECKSUM */
+
+/**
+Verify that stored post compression checksum matches calculated
+checksum. Note that old format did not have a checksum and
+in that case either original pre-compression page checksum will
+fail after decompression or page decompression fails.
+
+@param[in,out] page page frame
+@param[in] space_id Tablespace identifier
+@param[in] offset Page offset
+@return true if post compression checksum matches, false otherwise */
+UNIV_INTERN
+bool
+fil_verify_compression_checksum(
+ const byte* page,
+ ulint space_id,
+ ulint offset)
+ MY_ATTRIBUTE((warn_unused_result));
+
+#endif /* fil0pagecompress_h */
+
diff --git a/storage/xtradb/include/fsp0pagecompress.h b/storage/xtradb/include/fsp0pagecompress.h
index c623d11c326..00645616556 100644
--- a/storage/xtradb/include/fsp0pagecompress.h
+++ b/storage/xtradb/include/fsp0pagecompress.h
@@ -1,6 +1,6 @@
/*****************************************************************************
-Copyright (C) 2013, 2017, MariaDB Corporation. All Rights Reserved.
+Copyright (C) 2013, 2018, MariaDB Corporation. All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@@ -38,6 +38,7 @@ Created 11/12/2013 Jan Lindström jan.lindstrom@skysql.com
#define PAGE_SNAPPY_ALGORITHM 6
#define PAGE_ALGORITHM_LAST PAGE_SNAPPY_ALGORITHM
+#ifndef UNIV_INNOCHECKSUM
/**********************************************************************//**
Reads the page compression level from the first page of a tablespace.
@return page compression level, or 0 if uncompressed */
@@ -67,6 +68,7 @@ atomic_writes_t
fsp_flags_get_atomic_writes(
/*========================*/
ulint flags); /*!< in: tablespace flags */
+#endif /* !UNIV_INNOCHECKSUM */
#ifndef UNIV_NONINL
#include "fsp0pagecompress.ic"
diff --git a/storage/xtradb/include/fsp0pagecompress.ic b/storage/xtradb/include/fsp0pagecompress.ic
index 14f968e319e..0e9a574eef6 100644
--- a/storage/xtradb/include/fsp0pagecompress.ic
+++ b/storage/xtradb/include/fsp0pagecompress.ic
@@ -25,6 +25,7 @@ Created 11/12/2013 Jan Lindström jan.lindstrom@mariadb.com
***********************************************************************/
+#ifndef UNIV_INNOCHECKSUM
/********************************************************************//**
Determine the tablespace is page compression level from dict_table_t::flags.
@return page compression level or 0 if not compressed*/
@@ -84,6 +85,7 @@ fil_page_is_compressed_encrypted(
{
return(mach_read_from_2(buf+FIL_PAGE_TYPE) == FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED);
}
+#endif /*!UNIV_INNOCHECKSUM */
/****************************************************************//**
Get the name of the compression algorithm used for page