MDEV-11759: Encryption code in MariaDB 10.1/10.2 causes

compatibility problems

Pages that are encrypted contain post encryption checksum on
different location that normal checksum fields. Therefore,
we should before decryption check this checksum to avoid
unencrypting corrupted pages. After decryption we can use
traditional checksum check to detect if page is corrupted
or unencryption was done using incorrect key.

Pages that are page compressed do not contain any checksum,
here we need to fist unencrypt, decompress and finally
use tradional checksum check to detect page corruption
or that we used incorrect key in unencryption.

buf0buf.cc: buf_page_is_corrupted() mofified so that
compressed pages are skipped.

buf0buf.h, buf_block_init(), buf_page_init_low():
removed unnecessary page_encrypted, page_compressed,
stored_checksum, valculated_checksum fields from
buf_page_t

buf_page_get_gen(): use new buf_page_check_corrupt() function
to detect corrupted pages.

buf_page_check_corrupt(): If page was not yet decrypted
check if post encryption checksum still matches.
If page is not anymore encrypted, use buf_page_is_corrupted()
traditional checksum method.

If page is detected as corrupted and it is not encrypted
we print corruption message to error log.
If page is still encrypted or it was encrypted and now
corrupted, we will print message that page is
encrypted to error log.

buf_page_io_complete(): use new buf_page_check_corrupt()
function to detect corrupted pages.

buf_page_decrypt_after_read(): Verify post encryption
checksum before tring to decrypt.

fil0crypt.cc: fil_encrypt_buf() verify post encryption
checksum and ind fil_space_decrypt() return true
if we really decrypted the page.

fil_space_verify_crypt_checksum(): rewrite to use
the method used when calculating post encryption
checksum. We also check if post encryption checksum
matches that traditional checksum check does not
match.

fil0fil.ic: Add missed page type encrypted and page
compressed to fil_get_page_type_name()

Note that this change does not yet fix innochecksum tool,
that will be done in separate MDEV.

Fix test failures caused by buf page corruption injection.

4 files changed, 10 insertions, 0 deletions

diff --git a/mysql-test/suite/innodb/r/innodb-wl5522-debug-zip.result b/mysql-test/suite/innodb/r/innodb-wl5522-debug-zip.result
index 58721a8732a..05ecfe9f8ef 100644
--- a/mysql-test/suite/innodb/r/innodb-wl5522-debug-zip.result
+++ b/mysql-test/suite/innodb/r/innodb-wl5522-debug-zip.result
@@ -1,5 +1,7 @@
 call mtr.add_suppression("InnoDB: Page for tablespace ");
 call mtr.add_suppression("InnoDB: Invalid FSP_SPACE_FLAGS=0x");
+call mtr.add_suppression("InnoDB: Corruption: Block in space_id .* in file .* corrupted");
+call mtr.add_suppression("InnoDB: Based on page type .*");
 FLUSH TABLES;
 SET GLOBAL innodb_file_per_table = 1;
 SELECT @@innodb_file_per_table;
diff --git a/mysql-test/suite/innodb/r/innodb_bug14147491.result b/mysql-test/suite/innodb/r/innodb_bug14147491.result
index 5ff6f652f9e..1a8249f9e54 100644
--- a/mysql-test/suite/innodb/r/innodb_bug14147491.result
+++ b/mysql-test/suite/innodb/r/innodb_bug14147491.result
@@ -1,4 +1,6 @@
 call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed.*");
+call mtr.add_suppression("InnoDB: Corruption: Block in space_id .* in file .* corrupted.");
+call mtr.add_suppression("InnoDB: Based on page type .*");
 CALL mtr.add_suppression("InnoDB: Error: Unable to read tablespace .* page no .* into the buffer pool after 100 attempts");
 CALL mtr.add_suppression("InnoDB: Database page corruption on disk or a failed");
 CALL mtr.add_suppression("InnoDB: Space .* file test/t1 read of page .*");
@@ -10,6 +12,7 @@ CALL mtr.add_suppression("InnoDB: fix the corruption by dumping, dropping, and r
 CALL mtr.add_suppression("InnoDB: the corrupt table. You can use CHECK");
 CALL mtr.add_suppression("InnoDB: TABLE to scan your table for corruption.");
 CALL mtr.add_suppression("InnoDB: See also .* about forcing recovery.");
+flush tables;
 # Create and populate the table to be corrupted
 CREATE TABLE t1 (a INT AUTO_INCREMENT PRIMARY KEY, b TEXT) ENGINE=InnoDB;
 INSERT INTO t1 (b) VALUES ('corrupt me');
diff --git a/mysql-test/suite/innodb/t/innodb-wl5522-debug-zip.test b/mysql-test/suite/innodb/t/innodb-wl5522-debug-zip.test
index 0bfe67c06bc..f5e220ad4aa 100644
--- a/mysql-test/suite/innodb/t/innodb-wl5522-debug-zip.test
+++ b/mysql-test/suite/innodb/t/innodb-wl5522-debug-zip.test
@@ -19,6 +19,8 @@
 
 call mtr.add_suppression("InnoDB: Page for tablespace ");
 call mtr.add_suppression("InnoDB: Invalid FSP_SPACE_FLAGS=0x");
+call mtr.add_suppression("InnoDB: Corruption: Block in space_id .* in file .* corrupted");
+call mtr.add_suppression("InnoDB: Based on page type .*");
 FLUSH TABLES;
 
 let MYSQLD_DATADIR =`SELECT @@datadir`;
diff --git a/mysql-test/suite/innodb/t/innodb_bug14147491.test b/mysql-test/suite/innodb/t/innodb_bug14147491.test
index d48fa074147..853c7e63f35 100644
--- a/mysql-test/suite/innodb/t/innodb_bug14147491.test
+++ b/mysql-test/suite/innodb/t/innodb_bug14147491.test
@@ -5,6 +5,8 @@
 -- source include/not_encrypted.inc
 
 call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed.*");
+call mtr.add_suppression("InnoDB: Corruption: Block in space_id .* in file .* corrupted.");
+call mtr.add_suppression("InnoDB: Based on page type .*");
 
 # Don't test under valgrind, memory leaks will occur
 source include/not_valgrind.inc;
@@ -31,6 +33,7 @@ CALL mtr.add_suppression("InnoDB: fix the corruption by dumping, dropping, and r
 CALL mtr.add_suppression("InnoDB: the corrupt table. You can use CHECK");
 CALL mtr.add_suppression("InnoDB: TABLE to scan your table for corruption.");
 CALL mtr.add_suppression("InnoDB: See also .* about forcing recovery.");
+flush tables;
 
 
 --echo # Create and populate the table to be corrupted