MDEV-25105 Remove innodb_checksum_algorithm values none,innodb,...

Historically, InnoDB supported a buggy page checksum algorithm that did not
compute a checksum over the full page. Later, well before MySQL 4.1
introduced .ibd files and the innodb_file_per_table option, the algorithm
was corrected and the first 4 bytes of each page were redefined to be
a checksum.

The original checksum was so slow that an option to disable page checksum
was introduced for benchmarketing purposes.

The Intel Nehalem microarchitecture introduced the SSE4.2 instruction set
extension, which includes instructions for faster computation of CRC-32C.
In MySQL 5.6 (and MariaDB 10.0), innodb_checksum_algorithm=crc32 was
implemented to make of that. As that option was changed to be the default
in MySQL 5.7, a bug was found on big-endian platforms and some work-around
code was added to weaken that checksum further. MariaDB disables that
work-around by default since MDEV-17958.

Later, SIMD-accelerated CRC-32C has been implemented in MariaDB for POWER
and ARM and also for IA-32/AMD64, making use of carry-less multiplication
where available.

Long story short, innodb_checksum_algorithm=crc32 is faster and more secure
than the pre-MySQL 5.6 checksum, called innodb_checksum_algorithm=innodb.
It should have removed any need to use innodb_checksum_algorithm=none.

The setting innodb_checksum_algorithm=crc32 is the default in
MySQL 5.7 and MariaDB Server 10.2, 10.3, 10.4. In MariaDB 10.5,
MDEV-19534 made innodb_checksum_algorithm=full_crc32 the default.
It is even faster and more secure.

The default settings in MariaDB do allow old data files to be read,
no matter if a worse checksum algorithm had been used.
(Unfortunately, before innodb_checksum_algorithm=full_crc32,
the data files did not identify which checksum algorithm is being used.)

The non-default settings innodb_checksum_algorithm=strict_crc32 or
innodb_checksum_algorithm=strict_full_crc32 would only allow CRC-32C
checksums. The incompatibility with old data files is why they are
not the default.

The newest server not to support innodb_checksum_algorithm=crc32
were MySQL 5.5 and MariaDB 5.5. Both have reached their end of life.
A valid reason for using innodb_checksum_algorithm=innodb could have
been the ability to downgrade. If it is really needed, data files
can be converted with an older version of the innochecksum utility.

Because there is no good reason to allow data files to be written
with insecure checksums, we will reject those option values:

    innodb_checksum_algorithm=none
    innodb_checksum_algorithm=innodb
    innodb_checksum_algorithm=strict_none
    innodb_checksum_algorithm=strict_innodb

Furthermore, the following innochecksum options will be removed,
because only strict crc32 will be supported:

    innochecksum --strict-check=crc32
    innochecksum -C crc32
    innochecksum --write=crc32
    innochecksum -w crc32

If a user wishes to convert a data file to use a different checksum
(so that it might be used with the no-longer-supported
MySQL 5.5 or MariaDB 5.5, which do not support IMPORT TABLESPACE
nor system tablespace format changes that were made in MariaDB 10.3),
then the innochecksum tool from MariaDB 10.2, 10.3, 10.4, 10.5 or
MySQL 5.7 can be used.

Reviewed by: Thirunarayanan Balathandayuthapani

2 files changed, 8 insertions, 114 deletions

diff --git a/extra/innochecksum.cc b/extra/innochecksum.cc
index 833e66341c7..02aec1bd919 100644
--- a/extra/innochecksum.cc
+++ b/extra/innochecksum.cc
@@ -1,6 +1,6 @@
 /*
    Copyright (c) 2005, 2016, Oracle and/or its affiliates. All rights reserved.
-   Copyright (c) 2014, 2020, MariaDB Corporation.
+   Copyright (c) 2014, 2021, MariaDB Corporation.
 
    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
@@ -68,7 +68,6 @@ static bool			do_one_page;
 static my_bool do_leaf;
 static my_bool per_page_details;
 static ulint n_merge;
-extern ulong			srv_checksum_algorithm;
 static ulint physical_page_size;  /* Page size in bytes on disk. */
 ulong srv_page_size;
 ulong srv_page_size_shift;
@@ -78,8 +77,6 @@ unsigned long long		cur_page_num;
 unsigned long long		cur_space;
 /* Skip the checksum verification. */
 static bool			no_check;
-/* Enabled for strict checksum verification. */
-bool				strict_verify = 0;
 /* Enabled for rewrite checksum. */
 static bool			do_write;
 /* Mismatches count allowed (0 by default). */
@@ -102,11 +99,6 @@ static bool			is_log_enabled = false;
 struct flock			lk;
 #endif /* _WIN32 */
 
-/* Strict check algorithm name. */
-static ulong			strict_check;
-/* Rewrite checksum algorithm name. */
-static ulong			write_check;
-
 /* Innodb page type. */
 struct innodb_page_type {
 	int n_undo_state_active;
@@ -136,24 +128,6 @@ struct innodb_page_type {
 	int n_fil_page_type_page_compressed_encrypted;
 } page_type;
 
-/* Possible values for "--strict-check" for strictly verify checksum
-and "--write" for rewrite checksum. */
-static const char *innochecksum_algorithms[] = {
-	"crc32",
-	"crc32",
-	"innodb",
-	"innodb",
-	"none",
-	"none",
-	NullS
-};
-
-/* Used to define an enumerate type of the "innochecksum algorithm". */
-static TYPELIB innochecksum_algorithms_typelib = {
-	array_elements(innochecksum_algorithms)-1,"",
-	innochecksum_algorithms, NULL
-};
-
 #define SIZE_RANGES_FOR_PAGE 10
 #define NUM_RETRIES 3
 #define DEFAULT_RETRY_DELAY 1000000
@@ -646,10 +620,9 @@ static bool update_checksum(byte* page, ulint flags)
 	}
 
 	if (iscompressed) {
-		/* page is compressed */
-		checksum = page_zip_calc_checksum(
-			page, physical_page_size,
-			static_cast<srv_checksum_algorithm_t>(write_check));
+		/* ROW_FORMAT=COMPRESSED */
+		checksum = page_zip_calc_checksum(page, physical_page_size,
+						  false);
 
 		mach_write_to_4(page + FIL_PAGE_SPACE_OR_CHKSUM, checksum);
 		if (is_log_enabled) {
@@ -673,50 +646,16 @@ static bool update_checksum(byte* page, ulint flags)
 		/* page is uncompressed. */
 
 		/* Store the new formula checksum */
-		switch ((srv_checksum_algorithm_t) write_check) {
-
-		case SRV_CHECKSUM_ALGORITHM_FULL_CRC32:
-		case SRV_CHECKSUM_ALGORITHM_STRICT_FULL_CRC32:
-		case SRV_CHECKSUM_ALGORITHM_CRC32:
-		case SRV_CHECKSUM_ALGORITHM_STRICT_CRC32:
-			checksum = buf_calc_page_crc32(page);
-			break;
-
-		case SRV_CHECKSUM_ALGORITHM_INNODB:
-		case SRV_CHECKSUM_ALGORITHM_STRICT_INNODB:
-			checksum = (ib_uint32_t)
-					buf_calc_page_new_checksum(page);
-			break;
-
-		case SRV_CHECKSUM_ALGORITHM_NONE:
-		case SRV_CHECKSUM_ALGORITHM_STRICT_NONE:
-			checksum = BUF_NO_CHECKSUM_MAGIC;
-			break;
-
-		/* no default so the compiler will emit a warning if new
-		enum is added and not handled here */
-		}
+		checksum = buf_calc_page_crc32(page);
 
 		mach_write_to_4(page + FIL_PAGE_SPACE_OR_CHKSUM, checksum);
 		if (is_log_enabled) {
-			fprintf(log_file, "page::%llu; Updated checksum field1"
+			fprintf(log_file, "page::%llu; Updated checksum"
 				" = %u\n", cur_page_num, checksum);
 		}
 
-		if (write_check == SRV_CHECKSUM_ALGORITHM_STRICT_INNODB
-		    || write_check == SRV_CHECKSUM_ALGORITHM_INNODB) {
-			checksum = (ib_uint32_t)
-					buf_calc_page_old_checksum(page);
-		}
-
 		mach_write_to_4(page + physical_page_size -
 				FIL_PAGE_END_LSN_OLD_CHKSUM,checksum);
-
-		if (is_log_enabled) {
-			fprintf(log_file, "page::%llu; Updated checksum "
-				"field2 = %u\n", cur_page_num, checksum);
-		}
-
 	}
 
 func_exit:
@@ -1270,17 +1209,11 @@ static struct my_option innochecksum_options[] = {
   {"page", 'p', "Check only this page (0 based).",
     &do_page, &do_page, 0, GET_ULL, REQUIRED_ARG,
     0, 0, ULLONG_MAX, 0, 1, 0},
-  {"strict-check", 'C', "Specify the strict checksum algorithm by the user.",
-    &strict_check, &strict_check, &innochecksum_algorithms_typelib,
-    GET_ENUM, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
   {"no-check", 'n', "Ignore the checksum verification.",
     &no_check, &no_check, 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
   {"allow-mismatches", 'a', "Maximum checksum mismatch allowed.",
     &allow_mismatches, &allow_mismatches, 0,
     GET_ULL, REQUIRED_ARG, 0, 0, ULLONG_MAX, 0, 1, 0},
-  {"write", 'w', "Rewrite the checksum algorithm by the user.",
-    &write_check, &write_check, &innochecksum_algorithms_typelib,
-    GET_ENUM, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
   {"page-type-summary", 'S', "Display a count of each page type "
    "in a tablespace.", &page_type_summary, &page_type_summary, 0,
    GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
@@ -1321,7 +1254,7 @@ static void usage(void)
 	printf("InnoDB offline file checksum utility.\n");
 	printf("Usage: %s [-c] [-s <start page>] [-e <end page>] "
 		"[-p <page>] [-i] [-v]  [-a <allow mismatches>] [-n] "
-		"[-C <strict-check>] [-w <write>] [-S] [-D <page type dump>] "
+		"[-S] [-D <page type dump>] "
 		"[-l <log>] [-l] [-m <merge pages>] <filename or [-]>\n", my_progname);
 	printf("See https://mariadb.com/kb/en/library/innochecksum/"
 	       " for usage hints.\n");
@@ -1358,38 +1291,6 @@ innochecksum_get_one_option(
 		my_end(0);
 		exit(EXIT_SUCCESS);
 		break;
-	case 'C':
-		strict_verify = true;
-		switch ((srv_checksum_algorithm_t) strict_check) {
-
-		case SRV_CHECKSUM_ALGORITHM_STRICT_CRC32:
-		case SRV_CHECKSUM_ALGORITHM_CRC32:
-			srv_checksum_algorithm =
-				SRV_CHECKSUM_ALGORITHM_STRICT_CRC32;
-			break;
-
-		case SRV_CHECKSUM_ALGORITHM_STRICT_INNODB:
-		case SRV_CHECKSUM_ALGORITHM_INNODB:
-			srv_checksum_algorithm =
-				SRV_CHECKSUM_ALGORITHM_STRICT_INNODB;
-			break;
-
-		case SRV_CHECKSUM_ALGORITHM_STRICT_NONE:
-		case SRV_CHECKSUM_ALGORITHM_NONE:
-			srv_checksum_algorithm =
-				SRV_CHECKSUM_ALGORITHM_STRICT_NONE;
-			break;
-
-		case SRV_CHECKSUM_ALGORITHM_STRICT_FULL_CRC32:
-		case SRV_CHECKSUM_ALGORITHM_FULL_CRC32:
-			srv_checksum_algorithm =
-				SRV_CHECKSUM_ALGORITHM_STRICT_FULL_CRC32;
-			break;
-
-		default:
-			return(true);
-		}
-		break;
 	case 'n':
 		no_check = true;
 		break;
@@ -1592,13 +1493,6 @@ int main(
 		goto my_exit;
 	}
 
-	if (strict_verify && no_check) {
-		fprintf(stderr, "Error: --strict-check option cannot be used "
-			"together with --no-check option.\n");
-		exit_status = 1;
-		goto my_exit;
-	}
-
 	if (no_check && !do_write) {
 		fprintf(stderr, "Error: --no-check must be associated with "
 			"--write option.\n");
diff --git a/extra/mariabackup/xtrabackup.cc b/extra/mariabackup/xtrabackup.cc
index 56ce2a58eff..31add72d831 100644
--- a/extra/mariabackup/xtrabackup.cc
+++ b/extra/mariabackup/xtrabackup.cc
@@ -1556,7 +1556,7 @@ struct my_option xb_server_options[] =
 
   {"innodb_checksum_algorithm", OPT_INNODB_CHECKSUM_ALGORITHM,
   "The algorithm InnoDB uses for page checksumming. [CRC32, STRICT_CRC32, "
-   "INNODB, STRICT_INNODB, NONE, STRICT_NONE]", &srv_checksum_algorithm,
+   "FULL_CRC32, STRICT_FULL_CRC32]", &srv_checksum_algorithm,
    &srv_checksum_algorithm, &innodb_checksum_algorithm_typelib, GET_ENUM,
    REQUIRED_ARG, SRV_CHECKSUM_ALGORITHM_CRC32, 0, 0, 0, 0, 0},