MDEV-8410: Changing file-key-management to example-key-management causes crash and no real error

MDEV-8409: Changing file-key-management-encryption-algorithm causes crash and no real info why

Analysis: Both bugs has two different error cases. Firstly, at startup
when server reads latest checkpoint but requested key_version,
key management plugin or encryption algorithm or method is not found
leading corrupted log entry. Secondly, similarly when reading system
tablespace if requested key_version, key management plugin or encryption
algorithm or method is not found leading buffer pool page corruption.

Fix: Firsly, when reading checkpoint at startup check if the log record
may be encrypted and if we find that it could be encrypted, print error
message and do not start server. Secondly, if page is buffer pool seems
corrupted but we find out that there is crypt_info, print additional
error message before asserting.

1 files changed, 61 insertions, 9 deletions

diff --git a/storage/innobase/include/log0crypt.h b/storage/innobase/include/log0crypt.h
index b04f16d2a29..7e737853465 100644
--- a/storage/innobase/include/log0crypt.h
+++ b/storage/innobase/include/log0crypt.h
@@ -1,8 +1,27 @@
+/*****************************************************************************
+
+Copyright (C) 2013, 2015, Google Inc. All Rights Reserved.
+Copyright (C) 2014, 2015, MariaDB Corporation. All Rights Reserved.
+
+This program is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free Software
+Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful, but WITHOUT
+ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License along with
+this program; if not, write to the Free Software Foundation, Inc.,
+51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+
+*****************************************************************************/
 /**************************************************//**
 @file include/log0crypt.h
 Innodb log encrypt/decrypt
 
 Created 11/25/2013 Minli Zhu
+Modified           Jan Lindström jan.lindstrom@mariadb.com
 *******************************************************/
 #ifndef log0crypt_h
 #define log0crypt_h
@@ -22,7 +41,7 @@ UNIV_INTERN
 void
 log_crypt_set_ver_and_key(
 /*======================*/
-	ib_uint64_t next_checkpoint_no);
+	ib_uint64_t next_checkpoint_no);/*!< in: next checkpoint no */
 
 
 /*********************************************************************//**
@@ -43,17 +62,17 @@ UNIV_INTERN
 bool
 log_crypt_read_checkpoint_buf(
 /*===========================*/
-	const byte*	buf);			/*!< in: checkpoint buffer */
+	const byte*	buf);		/*!< in: checkpoint buffer */
 
 /********************************************************
 Encrypt one or more log block before it is flushed to disk */
 UNIV_INTERN
 void
 log_encrypt_before_write(
-/*===========================*/
-	ib_uint64_t next_checkpoint_no,	/*!< in: log group to be flushed */
-	byte* block,			/*!< in/out: pointer to a log block */
-	const ulint size);		/*!< in: size of log blocks */
+/*=====================*/
+	ib_uint64_t	next_checkpoint_no,	/*!< in: log group to be flushed */
+	byte* 		block,			/*!< in/out: pointer to a log block */
+	const ulint 	size);			/*!< in: size of log blocks */
 
 /********************************************************
 Decrypt a specified log segment after they are read from a log file to a buffer.
@@ -61,8 +80,41 @@ Decrypt a specified log segment after they are read from a log file to a buffer.
 UNIV_INTERN
 void
 log_decrypt_after_read(
-/*==========================*/
-	byte* frame,	/*!< in/out: log segment */
-	const ulint size);	/*!< in: log segment size */
+/*===================*/
+	byte*		frame,	/*!< in/out: log segment */
+	const ulint	size);	/*!< in: log segment size */
+
+/* Error codes for crypt info */
+typedef enum {
+	LOG_UNENCRYPTED = 0,
+	LOG_CRYPT_KEY_NOT_FOUND = 1,
+	LOG_DECRYPT_MAYBE_FAILED = 2
+} log_crypt_err_t;
+
+/********************************************************
+Check is the checkpoint information encrypted. This check
+is based on fact has log group crypt info and based
+on this crypt info was the key version different from
+unencrypted key version. There is no realible way to
+distinguish encrypted log block from corrupted log block,
+but if log block corruption is found this function is
+used to find out if log block is maybe encrypted but
+encryption key, key management plugin or encryption
+algorithm does not match.
+@return TRUE, if log block may be encrypted */
+UNIV_INTERN
+ibool
+log_crypt_block_maybe_encrypted(
+/*============================*/
+	const byte* 		log_block, 	/*!< in: log block */
+	log_crypt_err_t* 	err_info); 	/*!< out: error info */
+
+/********************************************************
+Print crypt error message to error log */
+UNIV_INTERN
+void
+log_crypt_print_error(
+/*==================*/
+	log_crypt_err_t 	err_info); 	/*!< out: error info */
 
 #endif  // log0crypt.h