MDEV-27180 Fully atomic partitioning DDL operationsbb-10.7-midenok-MDEV-27180

Atomic DDL for partitioning originally covers crash-safety but it does
not recover fully from failures. F.ex. if error happened during the
rename or drop of partitions the ALTER operation does not return the
table to its original state.

The patch solves the above problem similarly to MDEV-25292 (atomic
CREATE OR REPLACE): new partitions are created as temporary, old
partitions are backed up, everything is guarded by two DDL log chains:
rollback and cleanup. Rollback chain contains the actions that bring
back the original table. Cleanup chain deletes the backup files.

The generic operation of alter partition is as follows:

  1. Create new partitions as tmp partitions;
  2. Fill TMP partitions with data;
  3. Rename old and not needed partitions to backup partitions;
  4. Rename tmp partitions to original partititions;
  5. Do any required logging (mariabackup, binary);
  6. If everything is ok, drop the backup partitions;
  7. In the case any of pp.1-5 fails drop new partitions, rename back
     backup partitions.

Each rename operation is executed right after the corresponding DDL
logging. Originally they were done in different loops:
write_log_changed_partitions(), mysql_change_partitions(), etc. The
actual table operations were done in ha_partition handler:
ha_change_partitions(), ha_drop_partitions(),
ha_rename_partitions(). Now these calls are deprecated and removed.

Instead the above deprecated interfaces the following classes now
handle DDL logging and table operations:

      Alter_partition_logger
      Alter_partition_add
      Alter_partition_change

Alter_partition_logger handles the basic operation, DDL logging and
table renames. Alter_partition_add does the stuff for adding the
partitions. Alter_partition_change combines Alter_partition_logger and
Alter_partition_add for complex operations such as REORGANIZE.

ha_partition::change_partitions() call was not fully removed. Stages
1-4 from that call was renamed to allocate_partitions() and that is
used by Alter_partition_add for initializing file handler arrays
(m_added_file, m_reorged_file). ha_partition::prepare_new_partition()
is renamed to create_partition() because it actually does ha_create()
and the new name better describes what it does.

DDL_LOG_DELETE/DDL_LOG_RENAME/DDL_LOG_REPLACE are now pure file
operations. To process both .par and .frm files we now issue two
actions instead of one. That makes interface more simple and that will
be used by MDEV-16417 in future. All the table operations are done via
other actions, such as DDL_LOG_RENAME_TABLE, DDL_LOG_DROP_TABLE, etc.

DDL_LOG_RENAME_TABLE has new alter_partition option. With this option
it does simple ha_rename_table() which is needed for renaming the
partitions. The action for renaming the partitions must start from
phase DDL_RENAME_PHASE_TABLE.

The testing was refactored a bit. Many new fail/crash points was
added, the test was powered by binary and mariabackup logging, it
tests more ALTER commands with variations of partitions-only and
subpartitions, without and under LOCK TABLES.

EXCHANGE PARTITION was fixed: after failure under LOCK TABLES the
table became unlocked. Fixed by correcting place of
reopen_tables(). Tested by partition_debug.

1 files changed, 93 insertions, 1 deletions

diff --git a/mysql-test/suite/parts/inc/partition_fail.inc b/mysql-test/suite/parts/inc/partition_fail.inc
index bbe5924e358..0c6a42a8292 100644
--- a/mysql-test/suite/parts/inc/partition_fail.inc
+++ b/mysql-test/suite/parts/inc/partition_fail.inc
@@ -1,5 +1,22 @@
 # Include file to decrease test code duplication
 
+let $log_bin= `select @@global.log_bin`;
+if ($log_bin)
+{
+  RESET MASTER;
+  let $keep_include_silent= 1;
+  let $grep_script= ALTER;
+  let $binlog_file=master-bin.000001;
+}
+
+if ($create_statement2)
+{
+  --eval $create_statement2
+}
+if ($insert_statement2)
+{
+  --eval $insert_statement2
+}
 --eval $create_statement
 --eval $insert_statement
 --let $dbug_flag= `select @@session.debug_dbug`
@@ -11,16 +28,64 @@ SHOW CREATE TABLE t1;
 --sorted_result
 SELECT * FROM t1;
 --disable_abort_on_error
+
+--disable_query_log
+connect (con_backup,localhost,root,,);
+BACKUP STAGE START;
+connection default;
+--enable_query_log
+
 --eval $fail_statement
 --enable_abort_on_error
 --echo # $dbug_flag: AFTER failure
+show warnings;
 --replace_result #p# #P# #sp# #SP#
 --list_files $DATADIR/test
 SHOW CREATE TABLE t1;
 --sorted_result
 SELECT * FROM t1;
+if ($show_statement)
+{
+  --error 0,ER_NO_SUCH_TABLE
+  --eval $show_statement
+}
+if ($select_statement)
+{
+  --sorted_result
+  --error 0,ER_NO_SUCH_TABLE
+  --eval $select_statement
+}
+
+--source include/print_ddl_log.inc
+--disable_query_log
+--connection con_backup
+backup stage end;
+--connection default
+--disconnect con_backup
+--enable_query_log
+
+if ($log_bin)
+{
+  --source include/show_binlog_events.inc
+}
 DROP TABLE t1;
+if ($drop_statement)
+{
+  --eval $drop_statement
+}
 
+if ($log_bin)
+{
+  RESET MASTER;
+}
+if ($create_statement2)
+{
+  --eval $create_statement2
+}
+if ($insert_statement2)
+{
+  --eval $insert_statement2
+}
 --eval $create_statement
 --eval $insert_statement
 --echo # $dbug_flag: BEFORE failure (under LOCK TABLE)
@@ -29,15 +94,42 @@ DROP TABLE t1;
 SHOW CREATE TABLE t1;
 --sorted_result
 SELECT * FROM t1;
-LOCK TABLE t1 WRITE;
+if (!$create_statement2)
+{
+  LOCK TABLE t1 WRITE;
+}
+if ($create_statement2)
+{
+  LOCK TABLES t1 WRITE, t2 WRITE;
+}
 --disable_abort_on_error
 --eval $fail_statement
 --enable_abort_on_error
 --echo # $dbug_flag: AFTER failure (under LOCK TABLE)
+show warnings;
 --replace_result #p# #P# #sp# #SP#
 --list_files $DATADIR/test
 SHOW CREATE TABLE t1;
 --sorted_result
 SELECT * FROM t1;
+if ($show_statement)
+{
+  --error 0,ER_TABLE_NOT_LOCKED
+  --eval $show_statement
+}
+if ($select_statement)
+{
+  --sorted_result
+  --error 0,ER_TABLE_NOT_LOCKED
+  --eval $select_statement
+}
+if ($log_bin)
+{
+  --source include/show_binlog_events.inc
+}
 UNLOCK TABLES;
 DROP TABLE t1;
+if ($drop_statement)
+{
+  --eval $drop_statement
+}