MDEV-15250 UPSERT during ALTER TABLE results in 'Duplicate entry' error for alter

- InnoDB DDL results in `Duplicate entry' if concurrent DML throws
duplicate key error. The following scenario explains the problem

connection con1:
  ALTER TABLE t1 FORCE;

connection con2:
  INSERT INTO t1(pk, uk) VALUES (2, 2), (3, 2);

In connection con2, InnoDB throws the 'DUPLICATE KEY' error because
of unique index. Alter operation will throw the error when applying
the concurrent DML log.

- Inserting the duplicate key for unique index logs the insert
operation for online ALTER TABLE. When insertion fails,
transaction does rollback and it leads to logging of
delete operation for online ALTER TABLE.
While applying the insert log entries, alter operation
encounters 'DUPLICATE KEY' error.

- To avoid the above fake duplicate scenario, InnoDB should
not write any log for online ALTER TABLE before DML transaction
commit.

- User thread which does DML can apply the online log if
InnoDB ran out of online log and index is marked as completed.
Set online log error if apply phase encountered any error.
It can also clear all other indexes log, marks the newly
added indexes as corrupted.

- Removed the old online code which was a part of DML operations

commit_inplace_alter_table() : Does apply the online log
for the last batch of secondary index log and does frees
the log for the completed index.

trx_t::apply_online_log: Set to true while writing the undo
log if the modified table has active DDL

trx_t::apply_log(): Apply the DML changes to online DDL tables

dict_table_t::is_active_ddl(): Returns true if the table
has an active DDL

dict_index_t::online_log_make_dummy(): Assign dummy value
for clustered index online log to indicate the secondary
indexes are being rebuild.

dict_index_t::online_log_is_dummy(): Check whether the online
log has dummy value

ha_innobase_inplace_ctx::log_failure(): Handle the apply log
failure for online DDL transaction

row_log_mark_other_online_index_abort(): Clear out all other
online index log after encountering the error during
row_log_apply()

row_log_get_error(): Get the error happened during row_log_apply()

row_log_online_op(): Does apply the online log if index is
completed and ran out of memory. Returns false if apply log fails

UndorecApplier: Introduced a class to maintain the undo log
record, latched undo buffer page, parse the undo log record,
maintain the undo record type, info bits and update vector

UndorecApplier::get_old_rec(): Get the correct version of the
clustered index record that was modified by the current undo
log record

UndorecApplier::clear_undo_rec(): Clear the undo log related
information after applying the undo log record

UndorecApplier::log_update(): Handle the update, delete undo
log and apply it on online indexes

UndorecApplier::log_insert(): Handle the insert undo log
and apply it on online indexes

UndorecApplier::is_same(): Check whether the given roll pointer
is generated by the current undo log record information

trx_t::rollback_low(): Set apply_online_log for the transaction
after partially rollbacked transaction has any active DDL

prepare_inplace_alter_table_dict(): After allocating the online
log, InnoDB does create fulltext common tables. Fulltext index
doesn't allow the index to be online. So removed the dead
code of online log removal

Thanks to Marko Mäkelä for providing the initial prototype and
Matthias Leich for testing the issue patiently.

34 files changed, 484 insertions, 158 deletions

diff --git a/mysql-test/suite/gcol/r/innodb_virtual_debug.result b/mysql-test/suite/gcol/r/innodb_virtual_debug.result
index 806cf1a98c8..80b2bde6ca5 100644
--- a/mysql-test/suite/gcol/r/innodb_virtual_debug.result
+++ b/mysql-test/suite/gcol/r/innodb_virtual_debug.result
@@ -105,7 +105,7 @@ SET lock_wait_timeout = 1;
 ALTER TABLE t1 ADD UNIQUE INDEX(c, b);
 connection default;
 SET DEBUG_SYNC = 'now WAIT_FOR s1';
-SET DEBUG_SYNC = 'row_ins_sec_index_enter SIGNAL s2 WAIT_FOR s3';
+SET DEBUG_SYNC = 'row_log_insert_handle SIGNAL s2 WAIT_FOR s3';
 INSERT INTO t1(a, b) VALUES(2, 2);
 connection con1;
 ERROR HY000: Lock wait timeout exceeded; try restarting transaction
diff --git a/mysql-test/suite/gcol/t/innodb_virtual_debug.test b/mysql-test/suite/gcol/t/innodb_virtual_debug.test
index 40446b991cd..5ebc90dac19 100644
--- a/mysql-test/suite/gcol/t/innodb_virtual_debug.test
+++ b/mysql-test/suite/gcol/t/innodb_virtual_debug.test
@@ -295,7 +295,7 @@ SET lock_wait_timeout = 1;
 
 connection default;
 SET DEBUG_SYNC = 'now WAIT_FOR s1';
-SET DEBUG_SYNC = 'row_ins_sec_index_enter SIGNAL s2 WAIT_FOR s3';
+SET DEBUG_SYNC = 'row_log_insert_handle SIGNAL s2 WAIT_FOR s3';
 --send INSERT INTO t1(a, b) VALUES(2, 2)
 
 connection con1;
diff --git a/mysql-test/suite/innodb/r/alter_candidate_key.result b/mysql-test/suite/innodb/r/alter_candidate_key.result
index 79cb225e3b5..2ada5a499a8 100644
--- a/mysql-test/suite/innodb/r/alter_candidate_key.result
+++ b/mysql-test/suite/innodb/r/alter_candidate_key.result
@@ -74,7 +74,7 @@ connection con1;
 SET DEBUG_SYNC='now WAIT_FOR dml';
 BEGIN;
 INSERT INTO t1 SET a=NULL;
-ROLLBACK;
+COMMIT;
 set DEBUG_SYNC='now SIGNAL dml_done';
 connection default;
 ERROR 22004: Invalid use of NULL value
diff --git a/mysql-test/suite/innodb/r/alter_crash.result b/mysql-test/suite/innodb/r/alter_crash.result
index 46ea85d3e1e..a98aeb70a15 100644
--- a/mysql-test/suite/innodb/r/alter_crash.result
+++ b/mysql-test/suite/innodb/r/alter_crash.result
@@ -169,9 +169,6 @@ INSERT INTO t1(f1, f2) VALUES(2, "This is column2 value");
 ROLLBACK;
 set DEBUG_SYNC = 'now SIGNAL insert_done';
 connection default;
-Warnings:
-Warning	1265	Data truncated for column 'f3' at row 3
-Warning	1265	Data truncated for column 'f4' at row 3
 SHOW CREATE TABLE t1;
 Table	Create Table
 t1	CREATE TABLE `t1` (
@@ -202,6 +199,7 @@ connection default;
 SET DEBUG_SYNC = 'now WAIT_FOR scanned';
 BEGIN;
 INSERT INTO t1 VALUES(2,1);
+COMMIT;
 SET DEBUG_SYNC = 'now SIGNAL commit';
 SET DEBUG_SYNC = 'now WAIT_FOR c';
 SET GLOBAL innodb_fil_make_page_dirty_debug=0;
@@ -221,4 +219,5 @@ t1	CREATE TABLE `t1` (
 SELECT * FROM t1;
 a	b
 1	1
+2	1
 DROP TABLE t1;
diff --git a/mysql-test/suite/innodb/r/alter_dml_apply.result b/mysql-test/suite/innodb/r/alter_dml_apply.result
new file mode 100644
index 00000000000..c9def9d6a41
--- /dev/null
+++ b/mysql-test/suite/innodb/r/alter_dml_apply.result
@@ -0,0 +1,39 @@
+CREATE TABLE t1(f1 INT NOT NULL, f2 INT NOT NULL,
+f3 CHAR(200), f4 CHAR(200),
+PRIMARY KEY(f1))ENGINE=InnoDB;
+INSERT INTO t1 VALUES(6000, 6000, "InnoDB", "MariaDB");
+SET DEBUG_SYNC="inplace_after_index_build SIGNAL dml_start WAIT_FOR dml_commit";
+ALTER TABLE t1 ADD UNIQUE KEY(f2), ADD UNIQUE INDEX(f4(10));
+connect con1,localhost,root,,,;
+SET DEBUG_SYNC="now WAIT_FOR dml_start";
+BEGIN;
+DELETE FROM t1 WHERE f1= 6000;
+INSERT INTO t1 VALUES(6000, 6000, "InnoDB", "MariaDB");
+ROLLBACK;
+BEGIN;
+DELETE FROM t1 WHERE f1= 6000;
+INSERT INTO t1 VALUES(6000, 6000, "InnoDB", "MariaDB");
+INSERT INTO t1 SELECT seq, seq, repeat('a', 200), repeat('b', 200) FROM seq_1_to_4000;
+COMMIT;
+SET DEBUG_SYNC="now SIGNAL dml_commit";
+connection default;
+ERROR 23000: Duplicate entry '' for key '*UNKNOWN*'
+SET DEBUG_SYNC="inplace_after_index_build SIGNAL dml_start WAIT_FOR dml_commit";
+ALTER TABLE t1 ADD UNIQUE KEY(f2), ADD INDEX(f3(10));
+connection con1;
+SET DEBUG_SYNC="now WAIT_FOR dml_start";
+BEGIN;
+DELETE FROM t1;
+INSERT INTO t1 SELECT seq, seq, repeat('d', 200), repeat('e', 200) FROM
+seq_1_to_4000;
+UPDATE t1 SET f3=repeat('c', 200), f4= repeat('d', 200), f2=3;
+COMMIT;
+SET DEBUG_SYNC="now SIGNAL dml_commit";
+connection default;
+ERROR 23000: Duplicate entry '' for key '*UNKNOWN*'
+disconnect con1;
+CHECK TABLE t1;
+Table	Op	Msg_type	Msg_text
+test.t1	check	status	OK
+DROP TABLE t1;
+SET DEBUG_SYNC=reset;
diff --git a/mysql-test/suite/innodb/r/alter_mdl_timeout.result b/mysql-test/suite/innodb/r/alter_mdl_timeout.result
index 7af1362c69e..e4fba8e260f 100644
--- a/mysql-test/suite/innodb/r/alter_mdl_timeout.result
+++ b/mysql-test/suite/innodb/r/alter_mdl_timeout.result
@@ -10,7 +10,7 @@ begin;
 INSERT INTO t1 VALUES('e','e',5, 5);
 SET DEBUG_SYNC="now SIGNAL con1_insert";
 SET DEBUG_SYNC="now WAIT_FOR con1_wait";
-SET DEBUG_SYNC="before_row_upd_sec_new_index_entry SIGNAL con1_update WAIT_FOR alter_rollback";
+SET DEBUG_SYNC="after_row_upd_clust SIGNAL con1_update WAIT_FOR alter_rollback";
 UPDATE t1 set f4 = 10 order by f1 desc limit 2;
 connection default;
 ERROR HY000: Lock wait timeout exceeded; try restarting transaction
diff --git a/mysql-test/suite/innodb/r/alter_not_null_debug,STRICT.rdiff b/mysql-test/suite/innodb/r/alter_not_null_debug,STRICT.rdiff
index 09c717c44b0..81466d791ac 100644
--- a/mysql-test/suite/innodb/r/alter_not_null_debug,STRICT.rdiff
+++ b/mysql-test/suite/innodb/r/alter_not_null_debug,STRICT.rdiff
@@ -1,11 +1,16 @@
-18,21c18
-< affected rows: 0
-< info: Records: 0  Duplicates: 0  Warnings: 1
-< Warnings:
-< Warning	1265	Data truncated for column 'c2' at row 3
----
-> ERROR 01000: Data truncated for column 'c2' at row 3
-24c21
-< 2	0
----
-> 2	NULL
+@@ -15,13 +15,10 @@
+ SET DEBUG_SYNC= 'now SIGNAL flushed';
+ affected rows: 0
+ connection default;
+-affected rows: 0
+-info: Records: 0  Duplicates: 0  Warnings: 1
+-Warnings:
+-Warning	1265	Data truncated for column 'c2' at row 3
++ERROR 22004: Invalid use of NULL value
+ SELECT * FROM t1;
+ c1	c2
+-2	0
++2	NULL
+ 3	1
+ DROP TABLE t1;
+ CREATE TABLE t1(c1 INT NOT NULL, c2 INT, PRIMARY KEY(c1))ENGINE=INNODB;
diff --git a/mysql-test/suite/innodb/r/innodb-alter-debug.result b/mysql-test/suite/innodb/r/innodb-alter-debug.result
index 81fc67e55c7..aae9432fc35 100644
--- a/mysql-test/suite/innodb/r/innodb-alter-debug.result
+++ b/mysql-test/suite/innodb/r/innodb-alter-debug.result
@@ -43,9 +43,8 @@ SET DEBUG_SYNC = 'now SIGNAL s2';
 /* connection default */
 connection default;
 /* reap */ alter table t1 force, add b int, ALGORITHM=inplace;
-ERROR 23000: Duplicate entry '1' for key 'uk'
 SET DEBUG_SYNC = 'row_log_table_apply1_before SIGNAL s1 WAIT_FOR s2';
-alter table t1 force, add b int, ALGORITHM=inplace;;
+alter table t1 force, add c int, ALGORITHM=inplace;;
 /* connection con1 */
 connection con1;
 set DEBUG_SYNC = 'now WAIT_FOR s1';
@@ -55,7 +54,6 @@ SET DEBUG_SYNC = 'now SIGNAL s2';
 /* connection default */
 connection default;
 /* reap */ alter table t1 force, add b int, ALGORITHM=inplace;
-ERROR 23000: Duplicate entry '1' for key 'uk'
 SET DEBUG_SYNC = 'RESET';
 drop table t1;
 #
@@ -72,7 +70,6 @@ ERROR 23000: Duplicate entry '1' for key 'a'
 SET DEBUG_SYNC = 'now SIGNAL S2';
 disconnect con1;
 connection default;
-ERROR 23000: Duplicate entry '1' for key 'a'
 SET DEBUG_SYNC='RESET';
 DROP TABLE t1;
 #
diff --git a/mysql-test/suite/innodb/r/innodb-index-debug.result b/mysql-test/suite/innodb/r/innodb-index-debug.result
index f6b23eea41a..c36a0531b95 100644
--- a/mysql-test/suite/innodb/r/innodb-index-debug.result
+++ b/mysql-test/suite/innodb/r/innodb-index-debug.result
@@ -118,20 +118,21 @@ drop table t480;
 # MDEV-12827 Assertion failure when reporting duplicate key error
 # in online table rebuild
 #
-CREATE TABLE t1 (j INT UNIQUE, i INT UNIQUE) ENGINE=InnoDB;
+CREATE TABLE t1 (j INT UNIQUE, i INT) ENGINE=InnoDB;
 INSERT INTO t1 VALUES(2, 2);
 connect  con1,localhost,root,,test;
 SET DEBUG_SYNC='row_log_table_apply1_before SIGNAL built WAIT_FOR log';
-ALTER TABLE t1 DROP j, FORCE;
+ALTER TABLE t1 DROP j, ADD UNIQUE INDEX(i), FORCE;
 connection default;
 SET DEBUG_SYNC='now WAIT_FOR built';
 SET DEBUG_DBUG='+d,row_ins_row_level';
 INSERT INTO t1 (i) VALUES (0),(0);
-ERROR 23000: Duplicate entry '0' for key 'i'
 SET DEBUG_SYNC='now SIGNAL log';
 SET DEBUG_DBUG=@saved_debug_dbug;
 connection con1;
 ERROR 23000: Duplicate entry '0' for key 'i'
+DELETE FROM t1;
+ALTER TABLE t1 ADD UNIQUE INDEX(i);
 SET DEBUG_SYNC='row_log_table_apply1_before SIGNAL built2 WAIT_FOR log2';
 ALTER TABLE t1 DROP j, FORCE;
 connection default;
@@ -141,7 +142,6 @@ UPDATE t1 SET i=0;
 ERROR 23000: Duplicate entry '0' for key 'i'
 SET DEBUG_SYNC='now SIGNAL log2';
 connection con1;
-ERROR 23000: Duplicate entry '0' for key 'i'
 disconnect con1;
 connection default;
 SET DEBUG_SYNC='RESET';
diff --git a/mysql-test/suite/innodb/r/innodb-index-online.result b/mysql-test/suite/innodb/r/innodb-index-online.result
index 1ee352cd402..be86cf7565e 100644
--- a/mysql-test/suite/innodb/r/innodb-index-online.result
+++ b/mysql-test/suite/innodb/r/innodb-index-online.result
@@ -85,7 +85,8 @@ ddl_sort_file_alter_table	0
 ddl_log_file_alter_table	0
 BEGIN;
 INSERT INTO t1 VALUES(7,4,2);
-ROLLBACK;
+COMMIT;
+DELETE FROM t1 where c1 = 7;
 SET DEBUG_SYNC = 'now SIGNAL rollback_done';
 connection con1;
 ERROR 23000: Duplicate entry '4' for key 'c2'
@@ -96,14 +97,14 @@ SET DEBUG_SYNC = 'now WAIT_FOR created';
 SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
 name	count
 ddl_background_drop_indexes	0
-ddl_online_create_index	0
+ddl_online_create_index	1
 ddl_pending_alter_table	1
 ddl_sort_file_alter_table	0
 ddl_log_file_alter_table	0
 INSERT INTO t1 VALUES(6,3,1);
 SET DEBUG_SYNC = 'now SIGNAL dml_done';
 connection con1;
-ERROR 23000: Duplicate entry for key 'c2'
+ERROR 23000: Duplicate entry '' for key '*UNKNOWN*'
 DELETE FROM t1 WHERE c1=6;
 ALTER TABLE t1 ADD UNIQUE INDEX(c2);
 SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
@@ -243,38 +244,22 @@ ddl_online_create_index	1
 ddl_pending_alter_table	1
 ddl_sort_file_alter_table	0
 ddl_log_file_alter_table	0
-BEGIN;
-DELETE FROM t1;
-ROLLBACK;
 UPDATE t1 SET c2 = c2 + 1;
-BEGIN;
+UPDATE t1 SET c2 = c2 + 2;
 UPDATE t1 SET c2 = c2 + 1;
-DELETE FROM t1;
-ROLLBACK;
-BEGIN;
-DELETE FROM t1;
-ROLLBACK;
+UPDATE t1 SET c2 = c2 + 2;
 UPDATE t1 SET c2 = c2 + 1;
-BEGIN;
+UPDATE t1 SET c2 = c2 + 2;
 UPDATE t1 SET c2 = c2 + 1;
-DELETE FROM t1;
-ROLLBACK;
-BEGIN;
-DELETE FROM t1;
-ROLLBACK;
+UPDATE t1 SET c2 = c2 + 2;
 UPDATE t1 SET c2 = c2 + 1;
-BEGIN;
+UPDATE t1 SET c2 = c2 + 2;
 UPDATE t1 SET c2 = c2 + 1;
-DELETE FROM t1;
-ROLLBACK;
-BEGIN;
-DELETE FROM t1;
-ROLLBACK;
+UPDATE t1 SET c2 = c2 + 2;
 UPDATE t1 SET c2 = c2 + 1;
-BEGIN;
+UPDATE t1 SET c2 = c2 + 2;
 UPDATE t1 SET c2 = c2 + 1;
-DELETE FROM t1;
-ROLLBACK;
+UPDATE t1 SET c2 = c2 + 2;
 SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
 name	count
 ddl_background_drop_indexes	0
@@ -310,7 +295,7 @@ ERROR HY000: Creating index 'c2e' required more than 'innodb_online_alter_log_ma
 SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
 name	count
 ddl_background_drop_indexes	1
-ddl_online_create_index	0
+ddl_online_create_index	1
 ddl_pending_alter_table	0
 ddl_sort_file_alter_table	0
 ddl_log_file_alter_table	1
@@ -321,7 +306,7 @@ name	pos
 SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
 name	count
 ddl_background_drop_indexes	1
-ddl_online_create_index	0
+ddl_online_create_index	1
 ddl_pending_alter_table	0
 ddl_sort_file_alter_table	0
 ddl_log_file_alter_table	1
@@ -361,19 +346,45 @@ ddl_log_file_alter_table	1
 BEGIN;
 INSERT INTO t1 SELECT 320 + c1, c2, c3 FROM t1 WHERE c1 > 160;
 DELETE FROM t1 WHERE c1 > 320;
-ROLLBACK;
+COMMIT;
 BEGIN;
 UPDATE t1 SET c2 = c2 + 1;
-DELETE FROM t1;
-ROLLBACK;
+COMMIT;
 BEGIN;
 INSERT INTO t1 SELECT 320 + c1, c2, c3 FROM t1 WHERE c1 > 160;
 DELETE FROM t1 WHERE c1 > 320;
-ROLLBACK;
+COMMIT;
 BEGIN;
 UPDATE t1 SET c2 = c2 + 1;
-DELETE FROM t1;
-ROLLBACK;
+COMMIT;
+BEGIN;
+INSERT INTO t1 SELECT 320 + c1, c2, c3 FROM t1 WHERE c1 > 160;
+DELETE FROM t1 WHERE c1 > 320;
+COMMIT;
+BEGIN;
+UPDATE t1 SET c2 = c2 + 1;
+COMMIT;
+BEGIN;
+INSERT INTO t1 SELECT 320 + c1, c2, c3 FROM t1 WHERE c1 > 160;
+DELETE FROM t1 WHERE c1 > 320;
+COMMIT;
+BEGIN;
+UPDATE t1 SET c2 = c2 + 1;
+COMMIT;
+BEGIN;
+INSERT INTO t1 SELECT 320 + c1, c2, c3 FROM t1 WHERE c1 > 160;
+DELETE FROM t1 WHERE c1 > 320;
+COMMIT;
+BEGIN;
+UPDATE t1 SET c2 = c2 + 1;
+COMMIT;
+BEGIN;
+INSERT INTO t1 SELECT 320 + c1, c2, c3 FROM t1 WHERE c1 > 160;
+DELETE FROM t1 WHERE c1 > 320;
+COMMIT;
+BEGIN;
+UPDATE t1 SET c2 = c2 + 1;
+COMMIT;
 SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
 name	count
 ddl_background_drop_indexes	0
@@ -452,7 +463,7 @@ name	pos
 SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
 name	count
 ddl_background_drop_indexes	1
-ddl_online_create_index	0
+ddl_online_create_index	1
 ddl_pending_alter_table	0
 ddl_sort_file_alter_table	0
 ddl_log_file_alter_table	2
@@ -460,7 +471,7 @@ connection default;
 SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
 name	count
 ddl_background_drop_indexes	1
-ddl_online_create_index	0
+ddl_online_create_index	1
 ddl_pending_alter_table	0
 ddl_sort_file_alter_table	0
 ddl_log_file_alter_table	2
@@ -498,7 +509,6 @@ SET DEBUG_SYNC = 'row_log_apply_before SIGNAL t1u_created WAIT_FOR dup_done';
 ALTER TABLE t1 ADD UNIQUE(c);
 connection con1;
 SET DEBUG_SYNC = 'now WAIT_FOR t1u_created';
-BEGIN;
 INSERT INTO t1 VALUES('bar'),('bar');
 SET DEBUG_SYNC = 'now SIGNAL dup_done';
 connection default;
diff --git a/mysql-test/suite/innodb/r/innodb-table-online.result b/mysql-test/suite/innodb/r/innodb-table-online.result
index 91d9b355125..659d645b5cc 100644
--- a/mysql-test/suite/innodb/r/innodb-table-online.result
+++ b/mysql-test/suite/innodb/r/innodb-table-online.result
@@ -97,11 +97,11 @@ ddl_online_create_index	1
 ddl_pending_alter_table	1
 ddl_sort_file_alter_table	0
 ddl_log_file_alter_table	0
-BEGIN;
 INSERT INTO t1 VALUES(4,7,2);
 SET DEBUG_SYNC = 'now SIGNAL insert_done';
 connection con1;
 ERROR 23000: Duplicate entry '4' for key 'PRIMARY'
+DELETE FROM t1 WHERE c1=4 and c2=7;
 connection default;
 ROLLBACK;
 connection con1;
@@ -213,30 +213,22 @@ ddl_online_create_index	1
 ddl_pending_alter_table	1
 ddl_sort_file_alter_table	0
 ddl_log_file_alter_table	1
-BEGIN;
-DELETE FROM t1;
-ROLLBACK;
 UPDATE t1 SET c2 = c2 + 1;
-BEGIN;
+UPDATE t1 SET c2 = c2 + 2;
 UPDATE t1 SET c2 = c2 + 1;
-DELETE FROM t1;
-ROLLBACK;
-BEGIN;
-DELETE FROM t1;
-ROLLBACK;
+UPDATE t1 SET c2 = c2 + 2;
 UPDATE t1 SET c2 = c2 + 1;
-BEGIN;
+UPDATE t1 SET c2 = c2 + 2;
 UPDATE t1 SET c2 = c2 + 1;
-DELETE FROM t1;
-ROLLBACK;
-BEGIN;
-DELETE FROM t1;
-ROLLBACK;
+UPDATE t1 SET c2 = c2 + 2;
 UPDATE t1 SET c2 = c2 + 1;
-BEGIN;
+UPDATE t1 SET c2 = c2 + 2;
 UPDATE t1 SET c2 = c2 + 1;
-DELETE FROM t1;
-ROLLBACK;
+UPDATE t1 SET c2 = c2 + 2;
+UPDATE t1 SET c2 = c2 + 1;
+UPDATE t1 SET c2 = c2 + 2;
+UPDATE t1 SET c2 = c2 + 1;
+UPDATE t1 SET c2 = c2 + 2;
 SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
 name	count
 ddl_background_drop_indexes	0
@@ -286,7 +278,7 @@ SET DEBUG_SYNC = 'row_log_table_apply1_before SIGNAL rebuilt3 WAIT_FOR dml3_done
 ALTER TABLE t1 ADD PRIMARY KEY(c22f), CHANGE c2 c22f INT;
 ERROR 42000: Multiple primary key defined
 ALTER TABLE t1 DROP PRIMARY KEY, ADD PRIMARY KEY(c22f), CHANGE c2 c22f INT;
-ERROR 23000: Duplicate entry '5' for key 'PRIMARY'
+ERROR 23000: Duplicate entry '26' for key 'PRIMARY'
 ALTER TABLE t1 DROP PRIMARY KEY, ADD PRIMARY KEY(c22f,c1,c4(5)),
 CHANGE c2 c22f INT, CHANGE c3 c3 CHAR(255) NULL, CHANGE c1 c1 INT AFTER c22f,
 ADD COLUMN c4 VARCHAR(6) DEFAULT 'Online', LOCK=NONE;
@@ -302,11 +294,8 @@ ddl_log_file_alter_table	1
 BEGIN;
 INSERT INTO t1 SELECT 320 + c1, c2, c3 FROM t1 WHERE c1 > 240;
 DELETE FROM t1 WHERE c1 > 320;
-ROLLBACK;
-BEGIN;
 UPDATE t1 SET c2 = c2 + 1;
-DELETE FROM t1;
-ROLLBACK;
+COMMIT;
 SELECT name, count FROM INFORMATION_SCHEMA.INNODB_METRICS WHERE subsystem = 'ddl';
 name	count
 ddl_background_drop_indexes	0
@@ -374,12 +363,11 @@ SET DEBUG_SYNC = 'now WAIT_FOR c3p5_created0';
 BEGIN;
 INSERT INTO t1 VALUES(347,33101,'Pikku kakkosen posti','YLETV2');
 INSERT INTO t1 VALUES(33101,347,NULL,'');
+COMMIT;
 SET DEBUG_SYNC = 'now SIGNAL ins_done0';
 connection con1;
-ERROR 01000: Data truncated for column 'c3' at row 323
-connection default;
-ROLLBACK;
-connection con1;
+ERROR 22004: Invalid use of NULL value
+DELETE FROM t1 WHERE c1= 347 and c22f = 33101;
 ALTER TABLE t1 MODIFY c3 CHAR(255) NOT NULL;
 SET DEBUG_SYNC = 'row_log_table_apply1_before SIGNAL c3p5_created WAIT_FOR ins_done';
 ALTER TABLE t1 DROP PRIMARY KEY, DROP COLUMN c22f,
@@ -405,20 +393,20 @@ ddl_log_file_alter_table	2
 connection default;
 SELECT COUNT(*) FROM t1;
 COUNT(*)
-321
+322
 ALTER TABLE t1 ROW_FORMAT=REDUNDANT;
 SELECT * FROM t1 LIMIT 10;
 c22f	c1	c3	c4
-5	1	1foo	Online
-5	6	6foofoofoofoofoofoo	Online
-5	11	11foofoofoofoofoofoofoofoofoofoofoo	Online
-5	16	16foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo	Online
-5	21	21foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo	Online
-5	26	26foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo	Online
-5	31	31foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo	Online
-5	36	36foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo	Online
-5	41	41foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo	Online
-5	46	46foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo	Online
+27	1	1foo	Online
+27	6	6foofoofoofoofoofoo	Online
+27	11	11foofoofoofoofoofoofoofoofoofoofoo	Online
+27	16	16foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo	Online
+27	21	21foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo	Online
+27	26	26foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo	Online
+27	31	31foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo	Online
+27	36	36foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo	Online
+27	41	41foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo	Online
+27	46	46foofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo	Online
 connection con1;
 ALTER TABLE t1 DISCARD TABLESPACE;
 connection default;
diff --git a/mysql-test/suite/innodb/r/insert_into_empty_debug.result b/mysql-test/suite/innodb/r/insert_into_empty_debug.result
new file mode 100644
index 00000000000..0f5d6a63a17
--- /dev/null
+++ b/mysql-test/suite/innodb/r/insert_into_empty_debug.result
@@ -0,0 +1,61 @@
+CREATE TABLE t1(f1 INT NOT NULL, f2 INT NOT NULL,
+PRIMARY KEY(f1))ENGINE=InnoDB;
+INSERT INTO t1 VALUES(1, 2), (2, 2);
+SET DEBUG_SYNC="innodb_rollback_inplace_alter_table SIGNAL dml_start WAIT_FOR dml_commit";
+ALTER TABLE t1 ADD UNIQUE KEY(f2);
+connect con1,localhost,root,,,;
+SET DEBUG_SYNC="now WAIT_FOR dml_start";
+BEGIN;
+DELETE FROM t1;
+SET DEBUG_SYNC="now SIGNAL dml_commit";
+connection default;
+ERROR 23000: Duplicate entry '2' for key 'f2'
+connection con1;
+COMMIT;
+TRUNCATE TABLE t1;
+SET unique_checks=0, foreign_key_checks=0;
+BEGIN;
+INSERT INTO t1 VALUES(1, 2);
+ROLLBACK;
+connection default;
+SELECT * FROM t1;
+f1	f2
+SHOW CREATE TABLE t1;
+Table	Create Table
+t1	CREATE TABLE `t1` (
+  `f1` int(11) NOT NULL,
+  `f2` int(11) NOT NULL,
+  PRIMARY KEY (`f1`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1
+INSERT INTO t1 VALUES(1, 1);
+connection con1;
+START TRANSACTION WITH CONSISTENT SNAPSHOT;
+connect con2,localhost,root,,,;
+DELETE FROM t1;
+connection default;
+SET DEBUG_SYNC="innodb_inplace_alter_table_enter SIGNAL purge_resume WAIT_FOR dml_commit";
+ALTER TABLE t1 ADD INDEX(f2, f1);
+connection con1;
+COMMIT;
+connection con2;
+SET GLOBAL innodb_purge_rseg_truncate_frequency=1;
+InnoDB		1 transactions not purged
+SET unique_checks=0, foreign_key_checks=0;
+BEGIN;
+INSERT INTO t1 VALUES(2, 2);
+ROLLBACK;
+SET DEBUG_SYNC="now SIGNAL dml_commit";
+connection default;
+SHOW CREATE TABLE t1;
+Table	Create Table
+t1	CREATE TABLE `t1` (
+  `f1` int(11) NOT NULL,
+  `f2` int(11) NOT NULL,
+  PRIMARY KEY (`f1`),
+  KEY `f2` (`f2`,`f1`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1
+disconnect con1;
+disconnect con2;
+DROP TABLE t1;
+SET DEBUG_SYNC=RESET;
+SET GLOBAL innodb_purge_rseg_truncate_frequency=default;
diff --git a/mysql-test/suite/innodb/r/instant_alter_debug,dynamic.rdiff b/mysql-test/suite/innodb/r/instant_alter_debug,dynamic.rdiff
deleted file mode 100644
index 379514edad9..00000000000
--- a/mysql-test/suite/innodb/r/instant_alter_debug,dynamic.rdiff
+++ /dev/null
@@ -1,6 +0,0 @@
-@@ -470,4 +470,4 @@
- FROM information_schema.global_status
- WHERE variable_name = 'innodb_instant_alter_column';
- instants
--33
-+32
diff --git a/mysql-test/suite/innodb/r/instant_alter_debug,redundant.rdiff b/mysql-test/suite/innodb/r/instant_alter_debug,redundant.rdiff
index eafa8e5725d..cff4ff18c70 100644
--- a/mysql-test/suite/innodb/r/instant_alter_debug,redundant.rdiff
+++ b/mysql-test/suite/innodb/r/instant_alter_debug,redundant.rdiff
@@ -1,4 +1,4 @@
-@@ -509,4 +509,4 @@
+@@ -527,4 +527,4 @@
  FROM information_schema.global_status
  WHERE variable_name = 'innodb_instant_alter_column';
  instants
diff --git a/mysql-test/suite/innodb/r/instant_alter_debug.result b/mysql-test/suite/innodb/r/instant_alter_debug.result
index 82230573c44..5f74c234260 100644
--- a/mysql-test/suite/innodb/r/instant_alter_debug.result
+++ b/mysql-test/suite/innodb/r/instant_alter_debug.result
@@ -262,7 +262,6 @@ INSERT INTO t1 SET a=3;
 ROLLBACK;
 SET DEBUG_SYNC = 'now SIGNAL logged';
 connection ddl;
-ERROR 22004: Invalid use of NULL value
 disconnect ddl;
 connection default;
 SET DEBUG_SYNC = RESET;
diff --git a/mysql-test/suite/innodb/r/online_table_rebuild.result b/mysql-test/suite/innodb/r/online_table_rebuild.result
new file mode 100644
index 00000000000..0ba26fca1d9
--- /dev/null
+++ b/mysql-test/suite/innodb/r/online_table_rebuild.result
@@ -0,0 +1,47 @@
+CREATE TABLE t1(f1 INT NOT NULL, f2 CHAR(200), f3 CHAR(200))ENGINE=InnoDB;
+INSERT INTO t1 VALUES(3, "innodb", "alter log");
+SET DEBUG_SYNC="inplace_after_index_build SIGNAL dml_start WAIT_FOR dml_commit";
+ALTER TABLE t1 ADD PRIMARY KEY(f3(10)), ADD UNIQUE KEY(f2(10));
+CONNECT con1,localhost,root,,,;
+SET DEBUG_SYNC="now WAIT_FOR dml_start";
+BEGIN;
+INSERT INTO t1 VALUES(1, repeat('b', 100), repeat('c', 100));
+INSERT INTO t1 VALUES(2, repeat('b', 100), repeat('a', 100));
+COMMIT;
+SET DEBUG_SYNC="now SIGNAL dml_commit";
+connection default;
+ERROR 23000: Duplicate entry 'bbbbbbbbbb' for key 'f2'
+connection default;
+SET DEBUG_SYNC="inplace_after_index_build SIGNAL dml_start WAIT_FOR dml_commit";
+ALTER TABLE t1 ADD PRIMARY KEY(f1);
+connection con1;
+SET DEBUG_SYNC="now WAIT_FOR dml_start";
+INSERT INTO t1 SELECT 10, repeat('a', 100), repeat('b', 100) FROM seq_1_to_4800;
+SET DEBUG_SYNC="now SIGNAL dml_commit";
+connection default;
+ERROR HY000: Creating index 'PRIMARY' required more than 'innodb_online_alter_log_max_size' bytes of modification log. Please try again
+DELETE FROM t1;
+INSERT INTO t1 VALUES(1, repeat('a', 100), repeat('b', 100));
+ALTER TABLE t1 ADD PRIMARY KEY(f1);
+set DEBUG_SYNC="innodb_inplace_alter_table_enter SIGNAL dml_start WAIT_FOR dml_commit";
+ALTER TABLE t1 DROP PRIMARY KEY, ADD PRIMARY KEY(f3(10));
+connection con1;
+SET DEBUG_SYNC="now WAIT_FOR dml_start";
+BEGIN;
+INSERT INTO t1 VALUES(2, repeat('b', 100), repeat('c', 100));
+UPDATE t1 set f3=repeat('c', 100) where f1=1;
+COMMIT;
+SET DEBUG_SYNC="now SIGNAL dml_commit";
+connection default;
+ERROR 23000: Duplicate entry 'cccccccccc' for key 'PRIMARY'
+SHOW CREATE TABLE t1;
+Table	Create Table
+t1	CREATE TABLE `t1` (
+  `f1` int(11) NOT NULL,
+  `f2` char(200) DEFAULT NULL,
+  `f3` char(200) DEFAULT NULL,
+  PRIMARY KEY (`f1`)
+) ENGINE=InnoDB DEFAULT CHARSET=latin1
+DROP TABLE t1;
+disconnect con1;
+SET DEBUG_SYNC=reset;
diff --git a/mysql-test/suite/innodb/r/table_definition_cache_debug.result b/mysql-test/suite/innodb/r/table_definition_cache_debug.result
index df171c89cd4..6bd754aaca3 100644
--- a/mysql-test/suite/innodb/r/table_definition_cache_debug.result
+++ b/mysql-test/suite/innodb/r/table_definition_cache_debug.result
@@ -12,6 +12,7 @@ connection default;
 SET DEBUG_SYNC = 'now WAIT_FOR scanned';
 BEGIN;
 INSERT INTO to_be_evicted VALUES(3, 2);
+COMMIT;
 SET DEBUG_SYNC = 'now SIGNAL got_duplicate';
 connection ddl;
 ERROR 23000: Duplicate entry '2' for key 'b'
diff --git a/mysql-test/suite/innodb/t/alter_candidate_key.test b/mysql-test/suite/innodb/t/alter_candidate_key.test
index 7c8f5e30993..824ad1ea799 100644
--- a/mysql-test/suite/innodb/t/alter_candidate_key.test
+++ b/mysql-test/suite/innodb/t/alter_candidate_key.test
@@ -50,7 +50,7 @@ connection con1;
 SET DEBUG_SYNC='now WAIT_FOR dml';
 BEGIN;
 INSERT INTO t1 SET a=NULL;
-ROLLBACK;
+COMMIT;
 set DEBUG_SYNC='now SIGNAL dml_done';
 connection default;
 --error ER_INVALID_USE_OF_NULL
diff --git a/mysql-test/suite/innodb/t/alter_crash.test b/mysql-test/suite/innodb/t/alter_crash.test
index 1049efd3e12..e0e294ae4f0 100644
--- a/mysql-test/suite/innodb/t/alter_crash.test
+++ b/mysql-test/suite/innodb/t/alter_crash.test
@@ -213,6 +213,7 @@ connection default;
 SET DEBUG_SYNC = 'now WAIT_FOR scanned';
 BEGIN;
 INSERT INTO t1 VALUES(2,1);
+COMMIT;
 SET DEBUG_SYNC = 'now SIGNAL commit';
 SET DEBUG_SYNC = 'now WAIT_FOR c';
 # Make all pending changes durable for recovery.
diff --git a/mysql-test/suite/innodb/t/alter_dml_apply.opt b/mysql-test/suite/innodb/t/alter_dml_apply.opt
new file mode 100644
index 00000000000..fa3418284bf
--- /dev/null
+++ b/mysql-test/suite/innodb/t/alter_dml_apply.opt
@@ -0,0 +1,2 @@
+--innodb_online_alter_log_max_size=64k
+--innodb_sort_buffer_size=64k
diff --git a/mysql-test/suite/innodb/t/alter_dml_apply.test b/mysql-test/suite/innodb/t/alter_dml_apply.test
new file mode 100644
index 00000000000..4d6fd418691
--- /dev/null
+++ b/mysql-test/suite/innodb/t/alter_dml_apply.test
@@ -0,0 +1,58 @@
+--source include/have_innodb.inc
+--source include/have_debug.inc
+--source include/have_sequence.inc
+
+CREATE TABLE t1(f1 INT NOT NULL, f2 INT NOT NULL,
+		f3 CHAR(200), f4 CHAR(200),
+		PRIMARY KEY(f1))ENGINE=InnoDB;
+INSERT INTO t1 VALUES(6000, 6000, "InnoDB", "MariaDB");
+
+# InnoDB DML thread applies the online log, aborts other online index
+
+SET DEBUG_SYNC="inplace_after_index_build SIGNAL dml_start WAIT_FOR dml_commit";
+SEND ALTER TABLE t1 ADD UNIQUE KEY(f2), ADD UNIQUE INDEX(f4(10));
+
+# InnoDB DML thread applies insert log
+
+connect(con1,localhost,root,,,);
+SET DEBUG_SYNC="now WAIT_FOR dml_start";
+# Rollback should avoid online index
+BEGIN;
+DELETE FROM t1 WHERE f1= 6000;
+INSERT INTO t1 VALUES(6000, 6000, "InnoDB", "MariaDB");
+ROLLBACK;
+
+# Insert log will fetch the previous version in this case
+BEGIN;
+DELETE FROM t1 WHERE f1= 6000;
+INSERT INTO t1 VALUES(6000, 6000, "InnoDB", "MariaDB");
+INSERT INTO t1 SELECT seq, seq, repeat('a', 200), repeat('b', 200) FROM seq_1_to_4000;
+COMMIT;
+SET DEBUG_SYNC="now SIGNAL dml_commit";
+
+connection default;
+--error ER_DUP_ENTRY
+reap;
+
+# DML Thread applies update log
+
+SET DEBUG_SYNC="inplace_after_index_build SIGNAL dml_start WAIT_FOR dml_commit";
+SEND ALTER TABLE t1 ADD UNIQUE KEY(f2), ADD INDEX(f3(10));
+
+connection con1;
+SET DEBUG_SYNC="now WAIT_FOR dml_start";
+BEGIN;
+DELETE FROM t1;
+INSERT INTO t1 SELECT seq, seq, repeat('d', 200), repeat('e', 200) FROM
+seq_1_to_4000;
+UPDATE t1 SET f3=repeat('c', 200), f4= repeat('d', 200), f2=3;
+COMMIT;
+SET DEBUG_SYNC="now SIGNAL dml_commit";
+
+connection default;
+--error ER_DUP_ENTRY
+reap;
+disconnect con1;
+CHECK TABLE t1;
+DROP TABLE t1;
+SET DEBUG_SYNC=reset;
diff --git a/mysql-test/suite/innodb/t/alter_mdl_timeout.test b/mysql-test/suite/innodb/t/alter_mdl_timeout.test
index 15e7f524fd0..ff77921b2d2 100644
--- a/mysql-test/suite/innodb/t/alter_mdl_timeout.test
+++ b/mysql-test/suite/innodb/t/alter_mdl_timeout.test
@@ -14,7 +14,7 @@ begin;
 INSERT INTO t1 VALUES('e','e',5, 5);
 SET DEBUG_SYNC="now SIGNAL con1_insert";
 SET DEBUG_SYNC="now WAIT_FOR con1_wait";
-SET DEBUG_SYNC="before_row_upd_sec_new_index_entry SIGNAL con1_update WAIT_FOR alter_rollback";
+SET DEBUG_SYNC="after_row_upd_clust SIGNAL con1_update WAIT_FOR alter_rollback";
 SEND UPDATE t1 set f4 = 10 order by f1 desc limit 2;
 
 connection default;
diff --git a/mysql-test/suite/innodb/t/alter_not_null_debug.test b/mysql-test/suite/innodb/t/alter_not_null_debug.test
index 9c5ba0faff0..87113b2b3f8 100644
--- a/mysql-test/suite/innodb/t/alter_not_null_debug.test
+++ b/mysql-test/suite/innodb/t/alter_not_null_debug.test
@@ -7,7 +7,7 @@ let $sql_mode = `SELECT @@SQL_MODE`;
 let $error_code = 0;
 
 if ($sql_mode == "STRICT_TRANS_TABLES") {
- let $error_code = WARN_DATA_TRUNCATED;
+ let $error_code = ER_INVALID_USE_OF_NULL;
 }
 
 
diff --git a/mysql-test/suite/innodb/t/innodb-alter-debug.test b/mysql-test/suite/innodb/t/innodb-alter-debug.test
index 1789ec294e4..2241ef5d295 100644
--- a/mysql-test/suite/innodb/t/innodb-alter-debug.test
+++ b/mysql-test/suite/innodb/t/innodb-alter-debug.test
@@ -52,11 +52,10 @@ SET DEBUG_SYNC = 'now SIGNAL s2';
 --echo /* connection default */
 connection default;
 --echo /* reap */ alter table t1 force, add b int, ALGORITHM=inplace;
---error ER_DUP_ENTRY
 --reap
 
 SET DEBUG_SYNC = 'row_log_table_apply1_before SIGNAL s1 WAIT_FOR s2';
---send alter table t1 force, add b int, ALGORITHM=inplace;
+--send alter table t1 force, add c int, ALGORITHM=inplace;
 
 --echo /* connection con1 */
 connection con1;
@@ -68,7 +67,6 @@ SET DEBUG_SYNC = 'now SIGNAL s2';
 --echo /* connection default */
 connection default;
 --echo /* reap */ alter table t1 force, add b int, ALGORITHM=inplace;
---error ER_DUP_ENTRY
 --reap
 SET DEBUG_SYNC = 'RESET';
 
@@ -92,7 +90,6 @@ SET DEBUG_SYNC = 'now SIGNAL S2';
 disconnect con1;
 
 CONNECTION default;
---error ER_DUP_ENTRY
 reap;
 SET DEBUG_SYNC='RESET';
 
diff --git a/mysql-test/suite/innodb/t/innodb-index-debug.test b/mysql-test/suite/innodb/t/innodb-index-debug.test
index 204bdfe5540..f03ef061769 100644
--- a/mysql-test/suite/innodb/t/innodb-index-debug.test
+++ b/mysql-test/suite/innodb/t/innodb-index-debug.test
@@ -122,17 +122,16 @@ drop table t480;
 --echo # in online table rebuild
 --echo #
 
-CREATE TABLE t1 (j INT UNIQUE, i INT UNIQUE) ENGINE=InnoDB;
+CREATE TABLE t1 (j INT UNIQUE, i INT) ENGINE=InnoDB;
 INSERT INTO t1 VALUES(2, 2);
 --connect (con1,localhost,root,,test)
 SET DEBUG_SYNC='row_log_table_apply1_before SIGNAL built WAIT_FOR log';
 --send
-ALTER TABLE t1 DROP j, FORCE;
+ALTER TABLE t1 DROP j, ADD UNIQUE INDEX(i), FORCE;
 
 --connection default
 SET DEBUG_SYNC='now WAIT_FOR built';
 SET DEBUG_DBUG='+d,row_ins_row_level';
---error ER_DUP_ENTRY
 INSERT INTO t1 (i) VALUES (0),(0);
 SET DEBUG_SYNC='now SIGNAL log';
 SET DEBUG_DBUG=@saved_debug_dbug;
@@ -140,6 +139,8 @@ SET DEBUG_DBUG=@saved_debug_dbug;
 --connection con1
 --error ER_DUP_ENTRY
 reap;
+DELETE FROM t1;
+ALTER TABLE t1 ADD UNIQUE INDEX(i);
 SET DEBUG_SYNC='row_log_table_apply1_before SIGNAL built2 WAIT_FOR log2';
 --send
 ALTER TABLE t1 DROP j, FORCE;
@@ -152,7 +153,6 @@ UPDATE t1 SET i=0;
 SET DEBUG_SYNC='now SIGNAL log2';
 
 --connection con1
---error ER_DUP_ENTRY
 reap;
 --disconnect con1
 --connection default
diff --git a/mysql-test/suite/innodb/t/innodb-index-online.test b/mysql-test/suite/innodb/t/innodb-index-online.test
index 6f30dad531d..2cb84b18402 100644
--- a/mysql-test/suite/innodb/t/innodb-index-online.test
+++ b/mysql-test/suite/innodb/t/innodb-index-online.test
@@ -98,7 +98,8 @@ eval $innodb_metrics_select;
 # Insert a duplicate entry (4) for the already started UNIQUE INDEX(c2).
 BEGIN;
 INSERT INTO t1 VALUES(7,4,2);
-ROLLBACK;
+COMMIT;
+DELETE FROM t1 where c1 = 7;
 SET DEBUG_SYNC = 'now SIGNAL rollback_done';
 
 connection con1;
@@ -121,7 +122,7 @@ INSERT INTO t1 VALUES(6,3,1);
 SET DEBUG_SYNC = 'now SIGNAL dml_done';
 connection con1;
 # This is due to the duplicate entry (6,3,1).
---error ER_DUP_UNKNOWN_IN_INDEX
+--error ER_DUP_ENTRY
 reap;
 DELETE FROM t1 WHERE c1=6;
 ALTER TABLE t1 ADD UNIQUE INDEX(c2);
@@ -237,17 +238,11 @@ SET DEBUG_SYNC = 'now WAIT_FOR c2e_created';
 # At this point, the clustered index scan must have completed,
 # but the modification log keeps accumulating due to the DEBUG_SYNC.
 eval $innodb_metrics_select;
-let $c= 4;
+let $c= 8;
 while ($c)
 {
-  BEGIN;
-  DELETE FROM t1;
-  ROLLBACK;
   UPDATE t1 SET c2 = c2 + 1;
-  BEGIN;
-  UPDATE t1 SET c2 = c2 + 1;
-  DELETE FROM t1;
-  ROLLBACK;
+  UPDATE t1 SET c2 = c2 + 2;
   dec $c;
 }
 # Incomplete index c2e should exist until the DDL thread notices the overflow.
@@ -325,17 +320,16 @@ connection default;
 SET DEBUG_SYNC = 'now WAIT_FOR c2f_created';
 # Generate some log (delete-mark, delete-unmark, insert etc.)
 eval $innodb_metrics_select;
-let $c= 2;
+let $c= 6;
 while ($c)
 {
 BEGIN;
 INSERT INTO t1 SELECT 320 + c1, c2, c3 FROM t1 WHERE c1 > 160;
 DELETE FROM t1 WHERE c1 > 320;
-ROLLBACK;
+COMMIT;
 BEGIN;
 UPDATE t1 SET c2 = c2 + 1;
-DELETE FROM t1;
-ROLLBACK;
+COMMIT;
 dec $c;
 }
 eval $innodb_metrics_select;
@@ -481,7 +475,6 @@ send ALTER TABLE t1 ADD UNIQUE(c);
 
 connection con1;
 SET DEBUG_SYNC = 'now WAIT_FOR t1u_created';
-BEGIN;
 INSERT INTO t1 VALUES('bar'),('bar');
 SET DEBUG_SYNC = 'now SIGNAL dup_done';
 
diff --git a/mysql-test/suite/innodb/t/innodb-table-online.test b/mysql-test/suite/innodb/t/innodb-table-online.test
index 7ed87fcc26b..8e80c3fe8a3 100644
--- a/mysql-test/suite/innodb/t/innodb-table-online.test
+++ b/mysql-test/suite/innodb/t/innodb-table-online.test
@@ -101,7 +101,6 @@ SET DEBUG_SYNC = 'now WAIT_FOR scanned';
 eval $innodb_metrics_select;
 
 # Insert a duplicate entry (4) for the already started UNIQUE INDEX(c1).
-BEGIN;
 INSERT INTO t1 VALUES(4,7,2);
 SET DEBUG_SYNC = 'now SIGNAL insert_done';
 
@@ -111,7 +110,7 @@ connection con1;
 # error on the (4,7,2).
 --error ER_DUP_ENTRY
 reap;
-
+DELETE FROM t1 WHERE c1=4 and c2=7;
 connection default;
 ROLLBACK;
 
@@ -204,17 +203,11 @@ UPDATE t1 SET c2 = c2 + 1;
 # At this point, the clustered index scan must have completed,
 # but the modification log keeps accumulating due to the DEBUG_SYNC.
 eval $innodb_metrics_select;
-let $c= 3;
+let $c= 8;
 while ($c)
 {
-  BEGIN;
-  DELETE FROM t1;
-  ROLLBACK;
   UPDATE t1 SET c2 = c2 + 1;
-  BEGIN;
-  UPDATE t1 SET c2 = c2 + 1;
-  DELETE FROM t1;
-  ROLLBACK;
+  UPDATE t1 SET c2 = c2 + 2;
   dec $c;
 }
 # Temporary table should exist until the DDL thread notices the overflow.
@@ -279,11 +272,8 @@ eval $innodb_metrics_select;
 BEGIN;
 INSERT INTO t1 SELECT 320 + c1, c2, c3 FROM t1 WHERE c1 > 240;
 DELETE FROM t1 WHERE c1 > 320;
-ROLLBACK;
-BEGIN;
 UPDATE t1 SET c2 = c2 + 1;
-DELETE FROM t1;
-ROLLBACK;
+COMMIT;
 eval $innodb_metrics_select;
 # Release con1.
 SET DEBUG_SYNC = 'now SIGNAL dml3_done';
@@ -346,16 +336,13 @@ SET DEBUG_SYNC = 'now WAIT_FOR c3p5_created0';
 BEGIN;
 INSERT INTO t1 VALUES(347,33101,'Pikku kakkosen posti','YLETV2');
 INSERT INTO t1 VALUES(33101,347,NULL,'');
+COMMIT;
 SET DEBUG_SYNC = 'now SIGNAL ins_done0';
 
 connection con1;
---error WARN_DATA_TRUNCATED
+--error ER_INVALID_USE_OF_NULL
 reap;
-
-connection default;
-ROLLBACK;
-
-connection con1;
+DELETE FROM t1 WHERE c1= 347 and c22f = 33101;
 ALTER TABLE t1 MODIFY c3 CHAR(255) NOT NULL;
 
 SET DEBUG_SYNC = 'row_log_table_apply1_before SIGNAL c3p5_created WAIT_FOR ins_done';
diff --git a/mysql-test/suite/innodb/t/insert_into_empty_debug.test b/mysql-test/suite/innodb/t/insert_into_empty_debug.test
new file mode 100644
index 00000000000..3058883a5d0
--- /dev/null
+++ b/mysql-test/suite/innodb/t/insert_into_empty_debug.test
@@ -0,0 +1,63 @@
+--source include/have_innodb.inc
+--source include/have_debug.inc
+--source include/have_debug_sync.inc
+
+# Encounter aborted online index during rollback of bulk insert
+
+CREATE TABLE t1(f1 INT NOT NULL, f2 INT NOT NULL,
+                PRIMARY KEY(f1))ENGINE=InnoDB;
+INSERT INTO t1 VALUES(1, 2), (2, 2);
+
+SET DEBUG_SYNC="innodb_rollback_inplace_alter_table SIGNAL dml_start WAIT_FOR dml_commit";
+send ALTER TABLE t1 ADD UNIQUE KEY(f2);
+
+connect(con1,localhost,root,,,);
+SET DEBUG_SYNC="now WAIT_FOR dml_start";
+BEGIN;
+DELETE FROM t1;
+SET DEBUG_SYNC="now SIGNAL dml_commit";
+
+connection default;
+--error ER_DUP_ENTRY
+reap;
+connection con1;
+COMMIT;
+TRUNCATE TABLE t1;
+SET unique_checks=0, foreign_key_checks=0;
+BEGIN;
+INSERT INTO t1 VALUES(1, 2);
+ROLLBACK;
+
+connection default;
+SELECT * FROM t1;
+SHOW CREATE TABLE t1;
+
+# Online alter logs ROW_LOG_EMPTY when table does bulk insert
+INSERT INTO t1 VALUES(1, 1);
+connection con1;
+START TRANSACTION WITH CONSISTENT SNAPSHOT;
+connect(con2,localhost,root,,,);
+DELETE FROM t1;
+connection default;
+SET DEBUG_SYNC="innodb_inplace_alter_table_enter SIGNAL purge_resume WAIT_FOR dml_commit";
+send ALTER TABLE t1 ADD INDEX(f2, f1);
+connection con1;
+COMMIT;
+connection con2;
+let $wait_all_purged=1;
+SET GLOBAL innodb_purge_rseg_truncate_frequency=1;
+--source include/wait_all_purged.inc
+SET unique_checks=0, foreign_key_checks=0;
+BEGIN;
+INSERT INTO t1 VALUES(2, 2);
+ROLLBACK;
+SET DEBUG_SYNC="now SIGNAL dml_commit";
+
+connection default;
+reap;
+SHOW CREATE TABLE t1;
+disconnect con1;
+disconnect con2;
+DROP TABLE t1;
+SET DEBUG_SYNC=RESET;
+SET GLOBAL innodb_purge_rseg_truncate_frequency=default;
diff --git a/mysql-test/suite/innodb/t/instant_alter_debug.test b/mysql-test/suite/innodb/t/instant_alter_debug.test
index f102185c27f..c6eca884907 100644
--- a/mysql-test/suite/innodb/t/instant_alter_debug.test
+++ b/mysql-test/suite/innodb/t/instant_alter_debug.test
@@ -303,7 +303,6 @@ ROLLBACK;
 SET DEBUG_SYNC = 'now SIGNAL logged';
 
 connection ddl;
---error ER_INVALID_USE_OF_NULL
 reap;
 disconnect ddl;
 
diff --git a/mysql-test/suite/innodb/t/online_table_rebuild.opt b/mysql-test/suite/innodb/t/online_table_rebuild.opt
new file mode 100644
index 00000000000..fa3418284bf
--- /dev/null
+++ b/mysql-test/suite/innodb/t/online_table_rebuild.opt
@@ -0,0 +1,2 @@
+--innodb_online_alter_log_max_size=64k
+--innodb_sort_buffer_size=64k
diff --git a/mysql-test/suite/innodb/t/online_table_rebuild.test b/mysql-test/suite/innodb/t/online_table_rebuild.test
new file mode 100644
index 00000000000..94bac2f949c
--- /dev/null
+++ b/mysql-test/suite/innodb/t/online_table_rebuild.test
@@ -0,0 +1,62 @@
+--source include/have_innodb.inc
+--source include/have_debug.inc
+--source include/have_sequence.inc
+
+CREATE TABLE t1(f1 INT NOT NULL, f2 CHAR(200), f3 CHAR(200))ENGINE=InnoDB;
+INSERT INTO t1 VALUES(3, "innodb", "alter log");
+
+# InnoDB fails with DUPLICATE KEY error in commit phase
+
+SET DEBUG_SYNC="inplace_after_index_build SIGNAL dml_start WAIT_FOR dml_commit";
+send ALTER TABLE t1 ADD PRIMARY KEY(f3(10)), ADD UNIQUE KEY(f2(10));
+CONNECT(con1,localhost,root,,,);
+SET DEBUG_SYNC="now WAIT_FOR dml_start";
+BEGIN;
+INSERT INTO t1 VALUES(1, repeat('b', 100), repeat('c', 100));
+INSERT INTO t1 VALUES(2, repeat('b', 100), repeat('a', 100));
+COMMIT;
+SET DEBUG_SYNC="now SIGNAL dml_commit";
+
+connection default;
+--error ER_DUP_ENTRY
+reap;
+
+# ONLINE_LOG_TOO_BIG error during commit phase
+
+connection default;
+
+SET DEBUG_SYNC="inplace_after_index_build SIGNAL dml_start WAIT_FOR dml_commit";
+SEND ALTER TABLE t1 ADD PRIMARY KEY(f1);
+
+connection con1;
+SET DEBUG_SYNC="now WAIT_FOR dml_start";
+INSERT INTO t1 SELECT 10, repeat('a', 100), repeat('b', 100) FROM seq_1_to_4800;
+SET DEBUG_SYNC="now SIGNAL dml_commit";
+
+connection default;
+--error ER_INNODB_ONLINE_LOG_TOO_BIG
+reap;
+DELETE FROM t1;
+INSERT INTO t1 VALUES(1, repeat('a', 100), repeat('b', 100));
+ALTER TABLE t1 ADD PRIMARY KEY(f1);
+
+# Update operation leads to duplicate key error
+
+set DEBUG_SYNC="innodb_inplace_alter_table_enter SIGNAL dml_start WAIT_FOR dml_commit";
+SEND ALTER TABLE t1 DROP PRIMARY KEY, ADD PRIMARY KEY(f3(10));
+
+connection con1;
+SET DEBUG_SYNC="now WAIT_FOR dml_start";
+BEGIN;
+INSERT INTO t1 VALUES(2, repeat('b', 100), repeat('c', 100));
+UPDATE t1 set f3=repeat('c', 100) where f1=1;
+COMMIT;
+SET DEBUG_SYNC="now SIGNAL dml_commit";
+
+connection default;
+--error ER_DUP_ENTRY
+reap;
+SHOW CREATE TABLE t1;
+DROP TABLE t1;
+disconnect con1;
+SET DEBUG_SYNC=reset;
diff --git a/mysql-test/suite/innodb/t/table_definition_cache_debug.test b/mysql-test/suite/innodb/t/table_definition_cache_debug.test
index 6a466af4cc5..8950691e05c 100644
--- a/mysql-test/suite/innodb/t/table_definition_cache_debug.test
+++ b/mysql-test/suite/innodb/t/table_definition_cache_debug.test
@@ -29,6 +29,7 @@ SET DEBUG_SYNC = 'now WAIT_FOR scanned';
 # and then hogs the table lock, so that the unique index cannot be dropped.
 BEGIN;
 INSERT INTO to_be_evicted VALUES(3, 2);
+COMMIT;
 SET DEBUG_SYNC = 'now SIGNAL got_duplicate';
 
 connection ddl;
diff --git a/mysql-test/suite/innodb_fts/r/misc_debug.result b/mysql-test/suite/innodb_fts/r/misc_debug.result
index 2a2afacb052..f40ed1fe9c3 100644
--- a/mysql-test/suite/innodb_fts/r/misc_debug.result
+++ b/mysql-test/suite/innodb_fts/r/misc_debug.result
@@ -65,3 +65,12 @@ ERROR HY000: Got error -1 "Internal error < 0 (Not system error)" from storage e
 SET debug_dbug=@saved_debug_dbug;
 DROP TABLE t1;
 # End of 10.3 tests
+CREATE TABLE t1(f1 INT NOT NULL, f2 CHAR(100))ENGINE=InnoDB;
+SET DEBUG_DBUG="+d,stats_lock_fail";
+ALTER TABLE t1 ADD FULLTEXT(f2);
+ERROR HY000: Got error 15 "Block device required" from storage engine InnoDB
+SET DEBUG_DBUG="-d,stats_lock_fail";
+ALTER TABLE t1 DISCARD TABLESPACE;
+ALTER TABLE t1 ADD FULLTEXT(f2);
+ERROR HY000: Tablespace has been discarded for table `t1`
+DROP TABLE t1;
diff --git a/mysql-test/suite/innodb_fts/t/misc_debug.test b/mysql-test/suite/innodb_fts/t/misc_debug.test
index 84da1320264..6ab980a3948 100644
--- a/mysql-test/suite/innodb_fts/t/misc_debug.test
+++ b/mysql-test/suite/innodb_fts/t/misc_debug.test
@@ -95,3 +95,15 @@ TRUNCATE t1;
 SET debug_dbug=@saved_debug_dbug;
 DROP TABLE t1;
 --echo # End of 10.3 tests
+
+# Fulltext fails in commit phase
+
+CREATE TABLE t1(f1 INT NOT NULL, f2 CHAR(100))ENGINE=InnoDB;
+SET DEBUG_DBUG="+d,stats_lock_fail";
+--error ER_GET_ERRNO
+ALTER TABLE t1 ADD FULLTEXT(f2);
+SET DEBUG_DBUG="-d,stats_lock_fail";
+ALTER TABLE t1 DISCARD TABLESPACE;
+--error ER_TABLESPACE_DISCARDED
+ALTER TABLE t1 ADD FULLTEXT(f2);
+DROP TABLE t1;