MDEV-22632 wsrep XID checkpointing can happen out of order for certification failurebb-10.4-MDEV-22632

When a transaction fails in certification phase, it has connsumed one GTID, but as
transaction must rollback, it will not go for commit ordering, and because of this
also the wsrep XID checkpointing can happen out of order.
This PR will make the thread, which has failed for certiication failure to wait for its
commit order turn for checkpointing wsrep IXD in innodb rollback segment.

There is a specific test for wsrep XID checkpointing ordering in mtr test:
mysql-wsrep-bugs-607, which is added in this PR.

Test galera_slave_replay depends also on this fix, as the second test phase
may also assert for bad wsrep XID checkpointing order.
galera_slave_replay.test had also other problems, which caused the test to
fail immediately, thse are now fixes in this PR as well.

1 files changed, 13 insertions, 1 deletions

diff --git a/sql/wsrep_server_service.cc b/sql/wsrep_server_service.cc
index e6ccaca13b1..57b9c7fd626 100644
--- a/sql/wsrep_server_service.cc
+++ b/sql/wsrep_server_service.cc
@@ -303,9 +303,21 @@ wsrep::gtid Wsrep_server_service::get_position(wsrep::client_service&)
   return wsrep_get_SE_checkpoint();
 }
 
-void Wsrep_server_service::set_position(wsrep::client_service&,
+void Wsrep_server_service::set_position(wsrep::client_service& c WSREP_UNUSED,
                                         const wsrep::gtid& gtid)
 {
+  Wsrep_client_service& cs WSREP_UNUSED (static_cast<Wsrep_client_service&>(c));
+  DBUG_ASSERT(cs.m_client_state.transaction().state()
+              == wsrep::transaction::s_aborted);
+  // Wait until all prior committers have finished.
+  wsrep::gtid wait_for(gtid.id(),
+                       wsrep::seqno(gtid.seqno().get() - 1));
+  if (auto err = Wsrep_server_state::instance().provider()
+      .wait_for_gtid(wait_for, std::numeric_limits<int>::max()))
+  {
+    WSREP_WARN("Wait for gtid returned error %d while waiting for "
+               "prior transactions to commit before setting position", err);
+  }
   wsrep_set_SE_checkpoint(gtid);
 }