MDEV-17554 Auto-create new partition for system versioned tables with history partitioned by INTERVAL/LIMITpreview-10.8-MDEV-17554-auto-create-partition

== Syntax change ==

Keyword AUTO enables partition auto-creation.

create table t1 (x int) with system versioning
partition by system_time interval 1 hour auto;

create table t1 (x int) with system versioning
partition by system_time interval 1 month
starts '2021-01-01 00:00:00' auto partitions 12;

create table t1 (x int) with system versioning
partition by system_time limit 1000 auto;

Or with explicit partitions:

create table t1 (x int) with system versioning
partition by system_time interval 1 hour auto
(partition p0 history, partition pn current);

== Description ==

Before executing history-generating DML command add N history
partitions, so that N would be sufficient for potentially generated
history. N > 1 may be required when history is rotated by INTERVAL and
timestamp was jumped to future further than interval value.

If last history partition exceeds LIMIT records before DML command
then new history partition is created and the history filling is
switched to the new partition. Thus LIMIT does not carry strict
limitation and the history partition size must be planned as LIMIT
records plus average number of history one DML generates.

Auto-creation is implemented by synchronous
fast_alter_partition_table() call from the thread of the executed DML
command before the command itself (by the fallback and retry mechanism
similar to Discovery feature, see Open_table_context).

The name for newly added partitions are generated like default
partition names with extension of MDEV-22155 (which avoids name
clashes by extending assignment counter to next free-enough gap).

These DML commands trigger auto-creation:

* DELETE (including multi-delete, excluding DELETE HISTORY)
* UPDATE (including multi-update)
* REPLACE (including REPLACE .. SELECT)
* INSERT .. ON DUPLICATE KEY UPDATE
* LOAD DATA .. REPLACE

MDEV-23642 Locking timeout caused by auto-creation affects original DML

The reasons for this are:

- Do not disrupt main business process (the history is auxiliary
service);

- Consequences are non-fatal (history is not lost, but comes into wrong
partition; fixed by partitioning rebuild);

- There is more freedom for application to fail in this case or
not: it may read warning info and find corresponding error number.

- While non-failing command is easy to handle by an application and
fail it, the opposite is hard to handle: there is no automatic
actions to fix failed command and retry, DBA intervention is
required and until then application is non-functioning.

MDEV-23639 Auto-create does not work under LOCK TABLES or inside triggers

Don't do tdc_remove_table() for OT_ADD_HISTORY_PARTITION because it is
not possible in locked tables mode.

LTM_LOCK_TABLES mode (and LTM_PRELOCKED_UNDER_LOCK_TABLES) works out
of the box as fast_alter_partition_table() can reopen tables via
locked_tables_list.

In LTM_PRELOCKED we reopen and relock table manually.

More fixes

* some_table_marked_for_reopen flag fix

some_table_marked_for_reopen affets only reopen of
m_locked_tables. I.e. Locked_tables_list::reopen_tables() reopens only
tables from m_locked_tables.

* Unused can_recover_from_failed_open() condition

Is recover_from_failed_open() can be really used after
open_and_process_routine()?

1 files changed, 23 insertions, 9 deletions

diff --git a/sql/lock.cc b/sql/lock.cc
index 2eba5df35f1..bfa3e7baa0f 100644
--- a/sql/lock.cc
+++ b/sql/lock.cc
@@ -654,7 +654,7 @@ bool mysql_lock_abort_for_thread(THD *thd, TABLE *table)
   a and b are freed with my_free()
 */
 
-MYSQL_LOCK *mysql_lock_merge(MYSQL_LOCK *a,MYSQL_LOCK *b)
+MYSQL_LOCK *mysql_lock_merge(MYSQL_LOCK *a, MYSQL_LOCK *b, THD *thd)
 {
   MYSQL_LOCK *sql_lock;
   TABLE **table, **end_table;
@@ -662,16 +662,28 @@ MYSQL_LOCK *mysql_lock_merge(MYSQL_LOCK *a,MYSQL_LOCK *b)
   DBUG_PRINT("enter", ("a->lock_count: %u  b->lock_count: %u",
                        a->lock_count, b->lock_count));
 
-  if (!(sql_lock= (MYSQL_LOCK*)
-	my_malloc(key_memory_MYSQL_LOCK, sizeof(*sql_lock) +
-		  sizeof(THR_LOCK_DATA*)*((a->lock_count+b->lock_count)*2) +
-		  sizeof(TABLE*)*(a->table_count+b->table_count),MYF(MY_WME))))
-    DBUG_RETURN(0);				// Fatal error
+  const size_t lock_size= sizeof(*sql_lock) +
+    sizeof(THR_LOCK_DATA *) * ((a->lock_count + b->lock_count) * 2) +
+    sizeof(TABLE *) * (a->table_count + b->table_count);
+  if (thd)
+  {
+    sql_lock= (MYSQL_LOCK *) thd->alloc(lock_size);
+    if (!sql_lock)
+      DBUG_RETURN(0);
+    sql_lock->flags= GET_LOCK_ON_THD;
+  }
+  else
+  {
+    sql_lock= (MYSQL_LOCK *)
+      my_malloc(key_memory_MYSQL_LOCK, lock_size, MYF(MY_WME));
+    if (!sql_lock)
+      DBUG_RETURN(0);
+    sql_lock->flags= 0;
+  }
   sql_lock->lock_count=a->lock_count+b->lock_count;
   sql_lock->table_count=a->table_count+b->table_count;
   sql_lock->locks=(THR_LOCK_DATA**) (sql_lock+1);
   sql_lock->table=(TABLE**) (sql_lock->locks+sql_lock->lock_count*2);
-  sql_lock->flags= 0;
   memcpy(sql_lock->locks,a->locks,a->lock_count*sizeof(*a->locks));
   memcpy(sql_lock->locks+a->lock_count,b->locks,
 	 b->lock_count*sizeof(*b->locks));
@@ -705,8 +717,10 @@ MYSQL_LOCK *mysql_lock_merge(MYSQL_LOCK *a,MYSQL_LOCK *b)
                   a->lock_count, b->lock_count);
 
   /* Delete old, not needed locks */
-  my_free(a);
-  my_free(b);
+  if (!(a->flags & GET_LOCK_ON_THD))
+    my_free(a);
+  if (!(b->flags & GET_LOCK_ON_THD))
+    my_free(b);
   DBUG_RETURN(sql_lock);
 }