Fix for: LP #634955: Assert in _ma_update_at_original_place()

Added locking of lock mutex when updating status in external_unlock() for Aria and MyISAM tables.
Fixed that 'source' command doesn't cause mysql command line tool to exit on error.
DEBUG_EXECUTE() and DEBUG_EVALUATE_IF() should not execute things based on wildcards. (Allows one to run --debug with mysql-test-run scripts that uses @debug)
Fixed several core dump, deadlock and crashed table bugs in handling of LOCK TABLE with MERGE tables:
- Added priority of locks to avoid crashes with MERGE tables.
- Added thr_lock_merge() to allow one to merge two results of thr_lock().
Fixed 'not found row' bug in REPLACE with Aria tables.
Mark MyISAM tables that are part of MERGE with HA_OPEN_MERGE_TABLE and set the locks to have priority THR_LOCK_MERGE_PRIV.
- By sorting MERGE tables last in thr_multi_unlock() it's safer to release and relock them many times (can happen when TRIGGERS are created)
Avoid printing (null) in debug file (to easier find out wrong NULL pointer usage with %s).



client/mysql.cc:
  Fixed that 'source' command doesn't cause mysql command line tool to exit on error.
client/mysqltest.cc:
  Don't send NULL to fn_format(). (Can cause crash on Solaris when using --debug)
dbug/dbug.c:
  DEBUG_EXECUTE() and DEBUG_EVALUATE_IF() should not execute things based on wildcards.
include/my_base.h:
  Added flag to signal if one opens a MERGE table.
  Added extra() command to signal that one is not part of a MERGE table anymore.
include/thr_lock.h:
  Added priority for locks (needed to fix bug in thr_lock when using MERGE tables)
  Added option to thr_unlock() if get_status() should be called.
  Added prototype for thr_merge_locks().
mysql-test/mysql-test-run.pl:
  Ignore crashed table warnings for tables named 'crashed'.
mysql-test/r/merge.result:
  Renamed triggers to make debugging easier.
  Added some CHECK TABLES to catch errors earlier.
  Additional tests.
mysql-test/r/merge_debug.result:
  Test of error handling when reopening MERGE tables.
mysql-test/r/udf_query_cache.result:
  Added missing flush status
mysql-test/suite/parts/r/partition_repair_myisam.result:
  Update results
mysql-test/t/merge.test:
  Renamed triggers to make debugging easier.
  Added some CHECK TABLES to catch errors earlier.
  Additional tests.
mysql-test/t/merge_debug.test:
  Test of error handling when reopening MERGE tables.
mysql-test/t/udf_query_cache.test:
  Added missing flush status
mysys/my_getopt.c:
  Removed not used variable
mysys/my_symlink2.c:
  Changed (null) to (NULL) to make it easier to find NULL arguments to DBUG_PRINT() functions.
  (On linux, NULL to sprintf is printed 'null')
mysys/thr_lock.c:
  Added priority of locks to avoid crashes with MERGE tables.
  Added thr_lock_merge() to allow one to merge two results of thr_lock().
  - This is needed for MyISAM as all locked table must share the same status. If not, you will not see newly inserted rows in other instances of the table.
  If calling thr_unlock() with THR_UNLOCK_UPDATE_STATUS, call update_status() and restore_status() for the locks. This is needed in some rare cases where we call thr_unlock() followed by thr_lock() without calling external_unlock/external_lock in between.
  Simplify loop in thr_multi_lock().
  Added 'start_trans', which is called at end of thr_multi_lock() when all locks are taken.
  - This was needed by Aria to ensure that transaction is started when we got all locks, not at get_status(). Without this, some rows could not be visible when we lock two tables at the same time, causing REPLACE using two tables to fail unexpectedly.
sql/handler.cc:
  Add an assert() in handler::print_error() for "impossible errors" (like table is crashed) when --debug-assert-if-crashed-table is used.
sql/lock.cc:
  Simplify mysql_lock_tables() code if get_lock_data() returns 0 locks.
  Added new parameter to thr_multi_unlock()
  In mysql_unlock_read_tables(), call first externa_unlock(), then thr_multi_unlock();  This is same order as we do in mysql_unlock_tables().
  Don't abort locks in mysql_lock_abort() for merged tables when a MERGE table is deleted; Would cause a spin lock.
  Added call to thr_merge_locks() in mysql_lock_merge() to ensure consistency in thr_locks().
  - New locks of same type and table is stored after the old lock to ensure that we get the status from the original lock.
sql/mysql_priv.h:
  Added debug_assert_if_crashed_table
sql/mysqld.cc:
  Added --debug-assert-if-crashed-table
sql/parse_file.cc:
  Don't print '(null)' in DBUG_PRINT of no dir given
sql/set_var.cc:
  Increase default size of buffer for @debug variable.
sql/sql_base.cc:
  In case of error from reopen_table() in reopen_tables(), call unlock_open_table() and restart loop.
  - This fixed bug when we twice deleted same table from open_cache.
  Don't take name lock for already name locked table in open_unireg_entry().
  - Fixed bug when doing repair in reopen_table().
  - In detach_merge_children(), always detach if 'clear_refs' is given. We can't trust parent->children_attached as this function can be called twice, first time with clear_refs set to 0.
sql/sql_class.cc:
  Changed printing of (null) to "" in set_thd_proc_info()
sql/sql_parse.cc:
  Added DBUG
sql/sql_trigger.cc:
  Don't call unlink_open_table() if reopen_table() fails as the table may already be freed.
storage/maria/ma_bitmap.c:
  Fixed DBUG_ASSERT() in allocate_tail()
storage/maria/ma_blockrec.c:
  Fixed wrong calculation of row length for very small rows in undo_row_update().
  - Fixes ASSERT() when doing undo.
storage/maria/ma_blockrec.h:
  Added _ma_block_start_trans() and _ma_block_start_trans_no_versioning()
storage/maria/ma_locking.c:
  Call _ma_update_status_with_lock() when releasing write locks.
  - Fixes potential problem with updating status without the proper lock.
storage/maria/ma_open.c:
  Changed to use start_trans() instead of get_status() to ensure that we see all rows in all locked tables when we got the locks.
  - Fixed 'not found row' bug in REPLACE with Aria tables.
storage/maria/ma_state.c:
  Added _ma_update_status_with_lock() and _ma_block_start_trans().
  This is to ensure that we see all rows in all locked tables when we got the locks.
storage/maria/ma_state.h:
  Added _ma_update_status_with_lock()
storage/maria/ma_write.c:
  More DBUG_PRINT
storage/myisam/mi_check.c:
  Fixed error message
storage/myisam/mi_extra.c:
  Added HA_EXTRA_DETACH_CHILD:
  - Detach MyISAM table to not be part of MERGE table (remove flag & lock priority).
storage/myisam/mi_locking.c:
  Call mi_update_status_with_lock() when releasing write locks.
  - Fixes potential problem with updating status without the proper lock.
  Change to use new HA_OPEN_MERGE_TABLE flag to test if MERGE table.
  Added mi_fix_status(), called by thr_merge().
storage/myisam/mi_open.c:
  Added marker if part of MERGE table.
  Call mi_fix_status() in thr_lock() for transactional tables.
storage/myisam/myisamdef.h:
  Change my_once_flag to uint, as it stored different values than just 0/1
  Added 'open_flag' to store state given to mi_open()
storage/myisammrg/ha_myisammrg.cc:
  Add THR_LOCK_MERGE_PRIV to THR_LOCK_DATA to get MERGE locks sorted after other types of locks.
storage/myisammrg/myrg_locking.c:
  Remove windows specific code.
storage/myisammrg/myrg_open.c:
  Use HA_OPEN_MERGE_TABLE to mi_open().
  Set HA_OPEN_MERGE_TABLE for linked MyISAM tables.
storage/xtradb/buf/buf0buf.c:
  Fixed compiler warning
storage/xtradb/buf/buf0lru.c:
  Initialize variable that could be used not initialized.

1 files changed, 57 insertions, 34 deletions

diff --git a/sql/sql_base.cc b/sql/sql_base.cc
index 33ea834247e..708b4d6126b 100644
--- a/sql/sql_base.cc
+++ b/sql/sql_base.cc
@@ -2032,6 +2032,8 @@ static void unlink_open_merge(THD *thd, TABLE *table, TABLE ***prev_pp)
         Remove parent from open_tables list and close it.
         This includes detaching and hence clearing parent references.
       */
+      DBUG_PRINT("info", ("Closing parent to '%s'.'%s'",
+                          table->s->db.str, table->s->table_name.str));
       close_thread_table(thd, prv_p);
     }
   }
@@ -3061,8 +3063,9 @@ bool reopen_table(TABLE *table)
   TABLE_LIST table_list;
   THD *thd= table->in_use;
   DBUG_ENTER("reopen_table");
-  DBUG_PRINT("tcache", ("table: '%s'.'%s' 0x%lx", table->s->db.str,
-                        table->s->table_name.str, (long) table));
+  DBUG_PRINT("tcache", ("table: '%s'.'%s'  table: 0x%lx  share: 0x%lx",
+                        table->s->db.str, table->s->table_name.str,
+                        (long) table, (long) table->s));
 
   DBUG_ASSERT(table->s->ref_count == 0);
   DBUG_ASSERT(!table->sort.io_cache);
@@ -3349,7 +3352,8 @@ bool reopen_tables(THD *thd, bool get_locks, bool mark_share_as_old)
       merge_table_found= TRUE;
     if (!tables || (!db_stat && reopen_table(table)))
     {
-      my_error(ER_CANT_REOPEN_TABLE, MYF(0), table->alias);
+      my_error(ER_CANT_REOPEN_TABLE, MYF(0),
+               table->alias ? table->alias : table->s->table_name.str);
       /*
         If we could not allocate 'tables', we may close open tables
         here. If a MERGE table is affected, detach the children first.
@@ -3359,9 +3363,10 @@ bool reopen_tables(THD *thd, bool get_locks, bool mark_share_as_old)
         that they cannot be moved into the unused_tables chain with
         these pointers set.
       */
-      if (table->child_l || table->parent)
-        detach_merge_children(table, TRUE);
-      VOID(hash_delete(&open_cache,(uchar*) table));
+      unlink_open_table(thd, table, 0);
+      /* Restart loop */
+      prev= &thd->open_tables;
+      next= *prev;
       error=1;
     }
     else
@@ -3396,7 +3401,7 @@ bool reopen_tables(THD *thd, bool get_locks, bool mark_share_as_old)
   }
   DBUG_PRINT("tcache", ("open tables to lock: %u",
                         (uint) (tables_ptr - tables)));
-  if (tables != tables_ptr)			// Should we get back old locks
+  if (tables != tables_ptr)  // Should we get back old locks
   {
     MYSQL_LOCK *lock;
     /*
@@ -3545,7 +3550,7 @@ bool table_is_used(TABLE *table, bool wait_for_name_lock)
     char *key= table->s->table_cache_key.str;
     uint key_length= table->s->table_cache_key.length;
 
-    DBUG_PRINT("loop", ("table_name: %s", table->alias));
+    DBUG_PRINT("loop", ("table_name: %s", table->alias ? table->alias : ""));
     HASH_SEARCH_STATE state;
     for (TABLE *search= (TABLE*) hash_first(&open_cache, (uchar*) key,
                                              key_length, &state);
@@ -3883,6 +3888,7 @@ static int open_unireg_entry(THD *thd, TABLE *entry, TABLE_LIST *table_list,
   int error;
   TABLE_SHARE *share;
   uint discover_retry_count= 0;
+  bool locked_table;
   DBUG_ENTER("open_unireg_entry");
 
   safe_mutex_assert_owner(&LOCK_open);
@@ -4003,8 +4009,10 @@ retry:
     }
     if (!entry->s || !entry->s->crashed)
       goto err;
-     // Code below is for repairing a crashed file
-     if ((error= lock_table_name(thd, table_list, TRUE)))
+
+    // Code below is for repairing a crashed file
+    locked_table= table_list->table != 0;
+    if (! locked_table && (error= lock_table_name(thd, table_list, TRUE)))
      {
        if (error < 0)
  	goto err;
@@ -4038,12 +4046,13 @@ retry:
      else
        thd->clear_error();			// Clear error message
      pthread_mutex_lock(&LOCK_open);
-     unlock_table_name(thd, table_list);
+     if (!locked_table)
+       unlock_table_name(thd, table_list);
  
      if (error)
        goto err;
      break;
-   }
+  }
 
   if (Table_triggers_list::check_n_load(thd, share->db.str,
                                         share->table_name.str, entry, 0))
@@ -4289,7 +4298,7 @@ void detach_merge_children(TABLE *table, bool clear_refs)
     children attached yet. Also this is called for every child and the
     parent from close_thread_tables().
   */
-  if ((first_detach= parent->children_attached))
+  if (parent->children_attached)
   {
     VOID(parent->file->extra(HA_EXTRA_DETACH_CHILDREN));
     parent->children_attached= FALSE;
@@ -4301,38 +4310,50 @@ void detach_merge_children(TABLE *table, bool clear_refs)
 
   if (clear_refs)
   {
-    /* In any case clear the own parent reference. (***) */
-    table->parent= NULL;
+    if (table->parent)
+    {
+      /* In any case clear the own parent reference. (***) */
+      table->parent= NULL;
+      table->file->extra(HA_EXTRA_DETACH_CHILD);
+    }
 
     /*
-      On the first detach, clear all references. If this table is the
-      parent, we still may need to clear the child references. The first
-      detach might not have done this.
+      Clear all references. If this table is the parent, we still may
+      need to clear the child references. The first detach might not
+      have done this.
     */
-    if (first_detach || (table == parent))
+    for (child_l= parent->child_l; ; child_l= child_l->next_global)
     {
-      /* Clear TABLE references to force new assignment at next open. */
-      for (child_l= parent->child_l; ; child_l= child_l->next_global)
+      /*
+        Do not DBUG_ASSERT(child_l->table); open_tables might be
+        incomplete or we may have been called twice.
+
+        Clear the parent reference of the children only on the first
+        detach. The children might already be closed. They will clear
+        it themselves when this function is called for them with
+        'clear_refs' true. See above "(***)".
+      */
+      if (child_l->table)
       {
+        if (child_l->table->parent)
+        {
+          child_l->table->parent= NULL;
+          if (child_l->table->db_stat)
+            child_l->table->file->extra(HA_EXTRA_DETACH_CHILD);
+        }
         /*
-          Do not DBUG_ASSERT(child_l->table); open_tables might be
-          incomplete.
-
-          Clear the parent reference of the children only on the first
-          detach. The children might already be closed. They will clear
-          it themseves when this function is called for them with
-          'clear_refs' true. See above "(***)".
+          Set alias to "" to ensure that table is not used if we are in
+          LOCK TABLES
         */
-        if (first_detach && child_l->table)
-          child_l->table->parent= NULL;
+        ((char*) child_l->table->alias)[0]= 0;
 
         /* Clear the table reference to force new assignment at next open. */
         child_l->table= NULL;
-
-        /* Break when this was the last child. */
-        if (&child_l->next_global == parent->child_last_l)
-          break;
       }
+
+      /* Break when this was the last child. */
+      if (&child_l->next_global == parent->child_last_l)
+        break;
     }
   }
 
@@ -5129,9 +5150,11 @@ bool open_normal_and_derived_tables(THD *thd, TABLE_LIST *tables, uint flags)
 
 static void mark_real_tables_as_free_for_reuse(TABLE_LIST *table)
 {
+  DBUG_ENTER("mark_real_tables_as_free_for_reuse");
   for (; table; table= table->next_global)
     if (!table->placeholder())
       table->table->query_id= 0;
+  DBUG_VOID_RETURN;
 }