Bug#14397 - OPTIMIZE TABLE with an open HANDLER causes a crash

Version for 5.0.
It fixes three problems:
1. The cause of the bug was that we did not check the table version for
 the HANDLER ... READ commands. We did not notice when a table was
 replaced by a new one. This can happen during ALTER TABLE, REPAIR
 TABLE, and OPTIMIZE TABLE (there might be more cases). I call the fix
 for this problem "the primary bug fix".
2. mysql_ha_flush() was not always called with a locked LOCK_open.
 Though the function comment clearly said it must.
 I changed the code so that the locking is done when required. I call
 the fix for this problem "the secondary fix".
3. In 5.0 (not in 4.1 or 4.0) DROP TABLE had a possible deadlock flaw in
 concur with FLUSH TABLES WITH READ LOCK. I call the fix for this
 problem "the 5.0 addendum fix".


include/my_pthread.h:
  Bug#14397 - OPTIMIZE TABLE with an open HANDLER causes a crash
  Added a new macro for the 5.0 addendum fix.
mysql-test/r/handler.result:
  Bug#14397 - OPTIMIZE TABLE with an open HANDLER causes a crash
  The test result.
mysql-test/t/handler.test:
  Bug#14397 - OPTIMIZE TABLE with an open HANDLER causes a crash
  The test case.
sql/lock.cc:
  Bug#14397 - OPTIMIZE TABLE with an open HANDLER causes a crash
  Changed a comment which did confuse me and which is not fully
  correct anymore after the 5.0 addendum fix.
  Added an assertion which would fire without the 5.0 addendum fix.
sql/mysql_priv.h:
  Bug#14397 - OPTIMIZE TABLE with an open HANDLER causes a crash
  Changed a definition for the secondary fix.
sql/sql_base.cc:
  Bug#14397 - OPTIMIZE TABLE with an open HANDLER causes a crash
  Changed function calls for the secondary fix.
sql/sql_class.cc:
  Bug#14397 - OPTIMIZE TABLE with an open HANDLER causes a crash
  Changed a function call for the secondary fix.
sql/sql_handler.cc:
  Bug#14397 - OPTIMIZE TABLE with an open HANDLER causes a crash
  The first two diffs make the primary bug fix.
  The rest is for the secondary fix.
sql/sql_table.cc:
  Bug#14397 - OPTIMIZE TABLE with an open HANDLER causes a crash
  The first diff (four changed places) make the 5.0 addendum fix.
  The other three are changed function calls for the secondary fix.

1 files changed, 14 insertions, 4 deletions

diff --git a/sql/lock.cc b/sql/lock.cc
index f4c4a781e45..fe8dcb3aa5e 100644
--- a/sql/lock.cc
+++ b/sql/lock.cc
@@ -815,10 +815,13 @@ static void print_lock_error(int error, const char *table)
 
   access to them is protected with a mutex LOCK_global_read_lock
 
-  (XXX: one should never take LOCK_open if LOCK_global_read_lock is taken,
-  otherwise a deadlock may occur - see mysql_rm_table. Other mutexes could
-  be a problem too - grep the code for global_read_lock if you want to use
-  any other mutex here)
+  (XXX: one should never take LOCK_open if LOCK_global_read_lock is
+  taken, otherwise a deadlock may occur. Other mutexes could be a
+  problem too - grep the code for global_read_lock if you want to use
+  any other mutex here) Also one must not hold LOCK_open when calling
+  wait_if_global_read_lock(). When the thread with the global read lock
+  tries to close its tables, it needs to take LOCK_open in
+  close_thread_table().
 
   How blocking of threads by global read lock is achieved: that's
   advisory. Any piece of code which should be blocked by global read lock must
@@ -937,6 +940,13 @@ bool wait_if_global_read_lock(THD *thd, bool abort_on_refresh,
   DBUG_ENTER("wait_if_global_read_lock");
 
   LINT_INIT(old_message);
+  /*
+    Assert that we do not own LOCK_open. If we would own it, other
+    threads could not close their tables. This would make a pretty
+    deadlock.
+  */
+  safe_mutex_assert_not_owner(&LOCK_open);
+
   (void) pthread_mutex_lock(&LOCK_global_read_lock);
   if ((need_exit_cond= must_wait))
   {