From 32f81bab7d3ed46ddc2863c7be8d69f8dcf698c3 Mon Sep 17 00:00:00 2001 From: Michael Widenius Date: Wed, 3 Dec 2008 00:02:52 +0200 Subject: WL#3262 add mutex lock order checking to safemutex (also called safe_mutex_deadlock_detector) This writes a warning on stderr if one uses mutex in different order, like if one in one case would lock mutex in the order A,B and in another case would lock mutex in the order B,A This is inspired by and loosely based on the LOCKDEP patch by Jonas Wrong mutex order is either fixed or mutex are marked with MYF_NO_DEADLOCK_DETECTION if used inconsistently (need to be fixed by server team) KNOWN_BUGS.txt: Added information that one need to dump and restore Maria tables include/hash.h: Added prototype function for walking over all elements in a hash include/my_pthread.h: Added my_pthread_mutex_init() and my_pthread_mutex_lock(); These should be used if one wants to disable mutex order checking. Changed names of the nonposix mutex_init functions to not conflict with my_phread_mutex_init() Added and extended structures for mutex deadlock detection. New arguments to sage_mutex_init() and safe_mutex_lock() to allow one to disable mutex order checking. Added variable 'safe_mutex_deadlock_detector' to enable/disable deadlock detection for all pthread_mutex_init() mysys/Makefile.am: Added cleaning of test files Added test_thr_mutex mysys/hash.c: Added hash_iterate() to iterate over all elements in a hash More comments mysys/my_init.c: Added calls to destory all mutex uses by mysys() Added waiting for threads to end before calling TERMINATE() to list not freed memory mysys/my_pthread.c: Changed names to free my_pthread_mutex_init() for mutex-lock-order-checking mysys/my_sleep.c: Fixed too long wait if using 1000000L as argument mysys/my_thr_init.c: Mark THR_LOCK_threads and THR_LOCK_malloc to not have mutex deadlock detection. (We can't have it enabled for this as these are internal mutex used by the detector Call my_thread_init() early as we need thread specific variables enabled for the following pthread_mutex_init() Move code to wait for threads to end to my_wait_for_other_threads_to_die() Don't destroy mutex and conditions unless all threads have died Added my_thread_destroy_mutex() to destroy all mutex used by the mysys thread system Name the thread specific mutex as "mysys_var->mutex" Added my_thread_var_mutex_in_use() to return pointer to mutex in use or 0 if thread variables are not initialized mysys/mysys_priv.h: Added prototypes for functions used internally with mutex-wrong-usage detection mysys/thr_mutex.c: Added runtime detection of mutex used in conflicting order See WL#3262 or test_thr_mutex.c for examples The base idea is for each mutex have two hashes: - mutex->locked_mutex points to all mutex used after this one - mutex->used_mutex points to all mutex which has this mutex in it's mutex->locked_mutex There is a wrong mutex order if any mutex currently locked before this mutex is in the mutex->locked_mutex hash sql/event_queue.cc: Mark mutex used inconsistently (need to be fixed by server team) sql/event_scheduler.cc: Declare the right order to take the mutex sql/events.cc: Mark mutex used inconsistently (need to be fixed by server team) sql/ha_ndbcluster_binlog.cc: Mark mutex used inconsistently (need to be fixed by server team) sql/log.cc: Mark mutex used inconsistently (need to be fixed by server team) sql/mysqld.cc: Use pthread_mutex_trylock instead of pthread_mutex_unlock() when sending kill signal to thread This is needed to avoid wrong mutex order as normally one takes 'current_mutex' before mysys_var->mutex. Added call to free sp cache. Add destruction of LOCK_server_started and COND_server_started. Added register_mutex_order() function to register in which order mutex should be taken (to initiailize mutex_deadlock_detector). Added option to turn off safe_mutex_deadlock_detector sql/protocol.cc: Fixed wrong argument to DBUG_PRINT (found by valgrind) sql/rpl_mi.cc: Mark mutex used inconsistently (need to be fixed by server team) sql/set_var.cc: Remove wrong locking of LOCK_global_system_variables when reading and setting log variables (would cause inconsistent mutex order). Update global variables outside of logger.unlock() as LOCK_global_system_variables has to be taken before logger locks Reviewed by gluh sql/sp_cache.cc: Added function to destroy mutex used by sp cache sql/sp_cache.h: Added function to destroy mutex used by sp cache sql/sql_class.cc: Use pthread_mutex_trylock instead of pthread_mutex_unlock() when sending kill signal to thread This is needed to avoid wrong mutex order as normally one takes 'current_mutex' before mysys_var->mutex. Register order in which LOCK_delete and mysys_var->mutex is taken sql/sql_insert.cc: Give a name for Delayed_insert::mutex Mark mutex used inconsistently (need to be fixed by server team) Move closing of tables outside of di->mutex (to avoid wrong mutex order) sql/sql_show.cc: Don't keep LOCK_global_system_variables locked over value->show_type() as this leads to wrong mutex order storage/innobase/handler/ha_innodb.cc: Disable safe_muted_deadlock_detector for innobase intern mutex (to speed up page cache initialization) storage/maria/ha_maria.cc: Added flag to ha_maria::info() to signal if we need to lock table share or not. This is needed to avoid locking mutex in wrong order storage/maria/ha_maria.h: Added flag to ha_maria::info() to signal if we need to lock table share or not. storage/maria/ma_close.c: Destroy key_del_lock Simplify freeing ftparser_param storage/maria/ma_key.c: Better comment storage/maria/ma_loghandler.c: Mark mutex used inconsistently (need to be fixed by sanja) storage/maria/ma_state.c: More comments storage/maria/ma_test1.c: Ensure that safe_mutex_deadlock_detector is always on (should be, this is just for safety) storage/maria/ma_test2.c: Ensure that safe_mutex_deadlock_detector is always on (should be, this is just for safety) --- sql/event_queue.cc | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) (limited to 'sql/event_queue.cc') diff --git a/sql/event_queue.cc b/sql/event_queue.cc index 04d4f858b43..d68dc8ef479 100644 --- a/sql/event_queue.cc +++ b/sql/event_queue.cc @@ -94,7 +94,12 @@ Event_queue::Event_queue() mutex_queue_data_attempting_lock(FALSE), waiting_on_cond(FALSE) { - pthread_mutex_init(&LOCK_event_queue, MY_MUTEX_INIT_FAST); + /* + Inconsisent usage between LOCK_event_queue and LOCK_scheduler_state and + LOCK_open + */ + my_pthread_mutex_init(&LOCK_event_queue, MY_MUTEX_INIT_FAST, + "LOCK_event_queue", MYF_NO_DEADLOCK_DETECTION); pthread_cond_init(&COND_queue_state, NULL); } -- cgit v1.2.1 From ecbcddc03dc298ea1e6c0aa1a120bd0b4b04b3fd Mon Sep 17 00:00:00 2001 From: Michael Widenius Date: Fri, 16 Jul 2010 10:33:01 +0300 Subject: Improved speed of thr_alarm from O(N) to O(1). thr_alarm is used to handle timeouts and kill of connections. Fixed compiler warnings. queues.h and queues.c are now based on the UNIREG code and thus made BSD. Fix code to use new queue() interface. This mostly affects how you access elements in the queue. If USE_NET_CLEAR is not set, don't clear connection from unexpected characters. This should give a speed up when doing a lot of fast queries. Fixed some code in ma_ft_boolean_search.c that had not made it from myisam/ft_boolean_search.c include/queues.h: Use UNIREG code base (BSD) Changed init_queue() to take all initialization arguments. New interface to access elements in queue include/thr_alarm.h: Changed to use time_t instead of ulong (portability) Added index_in_queue, to be able to remove random element from queue in O(1) mysys/queues.c: Use UNIREG code base (BSD) init_queue() and reinit_queue() now takes more initialization arguments. (No need for init_queue_ex() anymore) Now one can tell queue_insert() to store in the element a pointer to where element is in queue. This allows one to remove elements from queue in O(1) instead of O(N) mysys/thr_alarm.c: Use new option in queue() to allow fast removal of elements. Do less inside LOCK_alarm mutex. This should give a major speed up of thr_alarm usage when there is many threads sql/create_options.cc: Fixed wrong printf sql/event_queue.cc: Use new queue interface() sql/filesort.cc: Use new queue interface() sql/ha_partition.cc: Use new queue interface() sql/ha_partition.h: Fixed compiler warning sql/item_cmpfunc.cc: Fixed compiler warning sql/item_subselect.cc: Use new queue interface() Removed not used variable sql/net_serv.cc: If USE_NET_CLEAR is not set, don't clear connection from unexpected characters. This should give a speed up when doing a lot of fast queries at the disadvantage that if there is a bug in the client protocol the connection will be dropped instead of being unnoticed. sql/opt_range.cc: Use new queue interface() Fixed compiler warnings sql/uniques.cc: Use new queue interface() storage/maria/ma_ft_boolean_search.c: Copy code from myisam/ft_boolean_search.c Use new queue interface() storage/maria/ma_ft_nlq_search.c: Use new queue interface() storage/maria/ma_sort.c: Use new queue interface() storage/maria/maria_pack.c: Use new queue interface() Use queue_fix() instead of own loop to fix queue. storage/myisam/ft_boolean_search.c: Use new queue interface() storage/myisam/ft_nlq_search.c: Use new queue interface() storage/myisam/mi_test_all.sh: Remove temporary file from last run storage/myisam/myisampack.c: Use new queue interface() Use queue_fix() instead of own loop to fix queue. storage/myisam/sort.c: Use new queue interface() storage/myisammrg/myrg_queue.c: Use new queue interface() storage/myisammrg/myrg_rnext.c: Use new queue interface() storage/myisammrg/myrg_rnext_same.c: Use new queue interface() storage/myisammrg/myrg_rprev.c: Use new queue interface() --- sql/event_queue.cc | 46 ++++++++++++++++++++++++++++++---------------- 1 file changed, 30 insertions(+), 16 deletions(-) (limited to 'sql/event_queue.cc') diff --git a/sql/event_queue.cc b/sql/event_queue.cc index d68dc8ef479..2a354fe6cfd 100644 --- a/sql/event_queue.cc +++ b/sql/event_queue.cc @@ -136,9 +136,9 @@ Event_queue::init_queue(THD *thd) LOCK_QUEUE_DATA(); - if (init_queue_ex(&queue, EVENT_QUEUE_INITIAL_SIZE , 0 /*offset*/, - 0 /*max_on_top*/, event_queue_element_compare_q, - NULL, EVENT_QUEUE_EXTENT)) + if (::init_queue(&queue, EVENT_QUEUE_INITIAL_SIZE , 0 /*offset*/, + 0 /*max_on_top*/, event_queue_element_compare_q, + NullS, 0, EVENT_QUEUE_EXTENT)) { sql_print_error("Event Scheduler: Can't initialize the execution queue"); goto err; @@ -325,11 +325,13 @@ void Event_queue::drop_matching_events(THD *thd, LEX_STRING pattern, bool (*comparator)(LEX_STRING, Event_basic *)) { - uint i= 0; + uint i; DBUG_ENTER("Event_queue::drop_matching_events"); DBUG_PRINT("enter", ("pattern=%s", pattern.str)); - while (i < queue.elements) + for (i= queue_first_element(&queue) ; + i <= queue_last_element(&queue) ; + ) { Event_queue_element *et= (Event_queue_element *) queue_element(&queue, i); DBUG_PRINT("info", ("[%s.%s]?", et->dbname.str, et->name.str)); @@ -339,7 +341,8 @@ Event_queue::drop_matching_events(THD *thd, LEX_STRING pattern, The queue is ordered. If we remove an element, then all elements after it will shift one position to the left, if we imagine it as an array from left to the right. In this case we should not - increment the counter and the (i < queue.elements) condition is ok. + increment the counter and the (i <= queue_last_element() condition + is ok. */ queue_remove(&queue, i); delete et; @@ -403,7 +406,9 @@ Event_queue::find_n_remove_event(LEX_STRING db, LEX_STRING name) uint i; DBUG_ENTER("Event_queue::find_n_remove_event"); - for (i= 0; i < queue.elements; ++i) + for (i= queue_first_element(&queue); + i <= queue_last_element(&queue); + i++) { Event_queue_element *et= (Event_queue_element *) queue_element(&queue, i); DBUG_PRINT("info", ("[%s.%s]==[%s.%s]?", db.str, name.str, @@ -441,7 +446,9 @@ Event_queue::recalculate_activation_times(THD *thd) LOCK_QUEUE_DATA(); DBUG_PRINT("info", ("%u loaded events to be recalculated", queue.elements)); - for (i= 0; i < queue.elements; i++) + for (i= queue_first_element(&queue); + i <= queue_last_element(&queue); + i++) { ((Event_queue_element*)queue_element(&queue, i))->compute_next_execution_time(); ((Event_queue_element*)queue_element(&queue, i))->update_timing_fields(thd); @@ -454,16 +461,19 @@ Event_queue::recalculate_activation_times(THD *thd) have removed all. The queue has been ordered in a way the disabled events are at the end. */ - for (i= queue.elements; i > 0; i--) + for (i= queue_last_element(&queue); + (int) i >= (int) queue_first_element(&queue); + i--) { - Event_queue_element *element = (Event_queue_element*)queue_element(&queue, i - 1); + Event_queue_element *element= + (Event_queue_element*)queue_element(&queue, i); if (element->status != Event_parse_data::DISABLED) break; /* This won't cause queue re-order, because we remove always the last element. */ - queue_remove(&queue, i - 1); + queue_remove(&queue, i); delete element; } UNLOCK_QUEUE_DATA(); @@ -499,7 +509,9 @@ Event_queue::empty_queue() sql_print_information("Event Scheduler: Purging the queue. %u events", queue.elements); /* empty the queue */ - for (i= 0; i < queue.elements; ++i) + for (i= queue_first_element(&queue); + i <= queue_last_element(&queue); + i++) { Event_queue_element *et= (Event_queue_element *) queue_element(&queue, i); delete et; @@ -525,7 +537,9 @@ Event_queue::dbug_dump_queue(time_t now) uint i; DBUG_ENTER("Event_queue::dbug_dump_queue"); DBUG_PRINT("info", ("Dumping queue . Elements=%u", queue.elements)); - for (i = 0; i < queue.elements; i++) + for (i= queue_first_element(&queue); + i <= queue_last_element(&queue); + i++) { et= ((Event_queue_element*)queue_element(&queue, i)); DBUG_PRINT("info", ("et: 0x%lx name: %s.%s", (long) et, @@ -592,7 +606,7 @@ Event_queue::get_top_for_execution_if_time(THD *thd, continue; } - top= ((Event_queue_element*) queue_element(&queue, 0)); + top= (Event_queue_element*) queue_top(&queue); thd->set_current_time(); /* Get current time */ @@ -634,10 +648,10 @@ Event_queue::get_top_for_execution_if_time(THD *thd, top->dbname.str, top->name.str, top->dropped? "Dropping.":""); delete top; - queue_remove(&queue, 0); + queue_remove_top(&queue); } else - queue_replaced(&queue); + queue_replace_top(&queue); dbug_dump_queue(thd->query_start()); break; -- cgit v1.2.1