diff options
| author | Jan Lindström <jan.lindstrom@mariadb.com> | 2020-03-12 15:34:50 +0200 |
|---|---|---|
| committer | Jan Lindström <jan.lindstrom@mariadb.com> | 2020-09-03 15:19:28 +0300 |
| commit | 1b6bd701b30f8297db38e605692aa91aff92a06d (patch) | |
| tree | fa977c8546b930b6f0a7df259abfdc725586dc29 /sql/sql_parse.cc | |
| parent | 97db6c15ea3e83a21df137c222dbd5a40fbe7c82 (diff) | |
| download | mariadb-git-bb-10.2-MDEV-21910.tar.gz | |
MDEV-21910 : KIlling thread on Galera could cause mutex deadlockbb-10.2-MDEV-21910
Whenever Galera BF (brute force) transaction decides to abort conflicting
transaction it will kill that thread using thd::awake()
User KILL [QUERY|CONNECTION] ... for a thread it will also call thd::awake()
Whenever one of these actions is executed we will hold number of InnoDB
internal mutexes and thd mutexes. Sometimes these mutexes are taken in
different order causing mutex deadlock.
Lets call BF kill as bf_thread and user KILL-query as kill_thread.
bf_thread takes mutexes in order:
(1) lock_sys->mutex (lock0lock.cc lock_rec_other_has_conflicting)
(2) victim_trx->mutex (lock0lock.cc lock_rec_other_has_conflicting)
(3) victim_thread->LOCK_thd_data (handler.cc wsrep_innobase_kill_one_trx)
kill_thread takes mutexes in order:
(1) victim_thread->LOCK_thd_data (sql_parse.cc find_thread_by_id)
(2) lock_sys->mutex (ha_innodb.cc innobase_kill_query)
(3) victim_trx->mutex (ha_innodb.cc innobase_kill_query)
Mutex deadlock is result of taking victim_thread->LOCK_thd_data
in different order.
In this patch we will fix Galera BF victim thread kill so that
it will not try to lock victim_thread->LOCK_thd_data mutex
while we hold InnoDB mutexes. Instead victim is inserted a list
for later kill processing.
A new background thread will pick victim thread from this new list and uses
thd::awake() with no InnoDB mutexes. Idea is similar to replication background
kill. This fix enforces that we take mutexes in same order:
(1) victim_thread->LOCK_thd_data
(2) lock_sys->mutex ->
(3) victim_trx->mutex
wsrep_mysqld.cc
Here we introduce a list where victim threads are stored,
condition variable to be used to wake up background thread
and mutex to protect list.
wsrep_thd.cc
Create a new background thread to handle victim thread
abort. We may take victim_thread->LOCK_thd_data mutex
here but not any InnoDB mutexes.
wsrep_innobase_kill_one_trx
Remove all the wsrep code that was moved to wsrep_thd.cc
We just enqueue required information to background kill
list and cancel victim trx lock wait if there is such.
Here we have InnoDB lock_sys->mutex and victim_trx->mutex
so here we can't take victim_thread->LOCK_thd_data mutex.
wsrep_abort_transaction
Cleanup only.
Diffstat (limited to 'sql/sql_parse.cc')
| -rw-r--r-- | sql/sql_parse.cc | 11 |
1 files changed, 9 insertions, 2 deletions
diff --git a/sql/sql_parse.cc b/sql/sql_parse.cc index 543c877b7f1..a755844e74e 100644 --- a/sql/sql_parse.cc +++ b/sql/sql_parse.cc @@ -8896,8 +8896,15 @@ kill_one_thread(THD *thd, longlong id, killed_state kill_signal, killed_type typ thd->security_ctx->user_matches(tmp->security_ctx)) && !wsrep_thd_is_BF(tmp, false)) { - tmp->awake(kill_signal); - error=0; +#ifdef WITH_WSREP + /* We allow kill to continue only if there is no concurrent + kill in processing. */ + if (wsrep_thd_set_wsrep_aborter(tmp, thd)) +#endif + { + tmp->awake(kill_signal); + error=0; + } } else error= (type == KILL_TYPE_QUERY ? ER_KILL_QUERY_DENIED_ERROR : |