MDEV-15016: multiple page cleaner threads use a lot of CPU on idle serverbb-10.3-MDEV-15016

Problem was that when number of page cleaner threads was increased
we sent a event is_requested indicating there is "work" to be done
when there was no work to be done and workers were left on busy loop.
When number of page cleaner threads is decreased we need to sent
is_requested event to indicate that there is "work" to be done.

buf_flush_set_page_cleaner_thread_cnt
        When number of page cleaner threads decrease we send
        is_requested event i.e. this is similar as in shutdown
        there is "work" to be done. We wait these threads
        to exit on is_started event. Page cleaner threads that
        id is less than n_workers will send is_started event
        before decreasing number of running threads and exit.

Manually tested on both (1) idle system and (2) on system with sysbench
workload by increasing and decreasing number of page cleaner threads
and monitoring mysqld CPU usage

1 files changed, 19 insertions, 3 deletions

diff --git a/storage/innobase/buf/buf0flu.cc b/storage/innobase/buf/buf0flu.cc
index 5c96096a8dd..5749c7c0b9e 100644
--- a/storage/innobase/buf/buf0flu.cc
+++ b/storage/innobase/buf/buf0flu.cc
@@ -3477,6 +3477,13 @@ buf_flush_set_page_cleaner_thread_cnt(ulong new_cnt)
 	mutex_enter(&page_cleaner.mutex);
 
 	srv_n_page_cleaners = new_cnt;
+
+	/* Startup is same as increasing number of page cleaner
+	threads i.e. we create requested number of threads and start
+	waiting is_started events until requested number of threads is
+	reached. When page cleaner thread starts it increases number of
+	running threads and sends a is_started event and starts to wait
+	for work in is_requested event. */
 	if (new_cnt > page_cleaner.n_workers) {
 		/* User has increased the number of page
 		cleaner threads. */
@@ -3489,12 +3496,21 @@ buf_flush_set_page_cleaner_thread_cnt(ulong new_cnt)
 
 	mutex_exit(&page_cleaner.mutex);
 
-	/* Wait until defined number of workers has started. */
+	/* Wait until defined number of workers has reached. */
 	while (page_cleaner.is_running &&
 	       page_cleaner.n_workers != (srv_n_page_cleaners - 1)) {
-		os_event_set(page_cleaner.is_requested);
+		/* When number of page cleaner threads decrease we send
+		is_requested event i.e. this is similar as in shutdown
+		there is "work" to be done. Again we wait these threads
+		to exit on is_started event. Page cleaner threads that
+		id is less than n_workers will send is_started event
+		before decreasing number of running threads and exists. */
+		if (page_cleaner.n_workers > (srv_n_page_cleaners -1 )) {
+			os_event_set(page_cleaner.is_requested);
+		}
+
 		os_event_reset(page_cleaner.is_started);
-		os_event_wait_time(page_cleaner.is_started, 1000000);
+		os_event_wait_time(page_cleaner.is_started, 10000);
 	}
 }