Bug #46165 server crash in dbug

This crash occured if the same debug trace file was closed twice,
leading to the same memory being free'd twice. This could occur
if the "debug" server system variable refered to the same trace
file in both global and session scope.

Example of an order of events that would lead to a crash:
1) Enable debug tracing to a trace file (global scope)
2) Enable debug tracing to the same trace file (session scope)
3) Reset debug settings (global scope)
4) Reset debug settings (session scope)

This caused a crash because the trace file was, by mistake, closed
in 3), leading to the same memory being free'd twice when the file
was closed again in 4).

Internally, the debug settings are stored in a stack, with session
settings (if any) on top and the global settings below. Each connection
has its own stack. When a set of settings is changed, it must be 
determined if its debug trace file is to be closed. Before, this was done
by only checking below on the settings stack. So if the global settings
were changed, an existing debug trace file reference in session settings
would be missed. This caused the file to be closed even if it was in use,
leading to a crash later when it was closed again.

This patch fixes the problem by preventing the trace file from being shared
between global and session settings. If session debug settings are set without
specifying a new trace file, stderr is used for output. This is a change
in behaviour and should be reflected in the documentation.

Test case added to variables.test.

2 files changed, 119 insertions, 0 deletions

diff --git a/mysql-test/r/variables.result b/mysql-test/r/variables.result
index be81afe1a43..971e4caf516 100644
--- a/mysql-test/r/variables.result
+++ b/mysql-test/r/variables.result
@@ -1699,3 +1699,47 @@ set @@session.autocommit=t1_min(), @@session.autocommit=t1_max(),
 drop table t1;
 drop function t1_min;
 drop function t1_max;
+#
+# Bug#46165 server crash in dbug
+#
+SET @old_globaldebug = @@global.debug;
+SET @old_sessiondebug= @@session.debug;
+# Test 1 - Bug test case, single connection
+SET GLOBAL  debug= '+O,../../log/bug46165.1.trace';
+SET SESSION debug= '-d:-t:-i';
+SET GLOBAL  debug= '';
+SET SESSION debug= '';
+# Test 2 - Bug test case, two connections
+# Connection default
+SET GLOBAL  debug= '+O,../../log/bug46165.2.trace';
+SET SESSION debug= '-d:-t:-i';
+# Connection con1
+SET GLOBAL  debug= '';
+# Connection default
+SET SESSION debug= '';
+# Connection con1
+# Connection default
+SET GLOBAL  debug= '';
+# Test 3 - Active session trace file on disconnect
+# Connection con1
+SET GLOBAL  debug= '+O,../../log/bug46165.3.trace';
+SET SESSION debug= '-d:-t:-i';
+SET GLOBAL  debug= '';
+# Test 4 - Active session trace file on two connections
+# Connection default
+SET GLOBAL  debug= '+O,../../log/bug46165.4.trace';
+SET SESSION debug= '-d:-t:-i';
+# Connection con1
+SET SESSION debug= '-d:-t:-i';
+SET GLOBAL  debug= '';
+SET SESSION debug= '';
+# Connection default
+SET SESSION debug= '';
+# Connection con1
+# Connection default
+# Test 5 - Different trace files
+SET SESSION debug= '+O,../../log/bug46165.5.trace';
+SET SESSION debug= '+O,../../log/bug46165.6.trace';
+SET SESSION debug= '-O';
+SET GLOBAL  debug= @old_globaldebug;
+SET SESSION debug= @old_sessiondebug;
diff --git a/mysql-test/t/variables.test b/mysql-test/t/variables.test
index 75099523062..0209fb00a77 100644
--- a/mysql-test/t/variables.test
+++ b/mysql-test/t/variables.test
@@ -1432,3 +1432,78 @@ drop function t1_max;
 
 
 ###########################################################################
+
+
+--echo #
+--echo # Bug#46165 server crash in dbug
+--echo #
+
+SET @old_globaldebug = @@global.debug;
+SET @old_sessiondebug= @@session.debug;
+
+--echo # Test 1 - Bug test case, single connection
+SET GLOBAL  debug= '+O,../../log/bug46165.1.trace';
+SET SESSION debug= '-d:-t:-i';
+
+SET GLOBAL  debug= '';
+SET SESSION debug= '';
+
+--echo # Test 2 - Bug test case, two connections
+--echo # Connection default
+connection default;
+SET GLOBAL  debug= '+O,../../log/bug46165.2.trace';
+SET SESSION debug= '-d:-t:-i';
+
+--echo # Connection con1
+connect (con1, localhost, root);
+SET GLOBAL  debug= '';
+
+--echo # Connection default
+connection default;
+SET SESSION debug= '';
+--echo # Connection con1
+connection con1;
+disconnect con1;
+--source include/wait_until_disconnected.inc
+--echo # Connection default
+connection default;
+SET GLOBAL  debug= '';
+
+--echo # Test 3 - Active session trace file on disconnect
+--echo # Connection con1
+connect (con1, localhost, root);
+SET GLOBAL  debug= '+O,../../log/bug46165.3.trace';
+SET SESSION debug= '-d:-t:-i';
+SET GLOBAL  debug= '';
+disconnect con1;
+--source include/wait_until_disconnected.inc
+
+--echo # Test 4 - Active session trace file on two connections
+--echo # Connection default
+connection default;
+SET GLOBAL  debug= '+O,../../log/bug46165.4.trace';
+SET SESSION debug= '-d:-t:-i';
+
+--echo # Connection con1
+connect (con1, localhost, root);
+SET SESSION debug= '-d:-t:-i';
+SET GLOBAL  debug= '';
+SET SESSION debug= '';
+
+--echo # Connection default
+connection default;
+SET SESSION debug= '';
+--echo # Connection con1
+connection con1;
+disconnect con1;
+--source include/wait_until_disconnected.inc
+--echo # Connection default
+connection default;
+
+--echo # Test 5 - Different trace files
+SET SESSION debug= '+O,../../log/bug46165.5.trace';
+SET SESSION debug= '+O,../../log/bug46165.6.trace';
+SET SESSION debug= '-O';
+
+SET GLOBAL  debug= @old_globaldebug;
+SET SESSION debug= @old_sessiondebug;