From a5efb91deaf8a9c3bf53cd82ebd574be0cdabc5d Mon Sep 17 00:00:00 2001 From: Davi Arnaut Date: Wed, 6 Oct 2010 11:34:28 -0300 Subject: Bug#49938: Failing assertion: inode or deadlock in fsp/fsp0fsp.c Bug#54678: InnoDB, TRUNCATE, ALTER, I_S SELECT, crash or deadlock - Incompatible change: truncate no longer resorts to a row by row delete if the storage engine does not support the truncate method. Consequently, the count of affected rows does not, in any case, reflect the actual number of rows. - Incompatible change: it is no longer possible to truncate a table that participates as a parent in a foreign key constraint, unless it is a self-referencing constraint (both parent and child are in the same table). To work around this incompatible change and still be able to truncate such tables, disable foreign checks with SET foreign_key_checks=0 before truncate. Alternatively, if foreign key checks are necessary, please use a DELETE statement without a WHERE condition. Problem description: The problem was that for storage engines that do not support truncate table via a external drop and recreate, such as InnoDB which implements truncate via a internal drop and recreate, the delete_all_rows method could be invoked with a shared metadata lock, causing problems if the engine needed exclusive access to some internal metadata. This problem originated with the fact that there is no truncate specific handler method, which ended up leading to a abuse of the delete_all_rows method that is primarily used for delete operations without a condition. Solution: The solution is to introduce a truncate handler method that is invoked when the engine does not support truncation via a table drop and recreate. This method is invoked under a exclusive metadata lock, so that there is only a single instance of the table when the method is invoked. Also, the method is not invoked and a error is thrown if the table is a parent in a non-self-referencing foreign key relationship. This was necessary to avoid inconsistency as some integrity checks are bypassed. This is inline with the fact that truncate is primarily a DDL operation that was designed to quickly remove all data from a table. mysql-test/suite/innodb/t/innodb-truncate.test: Add test cases for truncate and foreign key checks. Also test that InnoDB resets auto-increment on truncate. mysql-test/suite/innodb/t/innodb.test: FK is not necessary, test is related to auto-increment. Update error number, truncate is no longer invoked if table is parent in a FK relationship. mysql-test/suite/innodb/t/innodb_mysql.test: Update error number, truncate is no longer invoked if table is parent in a FK relationship. Use delete instead of truncate, test is used to check the interaction of FKs, triggers and delete. mysql-test/suite/parts/inc/partition_check.inc: Fix typo. mysql-test/suite/sys_vars/t/foreign_key_checks_func.test: Update error number, truncate is no longer invoked if table is parent in a FK relationship. mysql-test/t/mdl_sync.test: Modify test case to reflect and ensure that truncate takes a exclusive metadata lock. mysql-test/t/trigger-trans.test: Update error number, truncate is no longer invoked if table is parent in a FK relationship. sql/ha_partition.cc: Reorganize the various truncate methods. delete_all_rows is now passed directly to the underlying engines, so as truncate. The code responsible for truncating individual partitions is moved to ha_partition::truncate_partition, which is invoked when a ALTER TABLE t1 TRUNCATE PARTITION p statement is executed. Since the partition truncate no longer can be invoked via delete, the bitmap operations are not necessary anymore. The explicit reset of the auto-increment value is also removed as the underlying engines are now responsible for reseting the value. sql/handler.cc: Wire up the handler truncate method. sql/handler.h: Introduce and document the truncate handler method. It assumes certain use cases of delete_all_rows. Add method to retrieve the list of foreign keys referencing a table. Method is used to avoid truncating tables that are parent in a foreign key relationship. sql/share/errmsg-utf8.txt: Add error message for truncate and FK. sql/sql_lex.h: Introduce a flag so that the partition engine can detect when a partition is being truncated. Used to give a special error. sql/sql_parse.cc: Function mysql_truncate_table no longer exists. sql/sql_partition_admin.cc: Implement the TRUNCATE PARTITION statement. sql/sql_truncate.cc: Change the truncate table implementation to use the new truncate handler method and to not rely on row-by-row delete anymore. The truncate handler method is always invoked with a exclusive metadata lock. Also, it is no longer possible to truncate a table that is parent in some non-self-referencing foreign key. storage/archive/ha_archive.cc: Rename method as the description indicates that in the future this could be a truncate operation. storage/blackhole/ha_blackhole.cc: Implement truncate as no operation for the blackhole engine in order to remain compatible with older releases. storage/federated/ha_federated.cc: Introduce truncate method that invokes delete_all_rows. This is required to support partition truncate as this form of truncate does not implement the drop and recreate protocol. storage/heap/ha_heap.cc: Introduce truncate method that invokes delete_all_rows. This is required to support partition truncate as this form of truncate does not implement the drop and recreate protocol. storage/ibmdb2i/ha_ibmdb2i.cc: Introduce truncate method that invokes delete_all_rows. This is required to support partition truncate as this form of truncate does not implement the drop and recreate protocol. storage/innobase/handler/ha_innodb.cc: Rename delete_all_rows to truncate. InnoDB now does truncate under a exclusive metadata lock. Introduce and reorganize methods used to retrieve the list of foreign keys referenced by a or referencing a table. storage/myisammrg/ha_myisammrg.cc: Introduce truncate method that invokes delete_all_rows. This is required in order to remain compatible with earlier releases where truncate would resort to a row-by-row delete. --- sql/handler.cc | 15 +++++++++++++++ 1 file changed, 15 insertions(+) (limited to 'sql/handler.cc') diff --git a/sql/handler.cc b/sql/handler.cc index 567dbe6ea49..1542dd99ec6 100644 --- a/sql/handler.cc +++ b/sql/handler.cc @@ -3208,6 +3208,21 @@ handler::ha_delete_all_rows() } +/** + Truncate table: public interface. + + @sa handler::truncate() +*/ + +int +handler::ha_truncate() +{ + mark_trx_read_write(); + + return truncate(); +} + + /** Reset auto increment: public interface. -- cgit v1.2.1 From 9074307102644d43b934ce52cc0661890098698b Mon Sep 17 00:00:00 2001 From: Tor Didriksen Date: Mon, 18 Oct 2010 13:24:34 +0200 Subject: Bug#52172 test binlog.binlog_index needs --skip-core-file to avoid leaving core files For crash testing: kill the server without generating core file. include/my_dbug.h Use kill(getpid(), SIGKILL) which cannot be caught by signal handlers. All DBUG_XXX macros should be no-ops in optimized mode, do that for DBUG_ABORT as well. sql/handler.cc Kill server without generating core. sql/log.cc Kill server without generating core. --- sql/handler.cc | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) (limited to 'sql/handler.cc') diff --git a/sql/handler.cc b/sql/handler.cc index 19f397ef09f..a47a5fd8a3c 100644 --- a/sql/handler.cc +++ b/sql/handler.cc @@ -1127,7 +1127,7 @@ int ha_commit_trans(THD *thd, bool all) uint rw_ha_count; bool rw_trans; - DBUG_EXECUTE_IF("crash_commit_before", abort();); + DBUG_EXECUTE_IF("crash_commit_before", DBUG_SUICIDE();); /* Close all cursors that can not survive COMMIT */ if (is_real_trans) /* not a statement commit */ @@ -1179,7 +1179,7 @@ int ha_commit_trans(THD *thd, bool all) } status_var_increment(thd->status_var.ha_prepare_count); } - DBUG_EXECUTE_IF("crash_commit_after_prepare", abort();); + DBUG_EXECUTE_IF("crash_commit_after_prepare", DBUG_SUICIDE();); if (error || (is_real_trans && xid && (error= !(cookie= tc_log->log_xid(thd, xid))))) { @@ -1187,13 +1187,13 @@ int ha_commit_trans(THD *thd, bool all) error= 1; goto end; } - DBUG_EXECUTE_IF("crash_commit_after_log", abort();); + DBUG_EXECUTE_IF("crash_commit_after_log", DBUG_SUICIDE();); } error=ha_commit_one_phase(thd, all) ? (cookie ? 2 : 1) : 0; - DBUG_EXECUTE_IF("crash_commit_before_unlog", abort();); + DBUG_EXECUTE_IF("crash_commit_before_unlog", DBUG_SUICIDE();); if (cookie) tc_log->unlog(cookie, xid); - DBUG_EXECUTE_IF("crash_commit_after", abort();); + DBUG_EXECUTE_IF("crash_commit_after", DBUG_SUICIDE();); end: if (rw_trans) start_waiting_global_read_lock(thd); -- cgit v1.2.1 From cd1c6e220de1730615c145b5337f7cce554dfdae Mon Sep 17 00:00:00 2001 From: Davi Arnaut Date: Wed, 10 Nov 2010 19:14:47 -0200 Subject: Bug#58057: 5.1 libmysql/libmysql.c unused variable/compile failure Bug#57995: Compiler flag change build error on OSX 10.4: my_getncpus.c Bug#57996: Compiler flag change build error on OSX 10.5 : bind.c Bug#57994: Compiler flag change build error : my_redel.c Bug#57993: Compiler flag change build error on FreeBsd 7.0 : regexec.c Bug#57992: Compiler flag change build error on FreeBsd : mf_keycache.c Bug#57997: Compiler flag change build error on OSX 10.6: debug_sync.cc Fix assorted compiler generated warnings. cmd-line-utils/readline/bind.c: Bug#57996: Compiler flag change build error on OSX 10.5 : bind.c Initialize variable to work around a false positive warning. include/m_string.h: Bug#57994: Compiler flag change build error : my_redel.c The expansion of stpcpy (in glibc) causes warnings if the return value of strmov is not being used. Since stpcpy is a GNU extension and the expansion ends up using a built-in provided by GCC, use the compiler provided built-in directly when possible. include/my_compiler.h: Define a dummy MY_GNUC_PREREQ when not compiling with GCC. libmysql/libmysql.c: Bug#58057: 5.1 libmysql/libmysql.c unused variable/compile failure Variable might not be used in some cases. So, tag it as unused. mysys/mf_keycache.c: Bug#57992: Compiler flag change build error on FreeBsd : mf_keycache.c Use UNINIT_VAR to work around a false positive warning. mysys/my_getncpus.c: Bug#57995: Compiler flag change build error on OSX 10.4: my_getncpus.c Declare variable in the same block where it is used. regex/regexec.c: Bug#57993: Compiler flag change build error on FreeBsd 7.0 : regexec.c Work around a compiler bug which causes the cast to not be enforced. sql/debug_sync.cc: Bug#57997: Compiler flag change build error on OSX 10.6: debug_sync.cc Use UNINIT_VAR to work around a false positive warning. sql/handler.cc: Use UNINIT_VAR to work around a false positive warning. sql/slave.cc: Use UNINIT_VAR to work around a false positive warning. sql/sql_partition.cc: Use UNINIT_VAR to work around a false positive warning. storage/myisam/ft_nlq_search.c: Use UNINIT_VAR to work around a false positive warning. storage/myisam/mi_create.c: Use UNINIT_VAR to work around a false positive warning. storage/myisammrg/myrg_open.c: Use UNINIT_VAR to work around a false positive warning. tests/mysql_client_test.c: Change function to take a pointer to const, no need for a cast. --- sql/handler.cc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'sql/handler.cc') diff --git a/sql/handler.cc b/sql/handler.cc index a47a5fd8a3c..1525ca53bca 100644 --- a/sql/handler.cc +++ b/sql/handler.cc @@ -4141,7 +4141,7 @@ int handler::read_multi_range_first(KEY_MULTI_RANGE **found_range_p, */ int handler::read_multi_range_next(KEY_MULTI_RANGE **found_range_p) { - int result; + int UNINIT_VAR(result); DBUG_ENTER("handler::read_multi_range_next"); /* We should not be called after the last call returned EOF. */ -- cgit v1.2.1 From 6bf6272fdabf78e62f04fbcafb1d1e7dee2b27c2 Mon Sep 17 00:00:00 2001 From: Dmitry Lenev Date: Thu, 11 Nov 2010 20:11:05 +0300 Subject: Patch that refactors global read lock implementation and fixes bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK" and bug #54673 "It takes too long to get readlock for 'FLUSH TABLES WITH READ LOCK'". The first bug manifested itself as a deadlock which occurred when a connection, which had some table open through HANDLER statement, tried to update some data through DML statement while another connection tried to execute FLUSH TABLES WITH READ LOCK concurrently. What happened was that FTWRL in the second connection managed to perform first step of GRL acquisition and thus blocked all upcoming DML. After that it started to wait for table open through HANDLER statement to be flushed. When the first connection tried to execute DML it has started to wait for GRL/the second connection creating deadlock. The second bug manifested itself as starvation of FLUSH TABLES WITH READ LOCK statements in cases when there was a constant stream of concurrent DML statements (in two or more connections). This has happened because requests for protection against GRL which were acquired by DML statements were ignoring presence of pending GRL and thus the latter was starved. This patch solves both these problems by re-implementing GRL using metadata locks. Similar to the old implementation acquisition of GRL in new implementation is two-step. During the first step we block all concurrent DML and DDL statements by acquiring global S metadata lock (each DML and DDL statement acquires global IX lock for its duration). During the second step we block commits by acquiring global S lock in COMMIT namespace (commit code acquires global IX lock in this namespace). Note that unlike in old implementation acquisition of protection against GRL in DML and DDL is semi-automatic. We assume that any statement which should be blocked by GRL will either open and acquires write-lock on tables or acquires metadata locks on objects it is going to modify. For any such statement global IX metadata lock is automatically acquired for its duration. The first problem is solved because waits for GRL become visible to deadlock detector in metadata locking subsystem and thus deadlocks like one in the first bug become impossible. The second problem is solved because global S locks which are used for GRL implementation are given preference over IX locks which are acquired by concurrent DML (and we can switch to fair scheduling in future if needed). Important change: FTWRL/GRL no longer blocks DML and DDL on temporary tables. Before this patch behavior was not consistent in this respect: in some cases DML/DDL statements on temporary tables were blocked while in others they were not. Since the main use cases for FTWRL are various forms of backups and temporary tables are not preserved during backups we have opted for consistently allowing DML/DDL on temporary tables during FTWRL/GRL. Important change: This patch changes thread state names which are used when DML/DDL of FTWRL is waiting for global read lock. It is now either "Waiting for global read lock" or "Waiting for commit lock" depending on the stage on which FTWRL is. Incompatible change: To solve deadlock in events code which was exposed by this patch we have to replace LOCK_event_metadata mutex with metadata locks on events. As result we have to prohibit DDL on events under LOCK TABLES. This patch also adds extensive test coverage for interaction of DML/DDL and FTWRL. Performance of new and old global read lock implementations in sysbench tests were compared. There were no significant difference between new and old implementations. mysql-test/include/check_ftwrl_compatible.inc: Added helper script which allows to check that a statement is compatible with FLUSH TABLES WITH READ LOCK. mysql-test/include/check_ftwrl_incompatible.inc: Added helper script which allows to check that a statement is incompatible with FLUSH TABLES WITH READ LOCK. mysql-test/include/handler.inc: Adjusted test case to the fact that now DROP TABLE closes open HANDLERs for the table to be dropped before checking if there active FTWRL in this connection. mysql-test/include/wait_show_condition.inc: Fixed small error in the timeout message. The correct name of variable used as parameter for this script is "$condition" and not "$wait_condition". mysql-test/r/delayed.result: Added test coverage for scenario which triggered assert in metadata locking subsystem. mysql-test/r/events_2.result: Updated test results after prohibiting event DDL operations under LOCK TABLES. mysql-test/r/flush.result: Added test coverage for bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK". mysql-test/r/flush_read_lock.result: Added test coverage for various aspects of FLUSH TABLES WITH READ LOCK functionality. mysql-test/r/flush_read_lock_kill.result: Adjusted test case after replacing custom global read lock implementation with one based on metadata locks. Use new debug_sync point. Do not disable concurrent inserts as now InnoDB we always use InnoDB table. mysql-test/r/handler_innodb.result: Adjusted test case to the fact that now DROP TABLE closes open HANDLERs for the table to be dropped before checking if there active FTWRL in this connection. mysql-test/r/handler_myisam.result: Adjusted test case to the fact that now DROP TABLE closes open HANDLERs for the table to be dropped before checking if there active FTWRL in this connection. mysql-test/r/mdl_sync.result: Adjusted test case after replacing custom global read lock implementation with one based on metadata locks. Replaced usage of GRL-specific debug_sync's with appropriate sync points in MDL subsystem. mysql-test/suite/perfschema/r/dml_setup_instruments.result: Updated test results after removing global COND_global_read_lock condition variable. mysql-test/suite/perfschema/r/func_file_io.result: Ensure that this test doesn't affect subsequent tests. At the end of its execution enable back P_S instrumentation which this test disables at some point. mysql-test/suite/perfschema/r/func_mutex.result: Ensure that this test doesn't affect subsequent tests. At the end of its execution enable back P_S instrumentation which this test disables at some point. mysql-test/suite/perfschema/r/global_read_lock.result: Adjusted test case to take into account that new GRL implementation is based on MDL. mysql-test/suite/perfschema/r/server_init.result: Adjusted test case after replacing custom global read lock implementation with one based on MDL and replacing LOCK_event_metadata mutex with metadata lock. mysql-test/suite/perfschema/t/func_file_io.test: Ensure that this test doesn't affect subsequent tests. At the end of its execution enable back P_S instrumentation which this test disables at some point. mysql-test/suite/perfschema/t/func_mutex.test: Ensure that this test doesn't affect subsequent tests. At the end of its execution enable back P_S instrumentation which this test disables at some point. mysql-test/suite/perfschema/t/global_read_lock.test: Adjusted test case to take into account that new GRL implementation is based on MDL. mysql-test/suite/perfschema/t/server_init.test: Adjusted test case after replacing custom global read lock implementation with one based on MDL and replacing LOCK_event_metadata mutex with metadata lock. mysql-test/suite/rpl/r/rpl_tmp_table_and_DDL.result: Updated test results after prohibiting event DDL under LOCK TABLES. mysql-test/t/delayed.test: Added test coverage for scenario which triggered assert in metadata locking subsystem. mysql-test/t/events_2.test: Updated test case after prohibiting event DDL operations under LOCK TABLES. mysql-test/t/flush.test: Added test coverage for bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK". mysql-test/t/flush_block_commit.test: Adjusted test case after changing thread state name which is used when COMMIT waits for FLUSH TABLES WITH READ LOCK from "Waiting for release of readlock" to "Waiting for commit lock". mysql-test/t/flush_block_commit_notembedded.test: Adjusted test case after changing thread state name which is used when DML waits for FLUSH TABLES WITH READ LOCK. Now we use "Waiting for global read lock" in this case. mysql-test/t/flush_read_lock.test: Added test coverage for various aspects of FLUSH TABLES WITH READ LOCK functionality. mysql-test/t/flush_read_lock_kill-master.opt: We no longer need to use make_global_read_lock_block_commit_loop debug tag in this test. Instead we rely on an appropriate debug_sync point in MDL code. mysql-test/t/flush_read_lock_kill.test: Adjusted test case after replacing custom global read lock implementation with one based on metadata locks. Use new debug_sync point. Do not disable concurrent inserts as now InnoDB we always use InnoDB table. mysql-test/t/lock_multi.test: Adjusted test case after changing thread state names which are used when DML or DDL waits for FLUSH TABLES WITH READ LOCK to "Waiting for global read lock". mysql-test/t/mdl_sync.test: Adjusted test case after replacing custom global read lock implementation with one based on metadata locks. Replaced usage of GRL-specific debug_sync's with appropriate sync points in MDL subsystem. Updated thread state names which are used when DDL waits for FTWRL. mysql-test/t/trigger_notembedded.test: Adjusted test case after changing thread state names which are used when DML or DDL waits for FLUSH TABLES WITH READ LOCK to "Waiting for global read lock". sql/event_data_objects.cc: Removed Event_queue_element::status/last_executed_changed members and Event_queue_element::update_timing_fields() method. We no longer use this class for updating mysql.events once event is chosen for execution. Accesses to instances of this class in scheduler thread require protection by Event_queue::LOCK_event_queue mutex and we try to avoid updating table while holding this lock. sql/event_data_objects.h: Removed Event_queue_element::status/last_executed_changed members and Event_queue_element::update_timing_fields() method. We no longer use this class for updating mysql.events once event is chosen for execution. Accesses to instances of this class in scheduler thread require protection by Event_queue::LOCK_event_queue mutex and we try to avoid updating table while holding this lock. sql/event_db_repository.cc: - Changed Event_db_repository methods to not release all metadata locks once they are done updating mysql.events table. This allows to keep metadata lock protecting against GRL and lock protecting particular event around until corresponding DDL statement is written to the binary log. - Removed logic for conditional update of "status" and "last_executed" fields from update_timing_fields_for_event() method. In the only case when this method is called now "last_executed" is always modified and tracking change of "status" is too much hassle. sql/event_db_repository.h: Removed logic for conditional update of "status" and "last_executed" fields from Event_db_repository:: update_timing_fields_for_event() method. In the only case when this method is called now "last_executed" is always modified and tracking change of "status" field is too much hassle. sql/event_queue.cc: Changed event scheduler code not to update mysql.events table while holding Event_queue::LOCK_event_queue mutex. Doing so led to a deadlock with a new GRL implementation. This deadlock didn't occur with old implementation due to fact that code acquiring protection against GRL ignored pending GRL requests (which lead to GRL starvation). One of goals of new implementation is to disallow GRL starvation and so we have to solve problem with this deadlock in a different way. sql/events.cc: Changed methods of Events class to acquire protection against GRL while perfoming DDL statement and keep it until statement is written to the binary log. Unfortunately this step together with new GRL implementation exposed deadlock involving Events::LOCK_event_metadata and GRL. To solve it Events::LOCK_event_metadata mutex was replaced with a metadata lock on event. As a side-effect events DDL has to be prohibited under LOCK TABLES even in cases when mysql.events table was explicitly locked for write. sql/events.h: Replaced Events::LOCK_event_metadata mutex with a metadata lock on event. sql/ha_ndbcluster.cc: Updated code after replacing custom global read lock implementation with one based on MDL. Since MDL subsystem should now be able to detect deadlocks involving metadata locks and GRL there is no need for special handling of active GRL. sql/handler.cc: Replaced custom implementation of global read lock with one based on metadata locks. Consequently when doing commit instead of calling method of Global_read_lock class to acquire protection against GRL we simply acquire IX in COMMIT namespace. sql/lock.cc: Replaced custom implementation of global read lock with one based on metadata locks. This step allows to expose wait for GRL to deadlock detector of MDL subsystem and thus succesfully resolve deadlocks similar to one behind bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK". It also solves problem with GRL starvation described in bug #54673 "It takes too long to get readlock for 'FLUSH TABLES WITH READ LOCK'" since metadata locks used by GRL give preference to FTWRL statement instead of DML statements (if needed in future this can be changed to fair scheduling). Similar to old implementation of acquisition of GRL is two-step. During the first step we block all concurrent DML and DDL statements by acquiring global S metadata lock (each DML and DDL statement acquires global IX lock for its duration). During the second step we block commits by acquiring global S lock in COMMIT namespace (commit code acquires global IX lock in this namespace). Note that unlike in old implementation acquisition of protection against GRL in DML and DDL is semi-automatic. We assume that any statement which should be blocked by GRL will either open and acquires write-lock on tables or acquires metadata locks on objects it is going to modify. For any such statement global IX metadata lock is automatically acquired for its duration. To support this change: - Global_read_lock::lock/unlock_global_read_lock and make_global_read_lock_block_commit methods were changed accordingly. - Global_read_lock::wait_if_global_read_lock() and start_waiting_global_read_lock() methods were dropped. It is now responsibility of code acquiring metadata locks opening tables to acquire protection against GRL by explicitly taking global IX lock with statement duration. - Global variables, mutex and condition variable used by old implementation was removed. - lock_routine_name() was changed to use statement duration for its global IX lock. It was also renamed to lock_object_name() as it now also used to take metadata locks on events. - Global_read_lock::set_explicit_lock_duration() was added which allows not to release locks used for GRL when leaving prelocked mode. sql/lock.h: - Renamed lock_routine_name() to lock_object_name() and changed its signature to allow its usage for events. - Removed broadcast_refresh() function. It is no longer needed with new GRL implementation. sql/log_event.cc: Release metadata locks with statement duration at the end of processing legacy event for LOAD DATA. This ensures that replication thread processing such event properly releases its protection against global read lock. sql/mdl.cc: Changed MDL subsystem to support new MDL-based implementation of global read lock. Added COMMIT and EVENTS namespaces for metadata locks. Changed thread state name for GLOBAL namespace to "Waiting for global read lock". Optimized MDL_map::find_or_insert() method to avoid taking m_mutex mutex when looking up MDL_lock objects for GLOBAL or COMMIT namespaces. We keep pre-created MDL_lock objects for these namespaces around and simply return pointers to these global objects when needed. Changed MDL_lock/MDL_scoped_lock to properly handle notification of insert delayed handler threads when FTWRL takes global S lock. Introduced concept of lock duration. In addition to locks with transaction duration which work in the way which is similar to how locks worked before (i.e. they are released at the end of transaction), locks with statement and explicit duration were introduced. Locks with statement duration are automatically released at the end of statement. Locks with explicit duration require explicit release and obsolete concept of transactional sentinel. * Changed MDL_request and MDL_ticket classes to support notion of duration. * Changed MDL_context to keep locks with different duration in different lists. Changed code handling ticket list to take this into account. * Changed methods responsible for releasing locks to take into account duration of tickets. Particularly public MDL_context::release_lock() method now only can release tickets with explicit duration (there is still internal method which allows to specify duration). To release locks with statement or transaction duration one have to use release_statement/transactional_locks() methods. * Concept of savepoint for MDL subsystem now has to take into account locks with statement duration. Consequently MDL_savepoint class was introduced and methods working with savepoints were updated accordingly. * Added methods which allow to set duration for one or all locks in the context. sql/mdl.h: Changed MDL subsystem to support new MDL-based implementation of global read lock. Added COMMIT and EVENTS namespaces for metadata locks. Introduced concept of lock duration. In addition to locks with transaction duration which work in the way which is similar to how locks worked before (i.e. they are released at the end of transaction), locks with statement and explicit duration were introduced. Locks with statement duration are automatically released at the end of statement. Locks with explicit duration require explicit release and obsolete concept of transactional sentinel. * Changed MDL_request and MDL_ticket classes to support notion of duration. * Changed MDL_context to keep locks with different duration in different lists. Changed code handling ticket list to take this into account. * Changed methods responsible for releasing locks to take into account duration of tickets. Particularly public MDL_context::release_lock() method now only can release tickets with explicit duration (there is still internal method which allows to specify duration). To release locks with statement or transaction duration one have to use release_statement/transactional_locks() methods. * Concept of savepoint for MDL subsystem now has to take into account locks with statement duration. Consequently MDL_savepoint class was introduced and methods working with savepoints were updated accordingly. * Added methods which allow to set duration for one or all locks in the context. sql/mysqld.cc: Removed global mutex and condition variables which were used by old implementation of GRL. Also we no longer need to initialize Events::LOCK_event_metadata mutex as it was replaced with metadata locks on events. sql/mysqld.h: Removed global variable, mutex and condition variables which were used by old implementation of GRL. sql/rpl_rli.cc: When slave thread closes tables which were open for handling of RBR events ensure that it releases global IX lock which was acquired as protection against GRL. sql/sp.cc: Adjusted code to the new signature of lock_object/routine_name(), to the fact that one now needs specify duration of lock when initializing MDL_request and to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sp_head.cc: Ensure that statements in stored procedures release statement metadata locks and thus release their protectiong against GRL in proper moment in time. Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/sql_admin.cc: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/sql_base.cc: - Implemented support for new approach to acquiring protection against global read lock. We no longer acquire such protection explicitly on the basis of statement flags. Instead we always rely on code which is responsible for acquiring metadata locks on object to be changed acquiring this protection. This is achieved by acquiring global IX metadata lock with statement duration. Code doing this also responsible for checking that current connection has no active GRL by calling an Global_read_lock::can_acquire_protection() method. Changed code in open_table() and lock_table_names() accordingly. Note that as result of this change DDL and DML on temporary tables is always compatible with GRL (before it was incompatible in some cases and compatible in other cases). - To speed-up code acquiring protection against GRL introduced m_has_protection_against_grl member in Open_table_context class. It indicates that protection was already acquired sometime during open_tables() execution and new attempts can be skipped. - Thanks to new GRL implementation calls to broadcast_refresh() became unnecessary and were removed. - Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request and to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sql_base.h: Adjusted code to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. Also introduced Open_table_context::m_has_protection_against_grl member which allows to avoid acquiring protection against GRL while opening tables if such protection was already acquired. sql/sql_class.cc: Changed THD::leave_locked_tables_mode() after transactional sentinel for metadata locks was obsoleted by introduction of locks with explicit duration. sql/sql_class.h: - Adjusted code to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. - Changed Global_read_lock class according to changes in global read lock implementation: * wait_if_global_read_lock and start_waiting_global_read_lock are now gone. Instead code needing protection against GRL has to acquire global IX metadata lock with statement duration itself. To help it new can_acquire_protection() was introduced. Also as result of the above change m_protection_count member is gone too. * Added m_mdl_blocks_commits_lock member to store metadata lock blocking commits. * Adjusted code to the fact that concept of transactional sentinel was obsoleted by concept of lock duration. - Removed CF_PROTECT_AGAINST_GRL flag as it is no longer necessary. New GRL implementation acquires protection against global read lock automagically when statement acquires metadata locks on tables or other objects it is going to change. sql/sql_db.cc: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/sql_handler.cc: Removed call to broadcast_refresh() function. It is no longer needed with new GRL implementation. Adjusted code after introducing duration concept for metadata locks. Particularly to the fact transactional sentinel was replaced with explicit duration. sql/sql_handler.h: Renamed mysql_ha_move_tickets_after_trans_sentinel() to mysql_ha_set_explicit_lock_duration() after transactional sentinel was obsoleted by locks with explicit duration. sql/sql_insert.cc: Adjusted code handling delaying inserts after switching to new GRL implementation. Now connection thread initiating delayed insert has to acquire global IX lock in addition to metadata lock on table being inserted into. This IX lock protects against GRL and similarly to SW lock on table being inserted into has to be passed to handler thread in order to avoid deadlocks. sql/sql_lex.cc: LEX::protect_against_global_read_lock member is no longer necessary since protection against GRL is automatically taken by code acquiring metadata locks/opening tables. sql/sql_lex.h: LEX::protect_against_global_read_lock member is no longer necessary since protection against GRL is automatically taken by code acquiring metadata locks/opening tables. sql/sql_parse.cc: - Implemented support for new approach to acquiring protection against global read lock. We no longer acquire such protection explicitly on the basis of statement flags. Instead we always rely on code which is responsible for acquiring metadata locks on object to be changed acquiring this protection. This is achieved by acquiring global IX metadata lock with statement duration. This lock is automatically released at the end of statement execution. - Changed implementation of CREATE/DROP PROCEDURE/FUNCTION not to release metadata locks and thus protection against of GRL in the middle of statement execution. - Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request and to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sql_prepare.cc: Adjusted code to the to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sql_rename.cc: With new GRL implementation there is no need to explicitly acquire protection against GRL before renaming tables. This happens automatically in code which acquires metadata locks on tables being renamed. sql/sql_show.cc: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request and to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sql_table.cc: - With new GRL implementation there is no need to explicitly acquire protection against GRL before dropping tables. This happens automatically in code which acquires metadata locks on tables being dropped. - Changed mysql_alter_table() not to release lock on new table name explicitly and to rely on automatic release of locks at the end of statement instead. This was necessary since now MDL_context::release_lock() is supported only for locks for explicit duration. sql/sql_trigger.cc: With new GRL implementation there is no need to explicitly acquire protection against GRL before changing table triggers. This happens automatically in code which acquires metadata locks on tables which triggers are to be changed. sql/sql_update.cc: Fix bug exposed by GRL testing. During prepare phase acquire only S metadata locks instead of SW locks to keep prepare of multi-UPDATE compatible with concurrent LOCK TABLES WRITE and global read lock. sql/sql_view.cc: With new GRL implementation there is no need to explicitly acquire protection against GRL before creating view. This happens automatically in code which acquires metadata lock on view to be created. sql/sql_yacc.yy: LEX::protect_against_global_read_lock member is no longer necessary since protection against GRL is automatically taken by code acquiring metadata locks/opening tables. sql/table.cc: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/table.h: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/transaction.cc: Replaced custom implementation of global read lock with one based on metadata locks. Consequently when doing commit instead of calling method of Global_read_lock class to acquire protection against GRL we simply acquire IX in COMMIT namespace. Also adjusted code to the fact that MDL savepoint is now represented by MDL_savepoint class. --- sql/handler.cc | 40 ++++++++++++++++++++++++++++++++-------- 1 file changed, 32 insertions(+), 8 deletions(-) (limited to 'sql/handler.cc') diff --git a/sql/handler.cc b/sql/handler.cc index eb060002f48..22bb3069a9f 100644 --- a/sql/handler.cc +++ b/sql/handler.cc @@ -29,8 +29,6 @@ #include "sql_cache.h" // query_cache, query_cache_* #include "key.h" // key_copy, key_unpack, key_cmp_if_same, key_cmp #include "sql_table.h" // build_table_filename -#include "lock.h" // wait_if_global_read_lock, - // start_waiting_global_read_lock #include "sql_parse.h" // check_stack_overrun #include "sql_acl.h" // SUPER_ACL #include "sql_base.h" // free_io_cache @@ -41,6 +39,7 @@ #include "transaction.h" #include #include "probes_mysql.h" +#include "debug_sync.h" // DEBUG_SYNC #ifdef WITH_PARTITION_STORAGE_ENGINE #include "ha_partition.h" @@ -1155,6 +1154,7 @@ int ha_commit_trans(THD *thd, bool all) { uint rw_ha_count; bool rw_trans; + MDL_request mdl_request; DBUG_EXECUTE_IF("crash_commit_before", DBUG_SUICIDE();); @@ -1166,11 +1166,27 @@ int ha_commit_trans(THD *thd, bool all) /* rw_trans is TRUE when we in a transaction changing data */ rw_trans= is_real_trans && (rw_ha_count > 0); - if (rw_trans && - thd->global_read_lock.wait_if_global_read_lock(thd, FALSE, FALSE)) + if (rw_trans) { - ha_rollback_trans(thd, all); - DBUG_RETURN(1); + /* + Acquire a metadata lock which will ensure that COMMIT is blocked + by an active FLUSH TABLES WITH READ LOCK (and vice versa: + COMMIT in progress blocks FTWRL). + + We allow the owner of FTWRL to COMMIT; we assume that it knows + what it does. + */ + mdl_request.init(MDL_key::COMMIT, "", "", MDL_INTENTION_EXCLUSIVE, + MDL_EXPLICIT); + + if (thd->mdl_context.acquire_lock(&mdl_request, + thd->variables.lock_wait_timeout)) + { + ha_rollback_trans(thd, all); + DBUG_RETURN(1); + } + + DEBUG_SYNC(thd, "ha_commit_trans_after_acquire_commit_lock"); } if (rw_trans && @@ -1225,8 +1241,16 @@ int ha_commit_trans(THD *thd, bool all) DBUG_EXECUTE_IF("crash_commit_after", DBUG_SUICIDE();); RUN_HOOK(transaction, after_commit, (thd, FALSE)); end: - if (rw_trans) - thd->global_read_lock.start_waiting_global_read_lock(thd); + if (rw_trans && mdl_request.ticket) + { + /* + We do not always immediately release transactional locks + after ha_commit_trans() (see uses of ha_enable_transaction()), + thus we release the commit blocker lock as soon as it's + not needed. + */ + thd->mdl_context.release_lock(mdl_request.ticket); + } } /* Free resources and perform other cleanup even for 'empty' transactions. */ else if (is_real_trans) -- cgit v1.2.1 From 647c619393f713ad6725d990128a8d1911e956f8 Mon Sep 17 00:00:00 2001 From: Luis Soares Date: Tue, 30 Nov 2010 23:32:51 +0000 Subject: BUG#46166: MYSQL_BIN_LOG::new_file_impl is not propagating error when generating new name. If find_uniq_filename returns an error, then this error is not being propagated upwards, and execution does not report error to the user (although a entry in the error log is generated). Additionally, some more errors were ignored in new_file_impl: - when writing the rotate event - when reopening the index and binary log file This patch addresses this by propagating the error up in the execution stack. Furthermore, when rotation of the binary log fails, an incident event is written, because there may be a chance that some changes for a given statement, were not properly logged. For example, in SBR, LOAD DATA INFILE statement requires more than one event to be logged, should rotation fail while logging part of the LOAD DATA events, then the logged data would become inconsistent with the data in the storage engine. mysql-test/include/restart_mysqld.inc: Refactored restart_mysqld so that it is not hardcoded for mysqld.1, but rather for the current server. mysql-test/suite/binlog/t/binlog_index.test: The error on open of index and binary log on new_file_impl is now caught. Thence the user will get an error message. We need to accomodate this change in the test case for the failing FLUSH LOGS. mysql-test/suite/rpl/t/rpl_binlog_errors-master.opt: Sets max_binlog_size to 4096. mysql-test/suite/rpl/t/rpl_binlog_errors.test: Added some test cases for asserting that the error is found and reported. sql/handler.cc: Catching error now returned by unlog (in ha_commit_trans) and returning it. sql/log.cc: Propagating errors from new_file_impl upwards. The errors that new_file_impl catches now are: - error on generate_new_name - error on writing the rotate event - error when opening the index or the binary log file. sql/log.h: Changing declaration of: - rotate_and_purge - new_file - new_file_without_locking - new_file_impl - unlog They now return int instead of void. sql/mysql_priv.h: Change signature of reload_acl_and_cache so that write_to_binlog is an int instead of bool. sql/mysqld.cc: Redeclaring not_used var as int instead of bool. sql/rpl_injector.cc: Changes to catch the return from rotate_and_purge. sql/slave.cc: Changes to catch the return values for new_file and rotate_relay_log. sql/slave.h: Changes to rotate_relay_log declaration (now returns int instead of void). sql/sql_load.cc: In SBR, some logging of LOAD DATA events goes through IO_CACHE_CALLBACK invocation at mf_iocache.c:_my_b_get. The IO_CACHE implementation is ignoring the return value for from these callbacks (pre_read and post_read), so we need to find out at the end of the execution if the error is set or not in THD. sql/sql_parse.cc: Catching the rotate_relay_log and rotate_and_purge return values. Semantic change in reload_acl_and_cache so that we report errors in binlog interactions through the write_to_binlog output parameter. If there was any failure while rotating the binary log, we should then report the error to the client when handling SQLCOMM_FLUSH. --- sql/handler.cc | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) (limited to 'sql/handler.cc') diff --git a/sql/handler.cc b/sql/handler.cc index 1525ca53bca..18381bc552c 100644 --- a/sql/handler.cc +++ b/sql/handler.cc @@ -1192,7 +1192,11 @@ int ha_commit_trans(THD *thd, bool all) error=ha_commit_one_phase(thd, all) ? (cookie ? 2 : 1) : 0; DBUG_EXECUTE_IF("crash_commit_before_unlog", DBUG_SUICIDE();); if (cookie) - tc_log->unlog(cookie, xid); + if(tc_log->unlog(cookie, xid)) + { + error= 2; + goto end; + } DBUG_EXECUTE_IF("crash_commit_after", DBUG_SUICIDE();); end: if (rw_trans) -- cgit v1.2.1 From 1faf910eeb4e8f40253bf34b634af5332f367dc5 Mon Sep 17 00:00:00 2001 From: Sergey Glukhov Date: Mon, 13 Dec 2010 14:48:12 +0300 Subject: Bug#39828 : Autoinc wraps around when offset and increment > 1 Auto increment value wraps when performing a bulk insert with auto_increment_increment and auto_increment_offset greater than one. The fix: If overflow happened then return MAX_ULONGLONG value as an indication of overflow and check this before storing the value into the field in update_auto_increment(). mysql-test/r/auto_increment.result: test case mysql-test/suite/innodb/r/innodb-autoinc.result: test case fix mysql-test/suite/innodb/t/innodb-autoinc.test: test case fix mysql-test/suite/innodb_plugin/r/innodb-autoinc.result: test case fix mysql-test/suite/innodb_plugin/t/innodb-autoinc.test: test case fix mysql-test/t/auto_increment.test: test case sql/handler.cc: If overflow happened then return MAX_ULONGLONG value as an indication of overflow and check this before storing the value into the field in update_auto_increment(). --- sql/handler.cc | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) (limited to 'sql/handler.cc') diff --git a/sql/handler.cc b/sql/handler.cc index 1525ca53bca..33c7f518838 100644 --- a/sql/handler.cc +++ b/sql/handler.cc @@ -2166,7 +2166,8 @@ int handler::read_first_row(uchar * buf, uint primary_key) computes the lowest number - strictly greater than "nr" - of the form: auto_increment_offset + N * auto_increment_increment - + If overflow happened then return MAX_ULONGLONG value as an + indication of overflow. In most cases increment= offset= 1, in which case we get: @verbatim 1,2,3,4,5,... @endverbatim If increment=10 and offset=5 and previous number is 1, we get: @@ -2175,13 +2176,23 @@ int handler::read_first_row(uchar * buf, uint primary_key) inline ulonglong compute_next_insert_id(ulonglong nr,struct system_variables *variables) { + const ulonglong save_nr= nr; + if (variables->auto_increment_increment == 1) - return (nr+1); // optimization of the formula below - nr= (((nr+ variables->auto_increment_increment - - variables->auto_increment_offset)) / - (ulonglong) variables->auto_increment_increment); - return (nr* (ulonglong) variables->auto_increment_increment + - variables->auto_increment_offset); + nr= nr + 1; // optimization of the formula below + else + { + nr= (((nr+ variables->auto_increment_increment - + variables->auto_increment_offset)) / + (ulonglong) variables->auto_increment_increment); + nr= (nr* (ulonglong) variables->auto_increment_increment + + variables->auto_increment_offset); + } + + if (unlikely(nr <= save_nr)) + return ULONGLONG_MAX; + + return nr; } @@ -2392,7 +2403,7 @@ int handler::update_auto_increment() variables->auto_increment_increment, nb_desired_values, &nr, &nb_reserved_values); - if (nr == ~(ulonglong) 0) + if (nr == ULONGLONG_MAX) DBUG_RETURN(HA_ERR_AUTOINC_READ_FAILED); // Mark failure /* @@ -2423,6 +2434,9 @@ int handler::update_auto_increment() } } + if (unlikely(nr == ULONGLONG_MAX)) + DBUG_RETURN(HA_ERR_AUTOINC_ERANGE); + DBUG_PRINT("info",("auto_increment: %lu", (ulong) nr)); if (unlikely(table->next_number_field->store((longlong) nr, TRUE))) -- cgit v1.2.1 From 0a7cfad0804ab912d185c9478872b990783932bd Mon Sep 17 00:00:00 2001 From: Jan Wedvik Date: Tue, 11 Jan 2011 12:09:54 +0100 Subject: Fix for bug#58553, "Queries with pushed conditions causes 'explain extended' to crash mysqld". handler::pushed_cond was not always properly reset when table objects where recycled via the table cache. handler::pushed_cond is now set to NULL in handler::ha_reset(). This should prevent pushed conditions from (incorrectly) re-apperaring in later queries. --- sql/handler.cc | 1 + 1 file changed, 1 insertion(+) (limited to 'sql/handler.cc') diff --git a/sql/handler.cc b/sql/handler.cc index ce4e4a9e3a9..711d2942ce0 100644 --- a/sql/handler.cc +++ b/sql/handler.cc @@ -4657,6 +4657,7 @@ int handler::ha_reset() free_io_cache(table); /* reset the bitmaps to point to defaults */ table->default_column_bitmaps(); + pushed_cond= NULL; DBUG_RETURN(reset()); } -- cgit v1.2.1 From de3c4428b8c759e85631d8d70b5845c872de5400 Mon Sep 17 00:00:00 2001 From: Karen Langford Date: Tue, 25 Jan 2011 15:42:40 +0100 Subject: Updating header copyright/README in source for 2011 --- sql/handler.cc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'sql/handler.cc') diff --git a/sql/handler.cc b/sql/handler.cc index 711d2942ce0..5968a78b587 100644 --- a/sql/handler.cc +++ b/sql/handler.cc @@ -1,4 +1,4 @@ -/* Copyright 2000-2008 MySQL AB, 2008 Sun Microsystems, Inc. +/* Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by -- cgit v1.2.1 From d1d166875ac2eec18d15611d0d82ba49b6def288 Mon Sep 17 00:00:00 2001 From: Jorgen Loland Date: Mon, 21 Feb 2011 16:49:03 +0100 Subject: BUG#11762751: UPDATE STATEMENT THROWS AN ERROR, BUT STILL UPDATES THE TABLE ENTRIES (formerly 55385) BUG#11764529: MULTI UPDATE+INNODB REPORTS ER_KEY_NOT_FOUND IF A TABLE IS UPDATED TWICE (formerly 57373) If multiple-table update updates a row through two aliases and the first update physically moves the row, the second update will fail to locate the row. This results in different errors depending on storage engine: * MyISAM: Got error 134 from storage engine * InnoDB: Can't find record in 'tbl' None of these errors accurately describe the problem. Furthermore, since MyISAM is non-transactional, the update executed first will be performed while the second will not. In addition, for two equal multiple-table update statements, one could succeed and the other fail based on whether or not the record actually moved or not. This was inconsistent. Two update operations may physically move a row: 1) Update of a column in a clustered primary key 2) Update of a column used to calculate which partition the row belongs to BUG#11764529 is about case 1) above, BUG#11762751 was about case 2). The fix for these bugs is to return with an error if multiple-table update is about to: a) Update a table through multiple aliases, and b) Perform an update that may physically more the row in at least one of these aliases This avoids * partial updates as described for MyISAM above, * provides the same error message that describes the actual problem for all SEs * inconsistent behavior where a statement fails or succeeds based on e.g. the partitioning algorithm of the table. mysql-test/r/multi_update.result: Add test for bug#57373 mysql-test/r/multi_update_innodb.result: Add test for bug#57373 mysql-test/r/partition.result: Add test for bug#55385 mysql-test/t/multi_update.test: Add test for bug#57373 mysql-test/t/multi_update_innodb.test: Add test for bug#57373 mysql-test/t/partition.test: Add test for bug#55385 sql/handler.cc: Translate handler error HA_ERR_RECORD_DELETED to server error sql/share/errmsg-utf8.txt: New error message for multi-table update where the same table is updated multiple times. sql/sql_update.cc: Add function unsafe_key_update() --- sql/handler.cc | 1 + 1 file changed, 1 insertion(+) (limited to 'sql/handler.cc') diff --git a/sql/handler.cc b/sql/handler.cc index f7690c28827..9ca1f3a20f8 100644 --- a/sql/handler.cc +++ b/sql/handler.cc @@ -2700,6 +2700,7 @@ void handler::print_error(int error, myf errflag) break; case HA_ERR_KEY_NOT_FOUND: case HA_ERR_NO_ACTIVE_RECORD: + case HA_ERR_RECORD_DELETED: case HA_ERR_END_OF_FILE: textno=ER_KEY_NOT_FOUND; break; -- cgit v1.2.1 From 0db0e64f33bda17852d9de57aa5a1d376af95db2 Mon Sep 17 00:00:00 2001 From: Jon Olav Hauglid Date: Tue, 8 Mar 2011 09:41:57 +0100 Subject: Bug #11755431 (former 47205) MAP 'REPAIR TABLE' TO RECREATE +ANALYZE FOR ENGINES NOT SUPPORTING NATIVE REPAIR Executing 'mysqlcheck --check-upgrade --auto-repair ...' will first issue 'CHECK TABLE FOR UPGRADE' for all tables in the database in order to check if the tables are compatible with the current version of MySQL. Any tables that are found incompatible are then upgraded using 'REPAIR TABLE'. The problem was that some engines (e.g. InnoDB) do not support 'REPAIR TABLE'. This caused any such tables to be left incompatible. As a result such tables were not properly fixed by the mysql_upgrade tool. This patch fixes the problem by first changing 'CHECK TABLE FOR UPGRADE' to return a different error message if the engine does not support REPAIR. Instead of "Table upgrade required. Please do "REPAIR TABLE ..." it will report "Table rebuild required. Please do "ALTER TABLE ... FORCE ..." Second, the patch changes mysqlcheck to do 'ALTER TABLE ... FORCE' instead of 'REPAIR TABLE' in these cases. This patch also fixes 'ALTER TABLE ... FORCE' to actually rebuild the table. This change should be reflected in the documentation. Before this patch, 'ALTER TABLE ... FORCE' was unused (See Bug#11746162) Test case added to mysqlcheck.test client/mysqlcheck.c: Changed mysqlcheck to do 'ALTER TABLE ... FORCE' if 'CHECK TABLE FOR UPGRADE' reports ER_TABLE_NEEDS_REBUILD and not ER_TABLE_NEEDS_UPGRADE. mysql-test/r/mysqlcheck.result: Added regression test. mysql-test/std_data/bug47205.frm: InnoDB 5.0 FRM which contains a varchar primary key using utf8_general_ci. This is an incompatible FRM for 5.5. mysql-test/t/mysqlcheck.test: Added regression test. sql/handler.h: Added new HA_CAN_REPAIR flag. sql/share/errmsg-utf8.txt: Added new error message ER_TABLE_NEEDS_REBUILD sql/sql_admin.cc: Changed 'CHECK TABLE FOR UPDATE' to give ER_TABLE_NEEDS_REBUILD instead of ER_TABLE_NEEDS_UPGRADE if the engine does not support REPAIR (as indicated by the new HA_CAN_REPAIR flag). sql/sql_lex.h: Remove unused ALTER_FORCE flag. sql/sql_yacc.yy: Make sure ALTER TABLE ... FORCE recreates the table by setting the ALTER_RECREATE flag as the ALTER_FORCE flag was unused. storage/archive/ha_archive.h: Added new HA_CAN_REPAIR flag to Archive storage/csv/ha_tina.h: Added new HA_CAN_REPAIR flag to CSV storage/federated/ha_federated.h: Added new HA_CAN_REPAIR flag to Federated storage/myisam/ha_myisam.cc: Added new HA_CAN_REPAIR flag to MyISAM --- sql/handler.cc | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) (limited to 'sql/handler.cc') diff --git a/sql/handler.cc b/sql/handler.cc index 9ca1f3a20f8..fefc0553c88 100644 --- a/sql/handler.cc +++ b/sql/handler.cc @@ -3214,9 +3214,13 @@ int handler::ha_repair(THD* thd, HA_CHECK_OPT* check_opt) mark_trx_read_write(); - if ((result= repair(thd, check_opt))) - return result; - return update_frm_version(table); + result= repair(thd, check_opt); + DBUG_ASSERT(result == HA_ADMIN_NOT_IMPLEMENTED || + ha_table_flags() & HA_CAN_REPAIR); + + if (result == HA_ADMIN_OK) + result= update_frm_version(table); + return result; } -- cgit v1.2.1 From e0887df8e1127c0f1410b9d4ad61647cb5f93be2 Mon Sep 17 00:00:00 2001 From: Mattias Jonsson Date: Fri, 25 Mar 2011 12:36:02 +0100 Subject: Bug#11766249 bug#59316: PARTITIONING AND INDEX_MERGE MEMORY LEAK When executing row-ordered-retrieval index merge, the handler was cloned, but it used the wrong memory root, so instead of allocating memory on the thread/query's mem_root, it used the table's mem_root, resulting in non released memory in the table object, and was not freed until the table was closed. Solution was to ensure that memory used during cloning of a handler was allocated from the correct memory root. This was implemented by fixing handler::clone() to also take a name argument, so it can be used with partitioning. And in ha_partition only allocate the ha_partition's ref, and call the original ha_partition partitions clone() and set at cloned partitions. Fix of .bzrignore on Windows with VS 2010 --- sql/handler.cc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'sql/handler.cc') diff --git a/sql/handler.cc b/sql/handler.cc index 5968a78b587..8adb8e061a3 100644 --- a/sql/handler.cc +++ b/sql/handler.cc @@ -2037,9 +2037,9 @@ int ha_delete_table(THD *thd, handlerton *table_type, const char *path, /**************************************************************************** ** General handler functions ****************************************************************************/ -handler *handler::clone(MEM_ROOT *mem_root) +handler *handler::clone(const char *name, MEM_ROOT *mem_root) { - handler *new_handler= get_new_handler(table->s, mem_root, table->s->db_type()); + handler *new_handler= get_new_handler(table->s, mem_root, ht); /* Allocate handler->ref here because otherwise ha_open will allocate it on this->table->mem_root and we will not be able to reclaim that memory @@ -2048,7 +2048,7 @@ handler *handler::clone(MEM_ROOT *mem_root) if (!(new_handler->ref= (uchar*) alloc_root(mem_root, ALIGN_SIZE(ref_length)*2))) return NULL; if (new_handler && !new_handler->ha_open(table, - table->s->normalized_path.str, + name, table->db_stat, HA_OPEN_IGNORE_IF_LOCKED)) return new_handler; -- cgit v1.2.1 From bd92ea43116b8ce606de5e6fc825e1a8b87a7740 Mon Sep 17 00:00:00 2001 From: Mattias Jonsson Date: Wed, 20 Apr 2011 17:52:33 +0200 Subject: Bug#11766249 bug#59316: PARTITIONING AND INDEX_MERGE MEMORY LEAK Update for previous patch according to reviewers comments. Updated the constructors for ha_partitions to use the common init_handler_variables functions Added use of defines for size and offset to get better readability for the code that reads and writes the .par file. Also refactored the get_from_handler_file function. --- sql/handler.cc | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) (limited to 'sql/handler.cc') diff --git a/sql/handler.cc b/sql/handler.cc index 8adb8e061a3..718529fa5fc 100644 --- a/sql/handler.cc +++ b/sql/handler.cc @@ -2045,14 +2045,21 @@ handler *handler::clone(const char *name, MEM_ROOT *mem_root) on this->table->mem_root and we will not be able to reclaim that memory when the clone handler object is destroyed. */ - if (!(new_handler->ref= (uchar*) alloc_root(mem_root, ALIGN_SIZE(ref_length)*2))) - return NULL; - if (new_handler && !new_handler->ha_open(table, - name, - table->db_stat, - HA_OPEN_IGNORE_IF_LOCKED)) - return new_handler; - return NULL; + if (new_handler && + !(new_handler->ref= (uchar*) alloc_root(mem_root, + ALIGN_SIZE(ref_length)*2))) + new_handler= NULL; + /* + TODO: Implement a more efficient way to have more than one index open for + the same table instance. The ha_open call is not cachable for clone. + */ + if (new_handler && new_handler->ha_open(table, + name, + table->db_stat, + HA_OPEN_IGNORE_IF_LOCKED)) + new_handler= NULL; + + return new_handler; } -- cgit v1.2.1 From 12c42b980aa94105a9108d4f58c5dc75d96ec824 Mon Sep 17 00:00:00 2001 From: Guilhem Bichot Date: Mon, 16 May 2011 22:04:01 +0200 Subject: Fix for BUG#11755168 '46895: test "outfile_loaddata" fails (reproducible)'. In sql_class.cc, 'row_count', of type 'ha_rows', was used as last argument for ER_TRUNCATED_WRONG_VALUE_FOR_FIELD which is "Incorrect %-.32s value: '%-.128s' for column '%.192s' at row %ld". So 'ha_rows' was used as 'long'. On SPARC32 Solaris builds, 'long' is 4 bytes and 'ha_rows' is 'longlong' i.e. 8 bytes. So the printf-like code was reading only the first 4 bytes. Because the CPU is big-endian, 1LL is 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x01 so the first four bytes yield 0. So the warning message had "row 0" instead of "row 1" in test outfile_loaddata.test: -Warning 1366 Incorrect string value: '\xE1\xE2\xF7' for column 'b' at row 1 +Warning 1366 Incorrect string value: '\xE1\xE2\xF7' for column 'b' at row 0 All error-messaging functions which internally invoke some printf-life function are potential candidate for such mistakes. One apparently easy way to catch such mistakes is to use ATTRIBUTE_FORMAT (from my_attribute.h). But this works only when call site has both: a) the format as a string literal b) the types of arguments. So: func(ER(ER_BLAH), 10); will silently not be checked, because ER(ER_BLAH) is not known at compile time (it is known at run-time, and depends on the chosen language). And func("%s", a va_list argument); has the same problem, as the *real* type of arguments is not known at this site at compile time (it's known in some caller). Moreover, func(ER(ER_BLAH)); though possibly correct (if ER(ER_BLAH) has no '%' markers), will not compile (gcc says "error: format not a string literal and no format arguments"). Consequences: 1) ATTRIBUTE_FORMAT is here added only to functions which in practice take "string literal" formats: "my_error_reporter" and "print_admin_msg". 2) it cannot be added to the other functions: my_error(), push_warning_printf(), Table_check_intact::report_error(), general_log_print(). To do a one-time check of functions listed in (2), the following "static code analysis" has been done: 1) replace my_error(ER_xxx, arguments for substitution in format) with the equivalent my_printf_error(ER_xxx,ER(ER_xxx), arguments for substitution in format), so that we have ER(ER_xxx) and the arguments *in the same call site* 2) add ATTRIBUTE_FORMAT to push_warning_printf(), Table_check_intact::report_error(), general_log_print() 3) replace ER(xxx) with the hard-coded English text found in errmsg.txt (like: ER(ER_UNKNOWN_ERROR) is replaced with "Unknown error"), so that a call site has the format as string literal 4) this way, ATTRIBUTE_FORMAT can effectively do its job 5) compile, fix errors detected by ATTRIBUTE_FORMAT 6) revert steps 1-2-3. The present patch has no compiler error when submitted again to the static code analysis above. It cannot catch all problems though: see Field::set_warning(), in which a call to push_warning_printf() has a variable error (thus, not replacable by a string literal); I checked set_warning() calls by hand though. See also WL 5883 for one proposal to avoid such bugs from appearing again in the future. The issues fixed in the patch are: a) mismatch in types (like 'int' passed to '%ld') b) more arguments passed than specified in the format. This patch resolves mismatches by changing the type/number of arguments, not by changing error messages of sql/share/errmsg.txt. The latter would be wrong, per the following old rule: errmsg.txt must be as stable as possible; no insertions or deletions of messages, no changes of type or number of printf-like format specifiers, are allowed, as long as the change impacts a message already released in a GA version. If this rule is not followed: - Connectors, which use error message numbers, will be confused (by insertions/deletions of messages) - using errmsg.sys of MySQL 5.1.n with mysqld of MySQL 5.1.(n+1) could produce wrong messages or crash; such usage can easily happen if installing 5.1.(n+1) while /etc/my.cnf still has --language=/path/to/5.1.n/xxx; or if copying mysqld from 5.1.(n+1) into a 5.1.n installation. When fixing b), I have verified that the superfluous arguments were not used in the format in the first 5.1 GA (5.1.30 'bteam@astra04-20081114162938-z8mctjp6st27uobm'). Had they been used, then passing them today, even if the message doesn't use them anymore, would have been necessary, as explained above. include/my_getopt.h: this function pointer is used only with "string literal" formats, so we can add ATTRIBUTE_FORMAT. mysql-test/collections/default.experimental: test should pass now sql/derror.cc: by having a format as string literal, ATTRIBUTE_FORMAT check becomes effective. sql/events.cc: Change justified by the following excerpt from sql/share/errmsg.txt: ER_EVENT_SAME_NAME eng "Same old and new event name" ER_EVENT_SET_VAR_ERROR eng "Error during starting/stopping of the scheduler. Error code %u" sql/field.cc: ER_TOO_BIG_SCALE 42000 S1009 eng "Too big scale %d specified for column '%-.192s'. Maximum is %lu." ER_TOO_BIG_PRECISION 42000 S1009 eng "Too big precision %d specified for column '%-.192s'. Maximum is %lu." ER_TOO_BIG_DISPLAYWIDTH 42000 S1009 eng "Display width out of range for column '%-.192s' (max = %lu)" sql/ha_ndbcluster.cc: ER_OUTOFMEMORY HY001 S1001 eng "Out of memory; restart server and try again (needed %d bytes)" (sizeof() returns size_t) sql/ha_ndbcluster_binlog.cc: Too many arguments for: ER_GET_ERRMSG eng "Got error %d '%-.100s' from %s" Patch by Jonas Oreland. sql/ha_partition.cc: print_admin_msg() is used only with a literal as format, so ATTRIBUTE_FORMAT works. sql/handler.cc: ER_OUTOFMEMORY HY001 S1001 eng "Out of memory; restart server and try again (needed %d bytes)" (sizeof() returns size_t) sql/item_create.cc: ER_TOO_BIG_SCALE 42000 S1009 eng "Too big scale %d specified for column '%-.192s'. Maximum is %lu." ER_TOO_BIG_PRECISION 42000 S1009 eng "Too big precision %d specified for column '%-.192s'. Maximum is %lu." 'c_len' and 'c_dec' are char*, passed as %d !! We don't know their value (as strtoul() failed), but they are likely big, so we use INT_MAX. 'len' is ulong. sql/item_func.cc: ER_WARN_DATA_OUT_OF_RANGE 22003 eng "Out of range value for column '%s' at row %ld" ER_CANT_FIND_UDF eng "Can't load function '%-.192s'" sql/item_strfunc.cc: ER_TOO_BIG_FOR_UNCOMPRESS eng "Uncompressed data size too large; the maximum size is %d (probably, length of uncompressed data was corrupted)" max_allowed_packet is ulong. sql/mysql_priv.h: sql_print_message_func is a function _pointer_. sql/sp_head.cc: ER_SP_RECURSION_LIMIT eng "Recursive limit %d (as set by the max_sp_recursion_depth variable) was exceeded for routine %.192s" max_sp_recursion_depth is ulong sql/sql_acl.cc: ER_PASSWORD_NO_MATCH 42000 eng "Can't find any matching row in the user table" ER_CANT_CREATE_USER_WITH_GRANT 42000 eng "You are not allowed to create a user with GRANT" sql/sql_base.cc: ER_NOT_KEYFILE eng "Incorrect key file for table '%-.200s'; try to repair it" ER_TOO_MANY_TABLES eng "Too many tables; MySQL can only use %d tables in a join" MAX_TABLES is size_t. sql/sql_binlog.cc: ER_UNKNOWN_ERROR eng "Unknown error" sql/sql_class.cc: ER_TRUNCATED_WRONG_VALUE_FOR_FIELD eng "Incorrect %-.32s value: '%-.128s' for column '%.192s' at row %ld" WARN_DATA_TRUNCATED 01000 eng "Data truncated for column '%s' at row %ld" sql/sql_connect.cc: ER_HANDSHAKE_ERROR 08S01 eng "Bad handshake" ER_BAD_HOST_ERROR 08S01 eng "Can't get hostname for your address" sql/sql_insert.cc: ER_WRONG_VALUE_COUNT_ON_ROW 21S01 eng "Column count doesn't match value count at row %ld" sql/sql_parse.cc: ER_WARN_HOSTNAME_WONT_WORK eng "MySQL is started in --skip-name-resolve mode; you must restart it without this switch for this grant to work" ER_TOO_HIGH_LEVEL_OF_NESTING_FOR_SELECT eng "Too high level of nesting for select" ER_UNKNOWN_ERROR eng "Unknown error" sql/sql_partition.cc: ER_OUTOFMEMORY HY001 S1001 eng "Out of memory; restart server and try again (needed %d bytes)" sql/sql_plugin.cc: ER_OUTOFMEMORY HY001 S1001 eng "Out of memory; restart server and try again (needed %d bytes)" sql/sql_prepare.cc: ER_OUTOFMEMORY HY001 S1001 eng "Out of memory; restart server and try again (needed %d bytes)" ER_UNKNOWN_STMT_HANDLER eng "Unknown prepared statement handler (%.*s) given to %s" length value (for '%.*s') must be 'int', per the doc of printf() and the code of my_vsnprintf(). sql/sql_show.cc: ER_OUTOFMEMORY HY001 S1001 eng "Out of memory; restart server and try again (needed %d bytes)" sql/sql_table.cc: ER_TOO_BIG_FIELDLENGTH 42000 S1009 eng "Column length too big for column '%-.192s' (max = %lu); use BLOB or TEXT instead" sql/table.cc: ER_NOT_FORM_FILE eng "Incorrect information in file: '%-.200s'" ER_COL_COUNT_DOESNT_MATCH_PLEASE_UPDATE eng "Column count of mysql.%s is wrong. Expected %d, found %d. Created with MySQL %d, now running %d. Please use mysql_upgrade to fix this error." table->s->mysql_version is ulong. sql/unireg.cc: ER_TOO_LONG_TABLE_COMMENT eng "Comment for table '%-.64s' is too long (max = %lu)" ER_TOO_LONG_FIELD_COMMENT eng "Comment for field '%-.64s' is too long (max = %lu)" ER_TOO_BIG_ROWSIZE 42000 eng "Row size too large. The maximum row size for the used table type, not counting BLOBs, is %ld. You have to change some columns to TEXT or BLOBs" --- sql/handler.cc | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) (limited to 'sql/handler.cc') diff --git a/sql/handler.cc b/sql/handler.cc index 718529fa5fc..82f5f2ee841 100644 --- a/sql/handler.cc +++ b/sql/handler.cc @@ -275,7 +275,7 @@ handler *get_ha_partition(partition_info *part_info) } else { - my_error(ER_OUTOFMEMORY, MYF(0), sizeof(ha_partition)); + my_error(ER_OUTOFMEMORY, MYF(0), static_cast(sizeof(ha_partition))); } DBUG_RETURN(((handler*) partition)); } @@ -1604,7 +1604,8 @@ int ha_recover(HASH *commit_list) } if (!info.list) { - sql_print_error(ER(ER_OUTOFMEMORY), info.len*sizeof(XID)); + sql_print_error(ER(ER_OUTOFMEMORY), + static_cast(info.len*sizeof(XID))); DBUG_RETURN(1); } -- cgit v1.2.1 From 9e2b7fa7d5f0cbe4920be5567314b6de1af660a4 Mon Sep 17 00:00:00 2001 From: Jimmy Yang Date: Tue, 31 May 2011 02:12:32 -0700 Subject: Implement worklog #5743 InnoDB: Lift the limit of index key prefixes. With this change, the index prefix column length lifted from 767 bytes to 3072 bytes if "innodb_large_prefix" is set to "true". rb://603 approved by Marko --- sql/handler.cc | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'sql/handler.cc') diff --git a/sql/handler.cc b/sql/handler.cc index 7ee9467f4c1..580677242bd 100644 --- a/sql/handler.cc +++ b/sql/handler.cc @@ -357,6 +357,7 @@ int ha_init_errors(void) SETMSG(HA_ERR_AUTOINC_READ_FAILED, ER_DEFAULT(ER_AUTOINC_READ_FAILED)); SETMSG(HA_ERR_AUTOINC_ERANGE, ER_DEFAULT(ER_WARN_DATA_OUT_OF_RANGE)); SETMSG(HA_ERR_TOO_MANY_CONCURRENT_TRXS, ER_DEFAULT(ER_TOO_MANY_CONCURRENT_TRXS)); + SETMSG(HA_ERR_INDEX_COL_TOO_LONG, ER_DEFAULT(ER_INDEX_COLUMN_TOO_LONG)); /* Register the error messages for use with my_error(). */ return my_error_register(get_handler_errmsgs, HA_ERR_FIRST, HA_ERR_LAST); @@ -2861,6 +2862,9 @@ void handler::print_error(int error, myf errflag) case HA_ERR_TOO_MANY_CONCURRENT_TRXS: textno= ER_TOO_MANY_CONCURRENT_TRXS; break; + case HA_ERR_INDEX_COL_TOO_LONG: + textno= ER_INDEX_COLUMN_TOO_LONG; + break; default: { /* The error was "unknown" to this function. -- cgit v1.2.1