summaryrefslogtreecommitdiff
path: root/mysys/my_fopen.c
Commit message (Collapse)AuthorAgeFilesLines
* MDEV-27142 - postfixVladislav Vaintroub2022-11-041-0/+2
| | | | | | | | Fix build failure in comp_err, if git is configured with default, platform-specific EOL. The error happens because comp_err is not prepared to handle extraneous CR characters from errmgs-utf8.txt. Use fopen in text mode to fix.
* Merge 10.4 into 10.5Marko Mäkelä2020-04-251-1/+3
|\ | | | | | | | | The functional changes of commit 5836191c8f0658d5d75484766fdcc3d838b0a5c1 (MDEV-21168) are omitted due to MDEV-742 having addressed the issue.
| * Shrink my_atomic.h and my_cpu.h scopeSergey Vojtovich2020-04-151-0/+1
| |
| * my_file_opened to my_atomicSergey Vojtovich2020-04-151-1/+2
| |
* | perfschema memory related instrumentation changesSergei Golubchik2020-03-101-3/+4
|/
* Merge branch '10.3' into 10.4Oleksandr Byelkin2019-05-191-2/+2
|\
| * Merge 10.2 into 10.3Marko Mäkelä2019-05-141-1/+1
| |\
| | * Merge 10.1 into 10.2Marko Mäkelä2019-05-131-1/+1
| | |\
| | | * Merge branch '5.5' into 10.1Vicențiu Ciorbaru2019-05-111-1/+1
| | | |\
| | | | * Update FSF AddressVicențiu Ciorbaru2019-05-111-1/+1
| | | | | | | | | | | | | | | | | | | | * Update wrong zip-code
| * | | | MDEV-17845 Extreme high open file limit usedSergei Golubchik2019-05-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | SHOW STATUS LIKE 'Open_files' was showing 18446744073709551615 my_file_opened used statistic_increment/statistic_decrement, so one-off errors were normal and expected. But they confused monitoring tools, so let's move my_file_opened to use atomics.
* | | | | Merge 10.3 into 10.4Marko Mäkelä2019-02-111-4/+1
|\ \ \ \ \ | |/ / / /
| * | | | Don't retry on closeDaniel Black2019-02-061-4/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | According to close(2) "Retrying the close() after a failure return is the wrong thing to do" Corrects 5c81cb880a054f34803e2821489533274ebf6c4e in MDEV-15635
* | | | | mysys: remove dead ME_xxx flagsSergei Golubchik2018-06-041-4/+3
|/ / / /
* | | | MDEV-15635 mysys: THR_LOCK_open reduce usageDaniel Black2018-05-211-17/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change the following to statistic counters: * my_file_opened * my_file_total_opened * my_stream_opened * my_tmp_file_created There is one non-statistics use of my_file_opened/my_stream_opened in my_end which prints a warning if we shutdown and its still open. It seems excessive to hold locks to prevent this warning. A file descriptor is already a unique element per process - in Windows, protection occurs at fd allocation using THR_LOCK_open in my_win_{,f}open and in other OSes, a unique fd to file map exists at the OS level. So accesses to my_file_info[fd] don't need to be protected by the THR_LOCK_open. my_close/my_fclose where restructured to clear out the my_file_info before the close/my_win_close/my_win_fclose. After these calls another thread could gain the same file descriptor. So for Windows this the file_info elements available to the my_win_{,f}_open are released during the invalidate_fd call within my_win_close. No locking is needed as the my_win_{,f}open is searching for a invalidate entry which is determined by a single value change. my_fclose also changed for non-Windows to retry closing if EINTR was returned, same as my_close. Closes #657
* | | | my_fdopen: list all args in commentDaniel Black2018-03-071-0/+2
| | | |
* | | | mysys: Remove freebsd freopen implementationDaniel Black2017-11-151-60/+0
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=79887 was fixed in 7.4 and 8.2. Both of these reached end of life in February 2011 https://www.freebsd.org/releases/ Signed-off-by: Daniel Black <daniel@linux.vnet.ibm.com>
* | | MDEV-13844 : Fix Windows warnings. Fix DBUG_PRINT.Vladislav Vaintroub2017-09-281-3/+3
|/ / | | | | | | | | | | | | | | | | | | | | - Fix win64 pointer truncation warnings (usually coming from misusing 0x%lx and long cast in DBUG) - Also fix printf-format warnings Make the above mentioned warnings fatal. - fix pthread_join on Windows to set return value.
* | Merge branch '10.0' into 10.1Sergei Golubchik2017-08-081-1/+1
|\ \
| * \ Merge remote-tracking branch 'origin/5.5' into 10.0Vicențiu Ciorbaru2017-07-251-1/+1
| |\ \ | | |/
| | * MDEV-12144 Signal 6 crash corrupts ibd filesSergei Golubchik2017-07-151-1/+1
| | | | | | | | | | | | | | | Avoid using STDERR_FILENO. The server uses freopen(stderr), so stderr can be on any file descriptor.
* | | Merge 10.0 into 10.1Marko Mäkelä2017-03-091-12/+6
|\ \ \ | |/ / | | | | | | | | | | | | | | | | | | | | | Also, implement MDEV-11027 a little differently from 5.5 and 10.0: recv_apply_hashed_log_recs(): Change the return type back to void (DB_SUCCESS was always returned). Report progress also via systemd using sd_notifyf().
| * | Merge branch '5.5' into 10.0Vicențiu Ciorbaru2017-03-031-12/+6
| |\ \ | | |/
| | * cleanup: my_register_filename()Sergei Golubchik2017-02-271-12/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | Don't let my_register_filename() fail because strdup() failed. Better to have NULL for a filename, then to fail the already successful open(). Filenames are only used for error reporting and there was already code to ignore OOMs (my_fdopen()) and to cope with missing filenames (my_filename()).
* | | Merge branch '10.0' into 10.1Sergei Golubchik2016-09-291-3/+3
|\ \ \ | |/ /
| * | Merge branch '5.5' into 10.0Sergei Golubchik2016-09-291-3/+3
| |\ \ | | |/
| | * MDEV-10907 MTR and server writes can interleave in the error logVladislav Vaintroub2016-09-271-3/+3
| | | | | | | | | | | | | | | Ensure atomic appends to the error log by using CreateFile with FILE_APPEND_DATA flag to open error log file (both MTR and server)
* | | Merge branch '10.0-galera' into 10.1Nirbhay Choubey2015-07-141-3/+5
| | |
* | | small cleanupSergei Golubchik2015-04-081-2/+1
|/ / | | | | | | | | | | | | | | | | | | | | * my_aes.h doesn't compile without my_global.h * typo in a comment * redundant condition * if encryption plugin fails, there's no encryption_key_manager at plugin deinit time * encryption plugin tests must run when plugin.so is present, not when a plugin is active (otherwise the test will be skipped when plugin fails to initialize).
* | 5.5 mergeSergei Golubchik2014-02-251-2/+2
|\ \ | |/
| * MySQL-5.5.36 mergeSergei Golubchik2014-02-171-2/+2
| |\ | | | | | | | | | (without few incorrect bugfixes and with 1250 files where only a copyright year was changed)
| | * Updated/added copyright headersMurthy Narkedimilli2014-01-061-1/+1
| | |
* | | 5.5 mergeSergei Golubchik2013-07-171-1/+1
|\ \ \ | |/ /
| * | mysql-5.5.32 mergeSergei Golubchik2013-07-161-1/+1
| |\ \ | | |/
| | * Fix for Bug 16395495 - OLD FSF ADDRESS IN GPL HEADERMurthy Narkedimilli2013-03-191-1/+1
| | |
* | | MDEV-4011 Added per thread memory counting and usageMichael Widenius2013-01-231-3/+3
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Base code and idea from a patch from by plinux at Taobao. The idea is that we mark all memory that are thread specific with MY_THREAD_SPECIFIC. Memory counting is done per thread in the my_malloc_size_cb_func callback function from my_malloc(). There are plenty of new asserts to ensure that for a debug server the counting is correct. Information_schema.processlist gets two new columns: MEMORY_USED and EXAMINED_ROWS. - The later is there mainly to show how query is progressing. The following changes in interfaces was needed to get this to work: - init_alloc_root() amd init_sql_alloc() has extra option so that one can mark memory with MY_THREAD_SPECIFIC - One now have to use alloc_root_set_min_malloc() to set min memory to be allocated by alloc_root() - my_init_dynamic_array() has extra option so that one can mark memory with MY_THREAD_SPECIFIC - my_net_init() has extra option so that one can mark memory with MY_THREAD_SPECIFIC - Added flag for hash_init() so that one can mark hash table to be thread specific. - Added flags to init_tree() so that one can mark tree to be thread specific. - Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG. - Added flag to Warning_info::Warning_info() if the structure should be fully initialized. - String elements can now be marked as thread specific. - Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC. - Changed type of myf from int to ulong, as this is always a set of bit flags. Other things: - Removed calls to net_end() and thd->cleanup() as these are now done in ~THD() - We now also show EXAMINED_ROWS in SHOW PROCESSLIST - Added new variable 'memory_used' - Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory. - Removed calls to the obsoleted function init_dynamic_array() - Use set_current_thd() instead of my_pthread_setspecific_ptr(THR_THD,...) client/completion_hash.cc: Updated call to init_alloc_root() client/mysql.cc: Updated call to init_alloc_root() client/mysqlbinlog.cc: init_dynamic_array() -> my_init_dynamic_array() Updated call to init_alloc_root() client/mysqlcheck.c: Updated call to my_init_dynamic_array() client/mysqldump.c: Updated call to init_alloc_root() client/mysqltest.cc: Updated call to init_alloc_root() Updated call to my_init_dynamic_array() Fixed compiler warnings extra/comp_err.c: Updated call to my_init_dynamic_array() extra/resolve_stack_dump.c: Updated call to my_init_dynamic_array() include/hash.h: Added HASH_THREAD_SPECIFIC include/heap.h: Added flag is internal temporary table. include/my_dir.h: Safety fix: Ensure that MY_DONT_SORT and MY_WANT_STAT don't interfer with other mysys flags include/my_global.h: Changed type of myf from int to ulong, as this is always a set of bit flags. include/my_sys.h: Added MY_THREAD_SPECIFIC and MY_THREAD_MOVE Added malloc_flags to DYNAMIC_ARRAY Added extra mysys flag argument to my_init_dynamic_array() Removed deprecated functions init_dynamic_array() and my_init_dynamic_array.._ci Updated paramaters for init_alloc_root() include/my_tree.h: Added my_flags to allow one to use MY_THREAD_SPECIFIC with hash tables. Removed with_delete. One should now instead use MY_TREE_WITH_DELETE_FLAG Updated parameters to init_tree() include/myisamchk.h: Added malloc_flags to allow one to use MY_THREAD_SPECIFIC for checks. include/mysql.h: Added MYSQL_THREAD_SPECIFIC_MALLOC Used 'unused1' to mark memory as thread specific. include/mysql.h.pp: Updated file include/mysql_com.h: Used 'unused1' to mark memory as thread specific. Updated parameters for my_net_init() libmysql/libmysql.c: Updated call to init_alloc_root() to mark memory thread specific. libmysqld/emb_qcache.cc: Updated call to init_alloc_root() libmysqld/lib_sql.cc: Updated call to init_alloc_root() mysql-test/r/create.result: Updated results mysql-test/r/user_var.result: Updated results mysql-test/suite/funcs_1/datadict/processlist_priv.inc: Update to handle new format of SHOW PROCESSLIST mysql-test/suite/funcs_1/datadict/processlist_val.inc: Update to handle new format of SHOW PROCESSLIST mysql-test/suite/funcs_1/r/is_columns_is.result: Update to handle new format of SHOW PROCESSLIST mysql-test/suite/funcs_1/r/processlist_priv_no_prot.result: Updated results mysql-test/suite/funcs_1/r/processlist_val_no_prot.result: Updated results mysql-test/t/show_explain.test: Fixed usage of debug variable so that one can run test with --debug mysql-test/t/user_var.test: Added test of memory_usage variable. mysys/array.c: Added extra my_flags option to init_dynamic_array() and init_dynamic_array2() so that one can mark memory with MY_THREAD_SPECIFIC All allocated memory is marked with the given my_flags. Removed obsolete function init_dynamic_array() mysys/default.c: Updated call to init_alloc_root() Updated call to my_init_dynamic_array() mysys/hash.c: Updated call to my_init_dynamic_array_ci(). Allocated memory is marked with MY_THREAD_SPECIFIC if HASH_THREAD_SPECIFIC is used. mysys/ma_dyncol.c: init_dynamic_array() -> my_init_dynamic_array() Added #if to get rid of compiler warnings mysys/mf_tempdir.c: Updated call to my_init_dynamic_array() mysys/my_alloc.c: Added extra parameter to init_alloc_root() so that one can mark memory with MY_THREAD_SPECIFIC Extend MEM_ROOT with a flag if memory is thread specific. This is stored in block_size, to keep the size of the MEM_ROOT object identical as before. Allocated memory is marked with MY_THREAD_SPECIFIC if used with init_alloc_root() mysys/my_chmod.c: Updated DBUG_PRINT because of change of myf type mysys/my_chsize.c: Updated DBUG_PRINT because of change of myf type mysys/my_copy.c: Updated DBUG_PRINT because of change of myf type mysys/my_create.c: Updated DBUG_PRINT because of change of myf type mysys/my_delete.c: Updated DBUG_PRINT because of change of myf type mysys/my_error.c: Updated DBUG_PRINT because of change of myf type mysys/my_fopen.c: Updated DBUG_PRINT because of change of myf type mysys/my_fstream.c: Updated DBUG_PRINT because of change of myf type mysys/my_getwd.c: Updated DBUG_PRINT because of change of myf type mysys/my_lib.c: Updated call to init_alloc_root() Updated call to my_init_dynamic_array() Updated DBUG_PRINT because of change of myf type mysys/my_lock.c: Updated DBUG_PRINT because of change of myf type mysys/my_malloc.c: Store at start of each allocated memory block the size of the block and if the block is thread specific. Call malloc_size_cb_func, if set, with the memory allocated/freed. Updated DBUG_PRINT because of change of myf type mysys/my_open.c: Updated DBUG_PRINT because of change of myf type mysys/my_pread.c: Updated DBUG_PRINT because of change of myf type mysys/my_read.c: Updated DBUG_PRINT because of change of myf type mysys/my_redel.c: Updated DBUG_PRINT because of change of myf type mysys/my_rename.c: Updated DBUG_PRINT because of change of myf type mysys/my_seek.c: Updated DBUG_PRINT because of change of myf type mysys/my_sync.c: Updated DBUG_PRINT because of change of myf type mysys/my_thr_init.c: Ensure that one can call my_thread_dbug_id() even if thread is not properly initialized. mysys/my_write.c: Updated DBUG_PRINT because of change of myf type mysys/mysys_priv.h: Updated parameters to sf_malloc and sf_realloc() mysys/safemalloc.c: Added checking that for memory marked with MY_THREAD_SPECIFIC that it's the same thread that is allocation and freeing the memory. Added sf_malloc_dbug_id() to allow MariaDB to specify which THD is handling the memory. Added my_flags arguments to sf_malloc() and sf_realloc() to be able to mark memory with MY_THREAD_SPECIFIC. Added sf_report_leaked_memory() to get list of memory not freed by a thread. mysys/tree.c: Added flags to init_tree() so that one can mark tree to be thread specific. Removed with_delete option to init_tree(). Now one should instead use MY_TREE_WITH_DELETE_FLAG. Updated call to init_alloc_root() All allocated memory is marked with the given malloc flags mysys/waiting_threads.c: Updated call to my_init_dynamic_array() sql-common/client.c: Updated call to init_alloc_root() and my_net_init() to mark memory thread specific. Updated call to my_init_dynamic_array(). Added MYSQL_THREAD_SPECIFIC_MALLOC so that client can mark memory as MY_THREAD_SPECIFIC. sql-common/client_plugin.c: Updated call to init_alloc_root() sql/debug_sync.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/event_scheduler.cc: Removed calls to net_end() as this is now done in ~THD() Call set_current_thd() to ensure that memory is assigned to right thread. sql/events.cc: my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/filesort.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/filesort_utils.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/ha_ndbcluster.cc: Updated call to init_alloc_root() Updated call to my_net_init() Removed calls to net_end() and thd->cleanup() as these are now done in ~THD() sql/ha_ndbcluster_binlog.cc: Updated call to my_net_init() Updated call to init_sql_alloc() Removed calls to net_end() and thd->cleanup() as these are now done in ~THD() sql/ha_partition.cc: Updated call to init_alloc_root() sql/handler.cc: Added MY_THREAD_SPECIFIC to allocated memory. Added missing call to my_dir_end() sql/item_func.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/item_subselect.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/item_sum.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/log.cc: More DBUG Updated call to init_alloc_root() sql/mdl.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/mysqld.cc: Added total_memory_used Updated call to init_alloc_root() Move mysql_cond_broadcast() before my_thread_end() Added mariadb_dbug_id() to count memory per THD instead of per thread. Added my_malloc_size_cb_func() callback function for my_malloc() to count memory. Move initialization of mysqld_server_started and mysqld_server_initialized earlier. Updated call to my_init_dynamic_array(). Updated call to my_net_init(). Call my_pthread_setspecific_ptr(THR_THD,...) to ensure that memory is assigned to right thread. Added status variable 'memory_used'. Updated call to init_alloc_root() my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/mysqld.h: Added set_current_thd() sql/net_serv.cc: Added new parameter to my_net_init() so that one can mark memory with MY_THREAD_SPECIFIC. Store in net->thread_specific_malloc if memory is thread specific. Mark memory to be thread specific if requested. sql/opt_range.cc: Updated call to my_init_dynamic_array() Updated call to init_sql_alloc() Added MY_THREAD_SPECIFIC to allocated memory. sql/opt_subselect.cc: Updated call to init_sql_alloc() to mark memory thread specific. sql/protocol.cc: Fixed compiler warning sql/records.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/rpl_filter.cc: Updated call to my_init_dynamic_array() sql/rpl_handler.cc: Updated call to my_init_dynamic_array2() sql/rpl_handler.h: Updated call to init_sql_alloc() sql/rpl_mi.cc: Updated call to my_init_dynamic_array() sql/rpl_tblmap.cc: Updated call to init_alloc_root() sql/rpl_utility.cc: Updated call to my_init_dynamic_array() sql/slave.cc: Initialize things properly before calling functions that allocate memory. Removed calls to net_end() as this is now done in ~THD() sql/sp_head.cc: Updated call to init_sql_alloc() Updated call to my_init_dynamic_array() Added parameter to warning_info() that it should be fully initialized. sql/sp_pcontext.cc: Updated call to my_init_dynamic_array() sql/sql_acl.cc: Updated call to init_sql_alloc() Updated call to my_init_dynamic_array() my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/sql_admin.cc: Added parameter to warning_info() that it should be fully initialized. sql/sql_analyse.h: Updated call to init_tree() to mark memory thread specific. sql/sql_array.h: Updated call to my_init_dynamic_array() to mark memory thread specific. sql/sql_audit.cc: Updated call to my_init_dynamic_array() sql/sql_base.cc: Updated call to init_sql_alloc() my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/sql_cache.cc: Updated comment sql/sql_class.cc: Added parameter to warning_info() that not initialize it until THD is fully created. Updated call to init_sql_alloc() Mark THD::user_vars has to be thread specific. Updated call to my_init_dynamic_array() Ensure that memory allocated by THD is assigned to the THD. More DBUG Always acll net_end() in ~THD() Assert that all memory signed to this THD is really deleted at ~THD. Fixed set_status_var_init() to not reset memory_used. my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/sql_class.h: Added MY_THREAD_SPECIFIC to allocated memory. Added malloc_size to THD to record allocated memory per THD. sql/sql_delete.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/sql_error.cc: Added 'initialize' parameter to Warning_info() to say if should allocate memory for it's structures. This is used by THD::THD() to not allocate memory until THD is ready. Added Warning_info::free_memory() sql/sql_error.h: Updated Warning_info() class. sql/sql_handler.cc: Updated call to init_alloc_root() to mark memory thread specific. sql/sql_insert.cc: More DBUG sql/sql_join_cache.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/sql_lex.cc: Updated call to my_init_dynamic_array() sql/sql_lex.h: Updated call to my_init_dynamic_array() sql/sql_load.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/sql_parse.cc: Removed calls to net_end() and thd->cleanup() as these are now done in ~THD() Ensure that examined_row_count() is reset before query. Fixed bug where kill_threads_for_user() was using the wrong mem_root to allocate memory. my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() Don't restore thd->status_var.memory_used when restoring thd->status_var sql/sql_plugin.cc: Updated call to init_alloc_root() Updated call to my_init_dynamic_array() Don't allocate THD on the stack, as this causes problems with valgrind when doing thd memory counting. my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/sql_prepare.cc: Added parameter to warning_info() that it should be fully initialized. Updated call to init_sql_alloc() to mark memory thread specific. sql/sql_reload.cc: my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/sql_select.cc: Updated call to my_init_dynamic_array() and init_sql_alloc() to mark memory thread specific. Added MY_THREAD_SPECIFIC to allocated memory. More DBUG sql/sql_servers.cc: Updated call to init_sql_alloc() to mark memory some memory thread specific. my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/sql_show.cc: Updated call to my_init_dynamic_array() Mark my_dir() memory thread specific. Use my_pthread_setspecific_ptr(THR_THD,...) to mark that allocated memory should be allocated to calling thread. More DBUG. Added malloc_size and examined_row_count to SHOW PROCESSLIST. Added MY_THREAD_SPECIFIC to allocated memory. Updated call to init_sql_alloc() Added parameter to warning_info() that it should be fully initialized. sql/sql_statistics.cc: Fixed compiler warning sql/sql_string.cc: String elements can now be marked as thread specific. sql/sql_string.h: String elements can now be marked as thread specific. sql/sql_table.cc: Updated call to init_sql_alloc() and my_malloc() to mark memory thread specific my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() Fixed compiler warning sql/sql_test.cc: Updated call to my_init_dynamic_array() to mark memory thread specific. sql/sql_trigger.cc: Updated call to init_sql_alloc() sql/sql_udf.cc: Updated call to init_sql_alloc() my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/sql_update.cc: Added MY_THREAD_SPECIFIC to allocated memory. sql/table.cc: Updated call to init_sql_alloc(). Mark memory used by temporary tables, that are not for slave threads, as MY_THREAD_SPECIFIC Updated call to init_sql_alloc() sql/thr_malloc.cc: Added my_flags argument to init_sql_alloc() to be able to mark memory as MY_THREAD_SPECIFIC. sql/thr_malloc.h: Updated prototype for init_sql_alloc() sql/tztime.cc: Updated call to init_sql_alloc() Updated call to init_alloc_root() to mark memory thread specific. my_pthread_setspecific_ptr(THR_THD,...) -> set_current_thd() sql/uniques.cc: Updated calls to init_tree(), my_init_dynamic_array() and my_malloc() to mark memory thread specific. sql/unireg.cc: Added MY_THREAD_SPECIFIC to allocated memory. storage/csv/ha_tina.cc: Updated call to init_alloc_root() storage/federated/ha_federated.cc: Updated call to init_alloc_root() Updated call to my_init_dynamic_array() Ensure that memory allocated by fedarated is registered for the system, not for the thread. storage/federatedx/federatedx_io_mysql.cc: Updated call to my_init_dynamic_array() storage/federatedx/ha_federatedx.cc: Updated call to init_alloc_root() Updated call to my_init_dynamic_array() storage/heap/ha_heap.cc: Added MY_THREAD_SPECIFIC to allocated memory. storage/heap/heapdef.h: Added parameter to hp_get_new_block() to be able to do thread specific memory tagging. storage/heap/hp_block.c: Added parameter to hp_get_new_block() to be able to do thread specific memory tagging. storage/heap/hp_create.c: - Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC. - Use MY_TREE_WITH_DELETE instead of removed option 'with_delete'. storage/heap/hp_open.c: Internal HEAP tables are now marking it's memory as MY_THREAD_SPECIFIC. storage/heap/hp_write.c: Added new parameter to hp_get_new_block() storage/maria/ma_bitmap.c: Updated call to my_init_dynamic_array() storage/maria/ma_blockrec.c: Updated call to my_init_dynamic_array() storage/maria/ma_check.c: Updated call to init_alloc_root() storage/maria/ma_ft_boolean_search.c: Updated calls to init_tree() and init_alloc_root() storage/maria/ma_ft_nlq_search.c: Updated call to init_tree() storage/maria/ma_ft_parser.c: Updated call to init_tree() Updated call to init_alloc_root() storage/maria/ma_loghandler.c: Updated call to my_init_dynamic_array() storage/maria/ma_open.c: Updated call to my_init_dynamic_array() storage/maria/ma_sort.c: Updated call to my_init_dynamic_array() storage/maria/ma_write.c: Updated calls to my_init_dynamic_array() and init_tree() storage/maria/maria_pack.c: Updated call to init_tree() storage/maria/unittest/sequence_storage.c: Updated call to my_init_dynamic_array() storage/myisam/ft_boolean_search.c: Updated call to init_tree() Updated call to init_alloc_root() storage/myisam/ft_nlq_search.c: Updated call to init_tree() storage/myisam/ft_parser.c: Updated call to init_tree() Updated call to init_alloc_root() storage/myisam/ft_stopwords.c: Updated call to init_tree() storage/myisam/mi_check.c: Updated call to init_alloc_root() storage/myisam/mi_write.c: Updated call to my_init_dynamic_array() Updated call to init_tree() storage/myisam/myisamlog.c: Updated call to init_tree() storage/myisam/myisampack.c: Updated call to init_tree() storage/myisam/sort.c: Updated call to my_init_dynamic_array() storage/myisammrg/ha_myisammrg.cc: Updated call to init_sql_alloc() storage/perfschema/pfs_check.cc: Rest current_thd storage/perfschema/pfs_instr.cc: Removed DBUG_ENTER/DBUG_VOID_RETURN as at this point my_thread_var is not allocated anymore, which can cause problems. support-files/compiler_warnings.supp: Disable compiler warning from offsetof macro.
* | merge with 5.3Sergei Golubchik2011-10-191-1/+2
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | sql/sql_insert.cc: CREATE ... IF NOT EXISTS may do nothing, but it is still not a failure. don't forget to my_ok it. ****** CREATE ... IF NOT EXISTS may do nothing, but it is still not a failure. don't forget to my_ok it. sql/sql_table.cc: small cleanup ****** small cleanup
| * | Backport Fix for Bug#24509 - 2048 file descriptor limit on windows needs ↵Vladislav Vaintroub2011-06-121-23/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | increasing. The patch replaces the use of the POSIX I/O interfaces in mysys on Windows with the Win32 API calls (CreateFile, WriteFile, etc). The Windows HANDLE for the open file is stored in the my_file_info struct, along with a flag for append mode (because the Windows API does not support opening files in append mode in all cases) The default max open files has been increased to 16384 and can be increased further by setting --max-open-files=<value> during the server start. Noteworthy benefit of this patch is that it removes limits from the table_cache size - allowing for more simultaneus users
| * | Merge with MySQL 5.1.55Michael Widenius2011-02-201-2/+136
| |\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Fixed some issues with partitions and connection_string, which also fixed lp:716890 "Pre- and post-recovery crash in Aria" - Fixed wrong assert in Aria Now need to merge with latest xtradb before pushing sql/ha_partition.cc: Ensure that m_ordered_rec_buffer is not freed before close. sql/mysqld.cc: Changed to use opt_stack_trace instead of opt_pstack. Removed references to pstack sql/partition_element.h: Ensure that connect_string is initialized storage/maria/ma_key_recover.c: Fixed wrong assert
* | \ \ 5.5-mergeSergei Golubchik2011-07-021-2/+134
|\ \ \ \ | | |_|/ | |/| |
| * | | weave merge from mysql-5.1 to mysql-5.5Georgi Kodinov2011-02-081-1/+1
| |\ \ \ | | | |/ | | |/| | | | | | | | | Resolved an innodb conflict thanks to vasil.
| | * | Updating header copyright/README in source for 2011Karen Langford2011-01-251-1/+1
| | | |
| * | | Merge of mysql-5.1 into mysql-5.5.Davi Arnaut2011-01-071-1/+133
| |\ \ \ | | |/ /
| | * | Bug#51023: Mysql server crashes on SIGHUP and destroys InnoDB filesDavi Arnaut2011-01-071-3/+3
| | | | | | | | | | | | | | | | | | | | WIN32 compilation fixes: define ETIMEDOUT only if not available and fix typos and add a missing parameter.
| | * | Bug#51023: Mysql server crashes on SIGHUP and destroys InnoDB filesDavi Arnaut2011-01-071-1/+134
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | From a user perspective, the problem is that a FLUSH LOGS or SIGHUP signal could end up associating the stdout and stderr to random files. In the case of this bug report, the streams would end up associated to InnoDB ibd files. The freopen(3) function is not thread-safe on FreeBSD. What this means is that if another thread calls open(2) during freopen() is executing that another thread's fd returned by open(2) may get re-associated with the file being passed to freopen(3). See FreeBSD PR number 79887 for reference: http://www.freebsd.org/cgi/query-pr.cgi?pr=79887 This problem is worked around by substituting a internal hook within the FILE structure. This avoids the loss of atomicity by not having the original fd closed before its duplicated. Patch based on the original work by Vasil Dimov. include/my_sys.h: Export my_freopen. mysys/my_fopen.c: Add a my_freopen abstraction to workaround bugs in specific OSes. Add a prototype for getosreldate() as older FreeBSD versions did not define one. sql/log.cc: Move freopen abstraction code over to mysys. The streams are now only reopened for writing.
* | | | merge.Sergei Golubchik2010-11-251-1/+1
|\ \ \ \ | |/ / / |/| | / | | |/ | |/| | | | checkpoint. does not compile.
| * | Fixed several bugs in page CRC handlingunknown2007-12-181-1/+1
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Ignore CRC errors in REDO for potential new pages - Ignore CRC errors when repairing tables - Don't do readcheck callback on read error - Set my_errno to HA_ERR_WRONG_CRC if we find page with wrong CRC - Check index page for length before calculating CRC to catch bad pages Fixed bugs where we used wrong file descriptor to read/write bitmaps Fixed wrong hash key in 'files_in_flush' Fixed wrong lock method when writing bitmap Fixed some wrong printf statements in check/repair that caused core dumps Fixed argument to translog_page_validator that cause reading of log files to fail Store number of bytes used for delete-linked key pages to be able to use standard index CRC for deleted key pages. Use fast 'dummy' pagecheck callbacks for temporary tables Don't die silently if flush finds pinned pages Give error (for now) if one tries to create a transactional table with fulltext or spatial keys Removed some not needed calls to pagecache_file_init() Added checking of pagecache checksums to ma_test1 and ma_test2 More DBUG Fixed some DBUG_PRINT to be in line with rest of the code include/my_base.h: Added HA_ERR_INTERNAL_ERROR (used for flush with pinned pages) and HA_ERR_WRONG_CRC mysql-test/r/binlog_unsafe.result: Added missing DROP VIEW statement mysql-test/r/maria.result: Added TRANSACTIONAL=0 when testing with fulltext keys Added test that verifies we can't yet create transactional test with fulltext or spatial keys mysql-test/r/ps_maria.result: Added TRANSACTIONAL=0 when testing with fulltext keys mysql-test/t/binlog_unsafe.test: Added missing DROP VIEW statement mysql-test/t/maria.test: Added TRANSACTIONAL=0 when testing with fulltext keys Added test that verifies we can't yet create transactional test with fulltext or spatial keys mysql-test/t/ps_maria.test: Added TRANSACTIONAL=0 when testing with fulltext keys mysys/my_fopen.c: Fd: -> fd: mysys/my_handler.c: Added new error messages mysys/my_lock.c: Fd: -> fd: mysys/my_pread.c: Fd: -> fd: mysys/my_read.c: Fd: -> fd: mysys/my_seek.c: Fd: -> fd: mysys/my_sync.c: Fd: -> fd: mysys/my_write.c: Fd: -> fd: sql/mysqld.cc: Fixed wrong argument to my_uuid_init() sql/sql_plugin.cc: Unified DBUG_PRINT (for convert-dbug-for-diff) storage/maria/ma_bitmap.c: Fixed wrong lock method when writing bitmap Fixed valgrind error Use fast 'dummy' pagecheck callbacks for temporary tables Faster bitmap handling for non transational tables storage/maria/ma_blockrec.c: Fixed that bitmap reading is done with the correct filehandle Handle reading of pages with wrong CRC when page contect doesn't matter Use the page buffer also when we get WRONG CRC or FILE_TOO_SHORT. (Faster and fixed a couple of bugs) storage/maria/ma_check.c: Split long strings for readablity Fixed some wrong printf statements that caused core dumps Use bitmap.file for bitmaps Ignore pages with wrong CRC storage/maria/ma_close.c: More DBUG_PRINT storage/maria/ma_create.c: Give error (for now) if one tries to create a crash safe table with fulltext or spatial keys storage/maria/ma_key_recover.c: Ignore HA_ERR_WRONG_CRC for new pages info->s -> share Store number of bytes used for delete-linked key pages to be able to use standard index CRC for deleted key pages. storage/maria/ma_loghandler.c: Fixed argument to translog_page_validator() storage/maria/ma_open.c: Removed old VMS specific code Added function to setup pagecache callbacks Moved code around to set 'share->temporary' early Removed some not needed calls to pagecache_file_init() storage/maria/ma_page.c: Store number of bytes used for delete-linked key pages to be able to use standard index CRC for deleted key pages. storage/maria/ma_pagecache.c: Don't do readcheck callback on read error Reset PCBLOCK_ERROR in pagecache_unlock_by_link() if we write page Set my_errno to HA_ER_INTERNAL_ERROR if flush() finds pinned pages Don't die silently if flush finds pinned pages. Use correct file descriptor when flushing pages Fixed wrong hash key in 'files_in_flush'; This must be the file descriptor, not the PAGECACHE_FILE as there may be several PAGECACHE_FILE for same file descriptor More DBUG_PRINT storage/maria/ma_pagecrc.c: Removed inline from not tiny static function Set my_errno to HA_ERR_WRONG_CRC if we find page with wrong CRC (Otherwise my_errno may be 0, and a lot of other code will be confused) CRCerror -> error (to keep code uniform) Print crc with %lu, as in my_checksum() uchar* -> uchar * Check index page for length before calculating CRC to catch bad pages Added 'dummy' crc_check and filler functions that are used for temporary tables storage/maria/ma_recovery.c: More DBUG More message to users to give information what phase failed Better error message if recovery failed storage/maria/ma_test1.c: Added checking of page checksums (combined with 'c' to not have to add more test runs) storage/maria/ma_test2.c: Added checking of page checksums (combined with 'c' to not have to add more test runs) storage/maria/maria_chk.c: Fixed wrong argument to _ma_check_print_error() storage/maria/maria_def.h: Added format information to _ma_check_print_xxxx functions uchar* -> uchar *
* | Bug#34043: Server loops excessively in _checkchunk() when safemalloc is enabledDavi Arnaut2010-07-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Essentially, the problem is that safemalloc is excruciatingly slow as it checks all allocated blocks for overrun at each memory management primitive, yielding a almost exponential slowdown for the memory management functions (malloc, realloc, free). The overrun check basically consists of verifying some bytes of a block for certain magic keys, which catches some simple forms of overrun. Another minor problem is violation of aliasing rules and that its own internal list of blocks is prone to corruption. Another issue with safemalloc is rather the maintenance cost as the tool has a significant impact on the server code. Given the magnitude of memory debuggers available nowadays, especially those that are provided with the platform malloc implementation, maintenance of a in-house and largely obsolete memory debugger becomes a burden that is not worth the effort due to its slowness and lack of support for detecting more common forms of heap corruption. Since there are third-party tools that can provide the same functionality at a lower or comparable performance cost, the solution is to simply remove safemalloc. Third-party tools can provide the same functionality at a lower or comparable performance cost. The removal of safemalloc also allows a simplification of the malloc wrappers, removing quite a bit of kludge: redefinition of my_malloc, my_free and the removal of the unused second argument of my_free. Since free() always check whether the supplied pointer is null, redudant checks are also removed. Also, this patch adds unit testing for my_malloc and moves my_realloc implementation into the same file as the other memory allocation primitives. client/mysqldump.c: Pass my_free directly as its signature is compatible with the callback type -- which wasn't the case for free_table_ent.
* | WL#2360 Performance schemaMarc Alff2009-12-091-8/+8
| | | | | | | | | | Part III: mysys instrumentation
* | This is the downport ofVladislav Vaintroub2009-09-111-23/+26
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bug#24509 - 2048 file descriptor limit on windows needs increasing, also WL#3049 - improved Windows I/O The patch replaces the use of the POSIX I/O interfaces in mysys on Windows with the Win32 API calls (CreateFile, WriteFile, etc). The Windows HANDLE for the open file is stored in the my_file_info struct, along with a flag for append mode because the Windows API does not support opening files in append mode in all cases) The default max open files has been increased to 16384 and can be increased further by setting --max-open-files=<value> during the server start. Another major change in this patch that almost all Windows specific file IO code has been moved to a new file my_winfile.c, greatly reducing the amount of code in #ifdef blocks within mysys, thus improving readability. Minor enhancements: - my_(f)stat() is changed to use __stati64 structure with 64 file size and timestamps. It will return correct file size now (C runtime implementation used to report outdated information) - my_lock on Windows is prepared to handle additional timeout parameter - after review : changed __WIN__ to _WIN32 in the new and changed code. client/mysqlbinlog.cc: fileno -> my_fileno client/readline.cc: fileno -> my_fileno include/config-win.h: Increase OS_FILE_LIMIT for Windows. Remove O_SHARE - Windows does not support it. Its definition conflicts with O_SHORT_LIVED, that has different semantics. include/my_dir.h: Use stat64 for stat() family of functions on Windows, because of 64 bit file size. include/my_global.h: Increased default value for open file limit to 16K include/my_sys.h: - my_file_info got new structure members - file handle and open flags - 2 new Windows-only mysys functions : my_get_osfhandle and my_osmaperr, modelled after Windows C runtime functions _get_osfhandle and _dosmaperr libmysql/CMakeLists.txt: new files my_winfile.c and my_winerr.c mysql-test/suite/large_tests/r/lock_tables_big.result: test for more then 2048 open file descriptors on Windows mysql-test/suite/large_tests/t/lock_tables_big.test: test for more then 2048 open file descriptors on Windows mysys/CMakeLists.txt: new files my_winfile.c and my_winerr.c mysys/Makefile.am: new files my_winfile.c and my_winerr.c mysys/default_modify.c: fileno -> my_fileno mysys/my_chsize.c: implementation of chsize on Windows now moved to my_winfile.c mysys/my_create.c: - my_sopen->my_win_open - close open file before removing (won't generally work on Windows otherwise) mysys/my_file.c: On Windows, files returned by my_open will not start with 0, but 2048 (making it simple to detect incompatible mix of CRT and mysys io functions) mysys/my_fopen.c: fileno->my_win_fileno , fclose->my_win_fclose, fdopen->my_win_fdopen Check for legal filename is done by my_win_[f]open functions mysys/my_fstream.c: fileno->my_fileno mysys/my_lib.c: Windows stat() functions are moved to my_winfile.c mysys/my_lock.c: Move Windows code under #ifdef to a separate function win_lock(). Add a parameter for lock wait timeout mysys/my_mmap.c: _get_osfhandle->my_get_osfhandle mysys/my_open.c: my_sopen->my_win_open (simpler interface) mysys/my_pread.c: moved most windows specific code to my_win_file.c Use my_win_pread mysys/my_quick.c: Use my_win_read/my_win_write mysys/my_read.c: Moved most of windows specific code to my_win_file.c Use my_win_read() mysys/my_seek.c: On Windows, use my_win_lseek() in my_seek()/my_tell() Removed dead code (synchronization of lseeks) Improved DBUG tracing (file position is ulonglong, not ulong) mysys/my_static.c: Removed array initialization. my_file_info_default is global variable thus it is initialized with all zeros anyway mysys/my_sync.c: _commit->my_win_fsync mysys/my_winerr.c: New file my_winerr.c Exports my_osmaperr modelled after undocumented C runtime function _dosmaperr(). The problem with _dosmaperr() used previously is that 1) it is nowhere documented and thus code relying on it is not guaranteed to work in subsequent releases on the C runtime 2) it is present only in static C runtime (mysqld does not link if compiled with /MD) mysys/my_winfile.c: New file my_winfile.c Implements ANSI/Posix file IO routines, when possible using native Windows IO, without C runtime (C runtime dropped because of the 2048 file descriptor limit). mysys/my_write.c: write->my_win_write mysys/mysys_priv.h: Declaration of Windows Posix functions (private to mysys, shall not be visible outside) storage/innobase/handler/ha_innodb.cc: mysys native Windows IO : correct innodb tmp file handling mysql_tmpfile does not return valid CRT file descriptor, thus it is not possible to dup() it. Instead, the native file handle has to be duplicated and then converted to CRT descriptor. storage/myisam/mi_locking.c: _commit->my_sync
View Lorry Controller Status