diff options
author | unknown <guilhem@gbichot4.local> | 2007-12-30 21:32:07 +0100 |
---|---|---|
committer | unknown <guilhem@gbichot4.local> | 2007-12-30 21:32:07 +0100 |
commit | 18bc7b695af8b553e6b8151ce7ce2e2d8755881d (patch) | |
tree | 8f4ad01ebf35455a4a4de26ab4acf29d15baf88e /mysql-test | |
parent | 630169c6d896bd33b65866d91f2bd8894b79bfa1 (diff) | |
download | mariadb-git-18bc7b695af8b553e6b8151ce7ce2e2d8755881d.tar.gz |
WL#3072 - Maria Recovery
* to honour WAL we now force the whole log when flushing a bitmap page.
* ability to intentionally crash in various places for recovery testing
* bugfix (dirty pages list found in checkpoint record was ignored)
* smaller checkpoint record
* misc small cleanups and comments
mysql-test/include/maria_empty_logs.inc:
maria-purge.test creates ~11 logs, remove them all
mysql-test/r/maria-recovery-bitmap.result:
result is good; without the _ma_bitmap_get_log_address() call,
we got
check error Bitmap at 0 has pages reserved outside of data file length
mysql-test/r/maria-recovery.result:
result update
mysql-test/t/maria-recovery-bitmap.test:
enable test of "bitmap-flush should flush whole log otherwise
corrupted data file (bitmap ahead of data pages)".
mysql-test/t/maria-recovery.test:
test of checkpoint
sql/sql_table.cc:
comment
storage/maria/ha_maria.cc:
_ma_reenable_logging_for_table() now includes file->trn=0.
At the end of repair() we don't need to re-enable logging, it is
done already by caller (like copy_data_between_tables()); it sounds
strange that this function could decide to re-enable, it should be
up to caller who knows what other operations it plans. Removing this
line led to assertion failure in maria_lock_database(F_UNLCK), fixed
by removing the assertion: maria_lock_database()
is here called in a context where F_UNLCK does not make the
table visible to others so assertion is excessive, and external_lock()
is already designed to honour the asserted condition.
Ability to crash at the end of bulk insert when indices
have been enabled.
storage/maria/ma_bitmap.c:
Better use pagecache_file_init() than set pagecache callbacks directly;
and a new function to set those callbacks for bitmap so that we can
reuse it.
_ma_bitmap_get_log_address() is a pagecache get_log_address callback
which causes the whole log to be flushed when a bitmap page
is flushed by the page cache. This was required by WAL.
storage/maria/ma_blockrec.c:
get_log_address pagecache callback for data (non bitmap) pages:
just reads the LSN from the page's content, like was hard-coded
before in ma_pagecache.c.
storage/maria/ma_blockrec.h:
functions which need to be exported
storage/maria/ma_check.c:
create_new_data_handle() can be static.
Ability to crash after rebuilding the index in OPTIMIZE,
in REPAIR. my_lock() implemented already.
storage/maria/ma_checkpoint.c:
As MARIA_SHARE* is now accessible to pagecache_collect_changed_blocks_LSN(),
we don't need to store kfile/dfile descriptors in checkpoint record,
2-byte-id of the table plus one byte to say if this is data or index
file is enough. So we go from 4+4 bytes per table down to 2+1.
storage/maria/ma_commit.c:
removing duplicate functions (see _ma_tmp_disable_logging_for_table())
storage/maria/ma_extra.c:
Monty fixed
storage/maria/ma_key_recover.c:
comment
storage/maria/ma_locking.c:
Sometimes other code does funny things with maria_lock_database(),
like ha_maria::repair() calling it at start and end without going
through ha_maria::external_lock(). So it happens that maria_lock_database()
is called with now_transactional!=born_transactional.
storage/maria/ma_loghandler.c:
update to new prototype
storage/maria/ma_open.c:
set_data|index_pagecache_callbacks() need to be exported as
they are now called when disabling/enabling transactionality.
storage/maria/ma_pagecache.c:
Removing PAGE_LSN_OFFSET, as much of the code relies on it being
0 anyway (let's not give impression we can just change this constant).
When flushing a page to disk, call the get_log_address callback to
know up to which LSN the log should be flushed.
As we now can access MARIA_SHARE* we can know share->id and store
it into the checkpoint record; we thus go from 4 bytes per dirty page
to 2+1.
storage/maria/ma_pagecache.h:
get_log_address callback
storage/maria/ma_panic.c:
No reason to reset pagecache callbacks in HA_PANIC_READ:
all we do is reopen files if they were closed; callbacks should
be in place already as 'info' exists; we just want to modify
the file descriptors, not the full PAGECACHE_FILE structure.
If we open data file and it was closed, share->bitmap.file needs
to be set.
Note that the modified code is disabled anyway.
storage/maria/ma_recovery.c:
Checkpoint record does not contain kfile/dfile descriptors anymore
so code can be simplified. Hash key in all_dirty_pages is
not made from file_descriptor & pageno anymore, but
index_or_data & table-short-id & pageno.
If a table's create_rename_lsn is higher than record's LSN,
we skip the table and don't fail if it's corrupted (because the LSNs
say that we don't have to look at this table).
If a table is skipped (for example due to create_rename_lsn),
its UNDOs still cause undo_lsn to advance; this is so that if later
we notice the transaction has to rollback we fail (as table should
not be skipped in this case).
Fixing a bug: the dirty_pages list was never used, because
the LSN below which it was used was the minimum rec_lsn of dirty pages!
It is now the min(checkpoint_start_log_horizon, min(trn's rec_lsn)).
When we disable/reenable transactionality, we modify pagecache
callbacks (needed for example for get_log_address: changing
share->page_type is not enough anymore).
storage/maria/ma_write.c:
'records' and 'checksum' are protected: they are updated under
log's mutex in write-hooks when UNDO is written.
storage/maria/maria_chk.c:
remove use of duplicate functions.
storage/maria/maria_def.h:
set_data|index_pagecache_callbacks() need to be exported;
_ma_reenable_logging_for_table() changes to a real function.
storage/maria/unittest/ma_pagecache_consist.c:
new prototype
storage/maria/unittest/ma_pagecache_single.c:
new prototype
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
new prototype
Diffstat (limited to 'mysql-test')
-rw-r--r-- | mysql-test/include/maria_empty_logs.inc | 43 | ||||
-rw-r--r-- | mysql-test/r/maria-recovery-bitmap.result | 18 | ||||
-rw-r--r-- | mysql-test/r/maria-recovery.result | 31 | ||||
-rw-r--r-- | mysql-test/t/maria-recovery-bitmap.test | 5 | ||||
-rw-r--r-- | mysql-test/t/maria-recovery.test | 29 |
5 files changed, 119 insertions, 7 deletions
diff --git a/mysql-test/include/maria_empty_logs.inc b/mysql-test/include/maria_empty_logs.inc index 7613cea2a52..a5ec31e335c 100644 --- a/mysql-test/include/maria_empty_logs.inc +++ b/mysql-test/include/maria_empty_logs.inc @@ -12,11 +12,50 @@ EOF --exec $MYSQLADMIN --no-defaults -S $MASTER_MYSOCK -P $MASTER_MYPORT -u root --password= shutdown 2>&1; +# Depending on what tests were run before, the number of logs +# may vary (maria-purge.test creates ~11 logs). remove_file $MYSQLTEST_VARDIR/master-data/maria_log_control; +-- error 0,1 remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000001; --- error 0,1 # maybe there is just one log +-- error 0,1 remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000002; -# Hope there were not more than these logs. +-- error 0,1 +remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000003; +-- error 0,1 +remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000004; +-- error 0,1 +remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000005; +-- error 0,1 +remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000006; +-- error 0,1 +remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000007; +-- error 0,1 +remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000008; +-- error 0,1 +remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000009; +-- error 0,1 +remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000010; +-- error 0,1 +remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000011; +-- error 0,1 +remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000012; +-- error 0,1 +remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000013; +-- error 0,1 +remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000014; +-- error 0,1 +remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000015; +-- error 0,1 +remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000016; +-- error 0,1 +remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000017; +-- error 0,1 +remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000018; +-- error 0,1 +remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000019; +-- error 0,1 +remove_file $MYSQLTEST_VARDIR/master-data/maria_log.00000020; +# hope there are not more than these logs... -- error 0,1 remove_file $MYSQLTEST_VARDIR/master-data/maria_recovery.trace; diff --git a/mysql-test/r/maria-recovery-bitmap.result b/mysql-test/r/maria-recovery-bitmap.result index 4eb1d2f491b..9c8ee173041 100644 --- a/mysql-test/r/maria-recovery-bitmap.result +++ b/mysql-test/r/maria-recovery-bitmap.result @@ -25,5 +25,23 @@ mysqltest.t1 check status OK Checksum-check ok use mysqltest; +* TEST of bitmap flushed without REDO-UNDO in the log (WAL violation) +flush table t1; +* copied t1 for comparison +lock tables t1 write; +insert into t1 values (REPEAT('a', 6000)); +SET SESSION debug="+d,maria_flush_bitmap,maria_crash"; +* crashing mysqld intentionally +set global maria_checkpoint_interval=1; +ERROR HY000: Lost connection to MySQL server during query +* recovery happens +check table t1 extended; +Table Op Msg_type Msg_text +mysqltest.t1 check status OK +* testing that checksum after recovery is as expected +Checksum-check +ok +use mysqltest; +drop table t1; drop database mysqltest_for_comparison; drop database mysqltest; diff --git a/mysql-test/r/maria-recovery.result b/mysql-test/r/maria-recovery.result index 2d4b91c890d..832d90b4104 100644 --- a/mysql-test/r/maria-recovery.result +++ b/mysql-test/r/maria-recovery.result @@ -214,6 +214,37 @@ t1 CREATE TABLE `t1` ( KEY `c` (`c`) ) ENGINE=MARIA AUTO_INCREMENT=16 DEFAULT CHARSET=latin1 drop table t1; +* TEST of checkpoint +set global debug="+d,info,query,enter,exit,loop,maria_checkpoint_indirect"; +set global maria_checkpoint_interval=10000; +create table t1(a int, b varchar(10), index(a,b)) engine=maria; +insert into t1 values(1,"a"),(2,"b"),(3,"c"); +delete from t1 where b="b"; +update t1 set b="d" where a=1; +flush table t1; +* copied t1 for comparison +lock tables t1 write; +insert into t1 values(4,"e"),(5,"f"),(6,"g"); +update t1 set b="h" where a=5; +delete from t1 where b="g"; +show status like "Maria_pagecache_blocks_not_flushed"; +Variable_name Value +Maria_pagecache_blocks_not_flushed 3 +set global maria_checkpoint_interval=10000; +update t1 set b="i" where a=5; +SET SESSION debug="+d,maria_crash"; +* crashing mysqld intentionally +set global maria_checkpoint_interval=1; +ERROR HY000: Lost connection to MySQL server during query +* recovery happens +check table t1 extended; +Table Op Msg_type Msg_text +mysqltest.t1 check status OK +* testing that checksum after recovery is as expected +Checksum-check +ok +use mysqltest; +drop table t1; drop database mysqltest_for_feeding_recovery; drop database mysqltest_for_comparison; drop database mysqltest; diff --git a/mysql-test/t/maria-recovery-bitmap.test b/mysql-test/t/maria-recovery-bitmap.test index 28d122ed6f7..ee5f6cbadd3 100644 --- a/mysql-test/t/maria-recovery-bitmap.test +++ b/mysql-test/t/maria-recovery-bitmap.test @@ -57,10 +57,6 @@ sleep 5; set global maria_checkpoint_interval=1; -- source include/maria_verify_recovery.inc -# disabled until pagecache callback framework is coded at which point -# we can add a get_lsn() callback for bitmaps, fixing the below bug. -if (0) -{ --echo * TEST of bitmap flushed without REDO-UNDO in the log (WAL violation) # before crashing we'll flush the bitmap page let $mvr_debug_option="+d,maria_flush_bitmap,maria_crash"; @@ -71,7 +67,6 @@ insert into t1 values (REPEAT('a', 6000)); # log is not flushed the bitmap is inconsistent with the data. -- source include/maria_verify_recovery.inc drop table t1; -} # clean up everything let $mms_purpose=comparison; diff --git a/mysql-test/t/maria-recovery.test b/mysql-test/t/maria-recovery.test index 0b70c8702d9..3e2a4f87377 100644 --- a/mysql-test/t/maria-recovery.test +++ b/mysql-test/t/maria-recovery.test @@ -179,6 +179,35 @@ let $mvr_crash_statement= set global maria_checkpoint_interval=1; show create table t1; drop table t1; +# A basic checkpoint test +--echo * TEST of checkpoint +# Don't take a full checkpoints, we want to test checkpoint vs dirty pages +set global debug="+d,info,query,enter,exit,loop,maria_checkpoint_indirect"; +# restart checkpoint thread for it to notice the above +set global maria_checkpoint_interval=10000; +create table t1(a int, b varchar(10), index(a,b)) engine=maria; +insert into t1 values(1,"a"),(2,"b"),(3,"c"); +delete from t1 where b="b"; +update t1 set b="d" where a=1; +-- source include/maria_make_snapshot_for_comparison.inc +lock tables t1 write; +insert into t1 values(4,"e"),(5,"f"),(6,"g"); +update t1 set b="h" where a=5; +delete from t1 where b="g"; +show status like "Maria_pagecache_blocks_not_flushed"; +# force a checkpoint; there should be dirty pages and an open transaction +set global maria_checkpoint_interval=10000; +# do some more work +update t1 set b="i" where a=5; +let $mvr_restore_old_snapshot=0; +let $mms_compare_physically=0; +let $mvr_debug_option="+d,maria_crash"; +let $mvr_crash_statement= set global maria_checkpoint_interval=1; +# Now we have a recovery, which should use the checkpoint record +# and its dirty pages list. +-- source include/maria_verify_recovery.inc +drop table t1; + # clean up everything let $mms_purpose=feeding_recovery; eval drop database mysqltest_for_$mms_purpose; |