WL#3071 Maria checkpoint

Ability for flush_pagecache_blocks() to flush only certain pages of
a file, as instructed by an option "filter" pointer-to-function argument;
Checkpoint and background dirty page flushing use that to flush only
pages which have been dirty for long enough and bitmap pages.
Fix for a bug in flush_cached_blocks() (no idea if it could produce
a bug in real life, but theoretically it is).
Testing checkpoint in ma_test_recovery via ma_test1 and ma_test2.
Background checkpoint & dirty pages flush thread is still disabled
by default in ha_maria.


mysql-test/r/maria.result:
  result update
storage/maria/ha_maria.cc:
  blank after function comment
storage/maria/ma_checkpoint.c:
  Using an enum instead of 0/1/2 (applying Sanja's review comments).
  The comment about "this is an horizon" can be removed as Sanja
  created translog_next_LSN() which parse_checkpoint_record() uses.
  Variables in ma_checkpoint_background() cannot be declared in the
  for() as their value must not be reset at each iteration!
storage/maria/ma_pagecache.c:
  adding to flush_pagecache_blocks() optional arguments 'filter'
  (pointer to function) and 'filter_arg'; if filter!=NULL this function
  will be called for each block of the file and will reply if this
  block and following ones should be flushed or not (3 possible
  replies).
  Fixing a bug when flush_cached_blocks() skips a pinned page: it has
  to unset PCBLOCK_IN_FLUSH set by flush_pagecache_blocks_int().
storage/maria/ma_pagecache.h:
  flush_pagecache_blocks() is changed to take "filter" and "filter_arg"
  arguments. "filter", if it is not NULL, may return one value
  among enum pagecache_flush_filter_result.
storage/maria/ma_recovery.c:
  open_count=0 when closing tables at the end of recovery.
storage/maria/ma_test1.c:
  Optional checkpoints (-H#) at various stages (stages similar
  to --testflag), for testing of checkpoints.
storage/maria/ma_test2.c:
  Optional checkpoints (-H#) at various stages (stages similar
  to -t), for testing of checkpoints.
storage/maria/ma_test_recovery.expected:
  Result update: the results of the additional test run with -H#
  (checkpoints) are added here. They are exactly identical to without
  checkpoints except that the index's Root (printed by maria_chk)
  is more correct when using checkpoints. This is because checkpoint
  flushed the state, so it happens to be correct, while no-checkpoint
  does not flush the state, and recovery does not recover indexes
  so Root is never fixed. When we recover indices, this will go away.
storage/maria/ma_test_recovery:
  We duplicate the loop of tests to add an additional run with
  checkpoints at various stages, to see if maria_read_log
  uses them fine.

1 files changed, 16 insertions, 3 deletions

diff --git a/storage/maria/ma_test2.c b/storage/maria/ma_test2.c
index d9bd9bca9e0..b2b2c2fc33a 100644
--- a/storage/maria/ma_test2.c
+++ b/storage/maria/ma_test2.c
@@ -28,7 +28,7 @@
 #include "trnman.h"
 #include <m_ctype.h>
 #include <my_bit.h>
-
+#include "ma_checkpoint.h"
 
 #define STANDARD_LENGTH 37
 #define MARIA_KEYS 6
@@ -51,7 +51,7 @@ static	int verbose=0,testflag=0,
             opt_quick_mode=0, transactional= 0, skip_update= 0,
             die_in_middle_of_transaction= 0;
 static int pack_seg=HA_SPACE_PACK,pack_type=HA_PACK_KEY,remove_count=-1;
-static int create_flag= 0, srand_arg= 0;
+static int create_flag= 0, srand_arg= 0, checkpoint= 0;
 static ulong pagecache_size=IO_SIZE*16;
 static enum data_file_type record_type= DYNAMIC_RECORD;
 
@@ -98,7 +98,7 @@ int main(int argc, char *argv[])
       translog_init(maria_data_root, TRANSLOG_FILE_SIZE,
 		    0, 0, maria_log_pagecache,
 		    TRANSLOG_DEFAULT_FLAGS) ||
-      (transactional && trnman_init(0)))
+      (transactional && (trnman_init(0) || ma_checkpoint_init(FALSE))))
   {
     fprintf(stderr, "Error in initialization");
     exit(1);
@@ -240,6 +240,8 @@ int main(int argc, char *argv[])
   maria_begin(file);
   if (testflag == 1)
     goto end;
+  if (checkpoint == 1 && ma_checkpoint_execute(CHECKPOINT_MEDIUM, FALSE))
+    goto err;
   if (!silent)
     printf("- Writing key:s\n");
   if (locking)
@@ -302,6 +304,8 @@ int main(int argc, char *argv[])
   }
   if (testflag == 2)
     goto end;
+  if (checkpoint == 2 && ma_checkpoint_execute(CHECKPOINT_MEDIUM, FALSE))
+    goto err;
 
   if (write_cacheing)
   {
@@ -353,6 +357,8 @@ int main(int argc, char *argv[])
   }
   if (testflag == 3)
     goto end;
+  if (checkpoint == 3 && ma_checkpoint_execute(CHECKPOINT_MEDIUM, FALSE))
+    goto err;
 
   if (!silent)
     printf("- Update\n");
@@ -414,6 +420,8 @@ int main(int argc, char *argv[])
   }
   if (testflag == 4)
     goto end;
+  if (checkpoint == 4 && ma_checkpoint_execute(CHECKPOINT_MEDIUM, FALSE))
+    goto err;
 
   for (i=999, dupp_keys=j=0 ; i>0 ; i--)
   {
@@ -824,6 +832,8 @@ int main(int argc, char *argv[])
 
   if (testflag == 5)
     goto end;
+  if (checkpoint == 5 && ma_checkpoint_execute(CHECKPOINT_MEDIUM, FALSE))
+    goto err;
 
   if (!silent)
     printf("- Removing keys\n");
@@ -1057,6 +1067,9 @@ static void get_options(int argc, char **argv)
       if ((first_key=atoi(++pos)) < 0 || first_key >= MARIA_KEYS)
 	first_key=0;
       break;
+    case 'H':
+      checkpoint= atoi(++pos);
+      break;
     case 'k':
       if ((keys=(uint) atoi(++pos)) < 1 ||
 	   keys > (uint) (MARIA_KEYS-first_key))