Added test cases for preceding test

This includes all test changes from
"Changing all cost calculation to be given in milliseconds"
and forwards.

Some of the things that caused changes in the result files:

- As part of fixing tests, I added 'echo' to some comments to be able to
  easier find out where things where wrong.
- MATERIALIZED has now a higher cost compared to X than before. Because
  of this some MATERIALIZED types have changed to DEPENDEND SUBQUERY.
  - Some test cases that required MATERIALIZED to repeat a bug was
    changed by adding more rows to force MATERIALIZED to happen.
- 'Filtered' in SHOW EXPLAIN has in many case changed from 100.00 to
  something smaller. This is because now filtered also takes into
  account the smallest possible ref access and filters, even if they
  where not used. Another reason for 'Filtered' being smaller is that
  we now also take into account implicit filtering done for subqueries
  using FIRSTMATCH.
  (main.subselect_no_exists_to_in)
  This is caluculated in best_access_path() and stored in records_out.
- Table orders has changed because more accurate costs.
- 'index' and 'ALL' for small tables has changed to use 'range' or
   'ref' because of optimizer_scan_setup_cost.
- index can be changed to 'range' as 'range' optimizer assumes we don't
  have to read the blocks from disk that range optimizer has already read.
  This can be confusing in the case where there is no obvious where clause
  but instead there is a hidden 'key_column > NULL' added by the optimizer.
  (main.subselect_no_exists_to_in)
- Scan on primary clustered key does not report 'Using Index' anymore
  (It's a table scan, not an index scan).
- For derived tables, the number of rows is now 100 instead of 2,
  which can be seen in EXPLAIN.
- More tests have "Using index for group by" as the cost of this
  optimization is now more correct (lower).
- A primary key could be preferred for a normal key, even if it would
  access more rows, as it's faster to do 1 lokoup and 3 'index_next' on a
  clustered primary key than one lookup trough a secondary.
  (main.stat_tables_innodb)

Notes:

- There was a 4.7% more calls to best_extension_by_limited_search() in
  the main.greedy_optimizer test.  However examining the test results
  it looked that the plans where slightly better (eq_ref where more
  chained together) so I assume this is ok.
- I have verified a few test cases where there was notable/unexpected
  changes in the plan and in all cases the new optimizer plans where
  faster.  (main.greedy_optimizer and some others)

1 files changed, 50 insertions, 37 deletions

diff --git a/mysql-test/main/range_mrr_icp.result b/mysql-test/main/range_mrr_icp.result
index a3c8bb7b8c2..1373faedf8e 100644
--- a/mysql-test/main/range_mrr_icp.result
+++ b/mysql-test/main/range_mrr_icp.result
@@ -255,7 +255,7 @@ id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra
 1	SIMPLE	t1	ref	x	x	5	const	1	Using index
 explain select count(*) from t1 where x in (1,2,3,4);
 id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra
-1	SIMPLE	t1	index	x	x	5	NULL	9	Using where; Using index
+1	SIMPLE	t1	range	x	x	5	NULL	4	Using where; Using index
 drop table t1;
 CREATE TABLE t1 (key1 int(11) NOT NULL default '0', KEY i1 (key1));
 INSERT INTO t1 VALUES (0),(0),(0),(0),(0),(1),(1);
@@ -264,12 +264,12 @@ INSERT INTO t2 VALUES (0),(0),(1),(1),(2),(2);
 explain select * from t1, t2 where (t1.key1 <t2.keya + 1) and t2.keya=3;
 id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra
 1	SIMPLE	t2	ref	j1	j1	4	const	1	Using index
-1	SIMPLE	t1	index	i1	i1	4	NULL	7	Using where; Using index; Using join buffer (flat, BNL join)
+1	SIMPLE	t1	range	i1	i1	4	NULL	7	Using where; Using index; Using join buffer (flat, BNL join)
 explain select * from t1 force index(i1), t2 force index(j1) where 
 (t1.key1 <t2.keya + 1) and t2.keya=3;
 id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra
 1	SIMPLE	t2	ref	j1	j1	4	const	1	Using index
-1	SIMPLE	t1	index	i1	i1	4	NULL	7	Using where; Using index; Using join buffer (flat, BNL join)
+1	SIMPLE	t1	range	i1	i1	4	NULL	7	Using where; Using index; Using join buffer (flat, BNL join)
 DROP TABLE t1,t2;
 CREATE TABLE t1 (
 a int(11) default NULL,
@@ -312,6 +312,9 @@ a	b
 15	1
 47	1
 DROP TABLE t1;
+#
+# Test of problem with IN on many different keyparts. (Bug #4157)
+#
 CREATE TABLE t1 (
 id int( 11 ) unsigned NOT NULL AUTO_INCREMENT ,
 line int( 5 ) unsigned NOT NULL default '0',
@@ -328,10 +331,17 @@ KEY `LINES` ( owner, tableid, content, id ) ,
 KEY recount( owner, line ) 
 ) ENGINE = MYISAM;
 INSERT into t1 (owner,id,columnid,line) values (11,15,15,1),(11,13,13,5);
+INSERT into t1 (owner,id,columnid,line) select 11,seq+20,seq,seq from seq_1_to_100;
+explain SELECT id, columnid, tableid, content, showid, line, ordinal FROM t1 WHERE owner=11 AND ((columnid IN ( 15, 13, 14 ) AND line IN ( 1, 2, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 31 )) OR (columnid IN ( 13, 14 ) AND line IN ( 15 ))) LIMIT 0 , 30;
+id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra
+1	SIMPLE	t1	ref	PRIMARY,menu,COLUMN,LINES,recount	COLUMN	4	const	11	Using index condition
 SELECT id, columnid, tableid, content, showid, line, ordinal FROM t1 WHERE owner=11 AND ((columnid IN ( 15, 13, 14 ) AND line IN ( 1, 2, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 31 )) OR (columnid IN ( 13, 14 ) AND line IN ( 15 ))) LIMIT 0 , 30;
 id	columnid	tableid	content	showid	line	ordinal
-15	15	1	188	1	1	0
 13	13	1	188	1	5	0
+15	15	1	188	1	1	0
+33	13	1	188	1	13	0
+34	14	1	188	1	14	0
+35	15	1	188	1	15	0
 drop table t1;
 create  table t1 (id int(10) primary key);
 insert into t1 values (1),(2),(3),(4),(5),(6),(7),(8),(9);
@@ -679,7 +689,7 @@ create table t1(a char(2), key(a(1)));
 insert into t1 values ('x'), ('xx');
 explain select a from t1 where a > 'x';
 id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra
-1	SIMPLE	t1	ALL	a	NULL	NULL	NULL	2	Using where
+1	SIMPLE	t1	range	a	a	2	NULL	2	Using where
 select a from t1 where a > 'x';
 a
 xx
@@ -726,7 +736,7 @@ WHERE
 v.oxrootid ='d8c4177d09f8b11f5.52725521' AND
 s.oxleft > v.oxleft AND s.oxleft < v.oxright;
 id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra
-1	SIMPLE	v	ALL	OXLEFT,OXRIGHT,OXROOTID	NULL	NULL	NULL	12	Using where
+1	SIMPLE	v	ref	OXLEFT,OXRIGHT,OXROOTID	OXROOTID	34	const	6	Using index condition
 1	SIMPLE	s	ALL	OXLEFT	NULL	NULL	NULL	12	Range checked for each record (index map: 0x4)
 SELECT s.oxid FROM t1 v, t1 s 
 WHERE
@@ -1141,7 +1151,7 @@ INSERT INTO t1 VALUES
 ('A2','2005-12-01 08:00:00',1000);
 EXPLAIN SELECT * FROM t1 WHERE item='A1' AND started<='2005-12-01 24:00:00';
 id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra
-1	SIMPLE	t1	ALL	PRIMARY	NULL	NULL	NULL	4	Using where
+1	SIMPLE	t1	ref	PRIMARY	PRIMARY	20	const	3	Using index condition
 SELECT * FROM t1 WHERE item='A1' AND started<='2005-12-01 24:00:00';
 item	started	price
 Warnings:
@@ -1239,13 +1249,16 @@ insert into t1 values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9);
 create table t2 (a int, b int, filler char(100));
 insert into t2 select A.a + 10 * (B.a + 10 * C.a), 10, 'filler' from t1 A,
 t1 B, t1 C where A.a < 5;
-insert into t2 select 1000, b, 'filler' from t2 limit 250;
+insert into t2 select 1000, b, 'filler' from t2 limit 50;
+select count(*) from t2;
+count(*)
+550
 alter table t2 add index (a,b);
-# In following EXPLAIN the access method should be ref, #rows~=250
+# In following EXPLAIN the access method should be ref, #rows~=50
 # (and not 2) when we are not using rowid-ordered scans
 explain select * from t2 where a=1000 and b<11;
 id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra
-1	SIMPLE	t2	range	a	a	10	NULL	253	Using index condition; Rowid-ordered scan
+1	SIMPLE	t2	range	a	a	10	NULL	63	Using index condition; Rowid-ordered scan
 drop table t1, t2;
 CREATE TABLE t1( a INT, b INT, KEY( a, b ) );
 CREATE TABLE t2( a INT, b INT, KEY( a, b ) );
@@ -2565,7 +2578,7 @@ EXPLAIN
           "key_length": "5",
           "used_key_parts": ["d"],
           "rows": 3,
-          "filtered": 100,
+          "filtered": 60,
           "index_condition": "t2.d is not null",
           "attached_condition": "(t2.d,t2.e) in (<cache>((3,3)),<cache>((7,7)),<cache>((2,2)))",
           "mrr_type": "Rowid-ordered scan"
@@ -2644,7 +2657,7 @@ EXPLAIN
           "key_length": "5",
           "used_key_parts": ["d"],
           "rows": 8,
-          "filtered": 100,
+          "filtered": 14.42307663,
           "index_condition": "t2.d is not null",
           "attached_condition": "(t2.d,t2.e) in (<cache>((3,3)),<cache>((7,7)),<cache>((8,8))) and octet_length(t2.f) = 1",
           "mrr_type": "Rowid-ordered scan"
@@ -2660,7 +2673,7 @@ EXPLAIN
           "used_key_parts": ["a"],
           "ref": ["test.t2.d"],
           "rows": 8,
-          "filtered": 100
+          "filtered": 73.17073059
         }
       }
     ]
@@ -2689,40 +2702,40 @@ prepare stmt from "select * from t1,t2
 where a = d and (a,e) in ((3,3),(7,7),(8,8)) and length(f) = 1";
 execute stmt;
 a	b	c	d	e	f
-3	2	uuuw	3	3	i
 3	2	uuua	3	3	i
-3	3	zzzz	3	3	i
+3	2	uuua	3	3	i
+3	2	uuuw	3	3	i
+3	2	uuuw	3	3	i
+3	3	zyxa	3	3	i
+3	3	zyxa	3	3	i
+3	3	zyxw	3	3	i
 3	3	zyxw	3	3	i
 3	3	zzza	3	3	i
-3	3	zyxa	3	3	i
-7	7	xxxyy	7	7	h
+3	3	zzza	3	3	i
+3	3	zzzz	3	3	i
+3	3	zzzz	3	3	i
 7	7	xxxya	7	7	h
-7	8	xxxxx	7	7	h
+7	7	xxxyy	7	7	h
 7	8	xxxxa	7	7	h
-3	2	uuuw	3	3	i
-3	2	uuua	3	3	i
-3	3	zzzz	3	3	i
-3	3	zyxw	3	3	i
-3	3	zzza	3	3	i
-3	3	zyxa	3	3	i
+7	8	xxxxx	7	7	h
 execute stmt;
 a	b	c	d	e	f
-3	2	uuuw	3	3	i
 3	2	uuua	3	3	i
-3	3	zzzz	3	3	i
+3	2	uuua	3	3	i
+3	2	uuuw	3	3	i
+3	2	uuuw	3	3	i
+3	3	zyxa	3	3	i
+3	3	zyxa	3	3	i
+3	3	zyxw	3	3	i
 3	3	zyxw	3	3	i
 3	3	zzza	3	3	i
-3	3	zyxa	3	3	i
-7	7	xxxyy	7	7	h
+3	3	zzza	3	3	i
+3	3	zzzz	3	3	i
+3	3	zzzz	3	3	i
 7	7	xxxya	7	7	h
-7	8	xxxxx	7	7	h
+7	7	xxxyy	7	7	h
 7	8	xxxxa	7	7	h
-3	2	uuuw	3	3	i
-3	2	uuua	3	3	i
-3	3	zzzz	3	3	i
-3	3	zyxw	3	3	i
-3	3	zzza	3	3	i
-3	3	zyxa	3	3	i
+7	8	xxxxx	7	7	h
 deallocate prepare stmt;
 insert into t1 select * from t1;
 # join order: (t2,t1) with ref access of t1
@@ -2748,7 +2761,7 @@ EXPLAIN
           "key_length": "5",
           "used_key_parts": ["d"],
           "rows": 7,
-          "filtered": 100,
+          "filtered": 6.730769157,
           "index_condition": "t2.d is not null",
           "attached_condition": "(t2.d,t2.e) in (<cache>((4,4)),<cache>((7,7)),<cache>((8,8))) and octet_length(t2.f) = 1",
           "mrr_type": "Rowid-ordered scan"
@@ -3369,7 +3382,7 @@ insert into t2 select A.a + B.a*10 + C.a*100 from ten A, ten B,ten C where A.a +
 # expected type=range, rows=1487 , reason=using index dives
 analyze SELECT * FROM t1 where a in (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198);
 id	select_type	table	type	possible_keys	key	key_len	ref	rows	r_rows	filtered	r_filtered	Extra
-1	SIMPLE	t1	range	a	a	5	NULL	1487	1199.00	100.00	100.00	Using where; Using index
+1	SIMPLE	t1	index	a	a	5	NULL	2000	2000.00	74.35	59.95	Using where; Using index
 insert into t2 values (200),(201);
 # expected type=range, rows=201 , reason=using index statistics
 analyze SELECT * FROM t1 where a in (0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,200,201);