MDEV-9745 Crash with CASE WHEN TRUE THEN COALESCE(CAST(NULL AS UNSIGNED)) ELSE 4 END

This is a backport of the patch for MDEV-9653 (fixed earlier in 10.1.13).

The code in Item_func_case::fix_length_and_dec() did not
calculate max_length and decimals properly.

In case of any numeric result (DECIMAL, REAL, INT) a generic method
Item_func_case::agg_num_lengths() was called, which could erroneously result
into a DECIMAL item with max_length==0 and decimals==0, so the constructor of
Field_new_decimals tried to create a field of DECIMAL(0,0) type,
which caused a crash.

Unlike Item_func_case, the code responsible for merging attributes in
Item_func_coalesce::fix_length_and_dec() works fine: it has specific execution
branches for all distinct numeric types and correctly creates a DECIMAL(1,0)
column instead of DECIMAL(0,0) for the same set of arguments.

The fix does the following:
- Moves the attribute merging code from Item_func_coalesce::fix_length_and_dec()
  to a new method Item_func_hybrid_result_type::fix_attributes()
- Removes the wrong code from Item_func_case::fix_length_and_dec()
  and reuses fix_attributes() in both Item_func_coalesce::fix_length_and_dec()
  and Item_func_case::fix_length_and_dec()
- Fixes count_real_length() and count_decimal_length() to get an array
  of Items as an argument, instead of using Item::args directly.
  This is needed for Item_func_case::fix_length_and_dec().
- Moves methods Item_func::count_xxx_length() from "public" to "protected".
- Removes Item_func_case::agg_num_length(), as it's not used any more.
- Additionally removes Item_func_case::agg_str_length(),
  as it also was not used (dead code).

1 files changed, 0 insertions, 2 deletions

diff --git a/sql/item_cmpfunc.h b/sql/item_cmpfunc.h
index 0faba016ba8..0194f9cd0e0 100644
--- a/sql/item_cmpfunc.h
+++ b/sql/item_cmpfunc.h
@@ -1255,8 +1255,6 @@ public:
   Item *find_item(String *str);
   CHARSET_INFO *compare_collation() { return cmp_collation.collation; }
   void cleanup();
-  void agg_str_lengths(Item *arg);
-  void agg_num_lengths(Item *arg);
 };
 
 /*