Bug#33143: Incorrect ORDER BY for ROUND()/TRUNCATE() result

The ROUND(X, D) function would change the Item::decimals field during
execution to achieve the effect of a dynamic number of decimal digits.
This caused a series of bugs:
Bug #30617:Round() function not working under some circumstances in InnoDB
Bug #33402:ROUND with decimal and non-constant cannot round to 0 decimal places
Bug #30889:filesort and order by with float/numeric crashes server
Fixed by never changing the number of shown digits for DECIMAL when
used with a nonconstant number of decimal digits.


mysql-test/r/type_decimal.result:
  Bug#33143: Test result
mysql-test/t/type_decimal.test:
  Bug#33143: Test case
sql/item_func.cc:
  Bug#33143: 
  - Moved the DECIMAL_MAX_SCALE limitation to fix_length_and_dec.
  - Removed resetting of Item::decimals field.
  - set the frac field of the output value to current scale.
strings/decimal.c:
  Bug#33143: It is necessary to set all digits in the buffer following the 
  rounded one to zero, as they may now be displayed.

1 files changed, 6 insertions, 6 deletions

diff --git a/sql/item_func.cc b/sql/item_func.cc
index 96326ee2728..443360625d0 100644
--- a/sql/item_func.cc
+++ b/sql/item_func.cc
@@ -2008,7 +2008,7 @@ void Item_func_round::fix_length_and_dec()
     int length_increase= ((decimals_delta <= 0) || truncate) ? 0:1;
 
     precision-= decimals_delta - length_increase;
-    decimals= decimals_to_set;
+    decimals= min(decimals_to_set, DECIMAL_MAX_SCALE);
     max_length= my_decimal_precision_to_length(precision, decimals,
                                                unsigned_flag);
     break;
@@ -2107,18 +2107,18 @@ my_decimal *Item_func_round::decimal_op(my_decimal *decimal_value)
 {
   my_decimal val, *value= args[0]->val_decimal(&val);
   longlong dec= args[1]->val_int();
-  if (dec > 0 || (dec < 0 && args[1]->unsigned_flag))
-  {
+  if (dec >= 0 || args[1]->unsigned_flag)
     dec= min((ulonglong) dec, DECIMAL_MAX_SCALE);
-    decimals= (uint8) dec; // to get correct output
-  }
   else if (dec < INT_MIN)
     dec= INT_MIN;
     
   if (!(null_value= (args[0]->null_value || args[1]->null_value ||
                      my_decimal_round(E_DEC_FATAL_ERROR, value, (int) dec,
-                                      truncate, decimal_value) > 1)))
+                                      truncate, decimal_value) > 1))) 
+  {
+    decimal_value->frac= decimals;
     return decimal_value;
+  }
   return 0;
 }