ash: expand: Do not quote backslashes in unquoted parameter expansion

Upstream commit:

    Date: Wed, 28 Mar 2018 18:37:51 +0800
    expand: Do not quote backslashes in unquoted parameter expansion

    Here is a better example:

        a="/*/\nullx" b="/*/\null"; printf "%s\n" $a $b

    dash currently prints

        /*/\nullx
        /*/\null

    bash prints

        /*/\nullx
        /dev/null

    You may argue the bash behaviour is inconsistent but it actually
    makes sense.  What happens is that quote removal only applies to
    the original token as seen by the shell.  It is never applied to
    the result of parameter expansion.

    Now you may ask why on earth does the second line say "/dev/null"
    instead of "/dev/\null".  Well that's because it is not the quote
    removal step that removed the backslash, but the pathname expansion.

    The fact that the /de\v does not become /dev even though it exists
    is just the result of the optimisation to avoid unnecessarily
        calling stat(2).  I have checked POSIX and I don't see anything
    that forbids this behaviour.

    So going back to dash yes I think we should adopt the bash behaviour
    for pathname expansion and keep the existing case semantics.

    This patch does exactly that.  Note that this patch does not work
    unless you have already applied

        https://patchwork.kernel.org/patch/10306507/

    because otherwise the optimisation mentioned above does not get
    detected correctly and we will end up doing quote removal twice.

    This patch also updates expmeta to handle naked backslashes at
    the end of the pattern which is now possible.

    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

function                                             old     new   delta
expmeta                                              618     653     +35
memtodest                                            146     147      +1

Tested to work with both ASH_INTERNAL_GLOB on and off.

hush does not handle this correctly.

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>

5 files changed, 32 insertions, 6 deletions

diff --git a/shell/ash.c b/shell/ash.c
index ad50537a1..dc1a55a6b 100644
--- a/shell/ash.c
+++ b/shell/ash.c
@@ -6236,9 +6236,7 @@ memtodest(const char *p, size_t len, int syntax, int quotes)
 			if (quotes & QUOTES_ESC) {
 				int n = SIT(c, syntax);
 				if (n == CCTL
-				 || (((quotes & EXP_FULL) || syntax != BASESYNTAX)
-				     && n == CBACK
-				    )
+				 || (syntax != BASESYNTAX && n == CBACK)
 				) {
 					USTPUTC(CTLESC, q);
 				}
@@ -7639,7 +7637,7 @@ expmeta(exp_t *exp, char *name, unsigned name_len, unsigned expdir_len)
 				}
 			}
 		} else {
-			if (*p == '\\')
+			if (*p == '\\' && p[1])
 				esc++;
 			if (p[esc] == '/') {
 				if (metaflag)
@@ -7653,7 +7651,7 @@ expmeta(exp_t *exp, char *name, unsigned name_len, unsigned expdir_len)
 			return;
 		p = name;
 		do {
-			if (*p == '\\')
+			if (*p == '\\' && p[1])
 				p++;
 			*enddir++ = *p;
 		} while (*p++);
@@ -7665,7 +7663,7 @@ expmeta(exp_t *exp, char *name, unsigned name_len, unsigned expdir_len)
 	if (name < start) {
 		p = name;
 		do {
-			if (*p == '\\')
+			if (*p == '\\' && p[1])
 				p++;
 			*enddir++ = *p++;
 		} while (p < start);
diff --git a/shell/ash_test/ash-glob/glob_bkslash_in_var.right b/shell/ash_test/ash-glob/glob_bkslash_in_var.right
new file mode 100644
index 000000000..f1484b9e4
--- /dev/null
+++ b/shell/ash_test/ash-glob/glob_bkslash_in_var.right
@@ -0,0 +1,4 @@
+Unquoted non-matching glob in var:'test*.TMP/\name_doesnt_exist'
+Unquoted matching glob in var:    'testdir.TMP/name'
+Quoted non-matching glob in var:  'test*.TMP/\name_doesnt_exist'
+Quoted matching glob in var:      'test*.TMP/\name'
diff --git a/shell/ash_test/ash-glob/glob_bkslash_in_var.tests b/shell/ash_test/ash-glob/glob_bkslash_in_var.tests
new file mode 100755
index 000000000..e3dedc4ac
--- /dev/null
+++ b/shell/ash_test/ash-glob/glob_bkslash_in_var.tests
@@ -0,0 +1,10 @@
+mkdir testdir.TMP
+>testdir.TMP/name
+a="test*.TMP/\name_doesnt_exist"
+b="test*.TMP/\name"
+printf "Unquoted non-matching glob in var:'%s'\n" $a
+printf "Unquoted matching glob in var:    '%s'\n" $b
+printf "Quoted non-matching glob in var:  '%s'\n" "$a"
+printf "Quoted matching glob in var:      '%s'\n" "$b"
+rm -f testdir.TMP/name
+rmdir testdir.TMP
diff --git a/shell/hush_test/hush-glob/glob_bkslash_in_var.right b/shell/hush_test/hush-glob/glob_bkslash_in_var.right
new file mode 100644
index 000000000..f1484b9e4
--- /dev/null
+++ b/shell/hush_test/hush-glob/glob_bkslash_in_var.right
@@ -0,0 +1,4 @@
+Unquoted non-matching glob in var:'test*.TMP/\name_doesnt_exist'
+Unquoted matching glob in var:    'testdir.TMP/name'
+Quoted non-matching glob in var:  'test*.TMP/\name_doesnt_exist'
+Quoted matching glob in var:      'test*.TMP/\name'
diff --git a/shell/hush_test/hush-glob/glob_bkslash_in_var.tests b/shell/hush_test/hush-glob/glob_bkslash_in_var.tests
new file mode 100755
index 000000000..e3dedc4ac
--- /dev/null
+++ b/shell/hush_test/hush-glob/glob_bkslash_in_var.tests
@@ -0,0 +1,10 @@
+mkdir testdir.TMP
+>testdir.TMP/name
+a="test*.TMP/\name_doesnt_exist"
+b="test*.TMP/\name"
+printf "Unquoted non-matching glob in var:'%s'\n" $a
+printf "Unquoted matching glob in var:    '%s'\n" $b
+printf "Quoted non-matching glob in var:  '%s'\n" "$a"
+printf "Quoted matching glob in var:      '%s'\n" "$b"
+rm -f testdir.TMP/name
+rmdir testdir.TMP