Eshell variable expansion should always return strings inside quotes

This is closer in behavior to regular shells, and gives Eshell users
greater flexibility in how variables are expanded.

* lisp/eshell/esh-util.el (eshell-convert): Add TO-STRING argument.

* lisp/eshell/esh-var.el (eshell-parse-variable-ref): Add MODIFIER-P
argument and adjust how 'eshell-convert' and 'eshell-apply-indices'
are called.
(eshell-get-variable, eshell-apply-indices): Add QUOTED argument.

* test/lisp/eshell/esh-var-tests.el (eshell-test-value): New defvar.
(esh-var-test/interp-convert-var-number)
(esh-var-test/interp-convert-var-split-indices)
(esh-var-test/interp-convert-quoted-var-number)
(esh-var-test/interp-convert-quoted-var-split-indices)
(esh-var-test/interp-convert-cmd-string-newline)
(esh-var-test/interp-convert-cmd-multiline)
(esh-var-test/interp-convert-cmd-number)
(esh-var-test/interp-convert-cmd-split-indices)
(esh-var-test/quoted-interp-convert-var-number)
(esh-var-test/quoted-interp-convert-var-split-indices)
(esh-var-test/quoted-interp-convert-quoted-var-number)
(esh-var-test/quoted-interp-convert-quoted-var-split-indices)
(esh-var-test/quoted-interp-convert-cmd-string-newline)
(esh-var-test/quoted-interp-convert-cmd-multiline)
(esh-var-test/quoted-interp-convert-cmd-number)
(esh-var-test/quoted-interp-convert-cmd-split-indices): New tests.

* doc/misc/eshell.texi (Arguments): Expand this section, and document
the new behavior.
(Dollars Expansion): Provide more detail about '$(lisp)' and
'${command}' forms.

* etc/NEWS (Eshell): Announce this change (bug#55236).

1 files changed, 45 insertions, 12 deletions

diff --git a/doc/misc/eshell.texi b/doc/misc/eshell.texi
index c9c11a38693..fff06b527cf 100644
--- a/doc/misc/eshell.texi
+++ b/doc/misc/eshell.texi
@@ -228,15 +228,39 @@ other background process in Emacs.
 
 @node Arguments
 @section Arguments
-Command arguments are passed to the functions as either strings or
-numbers, depending on what the parser thinks they look like.  If you
-need to use a function that takes some other data type, you will need to
-call it in an Elisp expression (which can also be used with
-@ref{Expansion, expansions}).  As with other shells, you can
-escape special characters and spaces with the backslash (@code{\}) and
-apostrophes (@code{''}) and double quotes (@code{""}).  This is needed
-especially for file names with special characters like pipe
-(@code{|}), which could be part of remote file names.
+Ordinarily, command arguments are parsed by Eshell as either strings
+or numbers, depending on what the parser thinks they look like.  To
+specify an argument of some other data type, you can use an
+@ref{Dollars Expansion, Elisp expression}:
+
+@example
+~ $ echo (list 1 2 3)
+(1 2 3)
+@end example
+
+Additionally, many @ref{Built-ins, Eshell commands} will flatten the
+arguments they receive, so passing a list as an argument will
+``spread'' the elements into multiple arguments:
+
+@example
+~ $ printnl (list 1 2) 3
+1
+2
+3
+@end example
+
+@subsection Quoting and escaping
+
+As with other shells, you can escape special characters and spaces
+with by prefixing the character with a backslash (@code{\}), or by
+surrounding the string with apostrophes (@code{''}) or double quotes
+(@code{""}).  This is needed especially for file names with special
+characters like pipe (@code{|}), which could be part of remote file
+names.
+
+When using @ref{Expansion, expansions} in an Eshell command, the
+result may potentially be of any data type.  To ensure that the result
+is always a string, the expansion can be surrounded by double quotes.
 
 @node Built-ins
 @section Built-in commands
@@ -1026,11 +1050,20 @@ value, such as @samp{$"@var{var}"-suffix}.
 @item $(@var{lisp})
 Expands to the result of evaluating the S-expression @code{(@var{lisp})}.  On
 its own, this is identical to just @code{(@var{lisp})}, but with the @code{$},
-it can be used in a string, such as @samp{/some/path/$(@var{lisp}).txt}.
+it can be used inside double quotes or within a longer string, such as
+@samp{/some/path/$(@var{lisp}).txt}.
 
 @item $@{@var{command}@}
-Returns the output of @command{@var{command}}, which can be any valid Eshell
-command invocation, and may even contain expansions.
+Returns the output of @command{@var{command}}, which can be any valid
+Eshell command invocation, and may even contain expansions.  Similar
+to @code{$(@var{lisp})}, this is identical to @code{@{@var{command}@}}
+when on its own, but the @code{$} allows it to be used inside double
+quotes or as part of a string.
+
+Normally, the output is split line-by-line, returning a list (or the
+first element if there's only one line of output).  However, when this
+expansion is surrounded by double quotes, it returns the output as a
+single string instead.
 
 @item $<@var{command}>
 As with @samp{$@{@var{command}@}}, evaluates the Eshell command invocation