Handle weird cases of token pasting

Especially when token pasting involves floating-point numbers, we can
have some really strange effects from token pasting: for example,
pasting the two tokens "xyzzy" and "1e+10" ends up with *three*
tokens: "xyzzy1e" "+" "10".  The easiest way to deal with this is to
explicitly combine the string and then run tokenize() on it.

Signed-off-by: H. Peter Anvin <hpa@zytor.com>

1 files changed, 14 insertions, 0 deletions

diff --git a/test/weirdpaste.asm b/test/weirdpaste.asm
new file mode 100644
index 00000000..c6e98150
--- /dev/null
+++ b/test/weirdpaste.asm
@@ -0,0 +1,14 @@
+;Testname=preproc; Arguments=-E; Files=stdout stderr
+;Testname=bin; Arguments=-fbin -oweirdpaste.bin; Files=stdout stderr weirdpaste.bin
+
+	%define foo xyzzy
+%define bar 1e+10
+
+%define xyzzy1e 15
+
+%macro dx 2
+%assign	xx %1%2
+	dw xx
+%endmacro
+
+	dx foo, bar