ApiAnnotations: Add SourceText for unicode tokens

Summary:
At the moment there is no way to tell if a given token used its unicode
variant or its normal one, except to look at the length of the token.

This fails for the unicode '*'.

Expose the original source text for unicode variants so that API
Annotations can capture them specifically.

Test Plan: ./validate

Reviewers: mpickering, austin, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1473

GHC Trac Issues: #11018

1 files changed, 5 insertions, 0 deletions

diff --git a/testsuite/tests/ghc-api/annotations/T10358.stdout b/testsuite/tests/ghc-api/annotations/T10358.stdout
index ae1ec8587f..2bcbf68c09 100644
--- a/testsuite/tests/ghc-api/annotations/T10358.stdout
+++ b/testsuite/tests/ghc-api/annotations/T10358.stdout
@@ -15,18 +15,23 @@
 ((Test10358.hs:5:7-16,AnnEqual), [Test10358.hs:5:12]),
 ((Test10358.hs:5:7-16,AnnSemi), [Test10358.hs:5:17]),
 ((Test10358.hs:5:14-16,AnnVal), [Test10358.hs:5:15]),
+((Test10358.hs:5:15,AnnStar), [Test10358.hs:5:15]),
 ((Test10358.hs:5:19-22,AnnBang), [Test10358.hs:5:19]),
 ((Test10358.hs:5:19-32,AnnEqual), [Test10358.hs:5:24]),
 ((Test10358.hs:5:19-32,AnnSemi), [Test10358.hs:6:7]),
 ((Test10358.hs:5:26-32,AnnVal), [Test10358.hs:5:29]),
+((Test10358.hs:5:29,AnnStar), [Test10358.hs:5:29]),
 ((Test10358.hs:6:7-16,AnnEqual), [Test10358.hs:6:10]),
 ((Test10358.hs:6:7-16,AnnFunId), [Test10358.hs:6:7-8]),
 ((Test10358.hs:6:7-16,AnnSemi), [Test10358.hs:7:7]),
 ((Test10358.hs:6:12-14,AnnVal), [Test10358.hs:6:13]),
 ((Test10358.hs:6:12-16,AnnVal), [Test10358.hs:6:15]),
+((Test10358.hs:6:13,AnnStar), [Test10358.hs:6:13]),
+((Test10358.hs:6:15,AnnStar), [Test10358.hs:6:15]),
 ((Test10358.hs:7:7-17,AnnEqual), [Test10358.hs:7:10]),
 ((Test10358.hs:7:7-17,AnnFunId), [Test10358.hs:7:7-8]),
 ((Test10358.hs:7:12-17,AnnVal), [Test10358.hs:7:14]),
+((Test10358.hs:7:14,AnnStar), [Test10358.hs:7:14]),
 ((<no location info>,AnnEofPos), [Test10358.hs:9:1])
 ]