|  | Commit message (Collapse) | Author | Age | Files | Lines | 
|---|
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | Summary:
Add prettyprinter tests, which take a file, parse it, pretty print it,
re-parse the pretty printed version and then compare the original and
new ASTs (ignoring locations)
Updates haddock submodule to match the AST changes.
There are three issues outstanding
1. Extra parens around a context are not reproduced. This will require an
   AST change and will be done in a separate patch.
2. Currently if an `HsTickPragma` is found, this is not pretty-printed,
   to prevent noise in the output.
   I am not sure what the desired behaviour in this case is, so have left
   it as before. Test Ppr047 is marked as expected fail for this.
3. Apart from in a context, the ParsedSource AST keeps all the parens from
   the original source.  Something is happening in the renamer to remove the
   parens around visible type application, causing T12530 to fail, as the
   dumped splice decl is after the renamer.
   This needs to be fixed by keeping the parens, but I do not know where they
   are being removed.  I have amended the test to pass, by removing the parens
   in the expected output.
Test Plan: ./validate
Reviewers: goldfire, mpickering, simonpj, bgamari, austin
Reviewed By: simonpj, bgamari
Subscribers: simonpj, goldfire, thomie, mpickering
Differential Revision: https://phabricator.haskell.org/D2752
GHC Trac Issues: #3384 | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | This patch replaces calls to barf() in loadArchive() with proper
error handling.
Test Plan: GHC CI
Reviewers: rwbarton, erikd, hvr, austin, simonmar, bgamari
Reviewed By: bgamari
Subscribers: thomie
Tags: #ghc
Differential Revision: https://phabricator.haskell.org/D2652
GHC Trac Issues: #12388 | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | We want to avoid using SIGALRM whenever possible since we will interrupt
long-running system calls. See #10840.
Test Plan: Validate on Darwin
Reviewers: austin, erikd, simonmar
Reviewed By: simonmar
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2796
GHC Trac Issues: #10840 | 
| | 
| 
| 
| | See comments. | 
| | 
| 
| 
| 
| | Static string optimisation means we get a ForeignPtr with an IORef
inside it, leading to a different error. | 
| | |  | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | Summary:
This commit makes various improvements and addresses some issues with
Compact Regions (aka Compact Normal Forms).
This was the most important thing I wanted to fix.  Compaction
previously prevented GC from running until it was complete, which
would be a problem in a multicore setting.  Now, we compact using a
hand-written Cmm routine that can be interrupted at any point.  When a
GC is triggered during a sharing-enabled compaction, the GC has to
traverse and update the hash table, so this hash table is now stored
in the StgCompactNFData object.
Previously, compaction consisted of a deepseq using the NFData class,
followed by a traversal in C code to copy the data.  This is now done
in a single pass with hand-written Cmm (see rts/Compact.cmm). We no
longer use the NFData instances, instead the Cmm routine evaluates
components directly as it compacts.
The new compaction is about 50% faster than the old one with no
sharing, and a little faster on average with sharing (the cost of the
hash table dominates when we're doing sharing).
Static objects that don't (transitively) refer to any CAFs don't need
to be copied into the compact region.  In particular this means we
often avoid copying Char values and small Int values, because these
are static closures in the runtime.
Each Compact# object can support a single compactAdd# operation at any
given time, so the Data.Compact library now enforces mutual exclusion
using an MVar stored in the Compact object.
We now get exceptions rather than killing everything with a barf()
when we encounter an object that cannot be compacted (a function, or a
mutable object).  We now also detect pinned objects, which can't be
compacted either.
The Data.Compact API has been refactored and cleaned up.  A new
compactSize operation returns the size (in bytes) of the compact
object.
Most of the documentation is in the Haddock docs for the compact
library, which I've expanded and improved here.
Various comments in the code have been improved, especially the main
Note [Compact Normal Forms] in rts/sm/CNF.c.
I've added a few tests, and expanded a few of the tests that were
there.  We now also run the tests with GHCi, and in a new test way
that enables sanity checking (+RTS -DS).
There's a benchmark in libraries/compact/tests/compact_bench.hs for
measuring compaction speed and comparing sharing vs. no sharing.
The field totalDataW in StgCompactNFData was unnecessary.
Test Plan:
* new unit tests
* validate
* tested manually that we can compact Data.Aeson data
Reviewers: gcampax, bgamari, ezyang, austin, niteria, hvr, erikd
Subscribers: thomie, simonpj
Differential Revision: https://phabricator.haskell.org/D2751
GHC Trac Issues: #12455 | 
| | 
| 
| 
| | I failed at the last attempt. | 
| | |  | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | Test Plan: make test TEST=T12919
Reviewers: bgamari, austin
Reviewed By: bgamari
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2788
GHC Trac Issues: #12919 | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | Removed the alignment for strings and mark then as cstring sections in
the generated asm so the linker can merge duplicate sections.
Reviewers: rwbarton, trofi, austin, trommler, simonmar, hvr, bgamari
Reviewed By: hvr, bgamari
Subscribers: simonpj, hvr, thomie
Differential Revision: https://phabricator.haskell.org/D1290
GHC Trac Issues: #9577 | 
| | 
| 
| 
| | This reverts commit 19ae142364058e258122f4bb68ef4b9aa6e41890. | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | Summary:
This fixes tests rn017, T7672 and closed #12930.
Both these tests were self referential module imports through hs-boot
files. As a result, I am quite suspicious of what the ImpAll constructor is
used for. I had a brief hunt around but couldn't immediately see whether
it was necessary.
Reviewers: austin, bgamari
Subscribers: simonpj, thomie, nomeata
Differential Revision: https://phabricator.haskell.org/D2793
GHC Trac Issues: #12930 | 
| | |  | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | Summary:
Visible API changes:
* The C struct `GCDetails` gives the stats about a single GC.  This is
  passed to the `gcDone()` callback if one is set via the
  RtsConfig. (previously we just passed a collection of values, so this
  is more extensible, at the expense of breaking the existing API)
* `RTSStats` gives cumulative stats since the start of the program,
  and includes the `GCDetails` for the most recent GC.  This struct
  can be obtained via `getRTSStats()` (the old `getGCStats()` has been
  removed, and `getGCStatsEnabled()` has been renamed to
  `getRTSStatsEnabled()`)
Improvements:
* The per-GC stats and cumulative stats are now cleanly separated.
* Inside the RTS we have a top-level `RTSStats` struct to keep all our
  stats in, previously this was just a collection of strangely-named
  variables.  This struct is mostly just copied in `getRTSStats()`, so
  the implementation of that function is a lot shorter.
* Types are more consistent.  We use a uint64_t byte count for all
  memory values, and Time for all time values.
* Names are more consistent.  We use a suffix `_bytes` for all byte
  counts and `_ns` for all time values.
* We now collect information about the amount of memory in large
  objects and compact objects in `GCDetails`. (the latter was the reason
  I started doing this patch but it seems to have ballooned a bit!)
* I fixed a bug in the calculation of the elapsed MUT time, and added
  an ASSERT to stop the calculations going wrong in the future.
For now I kept the Haskell API in `GHC.Stats` the same, by
impedence-matching with the new API.  We could either break that API
and make it match the C API more closely, or we could add a new API
and deprecate the old one.  Opinions welcome.
This stuff is very easy to get wrong, and it's hard to test.  Reviews
welcome!
Test Plan:
manual testing
validate
Reviewers: bgamari, niteria, austin, ezyang, hvr, erikd, rwbarton, Phyx
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2756 | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | Summary:
We mentioned that there were "some programs" that failed to typecheck
due to #12784, but given how surprisingly common this issue has been, it'd
be prudent to at least give one example of the bug in the release notes.
Reviewers: simonpj, bgamari, austin, rwbarton
Reviewed By: rwbarton
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2786
GHC Trac Issues: #12784 | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | Summary:
Fix issues preventing x86 GHC to build on Windows and
fix segfault in the testsuite.
Test Plan: ./validate
Reviewers: austin, erikd, simonmar, bgamari
Reviewed By: bgamari
Subscribers: #ghc_windows_task_force, thomie
Differential Revision: https://phabricator.haskell.org/D2789 | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | Summary:
D2716 introduced colors into the output of GHC.
These color ourputs are done using escape characters output
to the terminal.
These however are wreaking havoc on the testsuite output as now
no stderr with a warning or error will match anymore.
Instead of accepting the new codes as expected values instead I
turn them off. So the testsuite is consistent on platforms/terminals we
don't support colors on.
Test Plan:
any test that outputs colors. e.g.
make test TEST=T9576
Reviewers: austin, Rufflewind, bgamari
Subscribers: thomie, #ghc_windows_task_force
Differential Revision: https://phabricator.haskell.org/D2787
GHC Trac Issues: #8809 | 
| | |  | 
| | 
| 
| 
| | Refactoring only | 
| | 
| 
| 
| 
| 
| 
| 
| 
| | The used-variable calculation for pattern synonyms is a little
tricky, for reasons described in RnBinds
Note [Pattern synonym builders don't yield dependencies]
It was right semantically, but the "unused-variable warning" was
wrong, which led to Trac #12548. | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | Trac #12425 and #12234 showed up a major and long-standing
bug in the occurrence analyser, whereby it could generate
explonentially large program!
There's a lot of commentary on #12425; and it's all described
in Note [Loop breakers, node scoring, and stability]
I did quite a lot of refactoring to make the code comprehensibe
again (its structure had bit-rotted rather), so the patch
looks bigger than it really is.
Hurrah!
I did a nofib run to check that I hadn't inadertently ruined
anything:
--------------------------------------------------------------------------------
        Program           Size    Allocs   Runtime   Elapsed  TotalMem
--------------------------------------------------------------------------------
          fluid          -0.3%     -1.5%      0.01      0.01     +0.0%
         parser          -0.9%     +0.6%      0.04      0.04     +0.0%
         prolog          -0.1%     +1.2%      0.00      0.00     +0.0%
--------------------------------------------------------------------------------
            Min          -0.9%     -1.5%     -8.6%     -8.7%     +0.0%
            Max          +0.1%     +1.2%     +7.7%     +7.8%     +2.4%
 Geometric Mean          -0.2%     -0.0%     -0.2%     -0.3%     +0.0%
I checked what happened in 'prolog'.  It seems that we have a
recursive data structure something like this
   f :: [blah]
   f x = build (\cn.  ...g...  )
   g :: [blah2]
   g y = ....(foldr k z (f y))....
If we inline 'f' into 'g' we get better fusion than the other
way round, but we don't have any way to spot that at the moment.
(I wonder if we could do worker/wrapper for functions returning
a 'build'?)  It was happening before by a fluke.
Anyway I decided to accept this; it's relatively rare I think. | 
| | 
| 
| 
| 
| | Previously, it would substitute e for n without an e being around.
I clarify that by naming the scrutinee e. | 
| | 
| 
| 
| 
| 
| | It's 2.27 (-2), not 1.27.2. Dropped stray '\r'.
Signed-off-by: Sergei Trofimovich <siarheit@google.com> | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | Maintain in-scope set in deeply_instantiate (Fixes T12549).
lint fixes
Test Plan: validate
Reviewers: simonpj, austin, goldfire, bgamari
Reviewed By: simonpj, bgamari
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2757
GHC Trac Issues: #12549 | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | When rts is forked it doesn't update toplevel handler, so UserInterrupt
exception is sent to Thread1 that doesn't exist in forked process.
We install toplevel handler when fork so signal will be delivered to the
new main thread.
Fixes #12903
Reviewers: simonmar, austin, erikd, bgamari
Reviewed By: bgamari
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2770
GHC Trac Issues: #12903 | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | select() is limited to 1024 file descriptors.  This actually blew up
in a very hard-to-debug way in our production system when using the
hinotify package.
Test Plan:
libraries/tests pass, paricularly hGetBuf001 which exercises this
code.
Reviewers: niteria, erikd, austin, hvr, bgamari
Reviewed By: bgamari
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2785
GHC Trac Issues: #12912 | 
| | 
| 
| 
| 
| | Mention that many GHC testsuite tests will fail with a compiler built with
the quickest profile. See Trac #12141. | 
| | 
| 
| 
| | The testsuite now requires python >=3.0. See #12909. | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | textEncodingName is notjust a string, it must be a valid input for
mkTextEncoding, as stated in libraries/base/GHC/IO/Encoding/Types.hs.
Test Plan:
A working latin1 locale is required on the system.  Reason: ghc's
initial locale encoding defaults to ASCII, if either an unknown locale
or unknown charset is used.  For the bug to show up, ghc must start up
using the latin1 encoding.
From main directory in ghc do:
$ ./configure && make clean && make boot && make inplace/bin/ghc-stage2
$ LC_CTYPE="de_DE.ISO-8859-1" ./inplace/bin/ghc-stage2
Before the patch, the last line leads to the exception thrown:
ghc-stage2: mkTextEncoding: does not exist (unknown
encoding:ISO8859-1(checked)//TRANSLIT)
After the patch, ghc-stage2 prints a short usage summary as expected.
Moreover,
$ make test TEST=encoding005
continues to  pass after the patch.
Reviewers: austin, hvr, rwbarton, bgamari
Reviewed By: bgamari
Subscribers: mpickering, rwbarton, thomie
Differential Revision: https://phabricator.haskell.org/D2667 | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | Test Plan: Validate on 64-bit Windows
Reviewers: geekosaur, austin, erikd, Phyx
Reviewed By: Phyx
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2781 | 
| | 
| 
| 
| | The previous attempt updated the comment but not the value. Silly me. | 
| | |  | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | As described in #12865, newer Linux kernels support both MADV_FREE and
MADV_DONTNEED. Previously a runtime would fail to try MADV_DONTNEED if
MADV_FREE failed (e.g. since the kernel which the image is running on
doesn't support the latter). Now we try MADV_DONTNEED if MADV_FREE
failed to ensure that binaries compiled on a kernel supporting MADV_FREE
don't fail on decommit.
Test Plan: Validate
Reviewers: austin, erikd, simonmar
Reviewed By: simonmar
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2780
GHC Trac Issues: #12865 | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | They are not supported by Python 3.0, 3.1, and 3.2 (but are supported by
>= 3.3; silliness!)
Test Plan: Validate on python 3.2
Reviewers: austin
Subscribers: simonpj, thomie
Differential Revision: https://phabricator.haskell.org/D2778
GHC Trac Issues: #12909, #9184 | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | Test Plan: Build `p` way
Reviewers: austin, erikd, simonmar
Reviewed By: simonmar
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2779 | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | Test Plan: Read it, commit it, merge it, ship it
Reviewers: goldfire, bgamari, austin, hvr, simonpj
Reviewed By: simonpj
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2774
GHC Trac Issues: #12907 | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | Reviewers: simonpj, austin, bgamari
Reviewed By: simonpj, bgamari
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2769
GHC Trac Issues: #12901 | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | GHC uses gcc, not ld, for linking. Consequently all flags to be
interpreted by ld need to be prefixed by -optl,-Wl on the GHC command
line.
Test Plan: Validate on OpenBSD
Reviewers: austin, rwbarton
Reviewed By: rwbarton
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2776 | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| | Reviewers: austin, bgamari
Reviewed By: bgamari
Subscribers: thomie
Differential Revision: https://phabricator.haskell.org/D2772 | 
| | |  | 
| | 
| 
| 
| 
| 
| 
| | This reverts commit ef37580d16a23d1601795548b13b88b9f5373010.
Now that commit e2330b6dde805a5507898c3c4ddf38599df969da has been reverted
this change is now needed. | 
| | 
| 
| 
| 
| | This reverts commit 6f7ed1e51bf360621a3c2a447045ab3012f68575 due to breakage of
the build on Windows. | 
| | 
| 
| 
| 
| | Richard and I decided to make this change in our paper, and I'm
just propagating it to GHC | 
| | |  | 
| | |  | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | This updates the binary dists for windows to GCC 6.2.0 and
binutils 2.27.2 which has fixes required for LLVM.
Test Plan: ./validate
Reviewers: simonmar, erikd, austin, bgamari
Reviewed By: simonmar, bgamari
Subscribers: thomie, #ghc_windows_task_force
Differential Revision: https://phabricator.haskell.org/D2749
GHC Trac Issues: #12871, #8974 | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | Summary:
It turns out that Phyx's fix for #12554 (D2684) still fails with mingw-w64
python 2.7. However, Python 3 (both msys2 and mingw-w64) work fine. Given that
supporting Python 2 has already become rather tiresome (as @thomie warned it
would), let's just move to python3 by default.
Test Plan: Validate
Reviewers: austin, Phyx
Reviewed By: Phyx
Subscribers: Phyx, thomie
Differential Revision: https://phabricator.haskell.org/D2766
GHC Trac Issues: #12554 | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | Following D2684 these two tests need to be updated:
 * T7037: timeout.exe now waits until all processes are finished.
          this makes T7037 reliable. So enabled.
 * T876: Unknown reason, allocations are much lower than before.
Test Plan: ./validate
Reviewers: austin, simonmar, bgamari
Reviewed By: bgamari
Subscribers: thomie, #ghc_windows_task_force
Differential Revision: https://phabricator.haskell.org/D2759
GHC Trac Issues: #12725, #12004 | 
| | 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| | In a land far far away, a project called Cygwin was born.
Cygwin used newlib as it's standard C library implementation.
But Cygwin wanted to emulate POSIX systems as closely as possible.
So it implemented `execv` using the Windows function `spawnve`.
Specifically
```
spawnve (_P_OVERLAY, path, argv, cur_environ ())
```
`_P_OVERLAY` is crucial, as it makes the function behave *sort of*
like execv on linux. the child process replaces the original process.
With one major difference because of the difference in process models
on Windows: the original process signals the caller that it's done.
this is why the file is still locked. because it's still running,
control was returned because the parent process was destroyed,
but the child is still running.
I think it's just pure dumb luck, that the older runtimes are slow
enough to give the process time to terminate before we tried deleting
the file.  Which explains why you do have sporadic failures even on
older runtimes like 2.5.0, of a test or two (like T7307).
So this patch fixes a couple of things. I leverage the existing
`timeout.exe` to implement a workaround for this issue.
a) The old timeout used to start the process then assign it to the job.
   This is slightly faulty since child processes are only assigned to a
   job is their parent were assigned at the time they started. So this
   was a race condition. I now create the process suspended, assign it
   to the job and then resume it. Which means all child processes are
   not running under the same job.
b) First things, Is to prevent dangling child processes. I mark the job
   with `JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE` so when the last process in
   the job is done, it insures all processes under the job are killed.
c) Secondly, I change the way we wait for results. Instead of waiting
   for the parent process to terminate, I wait for the job itself to
   terminate.
   There's a slight subtlety there, we can't wait on the job itself.
   Instead we have to create an I/O Completion port and wait for signals
   on it.  See
   https://blogs.msdn.microsoft.com/oldnewthing/20130405-00/?p=4743
This fixes the issues on all runtimes for me and makes T7307 pass
consistenly.
The threading was also simplified by hiding all the locking in a single
semaphore and a completion class. Futhermore some additional error
reporting was added.
For encoding the testsuite now no longer passes a file handle to the
subprocess since on windows, sh.exe seems to acquire a lock on the file
that is not released in a timely fashion.
I suspect this because cygwin seems to emulate console handles by
creating file handles and using those for std handles. So when we give
it an existing file handle it just locks the file. I what's happening is
that it's not releasing the handle until all shared cygwin processes are
dead. Which explains why it worked in single threaded mode.
So now instead we pass a pipe and do not interpret the resulting data.
Any bytes written to stdin or read out of stdout/stderr are done so in
binary mode and we do not interpret the data. The reason for this is
that we have encoding tests in GHC which pass invalid utf-8. If we try
to handle the data as text then python will throw an exception instead
of a test comparison failing.
Also I have fixed the ability to override `PYTHON` when calling `make
tests`. This now works the same as with `.\validate`.
Finally, after cleaning up the locks I was able to make the abort
behavior work correctly as I believe it was intended: when you press
Ctrl+C and send an interrupt signal, the testsuite finishes the active
tests and then gracefully exits showing you a report of the progress it
did make. So using Ctrl+C will not just *die* as it did before.
These changes lift the restriction on which python version you use
(msys/mingw) or which runtime or python 3 or python 2.  All combinations
should now be supported.
Test Plan:
PATH=/usr/local/bin:/mingw64/bin:$APPDATA/cabal/bin:$PATH &&
PYTHON=/usr/bin/python THREADS=9 make test
THREADS=9 make test
PATH=/usr/local/bin:/mingw64/bin:$APPDATA/cabal/bin:$PATH &&
PYTHON=/usr/bin/python ./validate --quiet --testsuite-only
Reviewers: erikd, RyanGlScott, bgamari, austin
Subscribers: jrtc27, mpickering, thomie, #ghc_windows_task_force
Differential Revision: https://phabricator.haskell.org/D2684
GHC Trac Issues: #12725, #12554, #12661, #12004 |