summaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* Adjust xz compression settingsMark Rousskov2023-02-271-12/+38
| | | | | | | | | | | | This adjusts our compression settings for xz compressed files, which should reduce download sizes by around 5-15%, at the cost of ~44% longer compression times. Given that compression happens almost universally in CI, rather than something humans are directly waiting for, this tradeoff feels worth it. If we end up seeing this be too significant an increase in CI times, moving the more aggressive compression solely to promote-release should be relatively straightforward and give most of the benefits.
* Write out manifest.in entries deterministicallyJon Gjengset2023-02-041-4/+12
| | | | | | | | | | Currently, the order of the entries in `manifest.in` depends on the iteration order of `copy_with_callback`, which in turn depends on `WalkDir`, which explicitly says iteration order is unspecified. It's possible to call `WalkDir::sort_by` to give an iteration order for each directory's entries, but it felt better to accumulate the lines and then sort them to a) make it more evident that this is happening, and b) enable the copying to be parallelized in the future.
* Remove unused importsMichael Howell2022-06-181-2/+1
|
* Bump to clap 3Michael Howell2022-06-178-271/+90
|
* tarball: allow choosing the compression formats to outputPietro Albini2020-12-102-0/+6
|
* combine: allow choosing the compression formats to outputPietro Albini2020-12-103-2/+15
|
* generate: allow choosing the compression formats to outputPietro Albini2020-12-105-8/+60
|
* combine: support combining both xz and gz tarballsPietro Albini2020-12-102-19/+43
|
* compression: support a dynamic list of output formatsPietro Albini2020-12-103-57/+114
|
* main: allow fallible conversions during CLI parsingPietro Albini2020-12-104-7/+12
|
* Migrate from failure to anyhowJosh Stone2020-04-067-54/+33
|
* Use at most 8 threads for the xz streamJosh Stone2019-08-281-2/+4
| | | | | | | | | | | | | | | At preset 6, xz2 uses about 173MB of memory per thread. This adds up quickly -- e.g. over 8GB of memory on a 48-CPU machine. If you happen to try this in a 32-bit build, you'll get `LZMA_MEM_ERROR`. We can limit this to a heuristic maximum number of threads to avoid using so much memory, like xz's [`04_compress_easy_mt` example]. // The number 8 is arbitrarily chosen and may be too low or // high depending on the compression preset and the computer // being used. [`04_compress_easy_mt` example]: https://github.com/xz-mirror/xz/blob/de1f47b2b40e960b7bc3acba754f66dd19705921/doc/examples/04_compress_easy_mt.c#L71
* Fix Windows buildAlex Crichton2019-07-042-4/+1
|
* Use a multi-threaded xz encoderAlex Crichton2019-06-261-1/+5
|
* Move from `error-chain` to `failure`Alex Crichton2019-06-267-97/+86
|
* Run `cargo fmt`Alex Crichton2019-06-268-178/+310
|
* Various cosmetic improvements.Alexander Regueiro2019-02-166-47/+48
|
* Removed copyright notices.Alexander Regueiro2019-02-167-70/+0
|
* Replace r#try! macro with ? operatorTaiki Endo2019-02-081-26/+26
|
* Remove unused import and replace deprecated functionHirokazu Hata2019-02-042-6/+1
|
* Transition to 2018 editionHirokazu Hata2019-02-048-52/+52
|
* Copy symlinksTom Tromey2018-07-131-3/+14
| | | | | lldb includes some symlinks in its install tree, and so generator must copy these symlinks as well. I believe only file symlinks are needed.
* Do not follow symlinksTom Tromey2018-07-121-15/+23
| | | | | The lldb repository has a symlink in it, and so for the tarballer to work when lldb is included, it must not follow symlinks.
* Update error-chain and don't allow warningsMateusz Mikuła2018-06-252-2/+0
|
* Fix warnings in library from error-chainMark Simulacrum2018-06-201-0/+1
|
* Permit warnings inside error-chain expansionMark Simulacrum2018-06-201-0/+1
|
* Bump winapi to 0.3Bastien Orivel2018-03-022-67/+30
|
* Turn down xz compression levelAlex Crichton2018-02-261-1/+1
| | | | | | Creating tarballs seems to be taking ridiculous amounts of time on the dist bots (upwards of 10-20 minutes) and we're not *that* small anyway so let's shave off some build time by turning down the compression level a bit.
* update to rayon 1.0Josh Stone2018-02-151-1/+1
|
* Stream the parallel xz/gz tarball generationJosh Stone2018-01-162-34/+56
| | | | | | | | | | | | | | | | | This melds the serial-`Tee` and parallel-batched approaches from before and after commit adea17e. Now we can get the same multithreaded speedup without having to build the entire uncompressed tarball in memory first. The new `impl Write for RayonTee` uses `rayon::join` to split the compression work for each buffer to separate threads. This is scoped, so it can be fully zero-copy, sharing the input buffer directly. This is all wrapped in a 1 MiB `BufWriter` to balance the cost of thread wake-ups and synchronization. The net performance is unchanged, using around 125% CPU -- approximately 4:1 time spent in xz versus gz. The overall memory use is much reduced, now independent of the tarball size -- just a few MiB on top of the fixed-cost 674 MiB compressor memory requirements of `xz -9`.
* Upgrade flate crate to 1.0.1Malo Jaffré2017-12-292-3/+3
| | | | Unblocks rust-lang/rust#46278.
* Create xz/gz files in parallelAlex Crichton2017-09-061-26/+21
|
* Use tar's append_data to add executables on WindowsJosh Stone2017-05-311-2/+21
| | | | | | | | | | | Since Windows doesn't really have Posix file modes, the tar crate only sets file modes based on whether the source is readonly. But we do want stuff like `install.sh` to be executable when extracted elsewhere. Now we create a manual header for our files, and add the executable bit on files matching whitelisted extensions (exe, dll, py, sh). That's not trying to be exhaustive or perfect, but it's hopefully good enough to have functional rust packages.
* Normalize the manifest to Unix-style path separatorsJosh Stone2017-05-201-2/+19
| | | | Fixes #65
* Revert "Move the combined components instead of copying"Josh Stone2017-05-122-10/+6
| | | | | | | | This reverts commit 4f6e020b179ef79a3e3717f80132ad1a4875a669. We need to leave the unpacked input packages intact for rustbuild to reuse in OS-specific installers on macOS and Windows. Add a test to make sure the components are still present.
* Include the root directory in tarballJosh Stone2017-05-051-1/+1
|
* Standardize File errorsJosh Stone2017-05-055-38/+46
|
* Use error-chain for more informative errorsJosh Stone2017-05-057-93/+177
|
* Move the combined components instead of copyingJosh Stone2017-05-051-4/+3
|
* Deal with remove_dir_all complicationsJosh Stone2017-05-054-2/+850
|
* Write all directories to the tarball for rustup.rs#1092Josh Stone2017-05-051-21/+20
|
* Write tarballs without an intermediate uncompressed fileJosh Stone2017-05-051-27/+32
|
* Use the xz2 crate for creating xz tarballsJosh Stone2017-05-053-84/+14
|
* Fix cfg for calling OpenOptionsExt::modeJosh Stone2017-05-051-3/+1
|
* Use the flate2 crate for creating gzip tarballsJosh Stone2017-05-052-8/+13
|
* Use the tar crate for creating tarballsJosh Stone2017-05-051-19/+15
|
* Use the flate2 and tar crates for combiningJosh Stone2017-05-052-15/+8
|
* Use a macro to reduce clap boilerplate a littleJosh Stone2017-05-051-92/+51
|
* Use a macro to reduce struct boilerplateJosh Stone2017-05-057-271/+139
|
* Rewrite combine-installers.sh in RustJosh Stone2017-05-057-78/+374
|
View Lorry Controller Status