| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
| |
It was just an unnecessary indirection in the end.
Change-Id: I956ed4858dcc2b528be1e1fce9ab24862b99ff62
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
|
|
|
|
|
|
|
| |
... into SetupProjectParameters, where it belongs.
Change-Id: I5cae2842200a053827739b947d1cd06e1bbe5ef9
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
|
|
|
|
|
| |
Change-Id: Ia4a8c144431c413701707c024185ca54f49b0829
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
|
|
|
|
|
|
|
|
| |
Project loading functionality is implemented in various source files
these days, so it makes sense to group them together.
Change-Id: Iba42b0246c40610d2a03bf6cc7ed7d3bec9d5536
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
===================
Problem description
===================
The ModuleLoader class as it exists before this patch has a number of
serious problems:
- It instantiates modules over and over again everywhere a
Depends item appears. The different instances are later
merged in a hopelessly obscure and error-prone way.
- It seriously pollutes the module scopes so that sloppily written
project files appear to work even though they shouldn't.
- Dependencies between products are handled twice: Once using the
normal module instantiation code (with the Export item
acting like a Module item), and also with a parallel mechanism
that does strange, seemingly redundant things especially regarding
transitive dependencies, which appear to introduce enormous
run-time overhead. It is also split up between ModuleLoader and
ProjectResolver, adding even more confusion.
- The code is messy, seriously under-documented and hardly
understood even by its original authors. It presents a huge obstacle
to potential contributors.
=================
Patch description
=================
- Each module is instantiated once per product. Property values are
merged on the fly. Special handling for dependencies between
products are kept to the absolutely required minimum.
- There are no more extra passes for establishing inter-product
dependencies. Instead, whenever an unhandled dependency is
encountered, processing the current product is paused and resumed
once the dependency is ready, with the product state getting saved
and restored in between so no work is done twice.
- The ModuleLoader class now really only locates and loads modules.
The new main class is called ProjectTreeBuilder, and we have split
off small, self-contained pieces wherever possible. This process
will be continued in follow-up patches (see next section).
=======
Outlook
=======
The ProjectTreeBuilder ist still too large and should be split up
further into small, easily digestible parts with clear responsibilities,
until the top-level code becomes tidy and self-documenting. In the end,
it can probably be merged with ProjectResolver and Loader.
However, this first requires the tight coupling between
ProjectTreeBuilder/ModuleProviderLoader/ProbesResolver/ProjectResolver
to be broken; otherwise we can't produce clean interfaces. As this would
involve touching a lot of otherwise unrelated code, it is out of scope
for this patch.
=================
Benchmarking data
=================
We first present wall-time clock results gathered by running
"qbs resolve --log-time" for qbs itself and Qt Creator on macOS
and Windows. The numbers are the average of several runs, with
outliers removed.
Then the same for a simple QML project using a static Qt on Linux
(this case is special because our current way of handling plugins
causes a huge amount of modules to be loaded).
Finally, we show the output of the qbs_benchmarker tool for
resolving qbs and Qt Creator on Linux.
The data shows a speed-up that is hardly noticeable for simple
projects, but increases sharply with project complexity.
This suggests that our new way of resolving does not suffer
anymore from the non-linear slowdown when the number
of dependencies gets large.
Resolving qbs on Windows:
Before this patch: ModuleLoader 3.6s, ProjectResolver 870ms
With this patch: ProjectTreeBuilder 3.6s, ProjectResolver 840ms
Resolving Qt Creator on Windows:
Before this patch: ModuleLoader 17s, ProjectResolver 6.8s
With this patch: ProjectTreeBuilder 10.0s, ProjectResolver 6.5s
Resolving qbs on macOS:
Before this patch: ModuleLoader 4.0s, ProjectResolver 2.3s
With this patch: ProjectTreeBuilder 4.0s, ProjectResolver 2.3s
Resolving Qt Creator on macOS:
Before this patch: ModuleLoader 32.0s, ProjectResolver 15.6s
With this patch: ProjectTreeBuilder 23.0s, ProjectResolver 15.3s
Note that the above numbers are for an initial resolve, so they include
the time for running Probes. The speed-up for re-resolving (with cached
Probes) is even higher, in particular on macOS, where Probes take
excessively long.
Resolving with static Qt on Linux (QBS-1521):
Before this patch: ModuleLoader 36s, ProjectResolver 18s
With this patch: ProjectTreeBuilder 1.5s, ProjectResolver 14s
Output of qbs_benchmarker for resolving qbs on Linux:
Old instruction count: 10029744668
New instruction count: 9079802661
Relative change: -10 %
Old peak memory usage: 69881840 Bytes
New peak memory usage: 82434624 Bytes
Relative change: +17 %
Output of qbs_benchmarker for resolving Qt Creator on Linux:
Old instruction count: 87364681688
New instruction count: 53634332869
Relative change: -39 %
Old peak memory usage: 578458840 Bytes
New peak memory usage: 567271960 Bytes
Relative change: -2 %
I don't know where the increased memory footprint for a small project
comes from, but since it goes away for larger projects, it doesn't seem
worth investigating.
Fixes: QBS-1037
Task-number: QBS-1521
Change-Id: Ieeebce8a7ff68cdffc15d645e2342ece2426fa94
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
|
|
|
|
|
|
|
|
|
|
|
| |
As opposed to QtScript, such values cannot be re-used in QuickJS. This
was overlooked in 087c22e17721f37490dd2048a567b6a58065d939.
I assume this is the culprit for seemingly random crashes and "not a
function" messages we have been seeing sometimes when building with Qt
Creator.
Change-Id: Ia4e7aed0cda97439ac75db5ecfbf08ff096a02b2
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Newer clang versions seem to expose serious bugs in QtScript, whose
complexity makes it difficult to track them down.
We therefore switch to the more light-weight QuickJS, which offers all
the features we need (most notably property access interception), as
well as good performance.
To save some porting effort, we removed the long-deprecated loadFile()
and loadExtension() functions.
During the porting procedure, we noticed and fixed thread safety issues
in artifact access from JS commands.
We consider this change important enough to bump the major version, so
the next release will be 2.0.
Detailed benchmarking data is below. In summary, we see a modest speed-
up at the cost of a similarly modest increase in memory consumption
(with the exception of project resolving on macOS, which has become a
bit slower). Importantly, the increase does not rise with project size,
as the comparison of qbs vs Qt Creator shows.
Output of qbs_benchmarker on Linux with qbs as test project:
========== Performance data for Resolving ==========
Old instruction count: 12870602895
New instruction count: 11923459780
Relative change: -8 %
Old peak memory usage: 61775848 Bytes
New peak memory usage: 67583424 Bytes
Relative change: +9 %
========== Performance data for Rule Execution ==========
Old instruction count: 4074062223
New instruction count: 3887473574
Relative change: -5 %
Old peak memory usage: 35123704 Bytes
New peak memory usage: 38398392 Bytes
Relative change: +9 %
========== Performance data for Null Build ==========
Old instruction count: 1104417596
New instruction count: 1011033948
Relative change: -9 %
Old peak memory usage: 24461824 Bytes
New peak memory usage: 25325920 Bytes
Relative change: +3 %
Output of qbs_benchmarker on Linux with Qt Creator as test project:
========== Performance data for Resolving ==========
Old instruction count: 67166450352
New instruction count: 60772791018
Relative change: -10 %
Old peak memory usage: 327011616 Bytes
New peak memory usage: 343724176 Bytes
Relative change: +5 %
========== Performance data for Rule Execution ==========
Old instruction count: 71684351183
New instruction count: 67051936965
Relative change: -7 %
Old peak memory usage: 374913688 Bytes
New peak memory usage: 387790992 Bytes
Relative change: +3 %
========== Performance data for Null Build ==========
Old instruction count: 8383156078
New instruction count: 7930705668
Relative change: -6 %
Old peak memory usage: 180468360 Bytes
New peak memory usage: 182490384 Bytes
Relative change: +1 %
Real-world data building Qt Creator (using qbs --log-time, several runs,
removing outliers):
macOS:
Resolving: 43s -> 47s
Rule execution: 17s -> 14s
Windows:
Resolving: 18s -> 16s
Rule execution: 22s -> 17s
Fixes: QBS-913
Fixes: QBS-1103
Fixes: QBS-1126
Fixes: QBS-1227
Fixes: QBS-1684
Change-Id: Ie5088155026e85bbd1e303f1c67addb15810a3cb
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
Reviewed-by: Orgad Shaneh <orgads@gmail.com>
|
|
|
|
|
|
|
|
|
|
| |
We have been supported CMake build for quite some time
so users should have plenty of time to adapt.
Ubuntu, Brew and macports also use CMake for building QBS.
Change-Id: Ib78177f4a7ca8cdea1a2f3a8eac8bfe804674f32
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
| |
Amends d8d7beb866b24793d9c04b6996276a4a8959bfa2.
Change-Id: I6eefff5640ad1213e2ae36d7f71ba476ea4d475b
Reviewed-by: Christian Stenger <christian.stenger@qt.io>
|
|
|
|
|
| |
Change-Id: I0fea777445be769080b5e5534eb8dd05cf4652be
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
| |
Change-Id: I3ca9909c4a9061132c245985c4fd738fb7bc3009
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|\
| |
| |
| |
| |
| |
| | |
It seems we completely forgot to merge 1.21 into master, losing at least
some of the bugfixes in 1.21.1.
Change-Id: I4e8e32a861da817d9b686991933d6a32332534c5
|
| |
| |
| |
| |
| |
| |
| |
| | |
It's unclear why that code was ever there.
Fixes: QBS-1686
Change-Id: I893216c9b27afeec8913f59e55f475d58964eb92
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
|
| |
| |
| |
| |
| |
| |
| | |
... to unify and simplify a code.
Change-Id: Ia493c5f428c1da4df3c8623e448be8ac72fd4f44
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
| |
| |
| |
| |
| | |
Change-Id: Ia267b9419227db5f4382a33aa99eb7dbe0ada45f
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
|
| |
| |
| |
| |
| | |
Change-Id: I11dc7ce71c7ce7f161d75e3fcd2fc34c87e7733e
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
| |
| |
| |
| |
| | |
Change-Id: Ic17ce136f1438dbdc4cf1d5c88947d004748fd70
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
| |
| |
| |
| |
| | |
Change-Id: Ia5c14b86e5e0952dbebf103c5355f49e41158cba
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
| |
| |
| |
| |
| | |
Change-Id: I785b2c065dbd3daf4ae74f1e7a39e6be61489467
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
| |
| |
| |
| |
| | |
Change-Id: I63255548d9191831324a84dfce9e9296585dba9b
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
| |
| |
| |
| |
| | |
Change-Id: Ie62da8d2a47fe9218d2246216e9affb5fe79374f
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
| |
| |
| |
| |
| | |
Change-Id: Ie424e43f328b6a5954efc8dfff3674c95e637600
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
| |
| |
| |
| |
| |
| |
| |
| | |
This patch fixes the "signal error is overloaded [clazy-overloaded-signal]"
warnings.
Change-Id: Ibf822da6eb34c781696ea5e0839afd8cc408af74
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
| |
| |
| |
| |
| |
| |
| |
| | |
This patch fixes the "Use multi-arg instead [clazy-qstring-arg]"
warnings.
Change-Id: I205c94d33574b6f7af486b3654dd2d8591746f6c
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
|
|/
|
|
|
| |
Change-Id: I89f510619196cc01a9e3b0c2273888b12b188928
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
| |
... and replace them with the new template rangeTo method
Also, add efficient Set(Iter, Iter) ctor.
Change-Id: I5a2345ca84373692e3ba815e5a8f38cb4cfc4308
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
| |
Required for ongoing patchsets - later patches will add more members.
Change-Id: Iad57e17139b104e55221059eedb8c0f057f6f4e0
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
| |
The user might not remember which properties were originally passed on
the command line, so show them if they don't match the new ones.
Change-Id: I41fbfc84832c0bf56a950a6b546bccca50c88564
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
|
|
|
|
|
| |
Change-Id: I0dad1ee403176a665e4e9dead2f94dd032e1d75b
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
| |
...and use QVariant::userType() instead since type() method was
deprecated (variant.metaType().id() or variant.typeId() or
variant.userType() should be user instead).
Change-Id: I00b93c5e010a0c7b733b3c6dfeae2d744be32ce1
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
| |
The type should match the one used in load() since in qt6 the type is
not int anymore
Change-Id: I0e7e402dec6f059279de933d5afd0c450cde8ddd
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
| |
Otherwise, Qt complains when emitting the signal
Change-Id: I1840eff8c041ca21475658cb3113e81298b3a8a6
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
| |
QStringRef does not exist anymore.
Qml parser is not ported yet.
Change-Id: Ifb99a529b69c0085363031e852ffd7576f1d93be
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
| |
Change-Id: I9220b1e6344e90401abf4fb91115e77a3a6c996f
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
| |
Change-Id: Ia38f08fb32b50761e7fb8642019aad39a25c1ef2
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
| |
MSVC complains about inheriting from std::iterator and suggests
declaring typedefs manually
Change-Id: I19cf1d9efd86fccb53827235d3659ef571d4f8b5
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
| |
It qt6, containers use qsizetype for the size instead of int, so
compiler cannot deduce the correct type.
Change-Id: Ifcade8926679389389f52610aed0d4561aa1287b
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
| |
Change-Id: I1d6968de823c43e42ca53eb68972ba5e69dc29ed
Reviewed-by: Ivan Komissarov <ABBAPOH@gmail.com>
|
|
|
|
|
|
|
| |
The warning says 'c++11 range-loop might detach Qt container'
Change-Id: I6b6a91d6b3298702246da94f05d21623d5f13faa
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
|
|
| |
Also, fix undefined behavior when setting and reading different field of
a union (Lexer, Token) - according to the C++ Standard, it is not
allowed to use a union to zero members of a struct.
Treat these warnings as errors now.
Change-Id: I0f6d071217ef55e2c75c51138fcff47048eca62f
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
| |
Also, treat those as errors
Change-Id: I0771aa656273fd0a01c7787870d9de9b4c631823
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
| |
Also, treat those as errors
Change-Id: I33e5adef456b2c3445e00297d48770d99781aedd
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Dependency scanning can generate inverse dependencies. For instance, a
Qt C++ file "myobject.cpp" containing the Q_OBJECT macro will typically
include a file "myobject.moc", which is generated from the cpp file.
When scanning the inputs of "myobject.moc", the artifact will encounter
itself as a scan dependency. There is code that prevents a dependency
cycle in this case.
Let's now consider the case where such a rule has a second output
artifact: When scanning the inputs of that artifact, we will encounter
the first artifact as a scan dependency. This case was not handled,
leading to an infinite loop in rule execution. We fix this by rejecting
scan dependencies that have the same transformer as the artifact whose
inputs are being scanned.
Change-Id: I767ceaad387c2e315738070e01ef9322764afb65
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If scanning finds an artifact in more than one product, we so far
randomly used one of them. We now prefer artifacts from product
dependencies, because the other ones might get lost during change
tracking, as we don't keep an "is scan result of" type of information in
our RescuableArtifactData.
This should fix the linked bug in the vast majority of actual projects.
It's also conceptually the right thing to do, because the artifact in
the product dependency is much more likely to be the one that was
intended to be found.
Task-number: QBS-1532
Change-Id: Idd8662bb00570bc57f5861ce83517f0adb845b49
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
|
|
|
|
|
|
|
|
|
| |
Also, treat this warning as error
This amends 936eee744db9b2fd1b9b66d168d80a113002aad2
Change-Id: I372c3b11baa08375d581052282f4546b2ba2e057
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
To be able to use cpp.driverFlags and cpp.driverLinkerFlags with clang-
cl. This patchset makes possible to use clang-cl with "-
fsanitize=address" flag without passing the sanitizer libraries manually
to the linker
There's also a behavior change in which linker is used - clang-cl uses
native linker by default. Old behavior can be restored by setting
cpp.linkerVariant to "lld"
Fixes: QBS-1522
Change-Id: I9528ce40aa5fdfab987672b15fffd830fa2d6376
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
| |
QMap::keys() method allocates memory for a new list which is not desired
during the object destruction. Also simplifies dtor and make code more
exception-safe
Change-Id: Ib76c2426e74ca79e2ab56b993007985ba8999c85
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
|
| |
Use more pass-by-const-ref since there is no point in passing objects by
value in these cases
Also, treat this warning as error
Change-Id: I5382dcd56639f535cf240205cc5c3ac667c4ece3
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
|
|
|
|
|
| |
std::vector was copied way too much, it's better to use copy-on-write
here
This amends de2fea7734
Change-Id: I51afddc88ec6862a1a623b0c9a69c810e9e15a55
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|
|
|
|
|
| |
Change-Id: I6680a2a3ea711b8a7fa01fc1edb8ce1040f603b3
Reviewed-by: Christian Kandeler <christian.kandeler@qt.io>
|