1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
|
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<html>
<head>
<meta http-equiv="Content-Language" content="en-us">
<link rel="stylesheet" type="text/css" href="../stylesheets/style.css">
<title>Parallel Task</title>
</head>
<body>
<h2>Parallel</h2>
<h3>Description</h3>
<p>Executes nested tasks in parallel with no guarantees of thread safety. Every task will run in
its own thread, with the likelihood of concurrency problems scaling with the number of CPUs on the
host system.</p>
<p><strong>Warning</strong>: While the Apache Ant core is believed to be thread safe, no such
guarantees are made about tasks, which are not tested for thread safety during Ant's test process.
Third party tasks may or may not be thread safe, and some of Ant's core tasks, such
as <code><javac></code> are definitely not re-entrant. This is because they use libraries that
were never designed to be used in a multithreaded environment.</p>
<p>The primary use case for <code><parallel></code> is to run external programs such as an
application server, and the JUnit or TestNG test suites at the same time. Anyone trying to run large
Ant task sequences in parallel, such as <code>javadoc</code> and <code>javac</code> at the same
time, is implicitly taking on the task of identifying and fixing all concurrency bugs the tasks that
they run.</p>
<p>Accordingly, while this task has uses, it should be considered an advanced task which should be
used in certain batch processing or testing situations, rather than an easy trick to speed up build
times on a multicore CPU.</p>
<h3>Parameters</h3>
<table class="attr">
<tr>
<th>Attribute</th>
<th>Description</th>
<th>Required</th>
</tr>
<tr>
<td>threadCount</td>
<td>Maximum numbers of thread to use.</td>
<td>No</td>
</tr>
<tr>
<td>threadsPerProcessor</td>
<td>Maximum number of threads to use per available processor (Java 1.4+)</td>
<td>No; defers to <var>threadCount</var></td>
</tr>
<tr>
<td>timeout</td>
<td>Number of milliseconds before execution is terminated</td>
<td>No</td>
</tr>
<tr>
<td>failonany</td>
<td>If any of the nested tasks fails, execution of the task completes at that point without
waiting for any other tasks to complete.</td>
<td>No; default is <q>false</q>.</td>
</tr>
<tr>
<td>pollInterval</td>
<td>Currently has no effect</td>
<td>No; default is <q>1000</q></td>
</tr>
</table>
<p>Parallel tasks have a number of uses in an Ant build file including:</p>
<ul>
<li>Taking advantage of available processing resources to execute external programs
simultaneously.</li>
<li>Testing servers, where the server can be run in one thread and the test harness is run in
another thread.</li>
</ul>
<p>Any valid Ant task may be embedded within a <code>parallel</code> task, including
other <code>parallel</code> tasks, though there is no guarantee that the tasks will be thread safe
in such an environment.</p>
<p>While the tasks within the <code>parallel</code> task are being run, the main thread will be
blocked waiting for all the child threads to complete. If execution is terminated by a timeout or a
nested task failure when the <var>failonany</var> flag is set, the <code>parallel</code> task will
complete without waiting for other nested tasks to complete in other threads.</p>
<p>If any of the tasks within the <code><parallel></code> task fails and <var>failonany</var>
is not set, the remaining tasks in other threads will continue to run until all threads have
completed. In this situation, the <code>parallel</code> task will also fail.</p>
<p>The <code>parallel</code> task may be combined with the <a href="sequential.html">sequential</a>
task to define sequences of tasks to be executed on each thread within the parallel block</p>
<p>The <var>threadCount</var> attribute can be used to place a maximum number of available threads
for the execution. When not present all child tasks will be executed at once. When present then
the maximum number of concurrently executing tasks will not exceed the number of threads specified.
Furthermore, each task will be started in the order they are given. But no guarantee is made as to
the speed of execution or the order of completion of the tasks, only that each will be started
before the next.<p>
<p>If you are using Java 1.4 or later you can also use the <var>threadsPerProcessor</var> and the
number of available threads will be the started multiple of the number of processors (there is no
affinity to a particular processor, however). This will override the value
in <var>threadCount</var>. If <var>threadsPerProcessor</var> is specified on any older JVM, then
the value in <var>threadCount</var> will be used as is.</p>
<p>When using <var>threadCount</var> and <var>threadsPerProcessor</var> care should be taken to
ensure that the build does not deadlock. This can be caused by tasks such as <code>waitfor</code>
taking up all available threads before the tasks that would unlock the <code>waitfor</code> would
occur. This is not a replacement for Java Language level thread semantics and is best used for
"embarrassingly parallel" tasks.</p>
<h3>Parameters specified as nested elements</h3>
<h4>daemons</h4>
<p>The <code>parallel</code> task supports a <code><daemons></code> nested element. This is a
list of tasks which are to be run in parallel daemon threads. The parallel task will not wait for
these tasks to complete. Being daemon threads, however, they will not prevent Ant from completing,
whereupon the threads are terminated. Failures in daemon threads which occur before
the <code>parallel</code> task itself finishes will be reported and can cause <code>parallel</code>
to throw an exception. Failures which occur after <code>parallel</code> has completed are not
reported.</p>
<p>Daemon tasks can be used, for example, to start test servers which might not be easily terminated
from Ant. By using <code><daemons></code> such servers do not halt the build.</p>
<h3>Examples</h3>
<pre>
<parallel>
<wlrun ... >
<sequential>
<sleep seconds="30"/>
<junit fork="true" forkmode="once" ... >
<wlstop/>
</sequential>
</parallel></pre>
<p>This example represents a typical pattern for testing a server application. In one thread the
server is started (the <code><wlrun></code> task). The other thread consists of a three tasks
which are performed in sequence. The <code><sleep></code> task is used to give the server time
to come up. Another task which is capable of validating that the server is available could be used
in place of the <code><sleep></code> task. The <code><junit></code> test harness then
runs, again in its own JVM. Once the tests are complete, the server is stopped
(using <code><wlstop></code> in this example), allowing both threads to complete. The
<code><parallel></code> task will also complete at this time and the build will then
continue.</p>
<pre>
<parallel>
<javac fork="true"...> <!-- compiler servlet code -->
<wljspc ...> <!-- precompile JSPs -->
</parallel></pre>
<p>This example shows two independent tasks being run to achieve better resource utilization during
the build. In this instance, some servlets are being compiled in one thread and a set of JSPs is
being precompiled in another. Developers need to be careful that the two tasks are independent, both
in terms of their dependencies and in terms of their potential interactions in Ant's external
environment. Here we set <var>fork</var>=<q>true</q> for the <code><javac></code> task, so
that it runs in a new process; if the <code><wljspc></code> task used the <code>javac</code>
compiler in-VM (it may), concurrency problems may arise.</p>
<pre>
<macrodef name="dbpurge">
<attribute file="file"/>
<sequential>
<java jar="utils/dbpurge.jar" fork="true" >
<arg file="@{file}/>
</java>
</sequential>
</macrodef>
<parallel threadCount="4">
<dbpurge file="db/one"/>
<dbpurge file="db/two"/>
<dbpurge file="db/three"/>
<dbpurge file="db/four"/>
<dbpurge file="db/five"/>
<dbpurge file="db/six"/>
<dbpurge file="db/seven"/>
<dbpurge file="db/eight"/>
<!-- repeated about 40 times -->
</parallel></pre>
<p>This example represents a typical need for use of the <var>threadCount</var>
and <var>threadsPerProcessor</var> attributes. Spinning up all 40 of those tasks could cripple the
system for memory and CPU time. By limiting the number of concurrent executions you can reduce
contention for CPU, memory and disk IO, and so actually finish faster. This is also a good
candidate for use of <var>threadCount</var> (and possibly <var>threadsPerProcessor</var>) because
each task is independent (every new JVM is forked) and has no dependencies on the other tasks.</p>
</body>
</html>
|
