1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
|
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
<title>greenpool – Green Thread Pools — Eventlet 0.33.0 documentation</title>
<link rel="stylesheet" type="text/css" href="../_static/pygments.css" />
<link rel="stylesheet" type="text/css" href="../_static/classic.css" />
<script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>
<script src="../_static/jquery.js"></script>
<script src="../_static/underscore.js"></script>
<script src="../_static/doctools.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="greenthread – Green Thread Implementation" href="greenthread.html" />
<link rel="prev" title="event – Cross-greenthread primitive" href="event.html" />
</head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="greenthread.html" title="greenthread – Green Thread Implementation"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="event.html" title="event – Cross-greenthread primitive"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">Eventlet 0.33.0 documentation</a> »</li>
<li class="nav-item nav-item-1"><a href="../modules.html" accesskey="U">Module Reference</a> »</li>
<li class="nav-item nav-item-this"><a href=""><code class="xref py py-mod docutils literal notranslate"><span class="pre">greenpool</span></code> – Green Thread Pools</a></li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<section id="module-eventlet.greenpool">
<span id="greenpool-green-thread-pools"></span><h1><code class="xref py py-mod docutils literal notranslate"><span class="pre">greenpool</span></code> – Green Thread Pools<a class="headerlink" href="#module-eventlet.greenpool" title="Permalink to this headline">¶</a></h1>
<dl class="py class">
<dt class="sig sig-object py" id="eventlet.greenpool.GreenPile">
<em class="property"><span class="pre">class</span> </em><span class="sig-prename descclassname"><span class="pre">eventlet.greenpool.</span></span><span class="sig-name descname"><span class="pre">GreenPile</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">size_or_pool</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">1000</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#eventlet.greenpool.GreenPile" title="Permalink to this definition">¶</a></dt>
<dd><p>GreenPile is an abstraction representing a bunch of I/O-related tasks.</p>
<p>Construct a GreenPile with an existing GreenPool object. The GreenPile will
then use that pool’s concurrency as it processes its jobs. There can be
many GreenPiles associated with a single GreenPool.</p>
<p>A GreenPile can also be constructed standalone, not associated with any
GreenPool. To do this, construct it with an integer size parameter instead
of a GreenPool.</p>
<p>It is not advisable to iterate over a GreenPile in a different greenthread
than the one which is calling spawn. The iterator will exit early in that
situation.</p>
<dl class="py method">
<dt class="sig sig-object py" id="eventlet.greenpool.GreenPile.next">
<span class="sig-name descname"><span class="pre">next</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#eventlet.greenpool.GreenPile.next" title="Permalink to this definition">¶</a></dt>
<dd><p>Wait for the next result, suspending the current greenthread until it
is available. Raises StopIteration when there are no more results.</p>
</dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="eventlet.greenpool.GreenPile.spawn">
<span class="sig-name descname"><span class="pre">spawn</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">func</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">args</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kw</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#eventlet.greenpool.GreenPile.spawn" title="Permalink to this definition">¶</a></dt>
<dd><p>Runs <em>func</em> in its own green thread, with the result available by
iterating over the GreenPile object.</p>
</dd></dl>
</dd></dl>
<dl class="py class">
<dt class="sig sig-object py" id="eventlet.greenpool.GreenPool">
<em class="property"><span class="pre">class</span> </em><span class="sig-prename descclassname"><span class="pre">eventlet.greenpool.</span></span><span class="sig-name descname"><span class="pre">GreenPool</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">size</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">1000</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#eventlet.greenpool.GreenPool" title="Permalink to this definition">¶</a></dt>
<dd><p>The GreenPool class is a pool of green threads.</p>
<dl class="py method">
<dt class="sig sig-object py" id="eventlet.greenpool.GreenPool.free">
<span class="sig-name descname"><span class="pre">free</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#eventlet.greenpool.GreenPool.free" title="Permalink to this definition">¶</a></dt>
<dd><p>Returns the number of greenthreads available for use.</p>
<p>If zero or less, the next call to <a class="reference internal" href="#eventlet.greenpool.GreenPool.spawn" title="eventlet.greenpool.GreenPool.spawn"><code class="xref py py-meth docutils literal notranslate"><span class="pre">spawn()</span></code></a> or <a class="reference internal" href="#eventlet.greenpool.GreenPool.spawn_n" title="eventlet.greenpool.GreenPool.spawn_n"><code class="xref py py-meth docutils literal notranslate"><span class="pre">spawn_n()</span></code></a> will
block the calling greenthread until a slot becomes available.</p>
</dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="eventlet.greenpool.GreenPool.imap">
<span class="sig-name descname"><span class="pre">imap</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">function</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">iterables</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#eventlet.greenpool.GreenPool.imap" title="Permalink to this definition">¶</a></dt>
<dd><p>This is the same as <code class="xref py py-func docutils literal notranslate"><span class="pre">itertools.imap()</span></code>, and has the same
concurrency and memory behavior as <a class="reference internal" href="#eventlet.greenpool.GreenPool.starmap" title="eventlet.greenpool.GreenPool.starmap"><code class="xref py py-meth docutils literal notranslate"><span class="pre">starmap()</span></code></a>.</p>
<p>It’s quite convenient for, e.g., farming out jobs from a file:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">worker</span><span class="p">(</span><span class="n">line</span><span class="p">):</span>
<span class="k">return</span> <span class="n">do_something</span><span class="p">(</span><span class="n">line</span><span class="p">)</span>
<span class="n">pool</span> <span class="o">=</span> <span class="n">GreenPool</span><span class="p">()</span>
<span class="k">for</span> <span class="n">result</span> <span class="ow">in</span> <span class="n">pool</span><span class="o">.</span><span class="n">imap</span><span class="p">(</span><span class="n">worker</span><span class="p">,</span> <span class="nb">open</span><span class="p">(</span><span class="s2">"filename"</span><span class="p">,</span> <span class="s1">'r'</span><span class="p">)):</span>
<span class="nb">print</span><span class="p">(</span><span class="n">result</span><span class="p">)</span>
</pre></div>
</div>
</dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="eventlet.greenpool.GreenPool.resize">
<span class="sig-name descname"><span class="pre">resize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">new_size</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#eventlet.greenpool.GreenPool.resize" title="Permalink to this definition">¶</a></dt>
<dd><p>Change the max number of greenthreads doing work at any given time.</p>
<p>If resize is called when there are more than <em>new_size</em> greenthreads
already working on tasks, they will be allowed to complete but no new
tasks will be allowed to get launched until enough greenthreads finish
their tasks to drop the overall quantity below <em>new_size</em>. Until
then, the return value of free() will be negative.</p>
</dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="eventlet.greenpool.GreenPool.running">
<span class="sig-name descname"><span class="pre">running</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#eventlet.greenpool.GreenPool.running" title="Permalink to this definition">¶</a></dt>
<dd><p>Returns the number of greenthreads that are currently executing
functions in the GreenPool.</p>
</dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="eventlet.greenpool.GreenPool.spawn">
<span class="sig-name descname"><span class="pre">spawn</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">function</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">args</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#eventlet.greenpool.GreenPool.spawn" title="Permalink to this definition">¶</a></dt>
<dd><p>Run the <em>function</em> with its arguments in its own green thread.
Returns the <code class="xref py py-class docutils literal notranslate"><span class="pre">GreenThread</span></code>
object that is running the function, which can be used to retrieve the
results.</p>
<p>If the pool is currently at capacity, <code class="docutils literal notranslate"><span class="pre">spawn</span></code> will block until one of
the running greenthreads completes its task and frees up a slot.</p>
<p>This function is reentrant; <em>function</em> can call <code class="docutils literal notranslate"><span class="pre">spawn</span></code> on the same
pool without risk of deadlocking the whole thing.</p>
</dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="eventlet.greenpool.GreenPool.spawn_n">
<span class="sig-name descname"><span class="pre">spawn_n</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">function</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">args</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n"><span class="pre">kwargs</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#eventlet.greenpool.GreenPool.spawn_n" title="Permalink to this definition">¶</a></dt>
<dd><p>Create a greenthread to run the <em>function</em>, the same as
<a class="reference internal" href="#eventlet.greenpool.GreenPool.spawn" title="eventlet.greenpool.GreenPool.spawn"><code class="xref py py-meth docutils literal notranslate"><span class="pre">spawn()</span></code></a>. The difference is that <a class="reference internal" href="#eventlet.greenpool.GreenPool.spawn_n" title="eventlet.greenpool.GreenPool.spawn_n"><code class="xref py py-meth docutils literal notranslate"><span class="pre">spawn_n()</span></code></a> returns
None; the results of <em>function</em> are not retrievable.</p>
</dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="eventlet.greenpool.GreenPool.starmap">
<span class="sig-name descname"><span class="pre">starmap</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">function</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">iterable</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#eventlet.greenpool.GreenPool.starmap" title="Permalink to this definition">¶</a></dt>
<dd><p>This is the same as <a class="reference external" href="https://docs.python.org/3/library/itertools.html#itertools.starmap" title="(in Python v3.10)"><code class="xref py py-func docutils literal notranslate"><span class="pre">itertools.starmap()</span></code></a>, except that <em>func</em> is
executed in a separate green thread for each item, with the concurrency
limited by the pool’s size. In operation, starmap consumes a constant
amount of memory, proportional to the size of the pool, and is thus
suited for iterating over extremely long input lists.</p>
</dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="eventlet.greenpool.GreenPool.waitall">
<span class="sig-name descname"><span class="pre">waitall</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#eventlet.greenpool.GreenPool.waitall" title="Permalink to this definition">¶</a></dt>
<dd><p>Waits until all greenthreads in the pool are finished working.</p>
</dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="eventlet.greenpool.GreenPool.waiting">
<span class="sig-name descname"><span class="pre">waiting</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#eventlet.greenpool.GreenPool.waiting" title="Permalink to this definition">¶</a></dt>
<dd><p>Return the number of greenthreads waiting to spawn.</p>
</dd></dl>
</dd></dl>
</section>
<div class="clearer"></div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h4>Previous topic</h4>
<p class="topless"><a href="event.html"
title="previous chapter"><code class="xref py py-mod docutils literal notranslate"><span class="pre">event</span></code> – Cross-greenthread primitive</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="greenthread.html"
title="next chapter"><code class="xref py py-mod docutils literal notranslate"><span class="pre">greenthread</span></code> – Green Thread Implementation</a></p>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../_sources/modules/greenpool.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
<input type="submit" value="Go" />
</form>
</div>
</div>
<script>$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="greenthread.html" title="greenthread – Green Thread Implementation"
>next</a> |</li>
<li class="right" >
<a href="event.html" title="event – Cross-greenthread primitive"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">Eventlet 0.33.0 documentation</a> »</li>
<li class="nav-item nav-item-1"><a href="../modules.html" >Module Reference</a> »</li>
<li class="nav-item nav-item-this"><a href=""><code class="xref py py-mod docutils literal notranslate"><span class="pre">greenpool</span></code> – Green Thread Pools</a></li>
</ul>
</div>
<div class="footer" role="contentinfo">
© Copyright 2005-2010, Eventlet Contributors.
Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 4.1.2.
</div>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-42952223-1', 'eventlet.net');
ga('send', 'pageview');
</script>
</body>
</html>
|
