1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<meta name="author" content="Philip Semanchuk">
<meta name="copyright" content="All contents © 2012 Philip Semanchuk">
<meta name="keywords" content="python posix ipc semaphore shared memory message queue">
<title>The posix_ipc Module for POSIX IPC Under Python -- Version History</title>
<style type="text/css">
li {
margin-top: .67em;
margin-bottom: .67em;
}
pre { margin-left: 2em; }
</style>
</head>
<body>
<h2>Version History</h2>
<p>This is the version history for the
<a href="http://semanchuk.com/philip/posix_ipc/">posix_ipc
module</a>.</p>
<ul id="history">
<li><strong><span id="current">Current</span> – 0.9.8 (20 Feb 2014) –</strong>
<p>As with 0.9.7, there are no code or feature changes in this version.
This version merely corrects a documentation error.
</p>
<p>This version comes with a big wish for peace in Ukraine. Мир!</p>
</li>
<li>0.9.7 (20 Feb 2014) –
<p>There are no code or feature changes in this version. The bump in
version number reflects that this is the first version
also available on PyPI.
</p>
<p>This version comes with a big wish for peace in Ukraine. Мир!</p>
</li>
<li>0.9.6 (23 Oct 2013) –
<p>Fixed two BSD-specific bugs introduced in version 0.9.5
that occurred if the kernel module <tt>mqueuefs</tt> wasn't
loaded at install time. Specifically --
</p>
<ul>
<li>The installer
would print a harmless but distracting error message from sysctl.
(This also affected OS X which is FreeBSD-ish.)
</li>
<li><tt>posix_ipc</tt> would build with an inappropriate
value for <tt>QUEUE_MESSAGES_MAX_DEFAULT</tt>.
Subsequent attempts to create a message queue would fail unless the
caller set the <tt>max_messages</tt> param to an appropriate
value. (This didn't affect OS X since OS X doesn't support message
queues at all.)
</li>
</ul>
<p>Also, rewrote the message queue thread notification code to address
the old bug (<tt>Fatal Python error: PyEval_AcquireLock: current thread state is NULL</tt>)
that appeared during release testing for 0.9.5 and which
has plagued me on and off since I wrote this code. The new code uses
<a href="http://docs.python.org/2/c-api/init.html#non-python-created-threads">the
algorithm recommended in the Python documentation</a> which may have
been flaky when I started using it in Python 2.4. It seems stable now
under Python 2.6+/3.
</p>
</li>
<li>0.9.5 (14 Oct 2013) –
<ul>
<li>Added the ability to use Semaphores in context managers.
Thanks to Matt Ruffalo for the suggestion and patch.
</li>
<li>Fixed a big under FreeBSD 9.x where I used overly ambitious
values for some message queue constants at build time. Now,
<tt>posix_ipc</tt> asks <tt>sysctl</tt> for the correct values.
<i>Köszönöm</i> to Attila Nagy for the bug report.
</li>
</ul>
</li>
<li>0.9.4 (2 Sept 2012) –
<p>Fixed a buglet. When creating shared memory under Linux and
specifying both a size and the read-only flag, creating the memory
would succeed but calling <tt>ftruncate()</tt> would fail.
The failure to change the size was correctly reported
but <tt>posix_ipc</tt> failed to clean up the shared memory segment
it had created. That's now fixed. Thanks to Kevin Miles for the bug
report.
</p>
</li>
<li>0.9.3 (2 Jan 2012) –
<p>Added a bugfix/feature to raise an error (rather than segault)
when trying to use a closed semaphore.
Thanks to Russel for the suggestion and patch.
</p>
</li>
<li>0.9.2 (6 Nov 2011) –
<ul>
<li>Fixed a bug where timeouts in <tt>Semaphore.acquire()</tt>,
<tt>MessageQueue.send()</tt> and <tt>MessageQueue.receive()</tt>
were only accurate to about one second due to use of the C call
<tt>time()</tt>. Switching to <tt>gettimeofday()</tt> fixes
the problem. Thanks to Douglas Young for the bug report and
patch.
</li>
<li>Fixed a bug in <tt>prober.py</tt> that caused install to fail
under Ubuntu 11.10. <tt>prober.py</tt> specified link options
in the wrong order, and so linking one of the test
applications that's built during setup was failing. Thanks
to Kevin Miles for the bug report.
</li>
<li>Added a check in <tt>prober.py</tt> to see if
<tt>sysconf_names</tt> exists in the <tt>os</tt> module. It
doesn't exist under Cygwin, and this code caused an error
on that platform. Thanks to Rizwan Raza for the bug report.
</li>
</ul>
</li>
<li>0.9.1 (7 Apr 2011) –
<ul>
<li>Fixed (?) a bug in message queue thread notification that caused
<tt>ceval: tstate mix-up</tt> and other fun messages. Thanks to
Lev Maximov for the bug report.
</li>
<li>Added the <tt>demo3</tt> directory with demos of message queue.
This was supposed be included in version 0.9.0 but I accidentally
left it out. (Also reported by Lev.)
</li>
</ul>
</li>
<li>0.9.0 (31 Dec 2010) –
<p>Added the <tt>demo3</tt> directory with demos of message queue
notification techniques. Also, fixed two bugs related to message
queue notification. Big thanks to
Philip D. Bober for debugging and providing a patch to the
most difficult part of the code. The bugs were –
</p>
<ul>
<li>First, the series of calls to set up the Python thread in
<tt>process_notification()</tt> were simply wrong. They worked
some (most?) of the time but would segfault eventually because
I was creating a Python thread state when I should not have.
</li>
<li>Second, the code in <tt>process_notification()</tt> failed
to consider that the user's callback might re-request
notification, thus overwriting pointers that I would later
decref. <tt>process_notification()</tt> is now thread-safe.
</li>
</ul>
</li>
<li>0.8.1 (15 Mar 2010) –
<p>Fixed a sloppy declaration that caused a compile error under
Cygwin 1.7.1. Thanks to Jill McCutcheon for the bug report.
</p>
</li>
<li>0.8.0 (2 Mar 2010) –
<ul>
<li>Fixed message queue support detection in FreeBSD and
the platform-specific documentation about FreeBSD.
</li>
<li>Rearranged the documentation and split the history
(which you're reading now) into a separate file.
</li>
<li>I fixed two small bugs related to the confusing
message queue constants. The bugs and associated changes are
explained below. The explanation is really long not
because the changes were big (they weren't), but because
they and rationale behind them are subtle.
<p>Fixing these bugs was made easier by this realization:
on all of the systems to which I have access that implement
message queues (FreeBSD, OpenSolaris, Linux, and Windows +
Cygwin), all except Linux implement them as
memory-mapped files or something similar. On these
non-Linux systems, the
maximum queue message count and size are pretty darn big
(<tt>LONG_MAX</tt>). Therefore, only on Linux is anyone likely to
encounter limits to message queue size and content.
</p>
<p>The first bug I fixed was related to four message queue
constants mentioned in <tt>posix_ipc</tt> documentation:
<tt>QUEUE_MESSAGES_MAX</tt>,
<tt>QUEUE_MESSAGES_MAX_DEFAULT</tt>,
<tt>QUEUE_MESSAGE_SIZE_MAX</tt> and
<tt>QUEUE_MESSAGE_SIZE_MAX_DEFAULT</tt>. All four were defined
in the <tt>C</tt>
code, but the two <tt>XXX_DEFAULT</tt> constants weren't exposed on
the Python side.
</p>
<p>The second bug was that under Linux, <tt>QUEUE_MESSAGES_MAX</tt> and
<tt>QUEUE_MESSAGE_SIZE_MAX</tt> were permanently fixed to their
values at <tt>posix_ipc</tt>'s compile/install time even if the
relevant system values changed later. Thanks to Kyle Tippetts
for bringing this to my attention.
</p>
<p><tt>QUEUE_MESSAGES_MAX_DEFAULT</tt> was arbitrarily limited to
(at most) 1024. This wasn't a bug, just a bad choice.
</p>
<p>I made a few changes in order to fix these problems –</p>
<ol>
<li>The constants <tt>QUEUE_MESSAGES_MAX</tt> and
<tt>QUEUE_MESSAGE_SIZE_MAX</tt>
<strong>have been deleted</strong> since they were only sure to
be accurate on systems where they were irrelevant. Furthermore,
Linux (the only place where they matter) exposes these values
through the file system (in
<tt>/proc/sys/fs/mqueue/msg_max</tt> and
<tt>/proc/sys/fs/mqueue/msgsize_max</tt> respectively) so Python
apps that need them can read them without any help
from <tt>posix_ipc</tt>.
</li>
<li><tt>QUEUE_MESSAGES_MAX_DEFAULT</tt> and
<tt>QUEUE_MESSAGE_SIZE_MAX_DEFAULT</tt> are now exposed to
Python as they should have been all along.
<tt>QUEUE_MESSAGES_MAX_DEFAULT</tt> is now set to
<tt>LONG_MAX</tt> on all platforms except Linux, where
it's set at compile time from <tt>/proc/sys/fs/mqueue/msg_max</tt>.
</li>
<li><tt>QUEUE_MESSAGE_SIZE_MAX_DEFAULT</tt> remains at the fairly
arbitrary value of 8k. It's not a good idea to make it too big
since a buffer of this size is allocated every time
<tt>MessageQueue.receive()</tt> is called. Under Linux, I
check the contents of <tt>/proc/sys/fs/mqueue/msgsize_max</tt>
and make <tt>QUEUE_MESSAGE_SIZE_MAX_DEFAULT</tt> smaller if
necessary.
</li>
</ol>
</li>
</ul>
</li>
<li>0.7.0 (21 Feb 2010) –
<p>Added Python 3.1 support.</p>
</li>
<li>0.6.3 (15 Feb 2009) –
<ul>
<li>Fixed a bug where creating an IPC object with invalid
parameters would correctly raise a <tt>ValueError</tt>, but
with a message that may or may not have correctly identified
the cause. (My code was making an educated guess that was
sometimes wrong.)
<p>As of this version, if initialization of an IPC object
fails with the error code <tt>EINVAL</tt>,
<tt>posix_ipc</tt> raises a <tt>ValueError</tt>
with the vague-but-correct message "Invalid parameter(s)".
</p>
</li>
<li>Cleaned up the code a little internally.</li>
</ul>
</li>
<li>0.6.2 (30 Dec 2009) –
<p>Fixed a bug where a <tt>MessageQueue</tt>'s <tt>mode</tt>
attribute returned garbage. <i>Grazie</i> to Stefano Debenedetti for
the bug report.
</p>
</li>
<li>0.6.1 (29 Nov 2009) –
<p>There were no functional changes to the module in this version, but
I added the convenience function <tt>close_fd()</tt> and fixed
some docmentation and demo bugs/sloppiness.
</p>
<ul>
<li>Added the convenience function <tt>SharedMemory.close_fd()</tt>.
Thanks to Kyle Tippetts for pointing out the usefulness
of this.
</li>
<li>Added the module attributes <tt>__version__</tt>,
<tt>__copyright__</tt>, <tt>__author__</tt> and
<tt>__license__</tt>.
</li>
<li>Fixed the license info embedded in <tt>posix_ipc_module.c</tt>
which was still referring to GPL.
<li>Replaced <tt>file()</tt> in <tt>setup.py</tt> with
<tt>open()</tt>/<tt>close()</tt>.</li>
<li>Demo changes –
<ul>
<li>Made the demo a bit faster, especially for large
shared memory chunks. Thanks to Andrew Trevorrow
for the suggestion and patch.
</li>
<li>Fixed a bug in premise.c; it wasn't closing the semaphore.</li>
<li>Fixed a bug in premise.py; it wasn't closing the
shared memory's file descriptor.
</li>
<li>Fixed bugs in conclusion.py; it wasn't closing the
shared memory's file descriptor, the semaphore or
the mapfile.
</li>
</ul>
</li>
</ul>
</li>
<li>0.6 (5 Oct 2009) –
<ul>
<li>Relicensed from the GPL to a BSD license to celebrate the
one year anniversary of this module.
<li>Updated Cygwin info.</li>
</ul>
</li>
<li>0.5.5 (17 Sept 2009) –
<ul>
<li>Set <tt>MQ_MAX_MESSAGES</tt> and <tt>MQ_MAX_MESSAGE_SIZE</tt> to
<tt>LONG_MAX</tt> under cygwin.
(<i>Danke</i> to René Liebscher.)
</li>
<li>Surrounded the <tt>#define PAGE_SIZE</tt> in probe_results.h with
<tt>#ifndef/#endif</tt> because it is already defined on some systems.
(<i>Danke</i> to René Liebscher, again.)
</li>
<li>Minor documentation changes.</li>
</ul>
</li>
<li>0.5.4 (21 Jun 2009) –
<ul>
<li>Added SignalError.</li>
<li>Fixed a bug where
<a href="http://groups.google.com/group/comp.lang.python/browse_thread/thread/ada39e984dfc3da6/fd6becbdce91a6be?#fd6becbdce91a6be">Python
would generate an uncatchable
KeyboardInterrupt when Ctrl-C was hit during a wait</a>
(e.g. <tt>sem.acquire()</tt>).
<p>Thanks to Maciek W. for reporting the problem and to
Piet van Oostrum and Greg for help with a solution.
</p>
</li>
<li>Minor documentation changes.</li>
</ul>
</li>
<li>0.5.3 (8 Mar 2009) –
<ul>
<li>Added automatic generation of names.</li>
<li>Changed status to beta.</li>
</ul>
</li>
<li>0.5.2 (12 Feb 2009) –
<ul>
<li>Fixed a memory leak in <tt>MessageQueue.receive()</tt>.</li>
<li>Fixed a bug where the name of the <tt>MessageQueue</tt>
<tt>current_messages</tt> attribute didn't match the name
given in the documentation.
</li>
<li>Added the VERSION attribute to the module.</li>
<li>Fixed a documentation bug that said message queue
notifications were not yet supported.
</li>
</ul>
</li>
<li>0.5.1 (8 Feb 2009) –
<ul>
<li>Fixed outdated info in setup.py that was showing up
in the Python package index. Updated README while I
was at it.
</li>
</ul>
</li>
<li>0.5 (8 Feb 2009) –
<ul>
<li>Added the message queue notification feature.</li>
<li>Added a <tt>mode</tt> attribute to each type.</li>
<li>Added <tt>str()</tt> and <tt>repr()</tt> support to
each object.
</li>
<li>Added a demo for message queues.</li>
<li>Fixed some minor documentation problems and added
some information (esp. about Windows + Cygwin).
</li>
</ul>
</li>
<li>0.4 (9 Jan 2009) –
<ul>
<li>Added message queue support.</li>
<li>Fixed the poor choices I'd made for names for classes and
errors by removing the leading "Posix" and "PosixIpc".
</li>
<li>Simplified the prober and expanded it (for message
queue support).
</li>
<li>Cleaned up this documentation.</li>
</ul>
</li>
<li>0.3.2 (4 Jan 2009) –
<ul>
<li>Fixed an uninitialized value passed to PyMem_Free() when
invalid params were passed to either constructor.
</li>
</ul>
</li>
<li>0.3.1 (1 Jan 2009) –
<ul>
<li>Fixed a big bug where the custom exceptions defined by this
module weren't visible.
</li>
<li>Fixed a compile complaint about the redefinition of
<tt>SEM_VALUE_MAX</tt> on Linux (Ubuntu) that I introduced
in the previous version.
</li>
<li>Fixed a bug in the demo program premise.c where I wasn't
closing the file descriptor associated with the shared
memory.
</li>
<li>Added the <tt>PAGE_SIZE</tt> attribute. This was already
available in the mmap module that you need to use shared
memory anyway, but adding it makes the interface more
consistent with the <tt>sysv_ipc</tt> module.
</li>
</ul>
</li>
<li>0.3 (19 Dec 2008) –
<ul>
<li>Added informative custom errors instead of raising
OSError when something goes wrong.
</li>
<li>Made the code friendly to multi-threaded applications.
</li>
<li>Added the constants <tt>O_CREX</tt> and
<tt>SEMAPHORE_VALUE_MAX</tt>.
</li>
<li>Added code to prohibit negative timeout values.</li>
</ul>
</li>
<li>0.2 (4 Dec 2008) –
<ul>
<li>Removed the un-Pythonic <tt>try_acquire()</tt> method. The
same functionality is now available by passing a timeout of
<tt>0</tt> to the <tt>.acquire()</tt> method.
</li>
<li>Renamed the module constant <tt>ACQUIRE_TIMEOUT_SUPPORTED</tt> to
<tt>SEMAPHORE_TIMEOUT_SUPPORTED</tt>.
</li>
<li>Moved the demo code into its own directory and added C
versions of the Python scripts. The parameters are now in a
text file shared by the Python and C program, so you can
run the C version of Mrs. Premise and have it communicate with
the Python version of Mrs. Conclusion and vice versa.
</li>
</ul>
</li>
<li>0.1 (9 Oct 2008) – Original (alpha) version.</li>
</ul>
</body>
</html>
|
