1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<meta name="author" content="Philip Semanchuk">
<meta name="copyright" content="All contents © 2014 Philip Semanchuk">
<meta name="keywords" content="python posix ipc semaphore shared memory message queue">
<title>POSIX IPC for Python</title>
<style type="text/css">
dt {
font-family: monospace;
font-weight: bold;
padding-bottom: .33em;
margin-top: 1em;
}
span[lang] { font-style: italic; }
span.param {
font-family: monospace;
font-style: italic;
}
li {
margin-top: .67em;
margin-bottom: .67em;
}
pre { margin-left: 2em; }
.dangerdanger {
border: 2px dashed red;
padding: .5em;
margin: 1em;
font-weight: bold;
clear: both;
}
form {
width: 20em;
float: right;
font-size: 90%;
margin-left: 1em;
margin-bottom: 1em;
float: right;
}
fieldset, legend {
background-color: #d0d0a9;
}
/* This style is only present on the local version of the readme.
In the online version, the RSS feed is displayed. */
div.rss { display: none; }
</style>
</head>
<body>
<h2>POSIX IPC for Python - Semaphores, Shared Memory and Message Queues</h2>
<div class="rss">
<a href="rss.xml"><img src="/common/images/rss.png" width="28" height="28" alt=""></a>
<br><a href="rss.xml">RSS</a>
</div>
<p>The Python extension module <tt>posix_ipc</tt> gives access
to POSIX inter-process semaphores, shared memory and message queues on
systems that support the POSIX Realtime Extensions a.k.a. POSIX 1003.1b-1993.
That includes most (all?) Linuxes with kernel ≥ 2.6, OpenSolaris ≥ 2008.05
and FreeBSD ≥ 7.2.
</p>
<p>OS X and other Unix-y platforms (including Windows +
<a href="http://www.cygwin.com/">Cygwin 1.7</a>) provide partial
(or partially broken) support. See <a href="#platforms">
the platform notes below</a> for more details.
</p>
<p>This module is known to work with Python 2.4 – 3.3 (but <em>not</em>
3.0). It is released
under a <a href="http://creativecommons.org/licenses/BSD/">BSD license</a>.
</p>
<p>You might be interested in the very similar module
<a href="/philip/sysv_ipc/"><tt>sysv_ipc</tt> which
provides Python access to IPC using System V semaphores, shared memory and
message queues</a>.
System V IPC has broader OS support but is a little less easy to use and
usually lags behind this module a little.
</p>
<p>You can <strong>download
<a href="posix_ipc-0.9.8.tar.gz">posix_ipc version 0.9.8</a>
</strong>
<a href="posix_ipc-0.9.8.md5.txt">[MD5 sum]</a>
<a href="posix_ipc-0.9.8.sha1.txt">[SHA1 sum]</a>
which contains the source code, setup.py, installation instructions and
<a href="#samples">sample code</a>. The exact same
<a href="https://pypi.python.org/pypi/posix_ipc">posix_ipc tarball is also available on PyPI</a>.
</p>
<p>
You might want to read
<a href="history.html">all of the changes in this version</a> and
about some <a href="#KnownBugs">known bugs</a>.
</p>
<p>Note that this module doesn't support unnamed (anonymous) POSIX
semaphores.
</p>
<h2>Module <tt>posix_ipc</tt> Documentation</h2>
<p>Jump to <a href="#semaphore">semaphores</a>,
<a href="#shared_memory">shared memory</a>, or
<a href="#message_queue">message queues</a>.</p>
<h3>Module Functions</h3>
<dl>
<dt>unlink_semaphore(name)<br>
unlink_shared_memory(name)<br>
unlink_message_queue(name)
</dt>
<dd>Convenience functions that unlink the IPC object described
by <span class="param">name</span>.</dd>
</dl>
<h3>Module Constants</h3>
<dl>
<dt>O_CREX, O_CREAT, O_EXCL and O_TRUNC</dt>
<dd>These flags are used when creating IPC objects.
All except <tt>O_CREX</tt> are bitwise unique and can be
ORed together. <tt>O_CREX</tt> is shorthand for
<tt>O_CREAT | O_EXCL</tt>.
<p><tt>O_TRUNC</tt> is only useful when
creating SharedMemory objects.</p>
</dd>
<dt>PAGE_SIZE</dt>
<dd>The operating system's memory page size, in bytes. It's probably a
good idea to make shared memory segments some multiple of this size.
</dd>
<dt>SEMAPHORE_TIMEOUT_SUPPORTED</dt>
<dd>True if the underlying OS supports <tt>sem_timedwait()</tt>. If False, all
timeouts > 0 passed to a semaphore's <tt>acquire()</tt> method are
treated as infinity.
<p>As far as I know, this is only False under OS X.</p>
</dd>
<dt>SEMAPHORE_VALUE_SUPPORTED</dt>
<dd>True if the underlying OS supports <tt>sem_getvalue()</tt>. If False,
accessing the <tt>value</tt> attribute on a <tt>Semaphore</tt> instance
will raise an AttributeError.
<p>As far as I know, this is only False under OS X.</p>
</dd>
<dt>SEMAPHORE_VALUE_MAX</dt>
<dd>The maximum value that can be assigned to a semaphore.
</dd>
<dt>MESSAGE_QUEUES_SUPPORTED</dt>
<dd>True if the underlying OS supports message queues, False otherwise.
</dd>
<dt>QUEUE_MESSAGES_MAX_DEFAULT</dt>
<dd>The default value for a message queue's <tt>max_messages</tt>
attribute. This can be quite small under Linux (e.g. 10)
but is usually LONG_MAX everywhere else.
</dd>
<dt>QUEUE_MESSAGE_SIZE_MAX_DEFAULT</dt>
<dd>The default value for a message queue's <tt>max_message_size</tt>
attribute. This is 8k (or possibly smaller under Linux).
</dd>
<dt>QUEUE_PRIORITY_MAX</dt>
<dd>The maximum message queue message priority.
</dd>
<dt>USER_SIGNAL_MIN, USER_SIGNAL_MAX</dt>
<dd>The constants define a range of signal values reserved for
use by user applications (like yours).
</dd>
</dl>
<h3>Module Errors</h3>
<p>In addition to standard Python errors (e.g. <tt>ValueError</tt>),
this module raises custom errors. These errors cover
situations specific to IPC.
</p>
<dl>
<dt>Error</dt>
<dd>The base error class for all the custom errors in this module.
</dd>
<dt>SignalError</dt>
<dd>Raised when a waiting call (e.g. <tt>sem.acquire()</tt>) is
interrupted by a signal other than KeyboardInterrupt.
</dd>
<dt>PermissionsError</dt>
<dd>Indicates that you've attempted something that the permissions on the
IPC object don't allow.
</dd>
<dt>ExistentialError</dt>
<dd>Indicates an error related to the existence or non-existence of
an IPC object.
</dd>
<dt>BusyError</dt>
<dd>Raised when a call times out.
</dd>
</dl>
<h3 id="semaphore">The Semaphore Class</h3>
<p>This is a handle to a semaphore.</p>
<h4>Methods</h4>
<dl>
<dt>Semaphore(name, [flags = 0, [mode = 0600, [initial_value = 0]]])</dt>
<dd>Creates a new semaphore or opens an existing one.
<p><span class="param">name</span> must be <tt>None</tt> or
a string. If it is <tt>None</tt>, the module chooses a random
unused name. If it is a string, it
should begin with a slash and be valid according
to pathname rules on your system, e.g.
<tt>/wuthering_heights_by_semaphore</tt>
</p>
<p>The <span class="param">flags</span> specify whether you want to create a
new semaphore or open an existing one.
</p>
<ul>
<li>With <span class="param">flags</span> set to the default of <tt>0</tt>, the module attempts
to open an existing semaphore and raises an error if that semaphore
doesn't exist.
</li>
<li>With <span class="param">flags</span> set to <tt>O_CREAT</tt>,
the module opens the semaphore if it exists (in which case mode and
initial value are ignored) or creates it if it doesn't.
</li>
<li>With <span class="param">flags</span> set to <tt>O_CREAT | O_EXCL</tt>
(or <tt>O_CREX</tt>),
the module creates a new semaphore identified by
<span class="param">name</span>. If a
semaphore with that name already exists, the call raises
an <tt>ExistentialError</tt>.
</li>
</ul>
</dd>
<dt>acquire([timeout=None])</dt>
<dd>Waits (conditionally) until the semaphore's value is > 0 and then returns,
decrementing the semaphore.
<p>The <span class="param">timeout</span> (which can be a float) specifies how
many seconds this call should wait, if at all.
</p>
<ul>
<li>A <span class="param">timeout</span> of None (the default)
implies no time limit. The call will not return until its wait
condition is satisfied.
</li>
<li>When <span class="param">timeout</span> is 0, the call
immediately raises a <tt>BusyError</tt>
if asked to wait. Since it will return immediately if not
asked to wait, this can be thought of as "non-blocking" mode.
</li>
<li>When the <span class="param">timeout</span> is > 0, the call
will wait no longer than <span class="param">timeout</span>
seconds before either returning (having acquired the semaphore)
or raising a <tt>BusyError</tt>.
<p>On platforms that don't support the <tt>sem_timedwait()</tt> API,
a <span class="param">timeout</span> > 0 is treated as
infinite. The call will not return until its wait
condition is satisfied.
</p>
<p>Most platforms provide <tt>sem_timedwait()</tt>. OS X is a
notable exception. The module's Boolean constant
<tt>SEMAPHORE_TIMEOUT_SUPPORTED</tt>
is True on platforms that support <tt>sem_timedwait()</tt>.
</p>
</li>
</ul>
</dd>
<dt>release()</dt>
<dd>
Releases (increments) the semaphore.
</dd>
<dt>close()</dt>
<dd>
Closes the semaphore, indicating that the current <em>process</em> is
done with the semaphore. The effect of subsequent use of the semaphore
by the current process is undefined. Assuming it still exists,
(see <tt>unlink()</tt>, below) the semaphore can be re-opened.
<p>You must call <tt>close()</tt> explicitly; it is
<strong>not</strong> called automatically
when a Semaphore object is garbage collected.
</p>
</dd>
<dt id="unlink_semaphore">unlink()</dt>
<dd>
Destroys the semaphore, with a caveat. If any processes have the semaphore
open when unlink is called, the call to unlink returns immediately
but destruction of the semaphore is postponed until all processes
have closed the semaphore.
<p>Note, however, that once a semaphore has been unlinked,
calls to <tt>open()</tt> with the same name should
refer to a new semaphore. Sound confusing? It is, and you'd
probably be wise structure your code so as to avoid
this situation.
</p>
</dd>
</dl>
<h4>Attributes</h4>
<dl>
<dt>name (read-only)</dt>
<dd>The name provided in the constructor.</dd>
<dt>value (read-only)</dt>
<dd>The integer value of the semaphore. Not available on OS X.
(See <a href="#platforms">Platforms</a>)
</dd>
</dl>
<h4>Context Manager Support</h4>
<p>These semaphores provide <tt>__enter__()</tt> and <tt>__exit__()</tt>
methods so they can be used in context managers. For instance --
</p>
<pre>
with posix_ipc.Semaphore(name) as sem:
# Do something...
</pre>
<p>Entering the context acquires the semaphore, exiting the context releases
the semaphore. See <tt>demo4/child.py</tt> for a complete example.
</p>
<h3 id="shared_memory">The SharedMemory Class</h3>
<p>This is a handle to a shared memory segment. POSIX shared memory segments
masquerade as files, and so the handle to a shared memory segment is just
a file descriptor that can be mmapped.
</p>
<h4>Methods</h4>
<dl>
<dt>SharedMemory(name, [flags = 0, [mode = 0600, [size = 0, [read_only = false]]]])</dt>
<dd>Creates a new shared memory segment or opens an existing one.
<p><span class="param">name</span> must be <tt>None</tt> or
a string. If it is <tt>None</tt>, the module chooses a random
unused name. If it is a string, it
should begin with a slash and be valid according
to pathname rules on your system, e.g.
<tt>/four_yorkshiremen_sharing_memories</tt>
</p>
<p>On some systems you need to have write access to the path.</p>
<p>The <span class="param">flags</span> specify whether you want to create a
new shared memory segment or open an existing one.
</p>
<ul>
<li>With <span class="param">flags</span> set to the default of <tt>0</tt>, the module attempts
to open an existing segment and raises an error if that segment
doesn't exist.
</li>
<li>With <span class="param">flags</span> set to <tt>O_CREAT</tt>,
the module opens the segment if it exists (in which case
<span class="param">size</span> and <span class="param">mode</span>
are ignored) or creates it if it doesn't.
</li>
<li>With <span class="param">flags</span> set to <tt>O_CREAT | O_EXCL</tt>
(or <tt>O_CREX</tt>),
the module creates a new shared memory segment identified by
<span class="param">name</span>. If a
segment with that name already exists, the call raises
an <tt>ExistentialError</tt>.
</li>
</ul>
<p>When opening an existing shared memory segment, one can also specify
the flag <tt>O_TRUNC</tt>
to truncate the shared memory to zero bytes.
</p>
</dd>
<dt>close_fd()</dt>
<dd>
Closes the file descriptor associated with this SharedMemory
object. Calling <tt>close_fd()</tt> is the same as calling
<tt><a href="http://www.python.org/doc/2.6/library/os.html#os.close">os.close()</a></tt>
on a SharedMemory object's <tt>fd</tt> attribute.
<p>You must call <tt>close_fd()</tt> or <tt>os.close()</tt>
explicitly; the file descriptor is <strong>not</strong> closed
automatically when a SharedMemory object is garbage collected.
</p>
<p>Closing the file descriptor has no effect on any <tt>mmap</tt>
objects that were created from it. See the demo for an
example.
</p>
</dd>
<dt>unlink()</dt>
<dd>
Marks the shared memory for destruction once all processes have unmapped it.
<p>
<a href="http://www.opengroup.org/onlinepubs/009695399/functions/shm_unlink.html">The
POSIX specification for <tt>shm_unlink()</tt></a> says, "Even if the object
continues to exist after the last shm_unlink(), reuse of the name shall subsequently
cause shm_open() to behave as if no shared memory object of this name exists
(that is, shm_open() will fail if O_CREAT is not set, or will create a new shared
memory object if O_CREAT is set)."
</p>
<p>I'll bet a virtual cup of coffee that this tricky part of the
standard is not well or consistently implemented in every OS. Caveat emptor.
</p>
</dd>
</dl>
<h4>Attributes</h4>
<dl>
<dt>name (read-only)</dt>
<dd>The name provided in the constructor.</dd>
<dt>fd (read-only)</dt>
<dd>The file descriptor that represents the memory segment.</dd>
<dt>size (read-only)</dt>
<dd>The size (in bytes) of the shared memory segment.</dd>
</dl>
<h3 id="message_queue">The MessageQueue Class</h3>
<p>This is a handle to a message queue.</p>
<h4>Methods</h4>
<dl>
<dt>MessageQueue(name, [flags = 0, [mode = 0600, [max_messages = QUEUE_MESSAGES_MAX_DEFAULT, [max_message_size = QUEUE_MESSAGE_SIZE_MAX_DEFAULT, [read = True, [write = True]]]]]])</dt>
<dd>Creates a new message queue or opens an existing one.
<p><span class="param">name</span> must be <tt>None</tt> or
a string. If it is <tt>None</tt>, the module chooses a random
unused name. If it is a string, it
should begin with a slash and be valid according
to pathname rules on your system, e.g.
<tt>/my_message_queue</tt>
</p>
<p>On some systems you need to have write access to the path.</p>
<p>The <span class="param">flags</span> specify whether you want to
create a new queue or open an existing one.
</p>
<ul>
<li>With <span class="param">flags</span> set to the default of
<tt>0</tt>, the module attempts
to open an existing queue and raises an error if that queue
doesn't exist.
</li>
<li>With <span class="param">flags</span> set to <tt>O_CREAT</tt>,
the module opens the queue if it exists (in which case
<span class="param">size</span> and <span class="param">mode</span>
are ignored) or creates it if it doesn't.
</li>
<li>With <span class="param">flags</span> set to <tt>O_CREAT | O_EXCL</tt>
(or <tt>O_CREX</tt>),
the module creates a new message queue identified by
<span class="param">name</span>. If a
queue with that name already exists, the call raises
an <tt>ExistentialError</tt>.
</li>
</ul>
<p><span class="param">Max_messages</span> defines how many messages
can be in the queue at one time. When the queue is full,
calls to <tt>.send()</tt> will wait.
</p>
<p><span class="param">Max_message_size</span> defines the maximum
size (in bytes) of a message.
</p>
<p><span class="param">Read</span> and
<span class="param">write</span>
default to True. If <span class="param">read/write </span>
is False, calling <tt>.receive()/.send()</tt> on this object
is not permitted.
This doesn't affect other handles to the same queue.
</p>
</dd>
<dt>send(message, [timeout = None, [priority = 0]])</dt>
<dd>
Sends a message via the queue.
<p>The <span class="param">message</span> string can contain embedded
NULLs (ASCII <tt>0x00</tt>). Under Python 3, the message can
also be a bytes object.
</p>
<p>The <span class="param">timeout</span> (which can be a float)
specifies how many seconds this call should wait if the
queue is full. Timeouts are irrelevant when the <tt>block</tt>
flag is False.
</p>
<ul>
<li>A <span class="param">timeout</span> of None (the default)
implies no time limit. The call will not return until the
message is sent.
</li>
<li>When <span class="param">timeout</span> is 0, the call
immediately raises a <tt>BusyError</tt>
if asked to wait.
</li>
<li>When the <span class="param">timeout</span> is > 0, the call
will wait no longer than <span class="param">timeout</span>
seconds before either returning (having sent the message)
or raising a <tt>BusyError</tt>.
</li>
</ul>
<p>The <span class="param">priority</span> allows you to order
messages in the queue. The highest priority message is received
first. By default, messages are sent at the lowest priority (0).
</p>
</dd>
<dt>receive([timeout = None])</dt>
<dd>
Receives a message from the queue, returning a tuple of
<tt>(message, priority)</tt>. Messages are received in the order of
highest priority to lowest, and in FIFO order for messages of
equal priority.
Under Python 3, the returned message is a bytes object.
<p>If the queue is empty, the call will not return immediately.
The <span class="param">timeout</span> parameter controls the
wait just as for the function <tt>send()</tt>.
</p>
</dd>
<dt>request_notification([notification = None])</dt>
<dd>Depending on the parameter, requests or cancels notification from the
operating system when the queue changes from empty to non-empty.
<ul>
<li>When <span class="param">notification</span> is <tt>None</tt>
(the default), any existing notification request is
cancelled.
</li>
<li>When <span class="param">notification</span> is an
integer, notification will be sent as a signal of this
value that can be caught using a signal handler installed
with <tt>signal.signal()</tt>.
</li>
<li>When <span class="param">notification</span> is a tuple
of <tt>(function, param)</tt>, notification will be sent
by invoking <tt><em>function(param)</em></tt> in a new
thread.
</li>
</ul>
<p>Message queues accept only one notification request at a time.
If another process has already requested notifications from
this queue, this call will fail with a <tt>BusyError</tt>.
</p>
<p>The operating system delivers (at most) one notification
per request. If you want subsequent notifications, you must
request them by calling
<tt>request_notification()</tt> again.
</p>
</dd>
<dt>close()</dt>
<dd>
Closes this reference to the queue.
<p>You must call <tt>close()</tt> explicitly; it is
<strong>not</strong> called automatically
when a MessageQueue object is garbage collected.
</p>
</dd>
<dt>unlink()</dt>
<dd>
Requests destruction of the queue. Although the call returns
immediately, actual destruction of the queue is postponed until all
references to it are closed.
</dd>
</dl>
<h4>Attributes</h4>
<dl>
<dt>name (read-only)</dt>
<dd>The name provided in the constructor.</dd>
<dt>mqd (read-only)</dt>
<dd>The message queue descriptor that represents the queue.</dd>
<dt>block</dt>
<dd>When True (the default), calls to <tt>.send()</tt> and
<tt>.receive()</tt> may wait (block) if they cannot immediately
satisfy the send/receive request. When <tt>block</tt> is False,
the module will raise <tt>BusyError</tt>
instead of waiting.
</dd>
<dt>max_messages (read-only)</dt>
<dd>The maximum number of messages the queue can hold.</dd>
<dt>max_message_size (read-only)</dt>
<dd>The maximum message size (in bytes).</dd>
<dt>current_messages (read-only)</dt>
<dd>The number of messages currently in the queue.</dd>
</dl>
<h3>Usage Tips</h3>
<h4 id="samples">Sample Code</h4>
<p>This module comes with three demonstrations. The first (in the
directory <tt>demo</tt>) shows how to use shared memory and semaphores.
The second (in the directory <tt>demo2</tt>) shows how to use
message queues. The third (<tt>demo3</tt>) shows how to use message queue
notifications.
</p>
<h4>Nobody Likes a Mr. Messy</h4>
<p>IPC objects are a little different from most Python objects
and therefore require a little more care on the part of the programmer. When a
program creates a IPC object, it creates something that
resides <em>outside of its own process</em>, just like a file on a hard drive. It
won't go away when your process ends unless you explicitly remove it. And since many
operating systems don't even give you a way to enumerate existing POSIX IPC
entities, it might be hard to figure out what you're leaving behind.
</p>
<p>In short, remember to clean up after yourself.</p>
<h4>Semaphores and References</h4>
<p>I know it's <em>verboten</em> to talk about pointers in Python, but I'm
going to do it anyway.
</p>
<p>Each Semaphore object created by this module contains a C pointer to
the IPC object created by the system. When you call <tt>sem.close()</tt>,
the object's internal pointer is set to <tt>NULL</tt>. This leaves the
object in a not-quite-useless state. You can still call <tt>sem.unlink()</tt>
or print <tt>sem.name</tt>, but calls to <tt>sem.aquire()</tt> or
<tt>sem.release()</tt> will raise an <tt>ExistentialError</tt>.
</p>
<p>If you know you're not going to use a Semaphore object after calling
<tt>sem.close()</tt> or <tt>sem.unlink()</tt>, you could you set your
semaphore variable to the return from the function (which is always
<tt>None</tt>) like so:
</p>
<pre>
my_sem = my_sem.close()
</pre>
<p>That will ensure you don't have any nearly useless objects laying around
that you might use by accident.
</p>
<p>This doesn't apply to shared memory and message queues because they're
referenced at the C level by a file descriptor rather than a pointer.
</p>
<h4>Permissions</h4>
<p>It appears that the read and write mode bits on IPC objects are
ignored by the operating system. For instance, on OS X, OpenSolaris and
Linux one can write to semaphores and message queues with a mode of
<tt>0400</tt>.
</p>
<h4>Message Queues</h4>
<p>When creating a new message queue, you specify a maximum message size
which defaults to <tt>QUEUE_MESSAGE_SIZE_MAX_DEFAULT</tt> (currently 8192
bytes). You can create a queue with a larger value, but be aware that
<tt>posix_ipc</tt> allocates a buffer the size of the maximum message size
every time <tt>receive()</tt> is called.
</p>
<h4>Resizing Shared Memory Segments</h4>
<p>
Under OS X/Darwin, <tt>ftruncate()</tt> can be used to set the memory size <em>once</em>
after the initial call to <tt>shm_open()</tt>. This module does that in the
<tt>SharedMemory</tt> constructor, so subsequent attempts to resize the shared memory
will fail.
</p>
<p>I don't know if this holds true on all platforms. If your platform supports multiple
calls to <tt>ftruncate()</tt>, you can call that via Python's <tt>os</tt> module,
passing the file descriptor exposed in the <tt>SharedMemory</tt> object.
</p>
<h4>Consult Your Local <tt>man</tt> Pages</h4>
<p>The posix_ipc module is just a wrapper around your system's API. If your
system's implementation has quirks, the <tt>man</tt> pages for
<tt>sem_open, sem_post,
sem_wait, sem_close, sem_unlink, shm_open, shm_unlink, mq_open, mq_send
mq_receive, mq_getattr, mq_close, mq_unlink</tt> and <tt>mq_notify</tt> will
probably cover them.
</p>
<h4>Last But Not Least</h4>
<p>For Pythonistas –</p>
<ul>
<li><a href="http://www.youtube.com/watch?v=13JK5kChbRw">A meditation on the
inaccuracy of shared memories</a>
</li>
</ul>
<h3><a name="KnownBugs">Known Bugs</a></h3>
<p>I don't know of any bugs in this code. However, under Python 3 the
standard library modules accept bytes and bytearray objects for filenames in
addition to strings. One could argue that this module should behave the
same way.
</p>
<p>Also, this module doesn't support Python 3 memory views, which it
probably should (for shared memory objects). Support for that might come in
a later version.
</p>
<h2 id="platforms">Platform Notes</h2>
<p>This module is just a wrapper around the operating system's functions,
so if the operating system doesn't provide a function, this module can't
either. The POSIX Realtime Extensions (POSIX 1003.1b-1993) are, as the name
implies, an extension to POSIX and so a platform can claim "<em>POSIX
conformance</em>" and still not support any or all of the IPC functions.
</p>
<dl>
<dt>Linux with kernel ≥ 2.6</dt>
<dd>All features supported.</dd>
<dt>OpenSolaris ≥ 2008.05</dt>
<dd>All features supported.</dd>
<dt>FreeBSD ≥ 7.2</dt>
<dd>All features supported.
<p>Under 7.2, <tt>posix_ipc</tt>'s demos fail unless they're run as
root. It's a simple permissions problem. Prefix the IPC object
names with <tt>/tmp</tt> in <tt>params.txt</tt> and the problem
goes away. I didn't see this behavior under FreeBSD 8.0, so it
must have been fixed at some point.
</p>
<p>If you don't have the <tt>sem</tt> and <tt>mqueuefs</tt> kernel
modules loaded, you'll get a message like this (or something
similarly discouraging) when you
try to create a semaphore or message queue:<br>
<tt>Bad system call: 12 (core dumped)</tt>
</p>
<p>Type <tt>kldstat</tt> to list loaded modules, and
<tt>kldload sem</tt> or <tt>kldload mqueuefs</tt> if you need
to load either of these. BTW,
<a href="http://www.freebsd.org/cgi/man.cgi?query=mqueuefs&apropos=0&sektion=5&manpath=FreeBSD+8.0-stable&format=html">mqueuefs</a> has
some cool features.
</p>
<p>
Prior to 7.2, FreeBSD POSIX semaphore support was
<a href="http://www.freebsd.org/cgi/query-pr.cgi?pr=127545">broken</a>.
</p>
</dd>
<dt>OS X (up to and including 10.8)</dt>
<dd>
Message queues are not supported by OS X. Also,
<tt>sem_getvalue()</tt> and <tt>sem_timedwait()</tt> are not
supported.
<p>From what I can tell, OS X does not support <tt>sem_init()</tt> or
<tt>sem_destroy()</tt>, so even if this module adds support for unnamed
semaphores, they won't be available under OS X.
</dd>
<dt>Windows + Cygwin 1.7</dt>
<dd><a href="http://www.cygwin.com/">Cygwin</a> is a Linux-like
environment for Windows.
<p>Versions of Cygwin prior to 1.7 didn't support POSIX IPC.
Under Cygwin 1.7 beta 62 (released in early October 2009),
<tt>posix_ipc</tt> compiles and runs both demos.
</p>
</dd>
</dl>
</body>
</html>
|
