1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Serialization — Kombu 2.0.0 documentation</title>
<link rel="stylesheet" href="../_static/celery.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '2.0.0',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="top" title="Kombu 2.0.0 documentation" href="../index.html" />
<link rel="up" title="User Guide" href="index.html" />
<link rel="next" title="Frequently Asked Questions" href="../faq.html" />
<link rel="prev" title="Connection and Producer Pools" href="pools.html" />
</head>
<body>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="../faq.html" title="Frequently Asked Questions"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="pools.html" title="Connection and Producer Pools"
accesskey="P">previous</a> |</li>
<li><a href="../index.html">Kombu 2.0.0 documentation</a> »</li>
<li><a href="index.html" accesskey="U">User Guide</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="serialization">
<span id="guide-serialization"></span><h1>Serialization<a class="headerlink" href="#serialization" title="Permalink to this headline">¶</a></h1>
<div class="section" id="serializers">
<span id="id1"></span><h2>Serializers<a class="headerlink" href="#serializers" title="Permalink to this headline">¶</a></h2>
<p>By default every message is encoded using <a class="reference external" href="http://www.json.org/">JSON</a>, so sending
Python data structures like dictionaries and lists works.
<a class="reference external" href="http://yaml.org/">YAML</a>, <a class="reference external" href="http://msgpack.sourceforge.net/">msgpack</a> and Python’s built-in <cite>pickle</cite> module is also supported,
and if needed you can register any custom serialization scheme you
want to use.</p>
<p>Each option has its advantages and disadvantages.</p>
<dl class="docutils">
<dt><cite>json</cite> – JSON is supported in many programming languages, is now</dt>
<dd><p class="first">a standard part of Python (since 2.6), and is fairly fast to
decode using the modern Python libraries such as <cite>cjson</cite> or
<cite>simplejson</cite>.</p>
<p>The primary disadvantage to <cite>JSON</cite> is that it limits you to
the following data types: strings, Unicode, floats, boolean,
dictionaries, and lists. Decimals and dates are notably missing.</p>
<p>Also, binary data will be transferred using Base64 encoding, which
will cause the transferred data to be around 34% larger than an
encoding which supports native binary types.</p>
<p class="last">However, if your data fits inside the above constraints and
you need cross-language support, the default setting of <cite>JSON</cite>
is probably your best choice.</p>
</dd>
<dt><cite>pickle</cite> – If you have no desire to support any language other than</dt>
<dd>Python, then using the <cite>pickle</cite> encoding will gain you
the support of all built-in Python data types (except class instances),
smaller messages when sending binary files, and a slight speedup
over <cite>JSON</cite> processing.</dd>
<dt><cite>yaml</cite> – YAML has many of the same characteristics as <cite>json</cite>,</dt>
<dd><p class="first">except that it natively supports more data types (including dates,
recursive references, etc.)</p>
<p>However, the Python libraries for YAML are a good bit slower
than the libraries for JSON.</p>
<p class="last">If you need a more expressive set of data types and need to maintain
cross-language compatibility, then <cite>YAML</cite> may be a better fit
than the above.</p>
</dd>
</dl>
<p>To instruct carrot to use an alternate serialization method,
use one of the following options.</p>
<blockquote>
<div><ol class="arabic">
<li><p class="first">Set the serialization option on a per-producer basis:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">producer</span> <span class="o">=</span> <span class="n">Producer</span><span class="p">(</span><span class="n">channel</span><span class="p">,</span>
<span class="gp">... </span> <span class="n">exchange</span><span class="o">=</span><span class="n">exchange</span><span class="p">,</span>
<span class="gp">... </span> <span class="n">serializer</span><span class="o">=</span><span class="s">"yaml"</span><span class="p">)</span>
</pre></div>
</div>
</li>
<li><p class="first">Set the serialization option per message:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="n">producer</span><span class="o">.</span><span class="n">publish</span><span class="p">(</span><span class="n">message</span><span class="p">,</span> <span class="n">routing_key</span><span class="o">=</span><span class="n">rkey</span><span class="p">,</span>
<span class="gp">... </span> <span class="n">serializer</span><span class="o">=</span><span class="s">"pickle"</span><span class="p">)</span>
</pre></div>
</div>
</li>
</ol>
</div></blockquote>
<p>Note that a <cite>Consumer</cite> do not need the serialization method specified.
They can auto-detect the serialization method as the
content-type is sent as a message header.</p>
</div>
<div class="section" id="sending-raw-data-without-serialization">
<span id="sending-raw-data"></span><h2>Sending raw data without Serialization<a class="headerlink" href="#sending-raw-data-without-serialization" title="Permalink to this headline">¶</a></h2>
<p>In some cases, you don’t need your message data to be serialized. If you
pass in a plain string or Unicode object as your message, then carrot will
not waste cycles serializing/deserializing the data.</p>
<p>You can optionally specify a <cite>content_type</cite> and <cite>content_encoding</cite>
for the raw data:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s">"~/my_picture.jpg"</span><span class="p">,</span> <span class="s">"rb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">fh</span><span class="p">:</span>
<span class="gp">... </span> <span class="n">producer</span><span class="o">.</span><span class="n">publish</span><span class="p">(</span><span class="n">fh</span><span class="o">.</span><span class="n">read</span><span class="p">(),</span>
<span class="go"> content_type="image/jpeg",</span>
<span class="go"> content_encoding="binary",</span>
<span class="go"> routing_key=rkey)</span>
</pre></div>
</div>
<p>The <cite>Message</cite> object returned by the <cite>Consumer</cite> class will have a
<cite>content_type</cite> and <cite>content_encoding</cite> attribute.</p>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper"><p class="logo"><a href="../index.html">
<img class="logo" width="128" height="128" src="http://cloud.github.com/downloads/ask/kombu/kombusmall.jpg" alt="Logo"/>
</a></p>
<h3><a href="../index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Serialization</a><ul>
<li><a class="reference internal" href="#serializers">Serializers</a></li>
<li><a class="reference internal" href="#sending-raw-data-without-serialization">Sending raw data without Serialization</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="pools.html"
title="previous chapter">Connection and Producer Pools</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="../faq.html"
title="next chapter">Frequently Asked Questions</a></p>
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../_sources/userguide/serialization.txt"
rel="nofollow">Show Source</a></li>
</ul>
<div id="searchbox" style="display: none">
<h3>Quick search</h3>
<form class="search" action="../search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="../faq.html" title="Frequently Asked Questions"
>next</a> |</li>
<li class="right" >
<a href="pools.html" title="Connection and Producer Pools"
>previous</a> |</li>
<li><a href="../index.html">Kombu 2.0.0 documentation</a> »</li>
<li><a href="index.html" >User Guide</a> »</li>
</ul>
</div>
<div class="footer">
© Copyright 2009-2012, Ask Solem.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.2.
</div>
</body>
</html>
|
