summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorDaniele Varrazzo <daniele.varrazzo@gmail.com>2010-04-04 03:10:18 +0100
committerDaniele Varrazzo <daniele.varrazzo@gmail.com>2010-04-21 15:21:32 +0100
commita54932ee9c086f658881b4d1141646d82fdc98db (patch)
tree11d0307618bc31761ad57e7bc3daa81f268d0c7e
parent127f92f9dbbbb54930b8e9c06ba5639ec3ef201b (diff)
downloadpsycopg2-a54932ee9c086f658881b4d1141646d82fdc98db.tar.gz
Added documentation for the green features.
-rw-r--r--doc/src/advanced.rst69
-rw-r--r--doc/src/connection.rst10
-rw-r--r--doc/src/extensions.rst18
-rw-r--r--doc/src/extras.rst8
-rw-r--r--psycopg/green.h15
5 files changed, 109 insertions, 11 deletions
diff --git a/doc/src/advanced.rst b/doc/src/advanced.rst
index b8ed537..13def38 100644
--- a/doc/src/advanced.rst
+++ b/doc/src/advanced.rst
@@ -362,6 +362,75 @@ this will be probably implemented in a future release.
+
+.. index::
+ single: Greenlet, Coroutine, eventlet, gevent, Wait callback
+
+.. _green-support:
+
+Support to coroutine libraries
+------------------------------
+
+.. versionadded:: 2.2.0
+
+Psycopg can be used together with coroutine_\-based libraries, and participate
+to cooperative multithread.
+
+Coroutine-based libraries (such as Eventlet_ or gevent_) can usually patch the
+Python standard library in order to enable a coroutine switch in presence of
+blocking I/O: the process is usually referred as making the system *green*, in
+reference to greenlet_, the basic Python micro-thread library.
+
+Because Psycopg is a C extension module, it is not possible for coroutine
+libraries to patch it: Psycopg instead enables cooperative multithreading by
+allowing the registration of a *wait callback* using the
+`psycopg2.extensions.set_wait_callback()` function. When a wait callback is
+registered, Psycopg will use `libpq non-blocking calls`__ instead of the regular
+blocking ones, and will delegate to the callback the responsibility to wait
+for available data.
+
+This way of working is less flexible of complete asynchronous I/O, but has the
+advantage of maintaining a complete |DBAPI| semantics: from the point of view
+of the end user, all Psycopg functions and objects will work transparently
+in the coroutine environment (the calling coroutine will be blocked while
+other coroutines can be scheduled to run), allowing non modified code and
+third party libraries (such as SQLAlchemy_) to be used in coroutine-based
+programs.
+
+Notice that, while I/O correctly yields control to other coroutines, each
+connection has a lock allowing a single cursor at time to communicate with the
+backend: such lock is not *green*, so blocking against it would block the
+entire program waiting for data, not the single coroutine. Therefore,
+programmers are advised to either avoid to share connections between coroutines
+or to use a library-friendly lock to synchronize shares connections, e.g. for
+pooling.
+
+Coroutine libraries authors should provide a callback implementation (and
+probably register it) to make Psycopg as green as they want. An example
+callback (using `!select()` to block) is provided as
+`psycopg2.extras.wait_select()`: it boils down to something similar to::
+
+ def wait_select(conn):
+ while 1:
+ state = conn.poll()
+ if state == extensions.POLL_OK:
+ break
+ elif state == extensions.POLL_READ:
+ select.select([conn.fileno()], [], [])
+ elif state == extensions.POLL_WRITE:
+ select.select([], [conn.fileno()], [])
+ else:
+ raise OperationalError("bad state from poll: %s" % state)
+
+.. _coroutine: http://en.wikipedia.org/wiki/Coroutine
+.. _greenlet: http://pypi.python.org/pypi/greenlet
+.. _Eventlet: http://eventlet.net/
+.. _gevent: http://www.gevent.org/
+.. _SQLAlchemy: http://www.sqlalchemy.org/
+.. __: http://www.postgresql.org/docs/8.4/static/libpq-async.html
+
+
+
.. testcode::
:hide:
diff --git a/doc/src/connection.rst b/doc/src/connection.rst
index 5d7077a..048bbcd 100644
--- a/doc/src/connection.rst
+++ b/doc/src/connection.rst
@@ -321,7 +321,7 @@ The ``connection`` class
.. versionadded:: 2.2.0
- .. seealso:: :ref:`Asynchronous support <async-support>`.
+ .. seealso:: :ref:`async-support` and :ref:`green-support`.
.. attribute:: async
@@ -338,9 +338,11 @@ The ``connection`` class
Return one of the constants defined in :ref:`poll-constants`. If it
returns `~psycopg2.extensions.POLL_OK` then the connection has been
estabilished or the query results are available on the client.
- Otherwise wait until the file descriptor returned by
- `~connection.fileno()` is ready to read or to write, as explained in
- :ref:`async-support`.
+ Otherwise wait until the file descriptor returned by `fileno()` is
+ ready to read or to write, as explained in :ref:`async-support`.
+ `poll()` should be also used by the function installed by
+ `~psycopg2.extensions.set_wait_callback()` as explained in
+ :ref:`green-support`.
`poll()` is also used to receive asynchronous notifications from the
database: see :ref:`async-notify` from further details.
diff --git a/doc/src/extensions.rst b/doc/src/extensions.rst
index 2d43c29..552dfdc 100644
--- a/doc/src/extensions.rst
+++ b/doc/src/extensions.rst
@@ -106,6 +106,14 @@ functionalities defined by the |DBAPI|_.
Close the object and remove it from the database.
+.. autofunction:: set_wait_callback(f)
+
+ .. versionadded:: 2.2.0
+
+.. autofunction:: get_wait_callback()
+
+ .. versionadded:: 2.2.0
+
.. _sql-adaptation-objects:
@@ -456,7 +464,9 @@ Poll constants
.. versionadded:: 2.2.0
These values can be returned by `connection.poll()` during asynchronous
-connection and communication. See :ref:`async-support`.
+connection and communication. They match the values in the libpq enum
+`!PostgresPollingStatusType`. See :ref:`async-support` and
+:ref:`green-support`.
.. data:: POLL_OK
@@ -481,6 +491,12 @@ connection and communication. See :ref:`async-support`.
select.select([], [conn.fileno()], [])
+.. data:: POLL_ERROR
+
+ There was a problem during connection polling. This value should actually
+ never be returned: in case of poll error usually an exception containing
+ the relevant details is raised.
+
Additional database types
diff --git a/doc/src/extras.rst b/doc/src/extras.rst
index 9eeddcc..9751d19 100644
--- a/doc/src/extras.rst
+++ b/doc/src/extras.rst
@@ -155,3 +155,11 @@ Fractional time zones
.. versionadded:: 2.0.9
+.. index::
+ pair: Example; Coroutine;
+
+Coroutine support
+-----------------
+
+.. autofunction:: wait_select(conn)
+
diff --git a/psycopg/green.h b/psycopg/green.h
index 16e8304..175f055 100644
--- a/psycopg/green.h
+++ b/psycopg/green.h
@@ -34,23 +34,26 @@ extern "C" {
#endif
#define psyco_set_wait_callback_doc \
-"set_wait_callback(f) -- Register a callback function to block waiting for data.\n" \
+"Register a callback function to block waiting for data.\n" \
"\n" \
"The callback should have signature :samp:`fun({conn})` and\n" \
"is called to wait for data available whenever a blocking function from the\n" \
-"libpq is called. Use `!register_wait_function(None)` to revert to the\n" \
-"original behaviour (using blocking libpq functions).\n" \
+"libpq is called. Use `!set_wait_callback(None)` to revert to the\n" \
+"original behaviour (i.e. using blocking libpq functions).\n" \
"\n" \
"The function is an hook to allow coroutine-based libraries (such as\n" \
-"eventlet_ or gevent_) to switch when Psycopg is blocked, allowing\n" \
+"Eventlet_ or gevent_) to switch when Psycopg is blocked, allowing\n" \
"other coroutines to run concurrently.\n" \
"\n" \
"See `~psycopg2.extras.wait_select()` for an example of a wait callback\n" \
-"implementation.\n"
+"implementation.\n" \
+"\n" \
+".. _Eventlet: http://eventlet.net/\n" \
+".. _gevent: http://www.gevent.org/\n"
HIDDEN PyObject *psyco_set_wait_callback(PyObject *self, PyObject *obj);
#define psyco_get_wait_callback_doc \
-"get_wait_callback() -- Return the currently registered wait callback.\n" \
+"Return the currently registered wait callback.\n" \
"\n" \
"Return `None` if no callback is currently registered.\n"
HIDDEN PyObject *psyco_get_wait_callback(PyObject *self, PyObject *obj);