path: root/docs/basic_usage.rst

.. _basic_usage:

===========
Basic Usage
===========

Connection Handling
===================

To begin using Kazoo, a :class:`~kazoo.client.KazooClient` object must be
created and a connection established:

.. code-block:: python

    from kazoo.client import KazooClient

    zk = KazooClient(hosts='127.0.0.1:2181')
    zk.start()

By default, the client will connect to a local Zookeeper server on the default
port (2181). You should make sure Zookeeper is actually running there first,
or the ``start`` command will be waiting until its default timeout.

Once connected, the client will attempt to stay connected regardless of
intermittent connection loss or Zookeeper session expiration. The client can be
instructed to drop a connection by calling `stop`:

.. code-block:: python

    zk.stop()

Listening for Connection Events
-------------------------------

It can be useful to know when the connection has been dropped, restored, or
when the Zookeeper session has expired. To simplify this process Kazoo uses a
state system and lets you register listener functions to be called when the
state changes.

.. code-block:: python

    from kazoo.client import KazooState

    def my_listener(state):
        if state == KazooState.LOST:
            # Register somewhere that the session was lost
        elif state == KazooState.SUSPENDED
            # Handle being disconnected from Zookeeper
        else:
            # Handle being connected/reconnected to Zookeeper

    zk.add_listener(my_listener)

When using the :class:`kazoo.recipe.lock.Lock` or creating ephemeral nodes, its
highly recommended to add a state listener so that your program can properly
deal with connection interruptions or a Zookeeper session loss.

Understanding Kazoo States
--------------------------

The :class:`~kazoo.protocol.states.KazooState` object represents several states
the client transitions through. The current state of the client can always be
determined by viewing the :attr:`~kazoo.client.KazooClient.state` property. The
possible states are:

- LOST
- CONNECTED
- SUSPENDED

When a :class:`~kazoo.client.KazooClient` instance is first created, it is in
the `LOST` state. After a connection is established it transitions to the
`CONNECTED` state. If any connection issues come up or if it needs to connect
to a different Zookeeper cluster node, it will transition to `SUSPENDED` to let
you know that commands cannot currently be run. The connection will also be
lost if the Zookeeper node is no longer part of the quorum, resulting in a
`SUSPENDED` state.

Upon re-establishing a connection the client could transition to `LOST` if the
session has expired, or `CONNECTED` if the session is still valid.

.. note::

    These states should be monitored using a listener as described previously
    so that the client behaves properly depending on the state of the
    connection.

When a connection transitions to `SUSPENDED`, if the client is performing an
action that requires agreement with other systems (using the Lock recipe for
example), it should pause what it's doing. When the connection has been
re-established the client can continue depending on if the state is `LOST` or
transitions directly to `CONNECTED` again.

When a connection transitions to `LOST`, any ephemeral nodes that have been
created will be removed by Zookeeper. This affects all recipes that create
ephemeral nodes, such as the Lock recipe. Lock's will need to be re-acquired
after the state transitions to `CONNECTED` again. This transition occurs when
a session expires or when you stop the clients connection.

**Valid State Transitions**

- *LOST -> CONNECTED*

  New connection, or previously lost one becoming connected.
- *CONNECTED -> SUSPENDED*

  Connection loss to server occurred on a connection.
- *CONNECTED -> LOST*

  Only occurs if invalid authentication credentials are provided after the
  connection was established.
- *SUSPENDED -> LOST*

  Connection resumed to server, but then lost as the session was expired.
- *SUSPENDED -> CONNECTED*

  Connection that was lost has been restored.

Read-Only Connections
---------------------

.. versionadded:: 0.6

Zookeeper 3.4 and above `supports a read-only mode
<http://wiki.apache.org/hadoop/ZooKeeper/GSoCReadOnlyMode>`_. This mode
must be turned on for the servers in the Zookeeper cluster for the
client to utilize it. To use this mode with Kazoo, the
:class:`~kazoo.client.KazooClient` should be called with the
`read_only` option set to `True`. This will let the client connect to
a Zookeeper node that has gone read-only, and the client will continue
to scan for other nodes that are read-write.

.. code-block:: python

    from kazoo.client import KazooClient

    zk = KazooClient(hosts='127.0.0.1:2181', read_only=True)
    zk.start()

A new attribute on :class:`~kazoo.protocol.states.KeeperState` has been
added, `CONNECTED_RO`. The connection states above are still valid,
however upon `CONNECTED`, you will need to check the clients non-
simplified state to see if the connection is `CONNECTED_RO`. For
example:

.. code-block:: python

    from kazoo.client import KazooState
    from kazoo.client import KeeperState

    @zk.add_listener
    def watch_for_ro(state):
        if state == KazooState.CONNECTED:
            if zk.client_state == KeeperState.CONNECTED_RO:
                print("Read only mode!")
            else:
                print("Read/Write mode!")

It's important to note that a `KazooState` is passed in to the listener
but the read-only information is only available by comparing the
non-simplified client state to the `KeeperState` object.

.. warning::

    A client using read-only mode should not use any of the recipes.


Zookeeper CRUD
==============

Zookeeper includes several functions for creating, reading, updating, and
deleting Zookeeper nodes (called znodes or nodes here). Kazoo adds several
convenience methods and a more Pythonic API.

Creating Nodes
--------------

Methods:

* :meth:`~kazoo.client.KazooClient.ensure_path`
* :meth:`~kazoo.client.KazooClient.create`

:meth:`~kazoo.client.KazooClient.ensure_path` will recursively create the node
and any nodes in the path necessary along the way, but can not set the data for
the node, only the ACL.

:meth:`~kazoo.client.KazooClient.create` creates a node and can set the data on
the node along with a watch function. It requires the path to it to exist
first, unless the `makepath` option is set to `True`.

.. code-block:: python

    # Ensure a path, create if necessary
    zk.ensure_path("/my/favorite")

    # Create a node with data
    zk.create("/my/favorite/node", b"a value")

Reading Data
------------

Methods:

* :meth:`~kazoo.client.KazooClient.exists`
* :meth:`~kazoo.client.KazooClient.get`
* :meth:`~kazoo.client.KazooClient.get_children`

:meth:`~kazoo.client.KazooClient.exists` checks to see if a node exists.

:meth:`~kazoo.client.KazooClient.get` fetches the data of the node along with
detailed node information in a :class:`~kazoo.protocol.states.ZnodeStat`
structure.

:meth:`~kazoo.client.KazooClient.get_children` gets a list of the children of
a given node.

.. code-block:: python

    # Determine if a node exists
    if zk.exists("/my/favorite"):
        # Do something

    # Print the version of a node and its data
    data, stat = zk.get("/my/favorite")
    print("Version: %s, data: %s" % (stat.version, data.decode("utf-8")))

    # List the children
    children = zk.get_children("/my/favorite")
    print("There are %s children with names %s" % (len(children), children))

Updating Data
-------------

Methods:

* :meth:`~kazoo.client.KazooClient.set`

:meth:`~kazoo.client.KazooClient.set` updates the data for a given node. A
version for the node can be supplied, which will be required to match before
updating the data, or a :exc:`~kazoo.exceptions.BadVersionError` will be
raised instead of updating.

.. code-block:: python

    zk.set("/my/favorite", b"some data")

Deleting Nodes
--------------

Methods:

* :meth:`~kazoo.client.KazooClient.delete`

:meth:`~kazoo.client.KazooClient.delete` deletes a node, and can optionally
recursively delete all children of the node as well. A version can be
supplied when deleting a node which will be required to match the version of
the node before deleting it or a :exc:`~kazoo.exceptions.BadVersionError`
will be raised instead of deleting.

.. code-block:: python

    zk.delete("/my/favorite/node", recursive=True)

.. _retrying_commands:

Retrying Commands
=================

Connections to Zookeeper may get interrupted if the Zookeeper server goes down
or becomes unreachable. By default, kazoo does not retry commands, so these
failures will result in an exception being raised. To assist with failures
kazoo comes with a :meth:`~kazoo.client.KazooClient.retry` helper that will
retry a function should one of the Zookeeper connection exceptions get raised.

Example:

.. code-block:: python

    result = zk.retry(zk.get, "/path/to/node")

Some commands may have unique behavior that doesn't warrant automatic retries
on a per command basis. For example, if one creates a node a connection might
be lost before the command returns successfully but the node actually got
created. This results in a :exc:`kazoo.exceptions.NodeExistsError` being
raised when it runs again. A similar unique situation arises when a node is
created with ephemeral and sequence options set,
`documented here on the Zookeeper site
<http://zookeeper.apache.org/doc/trunk/recipes.html#sc_recipes_errorHandlingNote>`_.

Since the :meth:`~kazoo.client.KazooClient.retry` method takes a function to
call and its arguments, a function that runs multiple Zookeeper commands could
be passed to it so that the entire function will be retried if the connection
is lost.

This snippet from the lock implementation shows how it uses retry to re-run the
function acquiring a lock, and checks to see if it was already created to
handle this condition:

.. code-block:: python

    # kazoo.recipe.lock snippet

    def acquire(self):
        """Acquire the mutex, blocking until it is obtained"""
        try:
            self.client.retry(self._inner_acquire)
            self.is_acquired = True
        except KazooException:
            # if we did ultimately fail, attempt to clean up
            self._best_effort_cleanup()
            self.cancelled = False
            raise

    def _inner_acquire(self):
        self.wake_event.clear()

        # make sure our election parent node exists
        if not self.assured_path:
            self.client.ensure_path(self.path)

        node = None
        if self.create_tried:
            node = self._find_node()
        else:
            self.create_tried = True

        if not node:
            node = self.client.create(self.create_path, self.data,
                ephemeral=True, sequence=True)
            # strip off path to node
            node = node[len(self.path) + 1:]

`create_tried` records whether it has tried to create the node already in the
event the connection is lost before the node name is returned.

Custom Retries
--------------

Sometimes you may wish to have specific retry policies for a command or
set of commands that differes from the
:meth:`~kazoo.client.KazooClient.retry` method. You can manually create
a :class:`~kazoo.retry.KazooRetry` instance with the specific retry
policy you prefer:

.. code-block:: python

    from kazoo.retry import KazooRetry

    kr = KazooRetry(max_tries=3, ignore_expire=False)
    result = kr(client.get, "/some/path")

This will retry the ``client.get`` command up to 3 times, and raise a
session expiration if it occurs. You can also make an instance with the
default behavior that ignores session expiration during a retry.

Watchers
========

Kazoo can set watch functions on a node that can be triggered either when the
node has changed or when the children of the node change. This change to the
node or children can also be the node or its children being deleted.

Watchers can be set in two different ways, the first is the style that
Zookeeper supports by default for one-time watch events. These watch functions
will be called once by kazoo, and do *not* receive session events, unlike the
native Zookeeper watches. Using this style requires the watch function to be
passed to one of these methods:

* :meth:`~kazoo.client.KazooClient.get`
* :meth:`~kazoo.client.KazooClient.get_children`
* :meth:`~kazoo.client.KazooClient.exists`

A watch function passed to :meth:`~kazoo.client.KazooClient.get` or
:meth:`~kazoo.client.KazooClient.exists` will be called when the data on the
node changes or the node itself is deleted. It will be passed a
:class:`~kazoo.protocol.states.WatchedEvent` instance.

.. code-block:: python

    def my_func(event):
        # check to see what the children are now

    # Call my_func when the children change
    children = zk.get_children("/my/favorite/node", watch=my_func)

Kazoo includes a higher level API that watches for data and children
modifications that's easier to use as it doesn't require re-setting the watch
every time the event is triggered. It also passes in the data and
:class:`~kazoo.protocol.states.ZnodeStat` when watching a node or the list of
children when watching a nodes children. Watch functions registered with this
API will be called immediately and every time there's a change, or until the
function returns False. If `allow_session_lost` is set to `True`, then the
function will no longer be called if the session is lost.

The following methods provide this functionality:

* :class:`~kazoo.recipe.watchers.ChildrenWatch`
* :class:`~kazoo.recipe.watchers.DataWatch`

These classes are available directly on the :class:`~kazoo.client.KazooClient`
instance and don't require the client object to be passed in when used in this
manner. The instance returned by instantiating either of the classes can be
called directly allowing them to be used as decorators:

.. code-block:: python

    @zk.ChildrenWatch("/my/favorite/node")
    def watch_children(children):
        print("Children are now: %s" % children)
    # Above function called immediately, and from then on

    @zk.DataWatch("/my/favorite")
    def watch_node(data, stat):
        print("Version: %s, data: %s" % (stat.version, data.decode("utf-8")))

Transactions
============

.. versionadded:: 0.6

Zookeeper 3.4 and above supports the sending of multiple commands at
once that will be committed as a single atomic unit. Either they will
all succeed or they will all fail. The result of a transaction will be
a list of the success/failure results for each command in the
transaction.

.. code-block:: python

    transaction = zk.transaction()
    transaction.check('/node/a', version=3)
    transaction.create('/node/b', b"a value")
    results = transaction.commit()

The :meth:`~kazoo.client.KazooClient.transaction` method returns a
:class:`~kazoo.client.TransactionRequest` instance. It's methods may be
called to queue commands to be completed in the transaction. When the
transaction is ready to be sent, the
:meth:`~kazoo.client.TransactionRequest.commit` method on it is called.

In the example above, there's a command not available unless a
transaction is being used, `check`. This can check nodes for a specific
version, which could be used to make the transaction fail if a node
doesn't match a version that it should be at. In this case the node
`/node/a` must be at version 3 or `/node/b` will not be created.