| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
| |
DNS resolution errors were previously not retriable, this commit allows it by changing the value returned when the DNS resolution fails.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Without this patch, a single select event is processed by iteration in
the 'ConnectionHandler' event loop.
In a scenario where the client issues a large number of async requests
with an important amplification factor, e.g. 'get_children_async' on a
large node, it is possible for the 'select' operation to almost always
return a "response ready" socket--as the server is often able to
process, serialize and ship a new reponse while Kazoo processes the
previous one.
That response socket often (always?) ends up at the beginning of the
list returned by 'select'.
As only 'select_result[0]' is processed in the loop, this can cause
the client to ignore the "request ready" FD for a long time, during
which no requests or pings are sent.
In effect, asynchronously "browsing" a large tree of nodes can stretch
that duration to the point where it exceeds the timeout--causing the
client to lose its session.
This patch considers both descriptors after 'select', and also
arranges for pings to be sent in case it encounters an "unending"
stream of responses to requests which were sent earlier.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
This adds a simple recovery path in case an SSL connection receives
an SSL_WANT_READ or WRITE error. Either error can occur while
reading or writing. The error indicates that the underlying
operation should be retried after the socket is once again readable
or writable (per the error code).
Closes #618
Co-authored-by: James E. Blair <jeblair@redhat.com>
|
| |
|
|
|
|
| |
This is a faster and more idiomatic way of using itertools.chain.
Instead of computing all the items in the iterable and storing them
in memory, they are computed one-by-one and never stored as a huge
list. This can save on both runtime and memory space.
|
| |
|
|
|
|
| |
As of Python 3.8, "is" with a literal is a syntax warning because of the
confusion between equality and instance identity it represents.
Issue #607
|
| | |
|
| |
|
|
|
| |
Allows configurable multi-implementations cooperations in locks (e.g.
Zookeeper python & go clients contending for the same lock).
|
| | |
|
| |
|
|
| |
Resolves #601
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
With this patch, requests issued while the client is in the
'CONNECTING' state get queued instead of raising a misleading
'SessionExpiredError'.
This fixes https://github.com/python-zk/kazoo/issues/374, and brings
Kazoo more in line with the Java and C clients.
See the 'kazoo.client.KazooClient.state' documentation as well as
these discussions for more details:
https://github.com/python-zk/kazoo/pull/570#issuecomment-554798550
https://github.com/python-zk/kazoo/pull/583#issuecomment-586422386
|
| |
|
|
|
| |
Satisfy new Hound style/lint checks
Upgrade to latest 3.5.6/3.4.14 Zookeeper releases.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
* Install debian packages for KDC as part of Travis init.
* Setup a loopback mini KDC for running tests.
* Run SASL tests as part of Travis builds.
* Improve harness cluster to support:
* Reconfiguration when environment changes.
* Different JAAS configurations (DIGEST/GSSAPI).
* Moved SASL tests into own module, with specially configured harness.
* Bumped default timeout to 15 sec to mitigate false negatives on
Travis.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New retry logic takes a maximum percentage off the canonical backoff,
ensure gradual predictable retries timings while still having a
controlable amount of jitter (re-introducing the `max_jitter` parameter)
to avoids swarming client retries.
Fix regression introduced in 60366d2c7910fc833991fad8e04bbe33817c0544
where retry/backoff logic produced only whole second (integer) retry
delays.
This produced inadequate retries on first retry and would generally not
work on fast network where sub miliseconds retries are desired.
Additionally, with high `max_delay` setting, as the range was always
spanning from 0 until the last delay, it would also produce extremely
random results with short delays following longer ones which is contrary
to the expected backoff logic.
|
| |
|
|
|
|
|
|
|
| |
When connection attempts fail repeatedly (e.g. all ZK servers are
unavailable), eventually the socketpair in the ConnectionHandler fills
up, and the Client gets stuck trying to write a single byte to the
socketpair.
Avoid this by ensuring we close the socketpair on a failed connection
attempt.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Switching to xenial distrib to make python 3.7 available
* Testing against 3.5.5 instead of 3.5.4-beta and deploying new version for python 3.7 and zk 3.5.5
* Testing against zk 3.4.14 instead of 3.4.13
* Change pattern for slf4j-log4j lib
* Since ZOOKEEPER-3156 it is now required to be authed to access ACLs
* Drop support for ZK 3.3.x and Python 3.{4,5}
|
| |
|
| |
In order to be better in sync with the official documentation (https://zookeeper.apache.org/doc/r3.5.5/zookeeperProgrammers.html#ch_zkSessions) it is better to rename those variables to `close` instead of `lost`.
|
| |
|
|
|
|
|
|
| |
Avoid
```
File "/home/tests/kazoo/recipe/lock.py", line 341, in contenders
contenders.append(data.decode('utf-8'))
AttributeError: 'NoneType' object has no attribute 'decode'
```
|
| |
|
|
|
|
|
|
|
|
| |
Add parameters to setup SSL context options and ciphers when playing with secure connection.
It can be set via a handler:
```
class MySequentialThreadingHandler(SequentialThreadingHandler):
def create_connection(self, *args, **kwargs):
return create_tcp_connection(socket, options=MY_OPTIONS, ciphers=MY_CIPHERS,
*args, **kwargs)
```
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* feat(recipe): add support for curator SharedCount recipe
This feature allows Java clients using curator's SharedCount recipe
and python clients using kazoo's Counter recipe to read and
write from the same path without receiving type errors.
example use:
counter = zk.Counter("/curator", support_curator=True)
counter += 2
counter -= 1
counter.value == 1
counter.pre_value == 2
counter.post_value == 1
Closes #558
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Move SASL configuration out of auth_data into its own dictionary which exposes more SASL features (e.g. server service name, client principal...). Legacy syntax is still supported for backward compatibilty.
Remove SASL from auth_data and place it between 'connection' and 'zookeeper protocol level authentication' to simplify connection logic and bring code in line with the protocol stack (SASL wraps Zookeeper, not the other way around).
Consistent exception, `AuthFailedError`, raised during authentication failure between SASL and ZK authentication.
New 'SASLException' exception raised in case of SASL intrisinc failures.
Add support for GSSAPI (Kerberos).
Example connection using Digest-MD5:
client = KazooClient(
sasl_options={'mechanism': 'DIGEST-MD5',
'username': 'myusername',
'password': 'mypassword'}
)
Example connection using GSSAPI (with some optional settings):
client = KazooClient(
sasl_options={'mechanism': 'GSSAPI',
'service': 'myzk', # optional
'principal': 'clt@EXAMPLE.COM'} # optional
)
|
| | |
|
| |
|
|
|
|
| |
the handler has stopped running (#549)
This avoids zombie thread to appear when creating and closing the client right after. A new unit case is added.
|
| | |
|
| |
|
|
| |
Accept kazoo<=2.5.0 KazooRetry 'max_jitter' argument and display a
warning for backward compatibility.
|
| |
|
|
|
|
|
| |
This ensures that the watcher is removed from the client listener when the func given to ChildrenWatch returns False.
Previously, the watcher was never removed so the ChildrenWatch object would endlessly grow in memory. A unit test is added to ensure this case never happen again.
Fix #542
|
| |
|
|
|
| |
In the case of a zookeeper server under pressure, it will typically try to maintain the quorum rather than handling client requests. In this kind of case, the quorum is maintained, the connection works, but the client is frozen there.
Retrying after a shorter timeout means we can reconnect to another server before losing the session altogether.
|
| |
|
|
|
|
|
|
|
|
| |
fix(recipe): Fix memory leak of TreeCache recipe.
Fix memory leak on idle handler and on closed TreeCache.
Add new memory tests for TreeCache recipe that uses
objgraph and other tests for various handler on TreeCache.
Let TreeCache start in a safe way. The doc now suggest to
close unused TreeCache.
|
| | |
|
| |
|
|
|
|
|
|
|
| |
Previously, a gap between calls to `time.time()` could lead to a
situation where the current time was less than `end` during the
`while` condition, but it was greater than `end` when assigning
a value to `timeout_at`.
Add tests to ensure a socket.error is raised instead of passing a
nonpositive value as a timeout to socket.create_connection.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
Version of Zookeeper is upgraded from 3.5.2-alpha to 3.5.4-beta for automated tests.
Reconfig now needs superuser authentification, a test is added to cover this feature.
Additionnal configuration and jvm parameters can be added when initializing
ManagedZookeeper. This is needed to ensure reconfig's tests to pass and can be used
in the future for similar needs.
Closes #477
|
| |
|
|
|
|
|
|
|
| |
Fixes the bug introduced by PR #512 where it is not possible to
connect in RO mode using authentication. Since the SASL
authentification feature, the _session_fallback method should
be called only after the SASL authentification is done. A new
method is added to the ConnectionHandler class in order to
correctly use KeeperState.CONNECTED or KeeperState.CONNECTED_RO.
|
| |
|
| |
It is possible to race between processing a new addAuth request(which updates the client.auth_data set) and iterating through it during reconnect. To avoid set changes during iteration, make a copy.
|
| |
|
|
|
|
|
|
| |
This adds the possibility to connect to Zookeeper using DIGEST-MD5 SASL.
It uses the pure-sasl library to connect using SASL. In case the library
is missing, connection to Zookeeper will be done without any
authentification and a warning message will be displayed. Tests have
been added for this feature. Documentation also has been updated.
|
| |
|
|
|
|
|
|
|
|
| |
interval (#521)
The previous implementation would add a fixed amount of jitter around
the calculated back-off time. Retry attempts were thus clustered
around the exponentially spaced backoff points.
This patch does exponential backoff but uniformly spreads the retries
over an interval [0, backoff**attempt]
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* client: Allow SSL use when communicating with Zookeeper, fixes #382
Zookeeper 3.5 supports SSL for client communications, this commit
adds support for it on the Kazoo side.
Note that you need to give the client the key, certificate and CA
files.
Co-Authored-By: Monty Taylor <mordred@inaugust.com>
* Added keyfile password for ssl connection
* Added a way to bypass ssl certification validation
* Added a timeout when using SSL connection
|
| |
|
|
|
|
|
| |
Delete lock node if already entry already consumed
Fix #366
Related: #347 / #373
|
| | |
|
| |
|
|
|
| |
gevent.event.Timeout is undocumented and will break when using gevent 1.3b1+.
Use gevent.Timeout as suggested in gevent docs.
|
| |
|
|
|
|
| |
Due to the use of unbound lambdas in AsyncResult, multiple callbacks
wouldn't actually get called, it would only call the final callback N
times, where N is the number of registered callbacks.
|
| |
|
|
| |
fix a typo.
|
| |
|
|
|
|
|
|
|
|
| |
Right now if the program taking the lock exits, the lock is also
released implicitly as the zk node is ephemeral. In some usecases
its desirable to make the lock release explicit. For example, in
scripting multiple programs that contend for a lock, or purposeful
failing lock acquirers to detect issues.
The ephemeral flag in acquire() allows for this behavior.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The previous implementation of session watcher triggers blocked operations.
If there is a huge tree in ZooKeeper, reconnecting event will lead an
initialized TreeCache into a bad performance state, because the connection
routine was blocked by the session watcher of TreeCache.
This commit put those blocked operations into a background queue to fix this.
There is an example code snippet:
from kazoo.client import KazooClient
from kazoo.recipe.cache import TreeCache
client = KazooClient()
client.start()
cache = TreeCache(client, '/a-huge-tree')
cache.start()
# Wait the cache be initialized and trigger a connection lost event.
client.get_children('/') # The connection is still broken
The patch of this commit has been used in the production environment of
https://github.com/eleme.
|
| | |
|
| |
|
|
| |
This commit lets TreeCache do not use queue of connection routine any more.
|
| |
|
|
| |
Then every handlers have queue_impl and queue_empty as their attributes.
|
