Commit message (Collapse) | Author | Age | Files | Lines | ||
---|---|---|---|---|---|---|
... | ||||||
* | Do not call state_change_callback with lock (#1775) | Dana Powers | 2019-04-02 | 2 | -21/+29 | |
| | ||||||
* | Additional BrokerConnection locks to synchronize protocol/IFR state (#1768) | Dana Powers | 2019-04-02 | 1 | -61/+85 | |
| | ||||||
* | Return connection state explicitly after close in connect() (#1778) | Dana Powers | 2019-04-02 | 1 | -1/+3 | |
| | ||||||
* | Reset reconnect backoff on SSL connection (#1777) | Dana Powers | 2019-04-02 | 1 | -0/+1 | |
| | ||||||
* | Fix possible AttribueError during conn._close_socket (#1776) | Dana Powers | 2019-04-01 | 1 | -1/+1 | |
| | ||||||
* | Dont treat popped conn.close() as failure in state change callback (#1773) | Dana Powers | 2019-04-01 | 1 | -3/+10 | |
| | ||||||
* | Avoid race condition on client._conns in send() (#1772) | Dana Powers | 2019-03-31 | 1 | -2/+3 | |
| | | | There was a very small possibility that between checking `self._can_send_request(node_id)` and grabbing the connection object via `self._conns[node_id]` that the connection could get closed / recycled / removed from _conns and cause a KeyError. This PR should prevent such a KeyError. In the case where the connection is disconnected by the time we call send(), we should expect conn.send() simply to fail the request. | |||||
* | lock client.check_version (#1771) | Dana Powers | 2019-03-31 | 1 | -0/+5 | |
| | ||||||
* | Dont wakeup during maybe_refresh_metadata -- it is only called by poll() (#1769) | Dana Powers | 2019-03-30 | 1 | -4/+4 | |
| | ||||||
* | Revert 703f0659 / fix 0.8.2 protocol quick detection (#1763) | Dana Powers | 2019-03-27 | 2 | -6/+9 | |
| | ||||||
* | Send pending requests before waiting for responses (#1762) | Dana Powers | 2019-03-27 | 1 | -2/+4 | |
| | ||||||
* | Dont do client wakeup when sending from sender thread (#1761) | Dana Powers | 2019-03-24 | 2 | -6/+10 | |
| | ||||||
* | Update sasl configuration docstrings | Dana Powers | 2019-03-23 | 5 | -24/+24 | |
| | ||||||
* | Support SASL OAuthBearer Authentication (#1750) | Phong Pham | 2019-03-22 | 7 | -6/+117 | |
| | ||||||
* | Maintain shadow cluster metadata for bootstrapping (#1753) | Dana Powers | 2019-03-21 | 2 | -35/+21 | |
| | ||||||
* | Allow configuration of SSL Ciphers (#1755) | Dana Powers | 2019-03-21 | 4 | -1/+28 | |
| | ||||||
* | Wrap SSL sockets after connecting (#1754) | Dana Powers | 2019-03-21 | 1 | -19/+11 | |
| | ||||||
* | Fix race condition in protocol.send_bytes (#1752) | Filip Stefanak | 2019-03-21 | 1 | -1/+2 | |
| | ||||||
* | Bump version for development | Dana Powers | 2019-03-14 | 1 | -1/+1 | |
| | ||||||
* | Release 1.4.51.4.5 | Dana Powers | 2019-03-14 | 1 | -1/+1 | |
| | ||||||
* | Error if connections_max_idle_ms not larger than request_timeout_ms (#1688) | Jeff Widman | 2019-03-14 | 1 | -3/+7 | |
| | ||||||
* | Retry bootstrapping after backoff when necessary (#1736) | Dana Powers | 2019-03-14 | 2 | -84/+89 | |
| | | | | | | | The current client attempts to bootstrap once during initialization, but if it fails there is no second attempt and the client will be inoperable. This can happen, for example, if an entire cluster is down at the time a long-running client starts execution. This commit attempts to fix this by removing the synchronous bootstrapping from `KafkaClient` init, and instead merges bootstrap metadata with the cluster metadata. The Java client uses a similar approach. This allows us to continue falling back to bootstrap data when necessary throughout the life of a long-running consumer or producer. Fix #1670 | |||||
* | Fix default protocol parser version | Dana Powers | 2019-03-13 | 1 | -0/+3 | |
| | ||||||
* | Minor updates to client_async.py | Dana Powers | 2019-03-13 | 1 | -4/+4 | |
| | ||||||
* | Recheck connecting nodes sooner when refreshing metadata (#1737) | Dana Powers | 2019-03-13 | 1 | -3/+1 | |
| | ||||||
* | Don't recheck version if api_versions data is already cached (#1738) | Dana Powers | 2019-03-13 | 1 | -0/+3 | |
| | | | I noticed during local testing that version probing was happening twice when connecting to newer broker versions. This was because we call check_version() once explicitly, and then again implicitly within get_api_versions(). But once we have _api_versions data cached, we can just return it and avoid probing versions a second time. | |||||
* | Attempt to join heartbeat thread during close() (#1735) | Dana Powers | 2019-03-13 | 1 | -3/+6 | |
| | | | | | | | Underlying issue here is a race on consumer.close() between the client, the connections/sockets, and the heartbeat thread. Although the heartbeat thread is signaled to close, we do not block for it. So when we go on to close the client and its underlying connections, if the heartbeat is still doing work it can cause errors/crashes if it attempts to access the now closed objects (selectors and/or sockets, primarily). So this commit adds a blocking thread join to the heartbeat close. This may cause some additional blocking time on consumer.close() while the heartbeat thread finishes. But it should be small in average case and in the worst case will be no longer than the heartbeat_timeout_ms (though if we timeout the join, race errors may still occur). Fix #1666 | |||||
* | 1701 use last offset from fetch v4 if available (#1724) | Keith So | 2019-03-13 | 3 | -0/+28 | |
| | ||||||
* | Catch thrown OSError by python 3.7 when creating a connection (#1694) | Daniel Johansson | 2019-03-12 | 1 | -0/+3 | |
| | ||||||
* | Ignore lookup_coordinator result in commit_offsets_async (#1712) | Faqa | 2019-03-12 | 1 | -1/+2 | |
| | ||||||
* | Synchronize puts to KafkaConsumer protocol buffer during async sends | Dana Powers | 2019-03-12 | 1 | -21/+36 | |
| | ||||||
* | Do network connections and writes in KafkaClient.poll() (#1729) | Dana Powers | 2019-03-08 | 4 | -49/+76 | |
| | | | | | | * Add BrokerConnection.send_pending_requests to support async network sends * Send network requests during KafkaClient.poll() rather than in KafkaClient.send() * Dont acquire lock during KafkaClient.send if node is connected / ready * Move all network connection IO into KafkaClient.poll() | |||||
* | Do not require client lock for read-only operations (#1730) | Dana Powers | 2019-03-06 | 1 | -50/+50 | |
| | | | In an effort to reduce the surface area of lock coordination, and thereby hopefully reduce lock contention, I think we can remove locking from the read-only KafkaClient methods: connected, is_disconnected, in_flight_request_count, and least_loaded_node . Given that the read data could change after the lock is released but before the caller uses it, the value of acquiring a lock here does not seem high to me. | |||||
* | Make NotEnoughReplicasError/NotEnoughReplicasAfterAppendError retriable (#1722) | le-linh | 2019-03-03 | 1 | -0/+2 | |
| | ||||||
* | Remove unused import | Jeff Widman | 2019-01-28 | 1 | -1/+0 | |
| | ||||||
* | Improve KafkaConsumer join group / only enable Heartbeat Thread during ↵ | Dana Powers | 2019-01-15 | 1 | -11/+23 | |
| | | | | stable group (#1695) | |||||
* | Remove unused `skip_double_compressed_messages` | Jeff Widman | 2019-01-13 | 2 | -16/+0 | |
| | | | | | | | | | | This `skip_double_compressed_messages` flag was added in https://github.com/dpkp/kafka-python/pull/755 in order to fix https://github.com/dpkp/kafka-python/issues/718. However, grep'ing through the code, it looks like it this is no longer used anywhere and doesn't do anything. So removing it. | |||||
* | Timeout all unconnected conns (incl SSL) after request_timeout_ms | Dana Powers | 2019-01-13 | 1 | -6/+8 | |
| | ||||||
* | Fix `AttributeError` caused by `getattr()` | Jeff Widman | 2019-01-07 | 1 | -1/+2 | |
| | | | | | | | `getattr(object, 'x', object.y)` will evaluate the default argument `object.y` regardless of whether `'x'` exists. For details see: https://stackoverflow.com/q/31443989/770425 | |||||
* | Fix SSL connection testing in Python 3.7 | Ben Weir | 2019-01-03 | 1 | -0/+7 | |
| | ||||||
* | Fix response error checking in KafkaAdminClient send_to_controller | Dana Powers | 2019-01-03 | 2 | -5/+15 | |
| | | | | | | | | | | | | Previously we weren't accounting for when the response tuple also has a `error_message` value. Note that in Java, the error fieldname is inconsistent: - `CreateTopicsResponse` / `CreatePartitionsResponse` uses `topic_errors` - `DeleteTopicsResponse` uses `topic_error_codes` So this updates the `CreateTopicsResponse` classes to match. The fix is a little brittle, but should suffice for now. | |||||
* | #1681 add copy() in metrics() to avoid thread safety issues (#1682) | Tosi Émeric | 2018-12-27 | 2 | -4/+4 | |
| | ||||||
* | Bugfix: Types need identity comparison | Jeff Widman | 2018-12-13 | 1 | -1/+1 | |
| | | | `isinstance()` won't work here, as the types require identity comparison. | |||||
* | Bump version for development | Dana Powers | 2018-11-20 | 1 | -1/+1 | |
| | ||||||
* | Release 1.4.41.4.4 | Dana Powers | 2018-11-20 | 1 | -1/+1 | |
| | ||||||
* | Rename KafkaAdmin to KafkaAdminClient | Jeff Widman | 2018-11-20 | 3 | -20/+20 | |
| | ||||||
* | Break KafkaClient poll if closed | Dana Powers | 2018-11-20 | 1 | -0/+2 | |
| | ||||||
* | Add protocols for {Describe,Create,Delete} Acls | Ulrik Johansson | 2018-11-19 | 1 | -0/+185 | |
| | ||||||
* | Bugfix: Always set this_groups_coordinator_id | Jeff Widman | 2018-11-19 | 1 | -1/+3 | |
| | ||||||
* | Various docstring / pep8 / code hygiene cleanups | Jeff Widman | 2018-11-18 | 1 | -71/+86 | |
| |