summaryrefslogtreecommitdiff
path: root/kafka/coordinator/base.py
Commit message (Collapse)AuthorAgeFilesLines
* Fix #1985: fix consumer deadlock when heartbeat thread request timeout (#2064)huangcuiyang2020-09-071-8/+12
|
* remove unused importsJames Lamb2020-07-101-1/+0
|
* Log retriable coordinator NodeNotReady, TooManyInFlightRequests as debug not ↵Dana Powers2019-12-291-2/+5
| | | | error (#1975)
* Fix simple typo: managementment -> managementTim Gates2019-12-081-1/+1
| | | | Closes #1965
* Change coordinator lock acquisition order (#1821)Dana Powers2019-09-291-42/+36
|
* Wrap consumer.poll() for KafkaConsumer iteration (#1902)Dana Powers2019-09-281-1/+5
|
* Use dedicated connection for group coordinator (#1822)Dana Powers2019-06-191-3/+3
| | | This changes the coordinator_id to be a unique string, e.g., `coordinator-1`, so that it will get a dedicated connection. This won't eliminate lock contention because the client lock applies to all connections, but it should improve in-flight-request contention.
* Attempt to join heartbeat thread during close() (#1735)Dana Powers2019-03-131-3/+6
| | | | | | | Underlying issue here is a race on consumer.close() between the client, the connections/sockets, and the heartbeat thread. Although the heartbeat thread is signaled to close, we do not block for it. So when we go on to close the client and its underlying connections, if the heartbeat is still doing work it can cause errors/crashes if it attempts to access the now closed objects (selectors and/or sockets, primarily). So this commit adds a blocking thread join to the heartbeat close. This may cause some additional blocking time on consumer.close() while the heartbeat thread finishes. But it should be small in average case and in the worst case will be no longer than the heartbeat_timeout_ms (though if we timeout the join, race errors may still occur). Fix #1666
* Do network connections and writes in KafkaClient.poll() (#1729)Dana Powers2019-03-081-2/+2
| | | | | | * Add BrokerConnection.send_pending_requests to support async network sends * Send network requests during KafkaClient.poll() rather than in KafkaClient.send() * Dont acquire lock during KafkaClient.send if node is connected / ready * Move all network connection IO into KafkaClient.poll()
* Improve KafkaConsumer join group / only enable Heartbeat Thread during ↵Dana Powers2019-01-151-11/+23
| | | | stable group (#1695)
* (Attempt to) Fix deadlock between consumer and heartbeat (#1628)Dana Powers2018-11-101-1/+1
|
* Always acquire client lock before coordinator lock to avoid deadlocks (#1464)Dana Powers2018-04-181-59/+64
|
* Heartbeat thread start / closeDana Powers2018-03-231-2/+4
|
* Change levels for some heartbeat thread loggingDana Powers2018-03-231-3/+3
|
* Check for immediate failure when looking up coordinator in heartbeat thread ↵Dana Powers2018-03-231-1/+5
| | | | (#1457)
* KAFKA-5512; Awake the heartbeat thread when timetoNextHeartbeat is equal to ↵Dana Powers2018-03-101-0/+4
| | | | 0 (#1439)
* use absolute imports everywhere (#1362)Kevin Tindall2018-02-061-7/+7
|
* KAFKA-3949: Avoid race condition when subscription changes during rebalance ↵Dana Powers2018-02-021-11/+13
| | | | (#1364)
* Name heartbeat thread with group_id; use backoff when polling (#1345)Dana Powers2018-01-161-1/+1
|
* Fix coordinator join_future race condition (#1338)Dana Powers2018-01-111-4/+8
| | | * Fix race condition in coordinator join_future handling
* Improve KafkaConsumer cleanup (#1339)Dana Powers2018-01-101-11/+27
|
* KAFKA-3888 Use background thread to process consumer heartbeats (#1266)Dana Powers2017-12-211-238/+436
|
* Handle lookup_coordinator send failures (#1279)Dana Powers2017-10-241-5/+11
|
* KAFKA-4034: Avoid unnecessary consumer coordinator lookup (#1254)Dana Powers2017-10-111-4/+19
|
* Use for join-time-max and sync-time-max metrics Max() measure function (#1146)Alexey Pervushin2017-07-201-2/+2
|
* Backoff on unavailable group coordinator retry (#1125)Dana Powers2017-06-191-0/+2
|
* Additional docstrings for autocommit close optionDana Powers2017-03-131-2/+2
|
* Short-circuit group coordinator requests when NodeNotReady (#995)Dana Powers2017-03-091-0/+23
|
* Avoid unknown coordinator after client poll (#1023)Dana Powers2017-03-091-7/+6
|
* For 0.8.2, only attempt connection to coordinator if least_loaded_node succeedsDana Powers2017-03-071-1/+2
|
* Minor additional logging for consumer coordinatorDana Powers2017-03-061-0/+2
|
* Dont refresh metadata on failed group coordinator request unless needed (#1006)Dana Powers2017-03-031-6/+4
|
* Add sphinx formatting to hyperlink methods (#898)Jeff Widman2017-03-031-5/+5
|
* PEP-8: Spacing & removed unused imports (#899)Jeff Widman2017-02-091-6/+6
|
* Fix possible request draining in ensure_active_group (#896)Dana Powers2016-12-021-3/+6
|
* Always include an error for logging when the coordinator is marked dead (#890)Dana Powers2016-11-181-7/+7
|
* Drain pending requests to the coordinator before initiating group rejoin (#798)Dana Powers2016-08-041-0/+8
|
* Change coordinator.needs_join_prepare -> coordinator.rejoining; fix consumer ↵Dana Powers2016-08-041-4/+4
| | | | group test (#795)
* Treat metric_group_prefix as config in KafkaConsumerDana Powers2016-08-041-2/+3
|
* Vendor six 1.10.0sixDana Powers2016-08-011-1/+1
|
* Add base coordinator metricsDana Powers2016-07-171-70/+71
|
* Fix KafkaConsumer autocommit for 0.8 brokers (#756 / #706)Dana Powers2016-07-161-0/+9
| | | | * Dont wait for group join to enable AutoCommitTask if broker version < 0.9 * For zookeeper offset storage, set a "coordinator" with least_loaded_node
* Avoid some exceptions in Coordinator.__del__ (#668)Dana Powers2016-04-251-1/+2
|
* Changing some commented out metrics to avoid future gotchas.Zack Dever2016-04-131-3/+3
|
* Default sleep=True in client.pollDana Powers2016-04-081-1/+1
|
* Can no longer have coordinator_id if coordinator_unknown()Dana Powers2016-04-081-6/+0
|
* Use version-indexed lists for request/response protocol structsprotocol_versionsDana Powers2016-04-051-15/+14
|
* KAFKA-3318: clean up consumer logging and error messagesDana Powers2016-04-051-36/+42
|
* Update imports from kafka.common -> kafka.errors / kafka.structsDana Powers2016-04-051-1/+1
|
* Improve Coordinator Error handlingDana Powers2016-04-041-8/+14
| | | | | | | - coordinator_unknown() should not check ready(node_id) - dont mark coordinator dead on not-ready or inflight-requests errors - improve coordinator and heartbeat logging - dont gate group metadata requests on ready() check [conn handles now]
View Lorry Controller Status