summaryrefslogtreecommitdiff
path: root/kafka/coordinator
Commit message (Collapse)AuthorAgeFilesLines
* Cover sticky assignor's metadata method with tests (#2161)Valeria Chernenko2020-11-051-3/+7
|
* Use six.viewitems instead of six.iteritems to avoid encoding problems (#2154)Valeria Chernenko2020-10-301-1/+1
|
* KIP-54: Implement sticky partition assignment strategy (#2057)Valeria Chernenko2020-09-295-1/+897
|
* Fix #1985: fix consumer deadlock when heartbeat thread request timeout (#2064)huangcuiyang2020-09-071-8/+12
|
* remove unused importsJames Lamb2020-07-101-1/+0
|
* Optionally return OffsetAndMetadata from consumer.committed(tp) (#1979)Dana Powers2019-12-291-3/+3
|
* Log retriable coordinator NodeNotReady, TooManyInFlightRequests as debug not ↵Dana Powers2019-12-291-2/+5
| | | | error (#1975)
* Fix simple typo: managementment -> managementTim Gates2019-12-081-1/+1
| | | | Closes #1965
* Fix typosCarson Ip2019-11-081-1/+1
|
* Issue #1780 - Consumer hang indefinitely in fetcher._retrieve_offsets() due ↵Commander Dishwasher2019-09-301-1/+5
| | | | to topic deletion while rebalancing (#1782)
* Change coordinator lock acquisition order (#1821)Dana Powers2019-09-291-42/+36
|
* Wrap consumer.poll() for KafkaConsumer iteration (#1902)Dana Powers2019-09-281-1/+5
|
* Allow the coordinator to auto-commit for all api_version.Jay Chan2019-06-201-1/+1
|
* Use dedicated connection for group coordinator (#1822)Dana Powers2019-06-191-3/+3
| | | This changes the coordinator_id to be a unique string, e.g., `coordinator-1`, so that it will get a dedicated connection. This won't eliminate lock contention because the client lock applies to all connections, but it should improve in-flight-request contention.
* A little python cleanup (#1805)Jeff Widman2019-05-171-4/+2
| | | | | | 1. Remove unused variable: `partitions_for_topic` 2. No need to cast to list as `sorted()` already returns a list 3. Using `enumerate()` is cleaner than `range(len())` and handles assigning `member`
* Attempt to join heartbeat thread during close() (#1735)Dana Powers2019-03-131-3/+6
| | | | | | | Underlying issue here is a race on consumer.close() between the client, the connections/sockets, and the heartbeat thread. Although the heartbeat thread is signaled to close, we do not block for it. So when we go on to close the client and its underlying connections, if the heartbeat is still doing work it can cause errors/crashes if it attempts to access the now closed objects (selectors and/or sockets, primarily). So this commit adds a blocking thread join to the heartbeat close. This may cause some additional blocking time on consumer.close() while the heartbeat thread finishes. But it should be small in average case and in the worst case will be no longer than the heartbeat_timeout_ms (though if we timeout the join, race errors may still occur). Fix #1666
* Ignore lookup_coordinator result in commit_offsets_async (#1712)Faqa2019-03-121-1/+2
|
* Do network connections and writes in KafkaClient.poll() (#1729)Dana Powers2019-03-081-2/+2
| | | | | | * Add BrokerConnection.send_pending_requests to support async network sends * Send network requests during KafkaClient.poll() rather than in KafkaClient.send() * Dont acquire lock during KafkaClient.send if node is connected / ready * Move all network connection IO into KafkaClient.poll()
* Improve KafkaConsumer join group / only enable Heartbeat Thread during ↵Dana Powers2019-01-151-11/+23
| | | | stable group (#1695)
* Be explicit with tuples for %s formattingJeff Widman2018-11-181-3/+3
| | | | Fix #1633
* (Attempt to) Fix deadlock between consumer and heartbeat (#1628)Dana Powers2018-11-101-1/+1
|
* Return future from commit_offsets_async (#1560)Mike Lang2018-08-311-1/+6
|
* Don't use `kafka.common` internally1.3.5Jeff Widman2018-06-052-2/+2
| | | | This finishes the split from `kafka.common` to `kafka.errors`/`kafka.structs`.
* Always acquire client lock before coordinator lock to avoid deadlocks (#1464)Dana Powers2018-04-181-59/+64
|
* Heartbeat thread start / closeDana Powers2018-03-231-2/+4
|
* Change levels for some heartbeat thread loggingDana Powers2018-03-231-3/+3
|
* Check for immediate failure when looking up coordinator in heartbeat thread ↵Dana Powers2018-03-231-1/+5
| | | | (#1457)
* KAFKA-5512; Awake the heartbeat thread when timetoNextHeartbeat is equal to ↵Dana Powers2018-03-101-0/+4
| | | | 0 (#1439)
* Remove old CommitFailed error message from coordinator (#1436)Dana Powers2018-03-101-11/+1
|
* Fix consumer poll stuck error when no available partition (#1375)Yu Kou2018-02-081-1/+1
|
* use absolute imports everywhere (#1362)Kevin Tindall2018-02-064-23/+23
|
* KAFKA-3949: Avoid race condition when subscription changes during rebalance ↵Dana Powers2018-02-022-50/+76
| | | | (#1364)
* Name heartbeat thread with group_id; use backoff when polling (#1345)Dana Powers2018-01-161-1/+1
|
* Fix coordinator join_future race condition (#1338)Dana Powers2018-01-111-4/+8
| | | * Fix race condition in coordinator join_future handling
* Improve KafkaConsumer cleanup (#1339)Dana Powers2018-01-102-11/+28
|
* KAFKA-3888 Use background thread to process consumer heartbeats (#1266)Dana Powers2017-12-213-350/+615
|
* Replace periods (java) with underscores (python)Jeff Widman2017-11-071-2/+2
|
* Handle lookup_coordinator send failures (#1279)Dana Powers2017-10-241-5/+11
|
* KAFKA-4034: Avoid unnecessary consumer coordinator lookup (#1254)Dana Powers2017-10-112-8/+43
|
* Explicitly check for `None` rather than FalseJeff Widman2017-10-061-1/+1
| | | | | If the group leader somehow gets in a state that it has an empty partition assignment, then `self._assignment_snapshot` will be `{}` which evaluates to `False`. So `self._subscription.mark_for_reassignment()` will never be triggered, even if `self._assignment_snapshot != self._metadata_snapshot`. Fixes the symptoms of https://github.com/dpkp/kafka-python/issues/1237 although I suspect there's an additional bug in that case that triggers the condition of the the group leader getting an empty partition assignment.
* Initialize metadata_snapshot in group coordinator (#1174)Dana Powers2017-08-131-6/+9
|
* Use for join-time-max and sync-time-max metrics Max() measure function (#1146)Alexey Pervushin2017-07-201-2/+2
|
* change_subscription called only when necessary (#1132)Petr Šebek2017-07-071-2/+3
| | | | | | When we are using subscription by pattern change subscription is called every metadata update even when nothing changes. This PR ensures that change subscription is called only when set of topics changes.
* Backoff on unavailable group coordinator retry (#1125)Dana Powers2017-06-191-0/+2
|
* Additional docstrings for autocommit close optionDana Powers2017-03-132-2/+10
|
* Optionally skip auto-commit during consumer.close (#1031)Dana Powers2017-03-131-2/+3
|
* Short-circuit group coordinator requests when NodeNotReady (#995)Dana Powers2017-03-091-0/+23
|
* Avoid unknown coordinator after client poll (#1023)Dana Powers2017-03-091-7/+6
|
* For 0.8.2, only attempt connection to coordinator if least_loaded_node succeedsDana Powers2017-03-071-1/+2
|
* Minor additional logging for consumer coordinatorDana Powers2017-03-061-0/+2
|