<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/python-packages/kafka-python.git/kafka/coordinator, branch benchmark_fixups</title>
<subtitle>github.com: mumrah/kafka-python.git
</subtitle>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/'/>
<entry>
<title>Attempt to join heartbeat thread during close() (#1735)</title>
<updated>2019-03-14T01:46:28+00:00</updated>
<author>
<name>Dana Powers</name>
<email>dana.powers@gmail.com</email>
</author>
<published>2019-03-14T01:46:28+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=921c553b6a62a34044e4ae444af65abea3717faa'/>
<id>921c553b6a62a34044e4ae444af65abea3717faa</id>
<content type='text'>
Underlying issue here is a race on consumer.close() between the client, the connections/sockets, and the heartbeat thread. Although the heartbeat thread is signaled to close, we do not block for it. So when we go on to close the client and its underlying connections, if the heartbeat is still doing work it can cause errors/crashes if it attempts to access the now closed objects (selectors and/or sockets, primarily).

So this commit adds a blocking thread join to the heartbeat close. This may cause some additional blocking time on consumer.close() while the heartbeat thread finishes. But it should be small in average case and in the worst case will be no longer than the heartbeat_timeout_ms (though if we timeout the join, race errors may still occur).

Fix #1666</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Underlying issue here is a race on consumer.close() between the client, the connections/sockets, and the heartbeat thread. Although the heartbeat thread is signaled to close, we do not block for it. So when we go on to close the client and its underlying connections, if the heartbeat is still doing work it can cause errors/crashes if it attempts to access the now closed objects (selectors and/or sockets, primarily).

So this commit adds a blocking thread join to the heartbeat close. This may cause some additional blocking time on consumer.close() while the heartbeat thread finishes. But it should be small in average case and in the worst case will be no longer than the heartbeat_timeout_ms (though if we timeout the join, race errors may still occur).

Fix #1666</pre>
</div>
</content>
</entry>
<entry>
<title>Ignore lookup_coordinator result in commit_offsets_async (#1712)</title>
<updated>2019-03-13T04:32:21+00:00</updated>
<author>
<name>Faqa</name>
<email>asafflesch@gmail.com</email>
</author>
<published>2019-03-13T04:32:21+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=994d2838fdc77cc8b0840db0679cdbb0a0fb487b'/>
<id>994d2838fdc77cc8b0840db0679cdbb0a0fb487b</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Do network connections and writes in KafkaClient.poll() (#1729)</title>
<updated>2019-03-08T16:01:48+00:00</updated>
<author>
<name>Dana Powers</name>
<email>dana.powers@gmail.com</email>
</author>
<published>2019-03-08T16:01:48+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=8c0792581d8a38822c01b40f5d3926c659b0c439'/>
<id>8c0792581d8a38822c01b40f5d3926c659b0c439</id>
<content type='text'>
* Add BrokerConnection.send_pending_requests to support async network sends
* Send network requests during KafkaClient.poll() rather than in KafkaClient.send()
* Dont acquire lock during KafkaClient.send if node is connected / ready
* Move all network connection IO into KafkaClient.poll()</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Add BrokerConnection.send_pending_requests to support async network sends
* Send network requests during KafkaClient.poll() rather than in KafkaClient.send()
* Dont acquire lock during KafkaClient.send if node is connected / ready
* Move all network connection IO into KafkaClient.poll()</pre>
</div>
</content>
</entry>
<entry>
<title>Improve KafkaConsumer join group / only enable Heartbeat Thread during stable group (#1695)</title>
<updated>2019-01-15T16:08:45+00:00</updated>
<author>
<name>Dana Powers</name>
<email>dana.powers@gmail.com</email>
</author>
<published>2019-01-15T16:08:45+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=a4f0cb881e8cb71f285ae802aecbf716bfe71d01'/>
<id>a4f0cb881e8cb71f285ae802aecbf716bfe71d01</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Be explicit with tuples for %s formatting</title>
<updated>2018-11-18T08:21:18+00:00</updated>
<author>
<name>Jeff Widman</name>
<email>jeff@jeffwidman.com</email>
</author>
<published>2018-11-13T19:57:45+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=1d443638e22c2d360086b8d7cee8b5d930741d12'/>
<id>1d443638e22c2d360086b8d7cee8b5d930741d12</id>
<content type='text'>
Fix #1633
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Fix #1633
</pre>
</div>
</content>
</entry>
<entry>
<title>(Attempt to) Fix deadlock between consumer and heartbeat (#1628)</title>
<updated>2018-11-10T20:45:01+00:00</updated>
<author>
<name>Dana Powers</name>
<email>dana.powers@gmail.com</email>
</author>
<published>2018-11-10T20:45:01+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=0a2ccba3cb1b8636f615a30821123720773a8dfa'/>
<id>0a2ccba3cb1b8636f615a30821123720773a8dfa</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Return future from commit_offsets_async (#1560)</title>
<updated>2018-08-31T13:11:23+00:00</updated>
<author>
<name>Mike Lang</name>
<email>ekimekim@users.noreply.github.com</email>
</author>
<published>2018-08-31T13:11:23+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=5a04bc78f3392038733d65fc1e4830c8b14cd6fd'/>
<id>5a04bc78f3392038733d65fc1e4830c8b14cd6fd</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Don't use `kafka.common` internally</title>
<updated>2018-06-05T21:32:01+00:00</updated>
<author>
<name>Jeff Widman</name>
<email>jeff@jeffwidman.com</email>
</author>
<published>2018-05-28T22:58:26+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=bc4cc434cddf403a35d0393d68ecfdbfad17c8e5'/>
<id>bc4cc434cddf403a35d0393d68ecfdbfad17c8e5</id>
<content type='text'>
This finishes the split from `kafka.common` to `kafka.errors`/`kafka.structs`.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This finishes the split from `kafka.common` to `kafka.errors`/`kafka.structs`.
</pre>
</div>
</content>
</entry>
<entry>
<title>Always acquire client lock before coordinator lock to avoid deadlocks (#1464)</title>
<updated>2018-04-18T15:29:19+00:00</updated>
<author>
<name>Dana Powers</name>
<email>dana.powers@gmail.com</email>
</author>
<published>2018-04-18T15:29:19+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=1c71dfc3c321372c808f45f569ae41352f420e8f'/>
<id>1c71dfc3c321372c808f45f569ae41352f420e8f</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Heartbeat thread start / close</title>
<updated>2018-03-23T20:54:12+00:00</updated>
<author>
<name>Dana Powers</name>
<email>dana.powers@gmail.com</email>
</author>
<published>2018-03-22T22:10:40+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=4c87d11c26e2aa5a60de0b2213dd8caa3b16d553'/>
<id>4c87d11c26e2aa5a60de0b2213dd8caa3b16d553</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
</feed>
