<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/python-packages/kafka-python.git/kafka, branch differentiate-between-client-and-broker-version-errors</title>
<subtitle>github.com: mumrah/kafka-python.git
</subtitle>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/'/>
<entry>
<title>Differentiate between client and broker version errors</title>
<updated>2018-11-20T19:03:46+00:00</updated>
<author>
<name>Jeff Widman</name>
<email>jeff@jeffwidman.com</email>
</author>
<published>2018-11-20T18:56:32+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=574c56ac7c25e481d775e54f621f73de95741dd6'/>
<id>574c56ac7c25e481d775e54f621f73de95741dd6</id>
<content type='text'>
While both indicate a mismatch between the broker and client, we want to
be able to differentiate between whether the error is thrown by the
client vs the broker.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
While both indicate a mismatch between the broker and client, we want to
be able to differentiate between whether the error is thrown by the
client vs the broker.
</pre>
</div>
</content>
</entry>
<entry>
<title>Rename KafkaAdmin to KafkaAdminClient</title>
<updated>2018-11-20T17:24:50+00:00</updated>
<author>
<name>Jeff Widman</name>
<email>jeff@jeffwidman.com</email>
</author>
<published>2018-11-20T17:03:50+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=fcc800f96f14192c44b09d1d37108377dcaed245'/>
<id>fcc800f96f14192c44b09d1d37108377dcaed245</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Break KafkaClient poll if closed</title>
<updated>2018-11-20T16:20:17+00:00</updated>
<author>
<name>Dana Powers</name>
<email>dana.powers@gmail.com</email>
</author>
<published>2018-11-20T16:20:17+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=2c77c8d73c3bb81011b3440b1f0308fd48d0e627'/>
<id>2c77c8d73c3bb81011b3440b1f0308fd48d0e627</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Add protocols for {Describe,Create,Delete} Acls</title>
<updated>2018-11-20T00:59:49+00:00</updated>
<author>
<name>Ulrik Johansson</name>
<email>ulrik.johansson@blocket.se</email>
</author>
<published>2018-11-15T22:59:05+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=66754624e84da89d94f6bc8dd55bd969ad520ee4'/>
<id>66754624e84da89d94f6bc8dd55bd969ad520ee4</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Bugfix: Always set this_groups_coordinator_id</title>
<updated>2018-11-19T19:08:05+00:00</updated>
<author>
<name>Jeff Widman</name>
<email>jeff@jeffwidman.com</email>
</author>
<published>2018-11-19T19:04:51+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=89243509b28d744ab26e8b67e1e2c9b74b4d902f'/>
<id>89243509b28d744ab26e8b67e1e2c9b74b4d902f</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Various docstring / pep8 / code hygiene cleanups</title>
<updated>2018-11-19T00:47:38+00:00</updated>
<author>
<name>Jeff Widman</name>
<email>jeff@jeffwidman.com</email>
</author>
<published>2018-11-17T12:11:52+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=24f41315889c23a5ea7d0ae26c3bbd8d68ae062c'/>
<id>24f41315889c23a5ea7d0ae26c3bbd8d68ae062c</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Fix describe_groups</title>
<updated>2018-11-18T23:37:36+00:00</updated>
<author>
<name>Jeff Widman</name>
<email>jeff@jeffwidman.com</email>
</author>
<published>2018-11-17T10:46:18+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=8dab14b6d73d8f1717fdeb46c79807827169fd2d'/>
<id>8dab14b6d73d8f1717fdeb46c79807827169fd2d</id>
<content type='text'>
This was completely broken previously because it didn't lookup the group
coordinator of the consumer group. Also added basic error
handling/raising.

Note:
I added the `group_coordinator_id` as an optional kwarg. As best I
can tell, the Java client doesn't include this and instead looks it up
every time. However, if we add this, it allows the caller the
flexibility to bypass the network round trip of the lookup if for some
reason they already know the `group_coordinator_id`.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This was completely broken previously because it didn't lookup the group
coordinator of the consumer group. Also added basic error
handling/raising.

Note:
I added the `group_coordinator_id` as an optional kwarg. As best I
can tell, the Java client doesn't include this and instead looks it up
every time. However, if we add this, it allows the caller the
flexibility to bypass the network round trip of the lookup if for some
reason they already know the `group_coordinator_id`.
</pre>
</div>
</content>
</entry>
<entry>
<title>Set a clear default value for `validate_only`/`include_synonyms`</title>
<updated>2018-11-18T23:33:56+00:00</updated>
<author>
<name>Jeff Widman</name>
<email>jeff@jeffwidman.com</email>
</author>
<published>2018-11-17T11:19:09+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=665f1e4dc4f809e4b70af71b1995d2fe6ed12a45'/>
<id>665f1e4dc4f809e4b70af71b1995d2fe6ed12a45</id>
<content type='text'>
Set a clear default value for `validate_only` / `include_synonyms`

Previously the kwarg defaulted to `None`, but then sent a `False` so this
makes it more explicit and reduces ambiguity.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Set a clear default value for `validate_only` / `include_synonyms`

Previously the kwarg defaulted to `None`, but then sent a `False` so this
makes it more explicit and reduces ambiguity.
</pre>
</div>
</content>
</entry>
<entry>
<title>Fix list_consumer_groups() to query all brokers</title>
<updated>2018-11-18T23:33:02+00:00</updated>
<author>
<name>Jeff Widman</name>
<email>jeff@jeffwidman.com</email>
</author>
<published>2018-11-13T21:10:19+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=232a2d6b428069722fd7f4dc9fba77e76f418594'/>
<id>232a2d6b428069722fd7f4dc9fba77e76f418594</id>
<content type='text'>
Previously, this only queried the controller. In actuality, the Kafka
protocol requires that the client query all brokers in order to get the
full list of consumer groups.

Note: The Java code (as best I can tell) doesn't allow limiting this to
specific brokers. And on the surface, this makes sense... you typically
don't care about specific brokers.

However, the inverse is true... consumer groups care about knowing their
group coordinator so they don't have to repeatedly query to find it.

In fact, a Kafka broker will only return the groups that it's a
coordinator for. While this is an implementation detail that is not
guaranteed by the upstream broker code, and technically should not be
relied upon, I think it very unlikely to change.

So monitoring scripts that fetch the offsets or describe the consumers
groups of all groups in the cluster can simply issue one call per broker
to identify all the coordinators, rather than having to issue one call
per consumer group. For an ad-hoc script this doesn't matter, but for
a monitoring script that runs every couple of minutes, this can be a big
deal. I know in the situations where I will use this, this matters more
to me than the risk of the interface unexpectedly breaking.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Previously, this only queried the controller. In actuality, the Kafka
protocol requires that the client query all brokers in order to get the
full list of consumer groups.

Note: The Java code (as best I can tell) doesn't allow limiting this to
specific brokers. And on the surface, this makes sense... you typically
don't care about specific brokers.

However, the inverse is true... consumer groups care about knowing their
group coordinator so they don't have to repeatedly query to find it.

In fact, a Kafka broker will only return the groups that it's a
coordinator for. While this is an implementation detail that is not
guaranteed by the upstream broker code, and technically should not be
relied upon, I think it very unlikely to change.

So monitoring scripts that fetch the offsets or describe the consumers
groups of all groups in the cluster can simply issue one call per broker
to identify all the coordinators, rather than having to issue one call
per consumer group. For an ad-hoc script this doesn't matter, but for
a monitoring script that runs every couple of minutes, this can be a big
deal. I know in the situations where I will use this, this matters more
to me than the risk of the interface unexpectedly breaking.
</pre>
</div>
</content>
</entry>
<entry>
<title>Add list_consumer_group_offsets()</title>
<updated>2018-11-18T23:29:16+00:00</updated>
<author>
<name>Jeff Widman</name>
<email>jeff@jeffwidman.com</email>
</author>
<published>2018-11-17T10:53:08+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/kafka-python.git/commit/?id=cc8e91426907f8ccadd60eedc4dc53b8729a84ec'/>
<id>cc8e91426907f8ccadd60eedc4dc53b8729a84ec</id>
<content type='text'>
Support fetching the offsets of a consumer group.

Note: As far as I can tell (the Java code is a little inscrutable), the
Java AdminClient doesn't allow specifying the `coordinator_id` or the
`partitions`.

But I decided to include them because they provide a lot of additional
flexibility:

1. allowing users to specify the partitions allows this method to be used even for
older brokers that don't support the OffsetFetchRequest_v2

2. allowing users to specify the coordinator ID gives them a way to
bypass a network round trip. This method will frequently be used for
monitoring, and if you've got 1,000 consumer groups that are being
monitored once a minute, that's ~1.5M requests a day that are
unnecessarily duplicated as the coordinator doesn't change unless
there's an error.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Support fetching the offsets of a consumer group.

Note: As far as I can tell (the Java code is a little inscrutable), the
Java AdminClient doesn't allow specifying the `coordinator_id` or the
`partitions`.

But I decided to include them because they provide a lot of additional
flexibility:

1. allowing users to specify the partitions allows this method to be used even for
older brokers that don't support the OffsetFetchRequest_v2

2. allowing users to specify the coordinator ID gives them a way to
bypass a network round trip. This method will frequently be used for
monitoring, and if you've got 1,000 consumer groups that are being
monitored once a minute, that's ~1.5M requests a day that are
unnecessarily duplicated as the coordinator doesn't change unless
there's an error.
</pre>
</div>
</content>
</entry>
</feed>
