delta/python-packages/kafka-python.git, branch struct-pre-compilation github.com: mumrah/kafka-python.git Pre-compile pack/unpack function calls 2018-10-29T04:44:00+00:00 billyevans pervushinai@gmail.com 2018-07-20T21:11:41+00:00 232aed4eca9acfc9fbf468059afabb0bc93f174b I noticed that pack/unpack functions from https://github.com/dpkp/kafka-python/blob/master/kafka/protocol/types.py might be slightly improved. I made pre-compilation for them. It gives about 10% better performance compared to the current implementation. Consumption of 100msg: ``` 239884 0.187 0.000 0.287 0.000 types.py:18(_unpack) # new version 239884 0.192 0.000 0.323 0.000 types.py:17(_unpack) ``` I also made some profiling for producers/consumers. It gives about 1-1.5% time savings. 
I noticed that pack/unpack functions from
https://github.com/dpkp/kafka-python/blob/master/kafka/protocol/types.py
might be slightly improved. I made pre-compilation for them. It gives
about 10% better performance compared to the current implementation.

Consumption of 100msg:
```
239884    0.187    0.000    0.287    0.000 types.py:18(_unpack) # new version
239884    0.192    0.000    0.323    0.000 types.py:17(_unpack)
```

I also made some profiling for producers/consumers. It gives about
1-1.5% time savings.
Cleanup fixture imports 2018-10-27T09:17:23+00:00 Jeff Widman jeff@jeffwidman.com 2018-10-27T07:11:10+00:00 f00016e7cec64cfc9697b233809cd37e0e19cc64 `random_string` now comes from `test.fixtures` and was being transparently imported via `test.testutil` so this bypasses the pointless indirect import. Similarly, `kafka_version` was transparently imported by `test.testutil` from `test.fixtures`. Also removed `random_port()` in `test.testutil` because its unused as its been replaced by the one in `test.fixtures`. This is part of the pytest migration that was started back in a1869c4be5f47b4f6433610249aaf29af4ec95e5. 
`random_string` now comes from `test.fixtures` and was being
transparently imported via `test.testutil` so this bypasses the
pointless indirect import.

Similarly, `kafka_version` was transparently imported by `test.testutil`
from `test.fixtures`.

Also removed `random_port()` in `test.testutil` because its unused as its been replaced
by the one in `test.fixtures`.

This is part of the pytest migration that was started back in
a1869c4be5f47b4f6433610249aaf29af4ec95e5.
Minor aesthetic cleanup of partitioner tests 2018-10-27T09:16:11+00:00 Jeff Widman jeff@jeffwidman.com 2018-10-27T09:01:08+00:00 1945ad16a15f53a07fae489b20ac616bb184ca89 






Remove unused ivy_root variable
2018-10-27T09:12:35+00:00

Jeff Widman
jeff@jeffwidman.com

2018-10-27T08:17:29+00:00

2b67493ff88aa3068bfc1be1aa089d95f4d60699

This is no longer used anywhere in the codebase





This is no longer used anywhere in the codebase







Minor cleanup of testing doc
2018-10-27T07:00:00+00:00

Jeff Widman
jeff@jeffwidman.com

2018-10-27T05:14:42+00:00

4f4e4fb5da8861593d084fa8a9de223ed7439ccf

Removed some of the hardcoded values as they are now outdated, and just
pointed to where to find the current value in the code.

Also some minor wordsmithing.





Removed some of the hardcoded values as they are now outdated, and just
pointed to where to find the current value in the code.

Also some minor wordsmithing.







Stop using deprecated log.warn()
2018-10-27T05:55:52+00:00

Jeff Widman
jeff@jeffwidman.com

2018-10-27T05:53:03+00:00

8791f0558fc103df43a2d7ab214904ad9a15f147

I missed this in my previous cleanup back in 9221fcf83528b5c3657e43636cb84c1d18025acd.





I missed this in my previous cleanup back in 9221fcf83528b5c3657e43636cb84c1d18025acd.







Stop pinning `pylint`
2018-10-27T04:03:48+00:00

Jeff Widman
jeff@jeffwidman.com

2018-10-25T05:50:06+00:00

b54607fbb88507e692f63c5c547742009f6bcaa8

We have many deprecation warnings in the travis logs for things that are
fixed in newer versions of `pylint` or `pylint`'s dependencies.

Note that `pylint` >= 2.0 does not support python 2, so this will result
in different versions of pylint running for python 2 vs python 3.
Personally, I am just fine with this.





We have many deprecation warnings in the travis logs for things that are
fixed in newer versions of `pylint` or `pylint`'s dependencies.

Note that `pylint` >= 2.0 does not support python 2, so this will result
in different versions of pylint running for python 2 vs python 3.
Personally, I am just fine with this.







Add temp workaround for upstream pylint bug
2018-10-27T04:03:48+00:00

Jeff Widman
jeff@jeffwidman.com

2018-10-26T21:09:54+00:00

cd0bd8e4db66a532f19a76908c677ebf004f642e

Temporarily workaround https://github.com/PyCQA/pylint/issues/2571 so that we can stop pinning `pylint`.





Temporarily workaround https://github.com/PyCQA/pylint/issues/2571 so that we can stop pinning `pylint`.







Add KafkaAdmin class
2018-10-25T05:42:12+00:00

Richard Lee
github@richardlee.name

2018-07-12T18:39:29+00:00

481f88068bdf0a18f12fd7a811b795f889d35fc7

Requires cluster version > 0.10.0.0, and uses new wire protocol
classes to do many things via broker connection that previously
needed to be done directly in zookeeper.





Requires cluster version > 0.10.0.0, and uses new wire protocol
classes to do many things via broker connection that previously
needed to be done directly in zookeeper.







Fix sphinx url
2018-10-24T22:37:41+00:00

Jeff Widman
jeff@jeffwidman.com

2018-10-24T20:30:45+00:00

ac9d5623116a5754c57a8ecd95b2954ba0f30c14

When I was fixing urls the other day, I noticed that sphinx hadn't added
https but there was an open ticket: https://github.com/sphinx-doc/sphinx/issues/5522

Now that that is resolved, I'm updating it here.





When I was fixing urls the other day, I noticed that sphinx hadn't added
https but there was an open ticket: https://github.com/sphinx-doc/sphinx/issues/5522

Now that that is resolved, I'm updating it here.