diff options
| author | Ilya Tyaptin <ityaptin@mirantis.com> | 2016-06-21 14:57:17 +0300 |
|---|---|---|
| committer | Mehdi Abaakouk <sileht@redhat.com> | 2017-01-02 11:46:48 +0100 |
| commit | f139eb258db12c10d921560db132263a7cf25b61 (patch) | |
| tree | 8892a69dcaecd3c76443ac5a0ba5e1db9da43efe /oslo_messaging/opts.py | |
| parent | 8fefbc064d30ec4fa530eb385629b4f05942ef0b (diff) | |
| download | oslo-messaging-f139eb258db12c10d921560db132263a7cf25b61.tar.gz | |
Moving driver to new kafka-python version
Currently Kafka driver for an oslo.messaging uses kafka-python==0.9.5
and mostly broken. This package version supports only low level Kafka
producer and consumer API which are marked as deprecated now [1]. Using
of these interfaces bring a big concern to the message processing,
because current KafkaConsumer has not any consuming coordination. This
fact causes a message duplication for the several consumers of one
topic. This behavior is specific to Ceilometer and services which read
and process notifications from other services.
New version of kafka-python allows to use async thread safe message
producers and coordinated consumers [1].
[1] http://kafka-python.readthedocs.io/en/master/changelog.html#feb-15-2016
The driver is currently experimental, python-kafka<1.0.0 API have major
issue described above that can't make the oslo.messaging driver works,
so we prefer having a working driver with a non-synced dependencies, that the
reverse.
Co-Authored-By: Mehdi Abaakouk <sileht@redhat.com>
Change-Id: I29862ed7bf56b9d8878fa8e9fb1cbd9d643908a4
Diffstat (limited to 'oslo_messaging/opts.py')
| -rw-r--r-- | oslo_messaging/opts.py | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/oslo_messaging/opts.py b/oslo_messaging/opts.py index 7c373c3..61bb692 100644 --- a/oslo_messaging/opts.py +++ b/oslo_messaging/opts.py @@ -26,6 +26,7 @@ from oslo_messaging._drivers import base as drivers_base from oslo_messaging._drivers import impl_pika from oslo_messaging._drivers import impl_rabbit from oslo_messaging._drivers.impl_zmq import zmq_options +from oslo_messaging._drivers import kafka_options from oslo_messaging._drivers.pika_driver import pika_connection_factory from oslo_messaging._drivers.zmq_driver.matchmaker import zmq_matchmaker_redis from oslo_messaging.notify import notifier @@ -53,6 +54,7 @@ _opts = [ pika_connection_factory.pika_opts, impl_pika.pika_pool_opts, impl_pika.message_opts, impl_pika.notification_opts, impl_pika.rpc_opts))), + ('oslo_messaging_kafka', kafka_options.KAFKA_OPTS), ] |