summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
...
| * Fix python3 str/bytes bug in KafkaConsumer.set_topic_partitions({(topic, ↵Dana Powers2015-04-041-1/+1
|/ | | | partition): offset, })
* Merge pull request #360 from dpkp/atexit_cleanupDana Powers2015-04-042-4/+51
|\ | | | | Register atexit handlers for consumer and producer thread/multiprocess cleanup
| * Register atexit handlers for consumer and producer thread/multiprocess ↵Dana Powers2015-03-312-4/+51
| | | | | | | | cleanup (not __del__)
* | Merge pull request #361 from kecaps/masterDana Powers2015-04-0411-49/+69
|\ \ | |/ |/| Make external API consistently support python3 strings for topic.
| * Make external API consistently support python3 strings for topic.Space2015-04-0311-49/+69
|/
* Merge pull request #356 from dpkp/always_fetch_offsetsDana Powers2015-03-305-14/+105
|\ | | | | fetch commit offsets in base consumer unless group is None
| * Bulk fetch offset partitions in base consumer -- suggested by ecanzonieriDana Powers2015-03-301-8/+9
| |
| * Fetch previously committed offsets in base consumer class so long asDana Powers2015-03-305-14/+34
| | | | | | | | | | | | | | | | a group is configured (but document that group must be None for old servers). This fixes multiprocessor consumer issue that prevented access to commit offsets if auto_commit is disabled. Also refactor fetch_last_known_offsets based on KafkaConsumer While still setting unknown offsets to 0
| * Skip these tests: no OffsetCommitRequest for 0.8.0Viktor Shlapakov2015-03-301-2/+2
| |
| * Added basic tests for load_initial_offsets optionViktor Shlapakov2015-03-301-0/+35
| |
| * Add test case for MP Consumer auto commitAli-Akber Saifee2015-03-301-0/+35
| | | | | | | | Tweak MP Consumer test to use iterator
* | Merge pull request #355 from dpkp/correlation_id_moduloDana Powers2015-03-302-7/+15
|\ \ | |/ |/| correlation_id modulo
| * Add test for correlation_id rolloverDana Powers2015-03-291-0/+8
| |
| * Rollover KafkaClient correlation ids at 2**31 to keep within int32 protocol ↵Dana Powers2015-03-291-7/+7
| | | | | | | | encoding
* | Merge pull request #357 from dpkp/del_consumer_stopDana Powers2015-03-301-0/+3
|\ \ | | | | | | Stop consumers on delete
| * | Stop consumers on deleteDana Powers2015-03-301-0/+3
| |/
* | Merge pull request #341 from dpkp/kafka_consumer_docsDana Powers2015-03-294-149/+185
|\ \ | |/ |/| KafkaConsumer documentation
| * Use bootstrap_servers interface in testsDana Powers2015-03-291-1/+1
| |
| * Add sphinx_rtd_theme to docs requirements.txtDana Powers2015-03-291-0/+1
| |
| * Updates to KafkaConsumer usage docsDana Powers2015-03-291-42/+26
| |
| * fixup kafka consumer docstringDana Powers2015-03-291-1/+1
| |
| * Move KafkaConsumer usage examples to docs/usage; Put KeyedProducer usage ↵Dana Powers2015-03-291-14/+105
| | | | | | | | right after SimpleProducer
| * Improve KafkaConsumer docstringsDana Powers2015-03-291-134/+94
|/
* Merge pull request #340 from dpkp/deprecate_metadata_broker_listDana Powers2015-03-291-8/+26
|\ | | | | Deprecate metadata_broker_list in favor of bootstrap_servers
| * Deprecate KafkaConsumer config metadata_broker_list in favor of ↵Dana Powers2015-03-231-8/+26
| | | | | | | | bootstrap_servers
* | Merge pull request #336 from scrapinghub/feature-mp-consumer-paramsDana Powers2015-03-295-38/+57
|\ \ | | | | | | Using additional params for MP consumer child process
| * | Manageable queue.put() operation for MPConsumer processesViktor Shlapakov2015-03-252-4/+12
| | |
| * | Added pylint.rc: ignoring SyncManager for pylintViktor Shlapakov2015-03-242-1/+3
| | |
| * | Using mp.manager to solve the issue with join for MPConsumerViktor Shlapakov2015-03-241-6/+7
| | |
| * | Moving to **kwargs for MPConsumer optionsViktor Shlapakov2015-03-242-6/+5
| | |
| * | Using iter_timeout=0 for MP inner consumer in testsViktor Shlapakov2015-03-131-1/+4
| | |
| * | Moved additional MP consumer options to **kwargsViktor Shlapakov2015-03-131-3/+3
| | |
| * | Using additional params for MP consumer child processViktor Shlapakov2015-03-131-26/+32
| | | | | | | | | | | | | | | | | | | | | - Moved the events params to a separate param for consistency - Passing additional params to internal SimpleConsumer worker for multiprocessing high-level consumer. It allows to use non-default consumer settings (fetch_size_bytes, buffer_size, max_buffer_size).
* | | Merge pull request #350 from ecanzonieri/enable_commit_on_seekDana Powers2015-03-291-1/+1
|\ \ \ | | | | | | | | Enable commit on seek
| * | | Increase count_since_commit on seekEnrico Canzonieri2015-03-201-1/+1
| |/ / | | | | | | | | | | | | | | | When auto_commit is False this change enables an explicit call to commit() to actually commit the offsets. Otherwise a consumer won't be able to commit until at least one message is read.
* | | Merge pull request #338 from dpkp/structs_reorgDana Powers2015-03-291-23/+26
|\ \ \ | | | | | | | | Fixup indentation in kafka/common.py; add links to protocol definitions
| * | | Fixup indentation in kafka/common.py; add links to protocol definitionsDana Powers2015-03-081-23/+26
| | |/ | |/|
* | | Merge pull request #337 from dpkp/gzip_contextDana Powers2015-03-291-23/+32
|\ \ \ | | | | | | | | Use context managers in gzip_encode / gzip_decode
| * | | Take the linter to kafka/codec.pyDana Powers2015-03-091-11/+10
| | | |
| * | | Gzip context manager not supported in py2.6, so use try/finally insteadDana Powers2015-03-091-2/+17
| | | |
| * | | Use context managers in gzip_encode / gzip_decodeDana Powers2015-03-081-12/+7
| |/ /
* | | Merge pull request #329 from vshlapakov/feature-batch-msg-keysDana Powers2015-03-294-16/+76
|\ \ \ | |_|/ |/| | Correct message keys for async batching mode
| * | Correct message keys for async batching modeViktor Shlapakov2015-02-254-16/+76
| | |
* | | Merge pull request #335 from scrapinghub/fix-mp-consumer-distributionDana Powers2015-03-121-10/+12
|\ \ \ | |_|/ |/| | Wrong partitions distribution logic for MP Consumer
| * | Used thread-safe dict.copy().keys() for MP consumer partitionsViktor Shlapakov2015-03-121-1/+4
| | |
| * | Cleaned code for MP consumer chunkingViktor Shlapakov2015-03-111-10/+9
| | |
| * | Fixing distribution for MP ConsumerViktor Shlapakov2015-03-031-1/+1
| |/
* | Merge branch 'vshlapakov-feature-async-threading'Dana Powers2015-03-083-26/+73
|\ \ | | | | | | | | | | | | | | | | | | PR 330: Threading for async batching Conflicts: kafka/producer/base.py
| * | Async producer stop() fixViktor Shlapakov2015-02-261-2/+2
| | |
| * | Using threading.Event to stop async producer threadViktor Shlapakov2015-02-261-5/+10
| | |
View Lorry Controller Status