| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
|
|
|
|
|
| |
* flake8: noqa the Python 2-only builtin cmp()
* flake8: buffer() was removed in Python 3
* flake8: unicode() was removed in Python3 because all str are Unicode
Adds the modifications from #1227
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* buffer() was removed in Python 3 so use memoryview
https://docs.python.org/2.7/library/stdtypes.html#memoryview-type
* buffer = bytes on Python 3
* from kombu.five import buffer
* buffer --> buffer_t
* @patch('buffer_t') in test_qpid.py
* @patch('kombu.five.buffer_t')
* Use buffer on legacy Python and bytes on Python
* Update test_qpid.py
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* tox.ini: Lint on Python, not on legacy Python
* Travis CI: Lint on Python, not on legacy Python
* Update .travis.yml
* Update .travis.yml
Co-authored-by: Omer Katz <omer.drow@gmail.com>
* flakeplus only works on legacy Python
Co-authored-by: Omer Katz <omer.drow@gmail.com>
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
| |
* Revert incompatible changes introduced in #1193
* Improved integration tests covering connection
* Fix unittests on python2 + flake8 fixes
|
| |
|
| |
|
| |
|
|\ |
|
| | |
|
| |
| |
| | |
Co-authored-by: Conor Stevenson <conor@lendingblock.com>
|
| | |
|
|/ |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
available workers to start acting on “in-flight” messages in the SQS queue (#1199)
* Fix for the issue #1172
* Unit test for the fix relating to the issue #1172
* Fix for issue #1198: Celery crashes in cases where there aren’t enough available workers to start acting on “in-flight” messages in the SQS queue
* Fix for issue #1198: fixed lint issues
* Fix for issue #1198: added unit tests
Co-authored-by: inauros <inauros@copyright.com>
|
| |
|
|
|
|
|
| |
dropped connection" (#1195)
This reverts commit 90f51bcbbd32146998e7c7e4491150344343776b.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Although a race condition in the SQLAlchmey Channel class was fixed in
a recent commit [0], a few other methods are also susceptible to race
conditions.
Indeed, if multiple threads call the class' `_open` method, a conflict
can occur with the `metadata.create_all` call which creates the schema
in the database, resulting in an exception which can look like this:
```
sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation)
duplicate key value violates unique constraint
"pg_type_typname_nsp_index" DETAIL: Key (typname,
typnamespace)=(queue_id_sequence, 2200) already exists.
```
Similarly, `_get_or_create` is susceptible to a race condition if two
threads try to create the unique queue object in the database at the
same time, resulting in this kind of exception:
```
sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation)
duplicate key value violates unique constraint "kombu_queue_name_key"
DETAIL: Key (name)=(celery) already exists.
[SQL: INSERT INTO kombu_queue (id, name) VALUES
(nextval('queue_id_sequence'), %(name)s) RETURNING kombu_queue.id]
[parameters: {'name': 'celery'}]
```
By using the mutex, we can ensure that only one thread at a time
executes these critical sections, eliminating the race conditions.
The mutex has been made re-entrant since `_open` can be called from
`_get_or_create`, which would cause a deadlock if a simple `Lock` was
used instead of `RLock`.
[0] 8f1de37a08318d388a048341ddca12af30019bf3
Signed-off-by: Antoine Busque <antoinebusque@gmail.com>
|
|
|
|
| |
Fixes #1190
|
| |
|
| |
|
|
|
|
| |
fix https://github.com/celery/celery/issues/5299
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
| |
Query.with_lockmode().
Based on SQLAlchemy documentation:
* method sqlalchemy.orm.query.Query.with_lockmode(mode) with mode='update' - translates to FOR UPDATE (Deprecated since version 0.9)
* method sqlalchemy.orm.query.Query.with_for_update() When called with no arguments, the resulting SELECT statement will have a FOR UPDATE clause appended.
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The init method for the SQLAlchemy transport's Channel implementation
is not currently thread-safe.
This means that if two threads in a process attempt to instantiate a
Channel concurrently, there can be a race condition when registering
the SQLAlchemy model classes, which leaves the SQLAlchemy ORM mapper
in a broken state.
This results in an exception like the following:
```
sqlalchemy.exc.InvalidRequestError: Table 'kombu_message' is already
defined for this MetaData instance. Specify 'extend_existing=True' to
redefine options and columns on an existing Table object.
```
Any subsequent calls to the SQLAlchemy ORM will then fail with an
exception like this:
```
sqlalchemy.exc.InvalidRequestError: Multiple classes found for path
"Message" in the registry of this declarative base. Please use a fully
module-qualified path.
```
This also applies to any SQLAlchemy calls in the same process, not
just those made by Kombu or Celery, and can only really be recovered
from by killing the process and starting over.
To avoid all of this, introduce a mutex which ensures that only one
thread at a time can register a SQLAlchemy model when instantiating
the Channel class.
Signed-off-by: Antoine Busque <antoinebusque@gmail.com>
|
|
|
|
|
|
|
|
|
| |
Flake8 compliant this time.
Unit test to follow if possible but see no unit testing infrastructure in the repo. Fail behaviour on this is Exchange.publish(text) crashes with an encoding error. A Pass is that the message is published, which can be confirmed with Queue.get() and comparing the payload with that which was published. Requires a running RabbitMQ instance to test (as that is the context of the crash).
A unit test though, while holding value is of the functionality of Exchange.publish() - currently broken for text messages, is not central to the value of fixing this break.
ref: https://github.com/celery/kombu/issues/1174
|