summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Update TOX_CONSTRAINTS_FILE for stable/zedstable/zedOpenStack Release Bot2022-09-091-1/+1
| | | | | | | | | | | | Update the URL to the upper-constraints file to point to the redirect rule on releases.openstack.org so that anyone working on this branch will switch to the correct upper-constraints list automatically when the requirements repository branches. Until the requirements repository has as stable/zed branch, tests will continue to use the upper-constraints list on master. Change-Id: I65dc8c04014f0b0697bb6fa056accb8e37a790f1
* Update .gitreview for stable/zedOpenStack Release Bot2022-09-091-0/+1
| | | | Change-Id: I5263dc510d8ba74b6ba4cf28feec805b16fe1a85
* Change default value of "heartbeat_in_pthread" to False14.0.0Slawek Kaplonski2022-08-162-2/+13
| | | | | | | | | | | | | | | | | | | | | As was reported in the related bug some time ago, setting that option to True for nova-compute can break it as it's non-wsgi service. We also noticed same problems with randomly stucked non-wsgi services like e.g. neutron agents and probably the same issue can happen with any other non-wsgi service. To avoid that this patch changes default value of that config option to be False. Together with [1] it effectively reverts change done in [2] some time ago. [1] https://review.opendev.org/c/openstack/oslo.messaging/+/800621 [2] https://review.opendev.org/c/openstack/oslo.messaging/+/747395 Related-Bug: #1934937 Closes-Bug: #1961402 Change-Id: I85f5b9d1b5d15ad61a9fcd6e25925b7eeb8bf6e7
* Merge "Add quorum queue control configurations"13.0.0Zuul2022-06-133-7/+80
|\
| * Add quorum queue control configurationshamza alqtaishat2022-04-063-7/+80
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the quorum queue type add features that did not exist before or not handled in rabbitmq the following link shows some of them https://blog.rabbitmq.com/posts/2020/04/rabbitmq-gets-an-ha-upgrade/ the options below control the quorum queue and ensure the stability of the quorum system x-max-in-memory-length x-max-in-memory-bytes x-delivery-limit which control the memory usage and handle message poisoning Closes-Bug: #1962348 Change-Id: I570227d6102681f4f9d8813ed0d7693a1160c21d
* | Drop python3.6/3.7 support in testing runtimeHervé Beraud2022-05-051-3/+1
| | | | | | | | | | | | | | | | | | | | In Zed cycle testing runtime, we are targetting to drop the python 3.6/3.7 support, project started adding python 3.8 as minimum, example nova: - https://github.com/openstack/nova/blob/56b5aed08c6a3ed81b78dc216f0165ebfe3c3350/setup.cfg#L13 Change-Id: Id23d3845db716d26175d71280dbedf93736d19de
* | Merge "Add EXTERNAL as rabbit login method"12.14.0Zuul2022-04-271-1/+1
|\ \
| * | Add EXTERNAL as rabbit login methodhamza alqtaishat2022-04-251-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As explained in the link below kombu has login method called external https://docs.celeryq.dev/projects/kombu/en/latest/_modules/kombu/connection.html The login method external is not listed as a choice in the Rabbit driver As explained in RabbitMQ documention https://www.rabbitmq.com/access-control.html for Authentication using Client TLS (x.509) Certificate Data clients must be configured to use the EXTERNAL mechanism. Closes-Bug: #1970276 Change-Id: I5c38d3a3cafd49f8abc031e36bc595f32a8631d2
* | | Merge "Add a new option to enforce the OpenSSL FIPS mode"Zuul2022-04-265-0/+98
|\ \ \
| * | | Add a new option to enforce the OpenSSL FIPS modeHervé Beraud2021-11-085-0/+98
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This option ``ssl_enforce_fips_mode`` allow us to enforce the FIPS mode if supported by the version of python in use. https://en.wikipedia.org/wiki/Federal_Information_Processing_Standards Change-Id: I50c7de71bfd38137eb83d23e910298946507ce9f
* | | | Merge "Add Python3 zed unit tests"Zuul2022-04-261-1/+1
|\ \ \ \ | |_|/ / |/| | |
| * | | Add Python3 zed unit testsOpenStack Release Bot2022-03-041-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is an automatically generated patch to ensure unit testing is in place for all the of the tested runtimes for zed. See also the PTI in governance [1]. [1]: https://governance.openstack.org/tc/reference/project-testing-interface.html Change-Id: I73a0700baa1c9edfb7a4b82be94df8bacff3c226
* | | | tests: Fix test failures with kombu >= 5.2.4Stephen Finucane2022-04-051-6/+21
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | kombu 5.2.4 fixed an off-by-one issue that meant we were attempting retries more than once [1]. We need to handle this to unblock the gate. This was discovered by examining the call stack and comparing this with recent changes in openstack/requirements. [1] https://github.com/celery/kombu/commit/5bed2a8f983a3bf61c12443e7704ffd89991ef9a Change-Id: I476e3c573523d5991c56b31ad4df1172196aa7f1 Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
* | | Update master for stable/yogaOpenStack Release Bot2022-03-042-0/+7
| |/ |/| | | | | | | | | | | | | | | | | | | | | Add file to the reno documentation build to show release notes for stable/yoga. Use pbr instruction to increment the minor version number automatically so that master versions are higher than the versions on stable/yoga. Sem-Ver: feature Change-Id: I3d2b041769c5c14a7d391c223dc499218a937e76
* | Merge "Adding support for rabbitmq quorum queues"12.13.0Zuul2022-02-084-9/+66
|\ \
| * | Adding support for rabbitmq quorum queuesHervé Beraud2022-02-054-9/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | https://www.rabbitmq.com/quorum-queues.html The quorum queue is a modern queue type for RabbitMQ implementing a durable, replicated FIFO queue based on the Raft consensus algorithm. It is available as of RabbitMQ 3.8.0. the quorum queues can not be set by policy so this should be done when declaring the queue. To declare a quorum queue set the x-queue-type queue argument to quorum (the default is classic). This argument must be provided by a client at queue declaration time; it cannot be set or changed using a policy. This is because policy definition or applicable policy can be changed dynamically but queue type cannot. It must be specified at the time of declaration. its good for the oslo messaging to add support for that type of queue that have multiple advantaged over mirroring. If quorum queues are sets mirrored queues will be ignored. Closes-Bug: #1942933 Change-Id: Id573e04c287e034e50626daf6e18a34735d45251
* | | Merge "[rabbit] use retry parameters during notification sending"12.12.0Zuul2022-01-128-26/+60
|\ \ \
| * | | [rabbit] use retry parameters during notification sendingBalazs Gibizer2022-01-128-26/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The rabbit backend now applies the [oslo_messaging_notifications]retry, [oslo_messaging_rabbit]rabbit_retry_interval, rabbit_retry_backoff and rabbit_interval_max configuration parameters when tries to establish the connection to the message bus during notification sending. This patch also clarifies the differences between the behavior of the kafka and the rabbit drivers in this regard. Closes-Bug: #1917645 Change-Id: Id4ccafc95314c86ae918336e42cca64a6acd4d94
* | | | Merge "Reproduce bug 1917645"Zuul2021-12-211-0/+68
|\ \ \ \ | |/ / /
| * | | Reproduce bug 1917645Balazs Gibizer2021-11-241-0/+68
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The [oslo_messaging_notification]retry parameter is not applied during connecting to the message bus. But the documentation implies it should[1][2]. The two possible drivers, rabbit and kafka, behaves differently. 1) The rabbit driver will retry the connection forever, blocking the caller process. 2) The kafka driver also ignores the retry configuration but the notifier call returns immediately even if the notification is not (cannot) be delivered. This patch adds test cases to show the wrong behavior. [1] https://docs.openstack.org/oslo.messaging/latest/configuration/opts.html#oslo_messaging_notifications.retry [2] https://github.com/openstack/oslo.messaging/blob/feb72de7b81e3919dedc697f9fb5484a92f85ad8/oslo_messaging/notify/messaging.py#L31-L36 Related-Bug: #1917645 Change-Id: Id8557050157aecd3abd75c9114d3fcaecdfc5dc9
* | | | Update python testing classifierdengzhaosen2021-12-211-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Yoga testing runtime[1] has been updated to add py39 testing as voting. Unit tests update are handled by the job template change in openstack-zuul-job - https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/820286 this commit updates the classifier in setup.cfg file. [1] https://governance.openstack.org/tc/reference/runtimes/yoga.html Change-Id: I26743858a0ca7a6e46bda821c8f29b6dff34ea15
* | | | amqp1: fix race when reconnecting12.11.1John Eckersberg2021-11-091-1/+2
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently this is how reconnect works: - pyngus detects failure and invokes callback Controller.connection_failed() which in turn calls Controller._handle_connection_loss() - The first thing that _handle_connection_loss does is to set self.addresser to None (important later) - Then it defers _do_reconnect after a delay (normally 1 second) - (1 second passes) - _do_reconnect calls _hard_reset which resets the controller state However, there is a race here. This can happen: - The above, up until it defers and waits for 1 second - Controller.send() is invoked on a task - A new Sender is created, and critically because self.reply_link still exists and is active, we call sender.attach and pass in self.addresser. Remember _handle_connection_loss sets self.addresser to None. - Eventually Sender.attach throws an AttributeError because it attempts to call addresser.resolve() but addresser is None The reason this happens is because although the connection is dead, the controller state is still half-alive because _hard_reset hasn't been called yet since it's deferred one second in _do_reconnect. The fix here is to move _hard_reset out of _do_reconnect and directly into _handle_connection_loss. The eventloop is woken up immediately to process _hard_reset but _do_reconnect is still deferred as before so as to retain the desired reconnect backoff behavior. Closes-Bug: #1941652 Change-Id: Ife62a7d76022908f0dc6a77f1ad607cb2fbd3e8f
* | | Merge "Remove deprecation of heartbeat_in_pthread"12.11.0Zuul2021-10-212-1/+7
|\ \ \
| * | | Remove deprecation of heartbeat_in_pthreadHervé Beraud2021-10-142-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In some circumstances services can be executed outside of mod_wsgi and in a monkey patched environment. In this context we need to leave the possibility to users to execute the heartbeat in a green thread. The heartbeat_in_pthread was tagged as depreacted few months and planned for a future removal. These changes drop this deprecation to allow to enable green threads if needed. Closes-Bug: #1934937 Change-Id: Iee2e5a6f7d71acba70bbc857f0bd7d83e32a7b8c
* | | | rabbit: move stdlib_threading bits into _utils12.10.0John Eckersberg2021-09-222-14/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The amqp1 driver also needs this same logic, so split it out and share it. Change-Id: I2e9dbfa27887e26807f199c9d359bacd7c15c67a
* | | | Merge "use message id cache for RPC listener"Zuul2021-09-132-1/+61
|\ \ \ \
| * | | | use message id cache for RPC listenerNikita Kalyanov2021-09-102-1/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Return back the message id cache feature to RPC listener, it was removed while refactoring in I708c3d6676b974d8daac6817c15f596cdf35817b See attached bug for more info. We should not raise DuplicateMessageError to avoid rejecting the previously ACK'ed message. Closes-Bug: #1935883 Change-Id: Ie237e9e3fdc3fc27b3deb18b94751cdc3afd190e
* | | | | Merge "limit maximum timeout in the poll loop"Zuul2021-09-132-2/+29
|\ \ \ \ \ | |/ / / /
| * | | | limit maximum timeout in the poll loopNikita Kalyanov2021-07-132-2/+29
| |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We should properly limit the maximum timeout with a 'min' to avoid long delays before message processing. Such delays may happen if the connection to a RabbitMQ server is re-established at the same time when the message arrives (see attached bug for more info). Moreover, this change is in line with the original intent to actually have an upper limit on maximum possible timeout (see comments in code and in the original review). Closes-Bug: #1935864 Change-Id: Iebc8a96e868d938a5d250bf9d66d20746c63d3d5
* | | | Add Python3 yoga unit testsOpenStack Release Bot2021-09-101-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is an automatically generated patch to ensure unit testing is in place for all the of the tested runtimes for yoga. See also the PTI in governance [1]. [1]: https://governance.openstack.org/tc/reference/project-testing-interface.html Change-Id: I0088ff54c4807f240a1db2457aeefcdf8b91375d
* | | | Update master for stable/xenaOpenStack Release Bot2021-09-102-0/+7
| |/ / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add file to the reno documentation build to show release notes for stable/xena. Use pbr instruction to increment the minor version number automatically so that master versions are higher than the versions on stable/xena. Sem-Ver: feature Change-Id: Ia40ac2ccee4fe230605f3183b0b432b0e31bff04
* | | amqp1: Do not reuse _socket_connection on reconnect12.9.1John Eckersberg2021-08-102-8/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Each _SocketConnection object is unique per-peer. For example, the properties attribute may contain keys such as 'x-ssl-peer-name'. Reusing the existing _socket_connection during failover will cause the TLS handshake to fail since the peer name will not match. There is potential for other similar-yet-unexplored bad things to happen as well. Instead, reconnect by waking up the eventloop via the _do_reconnect method, which reconstructs the connection properties to reflect the new (failed-over-to) host and ultimately crates a new _SocketConnection (or re-uses a *valid* old one) in eventloop.Thread.connect(). Closes-Bug: #1938945 Change-Id: I0c8dc447f4dc8d0d08c312a1f3e6fa1745fb69fd
* | | amqp1: re-organize TestFailover to be reused by TestSSLJohn Eckersberg2021-08-101-7/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This breaks out the generation of brokers and transport_url into separate methods. These methods are used in the next patch in this series, where TestSSL is updated to inherit from TestFailover, and TestSSL overrides the _gen_brokers and _gen_transport_url methods to supply the necessary SSL-aware options. Change-Id: Ia2f977795abc2e81a996e299867e05d41057f33f
* | | Revert "Disable AMQP 1.0 SSL unit tests"John Eckersberg2021-08-101-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | This reverts commit 8f5cfda6642ea7f75206d3183c2507e2e83c5693. Reason for revert: This was supposed to be temporary to unblock the gate. Whatever broke SSL cert generation in the first place appears to be fixed because I can run SSL tests now. Change-Id: I4f286cf3af0d578f472b84fe355c812910c7a121
* | | Merge "Changed minversion in tox to 3.18.0"12.9.0Zuul2021-08-101-3/+3
|\ \ \ | |/ / |/| |
| * | Changed minversion in tox to 3.18.0yangyawei2021-06-071-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | The patch bumps min version of tox to 3.18.0 in order to replace tox's whitelist_externals by allowlist_externals option: https://github.com/tox-dev/tox/blob/master/docs/changelog.rst#v3180-2020-07-23 Change-Id: If129b56be47952d018b9f6024d2a192950c1a974
* | | Merge "Remove the oslo_utils.fnmatch"Zuul2021-06-251-1/+1
|\ \ \ | |_|/ |/| |
| * | Remove the oslo_utils.fnmatchdengzhaosen2021-05-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Oslo.utils's fnmatch module was added to fix the py2.7 fnmatch module who was not thread safe [1]. Python 2.7 is no longer supported so now we can use the stdlib's fnmatch module and deprecate the one of oslo.utils. [1] https://bugs.python.org/issue23191$ Change-Id: Id5381a0a5216783f0df594b126786947db16a8d1
* | | Merge "Add Support For oslo.metrics"Zuul2021-06-107-17/+360
|\ \ \ | |_|/ |/| |
| * | Add Support For oslo.metricsChing Kuo2021-06-087-17/+360
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit added support to send rpc metrics to oslo.metrics. Changes includes: - Adding client wrapper for oslo.metrics to process metrics information and send to oslo.metrics socket - Modify rpc client to send metric when certain rpc events happens For more information on oslo.metrics https://opendev.org/openstack/oslo.metrics Change-Id: Idf8cc0e52ced1f697ac4048655eff4c956fd5c79
* | | Merge "Upgrade the pre-commit-hooks version"Zuul2021-05-261-1/+1
|\ \ \
| * | | Upgrade the pre-commit-hooks versionwu.shiming2021-05-181-1/+1
| |/ / | | | | | | | | | | | | | | | [1] https://github.com/pre-commit/pre-commit-hooks/commit/9136088a246768144165fcc3ecc3d31bb686920a Change-Id: Ifbea6ef06e230127bfa16fc3dc0ddd9f236cfbed
* | | setup.cfg: Replace dashes with underscoresyangyawei2021-05-131-4/+4
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Setuptools v54.1.0 introduces a warning that the use of dash-separated options in 'setup.cfg' will not be supported in a future version [1]. Get ahead of the issue by replacing the dashes with underscores. Without this, we see 'UserWarning' messages like the following on new enough versions of setuptools: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead [1] https://github.com/pypa/setuptools/commit/a2e9ae4cb Change-Id: I6e015bf3955e3ff7aa21bc86d3f6f69e0017ca29
* | Remove references to 'sys.version_info'12.8.0Jorhson Deng2021-04-281-12/+1
| | | | | | | | | | | | We support Python 3.6 as a minimum now, making these checks no-ops. Change-Id: I2cd6d5272eeacbda91e389efb1cfaaadf376d767
* | Merge "Fix formatting of release list"Zuul2021-04-161-13/+13
|\ \
| * | Fix formatting of release listPierre Riteau2021-04-161-13/+13
| | | | | | | | | | | | Change-Id: I1f859a964de7f96e5decdec0977faa355b6a2a60
* | | Merge "bindep: Add 'librdkafka-dev' dependency"Zuul2021-04-161-0/+1
|\ \ \ | |/ / |/| |
| * | bindep: Add 'librdkafka-dev' dependencyStephen Finucane2020-06-021-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We're seeing the following in our build logs: In file included from confluent_kafka/src/confluent_kafka.c:17:0: confluent_kafka/src/confluent_kafka.h:22:10: fatal error: librdkafka/rdkafka.h: No such file or directory #include <librdkafka/rdkafka.h> ^~~~~~~~~~~~~~~~~~~~~~ compilation terminated. error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 Resolve this by installing the development headers. Change-Id: Idda79dc87bcd0e3367a6abd1b52104c000ad1dcd Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
* | | Merge "Add Python3 xena unit tests"Zuul2021-04-151-1/+1
|\ \ \
| * | | Add Python3 xena unit testsOpenStack Release Bot2021-03-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is an automatically generated patch to ensure unit testing is in place for all the of the tested runtimes for xena. See also the PTI in governance [1]. [1]: https://governance.openstack.org/tc/reference/project-testing-interface.html Change-Id: I0be0881612b74a6d003616ee7747e303939be11f