summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Merge "Update the docker image to python3.9 and buster"HEADmasterZuul2022-04-192-6/+6
|\
| * Update the docker image to python3.9 and busterClark Boylan2022-04-192-6/+6
| | | | | | | | | | | | | | | | | | This is long overdue now that buster is out and python3.9 has been around for a bit. This update allows us to remove the buster 3.7 images that the gear images were based on. Depends-On: https://review.opendev.org/c/opendev/gear/+/838562 Change-Id: I8d57c06fe28276d6e15934a785b1f97125f0958f
* | Fix python3.9 testingClark Boylan2022-04-191-2/+7
|/ | | | | | | | | | Latest setuptools has deprecated the use of setup.py commands like `setup.py testr`. For some reason python3.9 hangs and fails to work at all when you run `setup.py testr`. Switch to running testr directly and not bother debugging this too aggressively as this functionality is going away eventually. Change-Id: I3ad9e0c00990fbb7d26b03674833666f32b3bcae
* Overhaul package metadata and contributor info0.16.0Jeremy Stanley2021-07-104-34/+59
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Modernize our package metadata in the following ways: * switch from description-file to long_description with the file attribute, and specify an explicit content type and encoding * replace the home-page parameter with the newer general url one * use the specific license metadata in addition to the corresponding trove classifier for it * make sure wheels when built also incorporate the LICENSE and AUTHORS files so that we're not distributing them without a copy of the license text * indicate support for all recent Python releases in trove classifiers * drop Python 3.4 cruft from the bindep list https://setuptools.readthedocs.io/en/latest/userguide/declarative_config.html Also replace the contributor documentation with a more up to date copy from opendev/bindep, and adjust the copyright assertions in the built Sphinx docs to refer to "OpenDev Contributors" and drop the unnecessary year. Change-Id: I39c5f5afc66edec0cf51709218f143b2a749eddd
* Add libffi header dependencyJeremy Stanley2021-07-101-0/+2
| | | | | | | | | Gear (indirectly) relies on cffi, which sometimes isn't built for the platforms on which we would like to install it. In those cases, the Python installation has to occur from source, and needs the headers for libffi to link against when compiling its extensions. Change-Id: Ifc876d93f95941236b78a88d5741467a10142d54
* Update testing to Python 3.9 and lintersJeremy Stanley2021-03-303-9/+18
| | | | | | | | | | | | | | | In preparation for an upcoming release, add testing for latest Python (3.9). Switch tox to use Python 3 by default, and rename the testenv for flake8 from pep8 to linters, consistent with other tools and libraries OpenDev maintains. Update to a newer hacking plugin, which will use newer flake8 as well. Ignore rules about line breaks around comparison operators, as well as those related to ambiguous variable names, at least for now. Also build distribution artifacts on a more recent platform so we get newer Setuptools with support for the latest package metadata, in preparation for a coming change to update that. Change-Id: I2130d66fc9aadaa9fe09635b59475be71938132e
* Create SSL context using PROTOCOL_TLS, fallback to highest supported versionGuillaume Chauvel2021-03-301-2/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Zuul test: tests.unit.test_scheduler.TestSchedulerSSL.test_jobs_executed fails on ubuntu focal with the following exception: Traceback (most recent call last): File "/home/gchauvel/zuul/zuul/.tox/py38/lib/python3.8/site-packages/gear/__init__.py", line 2835, in _doConnectLoop self.connectLoop() File "/home/gchauvel/zuul/zuul/.tox/py38/lib/python3.8/site-packages/gear/__init__.py", line 2865, in connectLoop c = context.wrap_socket(c, server_side=True) File "/usr/lib/python3.8/ssl.py", line 500, in wrap_socket return self.sslsocket_class._create( File "/usr/lib/python3.8/ssl.py", line 1040, in _create self.do_handshake() File "/usr/lib/python3.8/ssl.py", line 1309, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL] internal error (_ssl.c:1108) This is due to libssl1.1 being compiled with "-DOPENSSL_TLS_SECURITY_LEVEL=2" and gear forcing TLSv1.0 Extracted from ubuntu source package: The default security level for TLS connections was increased from level 1 to level 2. This moves from the 80 bit security level to the 112 bit security level and will require 2048 bit or larger RSA and DHE keys, 224 bit or larger ECC keys, SHA-2, TLSv1.2 or DTLSv1.2. Allowing to negotiate TLS to the highest available version between server and client solves the issue, provided that TLSv1.2 is useable. The option is supported by in the latest version of all pythons >=3.5 [1][2][3]. Unfortunately Xenial doesn't have latest 3.5 and lacks the ssl.PROTOCOL_TLS definition. We provide a fallback to select the highest version of TLS supported in that case. There is some risk using the fallback beacuse both the client and server need to agree on the version supported in this case. Xenial python 3.5 does support TLSv1_2 which means that for all practical purposes TLS v1.2 should be available on all platforms that gear runs avoiding this problem. Disable TLSv1.3: According to https://bugs.python.org/issue43622#msg389497, an event on ssl socket can happen without data being available at application level. As gear is using a polling loop with multiple file descriptors and ssl socket used as a blocking one, a blocked state could happen. This is highlighted by Zuul SSL test: TestSchedulerSSL, where such blocked state appears consistently. note: gear tests and zuul tests are ok when using TLSv1.2 but the previous behavior could also happen [1] https://docs.python.org/2.7/library/ssl.html?highlight=protocol_tls#ssl.PROTOCOL_TLS [2] https://docs.python.org/3.5/library/ssl.html?highlight=protocol_tls#ssl.PROTOCOL_TLS [3] https://docs.python.org/3/library/ssl.html?highlight=protocol_tls#ssl.PROTOCOL_TLS Change-Id: I5efb6c0576987815c5b93f8bc4020cdee2898d04
* Merge "wakeConnections: Randomize connections before scanning them"Zuul2021-03-101-1/+8
|\
| * wakeConnections: Randomize connections before scanning themAhmon Dancy2021-03-101-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | gear/__init__.py: Modified Server.wakeConnections() so that it randomizes the list of active connections before sending out NOOP's to them. This will hopefully spread workload across machines more evenly when there are multiple workers per machine. Reference: https://phabricator.wikimedia.org/T258630 Change-Id: I05dcb9fa383f3aefc8b5b1bb9dd8b3ff6ff7f37d
* | Merge "Move handleDisconnect into BaseClientServer"Zuul2021-03-101-11/+11
|\ \
| * | Move handleDisconnect into BaseClientServerTobias Henkel2020-03-241-11/+11
| | | | | | | | | | | | | | | | | | | | | It's called from there using self.handleDisconnect so define it there as well. Change-Id: I90fef55a79168e082e69f81a717c08badd4163a9
* | | Merge "remove unicode from code"Zuul2021-03-101-10/+10
|\ \ \
| * | | remove unicode from codeliyou012021-01-071-10/+10
| | | | | | | | | | | | | | | | Change-Id: If93af8c01cbbaad4e64dc78fb651815a4de6e741
* | | | Merge "Modify connection timeout process"Zuul2021-03-101-9/+12
|\ \ \ \ | |_|_|/ |/| | |
| * | | Modify connection timeout processshangxdy2018-01-261-9/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When connection to gearman server is timeout, it's necessary to release lock before raising a exception, otherwise the client may be dead locked. Change-Id: Idc9c9c4bd439b122dc7855ca05d962c4e6687829 Signed-off-by: shangxdy <shang.xiaodong@zte.com.cn>
* | | | Added Docker image buildsMohammed Naser2021-02-112-0/+92
| |/ / |/| | | | | | | | Change-Id: If0b5c982020faf0f512a1c0d9b7b495789b9aa4f
* | | Merge "Bump crypto requirement to accomodate security standards"Zuul2020-12-101-2/+2
|\ \ \
| * | | Bump crypto requirement to accomodate security standardsFabien Boucher2020-07-211-2/+2
| | |/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Depends-on: https://review.opendev.org/742165 On Fedora rawhide the gear package no longer build. https://koschei.fedoraproject.org/package/python-gear? This patch ensures that the ssl engine does not complains about: - ssl.SSLError: [SSL: EE_KEY_TOO_SMALL] ee key too small (_ssl.c:2951) - ssl.SSLError: [SSL: CA_MD_TOO_WEAK] ca md too weak (_ssl.c:2951) To reproduce the issue: podman run -it --root fedora:rawhide dnf install git libffi-devel python-devel tox gcc git clone https://opendev.org/opendev/gear.git && cd gear tox -epy39 tox -epy38 Change-Id: I57cd9c4750f27b7b76e92a0eef03e7de70c13dd5
* | | use python3 as context for build-python-releaseFabien Boucher2020-09-091-1/+3
|/ / | | | | | | Change-Id: Iebda27b20aaa94539766feeeb0236675c2ebdac6
* | Revert "Add BSD/Darwin support."0.15.1Tobias Henkel2020-02-173-90/+27
| | | | | | | | | | | | | | | | | | This floods the zuul test logs with poll messages which indicates that something changed to a busy loop. This reverts commit 103ad3e8ed78c6895c425115fda45f28441bbfaf. Change-Id: Id3347136507e7e65ccde937f1c2fd303aa3dfbbe
* | Merge "Ignore keepalive on unsupported platforms"0.15.0Zuul2020-02-031-2/+8
|\ \
| * | Ignore keepalive on unsupported platformsTobias Henkel2020-01-161-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | Gear currently supports keepalive only on linux platforms. On mac the socket must be configured differently. For now just ignore the keepalive flag in this case and emit a warning. Change-Id: I276967b720742fa64e5bc6eb769c75590141275c
* | | Merge "Add BSD/Darwin support."Zuul2020-02-033-27/+90
|\ \ \ | |/ /
| * | Add BSD/Darwin support.Tobias Henkel2019-10-153-27/+90
| | | | | | | | | | | | | | | | | | | | | | | | Switch between Epoll and Poll depending on the OS capabilities. Change-Id: Iaf1324d0c9d43c76e3f228f1a176e453a82a71a4 Co-Authored-By: Jan Kubovy <jan.kubovy@bmw.de> Co-Authored-By: Tobias Henkel <tobias.henkel@bmw.de>
* | | packaging: updated project urlsSorin Sbarnea2020-01-211-1/+6
| | | | | | | | | | | | | | | | | | Make package listing on PYPA more informative for the users. Change-Id: Ifd20ae3ee006c8018d04782f3328799769ce4f30
* | | Merge "Add in-repo zuul config"Zuul2019-10-141-0/+23
|\ \ \ | |/ / |/| |
| * | Add in-repo zuul configJames E. Blair2019-10-141-0/+23
| | | | | | | | | | | | | | | | | | | | | | | | We're moving this from the openstack to the opendev tenant. We should move the jobs in-repo at the same time. Depends-On: https://review.opendev.org/688451 Change-Id: I23b4a0d78d515557bfb676f206dcc0a91ecb8502
* | | Fix documentation buildsMohammed Naser2019-10-144-5/+9
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch does a few things to fix the documentation builds for this project - Move requirements to doc/requirements.txt for building docs to avoid installing extra dependencies. - Bump sphinx version to a newer release which is compatible with sphinxcontrib-programoutput - Remove default theme option to use the latest Sphinx theme that is shipped directly from upstream. - Bumped basepython for documentation jobs to Python 3. These are all squashed because the job is currently broken. Change-Id: Ib998923a5daaa5e9d3ddc748b76b6304e5c39b22
* | Merge "Add support for server name indication"0.14.0Zuul2019-06-171-11/+11
|\ \
| * | Add support for server name indicationTobias Henkel2019-05-041-11/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | According to the python docs [1] it is recommended to use SSLContext.wrap_socket to create an ssl connection since Python 3.2 and 2.7.9. This enables us to also leverage server name indication (SNI). One use case where SNI is beneficial is an easy and standard way to route traffic into an Openshift cluster. The most common way to get traffic into an Openshift cluster is using a routes. The routes in an openshift cluster work with either HTTP, HTTPS with SNI or TLS with SNI [2]. TLS with SNI in this case also works with non-http connections like gearman is using. [1] https://docs.python.org/3/library/ssl.html#socket-creation [2] https://docs.okd.io/3.11/dev_guide/expose_service/expose_internal_ip_router.html#overview Change-Id: I19c1edc4a14a303d2a91894e0065c8d31f89ce24
* | | Merge "add missing str to bytes conversion for Python3"Zuul2019-05-061-1/+1
|\ \ \ | |/ / |/| |
| * | add missing str to bytes conversion for Python3Benoit Bayszczak2018-08-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Using Python3, Gearman unexpectedly closed the socket with a client when typing the 'workers' command. gearman-debug.log: File "/usr/local/lib/python3.5/dist-packages/gear/__init__.py", line 3250, in handleWorkers (fd, ip, client_id.decode('utf8'), AttributeError: 'str' object has no attribute 'decode' Change-Id: I610bd44c76a0e52f8d4e8f24c82c636d4ebef0ae
* | | OpenDev Migration PatchOpenDev Sysadmins2019-04-191-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit was bulk generated and pushed by the OpenDev sysadmins as a part of the Git hosting and code review systems migration detailed in these mailing list posts: http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html Attempts have been made to correct repository namespaces and hostnames based on simple pattern matching, but it's possible some were updated incorrectly or missed entirely. Please reach out to us via the contact information listed at https://opendev.org/ with any questions you may have.
* | | Merge "Prefer ipv6 for listen addrs if available"0.13.0Zuul2019-01-281-3/+8
|\ \ \
| * | | Prefer ipv6 for listen addrs if availableClark Boylan2018-10-191-3/+8
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If the listen address allows for ipv4 or ipv6 values we want to prefer ipv6 if the host is configured with working ivp6. We add the ai_flag AF_ADDRCONFIG to filter out ipv6 (and ipv4) if the host isn't configure for this AF_INET version. Then we sort based on the family to prefer ipv6 over ipv4. The reason for this is clients will prefer ipv6 before falling back to ipv4 when attempting to connect to a hostname. If the server isn't listening on ipv6 this makes new connections happen slowly. Change-Id: I9f7a235b04068856c6cceeb2c230f3b56945572e
* | | Merge "Add support for keepalive to client"Zuul2018-12-111-3/+28
|\ \ \
| * | | Add support for keepalive to clientTobias Henkel2018-09-041-3/+28
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A gearman client only waiting for jobs will wait indefinitely if the gearman server vanishes (e.g. due to a VM crash). In this case there is no traffic on the connection and the client blocks forever if there is nothing in between that forcefully terminates the connection. Adding tcp keepalive can mitigate that and the connection will be terminated by the kernel in this situation which then triggers a reconnect. Change-Id: I8589cd45450245a25539c051355b38d16ee9f4b9
* | | Change openstack-dev to openstack-discussqingszhao2018-12-041-1/+1
|/ / | | | | | | | | | | Mailinglists have been updated. Openstack-discuss replaces openstack-dev. Change-Id: Ia1f74031b69da2d50f0404fac4698823526ecd83
* | Add --listen-address flag to geard0.12.0Paul Belanger2018-04-161-0/+4
| | | | | | | | | | | | | | | | Add the ability for an operator to control which interface to listen on. By default we use None to maintain backwards compatibility. Change-Id: I14c13ff500317d5a7b580e1b2a7f798a8db5de1d Signed-off-by: Paul Belanger <pabelanger@redhat.com>
* | Build universal wheelsPaul Belanger2018-04-131-0/+7
| | | | | | | | | | | | | | | | | | | | | | It is possible to use python3 for gear, so support both python2 and python3 wheels. Also update classifier to indicate which versions of python we support. Change-Id: I74384871cabc8d5b22f2d7555201c21f1bf37099 Signed-off-by: Paul Belanger <pabelanger@redhat.com>
* | Automatically send GRAB_JOB after CAN_DOJames E. Blair2018-02-022-0/+36
| | | | | | | | | | | | | | | | | | After registering a function, if a connection is sleeping (ie, waiting for a NOOP to wake it up), cause it to send another grab_job in case the newly registered job is something it can handle. Change-Id: Ibea13726f4a451ebc67850b17e168bd4accfbc0b
* | Make workers admin command py3 safe0.11.1James E. Blair2018-02-011-2/+3
|/ | | | Change-Id: I05598ceea20169a625bbb47f15288e6eadbc88d2
* server: make stats more efficient0.11.0James E. Blair2017-10-301-26/+15
| | | | | | | | | | | Keep track of the three main queue stats we report, so that we don't have to iterate over all the queued jobs to determine whether they are running. Also, drop the workers stat because it's not very useful and not entirely trivial to calculate. Change-Id: Id42a05e5626096d1100a9cb9e8166c8ec5103b41
* Merge "Add a send lock to the base Connection class"0.10.1Jenkins2017-10-091-11/+13
|\
| * Add a send lock to the base Connection classJames E. Blair2017-09-301-11/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The sendRaw method (and therefore sendPacket) was originally thread safe by virtue of consisting of a single socket.send() call, but when we added SSL, we added a loop within the method to handle the increased likelihood that not all data would be sent in one call. Of course, the method should have been written this way to start with. However, this means that we can end up with partial packets being sent before a context switch to another thread which may also want to send a packet. To handle that case, place the entire method in a lock. Note, this doesn't affect server connections as they use a non-blocking connection which has a send queue, so only one thread ever actuall transmits. Change-Id: I3bda6fda5f762d18f28b56a43b7dc28f37dbc427
* | Server: support background jobs0.10.0James E. Blair2017-09-132-9/+42
|/ | | | Change-Id: Ic15ab05c16f143f1d557d935aecb6d0afe419d59
* Fix exception setter0.9.1James E. Blair2017-05-181-1/+1
| | | | | | This method had a typo. Change-Id: I49b745387626f664235998de52d4c58927149b4d
* Merge "Replace assertEquals with assertEqual"0.9.0Jenkins2017-05-171-10/+10
|\
| * Replace assertEquals with assertEqualLuong Anh Tuan2016-11-221-10/+10
| | | | | | | | | | | | | | | | | | | | | | The method assertEquals has been deprecated since python 2.7. http://docs.python.org/2/library/unittest.html#deprecated-aliases Also in Python 3, a deprecated warning is raised when using assertEquals therefore we should use assertEqual instead. Change-Id: Ic99ea3f57bb6fd986ce2626e800e3d310a2e3df7 Closes-Bug: #1218185
* | Merge "Provide TextJob and TextWorker for convenience"Jenkins2017-05-172-105/+389
|\ \
View Lorry Controller Status