summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/testing/plugin
Commit message (Collapse)AuthorAgeFilesLines
* Improved wheel pipelineFederico Caselli2023-03-171-0/+34
| | | | | | | | - ensure that the compiled extensions are used - speed up job by parallelizing more Fixes: #9434 Change-Id: Ief750b28733ba24bb5ff8c105e1a4c9b7b928700
* happy new year 2023Mike Bayer2023-01-031-1/+1
| | | | Change-Id: I625af65b3fb1815b1af17dc2ef47dd697fdc3fb1
* Rewrite positional handling, test for "numeric"Federico Caselli2022-12-051-0/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Changed how the positional compilation is performed. It's rendered by the compiler the same as the pyformat compilation. The string is then processed to replace the placeholders with the correct ones, and to obtain the correct order of the parameters. This vastly simplifies the computation of the order of the parameters, that in case of nested CTE is very hard to compute correctly. Reworked how numeric paramstyle behavers: - added support for repeated parameter, without duplicating them like in normal positional dialects - implement insertmany support. This requires that the dialect supports out of order placehoders, since all parameters that are not part of the VALUES clauses are placed at the beginning of the parameter tuple - support for different identifiers for a numeric parameter. It's for example possible to use postgresql style placeholder $1, $2, etc Added two new dialect based on sqlite to test "numeric" fully using both :1 style and $1 style. Includes a workaround for SQLite's not-really-correct numeric implementation. Changed parmstyle of asyncpg dialect to use numeric, rendering with its native $ identifiers Fixes: #8926 Fixes: #8849 Change-Id: I7c640467d49adfe6d795cc84296fc7403dcad4d6
* Try running pyupgrade on the codeFederico Caselli2022-11-161-2/+2
| | | | | | | | command run is "pyupgrade --py37-plus --keep-runtime-typing --keep-percent-format <files...>" pyupgrade will change assert_ to assertTrue. That was reverted since assertTrue does not exists in sqlalchemy fixtures Change-Id: Ie1ed2675c7b11d893d78e028aad0d1576baebb55
* repair --disable-asyncio parameterMike Bayer2022-11-111-1/+1
| | | | | | | | | | | Fixed issue where the ``--disable-asyncio`` parameter to the test suite would fail to not actually run greenlet tests and would also not prevent the suite from using a "wrapping" greenlet for the whole suite. This parameter now ensures that no greenlet or asyncio use will occur within the entire run when set. Fixes: #8793 Change-Id: I87b510846b2cc24413cd54e7b7136e91aad3c309
* Tighten password security by removing `URL.__str__`Yassen Damyanov2022-09-231-1/+4
| | | | | | | | | | | | | | | For improved security, the :class:`_url.URL` object will now use password obfuscation by default when ``str(url)`` is called. To stringify a URL with cleartext password, the :meth:`_url.URL.render_as_string` may be used, passing the :paramref:`_url.URL.render_as_string.hide_password` parameter as ``False``. Thanks to our contributors for this pull request. Fixes: #8567 Closes: #8563 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8563 Pull-request-sha: d1f1127f753849eb70b8d6cc64badf34e1b9219b Change-Id: If756c8073ff99ac83876d9833c8fe1d7c76211f9
* Use cibuildwheel to create wheelsTrevor Gross2022-08-292-4/+13
| | | | | | | | | | | | | | | | Using cibuildwheel the following wheels are created: - windows x64 and x84 - macos x64 and arm - linux x64 and arm on manylinux and mosulinux (for alpine) - create a pure python wheel (for pypy and other archs) Fixes: #6702 Fixes: #7607 Closes: #7992 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7992 Pull-request-sha: 61d5e24e5b76c97db73aa2507af7f5c2d3a948fa Change-Id: If0c0b353766e0b61d421789d619eb2b940c08ad0
* Add ability to test using thick mode with oracledbFederico Caselli2022-06-201-6/+12
| | | | | Change-Id: Iee14750ba20422931bde4d61eaa570af482c7d8b References: #8147
* rearchitect reflection for batched performanceFederico Caselli2022-06-181-7/+9
| | | | | | | | | | | | | | | | | | | | | | | Rearchitected the schema reflection API to allow some dialects to make use of high performing batch queries to reflect the schemas of many tables at once using much fewer queries. The new performance features are targeted first at the PostgreSQL and Oracle backends, and may be applied to any dialect that makes use of SELECT queries against system catalog tables to reflect tables (currently this omits the MySQL and SQLite dialects which instead make use of parsing the "CREATE TABLE" statement, however these dialects do not have a pre-existing performance issue with reflection. MS SQL Server is still a TODO). The new API is backwards compatible with the previous system, and should require no changes to third party dialects to retain compatibility; third party dialects can also opt into the new system by implementing batched queries for schema reflection. Along with this change is an updated reflection API that is fully :pep:`484` typed, features many new methods and some changes. Fixes: #4379 Change-Id: I897ec09843543aa7012bcdce758792ed3d415d08
* inline mypy config; files ignoring type errors for the momentMike Bayer2022-04-283-0/+6
| | | | | | | | | | | | | | | | | | | to simplify pyproject.toml change the remaining files that aren't going to be typed on this first pass (unless of course someone wants to type some of these) to include # mypy: ignore-errors. for the moment, only a handful of ORM modules are to have more type checking implemented. It's important that ignore-errors is used and not "# type: ignore", as in the latter case, mypy doesn't even read the existing types in the file, which makes it impossible to type any files that refer to those modules at all. to simplify ongoing typing work use inline mypy config for remaining files that are "done" for now, indicating the level of type checking they currently have. Change-Id: I98669c1a305c2f0adba85d10b5425541f3fe9533
* pep-484: ORM public API, constructorsMike Bayer2022-04-201-16/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | for the moment, abandoning using @overload with relationship() and mapped_column(). The overloads are very difficult to get working at all, and the overloads that were there all wouldn't pass on mypy. various techniques of getting them to "work", meaning having right hand side dictate what's legal on the left, have mixed success and wont give consistent results; additionally, it's legal to have Optional / non-optional independent of nullable in any case for columns. relationship cases are less ambiguous but mypy was not going along with things. we have a comprehensive system of allowing left side annotations to drive the right side, in the absense of explicit settings on the right. so type-centric SQLAlchemy will be left-side driven just like dataclasses, and the various flags and switches on the right side will just not be needed very much. in other matters, one surprise, forgot to remove string support from orm.join(A, B, "somename") or do deprecations for it in 1.4. This is a really not-directly-used structure barely mentioned in the docs for many years, the example shows a relationship being used, not a string, so we will just change it to raise the usual error here. Change-Id: Iefbbb8d34548b538023890ab8b7c9a5d9496ec6e
* use code generation for scoped_sessionMike Bayer2022-04-122-13/+22
| | | | | | | | | | | | | | | | | | | our decorator thing generates code in any case, so point it at the file itself to generate real code for the blocks rather than doing things dynamically. this will allow typing tools to have no problem whatsoever and we also reduce import time overhead. file size will be a lot bigger though, shrugs. syntax / dupe method / etc. checking will be accomplished by our existing linting / typing / formatting tools. As we are also using "from __future__ import annotations", we also no longer have to apply quotes to generated annotations. Change-Id: I20962cb65bda63ff0fb67357ab346e9b1ef4f108
* establish mypy / typing approach for v2.0Mike Bayer2022-02-131-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | large patch to get ORM / typing efforts started. this is to support adding new test cases to mypy, support dropping sqlalchemy2-stubs entirely from the test suite, validate major ORM typing reorganization to eliminate the need for the mypy plugin. * New declarative approach which uses annotation introspection, fixes: #7535 * Mapped[] is now at the base of all ORM constructs that find themselves in classes, to support direct typing without plugins * Mypy plugin updated for new typing structures * Mypy test suite broken out into "plugin" tests vs. "plain" tests, and enhanced to better support test structures where we assert that various objects are introspected by the type checker as we expect. as we go forward with typing, we will add new use cases to "plain" where we can assert that types are introspected as we expect. * For typing support, users will be much more exposed to the class names of things. Add these all to "sqlalchemy" import space. * Column(ForeignKey()) no longer needs to be `@declared_attr` if the FK refers to a remote table * composite() attributes mapped to a dataclass no longer need to implement a `__composite_values__()` method * with_variant() accepts multiple dialect names Change-Id: I22797c0be73a8fbbd2d6f5e0c0b7258b17fe145d Fixes: #7535 Fixes: #7551 References: #6810
* join to existing mark expr with "and"Mike Bayer2022-01-251-6/+6
| | | | | | | | | | | ca48f461b2dcac2970829e4e0 considered an existing mark expression plus legacy tags to be an error condition; however these can be joined by "and" and will in our use case do the right thing. The github action scripts make use of legacy tags. We can change that also but I want to just make sure this combination works fully as well. Change-Id: Ifc506de3dd961c01d68d594ec2f5b2c9a0bbad31
* replace test tags with pytest.markMike Bayer2022-01-252-85/+90
| | | | | | | | | | | | | | | | | | | | | | | | | | replaced the __tags__ class attribute and the --exclude-tags / --include-tags test runner options with regular pytest.mark names so that we can take advantage of mark expressions. options --nomemory, --notimingintensive, --backend-only, --exclude-tags, --include-tags remain as legacy but make use of pytest mark for implemementation. Added a "mypy" mark for the section of tests that are doing mypy integration tests. The __backend__ and __sparse_backend__ class attributes also use pytest marks for their implementation, which also allows the marks "backend" and "sparse_backend" to be used explicitly. Also removed the no longer used "--cdecimal" option as this was python 2 specific. in theory, the usage of pytest marks could expand such that the whole exclusions system would be based on it, but this does not seem to have any advantage at the moment. Change-Id: Ideeb57d9d49f0efc7fc0b6b923b31207ab783025
* dont use exception catches for warnings; modernize xdist detectionMike Bayer2022-01-221-12/+7
| | | | | | | | | | | | | | | | | Improvements to the test suite's integration with pytest such that the "warnings" plugin, if manually enabled, will not interfere with the test suite, such that third parties can enable the warnings plugin or make use of the ``-W`` parameter and SQLAlchemy's test suite will continue to pass. Additionally, modernized the detection of the "pytest-xdist" plugin so that plugins can be globally disabled using PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 without breaking the test suite if xdist were still installed. Warning filters that promote deprecation warnings to errors are now localized to SQLAlchemy-specific warnings, or within SQLAlchemy-specific sources for general Python deprecation warnings, so that non-SQLAlchemy deprecation warnings emitted from pytest plugins should also not impact the test suite. Fixes: #7599 Change-Id: Ibcf09af25228d39ee5a943fda82d8a9302433726
* happy new year 2022Mike Bayer2022-01-061-1/+1
| | | | Change-Id: I49abf2607e0eb0623650efdf0091b1fb3db737ea
* Remove redundant code for EOL Python <= 3.6Hugo van Kemenade2022-01-061-15/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | <!-- Provide a general summary of your proposed changes in the Title field above --> ### Description <!-- Describe your changes in detail --> There's a few bits and pieces of code to support Python <= 3.6 which are no longer needed and can be removed, to slightly simplify the codebase. ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - Good to go, no issue or tests are needed - [x] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. - [ ] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. **Have a nice day!** Closes: #7544 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7544 Pull-request-sha: 282b4a91282902a57807aa2541b75b272b547127 Change-Id: I9ddf15fcf72551d52e3f027f337c7fee4aa9083b
* Update Black's target-version to py37Hugo van Kemenade2022-01-051-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | <!-- Provide a general summary of your proposed changes in the Title field above --> ### Description <!-- Describe your changes in detail --> Black's `target-version` was still set to `['py27', 'py36']`. Set it to `[py37]` instead. Also update Black and other pre-commit hooks and re-format with Black. ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - Good to go, no issue or tests are needed - [ ] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. - [ ] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. **Have a nice day!** Closes: #7536 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7536 Pull-request-sha: b3aedf5570d7e0ba6c354e5989835260d0591b08 Change-Id: I8be85636fd2c9449b07a8626050c8bd35bd119d5
* Merge "factor out UnboundLoad and rearchitect strategy_options.py" into mainmike bayer2021-12-271-1/+1
|\
| * factor out UnboundLoad and rearchitect strategy_options.pyMike Bayer2021-12-271-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The architecture of Load is mostly rewritten here. The change includes removal of the "pluggable" aspect of the loader options, which would patch new methods onto Load. This has been replaced by normal methods that respond normally to typing annotations. As part of this change, the bake_loaders() and unbake_loaders() options, which have no effect since 1.4 and were unlikely to be in any common use, have been removed. Additionally, to support annotations for methods that make use of @decorator, @generative etc., modified format_argspec_plus to no longer return "args", instead returns "grouped_args" which is always grouped and allows return annotations to format correctly. Fixes: #6986 Change-Id: I6117c642345cdde65a64389bba6057ddd5374427
* | implement cython for cache_anon_map, prefix_anon_mapMike Bayer2021-12-211-0/+15
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These are small bits where cache_anon_map in particular is part of the cache key generation scheme which is a key target for cython. changing such a tiny element of the cache key gen is doing basically nothing yet, as the cython impl is mostly the exact same speed as the python one. I guess for cython to be effective we'd need to redo the whole cache key generation and possibly not use the same kinds of structures, which might not be very easy to do. Additionally, some cython runtime import errors are being observed on jenkins, add an upfront check to the test suite to indicate if the expected build succeeded when REQUIRE_SQLALCHEMY_CEXT is set. Running case CacheAnonMap Running python .... Done Running cython .... Done | python | cython | cy / py | test_get_anon_non_present| 0.301266758 | 0.231203834 | 0.767438915 | test_get_anon_present| 0.300919362 | 0.227336695 | 0.755473803 | test_has_key_non_present| 0.152725077 | 0.133191719 | 0.872101171 | test_has_key_present| 0.152689778 | 0.133673095 | 0.875455428 | Running case PrefixAnonMap Running python .. Done Running cython .. Done | python | cython | cy / py | test_apply_non_present| 0.358715744 | 0.335245703 | 0.934572034 | test_apply_present | 0.354434996 | 0.338579782 | 0.955266229 | Change-Id: I0d3f1dd285c044afc234479141d831b2ee0455be
* adapt pytest plugin to support pytest v7Federico Caselli2021-11-293-78/+90
| | | | | | | | | | | | Implemented support for the test suite to run correctly under Pytest 7. Previously, only Pytest 6.x was supported for Python 3, however the version was not pinned on the upper bound in tox.ini. Pytest is not pinned in tox.ini to be lower than version 8 so that SQLAlchemy versions released with the current codebase will be able to be tested under tox without changes to the environment. Much thanks to the Pytest developers for their help with this issue. Change-Id: I3b12166199be2b913ee16e78b3ebbff415654396
* Added support for ``psycopg`` dialect.Federico Caselli2021-11-261-1/+2
| | | | | | | Both sync and async versions are supported. Fixes: #6842 Change-Id: I57751c5028acebfc6f9c43572562405453a2f2a4
* Clean up most py3k compatFederico Caselli2021-11-242-23/+6
| | | | Change-Id: I8172fdcc3103ff92aa049827728484c8779af6b7
* Remove object in class definitionFederico Caselli2021-11-221-1/+1
| | | | | References: #4600 Change-Id: I2a62ddfe00bc562720f0eae700a497495d7a987a
* First round of removal of python 2Federico Caselli2021-11-013-154/+5
| | | | | References: #4600 Change-Id: I61e35bc93fe95610ae75b31c18a3282558cd4ffe
* rename elements to mainMike Bayer2021-10-112-2/+2
| | | | | | | There are still some SQLite / MySQL specific occurrences of "master" but this is most of it. Change-Id: I0144c992e2f0207777e20e058b63a11c031986b9
* Modernize tests - calling_mapper_directlyGord Thompson2021-09-301-23/+24
| | | | | | | | | | | | | a few changes for py2k: * map_imperatively() includes the check that a class is being sent, this was only working for mapper() before * the test suite didn't place the py2k "autouse" workaround in the correct order, seemingly, tried to adjust the per-test ordering setup in pytestplugin.py Change-Id: I4cc39630724e810953cfda7b2afdadc8b948e3c2
* Surface driver connection object when using a proxied dialectFederico Caselli2021-09-171-5/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | Improve the interface used by adapted drivers, like the asyncio ones, to access the actual connection object returned by the driver. The :class:`_engine._ConnectionRecord` and :class:`_engine._ConnectionFairy` now have two new attributes: * ``dbapi_connection`` always represents a DBAPI compatible object. For pep-249 drivers, this is the DBAPI connection as it always has been, previously accessed under the ``.connection`` attribute. For asyncio drivers that SQLAlchemy adapts into a pep-249 interface, the returned object will normally be a SQLAlchemy adaption object called :class:`_engine.AdaptedConnection`. * ``driver_connection`` always represents the actual connection object maintained by the third party pep-249 DBAPI or async driver in use. For standard pep-249 DBAPIs, this will always be the same object as that of the ``dbapi_connection``. For an asyncio driver, it will be the underlying asyncio-only connection object. The ``.connection`` attribute remains available and is now a legacy alias of ``.dbapi_connection``. Fixes: #6832 Change-Id: Ib72f97deefca96dce4e61e7c38ba430068d6a82e
* Replace all http:// links to https://Federico Caselli2021-07-041-1/+1
| | | | | | Also replace http://pypi.python.org/pypi with https://pypi.org/project Change-Id: I84b5005c39969a82140706472989f2a30b0c7685
* Remove pep484 type comments from the codeFederico Caselli2021-05-161-13/+2
| | | | | | | | | | | | | Current effort is around the stub package, and having typing in two places makes thing worse, since the types here are usually outdated compared to the version in the stubs. Once v2 gets under way we can start consolidating the types here. Fixes: #6461 Change-Id: I7132a444bd7138123074bf5bc664b4bb119a85ce
* mypy: load extra test files from environment variableMaico Timmerman2021-05-041-0/+9
| | | | | | | | | | Add --mypy-extra-test-path parameter to pytest execution to list extra directories to load test files from. This enables the stubs repo to load this plugin and specify it's own test directory to run mypy tests. Supports both single file tests and incremental tests based on patch files. Change-Id: Id6424ff15b2f527183b9713384df3d625a8e6eb8
* Ensure sparse backend selection sorts on driver alsoMike Bayer2021-04-221-0/+1
| | | | | | | | | | | | | | | | Because we have --dbdriver now, we usually have lots of db drivers queued up which affect the selection of backend tests. the sort in possible_configs_for_cls() is used so that when we are only picking some backends due to the sparse flag, we get the same answer each time otherwise pytest-xdist raises an error if two workers have different tests. adding the sort for driver fixes an issue on jenkins master build where the memusage test run frequently failed due to mismatched test collection. Change-Id: I8584c1a3a4e5b759c0e5a2602a93432d9cc8168d
* CAST the elements in ARRAYs when using psycopg2Federico Caselli2021-03-151-1/+3
| | | | | | | | | | | | Adjusted the psycopg2 dialect to emit an explicit PostgreSQL-style cast for bound parameters that contain ARRAY elements. This allows the full range of datatypes to function correctly within arrays. The asyncpg dialect already generated these internal casts in the final statement. This also includes support for array slice updates as well as the PostgreSQL-specific :meth:`_postgresql.ARRAY.contains` method. Fixes: #6023 Change-Id: Ia7519ac4371a635f05ac69a3a4d0f4e6d2f04cad
* Create explicit GC ordering between ConnectionFairy/ConnectionRecordMike Bayer2021-01-151-0/+1
| | | | | | | | | | | | | | Fixed issue where connection pool would not return connections to the pool or otherwise be finalized upon garbage collection under pypy if the checked out connection fell out of scope without being closed. This is a long standing issue due to pypy's difference in GC behavior that does not call weakref finalizers if they are relative to another object that is also being garbage collected. A strong reference to the related record is now maintained so that the weakref has a strong-referenced "base" to trigger off of. Fixes: #5842 Change-Id: Id5448fdacb6cceaac1ea40b2fbc851f052ed8e86
* reinvent xdist hooks in terms of pytest fixturesMike Bayer2021-01-134-46/+298
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To allow the "connection" pytest fixture and others work correctly in conjunction with setup/teardown that expects to be external to the transaction, remove and prevent any usage of "xdist" style names that are hardcoded by pytest to run inside of fixtures, even function level ones. Instead use pytest autouse fixtures to implement our own r"setup|teardown_test(?:_class)?" methods so that we can ensure function-scoped fixtures are run within them. A new more explicit flow is set up within plugin_base and pytestplugin such that the order of setup/teardown steps, which there are now many, is fully documented and controllable. New granularity has been added to the test teardown phase to distinguish between "end of the test" when lock-holding structures on connections should be released to allow for table drops, vs. "end of the test plus its teardown steps" when we can perform final cleanup on connections and run assertions that everything is closed out. From there we can remove most of the defensive "tear down everything" logic inside of engines which for many years would frequently dispose of pools over and over again, creating for a broken and expensive connection flow. A quick test shows that running test/sql/ against a single Postgresql engine with the new approach uses 75% fewer new connections, creating 42 new connections total, vs. 164 new connections total with the previous system. As part of this, the new fixtures metadata/connection/future_connection have been integrated such that they can be combined together effectively. The fixture_session(), provide_metadata() fixtures have been improved, including that fixture_session() now strongly references sessions which are explicitly torn down before table drops occur afer a test. Major changes have been made to the ConnectionKiller such that it now features different "scopes" for testing engines and will limit its cleanup to those testing engines corresponding to end of test, end of test class, or end of test session. The system by which it tracks DBAPI connections has been reworked, is ultimately somewhat similar to how it worked before but is organized more clearly along with the proxy-tracking logic. A "testing_engine" fixture is also added that works as a pytest fixture rather than a standalone function. The connection cleanup logic should now be very robust, as we now can use the same global connection pools for the whole suite without ever disposing them, while also running a query for PostgreSQL locks remaining after every test and assert there are no open transactions leaking between tests at all. Additional steps are added that also accommodate for asyncio connections not explicitly closed, as is the case for legacy sync-style tests as well as the async tests themselves. As always, hundreds of tests are further refined to use the new fixtures where problems with loose connections were identified, largely as a result of the new PostgreSQL assertions, many more tests have moved from legacy patterns into the newest. An unfortunate discovery during the creation of this system is that autouse fixtures (as well as if they are set up by @pytest.mark.usefixtures) are not usable at our current scale with pytest 4.6.11 running under Python 2. It's unclear if this is due to the older version of pytest or how it implements itself for Python 2, as well as if the issue is CPU slowness or just large memory use, but collecting the full span of tests takes over a minute for a single process when any autouse fixtures are in place and on CI the jobs just time out after ten minutes. So at the moment this patch also reinvents a small version of "autouse" fixtures when py2k is running, which skips generating the real fixture and instead uses two global pytest fixtures (which don't seem to impact performance) to invoke the "autouse" fixtures ourselves outside of pytest. This will limit our ability to do more with fixtures until we can remove py2k support. py.test is still observed to be much slower in collection in the 4.6.11 version compared to modern 6.2 versions, so add support for new TOX_POSTGRESQL_PY2K and TOX_MYSQL_PY2K environment variables that will run the suite for fewer backends under Python 2. For Python 3 pin pytest to modern 6.2 versions where performance for collection has been improved greatly. Includes the following improvements: Fixed bug in asyncio connection pool where ``asyncio.TimeoutError`` would be raised rather than :class:`.exc.TimeoutError`. Also repaired the :paramref:`_sa.create_engine.pool_timeout` parameter set to zero when using the async engine, which previously would ignore the timeout and block rather than timing out immediately as is the behavior with regular :class:`.QueuePool`. For asyncio the connection pool will now also not interact at all with an asyncio connection whose ConnectionFairy is being garbage collected; a warning that the connection was not properly closed is emitted and the connection is discarded. Within the test suite the ConnectionKiller is now maintaining strong references to all DBAPI connections and ensuring they are released when tests end, including those whose ConnectionFairy proxies are GCed. Identified cx_Oracle.stmtcachesize as a major factor in Oracle test scalability issues, this can be reset on a per-test basis rather than setting it to zero across the board. the addition of this flag has resolved the long-standing oracle "two task" error problem. For SQL Server, changed the temp table style used by the "suite" tests to be the double-pound-sign, i.e. global, variety, which is much easier to test generically. There are already reflection tests that are more finely tuned to both styles of temp table within the mssql test suite. Additionally, added an extra step to the "dropfirst" mechanism for SQL Server that will remove all foreign key constraints first as some issues were observed when using this flag when multiple schemas had not been torn down. Identified and fixed two subtle failure modes in the engine, when commit/rollback fails in a begin() context manager, the connection is explicitly closed, and when "initialize()" fails on the first new connection of a dialect, the transactional state on that connection is still rolled back. Fixes: #5826 Fixes: #5827 Change-Id: Ib1d05cb8c7cf84f9a4bfd23df397dc23c9329bfe
* happy new yearMike Bayer2021-01-041-1/+1
| | | | Change-Id: Ic5bb19ca8be3cb47c95a0d3315d84cb484bac47c
* remove metadata.bind use from test suiteMike Bayer2021-01-031-64/+4
| | | | | | | | | | | | | | importantly this means we can remove bound metadata from the fixtures that are used by Alembic's test suite. hopefully this is the last one that has to happen to allow Alembic to be fully 1.4/2.0. Start moving from @testing.provide_metadata to a pytest metadata fixture. This does not seem to have any negative effects even though TablesTest uses a "self.metadata" attribute. Change-Id: Iae6ab95938a7e92b6d42086aec534af27b5577d3
* Repair async test refactorMike Bayer2021-01-021-2/+10
| | | | | | | | | | | | | | | in I4940d184a4dc790782fcddfb9873af3cca844398 we reworked how async tests run but apparently the async tests in test/ext/asyncio are reporting success without being run. This patch pushes pytestplugin further so that it won't instrument any test or function overall that declares itself async. This removes the need for the __async_wrap__ flag and also allows us to use a more strict "run_async_test" function that always runs the asyncio event loop from the top. Also start working asyncio into main testing suite. Change-Id: If7144e951a9db67eb7ea73b377f81c4440d39819
* Support testing of async drivers without fallback modeFederico Caselli2020-12-302-45/+152
| | | | Change-Id: I4940d184a4dc790782fcddfb9873af3cca844398
* Repair mssql dep tests; have __only_on__ imply __backend__Mike Bayer2020-12-191-0/+1
| | | | | | | | | | | | | | | | | | CI missed a few SQL Server tests because we run mssql-backendonly in the gerrit job. As there was a test that was "only on" mssql but didn't have backendonly, it never got run and then fails in master where we run mssql fully. Any suite that has an __only_on__ is inherently specific to a backend, so if present this should imply __backend__ so that it definitely runs when we have that backend present. This in turn meant we had to fix a few sqlite_file tests that weren't cleaning up or sharing well as they suddenly became backend tests under sqlite_file. Added a sqlite_file cleanup to test class cleanup for now. Change-Id: I9de1ceabd6596547a65c59059a55b7e5156103fd
* test fixes for oracle 18cMike Bayer2020-12-181-2/+5
| | | | Change-Id: I4968aa3bde3c4d11d7fe84f18b4a846ba357d16a
* generalize scoped_session proxying and apply to asyncio elementsMike Bayer2020-10-101-2/+2
| | | | | | | | | | | | | | | | | | | | Reworked the proxy creation used by scoped_session() to be based on fully copied code with augmented docstrings and moved it into langhelpers. asyncio session, engine, connection can now take advantage of it so that all non-async methods are availble. Overall implementation of most important accessors / methods on AsyncConnection, etc. , including awaitable versions of invalidate, execution_options, etc. In order to support an event dispatcher on the async classes while still allowing them to hold __slots__, make some adjustments to the event system to allow that to be present, at least rudimentally. Fixes: #5628 Change-Id: I5eb6929fc1e4fdac99e4b767dcfd49672d56e2b2
* add --notimingintensive; block from github jobsMike Bayer2020-10-071-0/+12
| | | | | | | | | | | | | | | | | | | | | | this provides a front-end option to disable tests marked as timing_intensive, all of which are in test_pool, which are more fragile and aren't consistent on the github runners. also remove /reduce unnecessary time.sleep() from two other pool tests that are not timing intensive. note that this removes test_hanging_connect_within_overflow from the github runs via the timing_intensive requirement. I've also removed MockReconnectTest from exclusions as those are really important tests and they use mocks so should not have platform dependent issues. Need to see what the windows failures are. Closes: #5633 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5633 Pull-request-sha: 166833e16ec342dfa10edb287d7aa495ddd1b59d Change-Id: Icb3d284a2a952e2495d80fa91e22e0b32a54340f
* Enable pypy tests on github workflowFederico Caselli2020-10-021-1/+1
| | | | | Fixes: #5223 Change-Id: I0952e54ed9af2952ea340be1945311376ffc1ad2
* Support pytest 6.xMike Bayer2020-09-261-7/+4
| | | | | | | pytest has removed support for pytest.Class().collect() and we need to use from_parent. Change-Id: Ia5fed9b22e76c99f71489283acee207f996f52a4
* Add support for identity columnsFederico Caselli2020-08-191-7/+2
| | | | | | | | | | | | | | Added the :class:`_schema.Identity` construct that can be used to configure identity columns rendered with GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY. Currently the supported backends are PostgreSQL >= 10, Oracle >= 12 and MSSQL (with different syntax and a subset of functionalities). Fixes: #5362 Fixes: #5324 Fixes: #5360 Change-Id: Iecea6f3ceb36821e8b96f0b61049b580507a1875
* Provision on different drivers dynamicallyMike Bayer2020-08-142-10/+31
| | | | | | | | | | | We want TOX_POSTGRESQL and similar to be the fixed variable that is configured from CI environment. These variables should refer to database servers but individual drivers like asyncpg mysqlconnector etc. should come from local tox.ini. add a new system to generate per-driver URLs from a simple list of hostname-based URLs delivered from CI environment. Change-Id: I4267b4a70742765388c7e7c4432c1da9d9adece2
* Implement rudimentary asyncio support w/ asyncpgMike Bayer2020-08-132-2/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | Using the approach introduced at https://gist.github.com/zzzeek/6287e28054d3baddc07fa21a7227904e We can now create asyncio endpoints that are then handled in "implicit IO" form within the majority of the Core internals. Then coroutines are re-exposed at the point at which we call into asyncpg methods. Patch includes: * asyncpg dialect * asyncio package * engine, result, ORM session classes * new test fixtures, tests * some work with pep-484 and a short plugin for the pyannotate package, which seems to have so-so results Change-Id: Idbcc0eff72c4cad572914acdd6f40ddb1aef1a7d Fixes: #3414
View Lorry Controller Status