summaryrefslogtreecommitdiff
path: root/test/dialect/test_sqlite.py
Commit message (Collapse)AuthorAgeFilesLines
* simplify AttachedDB test for SQLiteMike Bayer2023-03-131-68/+42
| | | | | | | | This test is failing on windows due to the new SQlite provisioning that seemed to be failing to delete schema files as they are still used by the main connection. Change-Id: I51093212ebfe1053f26b279c56c8fec0408806da
* Restore connectivity with ancient sqliteFederico Caselli2023-03-041-0/+37
| | | | | | | | Fixed bug that prevented SQLAlchemy to connect when using a very old sqlite version (before 3.9) on python 3.8+. Fixes: #9379 Change-Id: I10ca347398221c952e1a572dc6ef80e491d1f5cf
* check index_list pragma for number of columns returnedMike Bayer2022-12-131-0/+1
| | | | | | | | | | | Fixed regression caused by new support for reflection of partial indexes on SQLite added in 1.4.45 for :ticket:`8804`, where the ``index_list`` pragma command in very old versions of SQLite (possibly prior to 3.8.9) does not return the current expected number of columns, leading to exceptions raised when reflecting tables and indexes. Fixes: #8969 Change-Id: If317cdcfc6782f7e180df329b6ea0ddb48ce2269
* Rewrite positional handling, test for "numeric"Federico Caselli2022-12-051-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Changed how the positional compilation is performed. It's rendered by the compiler the same as the pyformat compilation. The string is then processed to replace the placeholders with the correct ones, and to obtain the correct order of the parameters. This vastly simplifies the computation of the order of the parameters, that in case of nested CTE is very hard to compute correctly. Reworked how numeric paramstyle behavers: - added support for repeated parameter, without duplicating them like in normal positional dialects - implement insertmany support. This requires that the dialect supports out of order placehoders, since all parameters that are not part of the VALUES clauses are placed at the beginning of the parameter tuple - support for different identifiers for a numeric parameter. It's for example possible to use postgresql style placeholder $1, $2, etc Added two new dialect based on sqlite to test "numeric" fully using both :1 style and $1 style. Includes a workaround for SQLite's not-really-correct numeric implementation. Changed parmstyle of asyncpg dialect to use numeric, rendering with its native $ identifiers Fixes: #8926 Fixes: #8849 Change-Id: I7c640467d49adfe6d795cc84296fc7403dcad4d6
* [sqlite] Reflect DEFERRABLE and INITIALLY options for foreign keysMichael Gorven2022-11-301-0/+59
| | | | | | | | | | | | | Added support for the SQLite backend to reflect the "DEFERRABLE" and "INITIALLY" keywords which may be present on a foreign key construct. Pull request courtesy Michael Gorven. Fixes: #8903 Closes: #8904 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8904 Pull-request-sha: 52aa4cf77482c4051899e21bea75b9830e4c3efa Change-Id: I713906db1a458d8f1be39625841ca3bbc03ec835
* add partial index predicate to SQLiteDialect.get_indexes() resultTobias Pfeiffer2022-11-281-0/+51
| | | | | | | | | | | | | Added support for reflection of expression-oriented WHERE criteria included in indexes on the SQLite dialect, in a manner similar to that of the PostgreSQL dialect. Pull request courtesy Tobias Pfeiffer. Fixes: #8804 Closes: #8806 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8806 Pull-request-sha: 539dfcb372360911b69aed2a804698bb1a2220b1 Change-Id: I0e34d47dbe2b9c1da6fce531363084843e5127a3
* Try running pyupgrade on the codeFederico Caselli2022-11-161-2/+0
| | | | | | | | command run is "pyupgrade --py37-plus --keep-runtime-typing --keep-percent-format <files...>" pyupgrade will change assert_ to assertTrue. That was reverted since assertTrue does not exists in sqlalchemy fixtures Change-Id: Ie1ed2675c7b11d893d78e028aad0d1576baebb55
* ensure RETURNING renders in stringify w/ no server versionMike Bayer2022-08-031-0/+16
| | | | | | | | | just in my own testing, if I say insert().return_defaults() and stringify, I should see it, so make sure all the dialects default to "insert_returning" etc. , with downgrade on server version check. Change-Id: Id64e78fcb03c48b5dcb0feb21cb9cc495edd15e9
* SQLite reflection ignores schema internal namesFederico Caselli2022-08-011-0/+72
| | | | | | | | | | | | | | Added new parameter to SQLite for reflection methods called ``sqlite_include_internal=True``; when omitted, local tables that start with the prefix ``sqlite_``, which per SQLite documentation are noted as "internal schema" tables such as the ``sqlite_sequence`` table generated to support "AUTOINCREMENT" columns, will not be included in reflection methods that return lists of local objects. This prevents issues for example when using Alembic autogenerate, which previously would consider these SQLite-generated tables as being remove from the model. Fixes: #8234 Change-Id: I36ee7a053e04b6c46c912aaa0d7e035a5b88a4f9
* Comments on (named) constraintscheremnov2022-06-291-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adds support for comments on named constraints, including `ForeignKeyConstraint`, `PrimaryKeyConstraint`, `CheckConstraint`, `UniqueConstraint`, solving the [Issue 5667](https://github.com/sqlalchemy/sqlalchemy/issues/5667). Supports only PostgreSQL backend. ### Description Following the example of [Issue 1546](https://github.com/sqlalchemy/sqlalchemy/issues/1546), supports comments on constraints. Specifically, enables comments on _named_ ones — as I get it, PostgreSQL prohibits comments on unnamed constraints. Enables setting the comments for named constraints like this: ``` Table( 'example', metadata, Column('id', Integer), Column('data', sa.String(30)), PrimaryKeyConstraint( "id", name="id_pk", comment="id_pk comment" ), CheckConstraint('id < 100', name="cc1", comment="Id value can't exceed 100"), UniqueConstraint(['data'], name="uc1", comment="Must have unique data field"), ) ``` Provides the DDL representation for constraint comments and routines to create and drop them. Class `.Inspector` reflects constraint comments via methods like `get_check_constraints` . ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - [ ] A short code fix - [x] A new feature implementation - Solves the issue 5667. - The commit message includes `Fixes: 5667`. - Includes tests based on comment reflection. **Have a nice day!** Fixes: #5667 Closes: #7742 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7742 Pull-request-sha: 42a5d3c3e9ccf9a9d5397fd007aeab0854f66130 Change-Id: Ia60f578595afdbd6089541c9a00e37997ef78ad3
* rearchitect reflection for batched performanceFederico Caselli2022-06-181-21/+10
| | | | | | | | | | | | | | | | | | | | | | | Rearchitected the schema reflection API to allow some dialects to make use of high performing batch queries to reflect the schemas of many tables at once using much fewer queries. The new performance features are targeted first at the PostgreSQL and Oracle backends, and may be applied to any dialect that makes use of SELECT queries against system catalog tables to reflect tables (currently this omits the MySQL and SQLite dialects which instead make use of parsing the "CREATE TABLE" statement, however these dialects do not have a pre-existing performance issue with reflection. MS SQL Server is still a TODO). The new API is backwards compatible with the previous system, and should require no changes to third party dialects to retain compatibility; third party dialects can also opt into the new system by implementing batched queries for schema reflection. Along with this change is an updated reflection API that is fully :pep:`484` typed, features many new methods and some changes. Fixes: #4379 Change-Id: I897ec09843543aa7012bcdce758792ed3d415d08
* render col name in on conflict set clause, not given keyMike Bayer2022-05-151-1/+46
| | | | | | | | | | | | | Fixed bug where the PostgreSQL :meth:`_postgresql.Insert.on_conflict` method and the SQLite :meth:`_sqlite.Insert.on_conflict` method would both fail to correctly accommodate a column with a separate ".key" when specifying the column using its key name in the dictionary passed to ``set_``, as well as if the :attr:`_sqlite.Insert.excluded` or :attr:`_postgresql.Insert.excluded` collection were used as the dictionary directly. Fixes: #8014 Change-Id: I67226aeedcb2c683e22405af64720cc1f990f274
* use .fromisoformat() for sqlite datetime, date, time parsingMike Bayer2022-04-031-3/+3
| | | | | | | | | | | | SQLite datetime, date, and time datatypes now use Python standard lib ``fromisoformat()`` methods in order to parse incoming datetime, date, and time string values. This improves performance vs. the previous regular expression-based approach, and also automatically accommodates for datetime and time formats that contain either a six-digit "microseconds" format or a three-digit "milliseconds" format. Fixes: #7029 Change-Id: I67aab4fe5ee3055e5996050cf4564981413cc221
* improve reflection of inline UNIQUE constraintsMike Bayer2022-02-201-0/+50
| | | | | | | | | | | | | | Fixed issue where SQLite unique constraint reflection would not work for an inline UNIQUE constraint where the column name had an underscore in its name. Added support for reflecting SQLite inline unique constraints where the column names are formatted with SQLite "escape quotes" ``[]`` or `` ` ``, which are discarded by the database when producing the column name. Fixes: #7736 Change-Id: I635003478dc27193995f7d7a6448f9333a498706
* use QueuePool for sqlite file databasesMike Bayer2022-02-101-6/+16
| | | | | | | | | | | | | The SQLite dialect now defaults to :class:`_pool.QueuePool` when a file based database is used. This is set along with setting the ``check_same_thread`` parameter to ``False``. It has been observed that the previous approach of defaulting to :class:`_pool.NullPool`, which does not hold onto database connections after they are released, did in fact have a measurable negative performance impact. As always, the pool class is always customizable via the :paramref:`_sa.create_engine.poolclass` parameter. Fixes: #7490 Change-Id: I5f6c259def0ef43d401c6163dc99f651e519148d
* Merge "Fix various source comment/doc typos" into mainmike bayer2022-01-071-1/+1
|\
| * Fix various source comment/doc typosluz paz2021-12-291-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ### Description Found via `codespell -q 3 -L ba,crate,datas,froms,gord,hist,inh,nd,selectin,strat,ue` Also added codespell to the pep8 tox env ### Checklist This pull request is: - [x] A documentation / typographical error fix - Good to go, no issue or tests are needed Closes: #7338 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7338 Pull-request-sha: 0deac2219396bc0eba7da53eb3a80932edbf2dd7 Change-Id: Icd61db31c8dc655d4a39d8a304194804d08555fe
* | Update Black's target-version to py37Hugo van Kemenade2022-01-051-2/+2
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | <!-- Provide a general summary of your proposed changes in the Title field above --> ### Description <!-- Describe your changes in detail --> Black's `target-version` was still set to `['py27', 'py36']`. Set it to `[py37]` instead. Also update Black and other pre-commit hooks and re-format with Black. ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - Good to go, no issue or tests are needed - [ ] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. - [ ] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. **Have a nice day!** Closes: #7536 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7536 Pull-request-sha: b3aedf5570d7e0ba6c354e5989835260d0591b08 Change-Id: I8be85636fd2c9449b07a8626050c8bd35bd119d5
* Clean up most py3k compatFederico Caselli2021-11-241-11/+6
| | | | Change-Id: I8172fdcc3103ff92aa049827728484c8779af6b7
* Remove object in class definitionFederico Caselli2021-11-221-1/+1
| | | | | References: #4600 Change-Id: I2a62ddfe00bc562720f0eae700a497495d7a987a
* fully support isolation_level parameter in base dialectMike Bayer2021-11-181-16/+0
| | | | | | | | | | | | | | | | | | | | Generalized the :paramref:`_sa.create_engine.isolation_level` parameter to the base dialect so that it is no longer dependent on individual dialects to be present. This parameter sets up the "isolation level" setting to occur for all new database connections as soon as they are created by the connection pool, where the value then stays set without being reset on every checkin. The :paramref:`_sa.create_engine.isolation_level` parameter is essentially equivalent in functionality to using the :paramref:`_engine.Engine.execution_options.isolation_level` parameter via :meth:`_engine.Engine.execution_options` for an engine-wide setting. The difference is in that the former setting assigns the isolation level just once when a connection is created, the latter sets and resets the given level on each connection checkout. Fixes: #6342 Change-Id: Id81d6b1c1a94371d901ada728a610696e09e9741
* change the POSTCOMPILE/ SCHEMA symbols to not conflict w mssql quotingMike Bayer2021-11-091-1/+1
| | | | | | | | | | | | | | | | | | Adjusted the compiler's generation of "post compile" symbols including those used for "expanding IN" as well as for the "schema translate map" to not be based directly on plain bracketed strings with underscores, as this conflicts directly with SQL Server's quoting format of also using brackets, which produces false matches when the compiler replaces "post compile" and "schema translate" symbols. The issue created easy to reproduce examples both with the :meth:`.Inspector.get_schema_names` method when used in conjunction with the :paramref:`_engine.Connection.execution_options.schema_translate_map` feature, as well in the unlikely case that a symbol overlapping with the internal name "POSTCOMPILE" would be used with a feature like "expanding in". Fixes: #7300 Change-Id: I6255c850b140522a4aba95085216d0bca18ce230
* fully implement future engine and remove legacyMike Bayer2021-11-071-4/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | The major action here is to lift and move future.Connection and future.Engine fully into sqlalchemy.engine.base. This removes lots of engine concepts, including: * autocommit * Connection running without a transaction, autobegin is now present in all cases * most "autorollback" is obsolete * Core-level subtransactions (i.e. MarkerTransaction) * "branched" connections, copies of connections * execution_options() returns self, not a new connection * old argument formats, distill_params(), simplifies calling scheme between engine methods * before/after_execute() events (oriented towards compiled constructs) don't emit for exec_driver_sql(). before/after_cursor_execute() is still included for this * old helper methods superseded by context managers, connection.transaction(), engine.transaction() engine.run_callable() * ancient engine-level reflection methods has_table(), table_names() * sqlalchemy.testing.engines.proxying_engine References: #7257 Change-Id: Ib20ed816642d873b84221378a9ec34480e01e82c
* First round of removal of python 2Federico Caselli2021-11-011-4/+0
| | | | | References: #4600 Change-Id: I61e35bc93fe95610ae75b31c18a3282558cd4ffe
* ensure pysqlite dialect enumerates correct isolation levelsMike Bayer2021-08-301-0/+17
| | | | | | | | | Fixed bug where the error message for SQLite invalid isolation level on the pysqlite driver would fail to indicate that "AUTOCOMMIT" is one of the valid isolation levels. Change-Id: Icbceab9a28af6a560859761fa92320b5473269a9 References: #6959
* Replace all http:// links to https://Federico Caselli2021-07-041-1/+1
| | | | | | Also replace http://pypi.python.org/pypi with https://pypi.org/project Change-Id: I84b5005c39969a82140706472989f2a30b0c7685
* Pass URL object, not the string, to on_connect_urlMike Bayer2021-06-061-0/+9
| | | | | | | | | | | | The fix for pysqlcipher released in version 1.4.3 :ticket:`5848` was unfortunately non-working, in that the new ``on_connect_url`` hook was erroneously not receiving a ``URL`` object under normal usage of :func:`_sa.create_engine` and instead received a string that was unhandled; the test suite failed to fully set up the actual conditions under which this hook is called. This has been fixed. Fixes: #6586 Change-Id: I3bf738daec35877a10fdad740f08dca9e7420829
* don't cache TypeDecorator by defaultMike Bayer2021-05-061-0/+2
| | | | | | | | | | The :class:`.TypeDecorator` class will now emit a warning when used in SQL compilation with caching unless the ``.cache_ok`` flag is set to ``True`` or ``False``. ``.cache_ok`` indicates that all the parameters passed to the object are safe to be used as a cache key, ``False`` means they are not. Fixes: #6436 Change-Id: Ib1bb7dc4b124e38521d615c2e2e691e4915594fb
* Use SingletonThreadPool for in-memory SQLiteYongJieYongJie2021-04-281-0/+5
| | | | | | | | | | | | | | | | database created using filename uri Default to using ``SingletonThreadPool`` for in-memory SQLite databases created using URI filenames. Previously the default pool used was the ``NullPool`` that precented sharing the same database between multiple engines. Fixes: #6379 Closes: #6380 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/6380 Pull-request-sha: 3b8024417a3a54c8269d43e40801aa4e58593540 Change-Id: Ice09622796455e796ede7711c98f3ceec13aa949
* Repair pysqlcipher and use sqlcipher3Mike Bayer2021-03-241-1/+15
| | | | | | | | | | | | | | | | | | | | | | | The ``pysqlcipher`` dialect now imports the ``sqlcipher3`` module for Python 3 by default. Regressions have been repaired such that the connection routine was not working. To better support the post-connection steps of the pysqlcipher dialect, a new hook Dialect.on_connect_url() is added, which supersedes Dialect.on_connect() and is passed the URL object. The dialect now pulls the passphrase and other cipher args from the URL directly without including them in the "connect" args. This will allow any user-defined extensibility to connecting to work as it would for other dialects. The commit also builds upon the extended routines in sqlite/provisioning.py to better support running tests against multiple simultaneous SQLite database files. Additionally enables backend for test_sqlite which was skipping everything for aiosqlite too, fortunately everything there is passing. Fixes: #5848 Change-Id: I43f53ebc62298a84a4abe149e1eb699a027b7915
* Add support for aiosqliteFederico Caselli2021-03-241-1/+1
| | | | | | | | Added support for the aiosqlite database driver for use with the SQLAlchemy asyncio extension. Fixes: #5920 Change-Id: Id11a320516a44e886a6f518d2866a0f992413e55
* Accept ColumnCollection in update_on_conflict(set_=Gord Thompson2021-02-151-2/+9
| | | | | Fixes: #5939 Change-Id: I21d7125765028e2a98d5ef4c32d8e7e457aa2d12
* Repair tests for older SQLite, timing, pypyMike Bayer2021-02-031-5/+3
| | | | Change-Id: I9e425d2415d18a893342244755c7748b546fb20d
* introduce generalized decorator to prevent invalid method callsMike Bayer2021-01-161-0/+21
| | | | | | | | | | | | | | | | | | | This introduces the ``_exclusive_against()`` utility decorator that can be used to prevent repeated invocations of methods that typically should only be called once. An informative error message is now raised for a selected set of DML methods (currently all part of :class:`_dml.Insert` constructs) if they are called a second time, which would implicitly cancel out the previous setting. The methods altered include: :class:`_sqlite.Insert.on_conflict_do_update`, :class:`_sqlite.Insert.on_conflict_do_nothing` (SQLite), :class:`_postgresql.Insert.on_conflict_do_update`, :class:`_postgresql.Insert.on_conflict_do_nothing` (PostgreSQL), :class:`_mysql.Insert.on_duplicate_key_update` (MySQL) Fixes: #5169 Change-Id: I9278fa87cd3470dcf296ff96bb0fb17a3236d49d
* Guard against re-entrant autobegin in Core, ORMMike Bayer2021-01-151-0/+4
| | | | | | | | | | | | | | | | | | | | Fixed bug in "future" version of :class:`.Engine` where emitting SQL during the :meth:`.EngineEvents.do_begin` event hook would cause a re-entrant condition due to autobegin, including the recipe documented for SQLite to allow for savepoints and serializable isolation support. Fixed issue in new :class:`_orm.Session` similar to that of the :class:`_engine.Connection` where the new "autobegin" logic could be tripped into a re-entrant state if SQL were executed within the :meth:`.SessionEvents.after_transaction_create` event hook. Also repair the new "testing_engine" pytest fixture to set up for "future" engine appropriately, which wasn't working leading to the test_execute.py tests not using the future engine since recent f1e96cb0874927a475d0c11139. Fixes: #5845 Change-Id: Ib2432d8c8bd753e24be60720ec47affb2df15a4a
* update execute() arg formats in modules and testsMike Bayer2021-01-151-9/+9
| | | | | | | | | | | | | continuing with producing a SQLAlchemy 1.4.0b2 that internally does not emit any of its own 2.0 deprecation warnings, migrate the *args and **kwargs passed to execute() methods that now must be a single list or dictionary. Alembic 1.5 is again waiting on this internal consistency to be present so that it can pass all tests with no 2.0 deprecation warnings. Change-Id: If6b792e57c8c5dff205419644ab68e631575a2fa
* reinvent xdist hooks in terms of pytest fixturesMike Bayer2021-01-131-12/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To allow the "connection" pytest fixture and others work correctly in conjunction with setup/teardown that expects to be external to the transaction, remove and prevent any usage of "xdist" style names that are hardcoded by pytest to run inside of fixtures, even function level ones. Instead use pytest autouse fixtures to implement our own r"setup|teardown_test(?:_class)?" methods so that we can ensure function-scoped fixtures are run within them. A new more explicit flow is set up within plugin_base and pytestplugin such that the order of setup/teardown steps, which there are now many, is fully documented and controllable. New granularity has been added to the test teardown phase to distinguish between "end of the test" when lock-holding structures on connections should be released to allow for table drops, vs. "end of the test plus its teardown steps" when we can perform final cleanup on connections and run assertions that everything is closed out. From there we can remove most of the defensive "tear down everything" logic inside of engines which for many years would frequently dispose of pools over and over again, creating for a broken and expensive connection flow. A quick test shows that running test/sql/ against a single Postgresql engine with the new approach uses 75% fewer new connections, creating 42 new connections total, vs. 164 new connections total with the previous system. As part of this, the new fixtures metadata/connection/future_connection have been integrated such that they can be combined together effectively. The fixture_session(), provide_metadata() fixtures have been improved, including that fixture_session() now strongly references sessions which are explicitly torn down before table drops occur afer a test. Major changes have been made to the ConnectionKiller such that it now features different "scopes" for testing engines and will limit its cleanup to those testing engines corresponding to end of test, end of test class, or end of test session. The system by which it tracks DBAPI connections has been reworked, is ultimately somewhat similar to how it worked before but is organized more clearly along with the proxy-tracking logic. A "testing_engine" fixture is also added that works as a pytest fixture rather than a standalone function. The connection cleanup logic should now be very robust, as we now can use the same global connection pools for the whole suite without ever disposing them, while also running a query for PostgreSQL locks remaining after every test and assert there are no open transactions leaking between tests at all. Additional steps are added that also accommodate for asyncio connections not explicitly closed, as is the case for legacy sync-style tests as well as the async tests themselves. As always, hundreds of tests are further refined to use the new fixtures where problems with loose connections were identified, largely as a result of the new PostgreSQL assertions, many more tests have moved from legacy patterns into the newest. An unfortunate discovery during the creation of this system is that autouse fixtures (as well as if they are set up by @pytest.mark.usefixtures) are not usable at our current scale with pytest 4.6.11 running under Python 2. It's unclear if this is due to the older version of pytest or how it implements itself for Python 2, as well as if the issue is CPU slowness or just large memory use, but collecting the full span of tests takes over a minute for a single process when any autouse fixtures are in place and on CI the jobs just time out after ten minutes. So at the moment this patch also reinvents a small version of "autouse" fixtures when py2k is running, which skips generating the real fixture and instead uses two global pytest fixtures (which don't seem to impact performance) to invoke the "autouse" fixtures ourselves outside of pytest. This will limit our ability to do more with fixtures until we can remove py2k support. py.test is still observed to be much slower in collection in the 4.6.11 version compared to modern 6.2 versions, so add support for new TOX_POSTGRESQL_PY2K and TOX_MYSQL_PY2K environment variables that will run the suite for fewer backends under Python 2. For Python 3 pin pytest to modern 6.2 versions where performance for collection has been improved greatly. Includes the following improvements: Fixed bug in asyncio connection pool where ``asyncio.TimeoutError`` would be raised rather than :class:`.exc.TimeoutError`. Also repaired the :paramref:`_sa.create_engine.pool_timeout` parameter set to zero when using the async engine, which previously would ignore the timeout and block rather than timing out immediately as is the behavior with regular :class:`.QueuePool`. For asyncio the connection pool will now also not interact at all with an asyncio connection whose ConnectionFairy is being garbage collected; a warning that the connection was not properly closed is emitted and the connection is discarded. Within the test suite the ConnectionKiller is now maintaining strong references to all DBAPI connections and ensuring they are released when tests end, including those whose ConnectionFairy proxies are GCed. Identified cx_Oracle.stmtcachesize as a major factor in Oracle test scalability issues, this can be reset on a per-test basis rather than setting it to zero across the board. the addition of this flag has resolved the long-standing oracle "two task" error problem. For SQL Server, changed the temp table style used by the "suite" tests to be the double-pound-sign, i.e. global, variety, which is much easier to test generically. There are already reflection tests that are more finely tuned to both styles of temp table within the mssql test suite. Additionally, added an extra step to the "dropfirst" mechanism for SQL Server that will remove all foreign key constraints first as some issues were observed when using this flag when multiple schemas had not been torn down. Identified and fixed two subtle failure modes in the engine, when commit/rollback fails in a begin() context manager, the connection is explicitly closed, and when "initialize()" fails on the first new connection of a dialect, the transactional state on that connection is still rolled back. Fixes: #5826 Fixes: #5827 Change-Id: Ib1d05cb8c7cf84f9a4bfd23df397dc23c9329bfe
* remove metadata.bind use from test suiteMike Bayer2021-01-031-79/+40
| | | | | | | | | | | | | | importantly this means we can remove bound metadata from the fixtures that are used by Alembic's test suite. hopefully this is the last one that has to happen to allow Alembic to be fully 1.4/2.0. Start moving from @testing.provide_metadata to a pytest metadata fixture. This does not seem to have any negative effects even though TablesTest uses a "self.metadata" attribute. Change-Id: Iae6ab95938a7e92b6d42086aec534af27b5577d3
* Repair mssql dep tests; have __only_on__ imply __backend__Mike Bayer2020-12-191-102/+103
| | | | | | | | | | | | | | | | | | CI missed a few SQL Server tests because we run mssql-backendonly in the gerrit job. As there was a test that was "only on" mssql but didn't have backendonly, it never got run and then fails in master where we run mssql fully. Any suite that has an __only_on__ is inherently specific to a backend, so if present this should imply __backend__ so that it definitely runs when we have that backend present. This in turn meant we had to fix a few sqlite_file tests that weren't cleaning up or sharing well as they suddenly became backend tests under sqlite_file. Added a sqlite_file cleanup to test class cleanup for now. Change-Id: I9de1ceabd6596547a65c59059a55b7e5156103fd
* Emit 2.0 deprecation warning for sub-transactionsMike Bayer2020-12-141-22/+0
| | | | | | | | The nesting pattern will be removed in 2.0, so the use of the MarkerTransaction should emit a 2.0 deprecation warning unconditionally. Change-Id: I96aed22c4e5db9b59e9b28a7f2d1283cd99a9cb6
* correct for "autocommit" deprecation warningMike Bayer2020-12-111-60/+76
| | | | | | | | | | | | Ensure no autocommit warnings occur internally or within tests. Also includes fixes for SQL Server full text tests which apparently have not been working at all for a long time, as it used long removed APIs. CI has not had fulltext running for some years and is now installed. Change-Id: Id806e1856c9da9f0a9eac88cebc7a94ecc95eb96
* Support Column objects in the SET clause for upsertMike Bayer2020-11-231-0/+47
| | | | | | | | | | | | | | Established support for :class:`_schema.Column` objects as well as ORM instrumented attributes as keys in the ``set_`` dictionary passed to the :meth:`_postgresql.Insert.on_conflict_do_update` and :meth:`_sqlite.Insert.on_conflict_do_update` methods, which match to the :class:`_schema.Column` objects in the ``.c`` collection of the target :class:`_schema.Table`. Previously, only string column names were expected; a column expression would be assumed to be an out-of-table expression that would render fully along with a warning. Fixes: #5722 Change-Id: Ice73b501d721c28d978a0277a83cedc6aff756a9
* Merge "Support SQLite WITHOUT ROWID tables"mike bayer2020-11-091-0/+10
|\
| * Support SQLite WITHOUT ROWID tablesSean Anderson2020-11-081-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds support for creating tables WITHOUT ROWID in the SQLite dialect. WITHOUT ROWID tables were introduced in SQLite version 3.8.2 (2013-12-06). They do not use an implicit rowid column as the primary key. This may result in space and performance savings for tables without INTEGER primary keys and tables with composite primary keys. For more information about this feature, see the sqlite documentation [1]. [1] https://www.sqlite.org/withoutrowid.html Fixes: #5685 ### Checklist This pull request is: - [x] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. Closes: #5686 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5686 Pull-request-sha: 2b44782d1b3d858e31ce1ff8e08e197af37344d8 Change-Id: Ifcf727b0c07c90e267b79828a8e3fd7a8260a074
* | Implement upsert for SQLiteRamonWill2020-11-081-0/+538
|/ | | | | | | | | | | | Implemented INSERT... ON CONFLICT clause for SQLite. Pull request courtesy Ramon Williams. Fixes: #4010 Closes: #5580 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5580 Pull-request-sha: fb422e0749fac442a455cbce539ef662d9512bc0 Change-Id: Ibeea44f4c2cee8dab5dc22b7ec3ae1ab95c12b65
* Convert to autoload_with internallyMike Bayer2020-11-071-82/+76
| | | | | | | | | Fixed bug where the now-deprecated ``autoload`` parameter was being called internally within the reflection routines when a related table were reflected. Fixes: #5684 Change-Id: I6ab439a2f49ff1ae2d3c7a15b531cbafbc3cf594
* Apply underscore naming to several more operatorsjonathan vanasco2020-10-301-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | The operator changes are: * `isfalse` is now `is_false` * `isnot_distinct_from` is now `is_not_distinct_from` * `istrue` is now `is_true` * `notbetween` is now `not_between` * `notcontains` is now `not_contains` * `notendswith` is now `not_endswith` * `notilike` is now `not_ilike` * `notlike` is now `not_like` * `notmatch` is now `not_match` * `notstartswith` is now `not_startswith` * `nullsfirst` is now `nulls_first` * `nullslast` is now `nulls_last` Because these are core operators, the internal migration strategy for this change is to support legacy terms for an extended period of time -- if not indefinitely -- but update all documentation, tutorials, and internal usage to the new terms. The new terms are used to define the functions, and the legacy terms have been deprecated into aliases of the new terms. Fixes: #5435 Change-Id: Ifbd7cb1cdda5981990243c4fc4b4ff467dc132ac
* upgrade to black 20.8b1Mike Bayer2020-09-281-7/+12
| | | | | | | It's better, the majority of these changes look more readable to me. also found some docstrings that had formatting / quoting issues. Change-Id: I582a45fde3a5648b2f36bab96bad56881321899b
* Update select usage to use the new 1.4 formatFederico Caselli2020-09-081-3/+1
| | | | | | | | | | | | | | | | This change includes mainly that the bracketed use within select() is moved to positional, and keyword arguments are removed from calls to the select() function. it does not yet fully address other issues such as keyword arguments passed to the table.select(). Additionally, allows False / None to both be considered as "disable" for all of select.correlate(), select.correlate_except(), query.correlate(), which establishes consistency with passing of ``False`` for the legact select(correlate=False) argument. Change-Id: Ie6c6e6abfbd3d75d4c8de504c0cf0159e6999108
* Add support for regular expression on supported backend.Federico Caselli2020-08-271-0/+75
| | | | | | | | | | | | Two operations have been defined: * :meth:`~.ColumnOperators.regexp_match` implementing a regular expression match like function. * :meth:`~.ColumnOperators.regexp_replace` implementing a regular expression string replace function. Fixes: #1390 Change-Id: I44556846e4668ccf329023613bd26861d5c674e6
View Lorry Controller Status