summaryrefslogtreecommitdiff
path: root/test/dialect/mssql/test_engine.py
Commit message (Collapse)AuthorAgeFilesLines
* add deterministic imv returning ordering using sentinel columnsMike Bayer2023-04-211-9/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Repaired a major shortcoming which was identified in the :ref:`engine_insertmanyvalues` performance optimization feature first introduced in the 2.0 series. This was a continuation of the change in 2.0.9 which disabled the SQL Server version of the feature due to a reliance in the ORM on apparent row ordering that is not guaranteed to take place. The fix applies new logic to all "insertmanyvalues" operations, which takes effect when a new parameter :paramref:`_dml.Insert.returning.sort_by_parameter_order` on the :meth:`_dml.Insert.returning` or :meth:`_dml.UpdateBase.return_defaults` methods, that through a combination of alternate SQL forms, direct correspondence of client side parameters, and in some cases downgrading to running row-at-a-time, will apply sorting to each batch of returned rows using correspondence to primary key or other unique values in each row which can be correlated to the input data. Performance impact is expected to be minimal as nearly all common primary key scenarios are suitable for parameter-ordered batching to be achieved for all backends other than SQLite, while "row-at-a-time" mode operates with a bare minimum of Python overhead compared to the very heavyweight approaches used in the 1.x series. For SQLite, there is no difference in performance when "row-at-a-time" mode is used. It's anticipated that with an efficient "row-at-a-time" INSERT with RETURNING batching capability, the "insertmanyvalues" feature can be later be more easily generalized to third party backends that include RETURNING support but not necessarily easy ways to guarantee a correspondence with parameter order. Fixes: #9618 References: #9603 Change-Id: I1d79353f5f19638f752936ba1c35e4dc235a8b7c
* turn off use_insertmanyvalues for SQL ServerMike Bayer2023-04-051-1/+10
| | | | | | | | | | | | | | | | | | we will keep trying to find workarounds, however this patch is the "turn it off" patch Due to a critical bug identified in SQL Server, the SQLAlchemy "insertmanyvalues" feature which allows fast INSERT of many rows while also supporting RETURNING unfortunately needs to be disabled for SQL Server. SQL Server is apparently unable to guarantee that the order of rows inserted matches the order in which they are sent back by OUTPUT inserted when table-valued rows are used with INSERT in conjunction with OUTPUT inserted. We are trying to see if Microsoft is able to confirm this undocumented behavior however there is no known workaround, other than it's not safe to use table-valued expressions with OUTPUT inserted for now. Fixes: #9603 Change-Id: I4b932fb8774390bbdf4e870a1f6cfe9a78c4b105
* favor fast_executemany over insertmanyvalues when setMike Bayer2023-04-021-1/+4
| | | | | | | | | | | | | | | | | | | | | | | Changed the bulk INSERT strategy used for SQL Server "executemany" with pyodbc when ``fast_executemany`` is set to ``True`` by using ``fast_executemany`` / ``cursor.executemany()`` for bulk INSERT that does not include RETURNING, restoring the same behavior as was used in SQLAlchemy 1.4 when this parameter is set. For INSERT statements that use RETURNING, the "insertmanyvalues" strategy continues to be used as it is the only current strategy that supports RETURNING with bulk INSERT. Previously, SQLAlchemy 2.0 would use "insertmanyvalues" for all INSERT statements when ``use_insertmanyvalues`` was left at its default of ``False``, ignoring if ``fast_executemany`` was set. New performance details from end users have shown that ``fast_executemany`` is still much faster for very large datasets as it uses ODBC commands that can receive all rows in a single round trip, allowing for much larger datasizes than the batches that can be sent by the current "insertmanyvalues" strategy. Fixes: #9586 Change-Id: I85955a10ba77c26cdc0c22e362a827d7aaef2852
* Make comment support conditional on fn_listextendedproperty availabilityMike Bayer2023-01-251-0/+71
| | | | | | | | | | | | | | | The newly added comment reflection and rendering capability of the MSSQL dialect, added in :ticket:`7844`, will now be disabled by default if it cannot be determined that an unsupported backend such as Azure Synapse may be in use; this backend does not support table and column comments and does not support the SQL Server routines in use to generate them as well as to reflect them. A new parameter ``supports_comments`` is added to the dialect which defaults to ``None``, indicating that comment support should be auto-detected. When set to ``True`` or ``False``, the comment support is either enabled or disabled unconditionally. Fixes: #9142 Change-Id: Ib5cac31806185e7353e15b3d83b580652d304b3b
* disable setinputsizes only for true DBAPI executemanyMike Bayer2022-12-011-61/+68
| | | | | | | | | | | | | | | | | | | | Fixed regression caused by the combination of :ticket:`8177`, re-enable setinputsizes for SQL server unless fast_executemany + DBAPI executemany is used for a statement, along with :ticket:`6047`, implement "insertmanyvalues", which bypasses DBAPI executemany in place of a custom DBAPI execute for INSERT statements. setinputsizes would incorrectly not be used for a multiple parameter-set INSERT statement that used "insertmanyvalues" if fast_executemany were turned on, as the check would incorrectly assume this is a DBAPI executemany call. The "regression" would then be that the "insertmanyvalues" statement format is apparently slightly more sensitive to multiple rows that don't use the same types for each row, so in such a case setinputsizes is especially needed. The fix repairs the fast_executemany check so that it only disables setinputsizes if true DBAPI executemany is to be used. Fixes: #8917 Change-Id: I78895606a99848d4f92ecf38ded92dc5d6d48c6f
* Try running pyupgrade on the codeFederico Caselli2022-11-161-2/+0
| | | | | | | | command run is "pyupgrade --py37-plus --keep-runtime-typing --keep-percent-format <files...>" pyupgrade will change assert_ to assertTrue. That was reverted since assertTrue does not exists in sqlalchemy fixtures Change-Id: Ie1ed2675c7b11d893d78e028aad0d1576baebb55
* implement batched INSERT..VALUES () () for executemanyMike Bayer2022-09-241-1/+3
| | | | | | | | | | | | | | | | | | | | the feature is enabled for all built in backends when RETURNING is used, except for Oracle that doesn't need it, and on psycopg2 and mssql+pyodbc it is used for all INSERT statements, not just those that use RETURNING. third party dialects would need to opt in to the new feature by setting use_insertmanyvalues to True. Also adds dialect-level guards against using returning with executemany where we dont have an implementation to suit it. execute single w/ returning still defers to the server without us checking. Fixes: #6047 Fixes: #7907 Change-Id: I3936d3c00003f02e322f2e43fb949d0e6e568304
* catch exception for system_views alsoMike Bayer2022-09-151-3/+30
| | | | | | | | | | | | Fixed yet another regression in SQL Server isolation level fetch (see :ticket:`8231`, :ticket:`8475`), this time with "Microsoft Dynamics CRM Database via Azure Active Directory", which apparently lacks the ``system_views`` view entirely. Error catching has been extended that under no circumstances will this method ever fail, provided database connectivity is present. Fixes: #8525 Change-Id: I76a429e3329926069a0367d2e77ca1124b9a059d
* Fix Azure Synapse connection errorGord Thompson2022-09-021-1/+19
| | | | | | | | | | Fixed regression caused by the fix for :ticket:`8231` released in 1.4.40 where connection would fail if the user does not have permission to query the dm_exec_sessions or dm_pdw_nodes_exec_sessions system view when trying to determine the current transaction isolation level. Fixes: #8475 Change-Id: Ie2bcda92f2ef2d12360ddda47eb6e896313c71f2
* Fix 'No transaction found' error on Synapse.Gord Thompson2022-08-021-11/+63
| | | | | | | | | | | | | | | | | | | | Fixed issue where the SQL Server dialect's query for the current isolation level would fail on Azure Synapse Analytics, due to the way in which this database handles transaction rollbacks after an error has occurred. The initial query has been modified to no longer rely upon catching an error when attempting to detect the appropriate system view. Additionally, to better support this database's very specific "rollback" behavior, implemented new parameter ``ignore_no_transaction_on_rollback`` indicating that a rollback should ignore Azure Synapse error 'No corresponding transaction found. (111214)', which is raised if no transaction is present in conflict with the Python DBAPI. Fixes: #8231 Closes: #8233 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8233 Pull-request-sha: c48bd44a9f53d00e5e94f1b8bf996711b6419562 Change-Id: I6407a03148f45cc9eba8fe1d31d4f59ebf9c7ef7
* Change setinputsizes behavior for mssql+pyodbcGord Thompson2022-06-291-5/+4
| | | | | | | | | | | | | | | | | | The ``use_setinputsizes`` parameter for the ``mssql+pyodbc`` dialect now defaults to ``True``; this is so that non-unicode string comparisons are bound by pyodbc to pyodbc.SQL_VARCHAR rather than pyodbc.SQL_WVARCHAR, allowing indexes against VARCHAR columns to take effect. In order for the ``fast_executemany=True`` parameter to continue functioning, the ``use_setinputsizes`` mode now skips the ``cursor.setinputsizes()`` call specifically when ``fast_executemany`` is True and the specific method in use is ``cursor.executemany()``, which doesn't support setinputsizes. The change also adds appropriate pyodbc DBAPI typing to values that are typed as :class:`_types.Unicode` or :class:`_types.UnicodeText`, as well as altered the base :class:`_types.JSON` datatype to consider JSON string values as :class:`_types.Unicode` rather than :class:`_types.String`. Fixes: #8177 Change-Id: I6c8886663254ae55cf904ad256c906e8f5e11f48
* mssql login failure if password starts with "{"Gord Thompson2022-05-291-15/+35
| | | | | | | | Fix issue where a password with a leading "{" would result in login failure. Fixes: #8062 Change-Id: If91c2c211937b5eac89b8d525c22a19b0a94c5c4
* pep-484 for engineMike Bayer2022-03-011-44/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | All modules in sqlalchemy.engine are strictly typed with the exception of cursor, default, and reflection. cursor and default pass with non-strict typing, reflection is waiting on the multi-reflection refactor. Behavioral changes: * create_connect_args() methods return a tuple of list, dict, rather than a list of list, dict * removed allow_chars parameter from pyodbc connector ._get_server_version_info() method * the parameter list passed to do_executemany is now a list in all cases. previously, this was being run through dialect.execute_sequence_format, which defaults to tuple and was only intended for individual tuple params. * broke up dialect.dbapi into dialect.import_dbapi class method and dialect.dbapi module object. added a deprecation path for legacy dialects. it's not really feasible to type a single attr as a classmethod vs. module type. The "type_compiler" attribute also has this problem with greater ability to work around, left that one for now. * lots of constants changing to be Enum, so that we can type them. for fixed tuple-position constants in cursor.py / compiler.py (which are used to avoid the speed overhead of namedtuple), using Literal[value] which seems to work well * some tightening up in Row regarding __getitem__, which we can do since we are on full 2.0 style result use * altered the set_connection_execution_options and set_engine_execution_options event flows so that the dictionary of options may be mutated within the event hook, where it will then take effect as the actual options used. Previously, changing the dict would be silently ignored which seems counter-intuitive and not very useful. * A lot of DefaultDialect/DefaultExecutionContext methods and attributes, including underscored ones, move to interfaces. This is not fully ideal as it means the Dialect/ExecutionContext interfaces aren't publicly subclassable directly, but their current purpose is more of documentation for dialect authors who should (and certainly are) still be subclassing the DefaultXYZ versions in all cases Overall, Result was the most extremely difficult class hierarchy to type here as this hierarchy passes through largely amorphous "row" datatypes throughout, which can in fact by all kinds of different things, like raw DBAPI rows, or Row objects, or "scalar"/Any, but at the same time these types have meaning so I tried still maintaining some level of semantic markings for these, it highlights how complex Result is now, as it's trying to be extremely efficient and inlined while also being very open-ended and extensible. Change-Id: I98b75c0c09eab5355fc7a33ba41dd9874274f12a
* Clean up most py3k compatFederico Caselli2021-11-241-1/+1
| | | | Change-Id: I8172fdcc3103ff92aa049827728484c8779af6b7
* De-emphasize notion of "default driver" (DBAPI)Gord Thompson2021-11-091-12/+13
| | | | | | | | | | | Fixes: #6960 Even though a default driver still exists for each dialect, remove most usages of `dialect://` to encourage users to explicitly specify `dialect+driver://` Change-Id: I0ad42167582df509138fca64996bbb53e379b1af
* Turn off pyodbc setinputsizes() by defaultMike Bayer2021-03-161-0/+148
| | | | | | | | | | | | | | Fixed regression where a new setinputsizes() API that's available for pyodbc was enabled, which is apparently incompatible with pyodbc's fast_executemany() mode in the absence of more accurate typing information, which as of yet is not fully implemented or tested. The pyodbc dialect and connector has been modified so that setinputsizes() is not used at all unless the parameter ``use_setinputsizes`` is passed to the dialect, e.g. via :func:`_sa.create_engine`, at which point its behavior can be customized using the :meth:`.DialectEvents.do_setinputsizes` hook. Fixes: #6058 Change-Id: I99c2be3a5cd76fc3e490d10865292ed85ffc23ae
* Fix test that incorrectly used ; as query separatorFederico Caselli2021-02-181-1/+1
| | | | Change-Id: I31e9973930d90184bbabda0bff6346eca4e00c37
* remove metadata.bind use from test suiteMike Bayer2021-01-031-8/+6
| | | | | | | | | | | | | | importantly this means we can remove bound metadata from the fixtures that are used by Alembic's test suite. hopefully this is the last one that has to happen to allow Alembic to be fully 1.4/2.0. Start moving from @testing.provide_metadata to a pytest metadata fixture. This does not seem to have any negative effects even though TablesTest uses a "self.metadata" attribute. Change-Id: Iae6ab95938a7e92b6d42086aec534af27b5577d3
* correct for "autocommit" deprecation warningMike Bayer2020-12-111-1/+1
| | | | | | | | | | | | Ensure no autocommit warnings occur internally or within tests. Also includes fixes for SQL Server full text tests which apparently have not been working at all for a long time, as it used long removed APIs. CI has not had fulltext running for some years and is now installed. Change-Id: Id806e1856c9da9f0a9eac88cebc7a94ecc95eb96
* upgrade to black 20.8b1Mike Bayer2020-09-281-2/+5
| | | | | | | It's better, the majority of these changes look more readable to me. also found some docstrings that had formatting / quoting issues. Change-Id: I582a45fde3a5648b2f36bab96bad56881321899b
* Add support for Azure authentication optionsGord Thompson2020-09-171-0/+20
| | | | | Fixes: #5592 Change-Id: I0688e5ea0fc6b01a0b72f397daea8f57a2ec0766
* Add deprecation warning for mssql legacy_schema_aliasingGord Thompson2020-08-241-13/+0
| | | | | Fixes: #4809 Change-Id: I9ce2a5dfb79d86624c187ee28b5911fd14328ce2
* Fix connection string escaping for mssql+pyodbcGord Thompson2020-06-041-3/+3
| | | | | Fixes: #5373 Change-Id: Ia41e8f1ef8644c54d23ebfdf3f909c785adf0fb0
* Fix is_disconnect false positive for mssql+pyodbcGord Thompson2020-06-011-2/+43
| | | | | | | | | | Fixed an issue where the ``is_disconnect`` function in the SQL Server pyodbc dialect was incorrectly reporting the disconnect state when the exception messsage had a substring that matched a SQL Server ODBC error code. Fixes: #5359 Change-Id: I450c6818405a20f4daee20d58fce2d5ecb33e17f
* Don't emit pyodbc "no driver" warning for empty URLMike Bayer2020-05-221-0/+7
| | | | | | | | | | | Fixed an issue in the pyodbc connector such that a warning about pyodbc "drivername" would be emitted when using a totally empty URL. Empty URLs are normal when producing a non-connected dialect object or when using the "creator" argument to create_engine(). The warning now only emits if the driver name is missing but other parameters are still present. Fixes: #5346 Change-Id: I0ee6f5fd5af7faca63bf0d7034410942f40834a8
* Deprecate unsupported dialects and dbapiFederico Caselli2020-04-291-27/+0
| | | | | | | | | | | | | | | | | - Deprecate dialects firebird and sybase. - Deprecate DBAPI - mxODBC for mssql - oursql for mysql - pygresql and py-postgresql for postgresql - Removed adodbapi DBAPI for mssql Fixes: #5189 Change-Id: Id9025f4f4de7e97d65aacd0eb4b0c21beb9a67b5
* Deprecate plain string in execute and introduce `exec_driver_sql`Federico Caselli2020-03-211-7/+19
| | | | | | | | | | | | | | | Execution of literal sql string is deprecated in the :meth:`.Connection.execute` and a warning is raised when used stating that it will be coerced to :func:`.text` in a future release. To execute a raw sql string the new connection method :meth:`.Connection.exec_driver_sql` was added, that will retain the previous behavior, passing the string to the DBAPI driver unchanged. Usage of scalar or tuple positional parameters in :meth:`.Connection.execute` is also deprecated. Fixes: #4848 Fixes: #5178 Change-Id: I2830181054327996d594f7f0d59c157d477c3aa9
* Recognize message 20047 as disconnect event in MSDialect_pymssqlJon Schuff2019-05-151-0/+3
| | | | | | | | | | | | Added error code 20047 to "is_disconnect" for pymssql. Pull request courtesy Jon Schuff. Fixes: #4680 Closes: #4681 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4681 Pull-request-sha: bc81c935ec0e352734d9ad1b322caf6d08079c3d Change-Id: Ifc7ffc4c933b08a34fad537dc48e05d2cfa66d42
* Commit transaction after SNAPSHOT isolation changeMike Bayer2019-03-081-0/+34
| | | | | | | | | A commit() is emitted after an isolation level change to SNAPSHOT, as both pyodbc and pymssql open an implicit transaction which blocks subsequent SQL from being emitted in the current transaction. Fixes: #4536 Change-Id: If3ba70f495bce2a35a873a3a72d1b30406e678c8
* Post black reformattingMike Bayer2019-01-061-12/+16
| | | | | | | | | | | | | Applied on top of a pure run of black -l 79 in I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes resolves all remaining flake8 conditions for those codes we have enabled in setup.cfg. Included are resolutions for all remaining flake8 issues including shadowed builtins, long lines, import order, unused imports, duplicate imports, and docstring issues. Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
* Run black -l 79 against all source filesMike Bayer2019-01-061-143/+224
| | | | | | | | | | | | | | This is a straight reformat run using black as is, with no edits applied at all. The black run will format code consistently, however in some cases that are prevalent in SQLAlchemy code it produces too-long lines. The too-long lines will be resolved in the following commit that will resolve all remaining flake8 issues including shadowed builtins, long lines, import order, unused imports, duplicate imports, and docstring issues. Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
* Add pyodbc fast_executemanyMike Bayer2018-07-101-0/+36
| | | | | | | | | Added ``fast_executemany=True`` parameter to the SQL Server pyodbc dialect, which enables use of pyodbc's new performance feature of the same name when using Microsoft ODBC drivers. Change-Id: I743fa7280e8f709addd330cfc7682623701cbb2e Fixes: #4158
* Fix UnboundLocalError in mssql during isolation level grabMike Bayer2018-06-251-2/+61
| | | | | | | | | | | | Fixed issue within the SQL Server dialect under Python 3 where when running against a non-standard SQL server database that does not contain either the "sys.dm_exec_sessions" or "sys.dm_pdw_nodes_exec_sessions" views, leading to a failure to fetch the isolation level, the error raise would fail due to an UnboundLocalError. Fixes: #4273 Co-authored-by: wikiped <wikiped@yandex.ru> Change-Id: I39877c1f65f9cf8602fb1dceaf03072357759564
* Filter non-integer characters from pyodbc SQL Server versionMike Bayer2018-03-301-0/+31
| | | | | | | | | | Adjusted the SQL Server version detection for pyodbc to only allow for numeric tokens, filtering out non-integers, since the dialect does tuple- numeric comparisons with this value. This is normally true for all known SQL Server / pyodbc drivers in any case. Change-Id: I4ab18a07e19231091b5e877ba1fccd5eda72a992 Fixes: #4227
* Add full list of pyodbc error codes for MSSQLMike Bayer2017-09-281-0/+29
| | | | | | | | | Moved the SQL server error codes out of connnectors/pyodbc.py and into mssql/pyodbc.py. Added complete list of odbc-related disconnect codes. Change-Id: Icd84a920dbfa1f188847f859654ff6f7a48170f1 Fixes: #4095
* Make all tests to be PEP8 compliantKhairi Hafsham2017-02-071-8/+10
| | | | | | | | tested using pycodestyle version 2.2.0 Fixes: #3885 Change-Id: I5df43adc3aefe318f9eeab72a078247a548ec566 Pull-request: https://github.com/zzzeek/sqlalchemy/pull/343
* Quote URL tokens with semicolons for pyodbc, adodbapiMike Bayer2016-11-111-1/+44
| | | | | | | | | | Fixed bug in pyodbc dialect (as well as in the mostly non-working adodbapi dialect) whereby a semicolon present in the password or username fields could be interpreted as a separator for another token; the values are now quoted when semicolons are present. Change-Id: I5f99fd8db53ebf8e805e7d9d60bc09b8f1af603f Fixes: #3762
* - fix this test to not require pyodbc installedMike Bayer2015-06-031-1/+2
|
* - Fixed bug where known boolean values used byMike Bayer2015-05-261-0/+10
| | | | | | | | :func:`.engine_from_config` were not being parsed correctly; these included ``pool_threadlocal`` and the psycopg2 argument ``use_native_unicode``. fixes #3435 - add legacy_schema_aliasing config parsing for mssql - move use_native_unicode config arg to the psycopg2 dialect
* - SQL Server 2012 now recommends VARCHAR(max), NVARCHAR(max),Mike Bayer2014-12-061-2/+1
| | | | | | | VARBINARY(max) for large text/binary types. The MSSQL dialect will now respect this based on version detection, as well as the new ``deprecate_large_types`` flag. fixes #3039
* - Fixed the version string detection in the pymssql dialect toMike Bayer2014-09-161-0/+20
| | | | | | work with Microsoft SQL Azure, which changes the word "SQL Server" to "SQL Azure". fixes #3151
* - The hostname-based connection format for SQL Server when usingMike Bayer2014-09-031-6/+20
| | | | | | | | pyodbc will no longer specify a default "driver name", and a warning is emitted if this is missing. The optimal driver name for SQL Server changes frequently and is per-platform, so hostname based connections need to specify this. DSN-based connections are preferred. fixes #3182
* Remove terminated connections from the pool.pr/51John Anderson2013-12-161-3/+4
| | | | | | In pymssql, if you terminate a long running query manually it will give you a connection reset by peer message, but this connection remains in the pool and will be re-used.
* refactor test suites for postgresql, mssql, mysql into packages.Mike Bayer2013-06-281-0/+154
View Lorry Controller Status