summaryrefslogtreecommitdiff
path: root/test/sql/test_compiler.py
Commit message (Collapse)AuthorAgeFilesLines
* add deterministic imv returning ordering using sentinel columnsMike Bayer2023-04-211-0/+241
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Repaired a major shortcoming which was identified in the :ref:`engine_insertmanyvalues` performance optimization feature first introduced in the 2.0 series. This was a continuation of the change in 2.0.9 which disabled the SQL Server version of the feature due to a reliance in the ORM on apparent row ordering that is not guaranteed to take place. The fix applies new logic to all "insertmanyvalues" operations, which takes effect when a new parameter :paramref:`_dml.Insert.returning.sort_by_parameter_order` on the :meth:`_dml.Insert.returning` or :meth:`_dml.UpdateBase.return_defaults` methods, that through a combination of alternate SQL forms, direct correspondence of client side parameters, and in some cases downgrading to running row-at-a-time, will apply sorting to each batch of returned rows using correspondence to primary key or other unique values in each row which can be correlated to the input data. Performance impact is expected to be minimal as nearly all common primary key scenarios are suitable for parameter-ordered batching to be achieved for all backends other than SQLite, while "row-at-a-time" mode operates with a bare minimum of Python overhead compared to the very heavyweight approaches used in the 1.x series. For SQLite, there is no difference in performance when "row-at-a-time" mode is used. It's anticipated that with an efficient "row-at-a-time" INSERT with RETURNING batching capability, the "insertmanyvalues" feature can be later be more easily generalized to third party backends that include RETURNING support but not necessarily easy ways to guarantee a correspondence with parameter order. Fixes: #9618 References: #9603 Change-Id: I1d79353f5f19638f752936ba1c35e4dc235a8b7c
* ensure single import per lineMike Bayer2023-02-281-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds the very small plugin flake8-import-single which will prevent us from having an import with more than one symbol on a line. Flake8 by itself prevents this pattern with E401: import collections, os, sys However does not do anything with this: from sqlalchemy import Column, text Both statements have the same issues generating merge artifacts as well as presenting a manual decision to be made. While zimports generally cleans up such imports at the top level, we don't enforce zimports / pre-commit use. the plugin finds the same issue for imports that are inside of test methods. We shouldn't usually have imports in test methods so most of them here are moved to be top level. The version is pinned at 0.1.5; the project seems to have no activity since 2019, however there are three 0.1.6dev releases on pypi which stopped in September 2019, they seem to be experiments with packaging. The source for 0.1.5 is extremely simple and only reveals one method to flake8 (the run() method). Change-Id: Icea894e43bad9c0b5d4feb5f49c6c666d6ea6aa1
* fix ORM support for column-named bindparam() in crud .values()Mike Bayer2023-01-111-0/+176
| | | | | | | | | | | | | | | | | | | Fixed bug / regression where using :func:`.bindparam()` with the same name as a column in the :meth:`.Update.values` method of :class:`.Update`, as well as the :meth:`.Insert.values` method of :class:`.Insert` in 2.0 only, would in some cases silently fail to honor the SQL expression in which the parameter were presented, replacing the expression with a new parameter of the same name and discarding any other elements of the SQL expression, such as SQL functions, etc. The specific case would be statements that were constructed against ORM entities rather than plain :class:`.Table` instances, but would occur if the statement were invoked with a :class:`.Session` or a :class:`.Connection`. :class:`.Update` part of the issue was present in both 2.0 and 1.4 and is backported to 1.4. Fixes: #9075 Change-Id: Ie954bc1f492ec6a566163588182ef4910c7ee452
* make bind escape lookup extensibleMike Bayer2022-12-161-0/+39
| | | | | | | | | | | | | | To accommodate for third party dialects with different character escaping needs regarding bound parameters, the system by which SQLAlchemy "escapes" (i.e., replaces with another character in its place) special characters in bound parameter names has been made extensible for third party dialects, using the :attr:`.SQLCompiler.bindname_escape_chars` dictionary which can be overridden at the class declaration level on any :class:`.SQLCompiler` subclass. As part of this change, also added the dot ``"."`` as a default "escaped" character. Fixes: #8994 Change-Id: I52fbbfa8c64497b123f57327113df3f022bd1419
* fix construct_params() for render_postcompile; add new APIMike Bayer2022-12-081-44/+389
| | | | | | | | | | | | | | | | | | | | | The :meth:`.SQLCompiler.construct_params` method, as well as the :attr:`.SQLCompiler.params` accessor, will now return the exact parameters that correspond to a compiled statement that used the ``render_postcompile`` parameter to compile. Previously, the method returned a parameter structure that by itself didn't correspond to either the original parameters or the expanded ones. Passing a new dictionary of parameters to :meth:`.SQLCompiler.construct_params` for a :class:`.SQLCompiler` that was constructed with ``render_postcompile`` is now disallowed; instead, to make a new SQL string and parameter set for an alternate set of parameters, a new method :meth:`.SQLCompiler.construct_expanded_state` is added which will produce a new expanded form for the given parameter set, using the :class:`.ExpandedState` container which includes a new SQL statement and new parameter dictionary, as well as a positional parameter tuple. Fixes: #6114 Change-Id: I9874905bb90f86799b82b244d57369558b18fd93
* Rewrite positional handling, test for "numeric"Federico Caselli2022-12-051-68/+298
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Changed how the positional compilation is performed. It's rendered by the compiler the same as the pyformat compilation. The string is then processed to replace the placeholders with the correct ones, and to obtain the correct order of the parameters. This vastly simplifies the computation of the order of the parameters, that in case of nested CTE is very hard to compute correctly. Reworked how numeric paramstyle behavers: - added support for repeated parameter, without duplicating them like in normal positional dialects - implement insertmany support. This requires that the dialect supports out of order placehoders, since all parameters that are not part of the VALUES clauses are placed at the beginning of the parameter tuple - support for different identifiers for a numeric parameter. It's for example possible to use postgresql style placeholder $1, $2, etc Added two new dialect based on sqlite to test "numeric" fully using both :1 style and $1 style. Includes a workaround for SQLite's not-really-correct numeric implementation. Changed parmstyle of asyncpg dialect to use numeric, rendering with its native $ identifiers Fixes: #8926 Fixes: #8849 Change-Id: I7c640467d49adfe6d795cc84296fc7403dcad4d6
* Fix positional compiling bugsFederico Caselli2022-12-011-0/+118
| | | | | | | | | | | Fixed a series of issues regarding positionally rendered bound parameters, such as those used for SQLite, asyncpg, MySQL and others. Some compiled forms would not maintain the order of parameters correctly, such as the PostgreSQL ``regexp_replace()`` function as well as within the "nesting" feature of the :class:`.CTE` construct first introduced in :ticket:`4123`. Fixes: #8827 Change-Id: I9813ed7c358cc5c1e26725c48df546b209a442cb
* Implement ScalarValueFederico Caselli2022-11-261-3/+9
| | | | | | | | | | | | | | Added :class:`_expression.ScalarValues` that can be used as a column element allowing using :class:`_expression.Values` inside IN clauses or in conjunction with ``ANY`` or ``ALL`` collection aggregates. This new class is generated using the method :meth:`_expression.Values.scalar_values`. The :class:`_expression.Values` instance is now coerced to a :class:`_expression.ScalarValues` when used in a ``IN`` or ``NOT IN`` operation. Fixes: #6289 Change-Id: Iac22487ccb01553684b908e54d01c0687fa739f1
* Try running pyupgrade on the codeFederico Caselli2022-11-161-19/+7
| | | | | | | | command run is "pyupgrade --py37-plus --keep-runtime-typing --keep-percent-format <files...>" pyupgrade will change assert_ to assertTrue. That was reverted since assertTrue does not exists in sqlalchemy fixtures Change-Id: Ie1ed2675c7b11d893d78e028aad0d1576baebb55
* add informative exception context for literal renderMike Bayer2022-11-141-0/+46
| | | | | | | | | | An informative re-raise is now thrown in the case where any "literal bindparam" render operation fails, indicating the value itself and the datatype in use, to assist in debugging when literal params are being rendered in a statement. Fixes: #8800 Change-Id: Id658f8b03359312353ddbb0c7563026239579f7b
* Revert automatic set of sequence start to 1Federico Caselli2022-10-171-3/+3
| | | | | | | | | | | | | | | | | The :class:`.Sequence` construct restores itself to the DDL behavior it had prior to the 1.4 series, where creating a :class:`.Sequence` with no additional arguments will emit a simple ``CREATE SEQUENCE`` instruction **without** any additional parameters for "start value". For most backends, this is how things worked previously in any case; **however**, for MS SQL Server, the default value on this database is ``-2**63``; to prevent this generally impractical default from taking effect on SQL Server, the :paramref:`.Sequence.start` parameter should be provided. As usage of :class:`.Sequence` is unusual for SQL Server which for many years has standardized on ``IDENTITY``, it is hoped that this change has minimal impact. Fixes: #7211 Change-Id: I1207ea10c8cb1528a1519a0fb3581d9621c27b31
* ensure RETURNING renders in stringify w/ no server versionMike Bayer2022-08-031-0/+15
| | | | | | | | | just in my own testing, if I say insert().return_defaults() and stringify, I should see it, so make sure all the dialects default to "insert_returning" etc. , with downgrade on server version check. Change-Id: Id64e78fcb03c48b5dcb0feb21cb9cc495edd15e9
* restore parameter escaping for public methodsMike Bayer2022-06-091-5/+28
| | | | | | | | | | | | | | Adjusted the fix made for :ticket:`8056` which adjusted the escaping of bound parameter names with special characters such that the escaped names were translated after the SQL compilation step, which broke a published recipe on the FAQ illustrating how to merge parameter names into the string output of a compiled SQL string. The change restores the escaped names that come from ``compiled.params`` and adds a conditional parameter to :meth:`.SQLCompiler.construct_params` named ``escape_names`` that defaults to ``True``, restoring the old behavior by default. Fixes: #8113 Change-Id: I9cbedb1080bc06d51f287fd2cbf26aaab1c74653
* propagate proxy_key from WrapsColumnExpressionMike Bayer2022-06-011-1/+2
| | | | | | | | | | | | | this allows cast() of a label() to propagate the proxy key outwards in the same way that it apparently works at the SQL level. This is stuffing even more rules into naming so basically seeing how far we can go without other cases starting to fail. Fixes: #8084 Change-Id: I20bd97dae798fee6492334c06934e807d0f269ef
* move bindparam quote application from compiler to defaultMike Bayer2022-05-291-5/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | in 296c84313ab29bf9599634f3 for #5653 we generalized Oracle's parameter escaping feature into the compiler, so that it could also work for PostgreSQL. The compiler used quoted names within parameter dictionaries, which then led to the complexity that all functions which interpreted keys from the compiled_params dict had to also quote the param names to use the dictionary. This extra complexity was not added to the ORM peristence.py however, which led to the versioning id feature being broken as well as other areas where persistence.py relies on naming schemes present in context.compiled_params. It also was not added to the "processors" lookup which led to #8053, that added this escaping to that part of the compiler. To both solve the whole problem as well as simplify the compiler quite a bit, move the actual application of the escaped names to be as late as possible, when default.py builds the final list of parameters. This is more similar to how it worked previously where OracleExecutionContext would be late-applying these escaped names. This re-establishes context.compiled_params as deterministically named regardless of dialect in use and moves out the complexity of the quoted param names to be only at the cursor.execute stage. Fixed bug, likely a regression from 1.3, where usage of column names that require bound parameter escaping, more concretely when using Oracle with column names that require quoting such as those that start with an underscore, or in less common cases with some PostgreSQL drivers when using column names that contain percent signs, would cause the ORM versioning feature to not work correctly if the versioning column itself had such a name, as the ORM assumes certain bound parameter naming conventions that were being interfered with via the quotes. This issue is related to :ticket:`8053` and essentially revises the approach towards fixing this, revising the original issue :ticket:`5653` that created the initial implementation for generalized bound-parameter name quoting. Fixes: #8056 Change-Id: I57b064e8f0d070e328b65789c30076f6a0ca0fef
* apply bindparam escape name to processors dictionaryMike Bayer2022-05-251-0/+42
| | | | | | | | | | | | | Fixed SQL compiler issue where the "bind processing" function for a bound parameter would not be correctly applied to a bound value if the bound parameter's name were "escaped". Concretely, this applies, among other cases, to Oracle when a :class:`.Column` has a name that itself requires quoting, such that the quoting-required name is then used for the bound parameters generated within DML statements, and the datatype in use requires bind processing, such as the :class:`.Enum` datatype. Fixes: #8053 Change-Id: I39d060a87e240b4ebcfccaa9c535e971b7255d99
* render select froms firstMike Bayer2022-05-221-5/+88
| | | | | | | | | | | | | | | | | | | The FROM clauses that are established on a :func:`_sql.select` construct when using the :meth:`_sql.Select.select_from` method will now render first in the FROM clause of the rendered SELECT, which serves to maintain the ordering of clauses as was passed to the :meth:`_sql.Select.select_from` method itself without being affected by the presence of those clauses also being mentioned in other parts of the query. If other elements of the :class:`_sql.Select` also generate FROM clauses, such as the columns clause or WHERE clause, these will render after the clauses delivered by :meth:`_sql.Select.select_from` assuming they were not explictly passed to :meth:`_sql.Select.select_from` also. This improvement is useful in those cases where a particular database generates a desirable query plan based on a particular ordering of FROM clauses and allows full control over the ordering of FROM clauses. Fixes: #7888 Change-Id: I740f262a3841f829239011120a59b5e58452db5b
* raise for same param name in expanding + non expandingMike Bayer2022-05-151-0/+19
| | | | | | | | | | | | | An informative error is raised if two individual :class:`.BindParameter` objects share the same name, yet one is used within an "expanding" context (typically an IN expression) and the other is not; mixing the same name in these two different styles of usage is not supported and typically the ``expanding=True`` parameter should be set on the parameters that are to receive list values outside of IN expressions (where ``expanding`` is set by default). Fixes: #8018 Change-Id: Ie707f29680eea16b9e421af93560ac1958e11a54
* maintain complete cloned_set for BindParameterMike Bayer2022-04-061-3/+14
| | | | | | | | | | | Fixed regression caused by :ticket:`7823` which impacted the caching system, such that bound parameters that had been "cloned" within ORM operations, such as polymorphic loading, would in some cases not acquire their correct execution-time value leading to incorrect bind values being rendered. Fixes: #7903 Change-Id: I61c802749b859bebeb127d24e66d6e77d13ce57a
* pep484 - sql.selectableMike Bayer2022-04-041-13/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the pep484 task becomes more intense as there is mounting pressure to come up with a consistency in how data moves from end-user to instance variable. current thinking is coming into: 1. there are _typing._XYZArgument objects that represent "what the user sent" 2. there's the roles, which represent a kind of "filter" for different kinds of objects. These are mostly important as the argument we pass to coerce(). 3. there's the thing that coerce() returns, which should be what the construct uses as its internal representation of the thing. This is _typing._XYZElement. but there's some controversy over whether or not we should pass actual ClauseElements around by their role or not. I think we shouldn't at the moment, but this makes the "role-ness" of something a little less portable. Like, we have to set DMLTableRole for TableClause, Join, and Alias, but then also we have to repeat those three types in order to set up _DMLTableElement. Other change introduced here, there was a deannotate=True for the left/right of a sql.join(). All tests pass without that. I'd rather not have that there as if we have a join(A, B) where A, B are mapped classes, we want them inside of the _annotations. The rationale seems to be performance, but this performance can be illustrated to be on the compile side which we hope is cached in the normal case. CTEs now accommodate for text selects including recursive. Get typing to accommodate "util.preloaded" cleanly; add "preloaded" as a real module. This seemed like we would have needed pep562 `__getattr__()` but we don't, just set names in globals() as we import them. References: #6810 Change-Id: I34d17f617de2fe2c086fc556bd55748dc782faf0
* repair broken truediv test suite; memusageMike Bayer2022-01-201-4/+4
| | | | | | | | | | | | | the truediv test suite didn't have __backend__ so wasn't running for every DB except in the main build. Repaired this as well as truediv support to preserve the right-hand side type when casting to numeric, if the right type is already a numeric type. also fixed a memusage test that relies on savepoints so was not running under gerrit runs. Change-Id: I3be223fdf697af9c1ed61b70d621f57cbbb7a92b
* track item schema names to identify name collisions w/ default schemaMike Bayer2022-01-141-0/+72
| | | | | | | | | | | | | | | | | | Added an additional lookup step to the compiler which will track all FROM clauses which are tables, that may have the same name shared in multiple schemas where one of the schemas is the implicit "default" schema; in this case, the table name when referring to that name without a schema qualification will be rendered with an anonymous alias name at the compiler level in order to disambiguate the two (or more) names. The approach of schema-qualifying the normally unqualified name with the server-detected "default schema name" value was also considered, however this approach doesn't apply to Oracle nor is it accepted by SQL Server, nor would it work with multiple entries in the PostgreSQL search path. The name collision issue resolved here has been identified as affecting at least Oracle, PostgreSQL, SQL Server, MySQL and MariaDB. Fixes: #7471 Change-Id: Id65e7ca8c43fe8d95777084e8d5ec140ebcd784d
* ensure correlate_except is checked for empty tupleMike Bayer2022-01-031-0/+8
| | | | | | | | | | | Fixed issue where :meth:`_sql.Select.correlate_except` method, when passed either the ``None`` value or no arguments, would not correlate any elements when used in an ORM context (that is, passing ORM entities as FROM clauses), rather than causing all FROM elements to be considered as "correlated" in the same way which occurs when using Core-only constructs. Fixes: #7514 Change-Id: Ic4a5252c8f3c1140aba6c308264948f3a91f33f5
* consider truediv as truediv; support floordiv operatorMike Bayer2021-12-261-4/+5
| | | | | | | | | | | | | | | | | | | | | Implemented full support for "truediv" and "floordiv" using the "/" and "//" operators. A "truediv" operation between two expressions using :class:`_types.Integer` now considers the result to be :class:`_types.Numeric`, and the dialect-level compilation will cast the right operand to a numeric type on a dialect-specific basis to ensure truediv is achieved. For floordiv, conversion is also added for those databases that don't already do floordiv by default (MySQL, Oracle) and the ``FLOOR()`` function is rendered in this case, as well as for cases where the right operand is not an integer (needed for PostgreSQL, others). The change resolves issues both with inconsistent behavior of the division operator on different backends and also fixes an issue where integer division on Oracle would fail to be able to fetch a result due to inappropriate outputtypehandlers. Fixes: #4926 Change-Id: Id54cc018c1fb7a49dd3ce1216d68d40f43fe2659
* The where method of exists now accepts multiple cluase.Federico Caselli2021-12-041-0/+8
| | | | | | | | | Support multiple clause elements in the :meth:`_sql.Exists.where` method, unifying the api with the on presented by a normal :func:`_sql.select` construct. Fixes: #7386 Change-Id: I5df20478008cd5167053d357cbfad8a641c62b44
* Clean up most py3k compatFederico Caselli2021-11-241-3/+2
| | | | Change-Id: I8172fdcc3103ff92aa049827728484c8779af6b7
* Support lightweight compiler column elements w/ slotsMike Bayer2021-11-221-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | the _CompileLabel class included ``__slots__`` but these weren't used as the superclasses included slots. Create a ``__slots__`` superclass for ``ClauseElement``, creating a new class of compilable SQL elements that don't include heavier features like caching, annotations and cloning, which are meant to be used only in an ad-hoc compiler fashion. Create new ``CompilerColumnElement`` from that which serves in column-oriented contexts, but similarly does not include any expression operator support as it is intended to be used only to generate a string. Apply this to both ``_CompileLabel`` as well as PostgreSQL ``_ColonCast``, which does not actually subclass ``ColumnElement`` as this class has memoized attributes that aren't worth changing, and does not include SQL operator capabilities as these are not needed for these compiler-only objects. this allows us to more inexpensively add new ad-hoc labels / casts etc. at compile time, as we will be seeking to expand out the typecasts that are needed for PostgreSQL dialects in a subsequent patch. Change-Id: I52973ae3295cb6e2eb0d7adc816c678a626643ed
* change the POSTCOMPILE/ SCHEMA symbols to not conflict w mssql quotingMike Bayer2021-11-091-37/+37
| | | | | | | | | | | | | | | | | | Adjusted the compiler's generation of "post compile" symbols including those used for "expanding IN" as well as for the "schema translate map" to not be based directly on plain bracketed strings with underscores, as this conflicts directly with SQL Server's quoting format of also using brackets, which produces false matches when the compiler replaces "post compile" and "schema translate" symbols. The issue created easy to reproduce examples both with the :meth:`.Inspector.get_schema_names` method when used in conjunction with the :paramref:`_engine.Connection.execution_options.schema_translate_map` feature, as well in the unlikely case that a symbol overlapping with the internal name "POSTCOMPILE" would be used with a feature like "expanding in". Fixes: #7300 Change-Id: I6255c850b140522a4aba95085216d0bca18ce230
* fully implement future engine and remove legacyMike Bayer2021-11-071-14/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | The major action here is to lift and move future.Connection and future.Engine fully into sqlalchemy.engine.base. This removes lots of engine concepts, including: * autocommit * Connection running without a transaction, autobegin is now present in all cases * most "autorollback" is obsolete * Core-level subtransactions (i.e. MarkerTransaction) * "branched" connections, copies of connections * execution_options() returns self, not a new connection * old argument formats, distill_params(), simplifies calling scheme between engine methods * before/after_execute() events (oriented towards compiled constructs) don't emit for exec_driver_sql(). before/after_cursor_execute() is still included for this * old helper methods superseded by context managers, connection.transaction(), engine.transaction() engine.run_callable() * ancient engine-level reflection methods has_table(), table_names() * sqlalchemy.testing.engines.proxying_engine References: #7257 Change-Id: Ib20ed816642d873b84221378a9ec34480e01e82c
* Remove deprecated dialects and driversFederico Caselli2021-10-311-8/+8
| | | | | Fixes: #7258 Change-Id: I3577f665eca04f2632b69bcb090f0a4ec9271db9
* fixes for usage of the null() and similar constantsMike Bayer2021-10-081-19/+77
| | | | | | | | | | | | | | | | | | | | | | | | Adjusted the "column disambiguation" logic that's new in 1.4, where the same expression repeated gets an "extra anonymous" label, so that the logic more aggressively deduplicates those labels when the repeated element is the same Python expression object each time, as occurs in cases like when using "singleton" values like :func:`_sql.null`. This is based on the observation that at least some databases (e.g. MySQL, but not SQLite) will raise an error if the same label is repeated inside of a subquery. Related to :ticket:`7153`, fixed an issue where result column lookups would fail for "adapted" SELECT statements that selected for "constant" value expressions most typically the NULL expression, as would occur in such places as joined eager loading in conjunction with limit/offset. This was overall a regression due to issue :ticket:`6259` which removed all "adaption" for constants like NULL, "true", and "false", but this broke the case where the same adaption logic were used to match the constant to a labeled expression referring to the constant in a subquery. Fixes: #7153 Fixes: #7154 Change-Id: I43823343721b9e70524ea3f5e8f39dd543a3e92b
* Fix various lib / test / examples typos (#7017)Kevin Kirsche2021-09-111-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * fix: lib/sqlalchemy/sql/lambdas.py * fix: lib/sqlalchemy/sql/compiler.py * fix: lib/sqlalchemy/sql/selectable.py * fix: lib/sqlalchemy/orm/relationships.py * fix: lib/sqlalchemy/dialects/mssql/base.py * fix: lib/sql/test_compiler.py * fix: lib/sqlalchemy/testing/requirements.py * fix: lib/sqlalchemy/orm/path_registry.py * fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py * fix: lib/sqlalchemy/cextension/immutabledict.c * fix: lib/sqlalchemy/cextension/resultproxy.c * fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py * fix: examples/versioned_rows/versioned_rows_w_versionid.py * fix: examples/elementtree/optimized_al.py * fix: test/orm/test_attribute.py * fix: test/sql/test_compare.py * fix: test/sql/test_type_expression.py * fix: capitalization in test/dialect/mysql/test_compiler.py * fix: typos in test/dialect/postgresql/test_reflection.py * fix: typo in tox.ini comment * fix: typo in /lib/sqlalchemy/orm/decl_api.py * fix: typo in test/orm/test_update_delete.py * fix: self-induced typo * fix: typo in test/orm/test_query.py * fix: typos in test/dialect/mssql/test_types.py * fix: typo in test/sql/test_types.py
* dont qualify literal_binds with literal_executeMike Bayer2021-08-071-0/+18
| | | | | | | | | | | | | | | this appears to be unnecessary and prevents end-user literal_binds case from working. Fixed issue where the ``literal_binds`` compiler flag, as used externally to render bound parameters inline, would fail to work when used with a certain class of parameters known as "literal_execute", which covers things like LIMIT and OFFSET values for dialects where the drivers don't allow a bound parameter, such as SQL Server's "TOP" clause. The issue locally seemed to affect only the MSSQL dialect. Fixes: #6863 Change-Id: Ia74cff5b0107b129a11b9b965883552b2962e449
* accommodate for cloned bindparams w/ maintain_keyMike Bayer2021-07-291-0/+16
| | | | | | | | | | | Fixed issue where a bound parameter object that was "cloned" would cause a name conflict in the compiler, if more than one clone of this parameter were used at the same time in a single statement. This could occur in particular with things like ORM single table inheritance queries that indicated the same "discriminator" value multiple times in one query. Fixes: #6824 Change-Id: Iba7a786fc5a2341ff7d07fc666d24ed790ad4fe8
* Adjust CTE recrusive col list to accommodate dupe col namesMike Bayer2021-07-131-0/+3
| | | | | | | | | | | | | | | | | | | Fixed issue in CTE constructs where a recursive CTE that referred to a SELECT that has duplicate column names, which are typically deduplicated using labeling logic in 1.4, would fail to refer to the deduplicated label name correctly within the WITH clause. As part of this change we are also attempting to remove the behavior of SelectStatementGrouping forcing off the "asfrom" contextual flag, which will have the result of additional labeling being applied to some UNION and similar statements when they are interpreted as subqueries. To maintain compatibility with "grouping", the Grouping/SelectStatementGrouping are now broken out into two separate compiler cases, as the "asfrom" logic appears to be tailored towards table valued SELECTS as column expressions. Fixes: #6710 Change-Id: I8af07a5c670dbe5736cd9f16084ef82f5e4c8642
* labeling refactorMike Bayer2021-07-121-5/+99
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To service #6718 and #6710, the system by which columns are given labels in a SELECT statement as well as the system that gives them keys in a .c or .selected_columns collection have been refactored to provide a single source of truth for both, in constrast to the previous approach that included similar logic repeated in slightly different ways. Main ideas: 1. ColumnElement attributes ._label, ._anon_label, ._key_label are renamed to include the letters "tq", meaning "table-qualified" - these labels are only used when rendering a SELECT that has LABEL_STYLE_TABLENAME_PLUS_COL for its label style; as this label style is primarily legacy, the "tq" names should be isolated so that in a 2.0 style application these aren't being used at all 2. The means by which the "labels" and "proxy keys" for the elements of a SELECT has been centralized to a single source of truth; previously, the three of _generate_columns_plus_names, _generate_fromclause_column_proxies, and _column_naming_convention all had duplicated rules between them, as well as that there were a little bit of labeling rules in compiler._label_select_column as well; by this we mean that the various "anon_label" "anon_key" methods on ColumnElement were called by all four of these methods, where there were many cases where it was necessary that one method comes up with the same answer as another of the methods. This has all been centralized into _generate_columns_plus_names for all the names except the "proxy key", which is generated by _column_naming_convention. 3. compiler._label_select_column has been rewritten to both not make any naming decisions nor any "proxy key" decisions, only whether to label or not to label; the _generate_columns_plus_names method gives it the information, where the proxy keys come from _column_naming_convention; previously, these proxy keys were matched based on restatement of similar (but not really the same) logic in two places. The heuristics of "whether to label or not to label" are also reorganized to be much easier to read and understand. 4. a new method compiler._label_returning_column is added for dialects to use in their "generate returning columns" methods. A github search reveals a small number of third party dialects also doing this using the prior _label_select_column method so we try to make sure _label_select_column continues to work the exact same way for that specific use case; for the "SELECT" use case it now needs 5. After some attempts to do it different ways, for the case where _proxy_key is giving us some kind of anon label, we are hard changing it to "_no_label" right now, as there's not currently a way to fully match anonymized labels from stmt.c or stmt.selected_columns to what will be in the result map. The idea of "_no_label" is to encourage the user to use label('name') for columns they want to be able to target by string name that don't have a natural name. Change-Id: I7a92a66f3a7e459ccf32587ac0a3c306650daf11
* accommodate for mutiple copies of bind in ckbmMike Bayer2021-04-281-0/+98
| | | | | | | | | | | | | | | | Fixed critical regression where bound parameter tracking as used in the SQL caching system could fail to track all parameters for the case where the same SQL expression containing a parameter were used in an ORM-related query using a feature such as class inheritance, which was then embedded in an enclosing expression which would make use of that same expression multiple times, such as a UNION. The ORM would individually copy the individual SELECT statements as part of compilation with class inheritance, which then embedded in the enclosing statement would fail to accommodate for all parameters. The logic that tracks this condition has been adjusted to work for multiple copies of a parameter. Fixes: #6391 Change-Id: I6db5dee0d361a3bb58d753a2d27ef2eee2b369c5
* Don't stringify unnamed column elements when proxyingMike Bayer2021-04-171-6/+11
| | | | | | | | | | | | | | | | | | | | | | | | Repaired and solidified issues regarding custom functions and other arbitrary expression constructs which within SQLAlchemy's column labeling mechanics would seek to use ``str(obj)`` to get a string representation to use as an anonymous column name in the ``.c`` collection of a subquery. This is a very legacy behavior that performs poorly and leads to lots of issues, so has been revised to no longer perform any compilation by establishing specific methods on :class:`.FunctionElement` to handle this case, as SQL functions are the only use case that it came into play. An effect of this behavior is that an unlabeled column expression with no derivable name will be given an arbitrary label starting with the prefix ``"_no_label"`` in the ``.c`` collection of a subquery; these were previously being represented either as the generic stringification of that expression, or as an internal symbol. This change seeks to make the concept of "anon name" more private and renames anon_label and anon_key_label to _anon_name_label and _anon_key_label. There's no end-user utility to these accessors and we need to be able to reorganize these as well. Fixes: #6256 Change-Id: Ie63c86b20ca45873affea78500388da94cf8bf94
* Merge "support multivalues insert on strsqlcompiler"mike bayer2021-04-071-0/+11
|\
| * support multivalues insert on strsqlcompilerMike Bayer2021-04-071-0/+11
| | | | | | | | | | | | | | | | Fixed the "stringify" compiler to support a basic stringification of a "multirow" INSERT statement, i.e. one with multiple tuples following the VALUES keyword. Change-Id: I1fe38d204d9965275d3a72157d5a72a53bec4b11
* | Expand reg for schema translate map for most special charactersMike Bayer2021-04-071-0/+70
|/ | | | | | | | | | | | Fixed regression where usage of a token in the :paramref:`_engine.Connection.execution_options.schema_translate_map` dictionary which contained special characters such as braces would fail to be substituted properly. Use of square bracket characters ``[]`` is now explicitly disallowed as these are used as a delimiter character in the current implementation. Fixes: #6216 Change-Id: I7ccfc2292b17340054cedf485ed1adf3119b96c8
* Merge "Disallow AliasedReturnsRows from execution"mike bayer2021-04-061-4/+4
|\
| * Disallow AliasedReturnsRows from executionMike Bayer2021-04-051-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Executing a :class:`_sql.Subquery` using :meth:`_engine.Connection.execute` is deprecated and will emit a deprecation warning; this use case was an oversight that should have been removed from 1.4. The operation will now execute the underlying :class:`_sql.Select` object directly for backwards compatibility. Similarly, the :class:`_sql.CTE` class is also not appropriate for execution. In 1.3, attempting to execute a CTE would result in an invalid "blank" SQL statement being executed; since this use case was not working it now raises :class:`_exc.ObjectNotExecutableError`. Previously, 1.4 was attempting to execute the CTE as a statement however it was working only erratically. The change also breaks out StatementRole from ReturnsRowsRole, as these roles should not be in the same lineage (some statements don't return rows, the whole class of ReturnsRows that are from clauses are not statements). Consolidate StatementRole and CoerceTextStatementRole as there's no usage difference between these. Simplify some old tests that were trying to make sure that "execution options" didn't transmit from a cte/subquery out to a select; as cte/subuqery() aren't executable in any case the options are removed. Fixes: #6204 Change-Id: I62613b7ab418afdd22c409eae75659e3f52fb65f
* | uniquify when popping literal_execute_params from param dictMike Bayer2021-04-051-0/+85
|/ | | | | | | | | | | Fixed further issues in the same area as that of :ticket:`6173` released in 1.4.5, where a "postcompile" parameter, again most typically those used for LIMIT/OFFSET rendering in Oracle and SQL Server, would fail to be processed correctly if the same parameter rendered in multiple places in the statement. Fixes: #6202 Change-Id: I95c355aa52a7546fe579ad67f9a8402a213cb79d
* improve targeting and labeling for unary() in columns clauseMike Bayer2021-03-061-0/+17
| | | | | | | | | | | | | | | | | | | | | | Fixed regression where usage of the standalone :func:`_sql.distinct()` used in the form of being directly SELECTed would fail to be locatable in the result set by column identity, which is how the ORM locates columns. While standalone :func:`_sql.distinct()` is not oriented towards being directly SELECTed (use :meth:`_sql.select.distinct` for a regular ``SELECT DISTINCT..``) , it was usable to a limited extent in this way previously (but wouldn't work in subqueries, for example). The column targeting for unary expressions such as "DISTINCT <col>" has been improved so that this case works again, and an additional improvement has been made so that usage of this form in a subquery at least generates valid SQL which was not the case previously. The change additionally enhances the ability to target elements in ``row._mapping`` based on SQL expression objects in ORM-enabled SELECT statements, including whether the statement was invoked by ``connection.execute()`` or ``session.execute()``. Fixes: #6008 Change-Id: I5cfa39435f5418861d70a7db8f52ab4ced6a792e
* Merge "support stringify for plain CTE"mike bayer2021-02-281-0/+11
|\
| * support stringify for plain CTEMike Bayer2021-02-271-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | Altered the compilation for the :class:`.CTE` construct so that a string is returned representing the inner SELECT statement if the :class:`.CTE` is stringified directly, outside of the context of an enclosing SELECT; This is the same behavior of :meth:`_FromClause.alias` and :meth:`_SelectStatement.subquery`. Previously, a blank string would be returned as the CTE is normally placed above a SELECT after that SELECT has been generated, which is generally misleading when debugging. Change-Id: Id3007c28e4a7a56d867e850bb890752946bd8f6f References: #5988
* | implement visit_unsupported_compilation for TypeCompilerMike Bayer2021-02-251-0/+14
|/ | | | | | | | Fixed regression where the "unsupported compilation error" for unknown datatypes would fail to raise correctly. Fixes: #5979 Change-Id: I984fe95666813832ab5bdfc568322e2aa7cc3db0
* Further refine labeling for renamed columnsMike Bayer2021-02-121-1/+2
| | | | | | | | | | | | | | | Forked from I22f6cf0f0b3360e55299cdcb2452cead2b2458ea we are attempting to decide the case for columns mapped under a different name. since the .key feature of Column seems to support this fully, see if an annotation can be used to indicate an effective .key for a column. The effective change is that the labeling of column expressions in rows has been improved to retain the original name of the ORM attribute even if used in a subquery. References: #5933 Change-Id: If251f556f7d723f50d349f765f1690d6c679d2ef
* Render NULL for bindparam w/ None value/literal_binds, warnMike Bayer2021-01-281-0/+68
| | | | | | | | | | | | | | | | | Adjusted the "literal_binds" feature of :class:`_sql.Compiler` to render NULL for a bound parameter that has ``None`` as the value, either explicitly passed or omitted. The previous error message "bind parameter without a renderable value" is removed, and a missing or ``None`` value will now render NULL in all cases. Previously, rendering of NULL was starting to happen for DML statements due to internal refactorings, but was not explicitly part of test coverage, which it now is. While no error is raised, when the context is within that of a column comparison, and the operator is not "IS"/"IS NOT", a warning is emitted that this is not generally useful from a SQL perspective. Fixes: #5888 Change-Id: Id5939d8dbfb1156a9f8a7f7e76cf18327155331a
View Lorry Controller Status