summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/sql
Commit message (Collapse)AuthorAgeFilesLines
...
* | Merge "Add Executable to DefaultGenerator"mike bayer2021-06-241-2/+3
|\ \
| * | Add Executable to DefaultGeneratorMike Bayer2021-06-231-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed the class hierarchy for the :class:`_schema.Sequence` and the more general :class:`_schema.DefaultGenerator` base, as these are "executable" as statements they need to include :class:`_sql.Executable` in their hierarchy, not just :class:`_roles.StatementRole` as was applied arbitrarily to :class:`_schema.Sequence` previously. The fix allows :class:`_schema.Sequence` to work in all ``.execute()`` methods including with :meth:`_orm.Session.execute` which was not working in the case that a ``do_orm_execute()`` handler was also established. Fixes: #6668 Change-Id: I0d192258c7cbd1bce2552f9e748e8fdd680dc45f
* | | consider "*" col as textual orderedMike Bayer2021-06-231-1/+1
|/ / | | | | | | | | | | | | | | | | Fixed old issue where a :func:`_sql.select()` made against the token "*", which then yielded exactly one column, would fail to correctly organize the ``cursor.description`` column name into the keys of the result object. Fixes: #6665 Change-Id: Ie8c00f62998972ad4a19a750d2642d00fde006f6
* | Export deferred columns but not col props; fix CTE labelingMike Bayer2021-06-223-51/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refined the behavior of ORM subquery rendering with regards to deferred columns and column properties to be more compatible with that of 1.3 while also providing for 1.4's newer features. As a subquery in 1.4 does not make use of loader options, including :func:`_orm.deferred`, a subquery that is against an ORM entity with deferred attributes will now render those deferred attributes that refer directly to mapped table columns, as these are needed in the outer SELECT if that outer SELECT makes use of these columns; however a deferred attribute that refers to a composed SQL expression as we normally do with :func:`_orm.column_property` will not be part of the subquery, as these can be selected explicitly if needed in the subquery. If the entity is being SELECTed from this subquery, the column expression can still render on "the outside" in terms of the derived subquery columns. This produces essentially the same behavior as when working with 1.3. However in this case the fix has to also make sure that the ``.selected_columns`` collection of an ORM-enabled :func:`_sql.select` also follows these rules, which in particular allows recursive CTEs to render correctly in this scenario, which were previously failing to render correctly due to this issue. As part of this change the _exported_columns_iterator() method has been removed and logic simplified to use ._all_selected_columns from any SelectBase object where _exported_columns_iterator() was used before. Additionally sets up UpdateBase to include ReturnsRows in its hierarchy; the literal point of ReturnsRows was to be a common base for UpdateBase and SelectBase so it was kind of weird it wasn't there. Fixes: #6661 Fixed issue in CTE constructs mostly relevant to ORM use cases where a recursive CTE against "anonymous" labels such as those seen in ORM ``column_property()`` mappings would render in the ``WITH RECURSIVE xyz(...)`` section as their raw internal label and not a cleanly anonymized name. Fixes: #6663 Change-Id: I26219d4d8e6c0915b641426e9885540f74fae4d2
* | Implement MySQL-specific MATCHAnton Kovalevich2021-06-211-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | Added new construct :class:`_mysql.match`, which provides for the full range of MySQL's MATCH operator including multiple column support and modifiers. Pull request courtesy Anton Kovalevich. Fixes: #6132 Closes: #6133 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/6133 Pull-request-sha: dc6842f13688849a848e2ecbb81600e6edf8b3a9 Change-Id: I66bbfd7947aa2e43a031772e9b5ae238d94e5223
* | memoize current options and joins w with_entities/with_only_colsMike Bayer2021-06-174-46/+121
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed further regressions in the same area as that of :ticket:`6052` where loader options as well as invocations of methods like :meth:`_orm.Query.join` would fail if the left side of the statement for which the option/join depends upon were replaced by using the :meth:`_orm.Query.with_entities` method, or when using 2.0 style queries when using the :meth:`_sql.Select.with_only_columns` method. A new set of state has been added to the objects which tracks the "left" entities that the options / join were made against which is memoized when the lead entities are changed. Fixes: #6503 Fixes: #6253 Change-Id: I211b2af98b0b20d1263fb15dc513884dcc5de6a4
* | document missing ColumnCollection methodsMike Bayer2021-06-131-25/+86
| | | | | | | | | | | | | | | | add additional notes regarding the methods that are on this object. Fixes: #6629 Change-Id: I68446cb16cbfcef9af9c2998687b3b536607cf0d
* | repair Join.is_derived_from() to not rely on simple identityMike Bayer2021-06-071-1/+3
| | | | | | | | | | | | | | | | | | Fixed issue where query production for joinedload against a complex left hand side involving joined-table inheritance could fail to produce a correct query, due to a clause adaption issue. Fixes: #6595 Change-Id: Id4b839d52447cdc103b392dd8946c4cfa7a829e1
* | don't enable "fast insert executemany" for ON CONFLICT etcMike Bayer2021-06-041-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | Fixed issue where using the PostgreSQL "INSERT..ON CONFLICT" structure would fail to work with the psycopg2 driver if it were used in an "executemany" context along with bound parameters in the "SET" clause, due to the implicit use of the psycopg2 fast execution helpers which are not appropriate for this style of INSERT statement. Additional checks to exclude this kind of statement from that particular extension have been added. Fixes: #6581 Change-Id: I3d6169e7e188dc087d1d1bfba9a42162db183265
* | removing unavailable parameters from documentationCalli Rogers2021-06-021-5/+0
| |
* | Ensure propagate_attrs available on PropComparatorMike Bayer2021-05-291-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed regression caused by just-released performance fix mentioned in #6550 where a query.join() to a relationship could produce an AttributeError if the query were made against non-ORM structures only, a fairly unusual calling pattern. In this fix, since we are no longer going through the production of ``__clause_element__()`` for Cls.some_relationship, I assumed we just throw this object away completely but I missed the one little bit where we might be getting ``_propagate_attrs`` from it. So we implement ``_propagate_attrs`` on ``PropComparator`` as well, since this is easy to define. Fixes: #6558 Change-Id: If781bf844e7e3d3b0841aff1c3668e9d6af9f097
* | Fix adaption in AnnotatedLabel; repair needless expense in coercionMike Bayer2021-05-283-6/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed regression involving clause adaption of labeled ORM compound elements, such as single-table inheritance discriminator expressions with conditionals or CASE expressions, which could cause aliased expressions such as those used in ORM join / joinedload operations to not be adapted correctly, such as referring to the wrong table in the ON clause in a join. This change also improves a performance bump that was located within the process of invoking :meth:`_sql.Select.join` given an ORM attribute as a target. Fixes: #6550 Change-Id: I98906476f0cce6f41ea00b77c789baa818e9d167
* | Fix minor glitches in the visitors.py docstrings (#6522)Lele Gaifax2021-05-251-5/+5
| | | | | | | | | | | | | | | | | | | | | | * Fix docstring typo * Capitalize first sentence word * Properly markup class name This is just for consistency with nearby references: the HasCacheKey class has no docstring so it does not appear in the documentation, moreover it's in the traversals module, so Sphinx cannot render it as a link anyway.
* | Merge "Remove pep484 type comments from the code"mike bayer2021-05-184-39/+10
|\ \
| * | Remove pep484 type comments from the codeFederico Caselli2021-05-164-39/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Current effort is around the stub package, and having typing in two places makes thing worse, since the types here are usually outdated compared to the version in the stubs. Once v2 gets under way we can start consolidating the types here. Fixes: #6461 Change-Id: I7132a444bd7138123074bf5bc664b4bb119a85ce
* | | Run SelectState from obj normalize ahead of calcing ORM joinsMike Bayer2021-05-171-21/+40
|/ / | | | | | | | | | | | | | | | | | | Fixed regression where the full combination of joined inheritance, global with_polymorphic, self-referential relationship and joined loading would fail to be able to produce a query with the scope of lazy loads and object refresh operations that also attempted to render the joined loader. Fixes: #6495 Change-Id: If74a744c237069e3a89617498096c18b9b6e5dde
* | Update black flak8 and zimportsFederico Caselli2021-05-122-3/+4
| | | | | | | | Change-Id: I488c9557eda390e4a88319affd4c8813ee274f80
* | Merge "allow CTE to be direct DML target"mike bayer2021-05-111-1/+7
|\ \
| * | allow CTE to be direct DML targetMike Bayer2021-05-111-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implemented support for a :class:`_sql.CTE` construct to be used directly as the target of a :func:`_sql.delete` construct, i.e. "WITH ... AS cte DELETE FROM cte". This appears to be a useful feature of SQL Server. The CTE is now generally usable as a DML target table however it's not clear if this syntax is valid beyond the use case of DELETE itself. Fixes: #6464 Change-Id: I3aac6fae2a1abb39bc0ffa87a044f5eb4f90f026
* | | set bindparam.expanding in coercion againMike Bayer2021-05-104-24/+26
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adjusted the logic added as part of :ticket:`6397` in 1.4.12 so that internal mutation of the :class:`.BindParameter` object occurs within the clause construction phase as it did before, rather than in the compilation phase. In the latter case, the mutation still produced side effects against the incoming construct and additionally could potentially interfere with other internal mutation routines. In order to solve the issue of the correct operator being present on the BindParameter.expand_op, we necessarily have to expand the BinaryExpression._negate() routine to flip the operator on the BindParameter also. Fixes: #6460 Change-Id: I1e53a9aeee4de4fc11af51d7593431532731561b
* | Correct cache key for proxy_owner, with_context_optionsMike Bayer2021-05-103-3/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed issue in subquery loader strategy which prevented caching from working correctly. This would have been seen in the logs as a "generated" message instead of "cached" for all subqueryload SQL emitted, which by saturating the cache with new keys would degrade overall performance; it also would produce "LRU size alert" warnings. In this issue we also observe that the local LRU cache for lazyloader and selectinloader will get used for all subsequent loads as well, which makes it more likely to hit the limit of 30. However rather than trying to work this out, it would be better if we removed the loader-local LRU caches altogether once we are confident these are working well. Fixes: #6459 Change-Id: Id953e8f75536bb87f7e3315929cebcd8f84a5a50
* | fix absolute importMike Bayer2021-05-061-1/+1
| | | | | | | | Change-Id: I57e7bb8ac8f53c601b2ec5462d7502f8e53f1b0f
* | Merge "don't cache TypeDecorator by default"mike bayer2021-05-063-1/+80
|\ \
| * | don't cache TypeDecorator by defaultMike Bayer2021-05-063-1/+80
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The :class:`.TypeDecorator` class will now emit a warning when used in SQL compilation with caching unless the ``.cache_ok`` flag is set to ``True`` or ``False``. ``.cache_ok`` indicates that all the parameters passed to the object are safe to be used as a cache key, ``False`` means they are not. Fixes: #6436 Change-Id: Ib1bb7dc4b124e38521d615c2e2e691e4915594fb
* | | Add a comment in the compiler on why the NOT IN operatorFederico Caselli2021-05-051-0/+3
| | | | | | | | | | | | | | | | | | is rendered using brackets. Change-Id: I39c2e6a03e8aad7ff27bc6000da5bc9f1de14dc2
* | | Parenthesize for empty not inMike Bayer2021-05-051-7/+9
|/ / | | | | | | | | | | | | | | | | | | | | | | | | Fixed regression caused by the "empty in" change just made in :ticket:`6397` 1.4.12 where the expression needs to be parenthesized for the "not in" use case, otherwise the condition will interfere with the other filtering criteria. also amends StrSQLCompiler to use the newer "empty IN" style for its compilation process. Fixes: #6428 Change-Id: I182a552fc0d3065a9e38c0f4ece2deb143735c36
* | Support filter_by() from columns, functions, Core SQLMike Bayer2021-05-013-1/+44
| | | | | | | | | | | | | | | | | | | | | | | | Fixed regression where :meth:`_orm.Query.filter_by` would not work if the lead entity were a SQL function or other expression derived from the primary entity in question, rather than a simple entity or column of that entity. Additionally, improved the behavior of :meth:`_sql.Select.filter_by` overall to work with column expressions even in a non-ORM context. Fixes: #6414 Change-Id: I316b5bf98293bec1ede08787f6181dd14be85419
* | track_on needs to be a fixed size, support sub-tuplesMike Bayer2021-04-301-1/+10
| | | | | | | | | | | | | | | | | | | | Fixed regression in ``selectinload`` loader strategy that would cause it to cache its internal state incorrectly when handling relationships that join across more than one column, such as when using a composite foreign key. The invalid caching would then cause other loader operations to fail. Fixes: #6410 Change-Id: I9f95ccca3553e7fd5794c619be4cf85c02b04626
* | Merge "Ensure iterable passed to Select is not a mapped class"mike bayer2021-04-291-2/+12
|\ \
| * | Ensure iterable passed to Select is not a mapped classMike Bayer2021-04-291-2/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed regression caused by :ticket:`5395` where tuning back the check for sequences in :func:`_sql.select` now caused failures when doing 2.0-style querying with a mapped class that also happens to have an ``__iter__()`` method. Tuned the check some more to accommodate this as well as some other interesting ``__iter__()`` scenarios. Fixes: #6300 Change-Id: Idf1983fd764b91a7d5fa8117aee8a3def3cfe5ff
* | | Merge "Fix ForeignKeyConstraint.copy() error"mike bayer2021-04-291-6/+6
|\ \ \ | |/ / |/| |
| * | Fix ForeignKeyConstraint.copy() errorGord Thompson2021-04-291-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed an issue with the (deprecated in 1.4) :meth:`_schema.ForeignKeyConstraint.copy` method that caused an error when invoked with the ``schema`` argument. Fixes: #6353 Change-Id: I03330d9ec254d64377f2b2e86af69a4eaff43ac6
* | | Merge "Use non-subquery form for empty IN"mike bayer2021-04-293-12/+59
|\ \ \ | |/ / |/| |
| * | Use non-subquery form for empty INMike Bayer2021-04-293-12/+59
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Revised the "EMPTY IN" expression to no longer rely upon using a subquery, as this was causing some compatibility and performance problems. The new approach for selected databases takes advantage of using a NULL-returning IN expression combined with the usual "1 != 1" or "1 = 1" expression appended by AND or OR. The expression is now the default for all backends other than SQLite, which still had some compatibility issues regarding tuple "IN" for older SQLite versions. Third party dialects can still override how the "empty set" expression renders by implementing a new compiler method ``def visit_empty_set_op_expr(self, type_, expand_op)``, which takes precedence over the existing ``def visit_empty_set_expr(self, element_types)`` which remains in place. Fixes: #6258 Fixes: #6397 Change-Id: I2df09eb00d2ad3b57039ae48128fdf94641b5e59
* | | Merge "Propertly ignore ``Identity`` in MySQL and MariaDb."mike bayer2021-04-291-2/+7
|\ \ \
| * | | Propertly ignore ``Identity`` in MySQL and MariaDb.Federico Caselli2021-04-281-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Ensure that the MySQL and MariaDB dialect ignore the :class:`_sql.Identity` construct while rendering the ``AUTO_INCREMENT`` keyword in a create table. The Oracle and PostgreSQL compiler was updated to not render :class:`_sql.Identity` if the database version does not support it (Oracle < 12 and PostgreSQL < 10). Previously it was rendered regardless of the database version. Fixes: #6338 Change-Id: I2ca0902fdd7b4be4fc1a563cf5585504cbea9360
* | | | Merge "ensure SelectState.all_selected_columns not memoized as a generator"mike bayer2021-04-281-2/+1
|\ \ \ \
| * | | | ensure SelectState.all_selected_columns not memoized as a generatorMike Bayer2021-04-281-2/+1
| | |/ / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | This regression was found from #6261 before release of 1.4.12 Fixes: #6261 Fixes: #6394 Change-Id: I4c2c5da02dbcf5c2cea26c81d3521de892ef428e
* | | | accommodate for mutiple copies of bind in ckbmMike Bayer2021-04-281-4/+4
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed critical regression where bound parameter tracking as used in the SQL caching system could fail to track all parameters for the case where the same SQL expression containing a parameter were used in an ORM-related query using a feature such as class inheritance, which was then embedded in an enclosing expression which would make use of that same expression multiple times, such as a UNION. The ORM would individually copy the individual SELECT statements as part of compilation with class inheritance, which then embedded in the enclosing statement would fail to accommodate for all parameters. The logic that tracks this condition has been adjusted to work for multiple copies of a parameter. Fixes: #6391 Change-Id: I6db5dee0d361a3bb58d753a2d27ef2eee2b369c5
* | | Merge "have SchemaType inherit schema from metadata"mike bayer2021-04-271-5/+27
|\ \ \
| * | | have SchemaType inherit schema from metadataMike Bayer2021-04-271-5/+27
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed very old issue where the :class:`_types.Enum` datatype would not inherit the :paramref:`_schema.MetaData.schema` parameter of a :class:`_schema.MetaData` object when that object were passed to the :class:`_types.Enum` using :paramref:`_types.Enum.metadata`. Fixes: #6373 Change-Id: Ie77d5e8cbc0bd7bfd0039fb60a4a0bde2df58ca9
* | | Handle Sequence in _process_multiparam_default_bindMike Bayer2021-04-261-1/+10
|/ / | | | | | | | | | | | | | | | | | | | | Fixed issue where usage of an explicit :class:`.Sequence` would produce inconsistent "inline" behavior for an :class:`.Insert` construct that includes multiple values phrases; the first seq would be inline but subsequent ones would be "pre-execute", leading to inconsistent sequence ordering. The sequence expressions are now fully inline. Fixes: #6361 Change-Id: Ie16794ec0e19979a7e6c8d1bef5716a9fc199889
* | dont assume ClauseElement in attributes, coercionsMike Bayer2021-04-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed two distinct issues, each of which would come into play under certain circumstances, most likely however one which is a common mis-configuration in :class:`_hybrid.hybrid_property`, where the "expression" implementation would return a non :class:`_sql.ClauseElement` such as a boolean value. For both issues, 1.3's behavior was to silently ignore the mis-configuration and ultimately attempt to interpret the value as a SQL expression, which would lead to an incorrect query. * Fixed issue regarding interaction of the attribute system with hybrid_property, where if the ``__clause_element__()`` method of the attribute returned a non-:class:`_sql.ClauseElement` object, an internal ``AttributeError`` would lead the attribute to return the ``expression`` function on the hybrid_property itself, as the attribute error was against the name ``.expression`` which would invoke the ``__getattr__()`` method as a fallback. This now raises explicitly. In 1.3 the non-:class:`_sql.ClauseElement` was returned directly. * Fixed issue in SQL argument coercions system where passing the wrong kind of object to methods that expect column expressions would fail if the object were altogether not a SQLAlchemy object, such as a Python function, in cases where the object were not just coerced into a bound value. Again 1.3 did not have a comprehensive argument coercion system so this case would also pass silently. Fixes: #6350 Change-Id: I5bba0a6b27f45e5f8ebadfd6d511fa773388ef7c
* | omit text from selected_columns; clear memoizationsMike Bayer2021-04-222-18/+88
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed regression where usage of the :func:`_sql.text` construct inside the columns clause of a :class:`_sql.Select` construct, which is better handled by using a :func:`_sql.literal_column` construct, would nonetheless prevent constructs like :func:`_sql.union` from working correctly. Other use cases, such as constructing subuqeries, continue to work the same as in prior versions where the :func:`_sql.text` construct is silently omitted from the collection of exported columns. Also repairs similar use within the ORM. This adds a new internal method _all_selected_columns. The existing "selected_columns" collection can't store a TextClause and this never worked, so they are omitted. The TextClause is also not "exported", i.e. available for SELECT from a subquery, as was already the case in 1.3, so the "exported_columns" and "exported_columns_iterator" accessors are where we now omit TextClause. Fixed regression involving legacy methods such as :meth:`_sql.Select.append_column` where internal assertions would fail. Fixes: #6343 Fixes: #6261 Change-Id: I7c2e5b9ae5d94131c77599a020f4310dcf812bcf
* | Propagate compiler kw for visit_values to parametersMike Bayer2021-04-201-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | Fixed issue in SQL compiler where the bound parameters set up for a :class:`.Values` construct wouldn't be positionally tracked correctly if inside of a :class:`_sql.CTE`, affecting database drivers that support VALUES + ctes and use positional parameters such as SQL Server in particular as well as asyncpg. The fix also repairs support for compiler flags such as ``literal_binds``. Fixes: #6327 Change-Id: I2d549228691d0bfc10dadd0955b1549d7584db51
* | Merge "Derive `next_value.type` from `Sequence.data_type` if available"mike bayer2021-04-201-0/+3
|\ \
| * | Derive `next_value.type` from `Sequence.data_type` if availableBryan Forbes2021-04-181-0/+3
| | | | | | | | | | | | | | | | | | Fixes #6287 Change-Id: I7d428ed86cd72cd910bfff9058a52c7fcb7c64ac
* | | add some cross-linking for orderby/groupby docs + label examplesMike Bayer2021-04-191-2/+7
| | | | | | | | | | | | | | | Change-Id: I34d4958ded8ec95e439ee47fdcb600f357cf7ae3 References: #6324
* | | Merge "Don't stringify unnamed column elements when proxying"mike bayer2021-04-173-29/+51
|\ \ \
| * | | Don't stringify unnamed column elements when proxyingMike Bayer2021-04-173-29/+51
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Repaired and solidified issues regarding custom functions and other arbitrary expression constructs which within SQLAlchemy's column labeling mechanics would seek to use ``str(obj)`` to get a string representation to use as an anonymous column name in the ``.c`` collection of a subquery. This is a very legacy behavior that performs poorly and leads to lots of issues, so has been revised to no longer perform any compilation by establishing specific methods on :class:`.FunctionElement` to handle this case, as SQL functions are the only use case that it came into play. An effect of this behavior is that an unlabeled column expression with no derivable name will be given an arbitrary label starting with the prefix ``"_no_label"`` in the ``.c`` collection of a subquery; these were previously being represented either as the generic stringification of that expression, or as an internal symbol. This change seeks to make the concept of "anon name" more private and renames anon_label and anon_key_label to _anon_name_label and _anon_key_label. There's no end-user utility to these accessors and we need to be able to reorganize these as well. Fixes: #6256 Change-Id: Ie63c86b20ca45873affea78500388da94cf8bf94
View Lorry Controller Status