summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/sql/expression.py
Commit message (Collapse)AuthorAgeFilesLines
* Add future=True to create_engine/Session; unify select()Mike Bayer2020-07-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Several weeks of using the future_select() construct has led to the proposal there be just one select() construct again which features the new join() method, and otherwise accepts both the 1.x and 2.x argument styles. This would make migration simpler and reduce confusion. However, confusion may be increased by the fact that select().join() is different Current thinking is we may be better off with a few hard behavioral changes to old and relatively unknown APIs rather than trying to play both sides within two extremely similar but subtly different APIs. At the moment, the .join() thing seems to be the only behavioral change that occurs without the user taking any explicit steps. Session.execute() will still behave the old way as we are adding a future flag. This change also adds the "future" flag to Session() and session.execute(), so that interpretation of the incoming statement, as well as that the new style result is returned, does not occur for existing applications unless they add the use of this flag. The change in general is moving the "removed in 2.0" system further along where we want the test suite to fully pass even if the SQLALCHEMY_WARN_20 flag is set. Get many tests to pass when SQLALCHEMY_WARN_20 is set; this should be ongoing after this patch merges. Improve the RemovedIn20 warning; these are all deprecated "since" 1.4, so ensure that's what the messages read. Make sure the inforamtion link is on all warnings. Add deprecation warnings for parameters present and add warnings to all FromClause.select() types of methods. Fixes: #5379 Fixes: #5284 Change-Id: I765a0b912b3dcd0e995426427d8bb7997cbffd51 References: #5159
* introduce deferred lambdasMike Bayer2020-07-031-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The coercions system allows us to add in lambdas as arguments to Core and ORM elements without changing them at all. By allowing the lambda to produce a deterministic cache key where we can also cheat and yank out literal parameters means we can move towards having 90% of "baked" functionality in a clearer way right in Core / ORM. As a second step, we can have whole statements inside the lambda, and can then add generation with __add__(), so then we have 100% of "baked" functionality with full support of ad-hoc literal values. Adds some more short_selects tests for the moment for comparison. Other tweaks inside cache key generation as we're trying to approach a certain level of performance such that we can remove the use of "baked" from the loader strategies. As we have not yet closed #4639, however the caching feature has been fully integrated as of b0cfa7379cf8513a821a3dbe3028c4965d9f85bd, we will also add complete caching documentation here and close that issue as well. Closes: #4639 Fixes: #5380 Change-Id: If91f61527236fd4d7ae3cad1f24c38be921c90ba
* Fix almost all read-level sphinx warningsMike Bayer2020-04-111-0/+2
| | | | | | | | There are some related to changelog that I can't figure out and are likely due to something in the changelog extension. also one thing with a "collection" I can't figure out. Change-Id: I0a9e6f4291c3589aa19a4abcb9245cd22a266fe0
* Implement SQL VALUES in core.Gord Thompson2020-03-241-0/+4
| | | | | | | | | | | | | Added a core :class:`Values` object that enables a VALUES construct to be used in the FROM clause of an SQL statement for databases that support it (mainly PostgreSQL and SQL Server). Fixes: #4868 Closes: #5030 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5030 Pull-request-sha: 84684038a8efa93b460318e0db53f6c644554588 Change-Id: Ib8109b63bc1a9dc04ab987c5322ca3375f7e824d
* Rework select(), CompoundSelect() in terms of CompileStateMike Bayer2020-03-101-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Continuation of I408e0b8be91fddd77cf279da97f55020871f75a9 - add an options() method to the base Generative construct. this will be where ORM options can go - Change Null, False_, True_ to be singletons, so that we aren't instantiating them and having to use isinstance. The previous issue with this was that they would produce dupe labels in SELECT statements. Apply the duplicate column logic, newly added in 1.4, to these objects as well as to non-apply-labels SELECT statements in general as a means of improving this. - create a revised system for generating ClauseList compilation constructs that simplfies up front creation to not actually use ClauseList; a simple tuple is rendered by the compiler using the same constrcution rules as what are used for ClauseList but without creating the actual object. Apply to Select, CompoundSelect, revise Update, Delete - Select, CompoundSelect get an initial CompileState implementation. All methods used only within compilation are moved here - refine update/insert/delete compile state to not require an outside boolean - refine and simplify Select._copy_internals - rework bind(), which is going away, to not use some of the internal traversal stuff - remove "autocommit", "for_update" parameters from Select, references #4643 - remove "autocommit" parameter from TextClause , references #4643 - add deprecation warnings for statement.execute(), engine.execute(), statement.scalar(), engine.scalar(). Fixes: #5193 Change-Id: I04ca0152b046fd42c5054ba10f37e43fc6e5a57b
* Fixes for public_factory and mysql/pg dml functionsMike Bayer2020-02-081-42/+46
| | | | | | | | | | | | | | | | * ensure that the location indicated by public_factory is importable * adjust all of sqlalchemy.sql.expression locations to be correct * support the case where a public_factory is against a function that has another public_factory already, and already replaced the __init__ on the target class * Use mysql.insert(), postgresql.insert(), don't include .dml in the class path. Change-Id: Iac285289455d8d7102349df3814f7cedc758e639
* happy new yearMike Bayer2020-01-011-1/+1
| | | | Change-Id: I08440dc25e40ea1ccea1778f6ee9e28a00808235
* Add anonymizing context to cache keys, comparison; convert traversalMike Bayer2019-11-041-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Created new visitor system called "internal traversal" that applies a data driven approach to the concept of a class that defines its own traversal steps, in contrast to the existing style of traversal now known as "external traversal" where the visitor class defines the traversal, i.e. the SQLCompiler. The internal traversal system now implements get_children(), _copy_internals(), compare() and _cache_key() for most Core elements. Core elements with special needs like Select still implement some of these methods directly however most of these methods are no longer explicitly implemented. The data-driven system is also applied to ORM elements that take part in SQL expressions so that these objects, like mappers, aliasedclass, query options, etc. can all participate in the cache key process. Still not considered is that this approach to defining traversibility will be used to create some kind of generic introspection system that works across Core / ORM. It's also not clear if real statement caching using the _cache_key() method is feasible, if it is shown that running _cache_key() is nearly as expensive as compiling in any case. Because it is data driven, it is more straightforward to optimize using inlined code, as is the case now, as well as potentially using C code to speed it up. In addition, the caching sytem now accommodates for anonymous name labels, which is essential so that constructs which have anonymous labels can be cacheable, that is, their position within a statement in relation to other anonymous names causes them to generate an integer counter relative to that construct which will be the same every time. Gathering of bound parameters from any cache key generation is also now required as there is no use case for a cache key that does not extract bound parameter values. Applies-to: #4639 Change-Id: I0660584def8627cad566719ee98d3be045db4b8d
* SelectBase no longer a FromClauseMike Bayer2019-07-061-0/+5
| | | | | | | | | | | | | | | | | | | | As part of the SQLAlchemy 2.0 migration project, a conceptual change has been made to the role of the :class:`.SelectBase` class hierarchy, which is the root of all "SELECT" statement constructs, in that they no longer serve directly as FROM clauses, that is, they no longer subclass :class:`.FromClause`. For end users, the change mostly means that any placement of a :func:`.select` construct in the FROM clause of another :func:`.select` requires first that it be wrapped in a subquery first, which historically is through the use of the :meth:`.SelectBase.alias` method, and is now also available through the use of :meth:`.SelectBase.subquery`. This was usually a requirement in any case since several databases don't accept unnamed SELECT subqueries in their FROM clause in any case. See the documentation in this change for lots more detail. Fixes: #4617 Change-Id: I0f6174ee24b9a1a4529168e52e855e12abd60667
* Implement new ClauseElement role and coercion systemMike Bayer2019-05-181-18/+1
| | | | | | | | | | | | | | | | | | | | A major refactoring of all the functions handle all detection of Core argument types as well as perform coercions into a new class hierarchy based on "roles", each of which identify a syntactical location within a SQL statement. In contrast to the ClauseElement hierarchy that identifies "what" each object is syntactically, the SQLRole hierarchy identifies the "where does it go" of each object syntactically. From this we define a consistent type checking and coercion system that establishes well defined behviors. This is a breakout of the patch that is reorganizing select() constructs to no longer be in the FromClause hierarchy. Also includes a rename of as_scalar() into scalar_subquery(); deprecates automatic coercion to scalar_subquery(). Partially-fixes: #4617 Change-Id: I26f1e78898693c6b99ef7ea2f4e7dfd0e8e1a1bd
* Prevent __init__ from being called for Alias, subclassesMike Bayer2019-02-211-3/+5
| | | | | | | | | | | The :class:`.Alias` class and related subclasses :class:`.CTE`, :class:`.Lateral` and :class:`.TableSample` have been reworked so that it is not possible for a user to construct the objects directly. These constructs require that the standalone construction function or selectable-bound method be used to instantiate new objects. Fixes: #4509 Change-Id: I74ae4786cb3ae625dab33b00bfd6bdc4e1219139
* Remove all remaining text() coercions and ensure identifiers are safeMike Bayer2019-02-061-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fully removed the behavior of strings passed directly as components of a :func:`.select` or :class:`.Query` object being coerced to :func:`.text` constructs automatically; the warning that has been emitted is now an ArgumentError or in the case of order_by() / group_by() a CompileError. This has emitted a warning since version 1.0 however its presence continues to create concerns for the potential of mis-use of this behavior. Note that public CVEs have been posted for order_by() / group_by() which are resolved by this commit: CVE-2019-7164 CVE-2019-7548 Added "SQL phrase validation" to key DDL phrases that are accepted as plain strings, including :paramref:`.ForeignKeyConstraint.on_delete`, :paramref:`.ForeignKeyConstraint.on_update`, :paramref:`.ExcludeConstraint.using`, :paramref:`.ForeignKeyConstraint.initially`, for areas where a series of SQL keywords only are expected.Any non-space characters that suggest the phrase would need to be quoted will raise a :class:`.CompileError`. This change is related to the series of changes committed as part of :ticket:`4481`. Fixed issue where using an uppercase name for an index type (e.g. GIST, BTREE, etc. ) or an EXCLUDE constraint would treat it as an identifier to be quoted, rather than rendering it as is. The new behavior converts these types to lowercase and ensures they contain only valid SQL characters. Quoting is applied to :class:`.Function` names, those which are usually but not necessarily generated from the :attr:`.sql.func` construct, at compile time if they contain illegal characters, such as spaces or punctuation. The names are as before treated as case insensitive however, meaning if the names contain uppercase or mixed case characters, that alone does not trigger quoting. The case insensitivity is currently maintained for backwards compatibility. Fixes: #4481 Fixes: #4473 Fixes: #4467 Change-Id: Ib22a27d62930e24702e2f0f7c74a0473385a08eb
* happy new yearMike Bayer2019-01-111-1/+1
| | | | Change-Id: I6a71f4924d046cf306961c58dffccf21e9c03911
* Post black reformattingMike Bayer2019-01-061-93/+91
| | | | | | | | | | | | | Applied on top of a pure run of black -l 79 in I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes resolves all remaining flake8 conditions for those codes we have enabled in setup.cfg. Included are resolutions for all remaining flake8 issues including shadowed builtins, long lines, import order, unused imports, duplicate imports, and docstring issues. Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
* Run black -l 79 against all source filesMike Bayer2019-01-061-44/+161
| | | | | | | | | | | | | | This is a straight reformat run using black as is, with no edits applied at all. The black run will format code consistently, however in some cases that are prevalent in SQLAlchemy code it produces too-long lines. The too-long lines will be resolved in the following commit that will resolve all remaining flake8 issues including shadowed builtins, long lines, import order, unused imports, duplicate imports, and docstring issues. Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
* happy new yearMike Bayer2018-01-121-1/+1
| | | | Change-Id: I3ef36bfd0cb0ba62b3123c8cf92370a43156cf8f
* Recognize brackets, quoted_name in SQL Server schemaMike Bayer2017-04-041-2/+3
| | | | | | | | | | | | | The SQL Server dialect now allows for a database and/or owner name with a dot inside of it, using brackets explicitly in the string around the owner and optionally the database name as well. In addition, sending the :class:`.quoted_name` construct for the schema name will not split on the dot and will deliver the full string as the "owner". :class:`.quoted_name` is also now available from the ``sqlalchemy.sql`` import space. Change-Id: I77491d63ce47638bd23787d903ccde2f35a9d43d Fixes: #2626
* update for 2017 copyrightMike Bayer2017-01-041-1/+1
| | | | Change-Id: I4e8c2aa8fe817bb2af8707410fa0201f938781de
* Correct any_, all_ spellingMike Bayer2016-12-281-1/+1
| | | | | | | | | - Fixed 1.1 regression where "import *" would not work for sqlalchemy.sql.expression, due to mis-spelled "any_" and "all_" functions. Change-Id: I25d1cd34c9239dbdcdb1889c5cda2474557e1418 Fixes: #3878
* Add TABLESAMPLE clause support.saarni2016-06-151-2/+3
| | | | | | | | | | The TABLESAMPLE clause allows randomly selecting an approximate percentage of rows from a table. At least DB2, Microsoft SQL Server and recent Postgresql support this standard clause. Fixes: #3718 Change-Id: I3fb8b9223e12a57100df30876b461884c58d72fa Pull-request: https://github.com/zzzeek/sqlalchemy/pull/277
* - Added :meth:`.Select.lateral` and related constructs to allowMike Bayer2016-03-291-4/+6
| | | | | for the SQL standard LATERAL keyword, currently only supported by Postgresql. fixes #2857
* - CTE functionality has been expanded to support all DML, allowingMike Bayer2016-02-111-2/+2
| | | | | | | INSERT, UPDATE, and DELETE statements to both specify their own WITH clause, as well as for these statements themselves to be CTE expressions when they include a RETURNING clause. fixes #2551
* - happy new yearMike Bayer2016-01-291-1/+1
|
* - The :func:`.type_coerce` construct is now a fully fledged CoreMike Bayer2015-09-161-1/+2
| | | | | | | | | | | expression element which is late-evaluated at compile time. Previously, the function was only a conversion function which would handle different expression inputs by returning either a :class:`.Label` of a column-oriented expression or a copy of a given :class:`.BindParameter` object, which in particular prevented the operation from being logically maintained when an ORM-level expression transformation would convert a column to a bound parameter (e.g. for lazy loading). fixes #3531
* - Added support for "set-aggregate" functions of the formticket_3516Mike Bayer2015-08-261-2/+3
| | | | | | | | | | | ``<function> WITHIN GROUP (ORDER BY <criteria>)``, using the method :class:`.FunctionElement.within_group`. A series of common set-aggregate functions with return types derived from the set have been added. This includes functions like :class:`.percentile_cont`, :class:`.dense_rank` and others. fixes #1370 - make sure we use func.name for all _literal_as_binds in functions.py so we get consistent naming behavior for parameters.
* - build out a new base type for Array, as well as new any/all operatorsMike Bayer2015-08-251-2/+4
| | | | | | - any/all work for Array as well as subqueries, accepted by MySQL - Postgresql ARRAY now subclasses Array - fixes #3516
* - copyright 2015Mike Bayer2015-03-101-1/+1
|
* - correctionsMike Bayer2014-12-271-1/+1
| | | | - attempt to add a script to semi-automate the fixing of links
* - Reversing a change that was made in 0.9, the "singleton" natureMike Bayer2014-10-191-3/+3
| | | | | | | | | of the "constants" :func:`.null`, :func:`.true`, and :func:`.false` has been reverted. These functions returning a "singleton" object had the effect that different instances would be treated as the same regardless of lexical use, which in particular would impact the rendering of the columns clause of a SELECT statement. fixes #3170
* renamed aggregatefilter to funcfilter, since it is thatIlja Everilä2014-09-111-3/+3
|
* implementation for <aggregate_fun> FILTER (WHERE ...)Ilja Everilä2014-09-101-1/+3
|
* - The :func:`~.expression.column` and :func:`~.expression.table`Mike Bayer2014-09-011-1/+2
| | | | | | | | | | | | | | | | | | | | | constructs are now importable from the "from sqlalchemy" namespace, just like every other Core construct. - The implicit conversion of strings to :func:`.text` constructs when passed to most builder methods of :func:`.select` as well as :class:`.Query` now emits a warning with just the plain string sent. The textual conversion still proceeds normally, however. The only method that accepts a string without a warning are the "label reference" methods like order_by(), group_by(); these functions will now at compile time attempt to resolve a single string argument to a column or label expression present in the selectable; if none is located, the expression still renders, but you get the warning again. The rationale here is that the implicit conversion from string to text is more unexpected than not these days, and it is better that the user send more direction to the Core / ORM when passing a raw string as to what direction should be taken. Core/ORM tutorials have been updated to go more in depth as to how text is handled. fixes #2992
* - apply pep8 formatting to sqlalchemy/sql, sqlalchemy/util, sqlalchemy/dialects,Brian Jarrett2014-07-201-20/+26
| | | | sqlalchemy/orm, sqlalchemy/event, sqlalchemy/testing
* - break up the <authors> copyright comment as part of a passMike Bayer2014-07-091-1/+2
| | | | to get all flake8 passing
* - Restored the import for :class:`.Function` to the ↵Mike Bayer2014-04-051-1/+1
| | | | | | ``sqlalchemy.sql.expression`` import namespace, which was removed at the beginning of 0.9.
* - happy new yearMike Bayer2014-01-051-1/+1
|
* make generativeselect availableMike Bayer2013-11-291-1/+1
|
* - New improvements to the :func:`.text` construct, includingMike Bayer2013-11-291-2/+2
| | | | | | | | more flexible ways to set up bound parameters and return types; in particular, a :func:`.text` can now be turned into a full FROM-object, embeddable in other statements as an alias or CTE using the new method :meth:`.TextClause.columns`. [ticket:2877]
* Import CTE so that API doc can be generatedVraj Mohan2013-11-141-1/+1
|
* An overhaul of expression handling for special symbols particularlyMike Bayer2013-10-231-5/+6
| | | | | | | | | | with conjunctions, e.g. ``None`` :func:`.expression.null` :func:`.expression.true` :func:`.expression.false`, including consistency in rendering NULL in conjunctions, "short-circuiting" of :func:`.and_` and :func:`.or_` expressions which contain boolean constants, and rendering of boolean constants and expressions as compared to "1" or "0" for backends that don't feature ``true``/``false`` constants. [ticket:2804]
* - The ``version_id_generator`` parameter of ``Mapper`` can now be specifiedMike Bayer2013-08-251-1/+2
| | | | | | | | | | | | | | | | | | | | | to rely upon server generated version identifiers, using triggers or other database-provided versioning features, by passing the value ``False``. The ORM will use RETURNING when available to immediately load the new version identifier, else it will emit a second SELECT. [ticket:2793] - The ``eager_defaults`` flag of :class:`.Mapper` will now allow the newly generated default values to be fetched using an inline RETURNING clause, rather than a second SELECT statement, for backends that support RETURNING. - Added a new variant to :meth:`.ValuesBase.returning` called :meth:`.ValuesBase.return_defaults`; this allows arbitrary columns to be added to the RETURNING clause of the statement without interfering with the compilers usual "implicit returning" feature, which is used to efficiently fetch newly generated primary key values. For supporting backends, a dictionary of all fetched values is present at :attr:`.ResultProxy.returned_defaults`. - add a glossary entry for RETURNING - add documentation for version id generation, [ticket:867]
* - apply an import refactoring to the ORM as wellMike Bayer2013-08-141-32/+31
| | | | | | | | | - rework the event system so that event modules load after their targets, dependencies are reversed - create an improved strategy lookup system for the ORM - rework the ORM to have very few import cycles - move out "importlater" to just util.dependency - other tricks to cross-populate modules in as clear a way as possible
* fix missing savepoint structuresMike Bayer2013-08-121-0/+3
|
* - A large refactoring of the ``sqlalchemy.sql`` package has reorganizedMike Bayer2013-08-121-6606/+79
| | | | | | | | | | | | | | | | | | | | | | the import structure of many core modules. ``sqlalchemy.schema`` and ``sqlalchemy.types`` remain in the top-level package, but are now just lists of names that pull from within ``sqlalchemy.sql``. Their implementations are now broken out among ``sqlalchemy.sql.type_api``, ``sqlalchemy.sql.sqltypes``, ``sqlalchemy.sql.schema`` and ``sqlalchemy.sql.ddl``, the last of which was moved from ``sqlalchemy.engine``. ``sqlalchemy.sql.expression`` is also a namespace now which pulls implementations mostly from ``sqlalchemy.sql.elements``, ``sqlalchemy.sql.selectable``, and ``sqlalchemy.sql.dml``. Most of the "factory" functions used to create SQL expression objects have been moved to classmethods or constructors, which are exposed in ``sqlalchemy.sql.expression`` using a programmatic system. Care has been taken such that all the original import namespaces remain intact and there should be no impact on any existing applications. The rationale here was to break out these very large modules into smaller ones, provide more manageable lists of function names, to greatly reduce "import cycles" and clarify the up-front importing of names, and to remove the need for redundant functions and documentation throughout the expression package.
* - The :meth:`.Operators.notin_` operator added in 0.8 now properlyMike Bayer2013-08-071-1/+4
| | | | | produces the negation of the expression "IN" returns when used against an empty collection. Also in 0.8.3.
* - Fixed bug in common table expression system where if the CTE wereMike Bayer2013-07-311-4/+4
| | | | | | used only as an ``alias()`` construct, it would not render using the WITH keyword. Also in 0.8.3, 0.7.11. [ticket:2783]
* - use inline=True for the insert..select here so it works on oracleMike Bayer2013-07-121-0/+11
|
* Fixed bug where the expression system relied upon the ``str()``Mike Bayer2013-07-121-1/+4
| | | | | | | | | | | | form of a some expressions when referring to the ``.c`` collection on a ``select()`` construct, but the ``str()`` form isn't available since the element relies on dialect-specific compilation constructs, notably the ``__getitem__()`` operator as used with a Postgresql ``ARRAY`` element. The fix also adds a new exception class :class:`.UnsupportedCompilationError` which is raised in those cases where a compiler is asked to compile something it doesn't know how to. Also in 0.8.3. [ticket:2780]
* - Added new method to the :func:`.insert` constructMike Bayer2013-07-051-2/+49
| | | | | | | | | | :meth:`.Insert.from_select`. Given a list of columns and a selectable, renders ``INSERT INTO (table) (columns) SELECT ..``. While this feature is highlighted as part of 0.9 it is also backported to 0.8.3. [ticket:722] - The :func:`.update`, :func:`.insert`, and :func:`.delete` constructs will now interpret ORM entities as FROM clauses to be operated upon, in the same way that select() already does. Also in 0.8.3.
* - rework of correlation, continuing on #2668, #2746Mike Bayer2013-06-261-43/+111
| | | | | | | | | | | | | | | | | | | | | | | | - add support for correlations to propagate all the way in; because correlations require context now, need to make sure a select enclosure of any level takes effect any number of levels deep. - fix what we said correlate_except() was supposed to do when we first released #2668 - "the FROM clause is left intact if the correlated SELECT is not used in the context of an enclosing SELECT..." - it was not considering the "existing_froms" collection at all, and prohibited additional FROMs from being placed in an any() or has(). - add test for multilevel any() - lots of docs, including glossary entries as we really need to define "WHERE clause", "columns clause" etc. so that we can explain correlation better - based on the insight that a SELECT can correlate anything that ultimately came from an enclosing SELECT that links to this one via WHERE/columns/HAVING/ORDER BY, have the compiler keep track of the FROM lists that correspond in this way, link it to the asfrom flag, so that we send to _get_display_froms() the exact list of candidate FROMs to correlate. no longer need any asfrom logic in the Select() itself - preserve 0.8.1's behavior for correlation when no correlate options are given, not to mention 0.7 and prior's behavior of not propagating implicit correlation more than one level.. this is to reduce surprises/hard-to-debug situations when a user isn't trying to correlate anything.
View Lorry Controller Status