summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/sql/elements.py
Commit message (Collapse)AuthorAgeFilesLines
* Update bindparam cache keyFederico Caselli2022-03-301-0/+2
| | | | | | | | | | | The ``literal_execute`` parameter now takes part of the cache generation of a bindparam, since it changes the sql string generated by the compiler. Previously the correct bind values were used, but the ``literal_execute`` would be ignored on subsequent executions of the same query. Fixes: #7876 Change-Id: I6bf887f1a2fe31f9d0ab68f5b4ff315004d006b2
* pep-484: the pep-484ening, SQL part threeMike Bayer2022-03-301-145/+199
| | | | | | | | | | | | | | | hitting DML which is causing us to open up the ColumnCollection structure a bit, as we do put anonymous column expressions with None here. However, we still want Table /TableClause to have named column collections that don't return None, so parametrize the "key" in this collection also. * rename some "immutable" elements to "readonly". we change the contents of immutablecolumncollection underneath, so it's not "immutable" Change-Id: I2593995a4e5c6eae874bed5bf76117198be8ae97
* pep484 - SQL internalsMike Bayer2022-03-241-9/+12
| | | | | | | non-strict checking for mostly internal or semi-internal code Change-Id: Ib91b47f1a8ccc15e666b94bad1ce78c4ab15b0ec
* pep 484 for typesMike Bayer2022-03-191-37/+82
| | | | | | | strict types type_api.py, including TypeDecorator, NativeForEmulated, etc. Change-Id: Ib2eba26de0981324a83733954cb7044a29bbd7db
* Merge "remove intermediary _is_clone_of entries when cloning" into mainmike bayer2022-03-171-1/+2
|\
| * remove intermediary _is_clone_of entries when cloningMike Bayer2022-03-171-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Improvements in memory usage by the ORM, removing a significant set of intermediary expression objects that are typically stored when a copy of an expression object is created. These clones have been greatly reduced, reducing the number of total expression objects stored in memory by ORM mappings by about 30%. note this change causes the tests to have a bit of a harder time with GC, which we would assume is because mappings now have a lot more garbage to clean up after mappers are configured. it remains to be seen what the long term effects of this are. Fixes: #7823 Change-Id: If8729747ffb9bf27e8974f069a994b5a823ee095
* | pep484 for hybridMike Bayer2022-03-171-3/+8
|/ | | | | Change-Id: I53274b13094d996e11b04acb03f9613edbddf87f References: #6810
* pep-484 - SQL column operationsMike Bayer2022-03-151-491/+793
| | | | | | | | | note we are taking out the ColumnOperartors[SQLCoreOperations] thing; not really clear why that was needed and at the moment it seems I was likely confused. Change-Id: I834b75f9b44f91b97e29f2e1a7b1029bd910e0a1
* pep-484: sqlalchemy.sql pass oneMike Bayer2022-03-131-9/+25
| | | | | | | | | | | | | | | | | | sqlalchemy.sql will require many passes to get all modules even gradually typed. Will have to pick and choose what modules can be strictly typed vs. which can be gradual. in this patch, emphasis is on visitors.py, cache_key.py, annotations.py for strict typing, compiler.py is on gradual typing but has much more structure, in particular where it connects with the outside world. The work within compiler.py also reached back out to engine/cursor.py , default.py quite a bit. References: #6810 Change-Id: I6e8a29f6013fd216e43d45091bc193f8be0368fd
* additional mypy strictnessMike Bayer2022-03-121-6/+26
| | | | | | | | | | | | | | | | | | enable type checking within untyped defs. This allowed some more internals to be fixed up with assertions etc. some internals that were unnecessary or not even used at all were removed. BaseCursorResult was no longer necessary since we only have one kind of CursorResult now. The different ResultProxy subclasses that had alternate "strategies" dont appear to be used at all even in 1.4.x, as there's no code that accesses the _cursor_strategy_cls attribute, which is also removed. As these were mostly private constructs that weren't even functioning correctly in any case, it's fine to remove these over the 2.0 boundary. Change-Id: Ifd536987d104b1cd8b546cefdbd5c1e5d1801082
* pep484 + abc bases for assocaitionproxyMike Bayer2022-03-011-61/+71
| | | | | | | | | | went to this one next as it was going to be hard, and also exercises the ORM expression hierarchy a bit. made some adjustments to SQLCoreOperations etc. Change-Id: Ie5dde9218dc1318252826b766d3e70b17dd24ea7 References: #6810 References: #7774
* pep-484 for engineMike Bayer2022-03-011-1/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | All modules in sqlalchemy.engine are strictly typed with the exception of cursor, default, and reflection. cursor and default pass with non-strict typing, reflection is waiting on the multi-reflection refactor. Behavioral changes: * create_connect_args() methods return a tuple of list, dict, rather than a list of list, dict * removed allow_chars parameter from pyodbc connector ._get_server_version_info() method * the parameter list passed to do_executemany is now a list in all cases. previously, this was being run through dialect.execute_sequence_format, which defaults to tuple and was only intended for individual tuple params. * broke up dialect.dbapi into dialect.import_dbapi class method and dialect.dbapi module object. added a deprecation path for legacy dialects. it's not really feasible to type a single attr as a classmethod vs. module type. The "type_compiler" attribute also has this problem with greater ability to work around, left that one for now. * lots of constants changing to be Enum, so that we can type them. for fixed tuple-position constants in cursor.py / compiler.py (which are used to avoid the speed overhead of namedtuple), using Literal[value] which seems to work well * some tightening up in Row regarding __getitem__, which we can do since we are on full 2.0 style result use * altered the set_connection_execution_options and set_engine_execution_options event flows so that the dictionary of options may be mutated within the event hook, where it will then take effect as the actual options used. Previously, changing the dict would be silently ignored which seems counter-intuitive and not very useful. * A lot of DefaultDialect/DefaultExecutionContext methods and attributes, including underscored ones, move to interfaces. This is not fully ideal as it means the Dialect/ExecutionContext interfaces aren't publicly subclassable directly, but their current purpose is more of documentation for dialect authors who should (and certainly are) still be subclassing the DefaultXYZ versions in all cases Overall, Result was the most extremely difficult class hierarchy to type here as this hierarchy passes through largely amorphous "row" datatypes throughout, which can in fact by all kinds of different things, like raw DBAPI rows, or Row objects, or "scalar"/Any, but at the same time these types have meaning so I tried still maintaining some level of semantic markings for these, it highlights how complex Result is now, as it's trying to be extremely efficient and inlined while also being very open-ended and extensible. Change-Id: I98b75c0c09eab5355fc7a33ba41dd9874274f12a
* pep-484 for sqlalchemy.event; use future annotationsMike Bayer2022-02-151-0/+2
| | | | | | | | | | __future__.annotations mode allows us to use non-string annotations for argument and return types in most cases, but more importantly it removes a large amount of runtime overhead that would be spent in evaluating the annotations. Change-Id: I2f5b6126fe0019713fc50001be3627b664019ede References: #6810
* Add new infrastructure to support greater use of __slots__Mike Bayer2022-01-211-2/+2
| | | | | | | | | | | | | | | | | | | * Changed AliasedInsp to use __slots__ * Migrated all of strategy_options to use __slots__ for objects. Adds new infrastructure to traversals to support shallow copy, to dict and from dict based on internal traversal attributes. Load / _LoadElement then leverage this to provide clone / generative / getstate without the need for __dict__ or explicit attribute lists. Doing this change revealed that there are lots of things that trigger off of whether or not a class has a __visit_name__ attribute. so to suit that we've gone back to having Visitable, which is a better name than Traversible at this point (I think Traversible is mis-spelled too). Change-Id: I13d04e494339fac9dbda0b8e78153418abebaf72 References: #7527
* Merge "track item schema names to identify name collisions w/ default ↵mike bayer2022-01-141-0/+1
|\ | | | | | | schema" into main
| * track item schema names to identify name collisions w/ default schemaMike Bayer2022-01-141-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added an additional lookup step to the compiler which will track all FROM clauses which are tables, that may have the same name shared in multiple schemas where one of the schemas is the implicit "default" schema; in this case, the table name when referring to that name without a schema qualification will be rendered with an anonymous alias name at the compiler level in order to disambiguate the two (or more) names. The approach of schema-qualifying the normally unqualified name with the server-detected "default schema name" value was also considered, however this approach doesn't apply to Oracle nor is it accepted by SQL Server, nor would it work with multiple entries in the PostgreSQL search path. The name collision issue resolved here has been identified as affecting at least Oracle, PostgreSQL, SQL Server, MySQL and MariaDB. Fixes: #7471 Change-Id: Id65e7ca8c43fe8d95777084e8d5ec140ebcd784d
* | Initial ORM typing layoutMike Bayer2022-01-141-271/+288
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | introduces: 1. new mapped_column() helper 2. DeclarativeBase helper 3. declared_attr has been re-typed 4. rework of Mapped[] to return InstrumentedAtribute for class get, so works without Mapped itself having expression methods 5. ORM constructs now generic on [_T] also includes some early typing work, most of which will be in later commits: 1. URL and History become typing.NamedTuple 2. come up with type-checking friendly way of type checking cy extensions, where type checking will be applied to the py versions, just needed to come up with a succinct conditional pattern for the imports References: #6810 References: #7535 References: #7562 Change-Id: Ie5d9a44631626c021d130ca4ce395aba623c71fb
* | initial reorganize for static typingMike Bayer2022-01-121-1490/+488
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | start applying foundational annotations to key elements. two main elements addressed here: 1. removal of public_factory() and replacement with explicit functions. this just works much better with typing. 2. typing support for column expressions and operators. The biggest part of this involves stubbing out all the ColumnOperators methods under ColumnElement in a TYPE_CHECKING section. Took me a while to see this method vs. much more complicated things I thought I needed. Also for this version implementing #7519, ColumnElement types against the Python type and not TypeEngine. it is hoped this leads to easier transferrence between ORM/Core as well as eventual support for result set typing. Not clear yet how well this approach will work and what new issues it may introduce. given the current approach we now get full, rich typing for scenarios like this: from sqlalchemy import column, Integer, String, Boolean c1 = column('a', String) c2 = column('a', Integer) expr1 = c2.in_([1, 2, 3]) expr2 = c2 / 5 expr3 = -c2 expr4_a = ~(c2 == 5) expr4_b = ~column('q', Boolean) expr5 = c1 + 'x' expr6 = c2 + 10 Fixes: #7519 Fixes: #6810 Change-Id: I078d9f57955549f6f7868314287175f6c61c44cb
* remove internal use of metaclassesMike Bayer2022-01-111-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | All but one metaclass used internally can now be replaced using __init_subclass__(). Within this patch we remove: * events._EventMeta * sql.visitors.TraversibleType * sql.visitors.InternalTraversibleType * testing.fixtures.FindFixture * testing.fixtures.FindFixtureDeclarative * langhelpers.EnsureKWArgType * sql.functions._GenericMeta * sql.type_api.VisitableCheckKWArg (was a mixture of TraversibleType and EnsureKWArgType) The remaining internal class is MetaOptions used by the sql.Options object which is in turn currently mostly for ORM internal use, as this type implements class level overrides for the ``+`` operator. For declarative, removing DeclarativeMeta in place of an `__init_subclass__()` class would not be fully feasible as it would break backwards compatibility with applications that refer to this class explicitly, but also DeclarativeMeta intercepts class-level attribute set and delete operations which is a widely used pattern. An option for declarative base to use `__init_subclass__()` should be provided but this is out of scope for this particular change. Change-Id: I8aa898c7ab59d887739037d34b1cbab36521ab78 References: #6810
* happy new year 2022Mike Bayer2022-01-061-1/+1
| | | | Change-Id: I49abf2607e0eb0623650efdf0091b1fb3db737ea
* Remove all remaining removed_in_20 warnings slated for removalMike Bayer2022-01-051-20/+2
| | | | | | | | | | | | | | | | | | | | | | | | | Finalize all remaining removed-in-2.0 changes so that we can begin doing pep-484 typing without old things getting in the way (we will also have to do public_factory). note there are a few "moved_in_20()" and "became_legacy_in_20()" warnings still in place. The SQLALCHEMY_WARN_20 variable is now removed. Also removed here are the legacy "in place mutators" for Select statements, and some keyword-only argument signatures in Core have been added. Also in the big change department, the ORM mapper() function is removed entirely; the Mapper class is otherwise unchanged, just the public-facing API function. Mappers are now always given a registry in which to participate, however the argument signature of Mapper is not changed. ideally "registry" would be the first positional argument. Fixes: #7257 Change-Id: Ic70c57b9f1cf7eb996338af5183b11bdeb3e1623
* Update Black's target-version to py37Hugo van Kemenade2022-01-051-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | <!-- Provide a general summary of your proposed changes in the Title field above --> ### Description <!-- Describe your changes in detail --> Black's `target-version` was still set to `['py27', 'py36']`. Set it to `[py37]` instead. Also update Black and other pre-commit hooks and re-format with Black. ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - Good to go, no issue or tests are needed - [ ] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. - [ ] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. **Have a nice day!** Closes: #7536 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7536 Pull-request-sha: b3aedf5570d7e0ba6c354e5989835260d0591b08 Change-Id: I8be85636fd2c9449b07a8626050c8bd35bd119d5
* Properly type _generative, decorator, public_factoryFederico Caselli2021-12-301-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Good new is that pylance likes it and copies over the singature and everything. Bad news is that mypy does not support this yet https://github.com/python/mypy/issues/8645 Other minor bad news is that non_generative is not typed. I've tried using a protocol like the one in the comment but the signature is not ported over by pylance, so it's probably best to just live without it to have the correct signature. notes from mike: these three decorators are at the core of getting the library to be typed, more good news is that pylance will do all the things we like re: public_factory, see https://github.com/microsoft/pyright/issues/2758#issuecomment-1002788656 . For @_generative, we will likely move to using pep 673 once mypy supports it which may be soon. but overall having the explicit "return self" in the methods, while a little inconvenient, makes the typing more straightforward and locally present in the files rather than being decided at a distance. having "return self" present, or not, both have problems, so maybe we will be able to change it again if things change as far as decorator support. As it is, I feel like we are barely squeaking by with our decorators, the typing is already pretty out there. Change-Id: Ic77e13fc861def76a5925331df85c0aa48d77807 References: #6810
* Replace raise_ with raise fromFederico Caselli2021-12-271-44/+26
| | | | | Change-Id: I7aaeb5bc130271624335b79cf586581d6c6c34c7 References: #4600
* Merge "implement cython for cache_anon_map, prefix_anon_map" into mainmike bayer2021-12-271-8/+3
|\
| * implement cython for cache_anon_map, prefix_anon_mapMike Bayer2021-12-211-8/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These are small bits where cache_anon_map in particular is part of the cache key generation scheme which is a key target for cython. changing such a tiny element of the cache key gen is doing basically nothing yet, as the cython impl is mostly the exact same speed as the python one. I guess for cython to be effective we'd need to redo the whole cache key generation and possibly not use the same kinds of structures, which might not be very easy to do. Additionally, some cython runtime import errors are being observed on jenkins, add an upfront check to the test suite to indicate if the expected build succeeded when REQUIRE_SQLALCHEMY_CEXT is set. Running case CacheAnonMap Running python .... Done Running cython .... Done | python | cython | cy / py | test_get_anon_non_present| 0.301266758 | 0.231203834 | 0.767438915 | test_get_anon_present| 0.300919362 | 0.227336695 | 0.755473803 | test_has_key_non_present| 0.152725077 | 0.133191719 | 0.872101171 | test_has_key_present| 0.152689778 | 0.133673095 | 0.875455428 | Running case PrefixAnonMap Running python .. Done Running cython .. Done | python | cython | cy / py | test_apply_non_present| 0.358715744 | 0.335245703 | 0.934572034 | test_apply_present | 0.354434996 | 0.338579782 | 0.955266229 | Change-Id: I0d3f1dd285c044afc234479141d831b2ee0455be
* | accommodate for "clone" of ColumnClauseMike Bayer2021-12-211-1/+17
|/ | | | | | | | | | | | | | | | | | | | | | | | for use with the ClauseElement.params() method, altered ColumnClause._clone() so that while the element stays immutable, if the column is associated with a subquery, it returns a new version of itself as corresponding to a clone of the subquery. this allows processing functions to access the parameters in the subquery and produce a copy of it. The use case here is the expanded use of .params() within loader strategies that use HasCacheKey._apply_params_to_element(). Fixed issue in new "loader criteria" method :meth:`_orm.PropComparator.and_` where usage with a loader strategy like :func:`_orm.selectinload` against a column that was a member of the ``.c.`` collection of a subquery object, where the subquery would be dynamically added to the FROM clause of the statement, would be subject to stale parameter values within the subquery in the SQL statement cache, as the process used by the loader strategy to replace the parameters at execution time would fail to accommodate the subquery when received in this form. Fixes: #7489 Change-Id: Ibb3b6af140b8a62a2c8d05b2ac92e86ca3013c46
* Warn when caching is disabled / documentMike Bayer2021-12-061-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds new warnings for all elements that don't indicate their caching behavior, including user-defined ClauseElement subclasses and third party dialects. it additionally adds new documentation to discuss an apparent performance degradation in 1.4 when caching is disabled as a result in the significant expense incurred by ORM lazy loaders, which in 1.3 used BakedQuery so were actually cached. As a result of adding the warnings, a fair degree of lesser used SQL expression objects identified that they did not define caching behavior so would have been producing ``[no key]``, including PostgreSQL constructs ``hstore`` and ``array``. These have been amended to use inherit cache where appropriate. "on conflict" constructs in PostgreSQL, MySQL, SQLite still explicitly don't generate a cache key at this time. The change also adds a test for all constructs via assert_compile() to assert they will not generate cache warnings. Fixes: #7394 Change-Id: I85958affbb99bfad0f5efa21bc8f2a95e7e46981
* Removals: MetaData.bind, Table.bind, all other .bindMike Bayer2021-12-021-24/+11
| | | | | Change-Id: I1ef2eb2018f4b68825fe40a2a8d99084cf217b35 References: #7257
* Clean up most py3k compatFederico Caselli2021-11-241-18/+13
| | | | Change-Id: I8172fdcc3103ff92aa049827728484c8779af6b7
* Support lightweight compiler column elements w/ slotsMike Bayer2021-11-221-86/+108
| | | | | | | | | | | | | | | | | | | | | | | | | | | | the _CompileLabel class included ``__slots__`` but these weren't used as the superclasses included slots. Create a ``__slots__`` superclass for ``ClauseElement``, creating a new class of compilable SQL elements that don't include heavier features like caching, annotations and cloning, which are meant to be used only in an ad-hoc compiler fashion. Create new ``CompilerColumnElement`` from that which serves in column-oriented contexts, but similarly does not include any expression operator support as it is intended to be used only to generate a string. Apply this to both ``_CompileLabel`` as well as PostgreSQL ``_ColonCast``, which does not actually subclass ``ColumnElement`` as this class has memoized attributes that aren't worth changing, and does not include SQL operator capabilities as these are not needed for these compiler-only objects. this allows us to more inexpensively add new ad-hoc labels / casts etc. at compile time, as we will be seeking to expand out the typecasts that are needed for PostgreSQL dialects in a subsequent patch. Change-Id: I52973ae3295cb6e2eb0d7adc816c678a626643ed
* Remove object in class definitionFederico Caselli2021-11-221-1/+1
| | | | | References: #4600 Change-Id: I2a62ddfe00bc562720f0eae700a497495d7a987a
* fully implement future engine and remove legacyMike Bayer2021-11-071-22/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | The major action here is to lift and move future.Connection and future.Engine fully into sqlalchemy.engine.base. This removes lots of engine concepts, including: * autocommit * Connection running without a transaction, autobegin is now present in all cases * most "autorollback" is obsolete * Core-level subtransactions (i.e. MarkerTransaction) * "branched" connections, copies of connections * execution_options() returns self, not a new connection * old argument formats, distill_params(), simplifies calling scheme between engine methods * before/after_execute() events (oriented towards compiled constructs) don't emit for exec_driver_sql(). before/after_cursor_execute() is still included for this * old helper methods superseded by context managers, connection.transaction(), engine.transaction() engine.run_callable() * ancient engine-level reflection methods has_table(), table_names() * sqlalchemy.testing.engines.proxying_engine References: #7257 Change-Id: Ib20ed816642d873b84221378a9ec34480e01e82c
* use ExpressionElementRole for case targets in case()Mike Bayer2021-11-031-22/+12
| | | | | | | | | | | | | | Fixed regression where the :func:`_sql.text` construct would no longer be accepted as a target case in the "whens" list within a :func:`_sql.case` construct. The regression appears related to an attempt to guard against some forms of literal values that were considered to be ambiguous when passed here; however, there's no reason the target cases shouldn't be interpreted as open-ended SQL expressions just like anywhere else, and a literal string or tuple will be converted to a bound parameter as would be the case elsewhere. Fixes: #7287 Change-Id: I75478adfa115f3292cb1362cc5b2fdf152b0ed6f
* First round of removal of python 2Federico Caselli2021-11-011-9/+0
| | | | | References: #4600 Change-Id: I61e35bc93fe95610ae75b31c18a3282558cd4ffe
* remove _resolve_label and related attributesMike Bayer2021-10-201-19/+6
| | | | | | | | | | | | | these seem to date back to 0.9 and revs like #3148 but none of it seems to be affecting things now, try removing it all and seeing what fails. we've noted that _resolve_label does not appear to be needed, however allow_label_resolve remains relevant both for non-column elements like text() as well as that it is used explicitly by the joined eager loader. Change-Id: Ic8a5d8001ef2a4133360f51a92a6f7b0cc389095
* process bulk_update_tuples before cache key or compilationMike Bayer2021-10-191-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed regression where the use of a :class:`_orm.hybrid_property` attribute or a mapped :func:`_orm.composite` attribute as a key passed to the :meth:`_dml.Update.values` method for an ORM-enabled :class:`_dml.Update` statement, as well as when using it via the legacy :meth:`_orm.Query.update` method, would be processed for incoming ORM/hybrid/composite values within the compilation stage of the UPDATE statement, which meant that in those cases where caching occurred, subsequent invocations of the same statement would no longer receive the correct values. This would include not only hybrids that use the :meth:`_orm.hybrid_property.update_expression` method, but any use of a plain hybrid attribute as well. For composites, the issue instead caused a non-repeatable cache key to be generated, which would break caching and could fill up the statement cache with repeated statements. The :class:`_dml.Update` construct now handles the processing of key/value pairs passed to :meth:`_dml.Update.values` and :meth:`_dml.Update.ordered_values` up front when the construct is first generated, before the cache key has been generated so that the key/value pairs are processed each time, and so that the cache key is generated against the individual column/value pairs that will ultimately be used in the statement. Fixes: #7209 Change-Id: I08f248d1d60ea9690b014c21439b775d951fb9e5
* use coercions for label element, ensure propagate_attrsMike Bayer2021-10-181-3/+15
| | | | | | | | | | Fixed bug where the ORM "plugin", necessary for features such as :func:`_orm.with_loader_criteria` to work correctly, would not be applied to a :func:`_sql.select` which queried from an ORM column expression if it made use of the :meth:`_sql.ColumnElement.label` modifier. Fixes: #7205 Change-Id: I72b84442e14df8b5ece33916f3c51ca3f358864b
* Pickling fixes for ORM / CoreMike Bayer2021-10-131-0/+3
| | | | | | | | | | | | | | | | | | Fixed regression where ORM loaded objects could not be pickled in cases where loader options making use of ``"*"`` were used in certain combinations, such as combining the :func:`_orm.joinedload` loader strategy with ``raiseload('*')`` of sub-elements. Fixes: #7134 Fixed issue where SQL queries using the :meth:`_functions.FunctionElement.within_group` construct could not be pickled, typically when using the ``sqlalchemy.ext.serializer`` extension but also for general generic pickling. Fixes: #6520 Change-Id: Ib73fd49c875e6da9898493c190f610e68b88ec72
* fixes for usage of the null() and similar constantsMike Bayer2021-10-081-13/+33
| | | | | | | | | | | | | | | | | | | | | | | | Adjusted the "column disambiguation" logic that's new in 1.4, where the same expression repeated gets an "extra anonymous" label, so that the logic more aggressively deduplicates those labels when the repeated element is the same Python expression object each time, as occurs in cases like when using "singleton" values like :func:`_sql.null`. This is based on the observation that at least some databases (e.g. MySQL, but not SQLite) will raise an error if the same label is repeated inside of a subquery. Related to :ticket:`7153`, fixed an issue where result column lookups would fail for "adapted" SELECT statements that selected for "constant" value expressions most typically the NULL expression, as would occur in such places as joined eager loading in conjunction with limit/offset. This was overall a regression due to issue :ticket:`6259` which removed all "adaption" for constants like NULL, "true", and "false", but this broke the case where the same adaption logic were used to match the constant to a labeled expression referring to the constant in a subquery. Fixes: #7153 Fixes: #7154 Change-Id: I43823343721b9e70524ea3f5e8f39dd543a3e92b
* repair any_() / all_() "implicit flip" behavior for NoneMike Bayer2021-10-041-22/+73
| | | | | | | | | | | | | | Fixed an inconsistency in the any_() / all_() functions / methods where the special behavior these functions have of "flipping" the expression such that the "ANY" / "ALL" expression is always on the right side would not function if the comparison were against the None value, that is, "column.any_() == None" should produce the same SQL expression as "null() == column.any_()". Added more docs to clarify this as well, plus mentions that any_() / all_() generally supersede the ARRAY version "any()" / "all()". Fixes: #7140 Change-Id: Ia5d55414ba40eb3fbda3598931fdd24c9b4a4411
* Fixes: #4504jonathan vanasco2021-09-291-0/+30
| | | | | | Docstring for EXTRACT Change-Id: Ie529ceb3551adedf0df873c22e65f8615cd1c9ef
* Merge "dont warn for dictionary passed positionally"mike bayer2021-07-211-8/+8
|\
| * dont warn for dictionary passed positionallyMike Bayer2021-07-211-8/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | Fixed issue where use of the :paramref:`_sql.case.whens` parameter passing a dictionary positionally and not as a keyword argument would emit a 2.0 deprecation warning, referring to the deprecation of passing a list positionally. The dictionary format of "whens", passed positionally, is still supported and was accidentally marked as deprecated. Removes warning filter for case statement. Fixes: #6786 Change-Id: I8efd1882563773bec89ae5e34f0dfede77fc4683
* | Merge "Documentation improvements"mike bayer2021-07-201-1/+1
|\ \ | |/ |/|
| * Documentation improvementsFederico Caselli2021-07-201-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Also remove deprecated usage: - load_only does not accept strings - case.whens is positional only Ref #6712 Ref #5994 Ref #6121 Ref #6785 Ref https://groups.google.com/g/sqlalchemy/c/-cnhThEu3kk Change-Id: I5db49a075b9d3d332518b9d189a24b13b502e2af
* | reset key/name when TableValuedColumn is adaptedMike Bayer2021-07-161-0/+4
| | | | | | | | | | | | | | | | | | | | Fixed issue in new :meth:`_schema.Table.table_valued` method where the resulting :class:`_sql.TableValuedColumn` construct would not respond correctly to alias adaptation as is used throughout the ORM, such as for eager loading, polymorphic loading, etc. Fixes: #6775 Change-Id: I77cec4b6e1b1003f2b6be242b54ada8e4a435250
* | labeling refactorMike Bayer2021-07-121-55/+188
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To service #6718 and #6710, the system by which columns are given labels in a SELECT statement as well as the system that gives them keys in a .c or .selected_columns collection have been refactored to provide a single source of truth for both, in constrast to the previous approach that included similar logic repeated in slightly different ways. Main ideas: 1. ColumnElement attributes ._label, ._anon_label, ._key_label are renamed to include the letters "tq", meaning "table-qualified" - these labels are only used when rendering a SELECT that has LABEL_STYLE_TABLENAME_PLUS_COL for its label style; as this label style is primarily legacy, the "tq" names should be isolated so that in a 2.0 style application these aren't being used at all 2. The means by which the "labels" and "proxy keys" for the elements of a SELECT has been centralized to a single source of truth; previously, the three of _generate_columns_plus_names, _generate_fromclause_column_proxies, and _column_naming_convention all had duplicated rules between them, as well as that there were a little bit of labeling rules in compiler._label_select_column as well; by this we mean that the various "anon_label" "anon_key" methods on ColumnElement were called by all four of these methods, where there were many cases where it was necessary that one method comes up with the same answer as another of the methods. This has all been centralized into _generate_columns_plus_names for all the names except the "proxy key", which is generated by _column_naming_convention. 3. compiler._label_select_column has been rewritten to both not make any naming decisions nor any "proxy key" decisions, only whether to label or not to label; the _generate_columns_plus_names method gives it the information, where the proxy keys come from _column_naming_convention; previously, these proxy keys were matched based on restatement of similar (but not really the same) logic in two places. The heuristics of "whether to label or not to label" are also reorganized to be much easier to read and understand. 4. a new method compiler._label_returning_column is added for dialects to use in their "generate returning columns" methods. A github search reveals a small number of third party dialects also doing this using the prior _label_select_column method so we try to make sure _label_select_column continues to work the exact same way for that specific use case; for the "SELECT" use case it now needs 5. After some attempts to do it different ways, for the case where _proxy_key is giving us some kind of anon label, we are hard changing it to "_no_label" right now, as there's not currently a way to fully match anonymized labels from stmt.c or stmt.selected_columns to what will be in the result map. The idea of "_no_label" is to encourage the user to use label('name') for columns they want to be able to target by string name that don't have a natural name. Change-Id: I7a92a66f3a7e459ccf32587ac0a3c306650daf11
* Merge "Add additional support to honor _proxy_key in Core selects"mike bayer2021-07-091-0/+15
|\
| * Add additional support to honor _proxy_key in Core selectsMike Bayer2021-07-051-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed ORM regression where ad-hoc label names generated for hybrid properties and potentially other similar types of ORM-enabled expressions would usually be propagated outwards through subqueries, allowing the name to be retained in the final keys of the result set even when selecting from subqueries. Additional state is now tracked in this case that isn't lost when a hybrid is selected out of a Core select / subquery. as we have removed things like column.label() from ORM, since we now have to export the cols with the same names as what we will render, experiment with giving a greater role to the _proxy_key annotation so that a desired name can be carried through more transarently. Fixes: #6718 Change-Id: Icb313244c13ea1a8f58d3e05d07aa3e1039e15bf
View Lorry Controller Status