summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/orm
Commit message (Collapse)AuthorAgeFilesLines
* Merge "Dont return outer transaction for _subtrans flag"mike bayer2021-04-092-4/+14
|\
| * Dont return outer transaction for _subtrans flagMike Bayer2021-04-092-4/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed critical regression where the :class:`_orm.Session` could fail to "autobegin" a new transaction when a flush occurred without an existing transaction in place, implicitly placing the :class:`_orm.Session` into legacy autocommit mode which commit the transaction. The :class:`_orm.Session` now has a check that will prevent this condition from occurring, in addition to repairing the flush issue. Additionally, scaled back part of the change made as part of :ticket:`5226` which can run autoflush during an unexpire operation, to not actually do this in the case of a :class:`_orm.Session` using legacy :paramref:`_orm.Session.autocommit` mode, as this incurs a commit within a refresh operation. Fixes: #6233 Change-Id: Ia980e62a090e39e3e2a7fb77c95832ae784cc9a5
* | Apply recursive check to immediateloader and generalizeMike Bayer2021-04-091-12/+33
|/ | | | | | | | | | | | | Fixed critical regression caused by the new feature added as part of :ticket:`1763`, eager loaders are invoked on unexpire operations. The new feature makes use of the "immediateload" eager loader strategy as a substitute for a collection loading strategy, which unlike the other "post-load" strategies was not accommodating for recursive invocations between mutually-dependent relationships, leading to recursion overflow errors. Fixes: #6232 Change-Id: Ifb2286281f40d1a04c24741261d4438659b6e3dd
* Merge "convert subqueryload paths for multilevel"mike bayer2021-04-081-0/+7
|\
| * convert subqueryload paths for multilevelMike Bayer2021-04-071-0/+7
| | | | | | | | | | | | | | | | | | | | Fixed regression where the :func:`_orm.subqueryload` loader strategy would fail to correctly accommodate sub-options, such as a :func:`_orm.defer` option on a column, if the "path" of the subqueryload were more than one level deep. Fixes: #6221 Change-Id: I2addef0be6aaa022fa1b2f20060c4f0769c3bdfb
* | Merge "Add test support for merge_frozen_result"mike bayer2021-04-071-2/+2
|\ \
| * | Add test support for merge_frozen_resultMike Bayer2021-04-071-2/+2
| |/ | | | | | | | | | | | | | | | | Fixed regression where the :func:`_orm.merge_frozen_result` function relied upon by the dogpile.caching example was not included in tests and began failing due to incorrect internal arguments. Fixes: #6211 Change-Id: I0b53d0f569c817994ad4827a3ddb1626fd2d082f
* | Check for hybrid's attribute name and support no nameMike Bayer2021-04-071-3/+15
|/ | | | | | | | | | | | | | | Fixed regression where the ORM compilation scheme would assume the function name of a hybrid property would be the same as the attribute name in such a way that an ``AttributeError`` would be raised, when it would attempt to determine the correct name for each element in a result tuple. A similar issue exists in 1.3 but only impacts the names of tuple rows. The fix here adds a check that the hybrid's function name is actually present in the ``__dict__`` of the class or its superclasses before assigning this name; otherwise, the hybrid is considered to be "unnamed" and ORM result tuples will use the naming scheme of the underlying expression. Fixes: #6215 Change-Id: I584c0c05efec957f4dcaccf5df371399a57dffe9
* Disable and disallow Result.unique() with yield_perMike Bayer2021-04-062-2/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed critical regression where the :meth:`_orm.Query.yield_per` method in the ORM would set up the internal :class:`_engine.Result` to yield chunks at a time, however made use of the new :meth:`_engine.Result.unique` method which uniques across the entire result. This would lead to lost rows since the ORM is using ``id(obj)`` as the uniquing function, which leads to repeated identifiers for new objects as already-seen objects are garbage collected. 1.3's behavior here was to "unique" across each chunk, which does not actually produce "uniqued" results when results are yielded in chunks. As the :meth:`_orm.Query.yield_per` method is already explicitly disallowed when joined eager loading is in place, which is the primary rationale for the "uniquing" feature, the "uniquing" feature is now turned off entirely when :meth:`_orm.Query.yield_per` is used. This regression only applies to the legacy :class:`_orm.Query` object; when using :term:`2.0 style` execution, "uniquing" is not automatically applied. To prevent the issue from arising from explicit use of :meth:`_engine.Result.unique`, an error is now raised if rows are fetched from a "uniqued" ORM-level :class:`_engine.Result` if any :ref:`yield per <orm_queryguide_yield_per>` API is also in use, as the purpose of ``yield_per`` is to allow for arbitrarily large numbers of rows, which cannot be uniqued in memory without growing the number of entries to fit the complete result size. Fixes: #6206 Change-Id: I3770d1f2e9be44d82c83ca992afb912dcc17af05
* Merge "Disallow AliasedReturnsRows from execution"mike bayer2021-04-063-4/+2
|\
| * Disallow AliasedReturnsRows from executionMike Bayer2021-04-053-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Executing a :class:`_sql.Subquery` using :meth:`_engine.Connection.execute` is deprecated and will emit a deprecation warning; this use case was an oversight that should have been removed from 1.4. The operation will now execute the underlying :class:`_sql.Select` object directly for backwards compatibility. Similarly, the :class:`_sql.CTE` class is also not appropriate for execution. In 1.3, attempting to execute a CTE would result in an invalid "blank" SQL statement being executed; since this use case was not working it now raises :class:`_exc.ObjectNotExecutableError`. Previously, 1.4 was attempting to execute the CTE as a statement however it was working only erratically. The change also breaks out StatementRole from ReturnsRowsRole, as these roles should not be in the same lineage (some statements don't return rows, the whole class of ReturnsRows that are from clauses are not statements). Consolidate StatementRole and CoerceTextStatementRole as there's no usage difference between these. Simplify some old tests that were trying to make sure that "execution options" didn't transmit from a cte/subquery out to a select; as cte/subuqery() aren't executable in any case the options are removed. Fixes: #6204 Change-Id: I62613b7ab418afdd22c409eae75659e3f52fb65f
* | Merge "Adjust for mypy incremental behaviors"mike bayer2021-04-062-0/+43
|\ \
| * | Adjust for mypy incremental behaviorsMike Bayer2021-04-052-0/+43
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Applied a series of refactorings and fixes to accommodate for Mypy "incremental" mode across multiple files, which previously was not taken into account. In this mode the Mypy plugin has to accommodate Python datatypes expressed in other files coming in with less information than they have on a direct run. Additionally, a new decorator :func:`_orm.declarative_mixin` is added, which is necessary for the Mypy plugin to be able to definifitely identify a Declarative mixin class that is otherwise not used inside a particular Python file. discussion: With incremental / deserialized mypy runs, it appears that when we look at a base class that comes from another file, cls.info is set to a special undefined node that matches CLASSDEF_NO_INFO, and we otherwise can't touch it without crashing. Additionally, sometimes cls.defs.body is present but empty. However, it appears that both of these cases can be sidestepped, first by doing a lookup() for the type name where we get a SymbolTableNode that then has the TypeInfo we wanted when we tried touching cls.info, and then however we got the TypeInfo, if cls.defs.body is empty we can just look in the names to get at the symbols for that class; we just can't access AssignmentStmts, but that's fine because we just need the information for classes we aren't actually type checking. This work also revealed there's no easy way to detect a mixin class so we just create a new decorator to mark that. will make code look better in any case. Fixes: #6147 Change-Id: Ia8fac8acfeec931d8f280491cffc5c6cb4a1204e
* | Detect (Entity, Entity) vs (Entity, onclause) in legacy joinMike Bayer2021-04-051-4/+24
|/ | | | | | | | | Fixed regression where a deprecated form of :meth:`_orm.Query.join` were used, passing a series of entities to join from without any ON clause in a single :meth:`_orm.Query.join` call, would fail to function correctly. Fixes: #6203 Change-Id: I5a6ec80de972af5b2ca9054e6f24a0b8af4a3e13
* Expand sibling tests for overlaps warningMike Bayer2021-03-311-1/+8
| | | | | | | | | | | | | Scaled back the warning message added in :ticket:`5171` to not warn for overlapping columns in an inheritance scenario where a particular relationship is local to a subclass and therefore does not represent an overlap. Add errors documentation for the warning and also expand ``util.warn()`` to include a code parameter. Fixes: #6171 Change-Id: Icb1f12d8d645d439ffd2bbb7371c6b00042b6ae3
* Merge "Commentary; run criteria.params() if statement isn't cached?"mike bayer2021-03-271-0/+13
|\
| * Commentary; run criteria.params() if statement isn't cached?Mike Bayer2021-03-271-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Considering adjustment to 56f9c7743e9083add69a10501a503f, if statement is not cached, skip the relatively expensive step of re-processing the criteria clause. However, this causes the overall cache key of the statement to come out differently which should also be avoided. Likely we would not merge the actual change, just the comment here. References: #6139 Change-Id: Idb555b78d8d7950d084315e004448f64cf59bb5c
* | Merge "Add ScopedSession.get()"mike bayer2021-03-261-0/+1
|\ \ | |/ |/|
| * Add ScopedSession.get()Mike Bayer2021-03-261-0/+1
| | | | | | | | | | | | | | | | Fixed missing method :meth:`_orm.Session.get` from the :class:`_orm.ScopedSession` interface. Fixes: #6144 Change-Id: I1d2425675f7f972c2479070a2e11b1d96a9aca8b
* | Adapt loader_criteria params for current queryMike Bayer2021-03-262-15/+118
|/ | | | | | | | | | | | | | | | | | Fixed critical issue in the new :meth:`_orm.PropComparator.and_` feature where loader strategies that emit secondary SELECT statements such as :func:`_orm.selectinload` and :func:`_orm.lazyload` would fail to accommodate for bound parameters in the user-defined criteria in terms of the current statement being executed, as opposed to the cached statement, causing stale bound values to be used. This also adds a warning for the case where an object that uses :func:`_orm.lazyload` in conjunction with :meth:`_orm.PropComparator.and_` is attempted to be serialized; the loader criteria cannot reliably be serialized and deserialized and eager loading should be used for this case. Fixes: #6139 Change-Id: I5a638bbecb7b583db2d3c0b76469f5a25c13dd3b
* Merge "Use class-local metadata for declarative base"mike bayer2021-03-251-1/+7
|\
| * Use class-local metadata for declarative baseMike Bayer2021-03-241-1/+7
| | | | | | | | | | | | | | | | | | Fixed regression where the ``.metadata`` attribute on a per class level would not be honored, breaking the use case of per-class-hierarchy :class:`.schema.MetaData` for abstract declarative classes and mixins. Fixes: #6128 Change-Id: I5c15436b5c5171105dc1a0192fa744daf79a344d
* | Support __visit_name__ on PropComparator to work in cloningMike Bayer2021-03-242-0/+6
|/ | | | | | | | | Repaired support so that the :meth:`_sql.Select.params` method can work correctly with a :class:`_sql.Select` object that includes joins across ORM relationship structures, which is a new feature in 1.4. Fixes: #6124 Change-Id: Ia92fc33c3acbe66910e9e3bf00af9100de19b2b8
* Merge "Remove internal use of string attr in loader option"mike bayer2021-03-233-66/+73
|\
| * Remove internal use of string attr in loader optionMike Bayer2021-03-233-66/+73
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed issue where a "removed in 2.0" warning were generated internally by the relationship loader mechanics. This changeset started the effort of converting all string usage in the test suite, however this is a much longer job as the use of strings in loader options is widespread. In particular I'm not totally comfortable with strings not being accepted in obvious spots like Load(User).load_only("x", "y", "z"), which points to a new string expecting functionality that's not what's there now. However at the moment it seems like we need to continue removing all support for strings and then figure out "immediate strings from an explicit class" later. Fixes: #6115 Change-Id: I6b314d135d2bc049fd66500914b772c1fe60b5b3
* | Merge "Adjust derivation rules for table vs. subquery against a join"mike bayer2021-03-231-0/+7
|\ \
| * | Adjust derivation rules for table vs. subquery against a joinMike Bayer2021-03-231-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed bug where ORM queries using a correlated subquery in conjunction with :func:`_orm.column_property` would fail to correlate correctly to an enclosing subquery or to a CTE when :meth:`_sql.Select.correlate_except` were used in the property to control correlation, in cases where the subquery contained the same selectables as ones within the correlated subquery that were intended to not be correlated. This is achieved by adding a limiting factor to ClauseAdapter which is to explicitly pass the selectables we will be adapting "from", which is then used by AliasedClass to limit "from" to the mappers represented by the AliasedClass. This did cause one test where an alias for a contains_eager() was missing to suddenly fail, and the test was corrected, however there may be some very edge cases like that one where the tighter criteria causes an existing use case that's relying on the more liberal aliasing to require modifications. Fixes: #6060 Change-Id: I8342042641886e1a220beafeb94fe45ea7aadb33
* | | warn / document for Query.with_polymorphic() with with_loader_criteria()Mike Bayer2021-03-232-2/+18
| |/ |/| | | | | | | | | | | | | | | These are illustrated as not working in #6111. As this is a highly complex and legacy method, encourage users to migrate off of it before using with_loader_criteria(). Fixes: #6111 Change-Id: I63c8187020c631d83259ea2200b66eabf74a0d0d
* | fix __all__Mike Bayer2021-03-201-1/+10
|/ | | | | | | sqlalchemy.engine had an oddly restrictive __all__ for some reason. Add missing symbols to session.__all__ Change-Id: I017fa1c2a93f559f2ccc366f88660266c50e9ca6
* support callable mapped attributes in dataclass mixinsMike Bayer2021-03-192-22/+90
| | | | | | | | Added support for the :class:`_orm.declared_attr` object to work in the context of dataclass fields. Fixes: #6100 Change-Id: Ifaf4a6482c866d6cfee99d8bc2c6294d923460d7
* Merge "Ensure ClauseAdapter treats FunctionElement as a ColumnElement"mike bayer2021-03-191-0/+2
|\
| * Ensure ClauseAdapter treats FunctionElement as a ColumnElementMike Bayer2021-03-181-0/+2
| | | | | | | | | | | | | | | | | | | | Fixed regression where use of an unnamed SQL expression such as a SQL function would raise a column targeting error if the query itself were using joinedload for an entity and was also being wrapped in a subquery by the joinedload eager loading process. Fixes: #6086 Change-Id: I22cf4d6974685267c4f903bd7639be8271c6c1ef
* | Merge "repair legacy_last_joined_entity for no onclause"mike bayer2021-03-191-1/+1
|\ \ | |/ |/|
| * repair legacy_last_joined_entity for no onclauseMike Bayer2021-03-181-1/+1
| | | | | | | | | | | | | | | | | | Fixed regression where the :meth:`_orm.Query.filter_by` method would fail to locate the correct source entity if the :meth:`_orm.Query.join` method had been used targeting an entity without any kind of ON clause. Fixes: #6092 Change-Id: I38d9099844f842f314c6673bd922467242409cdb
* | Merge "Adjust dataclass rules to account for field w/ default"mike bayer2021-03-181-7/+17
|\ \
| * | Adjust dataclass rules to account for field w/ defaultMike Bayer2021-03-181-7/+17
| |/ | | | | | | | | | | | | | | | | | | Fixed issue in new ORM dataclasses functionality where dataclass fields on an abstract base or mixin that contained column or other mapping constructs would not be mapped if they also included a "default" key within the dataclasses.field() object. Fixes: #6093 Change-Id: I628086ceb48ab1dd0702f239cd12be74074f58f1
* | Merge "Raise at Core / ORM concrete inh level for label overlap"mike bayer2021-03-181-0/+9
|\ \
| * | Raise at Core / ORM concrete inh level for label overlapMike Bayer2021-03-181-0/+9
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed regression where the :class:`.ConcreteBase` would fail to map at all when a mapped column name overlapped with the discriminator column name, producing an assertion error. The use case here did not function correctly in 1.3 as the polymorphic union would produce a query that ignored the discriminator column entirely, while emitting duplicate column warnings. As 1.4's architecture cannot easily reproduce this essentially broken behavior of 1.3 at the ``select()`` level right now, the use case now raises an informative error message instructing the user to use the ``.ConcreteBase._concrete_discriminator_name`` attribute to resolve the conflict. To assist with this configuration, ``.ConcreteBase._concrete_discriminator_name`` may be placed on the base class only where it will be automatically used by subclasses; previously this was not the case. Fixes: #6090 Change-Id: I8b7d01e4c9ea0dc97f30b8cd658b3505b24312a7
* | Restore Query.selectableMike Bayer2021-03-181-0/+12
|/ | | | | Fixes: #6088 Change-Id: Id014fbd081c0659d1939d059779780798cc8c1dd
* Merge "Use explicit names for mapper _get_clause parameters"mike bayer2021-03-173-2/+15
|\
| * Use explicit names for mapper _get_clause parametersMike Bayer2021-03-173-2/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed a critical regression in the relationship lazy loader where the SQL criteria used to fetch a related many-to-one object could go stale in relation to other memoized structures within the loader if the mapper had configuration changes, such as can occur when mappers are late configured or configured on demand, producing a comparison to None and returning no object. Huge thanks to Alan Hamlett for their help tracking this down late into the night. The primary change is that mapper._get_clause() uses a fixed name for its bound parameters, which is memoized under a lambda statement in the case of many-to-one lazy loading. This has implications for some other logic namely the .compare() used by loader strategies to determine use_get needed to be adjusted. This change also repairs the lambda module's behavior of removing the "required" flag from bound parameters, which caused this issue to also fail silently rather than issuing an error for a required bind parameter. Fixes: #6055 Change-Id: I19e1aba9207a049873e0f13c19bad7541e223cfd
* | Merge "Ensure entity or None returned from _entity_from_pre_ent_zero()"mike bayer2021-03-171-1/+7
|\ \
| * | Ensure entity or None returned from _entity_from_pre_ent_zero()Mike Bayer2021-03-171-1/+7
| |/ | | | | | | | | | | | | | | | | Fixed regression where the :meth:`_orm.Query.exists` method would fail to create an expression if the entity list of the :class:`_orm.Query` were an arbitrary SQL column expression. Fixes: #6076 Change-Id: I292dd5f527b2cbc1b76ca765b4ea321ef8535709
* | Merge "Fix typo in Session.identity_key"mike bayer2021-03-171-1/+1
|\ \
| * | Fix typo in Session.identity_keyMike Bayer2021-03-161-1/+1
| |/ | | | | | | | | | | | | | | | | Fixed regression in :meth:`_orm.Session.identity_key`, including that the method and related methods were not covered by any unit test as well as that the method contained a typo preventing it from functioning correctly. Fixes: #6067 Change-Id: I1a84f9ed095c4226d57eef1c46996601dc2f1eaa
* | turn off eager configure_mappers() outside of Query, LoadMike Bayer2021-03-162-3/+14
|/ | | | | | | | | | | | | | | | | | | | | | | | Fixed regression where producing a Core expression construct such as :func:`_sql.select` using ORM entities would eagerly configure the mappers, in an effort to maintain compatibility with the :class:`_orm.Query` object which necessarily does this to support many backref-related legacy cases. However, core :func:`_sql.select` constructs are also used in mapper configurations and such, and to that degree this eager configuration is more of an inconvenience, so eager configure has been disabled for the :func:`_sql.select` and other Core constructs in the absence of ORM loading types of functions such as :class:`_orm.Load`. The change maintains the behavior of :class:`_orm.Query` so that backwards compatibility is maintained. However, when using a :func:`_sql.select` in conjunction with ORM entities, a "backref" that isn't explicitly placed on one of the classes until mapper configure time won't be available unless :func:`_orm.configure_mappers` or the newer :func:`_orm.registry.configure` has been called elsewhere. Prefer using :paramref:`_orm.relationship.back_populates` for more explicit relationship configuration which does not have the eager configure requirement. Fixes: #6066 Change-Id: I7a953ddcc189471fbac63c97c51ab8956f64012e
* Early-assign Base.registry to a private nameMike Bayer2021-03-161-1/+16
| | | | | | | | | | | | | Fixed bug where user-mapped classes that contained an attribute named "registry" would cause conflicts with the new registry-based mapping system when using :class:`.DeclarativeMeta`. While the attribute remains something that can be set explicitly on a declarative base to be consumed by the metaclass, once located it is placed under a private class variable so it does not conflict with future subclasses that use the same name for other purposes. Fixes: #6054 Change-Id: I1f2e04b0d74c493e7e90eadead4e861d8960a794
* Move enable_eagerloads(False) out of _from_self() into count()Mike Bayer2021-03-151-2/+1
| | | | | | | | | | | | | | | | | Fixed regression where calling upon :meth:`_orm.Query.count` in conjunction with a loader option such as :func:`_orm.joinedload` would fail to ignore the loader option. This is a behavior that has always been very specific to the :meth:`_orm.Query.count` method; an error is normally raised if a given :class:`_orm.Query` has options that don't apply to what it is returning. Specifically, the call to enable_eagerloads(False) inside of _from_self() is not needed as loader options are now not invoked for subqueries. Instead, set enable_eagerloads(False) in the count() method itself, so that these options won't be considered in this specific case. Fixes: #6052 Change-Id: I0059ed3fb06156ef4116fd015cbef6f89808e8ef
* Implement Mypy pluginMike Bayer2021-03-133-2/+82
| | | | | | | | | | | Rudimentary and experimental support for Mypy has been added in the form of a new plugin, which itself depends on new typing stubs for SQLAlchemy. The plugin allows declarative mappings in their standard form to both be compatible with Mypy as well as to provide typing support for mapped classes and instances. Fixes: #4609 Change-Id: Ia035978c02ad3a5c0e5b3c6c30044dd5a3155170
* Merge "Ignore flake8 F401 on specific files"mike bayer2021-03-071-76/+76
|\