summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/util
Commit message (Collapse)AuthorAgeFilesLines
* fix link issue, add ref for pytest-xdist pinFederico Caselli2023-05-151-1/+1
| | | | Change-Id: If565b2b6b968457f221eb84f0270f617618d3319
* Avoid unnecessary nested exceptionsFederico Caselli2023-05-151-1/+3
| | | | Change-Id: Ibca9dc93f541992105cd915e02a3d3e9a0dbd67e
* Merge "fix test suite warnings" into mainmike bayer2023-05-101-0/+27
|\
| * fix test suite warningsMike Bayer2023-05-091-0/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | fix a handful of warnings that were emitting but not raising, usually because they were inside an "expect_warnings" block. modify "expect_warnings" to always use "raise_on_any_unexpected" behavior; remove this parameter. Fixed issue in semi-private ``await_only()`` and ``await_fallback()`` concurrency functions where the given awaitable would remain un-awaited if the function threw a ``GreenletError``, which could cause "was not awaited" warnings later on if the program continued. In this case, the given awaitable is now cancelled before the exception is thrown. Change-Id: I33668c5e8c670454a3d879e559096fb873b57244
* | adjust fwd_ref logicMike Bayer2023-04-281-1/+1
|/ | | | | | | | | | Fixed issue where ORM Annotated Declarative would not resolve forward references correctly in all cases; in particular, when using ``from __future__ import annotations`` in combination with Pydantic dataclasses. Change-Id: If643c9a4ac7e217d4cb3a7d09b96cfd49432c44e References: #9717
* add deterministic imv returning ordering using sentinel columnsMike Bayer2023-04-212-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Repaired a major shortcoming which was identified in the :ref:`engine_insertmanyvalues` performance optimization feature first introduced in the 2.0 series. This was a continuation of the change in 2.0.9 which disabled the SQL Server version of the feature due to a reliance in the ORM on apparent row ordering that is not guaranteed to take place. The fix applies new logic to all "insertmanyvalues" operations, which takes effect when a new parameter :paramref:`_dml.Insert.returning.sort_by_parameter_order` on the :meth:`_dml.Insert.returning` or :meth:`_dml.UpdateBase.return_defaults` methods, that through a combination of alternate SQL forms, direct correspondence of client side parameters, and in some cases downgrading to running row-at-a-time, will apply sorting to each batch of returned rows using correspondence to primary key or other unique values in each row which can be correlated to the input data. Performance impact is expected to be minimal as nearly all common primary key scenarios are suitable for parameter-ordered batching to be achieved for all backends other than SQLite, while "row-at-a-time" mode operates with a bare minimum of Python overhead compared to the very heavyweight approaches used in the 1.x series. For SQLite, there is no difference in performance when "row-at-a-time" mode is used. It's anticipated that with an efficient "row-at-a-time" INSERT with RETURNING batching capability, the "insertmanyvalues" feature can be later be more easily generalized to third party backends that include RETURNING support but not necessarily easy ways to guarantee a correspondence with parameter order. Fixes: #9618 References: #9603 Change-Id: I1d79353f5f19638f752936ba1c35e4dc235a8b7c
* establish column_property and query_expression as readonly from a dc perspectiveMike Bayer2023-04-122-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | Fixed bug in ORM Declarative Dataclasses where the :func:`_orm.queryable_attribute` and :func:`_orm.column_property` constructs, which are documented as read-only constructs in the context of a Declarative mapping, could not be used with a :class:`_orm.MappedAsDataclass` class without adding ``init=False``, which in the case of :func:`_orm.queryable_attribute` was not possible as no ``init`` parameter was included. These constructs have been modified from a dataclass perspective to be assumed to be "read only", setting ``init=False`` by default and no longer including them in the pep-681 constructor. The dataclass parameters for :func:`_orm.column_property` ``init``, ``default``, ``default_factory``, ``kw_only`` are now deprecated; these fields don't apply to :func:`_orm.column_property` as used in a Declarative dataclasses configuration where the construct would be read-only. Also added read-specific parameter :paramref:`_orm.queryable_attribute.compare` to :func:`_orm.queryable_attribute`; :paramref:`_orm.queryable_attribute.repr` was already present. Added missing :paramref:`_orm.mapped_column.active_history` parameter to :func:`_orm.mapped_column` construct. Fixes: #9628 Change-Id: I2ab44d6b763b20410bd1ebb5ac949a6d223f1ce2
* update for mypy 1.2.0Mike Bayer2023-04-081-1/+1
| | | | Change-Id: I6a46046af5b6ed484b470057761e77b485e2e061
* Add missing methods to OrderedSet.Federico Caselli2023-03-301-9/+28
| | | | | | | Implemented missing method ``copy`` and ``pop`` in OrderedSet class. Fixes: #9487 Change-Id: I1d2278b64939b44422e9d5857ec7d345fff53997
* check for recursion with container typesMike Bayer2023-03-271-8/+22
| | | | | | | | | | Fixed issue in ORM Annotated Declarative where using a recursive type (e.g. using a nested Dict type) would result in a recursion overflow in the ORM's annotation resolution logic, even if this datatype were not necessary to map the column. Fixes: #9553 Change-Id: Ied99dc0d47276c6e9c23fa9df5fc65f7736d65cf
* Improved wheel pipelineFederico Caselli2023-03-171-1/+5
| | | | | | | | - ensure that the compiled extensions are used - speed up job by parallelizing more Fixes: #9434 Change-Id: Ief750b28733ba24bb5ff8c105e1a4c9b7b928700
* update for mypy 1.1.1Mike Bayer2023-03-082-5/+16
| | | | | | | looks like mypy 1.1.1 slipped in after the last jenkins pep484 build for SQLAlchemy. update for its new issues. Change-Id: Ifca967a75d5e592c04b0138aeda9b646067fe59b
* KeyFuncDict regression fixes and dataclass fixesMike Bayer2023-03-051-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | adapt None-key warning for non-mapped attributes Fixed multiple regressions due to :ticket:`8372`, involving :func:`_orm.attribute_mapped_collection` (now called :func:`_orm.attribute_keyed_dict`). First, the collection was no longer usable with "key" attributes that were not themselves ordinary mapped attributes; attributes linked to descriptors and/or association proxy attributes have been fixed. Second, if an event or other operation needed access to the "key" in order to populate the dictionary from an mapped attribute that was not loaded, this also would raise an error inappropriately, rather than trying to load the attribute as was the behavior in 1.4. This is also fixed. For both cases, the behavior of :ticket:`8372` has been expanded. :ticket:`8372` introduced an error that raises when the derived key that would be used as a mapped dictionary key is effectively unassigned. In this change, a warning only is emitted if the effective value of the ".key" attribute is ``None``, where it cannot be unambiguously determined if this ``None`` was intentional or not. ``None`` will be not supported as mapped collection dictionary keys going forward (as it typically refers to NULL which means "unknown"). Setting :paramref:`_orm.attribute_keyed_dict.ignore_unpopulated_attribute` will now cause such ``None`` keys to be ignored as well. Add value constructors to dictionary collections Added constructor arguments to the built-in mapping collection types including :class:`.KeyFuncDict`, :func:`_orm.attribute_keyed_dict`, :func:`_orm.column_keyed_dict` so that these dictionary types may be constructed in place given the data up front; this provides further compatibility with tools such as Python dataclasses ``.asdict()`` which relies upon invoking these classes directly as ordinary dictionary classes. Fixes: #9418 Fixes: #9424 Change-Id: Ib16c4e690b7ac3fcc34df2f139cad61c6c4b2b19
* apply a fixed locals w/ Mapped to all de-stringifyMike Bayer2023-02-261-6/+29
| | | | | | | | | | | | | | | Continued the fix for :ticket:`8853`, allowing the :class:`_orm.Mapped` name to be fully qualified regardless of whether or not ``from __annotations__ import future`` were present. This issue first fixed in 2.0.0b3 confirmed that this case worked via the test suite, however the test suite apparently was not testing the behavior for the name ``Mapped`` not being locally present at all; string resolution has been updated to ensure the ``Mapped`` symbol is locatable as applies to how the ORM uses these functions. Fixes: #8853 Fixes: #9335 Change-Id: Id82d09aee906165a4d77c7da6a0b4177dd675c10
* Remove `typing.Self` workaroundYurii Karabas2023-02-081-1/+1
| | | | | | | | | | | | Remove ``typing.Self`` workaround, now using :pep:`673` for most methods that return ``Self``. Pull request courtesy Yurii Karabas. Fixes: #9254 Closes: #9255 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9255 Pull-request-sha: 2947df8ada79f5c3afe9c838e65993302199c2f7 Change-Id: Ic32015ad52e95a61f3913d43ea436aa9402804df
* Add support for typing.Literal in MappedFrederik Aalund2023-01-311-1/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added support for :pep:`586` ``Literal`` to be used in the :paramref:`_orm.registry.type_annotation_map` as well as within :class:`.Mapped` constructs. To use custom types such as these, they must appear explicitly within the :paramref:`_orm.registry.type_annotation_map` to be mapped. Pull request courtesy Frederik Aalund. As part of this change, the support for :class:`.sqltypes.Enum` in the :paramref:`_orm.registry.type_annotation_map` has been expanded to include support for ``Literal[]`` types consisting of string values to be used, in addition to ``enum.Enum`` datatypes. If a ``Literal[]`` datatype is used within ``Mapped[]`` that is not linked in :paramref:`_orm.registry.type_annotation_map` to a specific datatype, a :class:`.sqltypes.Enum` will be used by default. Fixed issue involving the use of :class:`.sqltypes.Enum` within the :paramref:`_orm.registry.type_annotation_map` where the :paramref:`_sqltypes.Enum.native_enum` parameter would not be correctly copied to the mapped column datatype, if it were overridden as stated in the documentation to set this parameter to False. Fixes: #9187 Fixes: #9200 Closes: #9191 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9191 Pull-request-sha: 7d13f705307bf62560fc831f6f049a425d411374 Change-Id: Ife3ba2655f4897f806d6a9cf0041c69fd4f39e9d
* support NewType in type_annotation_mapMike Bayer2023-01-311-11/+19
| | | | | | | | | | | | | | | | | Added support for :pep:`484` ``NewType`` to be used in the :paramref:`_orm.registry.type_annotation_map` as well as within :class:`.Mapped` constructs. These types will behave in the same way as custom subclasses of types right now; they must appear explicitly within the :paramref:`_orm.registry.type_annotation_map` to be mapped. Within this change, the lookup between decl_api._resolve_type and TypeEngine._resolve_for_python_type is streamlined to not inspect the given type multiple times, instead passing in from decl_api to TypeEngine the already "flattened" version of a Generic or NewType type. Fixes: #9175 Change-Id: I227cf84b4b88e4567fa2d1d7da0c05b54e00c562
* 2.0 prepMike Bayer2023-01-261-2/+2
| | | | | | still waiting on two more gerrits but this will set things up Change-Id: I966d4c683972c5b965b8db136509062cc8abfe91
* add context for warnings emitted from configure_mappers(), autoflush()jonathan vanasco2023-01-241-7/+40
| | | | | | | | | | | Improved the notification of warnings that are emitted within the configure mappers or flush process, which are often invoked as part of a different operation, to add additional context to the message that indicates one of these operations as the source of the warning within operations that may not be obviously related. Fixes: #7305 Change-Id: I79da7a6a5d4cf67d57615d0ffc2b8d8454011c84
* refactor code generation tools , include --check commandMike Bayer2023-01-182-44/+198
| | | | | | | | | | | | | | | | | in particular it looks like CI was not picking up on the "git diff" oriented commands, which were failing to run due to pathing issues. As we were setting cwd for black/zimports relative to sqlalchemy library, and tox installs it in the venv, black/zimports would fail to run from tox, and since these are subprocess.run we didn't pick up the failure. This overall locks down how zimports/black are run so that we are definitely from the source root, by using the location of tools/ to determine the root. Fixes: #8892 Change-Id: I7c54b747edd5a80e0c699b8456febf66d8b62375
* dont assume copy_with() on builtins list, dict, etc; improve error msg.Mike Bayer2023-01-161-3/+18
| | | | | | | | | | | | | | | | | | Fixed issue where using an ``Annotated`` type in the ``type_annotation_map`` which itself contained a plain container type (e.g. ``list``, ``dict``) generic type as the target type would produce an internal error where the ORM were trying to interpret the ``Annotated`` instance. Added an error message when a :func:`_orm.relationship` is mapped against an abstract container type, such as ``Mapped[Sequence[B]]``, without providing the :paramref:`_orm.relationship.container_class` parameter which is necessary when the type is abstract. Previously the the abstract container would attempt to be instantiated and fail. Fixes: #9099 Fixes: #9100 Change-Id: I18aa6abd5451c5ac75a9caed8441ff0cd8f44589
* replace @decorated_property decoratorMike Bayer2023-01-122-49/+0
| | | | | | | | | | | | | This decorator is no longer necessary as of Mypy 0.981 [1]. In current mypy versions, we require direct use of `@property` for return types of these methods to be recognized [1] https://github.com/python/mypy/issues/1362 Change-Id: Ibc36083dec854c5f9140a9b621e9bf9d5bb4fb61
* happy new year 2023Mike Bayer2023-01-0312-12/+12
| | | | Change-Id: I625af65b3fb1815b1af17dc2ef47dd697fdc3fb1
* dont call platform.architecture()Mike Bayer2022-12-161-1/+1
| | | | | | | | | | | Fixed regression where the base compat module was calling upon ``platform.architecture()`` in order to detect some system properties, which results in an over-broad system call against the system-level ``file`` call that is unavailable under some circumstances, including within some secure environment configurations. Fixes: #8995 Change-Id: Ib6171e75aff5a60a79dab81a0be21bee2456318b
* Improve support for enum in mapped classesFederico Caselli2022-11-291-1/+10
| | | | | | | | | | Add a new system by which TypeEngine objects have some say in how the declarative type registry interprets them. The Enum datatype is the primary target for this but it is hoped the system may be useful for other types as well. Fixes: #8859 Change-Id: I15ac3daee770408b5795746f47c1bbd931b7d26d
* Add new script to sync test files adding future annotationFederico Caselli2022-11-261-1/+1
| | | | | | | | | | | Create a new test file test_tm_future_annotations_sync.py that's obtained from test_tm_future_annotations.py by using the new script sync_test_files. This files includes the ``from __future__ import annotations`` It also turns out we had some bugs in there and we can also support some additional typing things Change-Id: Iac005df206d45a55345d9d88d67a80ce799d449f
* Merge "fix optionalized forms for dict[]" into mainmike bayer2022-11-221-9/+61
|\
| * fix optionalized forms for dict[]Mike Bayer2022-11-221-9/+61
| | | | | | | | | | | | | | | | | | | | | | Fixed a suite of issues involving :class:`.Mapped` use with dictionary types, such as ``Mapped[dict[str, str] | None]``, would not be correctly interpreted in Declarative ORM mappings. Support to correctly "de-optionalize" this type including for lookup in type_annotation_map has been fixed. Fixes: #8777 Change-Id: Iaba90791dea341d00eaff788d40b0a4e48dab02e
* | fall back to eval() for names that have dotsMike Bayer2022-11-211-0/+2
|/ | | | | | | | | | Fixed regression in 2.0.0b3 caused by :ticket:`8759` where indicating the :class:`.Mapped` name using a qualified name such as ``sqlalchemy.orm.Mapped`` would fail to be recognized by Declarative as indicating the :class:`.Mapped` construct. Fixes: #8853 Change-Id: Iddb6efaae864d4545e80c54658244670f81ef6cc
* Try running pyupgrade on the codeFederico Caselli2022-11-165-13/+9
| | | | | | | | command run is "pyupgrade --py37-plus --keep-runtime-typing --keep-percent-format <files...>" pyupgrade will change assert_ to assertTrue. That was reverted since assertTrue does not exists in sqlalchemy fixtures Change-Id: Ie1ed2675c7b11d893d78e028aad0d1576baebb55
* Merge "don't invoke fromclause.c when creating an annotated" into mainmike bayer2022-11-161-1/+1
|\
| * don't invoke fromclause.c when creating an annotatedMike Bayer2022-11-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The ``aliased()`` constructor calls upon ``__clause_element__()``, which internally annotates a ``FromClause``, like a subquery. This became expensive as ``AnnotatedFromClause`` has for many years called upon ``element.c`` so that the full ``.c`` collection is transferred to the Annotated. Taking this out proved to be challenging. A straight remove seemed to not break any tests except for the one that tested the exact condition. Nevertheless this seemed "spooky" so I instead moved the get of ``.c`` to be in a memoized proxy method. However, that then exposed a recursion issue related to loader_criteria; so the source of that behavior, which was an accidental behavioral artifact, is now made into an explcicit option that loader_criteria uses directly. The accidental behavioral artifact in question is still kind of strange since I was not able to fully trace out how it works, but the end result is that fixing the artifact to be "correct" causes loader_criteria, within the particular test for #7491, creates a select/ subquery structure with a cycle in it, so compilation fails with recursion overflow. The "solution" is to cause the artifact to occur in this case, which is that the ``AnnotatedFromClause`` will have a different ``.c`` collection than its element, which is a subquery. It's not totally clear how a cycle is generated when this is not done. This is commit one of two, which goes through some hoops to make essentially a one-line change. The next commit will rework ColumnCollection to optimize the corresponding_column() method significantly. Fixes: #8796 Change-Id: Id58ae6554db62139462c11a8be7313a3677456ad
* | Repair test in 32bit python builds.Federico Caselli2022-11-122-0/+2
|/ | | | Change-Id: I8287f3e1a975534c8a01a41c9dcc7e5e9f08bb52
* ensure anon_map is passed for most annotated traversalsMike Bayer2022-11-112-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | We can cache the annotated cache key for Table, but for selectables it's not safe, as it fails to pass the anon_map along and creates many redudant structures in observed test scenario. It is likely safe for a Column that's mapped to a Table also, however this is not implemented here. Will have to see if that part needs adjusting. Fixed critical memory issue identified in cache key generation, where for very large and complex ORM statements that make use of lots of ORM aliases with subqueries, cache key generation could produce excessively large keys that were orders of magnitude bigger than the statement itself. Much thanks to Rollo Konig Brock for their very patient, long term help in finally identifying this issue. Also within TypeEngine objects, when we generate elements for instance variables, skip the None elements at least. this also saves on tuple complexity. Fixes: #8790 Change-Id: I448ddbfb45ae0a648815be8dad4faad7d1977427
* work around Python 3.11 IntEnum issue; update FastIntFlagMike Bayer2022-11-101-6/+19
| | | | | | | | | | | | | | | | | in [1], Python 3.11 seems to have changed the behavior of IntEnum. We didn't notice this because we have our own workaround class already, but typing did. Ensure we remain compatible with IntFlag. This change also modifies FastIntFlag to no longer use global symbols; this is unnecessary as we assign FastIntFlag members explicitly. Use of ``symbol()`` should probably be phased out. [1] https://github.com/python/cpython/issues/99304 Fixes: #8783 Change-Id: I8ae2e871ff1467ae5ca1f63e66b5dae45d4a6c93
* try to support mypy 0.990Mike Bayer2022-11-092-19/+32
| | | | | | | | | | | mypy introduces a crash we need to work around, also some new rules. It also has either a behavioral change regarding how output is rendered in relationship to files being within sys.path or not, so work around that for test_mypy_plugin_py3k.py References: https://github.com/python/mypy/issues/14027 Change-Id: I689c7fe27dc52abee932de9e0fb23b2a2eba76fa
* support renamed symbols in annotation scansMike Bayer2022-11-041-14/+53
| | | | | | | | | | | | | | | | Added support in ORM declarative annotations for class names specified for :func:`_orm.relationship`, as well as the name of the :class:`_orm.Mapped` symbol itself, to be different names than their direct class name, to support scenarios such as where :class:`_orm.Mapped` is imported as ``from sqlalchemy.orm import Mapped as M``, or where related class names are imported with an alternate name in a similar fashion. Additionally, a target class name given as the lead argument for :func:`_orm.relationship` will always supersede the name given in the left hand annotation, so that otherwise un-importable names that also don't match the class name can still be used in annotations. Fixes: #8759 Change-Id: I74a00de7e1a45bf62dad50fd385bb75cf343f9f3
* Support result.close() for all iterator patternsMike Bayer2022-11-032-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change contains new features for 2.0 only as well as some behaviors that will be backported to 1.4. For 1.4 and 2.0: Fixed issue where the underlying DBAPI cursor would not be closed when using :class:`_orm.Query` with :meth:`_orm.Query.yield_per` and direct iteration, if a user-defined exception case were raised within the iteration process, interrupting the iterator. This would lead to the usual MySQL-related issues with server side cursors out of sync. For 1.4 only: A similar scenario can occur when using :term:`2.x` executions with direct use of :class:`.Result`, in that case the end-user code has access to the :class:`.Result` itself and should call :meth:`.Result.close` directly. Version 2.0 will feature context-manager calling patterns to address this use case. However within the 1.4 scope, ensured that ``.close()`` methods are available on all :class:`.Result` implementations including :class:`.ScalarResult`, :class:`.MappingResult`. For 2.0 only: To better support the use case of iterating :class:`.Result` and :class:`.AsyncResult` objects where user-defined exceptions may interrupt the iteration, both objects as well as variants such as :class:`.ScalarResult`, :class:`.MappingResult`, :class:`.AsyncScalarResult`, :class:`.AsyncMappingResult` now support context manager usage, where the result will be closed at the end of iteration. Corrected various typing issues within the engine and async engine packages. Fixes: #8710 Change-Id: I3166328bfd3900957eb33cbf1061d0495c9df670
* evaluate types in terms of the class in which they appearMike Bayer2022-10-311-3/+7
| | | | | | | | | | | Fixed issues within the declarative typing resolver (i.e. which resolves ``ForwardRef`` objects) where types that were declared for columns in one particular source file would raise ``NameError`` when the ultimate mapped class were in another source file. The types are now resolved in terms of the module for each class in which the types are used. Fixes: #8742 Change-Id: I236f94484ea79d47392a6201e671eeb89c305fd8
* Add pep 584 to python immutabledict fallbackFederico Caselli2022-10-221-1/+13
| | | | | Fixes: #8695 Change-Id: Ie0412c3a7b2b1ba5bd5112f204318ff763cbb8f4
* more many-to-one typingMike Bayer2022-10-191-2/+47
| | | | | | | | | | | | | | | | | | since we are typing centric, note this configuration as we have just supported in #8668. Note also I am just taking "backref" out of the basic version of the docs here totally, this doc is already a lot to read / take in without making it even more confusing; backref still has an entirely dedicated docs page which can have all the additional behaviors of backref() described. Additionally, get other "optional" forms to work including ``cls | None`` and ``Union[cls, None]``. Fixes: #8668 Change-Id: I2b026f496a1710ddebfb4aa6cf8459b4892cbc54
* implement write-only colletions, typing for dynamicMike Bayer2022-10-061-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For 2.0, we provide a truly "larger than memory collection" implementation, a write-only collection that will never under any circumstances implicitly load the entire collection, even during flush. This is essentially a much more "strict" version of the "dynamic" loader, which in fact has a lot of scenarios that it loads the full backing collection into memory, mostly defeating its purpose. Typing constructs are added that support both the new feature WriteOnlyMapping as well as the legacy feature DynamicMapping. These have been integrated with "annotion based mapping" so that relationship() uses these annotations to configure the loader strategy as well. additional changes: * the docs triggered a conflict in hybrid's "transformers" section, this section is hard-coded to Query using a pattern that doesnt seem to have any use and isn't part of the current select() interface, so just removed this section * As the docs for WriteOnlyMapping are very long, collections.rst is broken up into two pages now. Fixes: #6229 Fixes: #7123 Change-Id: I6929f3da6e441cad92285e7309030a9bac4e429d
* reorganize Mapped[] super outside of MapperPropertyMike Bayer2022-10-052-1/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | We made all the MapperProperty classes a subclass of Mapped[] to allow declarative mappings to name Mapped[] on the left side. this was cheating a bit because MapperProperty is not actually a descriptor, and the mapping process replaces the object with InstrumentedAttribute at mapping time, which is the actual Mapped[] descriptor. But now in I6929f3da6e441cad92285e7309030a9bac4e429d we are considering making the "cheating" a little more extensive by putting DynamicMapped / WriteOnlyMapped in Relationship's hierarchy, which need a flat out "type: ignore" to work. Instead of pushing more cheats into the core classes, move out the "Declarative"-facing versions of these classes to be typing only: Relationship, Composite, Synonym, and MappedSQLExpression added for ColumnProperty. Keep the internals expressed on the old names, RelationshipProperty, CompositeProperty, SynonymProperty, ColumnProprerty, which will remain "pure" with fully correct typing. then have the typing only endpoints be where the "cheating" and "type: ignores" have to happen, so that these are more or less slightly better forms of "Any". Change-Id: Ied7cc11196c9204da6851f49593d1b1fd2ef8ad8
* adjust for mypy 0.981Mike Bayer2022-09-261-2/+2
| | | | | | | new release is out today, fix a very small number of new issues. Change-Id: I443c78f3384319d56deb2c9309118ffb750bbf41
* ORM bulk insert via executeMike Bayer2022-09-241-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * ORM Insert now includes "bulk" mode that will run essentially the same process as session.bulk_insert_mappings; interprets the given list of values as ORM attributes for key names * ORM UPDATE has a similar feature, without RETURNING support, for session.bulk_update_mappings * Added support for upserts to do RETURNING ORM objects as well * ORM UPDATE/DELETE with list of parameters + WHERE criteria is a not implemented; use connection * ORM UPDATE/DELETE defaults to "auto" synchronize_session; use fetch if RETURNING is present, evaluate if not, as "fetch" is much more efficient (no expired object SELECT problem) and less error prone if RETURNING is available UPDATE: howver this is inefficient! please continue to use evaluate for simple cases, auto can move to fetch if criteria not evaluable * "Evaluate" criteria will now not preemptively unexpire and SELECT attributes that were individually expired. Instead, if evaluation of the criteria indicates that the necessary attrs were expired, we expire the object completely (delete) or expire the SET attrs unconditionally (update). This keeps the object in the same unloaded state where it will refresh those attrs on the next pass, for this generally unusual case. (originally #5664) * Core change! update/delete rowcount comes from len(rows) if RETURNING was used. SQLite at least otherwise did not support this. adjusted test_rowcount accordingly * ORM DELETE with a list of parameters at all is also a not implemented as this would imply "bulk", and there is no bulk_delete_mappings (could be, but we dont have that) * ORM insert().values() with single or multi-values translates key names based on ORM attribute names * ORM returning() implemented for insert, update, delete; explcit returning clauses now interpret rows in an ORM context, with support for qualifying loader options as well * session.bulk_insert_mappings() assigns polymorphic identity if not set. * explicit RETURNING + synchronize_session='fetch' is now supported with UPDATE and DELETE. * expanded return_defaults() to work with DELETE also. * added support for composite attributes to be present in the dictionaries used by bulk_insert_mappings and bulk_update_mappings, which is also the new ORM bulk insert/update feature, that will expand the composite values into their individual mapped attributes the way they'd be on a mapped instance. * bulk UPDATE supports "synchronize_session=evaluate", is the default. this does not apply to session.bulk_update_mappings, just the new version * both bulk UPDATE and bulk INSERT, the latter with or without RETURNING, support *heterogenous* parameter sets. session.bulk_insert/update_mappings did this, so this feature is maintained. now cursor result can be both horizontally and vertically spliced :) This is now a long story with a lot of options, which in itself is a problem to be able to document all of this in some way that makes sense. raising exceptions for use cases we haven't supported is pretty important here too, the tradition of letting unsupported things just not work is likely not a good idea at this point, though there are still many cases that aren't easily avoidable Fixes: #8360 Fixes: #7864 Fixes: #7865 Change-Id: Idf28379f8705e403a3c6a937f6a798a042ef2540
* additional de-stringify pass for unionsMike Bayer2022-09-081-2/+15
| | | | | | | | | | | | | | the change in c3cfee5b00a40790c18d took out a pass for de-stringify that broke some un-tested cases for Optional with future annotations mode. Adding tests for this revealed that this was a subset of a more general case where Union is presented with ForwardRefs inside of it matching up within the type map, which wasn't working before either, fixed that as well with an additional de-stringify for elements within the Union. Fixes: #8478 Change-Id: I8804cf6c67f14d10804584e1cddd2cfaa2376654
* Improve compiled extension detectionFederico Caselli2022-09-051-2/+20
| | | | | | | | | Ensure that all cython extension are imported by the compied detection logic. This is required since cython extensions moduels are marked as optional in the install, so it's possible that only some of them are compiled. The extensions are enabled only if all of them are correctly compiled Change-Id: I355cbac06f5c7a47d35661f42ebab3b0156c1965
* Detection of PEP 604 union syntax.Peter Schutt2022-09-011-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ### Description Fixes #8478 Handle `UnionType` as arguments to `Mapped`, e.g., `Mapped[str | None]`: - adds `utils.typing.is_optional_union()` used to detect if a column should be nullable. - adds `"UnionType"` to `utils.typing.is_optional()` names. - uses `get_origin()` in `utils.typing.is_origin_of()` as `UnionType` has no `__origin__` attribute. - tests with runtime type and postponed annotations and guard the tests running with `compat.py310`. ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - Good to go, no issue or tests are needed - [x] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. - [ ] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. **Have a nice day!** Closes: #8479 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8479 Pull-request-sha: 12417654822272c5847e684c53677f665553ef0e Change-Id: Ib3248043dd4a97324ac592c048385006536b2d49
* run update_subclass anytime we add new clslevel dispatchMike Bayer2022-08-311-1/+1
| | | | | | | | | | | Fixed event listening issue where event listeners added to a superclass would be lost if a subclass were created which then had its own listeners associated. The practical example is that of the :class:`.sessionmaker` class created after events have been associated with the :class:`_orm.Session` class. Fixes: #8467 Change-Id: I9bdba8769147e30110a09900d4a577e833ac3af9
* implement PG ranges/multiranges agnosticallyMike Bayer2022-08-051-0/+1
| | | | | | | | | | | | | | | Ranges now work using a new Range object, multiranges as lists of Range objects (this is what asyncpg does. not sure why psycopg has a "Multirange" type). psycopg, psycopg2, and asyncpg are currently supported. It's not clear how to make ranges work with pg8000, likely needs string conversion; this is straightforward with the new archicture and can be added later. Fixes: #8178 Change-Id: Iab8d8382873d5c14199adbe3f09fd0dc17e2b9f1
View Lorry Controller Status