diff options
44 files changed, 1111 insertions, 137 deletions
diff --git a/doc/build/changelog/changelog_14.rst b/doc/build/changelog/changelog_14.rst index 1e80c684c..4b6ed1d77 100644 --- a/doc/build/changelog/changelog_14.rst +++ b/doc/build/changelog/changelog_14.rst @@ -14,10 +14,48 @@ This document details individual issue-level changes made throughout .. changelog:: - :version: 1.4.48 + :version: 1.4.49 :include_notes_from: unreleased_14 .. changelog:: + :version: 1.4.48 + :released: April 30, 2023 + + .. change:: + :tags: bug, orm + :tickets: 9728 + :versions: 2.0.12 + + Fixed critical caching issue where the combination of + :func:`_orm.aliased()` and :func:`_hybrid.hybrid_property` expression + compositions would cause a cache key mismatch, leading to cache keys that + held onto the actual :func:`_orm.aliased` object while also not matching + that of equivalent constructs, filling up the cache. + + .. change:: + :tags: bug, orm + :tickets: 9634 + :versions: 2.0.10 + + Fixed bug where various ORM-specific getters such as + :attr:`.ORMExecuteState.is_column_load`, + :attr:`.ORMExecuteState.is_relationship_load`, + :attr:`.ORMExecuteState.loader_strategy_path` etc. would throw an + ``AttributeError`` if the SQL statement itself were a "compound select" + such as a UNION. + + .. change:: + :tags: bug, orm + :tickets: 9590 + :versions: 2.0.9 + + Fixed endless loop which could occur when using "relationship to aliased + class" feature and also indicating a recursive eager loader such as + ``lazy="selectinload"`` in the loader, in combination with another eager + loader on the opposite side. The check for cycles has been fixed to include + aliased class relationships. + +.. changelog:: :version: 1.4.47 :released: March 18, 2023 diff --git a/doc/build/changelog/changelog_20.rst b/doc/build/changelog/changelog_20.rst index ad54b7c5a..bb4144cd2 100644 --- a/doc/build/changelog/changelog_20.rst +++ b/doc/build/changelog/changelog_20.rst @@ -9,10 +9,25 @@ .. changelog:: - :version: 2.0.12 + :version: 2.0.13 :include_notes_from: unreleased_20 .. changelog:: + :version: 2.0.12 + :released: April 30, 2023 + + .. change:: + :tags: bug, mysql, mariadb + :tickets: 9722 + + Fixed issues regarding reflection of comments for :class:`_schema.Table` + and :class:`_schema.Column` objects, where the comments contained control + characters such as newlines. Additional testing support for these + characters as well as extended Unicode characters in table and column + comments (the latter of which aren't supported by MySQL/MariaDB) added to + testing overall. + +.. changelog:: :version: 2.0.11 :released: April 26, 2023 diff --git a/doc/build/changelog/unreleased_14/9590.rst b/doc/build/changelog/unreleased_14/9590.rst deleted file mode 100644 index 472cfc70e..000000000 --- a/doc/build/changelog/unreleased_14/9590.rst +++ /dev/null @@ -1,10 +0,0 @@ -.. change:: - :tags: bug, orm - :tickets: 9590 - :versions: 2.0.9 - - Fixed endless loop which could occur when using "relationship to aliased - class" feature and also indicating a recursive eager loader such as - ``lazy="selectinload"`` in the loader, in combination with another eager - loader on the opposite side. The check for cycles has been fixed to include - aliased class relationships. diff --git a/doc/build/changelog/unreleased_14/9634.rst b/doc/build/changelog/unreleased_14/9634.rst deleted file mode 100644 index 664e85716..000000000 --- a/doc/build/changelog/unreleased_14/9634.rst +++ /dev/null @@ -1,11 +0,0 @@ -.. change:: - :tags: bug, orm - :tickets: 9634 - :versions: 2.0.10 - - Fixed bug where various ORM-specific getters such as - :attr:`.ORMExecuteState.is_column_load`, - :attr:`.ORMExecuteState.is_relationship_load`, - :attr:`.ORMExecuteState.loader_strategy_path` etc. would throw an - ``AttributeError`` if the SQL statement itself were a "compound select" - such as a UNION. diff --git a/doc/build/changelog/unreleased_20/9656.rst b/doc/build/changelog/unreleased_20/9656.rst new file mode 100644 index 000000000..16c170aa8 --- /dev/null +++ b/doc/build/changelog/unreleased_20/9656.rst @@ -0,0 +1,8 @@ +.. change:: + :tags: typing, sql + :tickets: 9656 + + Added type ``ColumnExpressionArgument`` as a public alias of an internal + type. This type is useful since it's what' accepted by the sqlalchemy in + many api calls, such as :meth:`_sql.Select.where`, :meth:`_sql.and` and + many other. diff --git a/doc/build/changelog/unreleased_20/9715.rst b/doc/build/changelog/unreleased_20/9715.rst new file mode 100644 index 000000000..107051b72 --- /dev/null +++ b/doc/build/changelog/unreleased_20/9715.rst @@ -0,0 +1,8 @@ +.. change:: + :tags: bug, orm + :tickets: 9715 + + Fixed loader strategy pathing issues where eager loaders such as + :func:`_orm.joinedload` / :func:`_orm.selectinload` would fail to traverse + fully for many-levels deep following a load that had a + :func:`_orm.with_polymorphic` or similar construct as an interim member. diff --git a/doc/build/changelog/unreleased_20/9717.rst b/doc/build/changelog/unreleased_20/9717.rst new file mode 100644 index 000000000..d70ffe17a --- /dev/null +++ b/doc/build/changelog/unreleased_20/9717.rst @@ -0,0 +1,8 @@ +.. change:: + :tags: bug, orm + :tickets: 9717 + + Fixed issue where ORM Annotated Declarative would not resolve forward + references correctly in all cases; in particular, when using + ``from __future__ import annotations`` in combination with Pydantic + dataclasses. diff --git a/doc/build/changelog/unreleased_20/9731.rst b/doc/build/changelog/unreleased_20/9731.rst new file mode 100644 index 000000000..f5337b2c9 --- /dev/null +++ b/doc/build/changelog/unreleased_20/9731.rst @@ -0,0 +1,12 @@ +.. change:: + :tags: usecase, asyncio + :tickets: 9731 + + Added a new helper mixin :class:`_asyncio.AsyncAttrs` that seeks to improve + the use of lazy-loader and other expired or deferred ORM attributes with + asyncio, providing a simple attribute accessor that provides an ``await`` + interface to any ORM attribute, whether or not it needs to emit SQL. + + .. seealso:: + + :class:`_asyncio.AsyncAttrs` diff --git a/doc/build/changelog/unreleased_20/9739.rst b/doc/build/changelog/unreleased_20/9739.rst new file mode 100644 index 000000000..987d69ce2 --- /dev/null +++ b/doc/build/changelog/unreleased_20/9739.rst @@ -0,0 +1,9 @@ +.. change:: + :tags: bug, postgresql, regression + :tickets: 9739 + + Fixed another regression due to the "insertmanyvalues" change in 2.0.10 as + part of :ticket:`9618`, in a similar way as regression :ticket:`9701`, where + :class:`.LargeBinary` datatypes also need additional casts on when using the + asyncpg driver specifically in order to work with the new bulk INSERT + format. diff --git a/doc/build/changelog/unreleased_20/9746.rst b/doc/build/changelog/unreleased_20/9746.rst new file mode 100644 index 000000000..55d57925e --- /dev/null +++ b/doc/build/changelog/unreleased_20/9746.rst @@ -0,0 +1,8 @@ +.. change:: + :tags: bug, orm + :tickets: 9746 + + Fixed issue in new :ref:`orm_queryguide_upsert_returning` feature where the + ``populate_existing`` execution option was not being propagated to the + loading option, preventing existing attributes from being refreshed + in-place. diff --git a/doc/build/conf.py b/doc/build/conf.py index ccedb4906..0c1455bdb 100644 --- a/doc/build/conf.py +++ b/doc/build/conf.py @@ -242,9 +242,9 @@ copyright = "2007-2023, the SQLAlchemy authors and contributors" # noqa # The short X.Y version. version = "2.0" # The full version, including alpha/beta/rc tags. -release = "2.0.11" +release = "2.0.12" -release_date = "April 26, 2023" +release_date = "April 30, 2023" site_base = os.environ.get("RTD_SITE_BASE", "https://www.sqlalchemy.org") site_adapter_template = "docs_adapter.mako" diff --git a/doc/build/core/internals.rst b/doc/build/core/internals.rst index e3769b342..26aa9831d 100644 --- a/doc/build/core/internals.rst +++ b/doc/build/core/internals.rst @@ -69,3 +69,6 @@ Some key internal constructs are listed here. .. autoclass:: sqlalchemy.engine.AdaptedConnection :members: + +.. autoattribute:: sqlalchemy.sql.ColumnExpressionArgument + :members: diff --git a/doc/build/orm/extensions/associationproxy.rst b/doc/build/orm/extensions/associationproxy.rst index 036969f37..7b1ae3399 100644 --- a/doc/build/orm/extensions/associationproxy.rst +++ b/doc/build/orm/extensions/associationproxy.rst @@ -8,13 +8,18 @@ Association Proxy ``associationproxy`` is used to create a read/write view of a target attribute across a relationship. It essentially conceals the usage of a "middle" attribute between two endpoints, and -can be used to cherry-pick fields from a collection of -related objects or to reduce the verbosity of using the association -object pattern. Applied creatively, the association proxy allows +can be used to cherry-pick fields from both a collection of +related objects or scalar relationship. or to reduce the verbosity +of using the association object pattern. +Applied creatively, the association proxy allows the construction of sophisticated collections and dictionary views of virtually any geometry, persisted to the database using standard, transparently configured relational patterns. +.. contents:: Some example use cases for Association Proxy + :depth: 1 + :local: + .. _associationproxy_scalar_collections: Simplifying Scalar Collections @@ -124,7 +129,7 @@ the underlying collection or attribute does. .. _associationproxy_creator: Creation of New Values ----------------------- +^^^^^^^^^^^^^^^^^^^^^^ When a list ``append()`` event (or set ``add()``, dictionary ``__setitem__()``, or scalar assignment event) is intercepted by the association proxy, it @@ -689,6 +694,72 @@ deleted depends on the relationship cascade setting. :ref:`unitofwork_cascades` +Scalar Relationships +-------------------- + +The example below illustrates the use of the association proxy on the many +side of of a one-to-many relationship, accessing attributes of a scalar +object:: + + from __future__ import annotations + + from typing import List + + from sqlalchemy import ForeignKey + from sqlalchemy import String + from sqlalchemy.ext.associationproxy import association_proxy + from sqlalchemy.ext.associationproxy import AssociationProxy + from sqlalchemy.orm import DeclarativeBase + from sqlalchemy.orm import Mapped + from sqlalchemy.orm import mapped_column + from sqlalchemy.orm import relationship + + + class Base(DeclarativeBase): + pass + + + class Recipe(Base): + __tablename__ = "recipe" + id: Mapped[int] = mapped_column(primary_key=True) + name: Mapped[str] = mapped_column(String(64)) + + steps: Mapped[List[Step]] = relationship(back_populates="recipe") + step_descriptions: AssociationProxy[List[str]] = association_proxy( + "steps", "description" + ) + + + class Step(Base): + __tablename__ = "step" + id: Mapped[int] = mapped_column(primary_key=True) + description: Mapped[str] + recipe_id: Mapped[int] = mapped_column(ForeignKey("recipe.id")) + recipe: Mapped[Recipe] = relationship(back_populates="steps") + + recipe_name: AssociationProxy[str] = association_proxy("recipe", "name") + + def __init__(self, description: str) -> None: + self.description = description + + + my_snack = Recipe( + name="afternoon snack", + step_descriptions=[ + "slice bread", + "spread peanut butted", + "eat sandwich", + ], + ) + +A summary of the steps of ``my_snack`` can be printed using:: + + >>> for i, step in enumerate(my_snack.steps, 1): + ... print(f"Step {i} of {step.recipe_name!r}: {step.description}") + Step 1 of 'afternoon snack': slice bread + Step 2 of 'afternoon snack': spread peanut butted + Step 3 of 'afternoon snack': eat sandwich + API Documentation ----------------- diff --git a/doc/build/orm/extensions/asyncio.rst b/doc/build/orm/extensions/asyncio.rst index c57f1199c..0dff980e2 100644 --- a/doc/build/orm/extensions/asyncio.rst +++ b/doc/build/orm/extensions/asyncio.rst @@ -155,6 +155,7 @@ illustrates a complete example including mapper and session configuration:: from sqlalchemy import ForeignKey from sqlalchemy import func from sqlalchemy import select + from sqlalchemy.ext.asyncio import AsyncAttrs from sqlalchemy.ext.asyncio import async_sessionmaker from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import create_async_engine @@ -165,7 +166,7 @@ illustrates a complete example including mapper and session configuration:: from sqlalchemy.orm import selectinload - class Base(DeclarativeBase): + class Base(AsyncAttrs, DeclarativeBase): pass @@ -175,7 +176,7 @@ illustrates a complete example including mapper and session configuration:: id: Mapped[int] = mapped_column(primary_key=True) data: Mapped[str] create_date: Mapped[datetime.datetime] = mapped_column(server_default=func.now()) - bs: Mapped[List[B]] = relationship(lazy="raise") + bs: Mapped[List[B]] = relationship() class B(Base): @@ -225,6 +226,11 @@ illustrates a complete example including mapper and session configuration:: # expire_on_commit=False allows print(a1.data) + # alternatively, AsyncAttrs may be used to access any attribute + # as an awaitable (new in 2.0.13) + for b1 in await a1.awaitable_attrs.bs: + print(b1) + async def async_main() -> None: engine = create_async_engine( @@ -269,6 +275,64 @@ Using traditional asyncio, the application needs to avoid any points at which IO-on-attribute access may occur. Techniques that can be used to help this are below, many of which are illustrated in the preceding example. +* Attributes that are lazy-loading relationships, deferred columns or + expressions, or are being accessed in expiration scenarios can take advantage + of the :class:`_asyncio.AsyncAttrs` mixin. This mixin, when added to a + specific class or more generally to the Declarative ``Base`` superclass, + provides an accessor :attr:`_asyncio.AsyncAttrs.awaitable_attrs` + which delivers any attribute as an awaitable:: + + from __future__ import annotations + + from typing import List + + from sqlalchemy.ext.asyncio import AsyncAttrs + from sqlalchemy.orm import DeclarativeBase + from sqlalchemy.orm import Mapped + from sqlalchemy.orm import relationship + + + class Base(AsyncAttrs, DeclarativeBase): + pass + + + class A(Base): + __tablename__ = "a" + + # ... rest of mapping ... + + bs: Mapped[List[B]] = relationship() + + + class B(Base): + __tablename__ = "b" + + # ... rest of mapping ... + + Accessing the ``A.bs`` collection on newly loaded instances of ``A`` when + eager loading is not in use will normally use :term:`lazy loading`, which in + order to succeed will usually emit IO to the database, which will fail under + asyncio as no implicit IO is allowed. To access this attribute directly under + asyncio without any prior loading operations, the attribute can be accessed + as an awaitable by indicating the :attr:`_asyncio.AsyncAttrs.awaitable_attrs` + prefix:: + + a1 = await (session.scalars(select(A))).one() + for b1 in await a1.awaitable_attrs.bs: + print(b1) + + The :class:`_asyncio.AsyncAttrs` mixin provides a succinct facade over the + internal approach that's also used by the + :meth:`_asyncio.AsyncSession.run_sync` method. + + + .. versionadded:: 2.0.13 + + .. seealso:: + + :class:`_asyncio.AsyncAttrs` + + * Collections can be replaced with **write only collections** that will never emit IO implicitly, by using the :ref:`write_only_relationship` feature in SQLAlchemy 2.0. Using this feature, collections are never read from, only @@ -283,10 +347,9 @@ this are below, many of which are illustrated in the preceding example. bullets below address specific techniques when using traditional lazy-loaded relationships with asyncio, which requires more care. -* If using traditional ORM relationships which are subject to lazy loading, - relationships can be declared with ``lazy="raise"`` so that by - default they will not attempt to emit SQL. In order to load collections, - :term:`eager loading` must be used in all cases. +* If not using :class:`_asyncio.AsyncAttrs`, relationships can be declared + with ``lazy="raise"`` so that by default they will not attempt to emit SQL. + In order to load collections, :term:`eager loading` would be used instead. * The most useful eager loading strategy is the :func:`_orm.selectinload` eager loader, which is employed in the previous @@ -1019,6 +1082,9 @@ ORM Session API Documentation :members: :inherited-members: +.. autoclass:: AsyncAttrs + :members: + .. autoclass:: AsyncSession :members: :exclude-members: sync_session_class diff --git a/examples/asyncio/async_orm.py b/examples/asyncio/async_orm.py index 66501e545..eabc0250d 100644 --- a/examples/asyncio/async_orm.py +++ b/examples/asyncio/async_orm.py @@ -12,15 +12,18 @@ from typing import Optional from sqlalchemy import ForeignKey from sqlalchemy import func from sqlalchemy.ext.asyncio import async_sessionmaker +from sqlalchemy.ext.asyncio import AsyncAttrs from sqlalchemy.ext.asyncio import create_async_engine from sqlalchemy.future import select -from sqlalchemy.orm import declarative_base +from sqlalchemy.orm import DeclarativeBase from sqlalchemy.orm import Mapped from sqlalchemy.orm import mapped_column from sqlalchemy.orm import relationship from sqlalchemy.orm import selectinload -Base = declarative_base() + +class Base(AsyncAttrs, DeclarativeBase): + pass class A(Base): @@ -31,7 +34,7 @@ class A(Base): create_date: Mapped[datetime.datetime] = mapped_column( server_default=func.now() ) - bs: Mapped[List[B]] = relationship(lazy="raise") + bs: Mapped[List[B]] = relationship() class B(Base): @@ -93,11 +96,15 @@ async def async_main(): result = await session.scalars(select(A).order_by(A.id)) - a1 = result.first() + a1 = result.one() a1.data = "new data" await session.commit() + # use the AsyncAttrs interface to accommodate for a lazy load + for b1 in await a1.awaitable_attrs.bs: + print(b1) + asyncio.run(async_main()) diff --git a/examples/asyncio/async_orm_writeonly.py b/examples/asyncio/async_orm_writeonly.py index cdc486524..263c0d291 100644 --- a/examples/asyncio/async_orm_writeonly.py +++ b/examples/asyncio/async_orm_writeonly.py @@ -11,15 +11,18 @@ from typing import Optional from sqlalchemy import ForeignKey from sqlalchemy import func from sqlalchemy.ext.asyncio import async_sessionmaker +from sqlalchemy.ext.asyncio import AsyncAttrs from sqlalchemy.ext.asyncio import create_async_engine from sqlalchemy.future import select -from sqlalchemy.orm import declarative_base +from sqlalchemy.orm import DeclarativeBase from sqlalchemy.orm import Mapped from sqlalchemy.orm import mapped_column from sqlalchemy.orm import relationship from sqlalchemy.orm import WriteOnlyMapped -Base = declarative_base() + +class Base(AsyncAttrs, DeclarativeBase): + pass class A(Base): diff --git a/examples/asyncio/greenlet_orm.py b/examples/asyncio/greenlet_orm.py index 7429b6853..92880b992 100644 --- a/examples/asyncio/greenlet_orm.py +++ b/examples/asyncio/greenlet_orm.py @@ -10,13 +10,16 @@ from sqlalchemy import Column from sqlalchemy import ForeignKey from sqlalchemy import Integer from sqlalchemy import String +from sqlalchemy.ext.asyncio import AsyncAttrs from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import create_async_engine -from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.future import select +from sqlalchemy.orm import DeclarativeBase from sqlalchemy.orm import relationship -Base = declarative_base() + +class Base(AsyncAttrs, DeclarativeBase): + pass class A(Base): diff --git a/lib/sqlalchemy/__init__.py b/lib/sqlalchemy/__init__.py index e9f94f1d9..0bf16401c 100644 --- a/lib/sqlalchemy/__init__.py +++ b/lib/sqlalchemy/__init__.py @@ -79,6 +79,7 @@ from .schema import PrimaryKeyConstraint as PrimaryKeyConstraint from .schema import Sequence as Sequence from .schema import Table as Table from .schema import UniqueConstraint as UniqueConstraint +from .sql import ColumnExpressionArgument as ColumnExpressionArgument from .sql import SelectLabelStyle as SelectLabelStyle from .sql.expression import Alias as Alias from .sql.expression import alias as alias @@ -264,7 +265,7 @@ from .types import Uuid as Uuid from .types import VARBINARY as VARBINARY from .types import VARCHAR as VARCHAR -__version__ = "2.0.12" +__version__ = "2.0.13" def __go(lcls: Any) -> None: diff --git a/lib/sqlalchemy/dialects/mysql/reflection.py b/lib/sqlalchemy/dialects/mysql/reflection.py index ec3f82a60..ce1b9261d 100644 --- a/lib/sqlalchemy/dialects/mysql/reflection.py +++ b/lib/sqlalchemy/dialects/mysql/reflection.py @@ -146,11 +146,8 @@ class MySQLTableDefinitionParser: options = {} - if not line or line == ")": - pass - - else: - rest_of_line = line[:] + if line and line != ")": + rest_of_line = line for regex, cleanup in self._pr_options: m = regex.search(rest_of_line) if not m: @@ -310,7 +307,7 @@ class MySQLTableDefinitionParser: comment = spec.get("comment", None) if comment is not None: - comment = comment.replace("\\\\", "\\").replace("''", "'") + comment = cleanup_text(comment) sqltext = spec.get("generated") if sqltext is not None: @@ -585,11 +582,7 @@ class MySQLTableDefinitionParser: re.escape(directive), self._optional_equals, ) - self._pr_options.append( - _pr_compile( - regex, lambda v: v.replace("\\\\", "\\").replace("''", "'") - ) - ) + self._pr_options.append(_pr_compile(regex, cleanup_text)) def _add_option_word(self, directive): regex = r"(?P<directive>%s)%s" r"(?P<val>\w+)" % ( @@ -652,3 +645,28 @@ def _strip_values(values): a = a[1:-1].replace(a[0] * 2, a[0]) strip_values.append(a) return strip_values + + +def cleanup_text(raw_text: str) -> str: + if "\\" in raw_text: + raw_text = re.sub( + _control_char_regexp, lambda s: _control_char_map[s[0]], raw_text + ) + return raw_text.replace("''", "'") + + +_control_char_map = { + "\\\\": "\\", + "\\0": "\0", + "\\a": "\a", + "\\b": "\b", + "\\t": "\t", + "\\n": "\n", + "\\v": "\v", + "\\f": "\f", + "\\r": "\r", + # '\\e':'\e', +} +_control_char_regexp = re.compile( + "|".join(re.escape(k) for k in _control_char_map) +) diff --git a/lib/sqlalchemy/dialects/postgresql/asyncpg.py b/lib/sqlalchemy/dialects/postgresql/asyncpg.py index c879205e4..3f33600f9 100644 --- a/lib/sqlalchemy/dialects/postgresql/asyncpg.py +++ b/lib/sqlalchemy/dialects/postgresql/asyncpg.py @@ -182,6 +182,7 @@ from .base import PGExecutionContext from .base import PGIdentifierPreparer from .base import REGCLASS from .base import REGCONFIG +from .types import BYTEA from ... import exc from ... import pool from ... import util @@ -212,6 +213,10 @@ class AsyncpgTime(sqltypes.Time): render_bind_cast = True +class AsyncpgByteA(BYTEA): + render_bind_cast = True + + class AsyncpgDate(sqltypes.Date): render_bind_cast = True @@ -986,6 +991,7 @@ class PGDialect_asyncpg(PGDialect): sqltypes.Numeric: AsyncpgNumeric, sqltypes.Float: AsyncpgFloat, sqltypes.JSON: AsyncpgJSON, + sqltypes.LargeBinary: AsyncpgByteA, json.JSONB: AsyncpgJSONB, sqltypes.JSON.JSONPathType: AsyncpgJSONPathType, sqltypes.JSON.JSONIndexType: AsyncpgJSONIndexType, diff --git a/lib/sqlalchemy/engine/row.py b/lib/sqlalchemy/engine/row.py index da781334a..95789ddba 100644 --- a/lib/sqlalchemy/engine/row.py +++ b/lib/sqlalchemy/engine/row.py @@ -39,8 +39,8 @@ else: if TYPE_CHECKING: from .result import _KeyType - from .result import RMKeyView from .result import _ProcessorsType + from .result import RMKeyView _T = TypeVar("_T", bound=Any) _TP = TypeVar("_TP", bound=Tuple[Any, ...]) diff --git a/lib/sqlalchemy/ext/asyncio/__init__.py b/lib/sqlalchemy/ext/asyncio/__init__.py index 7195e1f07..ad6cd1526 100644 --- a/lib/sqlalchemy/ext/asyncio/__init__.py +++ b/lib/sqlalchemy/ext/asyncio/__init__.py @@ -19,5 +19,6 @@ from .scoping import async_scoped_session as async_scoped_session from .session import async_object_session as async_object_session from .session import async_session as async_session from .session import async_sessionmaker as async_sessionmaker +from .session import AsyncAttrs as AsyncAttrs from .session import AsyncSession as AsyncSession from .session import AsyncSessionTransaction as AsyncSessionTransaction diff --git a/lib/sqlalchemy/ext/asyncio/session.py b/lib/sqlalchemy/ext/asyncio/session.py index d819f546c..00fee9716 100644 --- a/lib/sqlalchemy/ext/asyncio/session.py +++ b/lib/sqlalchemy/ext/asyncio/session.py @@ -8,6 +8,7 @@ from __future__ import annotations import asyncio from typing import Any +from typing import Awaitable from typing import Callable from typing import Dict from typing import Generic @@ -73,6 +74,99 @@ _EXECUTE_OPTIONS = util.immutabledict({"prebuffer_rows": True}) _STREAM_OPTIONS = util.immutabledict({"stream_results": True}) +class AsyncAttrs: + """Mixin class which provides an awaitable accessor for all attributes. + + E.g.:: + + from __future__ import annotations + + from typing import List + + from sqlalchemy import ForeignKey + from sqlalchemy import func + from sqlalchemy.ext.asyncio import AsyncAttrs + from sqlalchemy.orm import DeclarativeBase + from sqlalchemy.orm import Mapped + from sqlalchemy.orm import mapped_column + from sqlalchemy.orm import relationship + + + class Base(AsyncAttrs, DeclarativeBase): + pass + + + class A(Base): + __tablename__ = "a" + + id: Mapped[int] = mapped_column(primary_key=True) + data: Mapped[str] + bs: Mapped[List[B]] = relationship() + + + class B(Base): + __tablename__ = "b" + id: Mapped[int] = mapped_column(primary_key=True) + a_id: Mapped[int] = mapped_column(ForeignKey("a.id")) + data: Mapped[str] + + In the above example, the :class:`_asyncio.AsyncAttrs` mixin is applied to + the declarative ``Base`` class where it takes effect for all subclasses. + This mixin adds a single new attribute + :attr:`_asyncio.AsyncAttrs.awaitable_attrs` to all classes, which will + yield the value of any attribute as an awaitable. This allows attributes + which may be subject to lazy loading or deferred / unexpiry loading to be + accessed such that IO can still be emitted:: + + a1 = (await async_session.scalars(select(A).where(A.id == 5))).one() + + # use the lazy loader on ``a1.bs`` via the ``.async_attrs`` + # interface, so that it may be awaited + for b1 in await a1.async_attrs.bs: + print(b1) + + The :attr:`_asyncio.AsyncAttrs.awaitable_attrs` performs a call against the + attribute that is approximately equivalent to using the + :meth:`_asyncio.AsyncSession.run_sync` method, e.g.:: + + for b1 in await async_session.run_sync(lambda sess: a1.bs): + print(b1) + + .. versionadded:: 2.0.13 + + .. seealso:: + + :ref:`asyncio_orm_avoid_lazyloads` + + """ + + class _AsyncAttrGetitem: + __slots__ = "_instance" + + def __init__(self, _instance: Any): + self._instance = _instance + + def __getattr__(self, name: str) -> Awaitable[Any]: + return greenlet_spawn(getattr, self._instance, name) + + @property + def awaitable_attrs(self) -> AsyncAttrs._AsyncAttrGetitem: + """provide a namespace of all attributes on this object wrapped + as awaitables. + + e.g.:: + + + a1 = (await async_session.scalars(select(A).where(A.id == 5))).one() + + some_attribute = await a1.async_attrs.some_deferred_attribute + some_collection = await a1.async_attrs.some_collection + + """ # noqa: E501 + + return AsyncAttrs._AsyncAttrGetitem(self) + + @util.create_proxy_methods( Session, ":class:`_orm.Session`", @@ -268,7 +362,7 @@ class AsyncSession(ReversibleProxy[Session]): to the database connection by running the given callable in a specially instrumented greenlet. - .. note:: + .. tip:: The provided callable is invoked inline within the asyncio event loop, and will block on traditional IO calls. IO within this @@ -277,6 +371,9 @@ class AsyncSession(ReversibleProxy[Session]): .. seealso:: + :class:`.AsyncAttrs` - a mixin for ORM mapped classes that provides + a similar feature more succinctly on a per-attribute basis + :meth:`.AsyncConnection.run_sync` :ref:`session_run_sync` diff --git a/lib/sqlalchemy/orm/attributes.py b/lib/sqlalchemy/orm/attributes.py index 69ddd3388..6a979219c 100644 --- a/lib/sqlalchemy/orm/attributes.py +++ b/lib/sqlalchemy/orm/attributes.py @@ -84,6 +84,9 @@ from ..sql import cache_key from ..sql import coercions from ..sql import roles from ..sql import visitors +from ..sql.cache_key import HasCacheKey +from ..sql.visitors import _TraverseInternalsType +from ..sql.visitors import InternalTraversal from ..util.typing import Literal from ..util.typing import Self from ..util.typing import TypeGuard @@ -326,13 +329,16 @@ class QueryableAttribute( # non-string keys. # ideally Proxy() would have a separate set of methods to deal # with this case. + entity_namespace = self._entity_namespace + assert isinstance(entity_namespace, HasCacheKey) + if self.key is _UNKNOWN_ATTR_KEY: # type: ignore[comparison-overlap] - annotations = {"entity_namespace": self._entity_namespace} + annotations = {"entity_namespace": entity_namespace} else: annotations = { "proxy_key": self.key, "proxy_owner": self._parententity, - "entity_namespace": self._entity_namespace, + "entity_namespace": entity_namespace, } ce = self.comparator.__clause_element__() @@ -558,13 +564,21 @@ class InstrumentedAttribute(QueryableAttribute[_T]): @dataclasses.dataclass(frozen=True) -class AdHocHasEntityNamespace: +class AdHocHasEntityNamespace(HasCacheKey): + _traverse_internals: ClassVar[_TraverseInternalsType] = [ + ("_entity_namespace", InternalTraversal.dp_has_cache_key), + ] + # py37 compat, no slots=True on dataclass - __slots__ = ("entity_namespace",) - entity_namespace: _ExternalEntityType[Any] + __slots__ = ("_entity_namespace",) + _entity_namespace: _InternalEntityType[Any] is_mapper: ClassVar[bool] = False is_aliased_class: ClassVar[bool] = False + @property + def entity_namespace(self): + return self._entity_namespace.entity_namespace + def create_proxied_attribute( descriptor: Any, @@ -638,7 +652,7 @@ def create_proxied_attribute( else: # used by hybrid attributes which try to remain # agnostic of any ORM concepts like mappers - return AdHocHasEntityNamespace(self.class_) + return AdHocHasEntityNamespace(self._parententity) @property def property(self): diff --git a/lib/sqlalchemy/orm/bulk_persistence.py b/lib/sqlalchemy/orm/bulk_persistence.py index cb416d69e..257d71db4 100644 --- a/lib/sqlalchemy/orm/bulk_persistence.py +++ b/lib/sqlalchemy/orm/bulk_persistence.py @@ -586,6 +586,7 @@ class ORMDMLState(AbstractORMCompileState): load_options = execution_options.get( "_sa_orm_load_options", QueryContext.default_load_options ) + querycontext = QueryContext( compile_state.from_statement_ctx, compile_state.select_statement, @@ -1140,6 +1141,7 @@ class BulkORMInsert(ORMDMLState, InsertDMLState): _return_defaults: bool = False _subject_mapper: Optional[Mapper[Any]] = None _autoflush: bool = True + _populate_existing: bool = False select_statement: Optional[FromStatement] = None @@ -1159,7 +1161,7 @@ class BulkORMInsert(ORMDMLState, InsertDMLState): execution_options, ) = BulkORMInsert.default_insert_options.from_execution_options( "_sa_orm_insert_options", - {"dml_strategy", "autoflush"}, + {"dml_strategy", "autoflush", "populate_existing"}, execution_options, statement._execution_options, ) @@ -1284,6 +1286,15 @@ class BulkORMInsert(ORMDMLState, InsertDMLState): if not bool(statement._returning): return result + if insert_options._populate_existing: + load_options = execution_options.get( + "_sa_orm_load_options", QueryContext.default_load_options + ) + load_options += {"_populate_existing": True} + execution_options = execution_options.union( + {"_sa_orm_load_options": load_options} + ) + return cls._return_orm_returning( session, statement, diff --git a/lib/sqlalchemy/orm/decl_api.py b/lib/sqlalchemy/orm/decl_api.py index 2f8289acf..a3a817162 100644 --- a/lib/sqlalchemy/orm/decl_api.py +++ b/lib/sqlalchemy/orm/decl_api.py @@ -666,7 +666,7 @@ class DeclarativeBase( from sqlalchemy import String from sqlalchemy.orm import DeclarativeBase - bigint = Annotated(int, "bigint") + bigint = Annotated[int, "bigint"] my_metadata = MetaData() class Base(DeclarativeBase): diff --git a/lib/sqlalchemy/orm/path_registry.py b/lib/sqlalchemy/orm/path_registry.py index e7084fbf6..e1cbd9313 100644 --- a/lib/sqlalchemy/orm/path_registry.py +++ b/lib/sqlalchemy/orm/path_registry.py @@ -119,6 +119,8 @@ class PathRegistry(HasCacheKey): is_property = False is_entity = False + is_unnatural: bool + path: _PathRepresentation natural_path: _PathRepresentation parent: Optional[PathRegistry] @@ -510,7 +512,12 @@ class PropRegistry(PathRegistry): # given MapperProperty's parent. insp = cast("_InternalEntityType[Any]", parent[-1]) natural_parent: AbstractEntityRegistry = parent - self.is_unnatural = False + + # inherit "is_unnatural" from the parent + if parent.parent.is_unnatural: + self.is_unnatural = True + else: + self.is_unnatural = False if not insp.is_aliased_class or insp._use_mapper_path: # type: ignore parent = natural_parent = parent.parent[prop.parent] @@ -570,6 +577,7 @@ class PropRegistry(PathRegistry): self.parent = parent self.path = parent.path + (prop,) self.natural_path = natural_parent.natural_path + (prop,) + self.has_entity = prop._links_to_entity if prop._is_relationship: if TYPE_CHECKING: @@ -674,7 +682,6 @@ class AbstractEntityRegistry(CreatesToken): # elif not parent.path and self.is_aliased_class: # self.natural_path = (self.entity._generate_cache_key()[0], ) else: - # self.natural_path = parent.natural_path + (entity, ) self.natural_path = self.path def _truncate_recursive(self) -> AbstractEntityRegistry: diff --git a/lib/sqlalchemy/orm/strategies.py b/lib/sqlalchemy/orm/strategies.py index 5581e5c7f..8e06c4f59 100644 --- a/lib/sqlalchemy/orm/strategies.py +++ b/lib/sqlalchemy/orm/strategies.py @@ -1063,6 +1063,7 @@ class LazyLoader( if extra_options: stmt._with_options += extra_options + stmt._compile_options += {"_current_path": effective_path} if use_get: diff --git a/lib/sqlalchemy/orm/strategy_options.py b/lib/sqlalchemy/orm/strategy_options.py index 48e69aef2..2e073f326 100644 --- a/lib/sqlalchemy/orm/strategy_options.py +++ b/lib/sqlalchemy/orm/strategy_options.py @@ -927,7 +927,9 @@ class _AbstractLoad(traversals.GenerativeOnTraversal, LoaderOption): ) -> Optional[_PathRepresentation]: i = -1 - for i, (c_token, p_token) in enumerate(zip(to_chop, path.path)): + for i, (c_token, p_token) in enumerate( + zip(to_chop, path.natural_path) + ): if isinstance(c_token, str): if i == 0 and c_token.endswith(f":{_DEFAULT_TOKEN}"): return to_chop @@ -942,36 +944,8 @@ class _AbstractLoad(traversals.GenerativeOnTraversal, LoaderOption): elif ( isinstance(c_token, InspectionAttr) and insp_is_mapper(c_token) - and ( - (insp_is_mapper(p_token) and c_token.isa(p_token)) - or ( - # a too-liberal check here to allow a path like - # A->A.bs->B->B.cs->C->C.ds, natural path, to chop - # against current path - # A->A.bs->B(B, B2)->B(B, B2)->cs, in an of_type() - # scenario which should only be occurring in a loader - # that is against a non-aliased lead element with - # single path. otherwise the - # "B" won't match into the B(B, B2). - # - # i>=2 prevents this check from proceeding for - # the first path element. - # - # if we could do away with the "natural_path" - # concept, we would not need guessy checks like this - # - # two conflicting tests for this comparison are: - # test_eager_relations.py-> - # test_lazyload_aliased_abs_bcs_two - # and - # test_of_type.py->test_all_subq_query - # - i >= 2 - and insp_is_aliased_class(p_token) - and p_token._is_with_polymorphic - and c_token in p_token.with_polymorphic_mappers - ) - ) + and insp_is_mapper(p_token) + and c_token.isa(p_token) ): continue @@ -1321,7 +1295,7 @@ class _WildcardLoad(_AbstractLoad): strategy: Optional[Tuple[Any, ...]] local_opts: _OptsType - path: Tuple[str, ...] + path: Union[Tuple[()], Tuple[str]] propagate_to_loaders = False def __init__(self) -> None: @@ -1366,6 +1340,7 @@ class _WildcardLoad(_AbstractLoad): it may be used as the sub-option of a :class:`_orm.Load` object. """ + assert self.path attr = self.path[0] if attr.endswith(_DEFAULT_TOKEN): attr = f"{attr.split(':')[0]}:{_WILDCARD_TOKEN}" @@ -1396,13 +1371,16 @@ class _WildcardLoad(_AbstractLoad): start_path: _PathRepresentation = self.path - # TODO: chop_path already occurs in loader.process_compile_state() - # so we will seek to simplify this if current_path: + # TODO: no cases in test suite where we actually get + # None back here new_path = self._chop_path(start_path, current_path) - if not new_path: + if new_path is None: return - start_path = new_path + + # chop_path does not actually "chop" a wildcard token path, + # just returns it + assert new_path == start_path # start_path is a single-token tuple assert start_path and len(start_path) == 1 @@ -1618,7 +1596,9 @@ class _LoadElement( """ - chopped_start_path = Load._chop_path(effective_path.path, current_path) + chopped_start_path = Load._chop_path( + effective_path.natural_path, current_path + ) if not chopped_start_path: return None diff --git a/lib/sqlalchemy/sql/__init__.py b/lib/sqlalchemy/sql/__init__.py index e5bbe65da..96cffed7d 100644 --- a/lib/sqlalchemy/sql/__init__.py +++ b/lib/sqlalchemy/sql/__init__.py @@ -7,6 +7,7 @@ from typing import Any from typing import TYPE_CHECKING +from ._typing import ColumnExpressionArgument as ColumnExpressionArgument from .base import Executable as Executable from .compiler import COLLECT_CARTESIAN_PRODUCTS as COLLECT_CARTESIAN_PRODUCTS from .compiler import FROM_LINTING as FROM_LINTING diff --git a/lib/sqlalchemy/sql/_typing.py b/lib/sqlalchemy/sql/_typing.py index 596493b7c..9e83c3f42 100644 --- a/lib/sqlalchemy/sql/_typing.py +++ b/lib/sqlalchemy/sql/_typing.py @@ -26,6 +26,7 @@ from .. import util from ..inspection import Inspectable from ..util.typing import Literal from ..util.typing import Protocol +from ..util.typing import TypeAlias if TYPE_CHECKING: from datetime import date @@ -175,7 +176,10 @@ _ColumnExpressionArgument = Union[ Callable[[], "ColumnElement[_T]"], "LambdaElement", ] -"""narrower "column expression" argument. +"See docs in public alias ColumnExpressionArgument." + +ColumnExpressionArgument: TypeAlias = _ColumnExpressionArgument[_T] +"""Narrower "column expression" argument. This type is used for all the other "column" kinds of expressions that typically represent a single SQL column expression, not a set of columns the @@ -184,10 +188,8 @@ way a table or ORM entity does. This includes ColumnElement, or ORM-mapped attributes that will have a `__clause_element__()` method, it also has the ExpressionElementRole overall which brings in the TextClause object also. - """ - _ColumnExpressionOrLiteralArgument = Union[Any, _ColumnExpressionArgument[_T]] _ColumnExpressionOrStrLabelArgument = Union[str, _ColumnExpressionArgument[_T]] diff --git a/lib/sqlalchemy/testing/fixtures.py b/lib/sqlalchemy/testing/fixtures.py index fc1fa1483..bff251b0f 100644 --- a/lib/sqlalchemy/testing/fixtures.py +++ b/lib/sqlalchemy/testing/fixtures.py @@ -13,6 +13,7 @@ import itertools import random import re import sys +from typing import Any import sqlalchemy as sa from . import assertions @@ -675,7 +676,7 @@ class MappedTest(TablesTest, assertions.AssertsExecutionResults): # 'once', 'each', None run_setup_mappers = "each" - classes = None + classes: Any = None @config.fixture(autouse=True, scope="class") def _setup_tables_test_class(self): diff --git a/lib/sqlalchemy/testing/requirements.py b/lib/sqlalchemy/testing/requirements.py index 7286cd81b..a2a9dc822 100644 --- a/lib/sqlalchemy/testing/requirements.py +++ b/lib/sqlalchemy/testing/requirements.py @@ -656,6 +656,13 @@ class SuiteRequirements(Requirements): return exclusions.closed() @property + def comment_reflection_full_unicode(self): + """Indicates if the database support table comment reflection in the + full unicode range, including emoji etc. + """ + return exclusions.closed() + + @property def constraint_comment_reflection(self): """indicates if the database support constraint on constraints and their reflection""" diff --git a/lib/sqlalchemy/testing/suite/test_insert.py b/lib/sqlalchemy/testing/suite/test_insert.py index d49eb3284..391503422 100644 --- a/lib/sqlalchemy/testing/suite/test_insert.py +++ b/lib/sqlalchemy/testing/suite/test_insert.py @@ -430,6 +430,8 @@ class ReturningTest(fixtures.TablesTest): this tests insertmanyvalues as well as decimal / floating point RETURNING types + TODO: this might be better in suite/test_types? + """ t = Table( diff --git a/lib/sqlalchemy/testing/suite/test_reflection.py b/lib/sqlalchemy/testing/suite/test_reflection.py index 8c26c265b..0162799d7 100644 --- a/lib/sqlalchemy/testing/suite/test_reflection.py +++ b/lib/sqlalchemy/testing/suite/test_reflection.py @@ -564,6 +564,7 @@ class ComponentReflectionTest(ComparesTables, OneConnectionTablesTest): sa.String(20), comment=r"""Comment types type speedily ' " \ '' Fun!""", ), + Column("d3", sa.String(42), comment="Comment\nwith\rescapes"), schema=schema, comment=r"""the test % ' " \ table comment""", ) @@ -572,6 +573,7 @@ class ComponentReflectionTest(ComparesTables, OneConnectionTablesTest): metadata, Column("data", sa.String(20)), schema=schema, + comment="no\nconstraints\rhas\fescaped\vcomment", ) if testing.requires.cross_schema_fk_reflection.enabled: @@ -831,7 +833,9 @@ class ComponentReflectionTest(ComparesTables, OneConnectionTablesTest): (schema, "comment_test"): { "text": r"""the test % ' " \ table comment""" }, - (schema, "no_constraints"): empty, + (schema, "no_constraints"): { + "text": "no\nconstraints\rhas\fescaped\vcomment" + }, (schema, "local_table"): empty, (schema, "remote_table"): empty, (schema, "remote_table_2"): empty, @@ -921,6 +925,7 @@ class ComponentReflectionTest(ComparesTables, OneConnectionTablesTest): "d2", comment=r"""Comment types type speedily ' " \ '' Fun!""", ), + col("d3", comment="Comment\nwith\rescapes"), ], (schema, "no_constraints"): [col("data")], (schema, "local_table"): [pk("id"), col("data"), col("remote_id")], @@ -2294,6 +2299,45 @@ class ComponentReflectionTest(ComparesTables, OneConnectionTablesTest): tables = [f"{schema}.{t}" for t in tables] eq_(sorted(m.tables), sorted(tables)) + @testing.requires.comment_reflection + def test_comments_unicode(self, connection, metadata): + Table( + "unicode_comments", + metadata, + Column("unicode", Integer, comment="é試蛇ẟΩ"), + Column("emoji", Integer, comment="☁️✨"), + comment="試蛇ẟΩ✨", + ) + + metadata.create_all(connection) + + insp = inspect(connection) + tc = insp.get_table_comment("unicode_comments") + eq_(tc, {"text": "試蛇ẟΩ✨"}) + + cols = insp.get_columns("unicode_comments") + value = {c["name"]: c["comment"] for c in cols} + exp = {"unicode": "é試蛇ẟΩ", "emoji": "☁️✨"} + eq_(value, exp) + + @testing.requires.comment_reflection_full_unicode + def test_comments_unicode_full(self, connection, metadata): + + Table( + "unicode_comments", + metadata, + Column("emoji", Integer, comment="🐍🧙🝝🧙♂️🧙♀️"), + comment="🎩🁰🝑🤷♀️🤷♂️", + ) + + metadata.create_all(connection) + + insp = inspect(connection) + tc = insp.get_table_comment("unicode_comments") + eq_(tc, {"text": "🎩🁰🝑🤷♀️🤷♂️"}) + c = insp.get_columns("unicode_comments")[0] + eq_({c["name"]: c["comment"]}, {"emoji": "🐍🧙🝝🧙♂️🧙♀️"}) + class TableNoColumnsTest(fixtures.TestBase): __requires__ = ("reflect_tables_no_columns",) diff --git a/lib/sqlalchemy/testing/suite/test_types.py b/lib/sqlalchemy/testing/suite/test_types.py index 72f1e8c10..ba2dda9ef 100644 --- a/lib/sqlalchemy/testing/suite/test_types.py +++ b/lib/sqlalchemy/testing/suite/test_types.py @@ -26,6 +26,7 @@ from ... import cast from ... import Date from ... import DateTime from ... import Float +from ... import Identity from ... import Integer from ... import JSON from ... import literal @@ -45,6 +46,7 @@ from ... import Unicode from ... import UnicodeText from ... import UUID from ... import Uuid +from ...dialects.postgresql import BYTEA from ...orm import declarative_base from ...orm import Session from ...sql.sqltypes import LargeBinary @@ -313,6 +315,68 @@ class BinaryTest(_LiteralRoundTripFixture, fixtures.TablesTest): row = connection.execute(select(binary_table.c.pickle_data)).first() eq_(row, ({"foo": [1, 2, 3], "bar": "bat"},)) + @testing.combinations( + ( + LargeBinary(), + b"this is binary", + ), + (LargeBinary(), b"7\xe7\x9f"), + (BYTEA(), b"7\xe7\x9f", testing.only_on("postgresql")), + argnames="type_,value", + ) + @testing.variation("sort_by_parameter_order", [True, False]) + @testing.variation("multiple_rows", [True, False]) + @testing.requires.insert_returning + def test_imv_returning( + self, + connection, + metadata, + sort_by_parameter_order, + type_, + value, + multiple_rows, + ): + """test #9739 (similar to #9701). + + this tests insertmanyvalues as well as binary + RETURNING types + + """ + t = Table( + "t", + metadata, + Column("id", Integer, Identity(), primary_key=True), + Column("value", type_), + ) + + t.create(connection) + + result = connection.execute( + t.insert().returning( + t.c.id, + t.c.value, + sort_by_parameter_order=bool(sort_by_parameter_order), + ), + [{"value": value} for i in range(10)] + if multiple_rows + else {"value": value}, + ) + + if multiple_rows: + i_range = range(1, 11) + else: + i_range = range(1, 2) + + eq_( + set(result), + {(id_, value) for id_ in i_range}, + ) + + eq_( + set(connection.scalars(select(t.c.value))), + {value}, + ) + class TextTest(_LiteralRoundTripFixture, fixtures.TablesTest): __requires__ = ("text_type",) diff --git a/lib/sqlalchemy/util/typing.py b/lib/sqlalchemy/util/typing.py index 9c38ae344..3ac67aad9 100644 --- a/lib/sqlalchemy/util/typing.py +++ b/lib/sqlalchemy/util/typing.py @@ -146,7 +146,7 @@ def de_stringify_annotation( original_annotation = annotation - if is_fwd_ref(annotation) and not annotation.__forward_evaluated__: + if is_fwd_ref(annotation): annotation = annotation.__forward_arg__ if isinstance(annotation, str): diff --git a/test/dialect/mysql/test_reflection.py b/test/dialect/mysql/test_reflection.py index f9975a973..a75c05f09 100644 --- a/test/dialect/mysql/test_reflection.py +++ b/test/dialect/mysql/test_reflection.py @@ -1368,6 +1368,29 @@ class ReflectionTest(fixtures.TestBase, AssertsCompiledSQL): }, ) + def test_reflect_comment_escapes(self, connection, metadata): + c = "\\ - \\\\ - \\0 - \\a - \\b - \\t - \\n - \\v - \\f - \\r" + Table("t", metadata, Column("c", Integer, comment=c), comment=c) + metadata.create_all(connection) + + insp = inspect(connection) + tc = insp.get_table_comment("t") + eq_(tc, {"text": c}) + col = insp.get_columns("t")[0] + eq_({col["name"]: col["comment"]}, {"c": c}) + + def test_reflect_comment_unicode(self, connection, metadata): + c = "☁️✨🐍🁰🝝" + c_exp = "☁️✨???" + Table("t", metadata, Column("c", Integer, comment=c), comment=c) + metadata.create_all(connection) + + insp = inspect(connection) + tc = insp.get_table_comment("t") + eq_(tc, {"text": c_exp}) + col = insp.get_columns("t")[0] + eq_({col["name"]: col["comment"]}, {"c": c_exp}) + class RawReflectionTest(fixtures.TestBase): def setup_test(self): diff --git a/test/ext/asyncio/test_session_py3k.py b/test/ext/asyncio/test_session_py3k.py index 36135a43d..a374728f7 100644 --- a/test/ext/asyncio/test_session_py3k.py +++ b/test/ext/asyncio/test_session_py3k.py @@ -1,20 +1,31 @@ +from __future__ import annotations + +from typing import List +from typing import Optional + from sqlalchemy import Column from sqlalchemy import event from sqlalchemy import exc from sqlalchemy import ForeignKey from sqlalchemy import func +from sqlalchemy import Identity from sqlalchemy import inspect from sqlalchemy import Integer from sqlalchemy import select from sqlalchemy import Sequence +from sqlalchemy import String from sqlalchemy import Table from sqlalchemy import testing from sqlalchemy import update from sqlalchemy.ext.asyncio import async_object_session from sqlalchemy.ext.asyncio import async_sessionmaker +from sqlalchemy.ext.asyncio import AsyncAttrs from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import exc as async_exc from sqlalchemy.ext.asyncio.base import ReversibleProxy +from sqlalchemy.orm import DeclarativeBase +from sqlalchemy.orm import Mapped +from sqlalchemy.orm import mapped_column from sqlalchemy.orm import relationship from sqlalchemy.orm import selectinload from sqlalchemy.orm import Session @@ -24,6 +35,7 @@ from sqlalchemy.testing import config from sqlalchemy.testing import engines from sqlalchemy.testing import eq_ from sqlalchemy.testing import expect_raises_message +from sqlalchemy.testing import fixtures from sqlalchemy.testing import is_ from sqlalchemy.testing import is_true from sqlalchemy.testing import mock @@ -45,6 +57,12 @@ class AsyncFixture(_AsyncFixture, _fixtures.FixtureTest): def async_engine(self): return engines.testing_engine(asyncio=True, transfer_staticpool=True) + # TODO: this seems to cause deadlocks in + # OverrideSyncSession for some reason + # @testing.fixture + # def async_engine(self, async_testing_engine): + # return async_testing_engine(transfer_staticpool=True) + @testing.fixture def async_session(self, async_engine): return AsyncSession(async_engine) @@ -1005,3 +1023,103 @@ class OverrideSyncSession(AsyncFixture): is_true(not isinstance(ass.sync_session, _MySession)) is_(ass.sync_session_class, Session) + + +class AsyncAttrsTest( + testing.AssertsExecutionResults, _AsyncFixture, fixtures.TestBase +): + __requires__ = ("async_dialect",) + + @config.fixture + def decl_base(self, metadata): + _md = metadata + + class Base(fixtures.ComparableEntity, AsyncAttrs, DeclarativeBase): + metadata = _md + type_annotation_map = { + str: String().with_variant( + String(50), "mysql", "mariadb", "oracle" + ) + } + + yield Base + Base.registry.dispose() + + @testing.fixture + def async_engine(self, async_testing_engine): + yield async_testing_engine(transfer_staticpool=True) + + @testing.fixture + def ab_fixture(self, decl_base): + class A(decl_base): + __tablename__ = "a" + + id: Mapped[int] = mapped_column(Identity(), primary_key=True) + data: Mapped[Optional[str]] + bs: Mapped[List[B]] = relationship(order_by=lambda: B.id) + + class B(decl_base): + __tablename__ = "b" + id: Mapped[int] = mapped_column(Identity(), primary_key=True) + a_id: Mapped[int] = mapped_column(ForeignKey("a.id")) + data: Mapped[Optional[str]] + + decl_base.metadata.create_all(testing.db) + + return A, B + + @async_test + async def test_lazyloaders(self, async_engine, ab_fixture): + A, B = ab_fixture + + async with AsyncSession(async_engine) as session: + b1, b2, b3 = B(data="b1"), B(data="b2"), B(data="b3") + a1 = A(data="a1", bs=[b1, b2, b3]) + session.add(a1) + + await session.commit() + + assert inspect(a1).expired + + with self.assert_statement_count(async_engine.sync_engine, 1): + eq_(await a1.awaitable_attrs.data, "a1") + + with self.assert_statement_count(async_engine.sync_engine, 1): + eq_(await a1.awaitable_attrs.bs, [b1, b2, b3]) + + # now it's loaded, lazy loading not used anymore + eq_(a1.bs, [b1, b2, b3]) + + @async_test + async def test_it_didnt_load_but_is_ok(self, async_engine, ab_fixture): + A, B = ab_fixture + + async with AsyncSession(async_engine) as session: + b1, b2, b3 = B(data="b1"), B(data="b2"), B(data="b3") + a1 = A(data="a1", bs=[b1, b2, b3]) + session.add(a1) + + await session.commit() + + async with AsyncSession(async_engine) as session: + a1 = ( + await session.scalars(select(A).options(selectinload(A.bs))) + ).one() + + with self.assert_statement_count(async_engine.sync_engine, 0): + eq_(await a1.awaitable_attrs.bs, [b1, b2, b3]) + + @async_test + async def test_the_famous_lazyloader_gotcha( + self, async_engine, ab_fixture + ): + A, B = ab_fixture + + async with AsyncSession(async_engine) as session: + a1 = A(data="a1") + session.add(a1) + + await session.flush() + + with self.assert_statement_count(async_engine.sync_engine, 1): + eq_(await a1.awaitable_attrs.bs, []) diff --git a/test/orm/dml/test_bulk_statements.py b/test/orm/dml/test_bulk_statements.py index ab03b251d..ca4e857ed 100644 --- a/test/orm/dml/test_bulk_statements.py +++ b/test/orm/dml/test_bulk_statements.py @@ -302,6 +302,78 @@ class InsertStmtTest(testing.AssertsExecutionResults, fixtures.TestBase): else: eq_(result.first(), (10, expected_qs[0])) + @testing.variation("populate_existing", [True, False]) + @testing.requires.provisioned_upsert + @testing.requires.update_returning + def test_upsert_populate_existing(self, decl_base, populate_existing): + """test #9742""" + + class Employee(ComparableEntity, decl_base): + __tablename__ = "employee" + + uuid: Mapped[uuid.UUID] = mapped_column(primary_key=True) + user_name: Mapped[str] = mapped_column(nullable=False) + + decl_base.metadata.create_all(testing.db) + s = fixture_session() + + uuid1 = uuid.uuid4() + uuid2 = uuid.uuid4() + e1 = Employee(uuid=uuid1, user_name="e1 old name") + e2 = Employee(uuid=uuid2, user_name="e2 old name") + s.add_all([e1, e2]) + s.flush() + + stmt = provision.upsert( + config, + Employee, + (Employee,), + set_lambda=lambda inserted: {"user_name": inserted.user_name}, + ).values( + [ + dict(uuid=uuid1, user_name="e1 new name"), + dict(uuid=uuid2, user_name="e2 new name"), + ] + ) + if populate_existing: + rows = s.scalars( + stmt, execution_options={"populate_existing": True} + ) + # SPECIAL: before we actually receive the returning rows, + # the existing objects have not been updated yet + eq_(e1.user_name, "e1 old name") + eq_(e2.user_name, "e2 old name") + + eq_( + set(rows), + { + Employee(uuid=uuid1, user_name="e1 new name"), + Employee(uuid=uuid2, user_name="e2 new name"), + }, + ) + + # now they are updated + eq_(e1.user_name, "e1 new name") + eq_(e2.user_name, "e2 new name") + else: + # no populate existing + rows = s.scalars(stmt) + eq_(e1.user_name, "e1 old name") + eq_(e2.user_name, "e2 old name") + eq_( + set(rows), + { + Employee(uuid=uuid1, user_name="e1 old name"), + Employee(uuid=uuid2, user_name="e2 old name"), + }, + ) + eq_(e1.user_name, "e1 old name") + eq_(e2.user_name, "e2 old name") + s.commit() + s.expire_all() + eq_(e1.user_name, "e1 new name") + eq_(e2.user_name, "e2 new name") + class UpdateStmtTest(fixtures.TestBase): __backend__ = True diff --git a/test/orm/inheritance/test_assorted_poly.py b/test/orm/inheritance/test_assorted_poly.py index 4bebc9b10..a40a9ae74 100644 --- a/test/orm/inheritance/test_assorted_poly.py +++ b/test/orm/inheritance/test_assorted_poly.py @@ -3,6 +3,10 @@ These are generally tests derived from specific user issues. """ +from __future__ import annotations + +from typing import Optional + from sqlalchemy import exists from sqlalchemy import ForeignKey from sqlalchemy import func @@ -17,10 +21,14 @@ from sqlalchemy.orm import aliased from sqlalchemy.orm import class_mapper from sqlalchemy.orm import column_property from sqlalchemy.orm import contains_eager +from sqlalchemy.orm import immediateload from sqlalchemy.orm import join from sqlalchemy.orm import joinedload +from sqlalchemy.orm import Mapped +from sqlalchemy.orm import mapped_column from sqlalchemy.orm import polymorphic_union from sqlalchemy.orm import relationship +from sqlalchemy.orm import selectinload from sqlalchemy.orm import Session from sqlalchemy.orm import sessionmaker from sqlalchemy.orm import with_polymorphic @@ -2554,3 +2562,206 @@ class Issue8168Test(AssertsCompiledSQL, fixtures.TestBase): ) else: scenario.fail() + + +class PolyIntoSelfReferentialTest( + fixtures.DeclarativeMappedTest, AssertsExecutionResults +): + """test for #9715""" + + @classmethod + def setup_classes(cls): + Base = cls.DeclarativeBasic + + class A(Base): + __tablename__ = "a" + + id: Mapped[int] = mapped_column( + primary_key=True, autoincrement=True + ) + + rel_id: Mapped[int] = mapped_column(ForeignKey("related.id")) + + related = relationship("Related") + + class Related(Base): + __tablename__ = "related" + + id: Mapped[int] = mapped_column( + primary_key=True, autoincrement=True + ) + rel_data: Mapped[str] + type: Mapped[str] = mapped_column() + + other_related_id: Mapped[int] = mapped_column( + ForeignKey("other_related.id") + ) + + other_related = relationship("OtherRelated") + + __mapper_args__ = { + "polymorphic_identity": "related", + "polymorphic_on": type, + } + + class SubRelated(Related): + __tablename__ = "sub_related" + + id: Mapped[int] = mapped_column( + ForeignKey("related.id"), primary_key=True + ) + sub_rel_data: Mapped[str] + + __mapper_args__ = {"polymorphic_identity": "sub_related"} + + class OtherRelated(Base): + __tablename__ = "other_related" + + id: Mapped[int] = mapped_column( + primary_key=True, autoincrement=True + ) + name: Mapped[str] + + parent_id: Mapped[Optional[int]] = mapped_column( + ForeignKey("other_related.id") + ) + parent = relationship("OtherRelated", lazy="raise", remote_side=id) + + @classmethod + def insert_data(cls, connection): + A, SubRelated, OtherRelated = cls.classes( + "A", "SubRelated", "OtherRelated" + ) + + with Session(connection) as sess: + + grandparent_otherrel1 = OtherRelated(name="GP1") + grandparent_otherrel2 = OtherRelated(name="GP2") + + parent_otherrel1 = OtherRelated( + name="P1", parent=grandparent_otherrel1 + ) + parent_otherrel2 = OtherRelated( + name="P2", parent=grandparent_otherrel2 + ) + + otherrel1 = OtherRelated(name="A1", parent=parent_otherrel1) + otherrel3 = OtherRelated(name="A2", parent=parent_otherrel2) + + address1 = SubRelated( + rel_data="ST1", other_related=otherrel1, sub_rel_data="w1" + ) + address3 = SubRelated( + rel_data="ST2", other_related=otherrel3, sub_rel_data="w2" + ) + + a1 = A(related=address1) + a2 = A(related=address3) + + sess.add_all([a1, a2]) + sess.commit() + + def _run_load(self, *opt): + A = self.classes.A + stmt = select(A).options(*opt) + + sess = fixture_session() + all_a = sess.scalars(stmt).all() + + sess.close() + + with self.assert_statement_count(testing.db, 0): + for a1 in all_a: + d1 = a1.related + d2 = d1.other_related + d3 = d2.parent + d4 = d3.parent + assert d4.name in ("GP1", "GP2") + + @testing.variation("use_workaround", [True, False]) + def test_workaround(self, use_workaround): + A, Related, SubRelated, OtherRelated = self.classes( + "A", "Related", "SubRelated", "OtherRelated" + ) + + related = with_polymorphic(Related, [SubRelated], flat=True) + + opt = [ + ( + joinedload(A.related.of_type(related)) + .joinedload(related.other_related) + .joinedload( + OtherRelated.parent, + ) + ) + ] + if use_workaround: + opt.append( + joinedload( + A.related, + Related.other_related, + OtherRelated.parent, + OtherRelated.parent, + ) + ) + else: + opt[0] = opt[0].joinedload(OtherRelated.parent) + + self._run_load(*opt) + + @testing.combinations( + (("joined", "joined", "joined", "joined"),), + (("selectin", "selectin", "selectin", "selectin"),), + (("selectin", "selectin", "joined", "joined"),), + (("selectin", "selectin", "joined", "selectin"),), + (("joined", "selectin", "joined", "selectin"),), + # TODO: immediateload (and lazyload) do not support the target item + # being a with_polymorphic. this seems to be a limitation in the + # current_path logic + # (("immediate", "joined", "joined", "joined"),), + argnames="loaders", + ) + @testing.variation("use_wpoly", [True, False]) + def test_all_load(self, loaders, use_wpoly): + A, Related, SubRelated, OtherRelated = self.classes( + "A", "Related", "SubRelated", "OtherRelated" + ) + + if use_wpoly: + related = with_polymorphic(Related, [SubRelated], flat=True) + else: + related = SubRelated + + opt = None + for i, (load_type, element) in enumerate( + zip( + loaders, + [ + A.related.of_type(related), + related.other_related, + OtherRelated.parent, + OtherRelated.parent, + ], + ) + ): + if i == 0: + if load_type == "joined": + opt = joinedload(element) + elif load_type == "selectin": + opt = selectinload(element) + elif load_type == "immediate": + opt = immediateload(element) + else: + assert False + else: + assert opt is not None + if load_type == "joined": + opt = opt.joinedload(element) + elif load_type == "selectin": + opt = opt.selectinload(element) + elif load_type == "immediate": + opt = opt.immediateload(element) + else: + assert False + + self._run_load(opt) diff --git a/test/orm/test_cache_key.py b/test/orm/test_cache_key.py index f1c3e6a54..884baed62 100644 --- a/test/orm/test_cache_key.py +++ b/test/orm/test_cache_key.py @@ -2,6 +2,7 @@ import random import sqlalchemy as sa from sqlalchemy import Column +from sqlalchemy import column from sqlalchemy import func from sqlalchemy import inspect from sqlalchemy import Integer @@ -16,6 +17,7 @@ from sqlalchemy import true from sqlalchemy import update from sqlalchemy import util from sqlalchemy.ext.declarative import ConcreteBase +from sqlalchemy.ext.hybrid import hybrid_property from sqlalchemy.orm import aliased from sqlalchemy.orm import Bundle from sqlalchemy.orm import defaultload @@ -785,6 +787,55 @@ class PolyCacheKeyTest(fixtures.CacheKeyFixture, _poly_fixtures._Polymorphic): compare_values=True, ) + @testing.variation( + "exprtype", ["plain_column", "self_standing_case", "case_w_columns"] + ) + def test_hybrid_w_case_ac(self, decl_base, exprtype): + """test #9728""" + + class Employees(decl_base): + __tablename__ = "employees" + id = Column(String(128), primary_key=True) + first_name = Column(String(length=64)) + + @hybrid_property + def name(self): + return self.first_name + + @name.expression + def name( + cls, + ): + if exprtype.plain_column: + return cls.first_name + elif exprtype.self_standing_case: + return case( + (column("x") == 1, column("q")), + else_=column("q"), + ) + elif exprtype.case_w_columns: + return case( + (column("x") == 1, column("q")), + else_=cls.first_name, + ) + else: + exprtype.fail() + + def go1(): + employees_2 = aliased(Employees, name="employees_2") + stmt = select(employees_2.name) + return stmt + + def go2(): + employees_2 = aliased(Employees, name="employees_2") + stmt = select(employees_2) + return stmt + + self._run_cache_key_fixture( + lambda: stmt_20(go1(), go2()), + compare_values=True, + ) + class RoundTripTest(QueryTest, AssertsCompiledSQL): __dialect__ = "default" diff --git a/test/profiles.txt b/test/profiles.txt index fd229ed03..5bdd5ac07 100644 --- a/test/profiles.txt +++ b/test/profiles.txt @@ -20,8 +20,8 @@ test.aaa_profiling.test_compiler.CompileTest.test_insert x86_64_linux_cpython_3. test.aaa_profiling.test_compiler.CompileTest.test_insert x86_64_linux_cpython_3.10_mssql_pyodbc_dbapiunicode_cextensions 75 test.aaa_profiling.test_compiler.CompileTest.test_insert x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_cextensions 75 test.aaa_profiling.test_compiler.CompileTest.test_insert x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_nocextensions 75 -test.aaa_profiling.test_compiler.CompileTest.test_insert x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 75 -test.aaa_profiling.test_compiler.CompileTest.test_insert x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 75 +test.aaa_profiling.test_compiler.CompileTest.test_insert x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 77 +test.aaa_profiling.test_compiler.CompileTest.test_insert x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 77 test.aaa_profiling.test_compiler.CompileTest.test_insert x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_cextensions 75 test.aaa_profiling.test_compiler.CompileTest.test_insert x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_nocextensions 75 @@ -34,8 +34,8 @@ test.aaa_profiling.test_compiler.CompileTest.test_select x86_64_linux_cpython_3. test.aaa_profiling.test_compiler.CompileTest.test_select x86_64_linux_cpython_3.10_mssql_pyodbc_dbapiunicode_cextensions 195 test.aaa_profiling.test_compiler.CompileTest.test_select x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_cextensions 195 test.aaa_profiling.test_compiler.CompileTest.test_select x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_nocextensions 195 -test.aaa_profiling.test_compiler.CompileTest.test_select x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 195 -test.aaa_profiling.test_compiler.CompileTest.test_select x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 195 +test.aaa_profiling.test_compiler.CompileTest.test_select x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 219 +test.aaa_profiling.test_compiler.CompileTest.test_select x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 219 test.aaa_profiling.test_compiler.CompileTest.test_select x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_cextensions 193 test.aaa_profiling.test_compiler.CompileTest.test_select x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_nocextensions 193 @@ -48,8 +48,8 @@ test.aaa_profiling.test_compiler.CompileTest.test_select_labels x86_64_linux_cpy test.aaa_profiling.test_compiler.CompileTest.test_select_labels x86_64_linux_cpython_3.10_mssql_pyodbc_dbapiunicode_cextensions 219 test.aaa_profiling.test_compiler.CompileTest.test_select_labels x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_cextensions 219 test.aaa_profiling.test_compiler.CompileTest.test_select_labels x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_nocextensions 219 -test.aaa_profiling.test_compiler.CompileTest.test_select_labels x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 219 -test.aaa_profiling.test_compiler.CompileTest.test_select_labels x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 219 +test.aaa_profiling.test_compiler.CompileTest.test_select_labels x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 243 +test.aaa_profiling.test_compiler.CompileTest.test_select_labels x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 243 test.aaa_profiling.test_compiler.CompileTest.test_select_labels x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_cextensions 217 test.aaa_profiling.test_compiler.CompileTest.test_select_labels x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_nocextensions 217 @@ -62,8 +62,8 @@ test.aaa_profiling.test_compiler.CompileTest.test_update x86_64_linux_cpython_3. test.aaa_profiling.test_compiler.CompileTest.test_update x86_64_linux_cpython_3.10_mssql_pyodbc_dbapiunicode_cextensions 81 test.aaa_profiling.test_compiler.CompileTest.test_update x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_cextensions 81 test.aaa_profiling.test_compiler.CompileTest.test_update x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_nocextensions 81 -test.aaa_profiling.test_compiler.CompileTest.test_update x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 81 -test.aaa_profiling.test_compiler.CompileTest.test_update x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 81 +test.aaa_profiling.test_compiler.CompileTest.test_update x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 86 +test.aaa_profiling.test_compiler.CompileTest.test_update x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 86 test.aaa_profiling.test_compiler.CompileTest.test_update x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_cextensions 79 test.aaa_profiling.test_compiler.CompileTest.test_update x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_nocextensions 79 @@ -76,8 +76,8 @@ test.aaa_profiling.test_compiler.CompileTest.test_update_whereclause x86_64_linu test.aaa_profiling.test_compiler.CompileTest.test_update_whereclause x86_64_linux_cpython_3.10_mssql_pyodbc_dbapiunicode_cextensions 180 test.aaa_profiling.test_compiler.CompileTest.test_update_whereclause x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_cextensions 180 test.aaa_profiling.test_compiler.CompileTest.test_update_whereclause x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_nocextensions 180 -test.aaa_profiling.test_compiler.CompileTest.test_update_whereclause x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 180 -test.aaa_profiling.test_compiler.CompileTest.test_update_whereclause x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 180 +test.aaa_profiling.test_compiler.CompileTest.test_update_whereclause x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 184 +test.aaa_profiling.test_compiler.CompileTest.test_update_whereclause x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 187 test.aaa_profiling.test_compiler.CompileTest.test_update_whereclause x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_cextensions 180 test.aaa_profiling.test_compiler.CompileTest.test_update_whereclause x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_nocextensions 180 @@ -328,8 +328,8 @@ test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_connection_execute test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_connection_execute x86_64_linux_cpython_3.10_mssql_pyodbc_dbapiunicode_nocextensions 53 test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_connection_execute x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_cextensions 53 test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_connection_execute x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_nocextensions 53 -test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_connection_execute x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 53 -test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_connection_execute x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 53 +test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_connection_execute x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 55 +test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_connection_execute x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 55 test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_connection_execute x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_cextensions 53 test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_connection_execute x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_nocextensions 53 @@ -343,8 +343,8 @@ test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_engine_execute x86_ test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_engine_execute x86_64_linux_cpython_3.10_mssql_pyodbc_dbapiunicode_nocextensions 106 test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_engine_execute x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_cextensions 106 test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_engine_execute x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_nocextensions 106 -test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_engine_execute x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 106 -test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_engine_execute x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 106 +test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_engine_execute x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 110 +test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_engine_execute x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 110 test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_engine_execute x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_cextensions 105 test.aaa_profiling.test_resultset.ExecutionTest.test_minimal_engine_execute x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_nocextensions 105 @@ -373,8 +373,8 @@ test.aaa_profiling.test_resultset.ResultSetTest.test_fetch_by_key_mappings x86_6 test.aaa_profiling.test_resultset.ResultSetTest.test_fetch_by_key_mappings x86_64_linux_cpython_3.10_mssql_pyodbc_dbapiunicode_nocextensions 15601 test.aaa_profiling.test_resultset.ResultSetTest.test_fetch_by_key_mappings x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_cextensions 2637 test.aaa_profiling.test_resultset.ResultSetTest.test_fetch_by_key_mappings x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_nocextensions 15641 -test.aaa_profiling.test_resultset.ResultSetTest.test_fetch_by_key_mappings x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 2592 -test.aaa_profiling.test_resultset.ResultSetTest.test_fetch_by_key_mappings x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 25595 +test.aaa_profiling.test_resultset.ResultSetTest.test_fetch_by_key_mappings x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 2651 +test.aaa_profiling.test_resultset.ResultSetTest.test_fetch_by_key_mappings x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 14655 test.aaa_profiling.test_resultset.ResultSetTest.test_fetch_by_key_mappings x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_cextensions 2539 test.aaa_profiling.test_resultset.ResultSetTest.test_fetch_by_key_mappings x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_nocextensions 14614 @@ -404,7 +404,7 @@ test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[False-1] x86_64 test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[False-1] x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_cextensions 14 test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[False-1] x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_nocextensions 16 test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[False-1] x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 14 -test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[False-1] x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 16 +test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[False-1] x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 15 test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[False-1] x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_cextensions 14 test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[False-1] x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_nocextensions 15 @@ -419,7 +419,7 @@ test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[False-2] x86_64 test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[False-2] x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_cextensions 14 test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[False-2] x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_nocextensions 16 test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[False-2] x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 14 -test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[False-2] x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 16 +test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[False-2] x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 15 test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[False-2] x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_cextensions 14 test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[False-2] x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_nocextensions 15 @@ -434,7 +434,7 @@ test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[True-1] x86_64_ test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[True-1] x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_cextensions 17 test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[True-1] x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_nocextensions 19 test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[True-1] x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 17 -test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[True-1] x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 19 +test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[True-1] x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 18 test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[True-1] x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_cextensions 17 test.aaa_profiling.test_resultset.ResultSetTest.test_one_or_none[True-1] x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_nocextensions 18 @@ -448,8 +448,8 @@ test.aaa_profiling.test_resultset.ResultSetTest.test_raw_string x86_64_linux_cpy test.aaa_profiling.test_resultset.ResultSetTest.test_raw_string x86_64_linux_cpython_3.10_mssql_pyodbc_dbapiunicode_nocextensions 6269 test.aaa_profiling.test_resultset.ResultSetTest.test_raw_string x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_cextensions 361 test.aaa_profiling.test_resultset.ResultSetTest.test_raw_string x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_nocextensions 6361 -test.aaa_profiling.test_resultset.ResultSetTest.test_raw_string x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 291 -test.aaa_profiling.test_resultset.ResultSetTest.test_raw_string x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 6291 +test.aaa_profiling.test_resultset.ResultSetTest.test_raw_string x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 301 +test.aaa_profiling.test_resultset.ResultSetTest.test_raw_string x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 5301 test.aaa_profiling.test_resultset.ResultSetTest.test_raw_string x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_cextensions 257 test.aaa_profiling.test_resultset.ResultSetTest.test_raw_string x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_nocextensions 5277 @@ -463,8 +463,8 @@ test.aaa_profiling.test_resultset.ResultSetTest.test_raw_unicode x86_64_linux_cp test.aaa_profiling.test_resultset.ResultSetTest.test_raw_unicode x86_64_linux_cpython_3.10_mssql_pyodbc_dbapiunicode_nocextensions 6269 test.aaa_profiling.test_resultset.ResultSetTest.test_raw_unicode x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_cextensions 361 test.aaa_profiling.test_resultset.ResultSetTest.test_raw_unicode x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_nocextensions 6361 -test.aaa_profiling.test_resultset.ResultSetTest.test_raw_unicode x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 291 -test.aaa_profiling.test_resultset.ResultSetTest.test_raw_unicode x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 6291 +test.aaa_profiling.test_resultset.ResultSetTest.test_raw_unicode x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 301 +test.aaa_profiling.test_resultset.ResultSetTest.test_raw_unicode x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 5301 test.aaa_profiling.test_resultset.ResultSetTest.test_raw_unicode x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_cextensions 257 test.aaa_profiling.test_resultset.ResultSetTest.test_raw_unicode x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_nocextensions 5277 @@ -478,8 +478,8 @@ test.aaa_profiling.test_resultset.ResultSetTest.test_string x86_64_linux_cpython test.aaa_profiling.test_resultset.ResultSetTest.test_string x86_64_linux_cpython_3.10_mssql_pyodbc_dbapiunicode_nocextensions 6594 test.aaa_profiling.test_resultset.ResultSetTest.test_string x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_cextensions 630 test.aaa_profiling.test_resultset.ResultSetTest.test_string x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_nocextensions 6634 -test.aaa_profiling.test_resultset.ResultSetTest.test_string x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 585 -test.aaa_profiling.test_resultset.ResultSetTest.test_string x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 6589 +test.aaa_profiling.test_resultset.ResultSetTest.test_string x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 642 +test.aaa_profiling.test_resultset.ResultSetTest.test_string x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 5646 test.aaa_profiling.test_resultset.ResultSetTest.test_string x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_cextensions 532 test.aaa_profiling.test_resultset.ResultSetTest.test_string x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_nocextensions 5605 @@ -493,7 +493,7 @@ test.aaa_profiling.test_resultset.ResultSetTest.test_unicode x86_64_linux_cpytho test.aaa_profiling.test_resultset.ResultSetTest.test_unicode x86_64_linux_cpython_3.10_mssql_pyodbc_dbapiunicode_nocextensions 6594 test.aaa_profiling.test_resultset.ResultSetTest.test_unicode x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_cextensions 630 test.aaa_profiling.test_resultset.ResultSetTest.test_unicode x86_64_linux_cpython_3.10_oracle_cx_oracle_dbapiunicode_nocextensions 6634 -test.aaa_profiling.test_resultset.ResultSetTest.test_unicode x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 585 -test.aaa_profiling.test_resultset.ResultSetTest.test_unicode x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 6589 +test.aaa_profiling.test_resultset.ResultSetTest.test_unicode x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_cextensions 642 +test.aaa_profiling.test_resultset.ResultSetTest.test_unicode x86_64_linux_cpython_3.10_postgresql_psycopg2_dbapiunicode_nocextensions 5646 test.aaa_profiling.test_resultset.ResultSetTest.test_unicode x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_cextensions 532 test.aaa_profiling.test_resultset.ResultSetTest.test_unicode x86_64_linux_cpython_3.10_sqlite_pysqlite_dbapiunicode_nocextensions 5605 diff --git a/test/requirements.py b/test/requirements.py index 2023428b8..436f7d3d7 100644 --- a/test/requirements.py +++ b/test/requirements.py @@ -167,6 +167,10 @@ class DefaultRequirements(SuiteRequirements): return only_on(["postgresql", "mysql", "mariadb", "oracle", "mssql"]) @property + def comment_reflection_full_unicode(self): + return only_on(["postgresql", "oracle", "mssql"]) + + @property def constraint_comment_reflection(self): return only_on(["postgresql"]) |