diff options
| author | Mike Bayer <mike_mp@zzzcomputing.com> | 2020-08-31 11:46:55 -0400 |
|---|---|---|
| committer | Mike Bayer <mike_mp@zzzcomputing.com> | 2020-09-10 17:53:53 -0400 |
| commit | 450f5c0d6519a439f4025c3892fe4cf3ee2d892c (patch) | |
| tree | 1f3f2467306304a5e9ccb25f10bfdf9989327ae2 /doc | |
| parent | 96bb6dc56d1da2b4fa30afd08ac4dfa665752913 (diff) | |
| download | sqlalchemy-450f5c0d6519a439f4025c3892fe4cf3ee2d892c.tar.gz | |
Build out new declarative systems; deprecate mapper()
The ORM Declarative system is now unified into the ORM itself, with new
import spaces under ``sqlalchemy.orm`` and new kinds of mappings. Support
for decorator-based mappings without using a base class, support for
classical style-mapper() calls that have access to the declarative class
registry for relationships, and full integration of Declarative with 3rd
party class attribute systems like ``dataclasses`` and ``attrs`` is now
supported.
Fixes: #5508
Change-Id: I130b2b6edff6450bfe8a3e6baa099ff04b5471ff
Diffstat (limited to 'doc')
33 files changed, 2371 insertions, 1473 deletions
diff --git a/doc/build/changelog/migration_14.rst b/doc/build/changelog/migration_14.rst index 91b856605..ca51ea26a 100644 --- a/doc/build/changelog/migration_14.rst +++ b/doc/build/changelog/migration_14.rst @@ -224,6 +224,117 @@ driven in order to support this new feature. :ticket:`4808` :ticket:`5004` +.. _change_5508: + +Declarative is now integrated into the ORM with new features +------------------------------------------------------------- + +After ten years or so of popularity, the ``sqlalchemy.ext.declarative`` +package is now integrated into the ``sqlalchemy.orm`` namespace, with the +exception of the declarative "extension" classes which remain as Declarative +extensions. + +The new classes added to ``sqlalchemy.orm`` include: + +* :class:`_orm.registry` - a new class that supercedes the role of the + "declarative base" class, serving as a registry of mapped classes which + can be referenced via string name within :func:`_orm.relationship` calls + and is agnostic of the style in which any particular class was mapped. + +* :func:`_orm.declarative_base` - this is the same declarative base class that + has been in use throughout the span of the declarative system, except it now + references a :class:`_orm.registry` object internally and is implemented + by the :meth:`_orm.registry.generate_base` method which can be invoked + from a :class:`_orm.registry` directly. The :func:`_orm.declarative_base` + function creates this registry automatically so there is no impact on + existing code. The ``sqlalchemy.ext.declarative.declarative_base`` name + is still present, emitting a 2.0 deprecation warning when + :ref:`2.0 deprecations mode <deprecation_20_mode>` is enabled. + +* :func:`_orm.declared_attr` - the same "declared attr" function call now + part of ``sqlalchemy.orm``. The ``sqlalchemy.ext.declarative.declared_attr`` + name is still present, emitting a 2.0 deprecation warning when + :ref:`2.0 deprecations mode <deprecation_20_mode>` is enabled. + +* Other names moved into ``sqlalchemy.orm`` include :func:`_orm.has_inherited_table`, + :func:`_orm.synonym_for`, :class:`_orm.DeclarativeMeta`, :func:`_orm.as_declarative`. + +In addition, The :func:`_declarative.instrument_declarative` function is +deprecated, superseded by :meth:`_orm.registry.map_declaratively`. The +:class:`_declarative.ConcreteBase`, :class:`_declarative.AbstractConcreteBase`, +and :class:`_declarative.DeferredReflection` classes remain as extensions in the +:ref:`declarative_toplevel` package. + +Mapping styles have now been organized such that they all extend from +the :class:`_orm.registry` object, and fall into these categories: + +* :ref:`orm_declarative_mapping` + * Using :func:`_orm.declarative_base` Base class w/ metaclass + * :ref:`orm_declarative_table` + * :ref:`Imperative Table (a.k.a. "hybrid table") <orm_imperative_table_configuration>` + * Using :meth:`_orm.registry.mapped` Declarative Decorator + * Declarative Table + * Imperative Table (Hybrid) + * :ref:`orm_declarative_dataclasses` +* :ref:`Imperative (a.k.a. "classical" mapping) <classical_mapping>` + * Using :meth:`_orm.registry.map_imperatively` + * :ref:`orm_imperative_dataclasses` + +The existing classical mapping function :func:`_orm.mapper` remains, however +it is deprecated to call upon :func:`_orm.mapper` directly; the new +:meth:`_orm.registry.map_imperatively` method now routes the request through +the :meth:`_orm.registry` so that it integrates with other declarative mappings +unambiguously. + +The new approach interoperates with 3rd party class instrumentation systems +which necessarily must take place on the class before the mapping process +does, allowing declartive mapping to work via a decorator instead of a +declarative base so that packages like dataclasses_ and attrs_ can be +used with declarative mappings, in addition to working with classical +mappings. + +Declarative documentation has now been fully integrated into the ORM mapper +configuration documentation and includes examples for all styles of mappings +organized into one place. See the section +:ref:`orm_mapping_classes_toplevel` for the start of the newly reorganized +documentation. + +.. _dataclasses: https://docs.python.org/3/library/dataclasses.html +.. _attrs: https://pypi.org/project/attrs/ + +.. seealso:: + + :ref:`orm_mapping_classes_toplevel` + + :ref:`change_5027` + +:ticket:`5508` + + +.. _change_5027: + +Python Dataclasses, attrs Supported w/ Declarative, Imperative Mappings +----------------------------------------------------------------------- + +Along with the new declarative decorator styles introduced in :ref:`change_5508`, +the :class:`_orm.Mapper` is now explicitly aware of the Python ``dataclasses`` +module and will recognize attributes that are configured in this way, and +proceed to map them without skipping them as was the case previously. In the +case of the ``attrs`` module, ``attrs`` already removes its own attributes +from the class so was already compatible with SQLAlchemy classical mappings. +With the addition of the :meth:`_orm.registry.mapped` decorator, both +attribute systems can now interoperate with Declarative mappings as well. + +.. seealso:: + + :ref:`orm_declarative_dataclasses` + + :ref:`orm_imperative_dataclasses` + + +:ticket:`5027` + + .. _change_3414: Asynchronous IO Support for Core and ORM diff --git a/doc/build/changelog/unreleased_14/5027.rst b/doc/build/changelog/unreleased_14/5027.rst index 6fd2bc9b2..fba8c6ba3 100644 --- a/doc/build/changelog/unreleased_14/5027.rst +++ b/doc/build/changelog/unreleased_14/5027.rst @@ -3,6 +3,13 @@ :tickets: 5027 Added support for direct mapping of Python classes that are defined using - the Python ``dataclasses`` decorator. See the section - :ref:`mapping_dataclasses` for background. Pull request courtesy Václav - Klusák.
\ No newline at end of file + the Python ``dataclasses`` decorator. Pull request courtesy Václav + Klusák. The new feature integrates into new support at the Declarative + level for systems such as ``dataclasses`` and ``attrs``. + + .. seealso:: + + :ref:`change_5027` + + :ref:`change_5508` + diff --git a/doc/build/changelog/unreleased_14/5508.rst b/doc/build/changelog/unreleased_14/5508.rst new file mode 100644 index 000000000..d1304c737 --- /dev/null +++ b/doc/build/changelog/unreleased_14/5508.rst @@ -0,0 +1,17 @@ +.. change:: + :tags: change, orm + :tickets: 5508 + + The ORM Declarative system is now unified into the ORM itself, with new + import spaces under ``sqlalchemy.orm`` and new kinds of mappings. Support + for decorator-based mappings without using a base class, support for + classical style-mapper() calls that have access to the declarative class + registry for relationships, and full integration of Declarative with 3rd + party class attribute systems like ``dataclasses`` and ``attrs`` is now + supported. + + .. seealso:: + + :ref:`change_5508` + + :ref:`change_5027` diff --git a/doc/build/core/engines_connections.rst b/doc/build/core/engines_connections.rst index f163a7629..70ece2ca5 100644 --- a/doc/build/core/engines_connections.rst +++ b/doc/build/core/engines_connections.rst @@ -3,7 +3,7 @@ Engine and Connection Use ========================= .. toctree:: - :maxdepth: 2 + :maxdepth: 3 engines connections diff --git a/doc/build/core/expression_api.rst b/doc/build/core/expression_api.rst index c080b3a63..944222fbd 100644 --- a/doc/build/core/expression_api.rst +++ b/doc/build/core/expression_api.rst @@ -10,7 +10,7 @@ see :ref:`sqlexpression_toplevel`. .. toctree:: - :maxdepth: 1 + :maxdepth: 3 sqlelement selectable diff --git a/doc/build/core/index.rst b/doc/build/core/index.rst index a3574341a..aaa63ca26 100644 --- a/doc/build/core/index.rst +++ b/doc/build/core/index.rst @@ -17,4 +17,4 @@ Language provides a schema-centric usage paradigm. types engines_connections api_basics - future
\ No newline at end of file + future diff --git a/doc/build/core/schema.rst b/doc/build/core/schema.rst index 5de685c7f..5a4f939bf 100644 --- a/doc/build/core/schema.rst +++ b/doc/build/core/schema.rst @@ -33,7 +33,7 @@ real DDL. They are therefore most intuitive to those who have some background in creating real schema generation scripts. .. toctree:: - :maxdepth: 2 + :maxdepth: 3 metadata reflection diff --git a/doc/build/core/types.rst b/doc/build/core/types.rst index ab761a1cb..762105646 100644 --- a/doc/build/core/types.rst +++ b/doc/build/core/types.rst @@ -4,7 +4,7 @@ Column and Data Types ===================== .. toctree:: - :maxdepth: 2 + :maxdepth: 3 type_basics custom_types diff --git a/doc/build/faq/index.rst b/doc/build/faq/index.rst index 5238490a4..810a04011 100644 --- a/doc/build/faq/index.rst +++ b/doc/build/faq/index.rst @@ -8,7 +8,7 @@ The Frequently Asked Questions section is a growing collection of commonly observed questions to well-known issues. .. toctree:: - :maxdepth: 1 + :maxdepth: 2 connections metadata_schema diff --git a/doc/build/orm/basic_relationships.rst b/doc/build/orm/basic_relationships.rst index b05701802..0ea699180 100644 --- a/doc/build/orm/basic_relationships.rst +++ b/doc/build/orm/basic_relationships.rst @@ -456,3 +456,200 @@ associated object, and a second to a target attribute. two-object ``Parent->Child`` relationship while still using the association object pattern, use the association proxy extension as documented at :ref:`associationproxy_toplevel`. + +.. _orm_declarative_relationship_eval: + +Late-Evaluation of Relationship Arguments +----------------------------------------- + +Many of the examples in the preceding sections illustrate mappings +where the various :func:`_orm.relationship` constructs refer to their target +classes using a string name, rather than the class itself:: + + class Parent(Base): + # ... + + children = relationship("Child", back_populates="parent") + + class Child(Base): + # ... + + parent = relationship("Parent", back_populates="children") + +These string names are resolved into classes in the mapper resolution stage, +which is an internal process that occurs typically after all mappings have +been defined and is normally triggered by the first usage of the mappings +themselves. The :class:`_orm.registry` object is the container in which +these names are stored and resolved to the mapped classes they refer towards. + +In addition to the main class argument for :func:`_orm.relationship`, +other arguments which depend upon the columns present on an as-yet +undefined class may also be specified either as Python functions, or more +commonly as strings. For most of these +arguments except that of the main argument, string inputs are +**evaluated as Python expressions using Python's built-in eval() function.**, +as they are intended to recieve complete SQL expressions. + +.. warning:: As the Python ``eval()`` function is used to interpret the + late-evaluated string arguments passed to :func:`_orm.relationship` mapper + configuration construct, these arguments should **not** be repurposed + such that they would receive untrusted user input; ``eval()`` is + **not secure** against untrusted user input. + +The full namespace available within this evaluation includes all classes mapped +for this declarative base, as well as the contents of the ``sqlalchemy`` +package, including expression functions like :func:`_sql.desc` and +:attr:`_functions.func`:: + + class Parent(Base): + # ... + + children = relationship( + "Child", + order_by="desc(Child.email_address)", + primaryjoin="Parent.id == Child.parent_id" + ) + +For the case where more than one module contains a class of the same name, +string class names can also be specified as module-qualified paths +within any of these string expressions:: + + class Parent(Base): + # ... + + children = relationship( + "myapp.mymodel.Child", + order_by="desc(myapp.mymodel.Child.email_address)", + primaryjoin="myapp.mymodel.Parent.id == myapp.mymodel.Child.parent_id" + ) + +The qualified path can be any partial path that removes ambiguity between +the names. For example, to disambiguate between +``myapp.model1.Child`` and ``myapp.model2.Child``, +we can specify ``model1.Child`` or ``model2.Child``:: + + class Parent(Base): + # ... + + children = relationship( + "model1.Child", + order_by="desc(mymodel1.Child.email_address)", + primaryjoin="Parent.id == model1.Child.parent_id" + ) + +The :func:`_orm.relationship` construct also accepts Python functions or +lambdas as input for these arguments. This has the advantage of providing +more compile-time safety and better support for IDEs and :pep:`484` scenarios. + +A Python functional approach might look like the following:: + + from sqlalchemy import desc + + def _resolve_child_model(): + from myapplication import Child + return Child + + class Parent(Base): + # ... + + children = relationship( + _resolve_child_model(), + order_by=lambda: desc(_resolve_child_model().email_address), + primaryjoin=lambda: Parent.id == _resolve_child_model().parent_id + ) + +The full list of parameters which accept Python functions/lambdas or strings +that will be passed to ``eval()`` are: + +* :paramref:`_orm.relationship.order_by` + +* :paramref:`_orm.relationship.primaryjoin` + +* :paramref:`_orm.relationship.secondaryjoin` + +* :paramref:`_orm.relationship.secondary` + +* :paramref:`_orm.relationship.remote_side` + +* :paramref:`_orm.relationship.foreign_keys` + +* :paramref:`_orm.relationship._user_defined_foreign_keys` + +.. versionchanged:: 1.3.16 + + Prior to SQLAlchemy 1.3.16, the main :paramref:`_orm.relationship.argument` + to :func:`_orm.relationship` was also evaluated throught ``eval()`` As of + 1.3.16 the string name is resolved from the class resolver directly without + supporting custom Python expressions. + +.. warning:: + + As stated previously, the above parameters to :func:`_orm.relationship` + are **evaluated as Python code expressions using eval(). DO NOT PASS + UNTRUSTED INPUT TO THESE ARGUMENTS.** + +It should also be noted that in a similar way as described at +:ref:`orm_declarative_table_adding_columns`, any :class:`_orm.MapperProperty` +construct can be added to a declarative base mapping at any time. If +we wanted to implement this :func:`_orm.relationship` after the ``Address`` +class were available, we could also apply it afterwards:: + + # first, module A, where Child has not been created yet, + # we create a Parent class which knows nothing about Child + + class Parent(Base): + # ... + + + #... later, in Module B, which is imported after module A: + + class Child(Base): + # ... + + from module_a import Parent + + # assign the User.addresses relationship as a class variable. The + # declarative base class will intercept this and map the relationship. + Parent.children = relationship( + Child, + primaryjoin=Child.parent_id==Parent.id + ) + +.. note:: assignment of mapped properties to a declaratively mapped class will only + function correctly if the "declarative base" class is used, which also + provides for a metaclass-driven ``__setattr__()`` method which will + intercept these operations. It will **not** work if the declarative + decorator provided by :meth:`_orm.registry.mapped` is used, nor will it + work for an imperatively mapped class mapped by + :meth:`_orm.registry.map_imperatively`. + + +.. _orm_declarative_relationship_secondary_eval: + +Late-Evaluation for a many-to-many relationship +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Many-to-many relationships include a reference to an additional, non-mapped +:class:`_schema.Table` object that is typically present in the :class:`_schema.MetaData` +collection referred towards by the :class:`_orm.registry`. The late-evaluation +system includes support for having this attribute also be specified as a +string argument which will be resolved from this :class:`_schema.MetaData` +collection. Below we specify an association table ``keyword_author``, +sharing the :class:`_schema.MetaData` collection associated with our +declarative base and its :class:`_orm.registry`. We can then refer to this +:class:`_schema.Table` by name in the :paramref:`_orm.relationship.secondary` +parameter:: + + keyword_author = Table( + 'keyword_author', Base.metadata, + Column('author_id', Integer, ForeignKey('authors.id')), + Column('keyword_id', Integer, ForeignKey('keywords.id')) + ) + + class Author(Base): + __tablename__ = 'authors' + id = Column(Integer, primary_key=True) + keywords = relationship("Keyword", secondary="keyword_author") + +For additional detail on many-to-many relationships see the section +:ref:`relationships_many_to_many`. diff --git a/doc/build/orm/declarative_config.rst b/doc/build/orm/declarative_config.rst new file mode 100644 index 000000000..bf5bd14f6 --- /dev/null +++ b/doc/build/orm/declarative_config.rst @@ -0,0 +1,335 @@ +.. _orm_declarative_mapper_config_toplevel: + +============================================= +Mapper Configuration with Declarative +============================================= + +The section :ref:`orm_mapper_configuration_overview` discusses the general +configurational elements of a :class:`_orm.Mapper` construct, which is the +structure that defines how a particular user defined class is mapped to a +database table or other SQL construct. The following sections describe +specific details about how the declarative system goes about constructing +the :class:`_orm.Mapper`. + +.. _orm_declarative_properties: + +Defining Mapped Properties with Declarative +-------------------------------------------- + +The examples given at :ref:`orm_declarative_table_config_toplevel` +illustrate mappings against table-bound columns; +the mapping of an individual column to an ORM class attribute is represented +internally by the :class:`_orm.ColumnProperty` construct. There are many +other varieties of mapper properties, the most common being the +:func:`_orm.relationship` construct. Other kinds of properties include +synonyms to columns which are defined using the :func:`_orm.synonym` +construct, SQL expressions that are defined using the :func:`_orm.column_property` +construct, and deferred columns and SQL expressions which load only when +accessed, defined using the :func:`_orm.deferred` construct. + +While an :ref:`imperative mapping <orm_imperative_mapping>` makes use of +the :ref:`properties <orm_mapping_properties>` dictionary to establish +all the mapped class attributes, in the declarative +mapping, these properties are all specified inline with the class definition, +which in the case of a declarative table mapping are inline with the +:class:`_schema.Column` objects that will be used to generate a +:class:`_schema.Table` object. + +Working with the example mapping of ``User`` and ``Address``, we may illustrate +a declarative table mapping that includes not just :class:`_schema.Column` +objects but also relationships and SQL expressions:: + + # mapping attributes using declarative with declarative table + # i.e. __tablename__ + + from sqlalchemy import Column, Integer, String, Text, ForeignKey + from sqlalchemy.orm import column_property, relationship, deferred + from sqlalchemy.orm import declarative_base + + Base = declarative_base() + + class User(Base): + __tablename__ = 'user' + + id = Column(Integer, primary_key=True) + name = Column(String) + firstname = Column(String(50)) + lastname = Column(String(50)) + + fullname = column_property(firstname + " " + lastname) + + addresses = relationship("Address", back_populates="user") + + class Address(Base): + __tablename__ = 'address' + + id = Column(Integer, primary_key=True) + user_id = Column(ForeignKey("user.id")) + email_address = Column(String) + address_statistics = deferred(Column(Text)) + + user = relationship("User", back_populates="addresses") + +The above declarative table mapping features two tables, each with a +:func:`_orm.relationship` referring to the other, as well as a simple +SQL expression mapped by :func:`_orm.column_property`, and an additional +:class:`_schema.Column` that will be loaded on a "deferred" basis as defined +by the :func:`_orm.deferred` construct. More documentation +on these particular concepts may be found at :ref:`relationship_patterns`, +:ref:`mapper_column_property_sql_expressions`, and :ref:`deferred`. + +Properties may be specified with a declarative mapping as above using +"hybrid table" style as well; the :class:`_schema.Column` objects that +are directly part of a table move into the :class:`_schema.Table` definition +but everything else, including composed SQL expressions, would still be +inline with the class definition. Constructs that need to refer to a +:class:`_schema.Column` directly would reference it in terms of the +:class:`_schema.Table` object. To illustrate the above mapping using +hybrid table style:: + + # mapping attributes using declarative with imperative table + # i.e. __table__ + + from sqlalchemy import Table + from sqlalchemy import Column, Integer, String, Text, ForeignKey + from sqlalchemy.orm import column_property, relationship, deferred + from sqlalchemy.orm import declarative_base + + Base = declarative_base() + + class User(Base): + __table__ = Table( + "user", + Base.metadata, + Column("id", Integer, primary_key=True), + Column("name", String), + Column("firstname", String(50)), + Column("lastname", String(50)) + ) + + fullname = column_property(__table__.c.firstname + " " + __table__.c.lastname) + + addresses = relationship("Address", back_populates="user") + + class Address(Base): + __table__ = Table( + "address", + Base.metadata, + Column("id", Integer, primary_key=True), + Column("user_id", ForeignKey("user.id")), + Column("email_address", String), + Column("address_statistics", Text) + ) + + address_statistics = deferred(__table__.c.address_statistics) + + user = relationship("User", back_populates="addresses") + +Things to note above: + +* The address :class:`_schema.Table` contains a column called ``address_statistics``, + however we re-map this column under the same attribute name to be under + the control of a :func:`_orm.deferred` construct. + +* With both declararative table and hybrid table mappings, when we define a + :class:`_schema.ForeignKey` construct, we always name the target table + using the **table name**, and not the mapped class name. + +* When we define :func:`_orm.relationship` constructs, as these constructs + create a linkage between two mapped classes where one necessarily is defined + before the other, we can refer to the remote class using its string name. + This functionality also extends into the area of other arguments specified + on the :func:`_orm.relationship` such as the "primary join" and "order by" + arguments. See the next section for details on this. + + +.. _orm_declarative_mapper_options: + +Mapper Configuration Options with Declarative +---------------------------------------------- + +With all mapping forms, the mapping of the class is configured through +parameters that become part of the :class:`_orm.Mapper` object. +The function which ultimately receives these arguments is the +:func:`_orm.mapper` function, and are delivered to it from one of +the front-facing mapping functions defined on the :class:`_orm.registry` +object. + +For the declarative form of mapping, mapper arguments are specified +using the ``__mapper_args__`` declarative class variable, which is a dictionary +that is passed as keyword arguments to the :func:`_orm.mapper` function. +Some examples: + +**Version ID Column** + +The :paramref:`_orm.mapper.version_id_col` and +:paramref:`_orm.mapper.version_id_generator` parameters:: + + from datetime import datetime + + class Widget(Base): + __tablename__ = 'widgets' + + id = Column(Integer, primary_key=True) + timestamp = Column(DateTime, nullable=False) + + __mapper_args__ = { + 'version_id_col': timestamp, + 'version_id_generator': lambda v:datetime.now() + } + +**Single Table Inheritance** + +The :paramref:`_orm.mapper.polymorphic_on` and +:paramref:`_orm.mapper.polymorphic_identity` parameters:: + + class Person(Base): + __tablename__ = 'person' + + person_id = Column(Integer, primary_key=True) + type = Column(String, nullable=False) + + __mapper_args__ = dict( + polymorphic_on=type, + polymorphic_identity="person" + ) + + class Employee(Person): + __mapper_args__ = dict( + polymorphic_identity="employee" + ) + +The ``__mapper_args__`` dictionary may be generated from a class-bound +descriptor method rather than from a fixed dictionary by making use of the +:func:`_orm.declared_attr` construct. The section :ref:`orm_mixins_toplevel` +discusses this concept further. + +.. seealso:: + + :ref:`orm_mixins_toplevel` + +Other Declarative Mapping Directives +-------------------------------------- + +``__declare_last__()`` +~~~~~~~~~~~~~~~~~~~~~~ + +The ``__declare_last__()`` hook allows definition of +a class level function that is automatically called by the +:meth:`.MapperEvents.after_configured` event, which occurs after mappings are +assumed to be completed and the 'configure' step has finished:: + + class MyClass(Base): + @classmethod + def __declare_last__(cls): + "" + # do something with mappings + +``__declare_first__()`` +~~~~~~~~~~~~~~~~~~~~~~~ + +Like ``__declare_last__()``, but is called at the beginning of mapper +configuration via the :meth:`.MapperEvents.before_configured` event:: + + class MyClass(Base): + @classmethod + def __declare_first__(cls): + "" + # do something before mappings are configured + +.. versionadded:: 0.9.3 + +.. _declarative_abstract: + +``__abstract__`` +~~~~~~~~~~~~~~~~ + +``__abstract__`` causes declarative to skip the production +of a table or mapper for the class entirely. A class can be added within a +hierarchy in the same way as mixin (see :ref:`declarative_mixins`), allowing +subclasses to extend just from the special class:: + + class SomeAbstractBase(Base): + __abstract__ = True + + def some_helpful_method(self): + "" + + @declared_attr + def __mapper_args__(cls): + return {"helpful mapper arguments":True} + + class MyMappedClass(SomeAbstractBase): + "" + +One possible use of ``__abstract__`` is to use a distinct +:class:`_schema.MetaData` for different bases:: + + Base = declarative_base() + + class DefaultBase(Base): + __abstract__ = True + metadata = MetaData() + + class OtherBase(Base): + __abstract__ = True + metadata = MetaData() + +Above, classes which inherit from ``DefaultBase`` will use one +:class:`_schema.MetaData` as the registry of tables, and those which inherit from +``OtherBase`` will use a different one. The tables themselves can then be +created perhaps within distinct databases:: + + DefaultBase.metadata.create_all(some_engine) + OtherBase.metadata.create_all(some_other_engine) + + +``__table_cls__`` +~~~~~~~~~~~~~~~~~ + +Allows the callable / class used to generate a :class:`_schema.Table` to be customized. +This is a very open-ended hook that can allow special customizations +to a :class:`_schema.Table` that one generates here:: + + class MyMixin(object): + @classmethod + def __table_cls__(cls, name, metadata, *arg, **kw): + return Table( + "my_" + name, + metadata, *arg, **kw + ) + +The above mixin would cause all :class:`_schema.Table` objects generated to include +the prefix ``"my_"``, followed by the name normally specified using the +``__tablename__`` attribute. + +``__table_cls__`` also supports the case of returning ``None``, which +causes the class to be considered as single-table inheritance vs. its subclass. +This may be useful in some customization schemes to determine that single-table +inheritance should take place based on the arguments for the table itself, +such as, define as single-inheritance if there is no primary key present:: + + class AutoTable(object): + @declared_attr + def __tablename__(cls): + return cls.__name__ + + @classmethod + def __table_cls__(cls, *arg, **kw): + for obj in arg[1:]: + if (isinstance(obj, Column) and obj.primary_key) or \ + isinstance(obj, PrimaryKeyConstraint): + return Table(*arg, **kw) + + return None + + class Person(AutoTable, Base): + id = Column(Integer, primary_key=True) + + class Employee(Person): + employee_name = Column(String) + +The above ``Employee`` class would be mapped as single-table inheritance +against ``Person``; the ``employee_name`` column would be added as a member +of the ``Person`` table. + diff --git a/doc/build/orm/declarative_mapping.rst b/doc/build/orm/declarative_mapping.rst new file mode 100644 index 000000000..9d2f3af40 --- /dev/null +++ b/doc/build/orm/declarative_mapping.rst @@ -0,0 +1,17 @@ +.. _declarative_config_toplevel: + +================================ +Mapping Classes with Declarative +================================ + +The Declarative mapping style is the primary style of mapping that is used +with SQLAlchemy. See the section :ref:`orm_declarative_mapping` for the +top level introduction. + + +.. toctree:: + :maxdepth: 3 + + declarative_tables + declarative_config + declarative_mixins diff --git a/doc/build/orm/declarative_mixins.rst b/doc/build/orm/declarative_mixins.rst new file mode 100644 index 000000000..c5912181d --- /dev/null +++ b/doc/build/orm/declarative_mixins.rst @@ -0,0 +1,548 @@ +.. _orm_mixins_toplevel: + +Composing Mapped Hierarchies with Mixins +======================================== + +A common need when mapping classes using the :ref:`Declarative +<orm_declarative_mapping>` style is to share some functionality, such as a set +of common columns, some common table options, or other mapped properties, +across many classes. The standard Python idioms for this is to have the +classes inherit from a superclass which includes these common features. + +When using declarative mappings, this idiom is allowed via the +usage of mixin classes, as well as via augmenting the declarative base +produced by either the :meth:`_orm.registry.generate_base` method +or :func:`_orm.declarative_base` functions. + +An example of some commonly mixed-in idioms is below:: + + from sqlalchemy.orm import declared_attr + + class MyMixin(object): + + @declared_attr + def __tablename__(cls): + return cls.__name__.lower() + + __table_args__ = {'mysql_engine': 'InnoDB'} + __mapper_args__= {'always_refresh': True} + + id = Column(Integer, primary_key=True) + + class MyModel(MyMixin, Base): + name = Column(String(1000)) + +Where above, the class ``MyModel`` will contain an "id" column +as the primary key, a ``__tablename__`` attribute that derives +from the name of the class itself, as well as ``__table_args__`` +and ``__mapper_args__`` defined by the ``MyMixin`` mixin class. + +There's no fixed convention over whether ``MyMixin`` precedes +``Base`` or not. Normal Python method resolution rules apply, and +the above example would work just as well with:: + + class MyModel(Base, MyMixin): + name = Column(String(1000)) + +This works because ``Base`` here doesn't define any of the +variables that ``MyMixin`` defines, i.e. ``__tablename__``, +``__table_args__``, ``id``, etc. If the ``Base`` did define +an attribute of the same name, the class placed first in the +inherits list would determine which attribute is used on the +newly defined class. + +Augmenting the Base +~~~~~~~~~~~~~~~~~~~ + +In addition to using a pure mixin, most of the techniques in this +section can also be applied to the base class itself, for patterns that +should apply to all classes derived from a particular base. This is achieved +using the ``cls`` argument of the :func:`_orm.declarative_base` function:: + + from sqlalchemy.orm import declared_attr + + class Base(object): + @declared_attr + def __tablename__(cls): + return cls.__name__.lower() + + __table_args__ = {'mysql_engine': 'InnoDB'} + + id = Column(Integer, primary_key=True) + + from sqlalchemy.orm import declarative_base + + Base = declarative_base(cls=Base) + + class MyModel(Base): + name = Column(String(1000)) + +Where above, ``MyModel`` and all other classes that derive from ``Base`` will +have a table name derived from the class name, an ``id`` primary key column, +as well as the "InnoDB" engine for MySQL. + +Mixing in Columns +~~~~~~~~~~~~~~~~~ + +The most basic way to specify a column on a mixin is by simple +declaration:: + + class TimestampMixin(object): + created_at = Column(DateTime, default=func.now()) + + class MyModel(TimestampMixin, Base): + __tablename__ = 'test' + + id = Column(Integer, primary_key=True) + name = Column(String(1000)) + +Where above, all declarative classes that include ``TimestampMixin`` +will also have a column ``created_at`` that applies a timestamp to +all row insertions. + +Those familiar with the SQLAlchemy expression language know that +the object identity of clause elements defines their role in a schema. +Two ``Table`` objects ``a`` and ``b`` may both have a column called +``id``, but the way these are differentiated is that ``a.c.id`` +and ``b.c.id`` are two distinct Python objects, referencing their +parent tables ``a`` and ``b`` respectively. + +In the case of the mixin column, it seems that only one +:class:`_schema.Column` object is explicitly created, yet the ultimate +``created_at`` column above must exist as a distinct Python object +for each separate destination class. To accomplish this, the declarative +extension creates a **copy** of each :class:`_schema.Column` object encountered on +a class that is detected as a mixin. + +This copy mechanism is limited to simple columns that have no foreign +keys, as a :class:`_schema.ForeignKey` itself contains references to columns +which can't be properly recreated at this level. For columns that +have foreign keys, as well as for the variety of mapper-level constructs +that require destination-explicit context, the +:class:`_orm.declared_attr` decorator is provided so that +patterns common to many classes can be defined as callables:: + + from sqlalchemy.orm import declared_attr + + class ReferenceAddressMixin(object): + @declared_attr + def address_id(cls): + return Column(Integer, ForeignKey('address.id')) + + class User(ReferenceAddressMixin, Base): + __tablename__ = 'user' + id = Column(Integer, primary_key=True) + +Where above, the ``address_id`` class-level callable is executed at the +point at which the ``User`` class is constructed, and the declarative +extension can use the resulting :class:`_schema.Column` object as returned by +the method without the need to copy it. + +Columns generated by :class:`_orm.declared_attr` can also be +referenced by ``__mapper_args__`` to a limited degree, currently +by ``polymorphic_on`` and ``version_id_col``; the declarative extension +will resolve them at class construction time:: + + class MyMixin: + @declared_attr + def type_(cls): + return Column(String(50)) + + __mapper_args__= {'polymorphic_on':type_} + + class MyModel(MyMixin, Base): + __tablename__='test' + id = Column(Integer, primary_key=True) + + +Mixing in Relationships +~~~~~~~~~~~~~~~~~~~~~~~ + +Relationships created by :func:`~sqlalchemy.orm.relationship` are provided +with declarative mixin classes exclusively using the +:class:`_orm.declared_attr` approach, eliminating any ambiguity +which could arise when copying a relationship and its possibly column-bound +contents. Below is an example which combines a foreign key column and a +relationship so that two classes ``Foo`` and ``Bar`` can both be configured to +reference a common target class via many-to-one:: + + class RefTargetMixin(object): + @declared_attr + def target_id(cls): + return Column('target_id', ForeignKey('target.id')) + + @declared_attr + def target(cls): + return relationship("Target") + + class Foo(RefTargetMixin, Base): + __tablename__ = 'foo' + id = Column(Integer, primary_key=True) + + class Bar(RefTargetMixin, Base): + __tablename__ = 'bar' + id = Column(Integer, primary_key=True) + + class Target(Base): + __tablename__ = 'target' + id = Column(Integer, primary_key=True) + + +Using Advanced Relationship Arguments (e.g. ``primaryjoin``, etc.) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:func:`~sqlalchemy.orm.relationship` definitions which require explicit +primaryjoin, order_by etc. expressions should in all but the most +simplistic cases use **late bound** forms +for these arguments, meaning, using either the string form or a lambda. +The reason for this is that the related :class:`_schema.Column` objects which are to +be configured using ``@declared_attr`` are not available to another +``@declared_attr`` attribute; while the methods will work and return new +:class:`_schema.Column` objects, those are not the :class:`_schema.Column` objects that +Declarative will be using as it calls the methods on its own, thus using +*different* :class:`_schema.Column` objects. + +The canonical example is the primaryjoin condition that depends upon +another mixed-in column:: + + class RefTargetMixin(object): + @declared_attr + def target_id(cls): + return Column('target_id', ForeignKey('target.id')) + + @declared_attr + def target(cls): + return relationship(Target, + primaryjoin=Target.id==cls.target_id # this is *incorrect* + ) + +Mapping a class using the above mixin, we will get an error like:: + + sqlalchemy.exc.InvalidRequestError: this ForeignKey's parent column is not + yet associated with a Table. + +This is because the ``target_id`` :class:`_schema.Column` we've called upon in our +``target()`` method is not the same :class:`_schema.Column` that declarative is +actually going to map to our table. + +The condition above is resolved using a lambda:: + + class RefTargetMixin(object): + @declared_attr + def target_id(cls): + return Column('target_id', ForeignKey('target.id')) + + @declared_attr + def target(cls): + return relationship(Target, + primaryjoin=lambda: Target.id==cls.target_id + ) + +or alternatively, the string form (which ultimately generates a lambda):: + + class RefTargetMixin(object): + @declared_attr + def target_id(cls): + return Column('target_id', ForeignKey('target.id')) + + @declared_attr + def target(cls): + return relationship("Target", + primaryjoin="Target.id==%s.target_id" % cls.__name__ + ) + +.. seealso:: + + :ref:`orm_declarative_relationship_eval` + +Mixing in deferred(), column_property(), and other MapperProperty classes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Like :func:`~sqlalchemy.orm.relationship`, all +:class:`~sqlalchemy.orm.interfaces.MapperProperty` subclasses such as +:func:`~sqlalchemy.orm.deferred`, :func:`~sqlalchemy.orm.column_property`, +etc. ultimately involve references to columns, and therefore, when +used with declarative mixins, have the :class:`_orm.declared_attr` +requirement so that no reliance on copying is needed:: + + class SomethingMixin(object): + + @declared_attr + def dprop(cls): + return deferred(Column(Integer)) + + class Something(SomethingMixin, Base): + __tablename__ = "something" + +The :func:`.column_property` or other construct may refer +to other columns from the mixin. These are copied ahead of time before +the :class:`_orm.declared_attr` is invoked:: + + class SomethingMixin(object): + x = Column(Integer) + + y = Column(Integer) + + @declared_attr + def x_plus_y(cls): + return column_property(cls.x + cls.y) + + +.. versionchanged:: 1.0.0 mixin columns are copied to the final mapped class + so that :class:`_orm.declared_attr` methods can access the actual column + that will be mapped. + +Mixing in Association Proxy and Other Attributes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Mixins can specify user-defined attributes as well as other extension +units such as :func:`.association_proxy`. The usage of +:class:`_orm.declared_attr` is required in those cases where the attribute must +be tailored specifically to the target subclass. An example is when +constructing multiple :func:`.association_proxy` attributes which each +target a different type of child object. Below is an +:func:`.association_proxy` / mixin example which provides a scalar list of +string values to an implementing class:: + + from sqlalchemy import Column, Integer, ForeignKey, String + from sqlalchemy.orm import relationship + from sqlalchemy.ext.associationproxy import association_proxy + from sqlalchemy.orm import declarative_base, declared_attr + + Base = declarative_base() + + class HasStringCollection(object): + @declared_attr + def _strings(cls): + class StringAttribute(Base): + __tablename__ = cls.string_table_name + id = Column(Integer, primary_key=True) + value = Column(String(50), nullable=False) + parent_id = Column(Integer, + ForeignKey('%s.id' % cls.__tablename__), + nullable=False) + def __init__(self, value): + self.value = value + + return relationship(StringAttribute) + + @declared_attr + def strings(cls): + return association_proxy('_strings', 'value') + + class TypeA(HasStringCollection, Base): + __tablename__ = 'type_a' + string_table_name = 'type_a_strings' + id = Column(Integer(), primary_key=True) + + class TypeB(HasStringCollection, Base): + __tablename__ = 'type_b' + string_table_name = 'type_b_strings' + id = Column(Integer(), primary_key=True) + +Above, the ``HasStringCollection`` mixin produces a :func:`_orm.relationship` +which refers to a newly generated class called ``StringAttribute``. The +``StringAttribute`` class is generated with its own :class:`_schema.Table` +definition which is local to the parent class making usage of the +``HasStringCollection`` mixin. It also produces an :func:`.association_proxy` +object which proxies references to the ``strings`` attribute onto the ``value`` +attribute of each ``StringAttribute`` instance. + +``TypeA`` or ``TypeB`` can be instantiated given the constructor +argument ``strings``, a list of strings:: + + ta = TypeA(strings=['foo', 'bar']) + tb = TypeA(strings=['bat', 'bar']) + +This list will generate a collection +of ``StringAttribute`` objects, which are persisted into a table that's +local to either the ``type_a_strings`` or ``type_b_strings`` table:: + + >>> print(ta._strings) + [<__main__.StringAttribute object at 0x10151cd90>, + <__main__.StringAttribute object at 0x10151ce10>] + +When constructing the :func:`.association_proxy`, the +:class:`_orm.declared_attr` decorator must be used so that a distinct +:func:`.association_proxy` object is created for each of the ``TypeA`` +and ``TypeB`` classes. + +.. _decl_mixin_inheritance: + +Controlling table inheritance with mixins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``__tablename__`` attribute may be used to provide a function that +will determine the name of the table used for each class in an inheritance +hierarchy, as well as whether a class has its own distinct table. + +This is achieved using the :class:`_orm.declared_attr` indicator in conjunction +with a method named ``__tablename__()``. Declarative will always +invoke :class:`_orm.declared_attr` for the special names +``__tablename__``, ``__mapper_args__`` and ``__table_args__`` +function **for each mapped class in the hierarchy, except if overridden +in a subclass**. The function therefore +needs to expect to receive each class individually and to provide the +correct answer for each. + +For example, to create a mixin that gives every class a simple table +name based on class name:: + + from sqlalchemy.orm import declared_attr + + class Tablename: + @declared_attr + def __tablename__(cls): + return cls.__name__.lower() + + class Person(Tablename, Base): + id = Column(Integer, primary_key=True) + discriminator = Column('type', String(50)) + __mapper_args__ = {'polymorphic_on': discriminator} + + class Engineer(Person): + __tablename__ = None + __mapper_args__ = {'polymorphic_identity': 'engineer'} + primary_language = Column(String(50)) + +Alternatively, we can modify our ``__tablename__`` function to return +``None`` for subclasses, using :func:`.has_inherited_table`. This has +the effect of those subclasses being mapped with single table inheritance +against the parent:: + + from sqlalchemy.orm import declared_attr + from sqlalchemy.orm import has_inherited_table + + class Tablename(object): + @declared_attr + def __tablename__(cls): + if has_inherited_table(cls): + return None + return cls.__name__.lower() + + class Person(Tablename, Base): + id = Column(Integer, primary_key=True) + discriminator = Column('type', String(50)) + __mapper_args__ = {'polymorphic_on': discriminator} + + class Engineer(Person): + primary_language = Column(String(50)) + __mapper_args__ = {'polymorphic_identity': 'engineer'} + +.. _mixin_inheritance_columns: + +Mixing in Columns in Inheritance Scenarios +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In contrast to how ``__tablename__`` and other special names are handled when +used with :class:`_orm.declared_attr`, when we mix in columns and properties (e.g. +relationships, column properties, etc.), the function is +invoked for the **base class only** in the hierarchy. Below, only the +``Person`` class will receive a column +called ``id``; the mapping will fail on ``Engineer``, which is not given +a primary key:: + + class HasId(object): + @declared_attr + def id(cls): + return Column('id', Integer, primary_key=True) + + class Person(HasId, Base): + __tablename__ = 'person' + discriminator = Column('type', String(50)) + __mapper_args__ = {'polymorphic_on': discriminator} + + class Engineer(Person): + __tablename__ = 'engineer' + primary_language = Column(String(50)) + __mapper_args__ = {'polymorphic_identity': 'engineer'} + +It is usually the case in joined-table inheritance that we want distinctly +named columns on each subclass. However in this case, we may want to have +an ``id`` column on every table, and have them refer to each other via +foreign key. We can achieve this as a mixin by using the +:attr:`.declared_attr.cascading` modifier, which indicates that the +function should be invoked **for each class in the hierarchy**, in *almost* +(see warning below) the same way as it does for ``__tablename__``:: + + class HasIdMixin(object): + @declared_attr.cascading + def id(cls): + if has_inherited_table(cls): + return Column(ForeignKey('person.id'), primary_key=True) + else: + return Column(Integer, primary_key=True) + + class Person(HasIdMixin, Base): + __tablename__ = 'person' + discriminator = Column('type', String(50)) + __mapper_args__ = {'polymorphic_on': discriminator} + + class Engineer(Person): + __tablename__ = 'engineer' + primary_language = Column(String(50)) + __mapper_args__ = {'polymorphic_identity': 'engineer'} + +.. warning:: + + The :attr:`.declared_attr.cascading` feature currently does + **not** allow for a subclass to override the attribute with a different + function or value. This is a current limitation in the mechanics of + how ``@declared_attr`` is resolved, and a warning is emitted if + this condition is detected. This limitation does **not** + exist for the special attribute names such as ``__tablename__``, which + resolve in a different way internally than that of + :attr:`.declared_attr.cascading`. + + +.. versionadded:: 1.0.0 added :attr:`.declared_attr.cascading`. + +Combining Table/Mapper Arguments from Multiple Mixins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the case of ``__table_args__`` or ``__mapper_args__`` +specified with declarative mixins, you may want to combine +some parameters from several mixins with those you wish to +define on the class itself. The +:class:`_orm.declared_attr` decorator can be used +here to create user-defined collation routines that pull +from multiple collections:: + + from sqlalchemy.orm import declared_attr + + class MySQLSettings(object): + __table_args__ = {'mysql_engine':'InnoDB'} + + class MyOtherMixin(object): + __table_args__ = {'info':'foo'} + + class MyModel(MySQLSettings, MyOtherMixin, Base): + __tablename__='my_model' + + @declared_attr + def __table_args__(cls): + args = dict() + args.update(MySQLSettings.__table_args__) + args.update(MyOtherMixin.__table_args__) + return args + + id = Column(Integer, primary_key=True) + +Creating Indexes with Mixins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To define a named, potentially multicolumn :class:`.Index` that applies to all +tables derived from a mixin, use the "inline" form of :class:`.Index` and +establish it as part of ``__table_args__``:: + + class MyMixin(object): + a = Column(Integer) + b = Column(Integer) + + @declared_attr + def __table_args__(cls): + return (Index('test_idx_%s' % cls.__tablename__, 'a', 'b'),) + + class MyModel(MyMixin, Base): + __tablename__ = 'atable' + c = Column(Integer,primary_key=True) diff --git a/doc/build/orm/declarative_tables.rst b/doc/build/orm/declarative_tables.rst new file mode 100644 index 000000000..bbad7a321 --- /dev/null +++ b/doc/build/orm/declarative_tables.rst @@ -0,0 +1,372 @@ + +.. _orm_declarative_table_config_toplevel: + +============================================= +Table Configuration with Declarative +============================================= + +As introduced at :ref:`orm_declarative_mapping`, the Declarative style +includses the ability to generate a mapped :class:`_schema.Table` object +at the same time, or to accommodate a :class:`_schema.Table` or other +:class:`_sql.FromClause` object directly. + +The following examples assume a declarative base class as:: + + from sqlalchemy.orm import declarative_base + + Base = declarative_base() + +All of the examples that follow illustrate a class inheriting from the above +``Base``. The decorator style introduced at :ref:`orm_declarative_decorator` +is fully supported with all the following examples as well. + +.. _orm_declarative_table: + +Declarative Table +----------------- + +With the declarative base class, the typical form of mapping includes an +attribute ``__tablename__`` that indicates the name of a :class:`_schema.Table` +that should be generated along with the mapping:: + + from sqlalchemy import Column, Integer, String, ForeignKey + from sqlalchemy.orm import declarative_base + + Base = declarative_base() + + class User(Base): + __tablename__ = 'user' + + id = Column(Integer, primary_key=True) + name = Column(String) + fullname = Column(String) + nickname = Column(String) + +Above, :class:`_schema.Column` objects are placed inline with the class +definition. The declarative mapping process will generate a new +:class:`_schema.Table` object against the :class:`_schema.MetaData` collection +associated with the declarative base, and each specified +:class:`_schema.Column` object will become part of the :attr:`.schema.Table.columns` +collection of this :class:`_schema.Table` object. The :class:`_schema.Column` +objects can omit their "name" field, which is usually the first positional +argument to the :class:`_schema.Column` constructor; the declarative system +will assign the key associated with each :class:`_schema.Column` as the name, +to produce a :class:`_schema.Table` that is equvialent to:: + + # equivalent Table object produced + user_table = Table( + "user", + Base.metadata, + Column("id", Integer, primary_key=True), + Column("name", String), + Column("fullname", String), + Column("nickname", String), + ) + +.. _orm_declarative_metadata: + +Accessing Table and Metadata +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +A declaratively mapped class will always include an attribute called +``__table__``; when the above configuration using ``__tablename__`` is +complete, the declarative process makes the :class:`_schema.Table` +available via the ``__table__`` attribute:: + + + # access the Table + user_table = User.__table__ + +The above table is ultimately the same one that corresponds to the +:attr:`_orm.Mapper.local_table` attribute, which we can see through the +:ref:`runtime inspection system <inspection_toplevel>`:: + + from sqlalchemy import inspect + + user_table = inspect(User).local_table + +The :class:`_schema.MetaData` collection associated with both the declarative +:class:`_orm.registry` as well as the base class is frequently necessary in +order to run DDL operations such as CREATE, as well as in use with migration +tools such as Alembic. This object is available via the ``.metadata`` +attribute of :class:`_orm.registry` as well as the declarative base class. +Below, for a small script we may wish to emit a CREATE for all tables against a +SQLite database:: + + engine = create_engine("sqlite://") + + Base.metadata.create_all(engine) + +.. _orm_declarative_table_configuration: + +Declarative Table Configuration +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When using Declarative Table configuration with the ``__tablename__`` +declarative class attribute, additional arguments to be supplied to the +:class:`_schema.Table` constructor should be provided using the +``__table_args__`` declarative class attribute. + +This attribute accommodates both positional as well as keyword +arguments that are normally sent to the +:class:`_schema.Table` constructor. +The attribute can be specified in one of two forms. One is as a +dictionary:: + + class MyClass(Base): + __tablename__ = 'sometable' + __table_args__ = {'mysql_engine':'InnoDB'} + +The other, a tuple, where each argument is positional +(usually constraints):: + + class MyClass(Base): + __tablename__ = 'sometable' + __table_args__ = ( + ForeignKeyConstraint(['id'], ['remote_table.id']), + UniqueConstraint('foo'), + ) + +Keyword arguments can be specified with the above form by +specifying the last argument as a dictionary:: + + class MyClass(Base): + __tablename__ = 'sometable' + __table_args__ = ( + ForeignKeyConstraint(['id'], ['remote_table.id']), + UniqueConstraint('foo'), + {'autoload':True} + ) + +A class may also specify the ``__table_args__`` declarative attribute, +as well as the ``__tablename__`` attribute, in a dynamic style using the +:func:`_orm.declared_attr` method decorator. See the section +:ref:`declarative_mixins` for examples on how this is often used. + +.. _orm_declarative_table_adding_columns: + +Adding New Columns +^^^^^^^^^^^^^^^^^^^ + +The declarative table configuration allows the addition of new +:class:`_schema.Column` objects under two scenarios. The most basic +is that of simply assigning new :class:`_schema.Column` objects to the +class:: + + MyClass.some_new_column = Column('data', Unicode) + +The above operation performed against a declarative class that has been +mapped using the declarative base (note, not the decorator form of declarative) +will add the above :class:`_schema.Column` to the :class:`_schema.Table` +using the :meth:`_schema.Table.append_column` method and will also add the +column to the :class:`_orm.Mapper` to be fully mapped. + +.. note:: assignment of new columns to an existing declaratively mapped class + will only function correctly if the "declarative base" class is used, which + also provides for a metaclass-driven ``__setattr__()`` method which will + intercept these operations. It will **not** work if the declarative + decorator provided by + :meth:`_orm.registry.mapped` is used, nor will it work for an imperatively + mapped class mapped by :meth:`_orm.registry.map_imperatively`. + + +The other scenario where a :class:`_schema.Column` is added on the fly is +when an inheriting subclass that has no table of its own indicates +additional columns; these columns will be added to the superclass table. +The section :ref:`single_inheritance` discusses single table inheritance. + + +.. _orm_imperative_table_configuration: + +Declarative with Imperative Table (a.k.a. Hybrid Declarative) +------------------------------------------------------------- + +Declarative mappings may also be provided with a pre-existing +:class:`_schema.Table` object, or otherwise a :class:`_schema.Table` or other +arbitrary :class:`_sql.FromClause` construct (such as a :class:`_sql.Join` +or :class:`_sql.Subquery`) that is constructed separately. + +This is referred to as a "hybrid declarative" +mapping, as the class is mapped using the declarative style for everything +involving the mapper configuration, however the mapped :class:`_schema.Table` +object is produced separately and passed to the declarative process +directly:: + + + from sqlalchemy.orm import declarative_base + from sqlalchemy import Column, Integer, String, ForeignKey + + + Base = declarative_base() + + # construct a Table directly. The Base.metadata collection is + # usually a good choice for MetaData but any MetaData + # collection may be used. + + user_table = Table( + "user", + Base.metadata, + Column("id", Integer, primary_key=True), + Column("name", String), + Column("fullname", String), + Column("nickname", String), + ) + + # construct the User class using this table. + class User(Base): + __table__ = user_table + +Above, a :class:`_schema.Table` object is constructed using the approach +described at :ref:`metadata_describing`. It can then be applied directly +to a class that is declaratively mapped. The ``__tablename__`` and +``__table_args__`` declarative class attributes are not used in this form. +The above configuration is often more readable as an inline definition:: + + class User(Base): + __table__ = Table( + "user", + Base.metadata, + Column("id", Integer, primary_key=True), + Column("name", String), + Column("fullname", String), + Column("nickname", String), + ) + +A natural effect of the above style is that the ``__table__`` attribute is +itself defined within the class definition block. As such it may be +immediately referred towards within subsequent attributes, such as the example +below which illustrates referring to the ``type`` column in a polymorphic +mapper configuration:: + + class Person(Base): + __table__ = Table( + 'person', + Base.metadata, + Column('id', Integer, primary_key=True), + Column('name', String(50)), + Column('type', String(50)) + ) + + __mapper_args__ = { + "polymorphic_on": __table__.c.type, + "polymorhpic_identity": "person" + } + +The "imperative table" form is also used when a non-:class:`_schema.Table` +construct, such as a :class:`_sql.Join` or :class:`_sql.Subquery` object, +is to be mapped. An example below:: + + from sqlalchemy import select, func + + subq = select( + func.count(orders.c.id).label('order_count'), + func.max(orders.c.price).label('highest_order'), + orders.c.customer_id + ).group_by(orders.c.customer_id).subquery() + + customer_select = select(customers, subq).join_from( + customers, subq, customers.c.id == subq.c.customer_id + ).subquery() + + class Customer(Base): + __table__ = customer_select + +For background on mapping to non-:class:`_schema.Table` constructs see +the sections :ref:`orm_mapping_joins` and :ref:`orm_mapping_arbitrary_subqueries`. + +The "imperative table" form is of particular use when the class itself +is using an alternative form of attribute declaration, such as Python +dataclasses. See the section :ref:`orm_declarative_dataclasses` for detail. + +.. seealso:: + + :ref:`metadata_describing` + + :ref:`orm_declarative_dataclasses` + +.. _orm_declarative_reflected: + +Mapping Declaratively with Reflected Tables +-------------------------------------------- + +There are several patterns available which provide for producing mapped +classes against a series of :class:`_schema.Table` objects that were +introspected from the database, using the reflection process described at +:ref:`metadata_reflection`. + +A very simple way to map a class to a table reflected from the database is to +use a declarative hybrid mapping, passing the +:paramref:`_schema.Table.autoload_with` parameter to the +:class:`_schema.Table`:: + + engine = create_engine("postgresql://user:pass@hostname/my_existing_database") + + class MyClass(Base): + __table__ = Table( + 'mytable', + Base.metadata, + autoload_with=engine + ) + +A major downside of the above approach however is that it requires the database +connectivity source to be present while the application classes are being +declared; it's typical that classes are declared as the modules of an +application are being imported, but database connectivity isn't available +until the application starts running code so that it can consume configuration +information and create an engine. + +Using DeferredReflection +^^^^^^^^^^^^^^^^^^^^^^^^^ + +To accommodate this case, a simple extension called the +:class:`.DeferredReflection` mixin is available, which alters the declarative +mapping process to be delayed until a special class-level +:meth:`.DeferredReflection.prepare` method is called, which will perform +the reflection process against a target database, and will integrate the +results with the declarative table mapping process, that is, classes which +use the ``__tablename__`` attribute:: + + from sqlalchemy.orm import declarative_base + from sqlalchemy.ext.declarative import DeferredReflection + + Base = declarative_base() + + class Reflected(DeferredReflection): + __abstract__ = True + + class Foo(Reflected, Base): + __tablename__ = 'foo' + bars = relationship("Bar") + + class Bar(Reflected, Base): + __tablename__ = 'bar' + + foo_id = Column(Integer, ForeignKey('foo.id')) + +Above, we create a mixin class ``Reflected`` that will serve as a base +for classes in our declarative hierarchy that should become mapped when +the ``Reflected.prepare`` method is called. The above mapping is not +complete until we do so, given an :class:`_engine.Engine`:: + + + engine = create_engine("postgresql://user:pass@hostname/my_existing_database") + Reflected.prepare(engine) + +The purpose of the ``Reflected`` class is to define the scope at which +classes should be reflectively mapped. The plugin will search among the +subclass tree of the target against which ``.prepare()`` is called and reflect +all tables. + +Using Automap +^^^^^^^^^^^^^^ + +A more automated solution to mapping against an existing database where +table reflection is to be used is to use the :ref:`automap_toplevel` +extension. This extension will generate entire mapped classes from a +database schema, and allows several hooks for customization including the +ability to explicitly map some or all classes while still making use of +reflection to fill in the remaining columns. + +.. seealso:: + + :ref:`automap_toplevel` diff --git a/doc/build/orm/extensions/declarative/api.rst b/doc/build/orm/extensions/declarative/api.rst index be97604d3..6e413a07e 100644 --- a/doc/build/orm/extensions/declarative/api.rst +++ b/doc/build/orm/extensions/declarative/api.rst @@ -7,18 +7,19 @@ Declarative API API Reference ============= -.. autofunction:: declarative_base +.. versionchanged:: 1.4 The fundamental structures of the declarative + system are now part of SQLAlchemy ORM directly. For these components + see: -.. autofunction:: as_declarative + * :func:`_orm.declarative_base` -.. autoclass:: declared_attr - :members: + * :class:`_orm.declared_attr` -.. autofunction:: sqlalchemy.ext.declarative.api._declarative_constructor + * :func:`_orm.has_inherited_table` -.. autofunction:: has_inherited_table + * :func:`_orm.synonym_for` -.. autofunction:: synonym_for + * :meth:`_orm.as_declarative` .. autofunction:: instrument_declarative @@ -30,130 +31,3 @@ API Reference :members: -Special Directives ------------------- - -``__declare_last__()`` -~~~~~~~~~~~~~~~~~~~~~~ - -The ``__declare_last__()`` hook allows definition of -a class level function that is automatically called by the -:meth:`.MapperEvents.after_configured` event, which occurs after mappings are -assumed to be completed and the 'configure' step has finished:: - - class MyClass(Base): - @classmethod - def __declare_last__(cls): - "" - # do something with mappings - -``__declare_first__()`` -~~~~~~~~~~~~~~~~~~~~~~~ - -Like ``__declare_last__()``, but is called at the beginning of mapper -configuration via the :meth:`.MapperEvents.before_configured` event:: - - class MyClass(Base): - @classmethod - def __declare_first__(cls): - "" - # do something before mappings are configured - -.. versionadded:: 0.9.3 - -.. _declarative_abstract: - -``__abstract__`` -~~~~~~~~~~~~~~~~ - -``__abstract__`` causes declarative to skip the production -of a table or mapper for the class entirely. A class can be added within a -hierarchy in the same way as mixin (see :ref:`declarative_mixins`), allowing -subclasses to extend just from the special class:: - - class SomeAbstractBase(Base): - __abstract__ = True - - def some_helpful_method(self): - "" - - @declared_attr - def __mapper_args__(cls): - return {"helpful mapper arguments":True} - - class MyMappedClass(SomeAbstractBase): - "" - -One possible use of ``__abstract__`` is to use a distinct -:class:`_schema.MetaData` for different bases:: - - Base = declarative_base() - - class DefaultBase(Base): - __abstract__ = True - metadata = MetaData() - - class OtherBase(Base): - __abstract__ = True - metadata = MetaData() - -Above, classes which inherit from ``DefaultBase`` will use one -:class:`_schema.MetaData` as the registry of tables, and those which inherit from -``OtherBase`` will use a different one. The tables themselves can then be -created perhaps within distinct databases:: - - DefaultBase.metadata.create_all(some_engine) - OtherBase.metadata.create_all(some_other_engine) - - -``__table_cls__`` -~~~~~~~~~~~~~~~~~ - -Allows the callable / class used to generate a :class:`_schema.Table` to be customized. -This is a very open-ended hook that can allow special customizations -to a :class:`_schema.Table` that one generates here:: - - class MyMixin(object): - @classmethod - def __table_cls__(cls, name, metadata, *arg, **kw): - return Table( - "my_" + name, - metadata, *arg, **kw - ) - -The above mixin would cause all :class:`_schema.Table` objects generated to include -the prefix ``"my_"``, followed by the name normally specified using the -``__tablename__`` attribute. - -``__table_cls__`` also supports the case of returning ``None``, which -causes the class to be considered as single-table inheritance vs. its subclass. -This may be useful in some customization schemes to determine that single-table -inheritance should take place based on the arguments for the table itself, -such as, define as single-inheritance if there is no primary key present:: - - class AutoTable(object): - @declared_attr - def __tablename__(cls): - return cls.__name__ - - @classmethod - def __table_cls__(cls, *arg, **kw): - for obj in arg[1:]: - if (isinstance(obj, Column) and obj.primary_key) or \ - isinstance(obj, PrimaryKeyConstraint): - return Table(*arg, **kw) - - return None - - class Person(AutoTable, Base): - id = Column(Integer, primary_key=True) - - class Employee(Person): - employee_name = Column(String) - -The above ``Employee`` class would be mapped as single-table inheritance -against ``Person``; the ``employee_name`` column would be added as a member -of the ``Person`` table. - - -.. versionadded:: 1.0.0 diff --git a/doc/build/orm/extensions/declarative/basic_use.rst b/doc/build/orm/extensions/declarative/basic_use.rst index b939f7e39..f1ce1d4a0 100644 --- a/doc/build/orm/extensions/declarative/basic_use.rst +++ b/doc/build/orm/extensions/declarative/basic_use.rst @@ -2,109 +2,19 @@ Basic Use ========= -.. seealso:: - - This section describes specifics about how the Declarative system - interacts with the SQLAlchemy ORM. For a general introduction - to class mapping, see :ref:`ormtutorial_toplevel` as well as - :ref:`mapper_config_toplevel`. - -SQLAlchemy object-relational configuration involves the -combination of :class:`_schema.Table`, :func:`.mapper`, and class -objects to define a mapped class. -:mod:`~sqlalchemy.ext.declarative` allows all three to be -expressed at once within the class declaration. As much as -possible, regular SQLAlchemy schema and ORM constructs are -used directly, so that configuration between "classical" ORM -usage and declarative remain highly similar. - -As a simple example:: - - from sqlalchemy import Column, Integer, String - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - class SomeClass(Base): - __tablename__ = 'some_table' - id = Column(Integer, primary_key=True) - name = Column(String(50)) - -Above, the :func:`declarative_base` callable returns a new base class from -which all mapped classes should inherit. When the class definition is -completed, a new :class:`_schema.Table` and :func:`.mapper` will have been generated. - -The resulting table and mapper are accessible via -``__table__`` and ``__mapper__`` attributes on the -``SomeClass`` class:: - - # access the mapped Table - SomeClass.__table__ - - # access the Mapper - SomeClass.__mapper__ +This section has moved to :ref:`orm_declarative_mapping`. Defining Attributes =================== -In the previous example, the :class:`_schema.Column` objects are -automatically named with the name of the attribute to which they are -assigned. - -To name columns explicitly with a name distinct from their mapped attribute, -just give the column a name. Below, column "some_table_id" is mapped to the -"id" attribute of `SomeClass`, but in SQL will be represented as -"some_table_id":: - - class SomeClass(Base): - __tablename__ = 'some_table' - id = Column("some_table_id", Integer, primary_key=True) - -Attributes may be added to the class after its construction, and they will be -added to the underlying :class:`_schema.Table` and -:func:`.mapper` definitions as appropriate:: +This section is covered by :ref:`mapping_columns_toplevel` - SomeClass.data = Column('data', Unicode) - SomeClass.related = relationship(RelatedInfo) -Classes which are constructed using declarative can interact freely -with classes that are mapped explicitly with :func:`.mapper`. - - -.. sidebar:: Using MyPy with SQLAlchemy models - - If you are using PEP 484 static type checkers for Python, a `MyPy <http://mypy-lang.org/>`_ - plugin is included with - `type stubs for SQLAlchemy <https://github.com/dropbox/sqlalchemy-stubs>`_. The plugin - is tailored towards SQLAlchemy declarative models. - - -It is recommended, though not required, that all tables -share the same underlying :class:`~sqlalchemy.schema.MetaData` object, -so that string-configured :class:`~sqlalchemy.schema.ForeignKey` -references can be resolved without issue. Accessing the MetaData ====================== -The :func:`declarative_base` base class contains a -:class:`_schema.MetaData` object where newly defined -:class:`_schema.Table` objects are collected. This object is -intended to be accessed directly for -:class:`_schema.MetaData`-specific operations. Such as, to issue -CREATE statements for all tables:: - - engine = create_engine('sqlite://') - Base.metadata.create_all(engine) - -:func:`declarative_base` can also receive a pre-existing -:class:`_schema.MetaData` object, which allows a -declarative setup to be associated with an already -existing traditional collection of :class:`~sqlalchemy.schema.Table` -objects:: - - mymetadata = MetaData() - Base = declarative_base(metadata=mymetadata) +This section has moved to :ref:`orm_declarative_metadata`. Class Constructor @@ -119,25 +29,7 @@ to the named attributes:: Mapper Configuration ==================== -Declarative makes use of the :func:`_orm.mapper` function internally -when it creates the mapping to the declared table. The options -for :func:`_orm.mapper` are passed directly through via the -``__mapper_args__`` class attribute. As always, arguments which reference -locally mapped columns can reference them directly from within the -class declaration:: - - from datetime import datetime - - class Widget(Base): - __tablename__ = 'widgets' - - id = Column(Integer, primary_key=True) - timestamp = Column(DateTime, nullable=False) - - __mapper_args__ = { - 'version_id_col': timestamp, - 'version_id_generator': lambda v:datetime.now() - } +This section is moved to :ref:`orm_declarative_mapper_options`. .. _declarative_sql_expressions: diff --git a/doc/build/orm/extensions/declarative/index.rst b/doc/build/orm/extensions/declarative/index.rst index 43972b03e..36700f812 100644 --- a/doc/build/orm/extensions/declarative/index.rst +++ b/doc/build/orm/extensions/declarative/index.rst @@ -1,32 +1,23 @@ .. _declarative_toplevel: -=========== -Declarative -=========== +.. currentmodule:: sqlalchemy.ext.declarative -The Declarative system is the typically used system provided by the SQLAlchemy -ORM in order to define classes mapped to relational database tables. However, -as noted in :ref:`classical_mapping`, Declarative is in fact a series of -extensions that ride on top of the SQLAlchemy :func:`.mapper` construct. - -While the documentation typically refers to Declarative for most examples, -the following sections will provide detailed information on how the -Declarative API interacts with the basic :func:`.mapper` and Core :class:`_schema.Table` -systems, as well as how sophisticated patterns can be built using systems -such as mixins. - - -.. toctree:: - :maxdepth: 2 - - basic_use - relationships - table_config - inheritance - mixins - api +====================== +Declarative Extensions +====================== +Extensions specific to the :ref:`Declarative <orm_declarative_mapping>` +mapping API. +.. versionchanged:: 1.4 The vast majority of the Declarative extension is now + integrated into the SQLAlchemy ORM and is importable from the + ``sqlalchemy.orm`` namespace. See the documentation at + :ref:`orm_declarative_mapping` for new documentation. + For an overview of the change, see :ref:`change_5508`. +.. autoclass:: AbstractConcreteBase +.. autoclass:: ConcreteBase +.. autoclass:: DeferredReflection + :members: diff --git a/doc/build/orm/extensions/declarative/inheritance.rst b/doc/build/orm/extensions/declarative/inheritance.rst index fcbdc0a94..70148986b 100644 --- a/doc/build/orm/extensions/declarative/inheritance.rst +++ b/doc/build/orm/extensions/declarative/inheritance.rst @@ -1,250 +1,3 @@ .. _declarative_inheritance: -Inheritance Configuration -========================= - -Declarative supports all three forms of inheritance as intuitively -as possible. The ``inherits`` mapper keyword argument is not needed -as declarative will determine this from the class itself. The various -"polymorphic" keyword arguments are specified using ``__mapper_args__``. - -.. seealso:: - - This section describes some specific details on how the Declarative system - interacts with SQLAlchemy ORM inheritance configuration. See - :ref:`inheritance_toplevel` for a general introduction to inheritance - mapping. - -Joined Table Inheritance -~~~~~~~~~~~~~~~~~~~~~~~~ - -Joined table inheritance is defined as a subclass that defines its own -table:: - - class Person(Base): - __tablename__ = 'people' - id = Column(Integer, primary_key=True) - discriminator = Column('type', String(50)) - __mapper_args__ = {'polymorphic_on': discriminator} - - class Engineer(Person): - __tablename__ = 'engineers' - __mapper_args__ = {'polymorphic_identity': 'engineer'} - id = Column(Integer, ForeignKey('people.id'), primary_key=True) - primary_language = Column(String(50)) - -Note that above, the ``Engineer.id`` attribute, since it shares the -same attribute name as the ``Person.id`` attribute, will in fact -represent the ``people.id`` and ``engineers.id`` columns together, -with the "Engineer.id" column taking precedence if queried directly. -To provide the ``Engineer`` class with an attribute that represents -only the ``engineers.id`` column, give it a different attribute name:: - - class Engineer(Person): - __tablename__ = 'engineers' - __mapper_args__ = {'polymorphic_identity': 'engineer'} - engineer_id = Column('id', Integer, ForeignKey('people.id'), - primary_key=True) - primary_language = Column(String(50)) - - -.. _declarative_single_table: - -Single Table Inheritance -~~~~~~~~~~~~~~~~~~~~~~~~ - -Single table inheritance is defined as a subclass that does not have -its own table; you just leave out the ``__table__`` and ``__tablename__`` -attributes:: - - class Person(Base): - __tablename__ = 'people' - id = Column(Integer, primary_key=True) - discriminator = Column('type', String(50)) - __mapper_args__ = {'polymorphic_on': discriminator} - - class Engineer(Person): - __mapper_args__ = {'polymorphic_identity': 'engineer'} - primary_language = Column(String(50)) - -When the above mappers are configured, the ``Person`` class is mapped -to the ``people`` table *before* the ``primary_language`` column is -defined, and this column will not be included in its own mapping. -When ``Engineer`` then defines the ``primary_language`` column, the -column is added to the ``people`` table so that it is included in the -mapping for ``Engineer`` and is also part of the table's full set of -columns. Columns which are not mapped to ``Person`` are also excluded -from any other single or joined inheriting classes using the -``exclude_properties`` mapper argument. Below, ``Manager`` will have -all the attributes of ``Person`` and ``Manager`` but *not* the -``primary_language`` attribute of ``Engineer``:: - - class Manager(Person): - __mapper_args__ = {'polymorphic_identity': 'manager'} - golf_swing = Column(String(50)) - -The attribute exclusion logic is provided by the -``exclude_properties`` mapper argument, and declarative's default -behavior can be disabled by passing an explicit ``exclude_properties`` -collection (empty or otherwise) to the ``__mapper_args__``. - -.. _declarative_column_conflicts: - -Resolving Column Conflicts -^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Note above that the ``primary_language`` and ``golf_swing`` columns -are "moved up" to be applied to ``Person.__table__``, as a result of their -declaration on a subclass that has no table of its own. A tricky case -comes up when two subclasses want to specify *the same* column, as below:: - - class Person(Base): - __tablename__ = 'people' - id = Column(Integer, primary_key=True) - discriminator = Column('type', String(50)) - __mapper_args__ = {'polymorphic_on': discriminator} - - class Engineer(Person): - __mapper_args__ = {'polymorphic_identity': 'engineer'} - start_date = Column(DateTime) - - class Manager(Person): - __mapper_args__ = {'polymorphic_identity': 'manager'} - start_date = Column(DateTime) - -Above, the ``start_date`` column declared on both ``Engineer`` and ``Manager`` -will result in an error:: - - sqlalchemy.exc.ArgumentError: Column 'start_date' on class - <class '__main__.Manager'> conflicts with existing - column 'people.start_date' - -In a situation like this, Declarative can't be sure -of the intent, especially if the ``start_date`` columns had, for example, -different types. A situation like this can be resolved by using -:class:`.declared_attr` to define the :class:`_schema.Column` conditionally, taking -care to return the **existing column** via the parent ``__table__`` if it -already exists:: - - from sqlalchemy.ext.declarative import declared_attr - - class Person(Base): - __tablename__ = 'people' - id = Column(Integer, primary_key=True) - discriminator = Column('type', String(50)) - __mapper_args__ = {'polymorphic_on': discriminator} - - class Engineer(Person): - __mapper_args__ = {'polymorphic_identity': 'engineer'} - - @declared_attr - def start_date(cls): - "Start date column, if not present already." - return Person.__table__.c.get('start_date', Column(DateTime)) - - class Manager(Person): - __mapper_args__ = {'polymorphic_identity': 'manager'} - - @declared_attr - def start_date(cls): - "Start date column, if not present already." - return Person.__table__.c.get('start_date', Column(DateTime)) - -Above, when ``Manager`` is mapped, the ``start_date`` column is -already present on the ``Person`` class. Declarative lets us return -that :class:`_schema.Column` as a result in this case, where it knows to skip -re-assigning the same column. If the mapping is mis-configured such -that the ``start_date`` column is accidentally re-assigned to a -different table (such as, if we changed ``Manager`` to be joined -inheritance without fixing ``start_date``), an error is raised which -indicates an existing :class:`_schema.Column` is trying to be re-assigned to -a different owning :class:`_schema.Table`. - -The same concept can be used with mixin classes (see -:ref:`declarative_mixins`):: - - class Person(Base): - __tablename__ = 'people' - id = Column(Integer, primary_key=True) - discriminator = Column('type', String(50)) - __mapper_args__ = {'polymorphic_on': discriminator} - - class HasStartDate(object): - @declared_attr - def start_date(cls): - return cls.__table__.c.get('start_date', Column(DateTime)) - - class Engineer(HasStartDate, Person): - __mapper_args__ = {'polymorphic_identity': 'engineer'} - - class Manager(HasStartDate, Person): - __mapper_args__ = {'polymorphic_identity': 'manager'} - -The above mixin checks the local ``__table__`` attribute for the column. -Because we're using single table inheritance, we're sure that in this case, -``cls.__table__`` refers to ``Person.__table__``. If we were mixing joined- -and single-table inheritance, we might want our mixin to check more carefully -if ``cls.__table__`` is really the :class:`_schema.Table` we're looking for. - -.. _declarative_concrete_table: - -Concrete Table Inheritance -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Concrete is defined as a subclass which has its own table and sets the -``concrete`` keyword argument to ``True``:: - - class Person(Base): - __tablename__ = 'people' - id = Column(Integer, primary_key=True) - name = Column(String(50)) - - class Engineer(Person): - __tablename__ = 'engineers' - __mapper_args__ = {'concrete':True} - id = Column(Integer, primary_key=True) - primary_language = Column(String(50)) - name = Column(String(50)) - -Usage of an abstract base class is a little less straightforward as it -requires usage of :func:`~sqlalchemy.orm.util.polymorphic_union`, -which needs to be created with the :class:`_schema.Table` objects -before the class is built:: - - engineers = Table('engineers', Base.metadata, - Column('id', Integer, primary_key=True), - Column('name', String(50)), - Column('primary_language', String(50)) - ) - managers = Table('managers', Base.metadata, - Column('id', Integer, primary_key=True), - Column('name', String(50)), - Column('golf_swing', String(50)) - ) - - punion = polymorphic_union({ - 'engineer':engineers, - 'manager':managers - }, 'type', 'punion') - - class Person(Base): - __table__ = punion - __mapper_args__ = {'polymorphic_on':punion.c.type} - - class Engineer(Person): - __table__ = engineers - __mapper_args__ = {'polymorphic_identity':'engineer', 'concrete':True} - - class Manager(Person): - __table__ = managers - __mapper_args__ = {'polymorphic_identity':'manager', 'concrete':True} - -The helper classes :class:`.AbstractConcreteBase` and :class:`.ConcreteBase` -provide automation for the above system of creating a polymorphic union. -See the documentation for these helpers as well as the main ORM documentation -on concrete inheritance for details. - -.. seealso:: - - :ref:`concrete_inheritance` - +See :ref:`inheritance_toplevel` for this section. diff --git a/doc/build/orm/extensions/declarative/mixins.rst b/doc/build/orm/extensions/declarative/mixins.rst index 509b1d34c..221e8f8f8 100644 --- a/doc/build/orm/extensions/declarative/mixins.rst +++ b/doc/build/orm/extensions/declarative/mixins.rst @@ -3,542 +3,4 @@ Mixin and Custom Base Classes ============================= -A common need when using :mod:`~sqlalchemy.ext.declarative` is to -share some functionality, such as a set of common columns, some common -table options, or other mapped properties, across many -classes. The standard Python idioms for this is to have the classes -inherit from a base which includes these common features. - -When using :mod:`~sqlalchemy.ext.declarative`, this idiom is allowed -via the usage of a custom declarative base class, as well as a "mixin" class -which is inherited from in addition to the primary base. Declarative -includes several helper features to make this work in terms of how -mappings are declared. An example of some commonly mixed-in -idioms is below:: - - from sqlalchemy.ext.declarative import declared_attr - - class MyMixin(object): - - @declared_attr - def __tablename__(cls): - return cls.__name__.lower() - - __table_args__ = {'mysql_engine': 'InnoDB'} - __mapper_args__= {'always_refresh': True} - - id = Column(Integer, primary_key=True) - - class MyModel(MyMixin, Base): - name = Column(String(1000)) - -Where above, the class ``MyModel`` will contain an "id" column -as the primary key, a ``__tablename__`` attribute that derives -from the name of the class itself, as well as ``__table_args__`` -and ``__mapper_args__`` defined by the ``MyMixin`` mixin class. - -There's no fixed convention over whether ``MyMixin`` precedes -``Base`` or not. Normal Python method resolution rules apply, and -the above example would work just as well with:: - - class MyModel(Base, MyMixin): - name = Column(String(1000)) - -This works because ``Base`` here doesn't define any of the -variables that ``MyMixin`` defines, i.e. ``__tablename__``, -``__table_args__``, ``id``, etc. If the ``Base`` did define -an attribute of the same name, the class placed first in the -inherits list would determine which attribute is used on the -newly defined class. - -Augmenting the Base -~~~~~~~~~~~~~~~~~~~ - -In addition to using a pure mixin, most of the techniques in this -section can also be applied to the base class itself, for patterns that -should apply to all classes derived from a particular base. This is achieved -using the ``cls`` argument of the :func:`.declarative_base` function:: - - from sqlalchemy.ext.declarative import declared_attr - - class Base(object): - @declared_attr - def __tablename__(cls): - return cls.__name__.lower() - - __table_args__ = {'mysql_engine': 'InnoDB'} - - id = Column(Integer, primary_key=True) - - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base(cls=Base) - - class MyModel(Base): - name = Column(String(1000)) - -Where above, ``MyModel`` and all other classes that derive from ``Base`` will -have a table name derived from the class name, an ``id`` primary key column, -as well as the "InnoDB" engine for MySQL. - -Mixing in Columns -~~~~~~~~~~~~~~~~~ - -The most basic way to specify a column on a mixin is by simple -declaration:: - - class TimestampMixin(object): - created_at = Column(DateTime, default=func.now()) - - class MyModel(TimestampMixin, Base): - __tablename__ = 'test' - - id = Column(Integer, primary_key=True) - name = Column(String(1000)) - -Where above, all declarative classes that include ``TimestampMixin`` -will also have a column ``created_at`` that applies a timestamp to -all row insertions. - -Those familiar with the SQLAlchemy expression language know that -the object identity of clause elements defines their role in a schema. -Two ``Table`` objects ``a`` and ``b`` may both have a column called -``id``, but the way these are differentiated is that ``a.c.id`` -and ``b.c.id`` are two distinct Python objects, referencing their -parent tables ``a`` and ``b`` respectively. - -In the case of the mixin column, it seems that only one -:class:`_schema.Column` object is explicitly created, yet the ultimate -``created_at`` column above must exist as a distinct Python object -for each separate destination class. To accomplish this, the declarative -extension creates a **copy** of each :class:`_schema.Column` object encountered on -a class that is detected as a mixin. - -This copy mechanism is limited to simple columns that have no foreign -keys, as a :class:`_schema.ForeignKey` itself contains references to columns -which can't be properly recreated at this level. For columns that -have foreign keys, as well as for the variety of mapper-level constructs -that require destination-explicit context, the -:class:`~.declared_attr` decorator is provided so that -patterns common to many classes can be defined as callables:: - - from sqlalchemy.ext.declarative import declared_attr - - class ReferenceAddressMixin(object): - @declared_attr - def address_id(cls): - return Column(Integer, ForeignKey('address.id')) - - class User(ReferenceAddressMixin, Base): - __tablename__ = 'user' - id = Column(Integer, primary_key=True) - -Where above, the ``address_id`` class-level callable is executed at the -point at which the ``User`` class is constructed, and the declarative -extension can use the resulting :class:`_schema.Column` object as returned by -the method without the need to copy it. - -Columns generated by :class:`~.declared_attr` can also be -referenced by ``__mapper_args__`` to a limited degree, currently -by ``polymorphic_on`` and ``version_id_col``; the declarative extension -will resolve them at class construction time:: - - class MyMixin: - @declared_attr - def type_(cls): - return Column(String(50)) - - __mapper_args__= {'polymorphic_on':type_} - - class MyModel(MyMixin, Base): - __tablename__='test' - id = Column(Integer, primary_key=True) - - -Mixing in Relationships -~~~~~~~~~~~~~~~~~~~~~~~ - -Relationships created by :func:`~sqlalchemy.orm.relationship` are provided -with declarative mixin classes exclusively using the -:class:`.declared_attr` approach, eliminating any ambiguity -which could arise when copying a relationship and its possibly column-bound -contents. Below is an example which combines a foreign key column and a -relationship so that two classes ``Foo`` and ``Bar`` can both be configured to -reference a common target class via many-to-one:: - - class RefTargetMixin(object): - @declared_attr - def target_id(cls): - return Column('target_id', ForeignKey('target.id')) - - @declared_attr - def target(cls): - return relationship("Target") - - class Foo(RefTargetMixin, Base): - __tablename__ = 'foo' - id = Column(Integer, primary_key=True) - - class Bar(RefTargetMixin, Base): - __tablename__ = 'bar' - id = Column(Integer, primary_key=True) - - class Target(Base): - __tablename__ = 'target' - id = Column(Integer, primary_key=True) - - -Using Advanced Relationship Arguments (e.g. ``primaryjoin``, etc.) -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -:func:`~sqlalchemy.orm.relationship` definitions which require explicit -primaryjoin, order_by etc. expressions should in all but the most -simplistic cases use **late bound** forms -for these arguments, meaning, using either the string form or a lambda. -The reason for this is that the related :class:`_schema.Column` objects which are to -be configured using ``@declared_attr`` are not available to another -``@declared_attr`` attribute; while the methods will work and return new -:class:`_schema.Column` objects, those are not the :class:`_schema.Column` objects that -Declarative will be using as it calls the methods on its own, thus using -*different* :class:`_schema.Column` objects. - -The canonical example is the primaryjoin condition that depends upon -another mixed-in column:: - - class RefTargetMixin(object): - @declared_attr - def target_id(cls): - return Column('target_id', ForeignKey('target.id')) - - @declared_attr - def target(cls): - return relationship(Target, - primaryjoin=Target.id==cls.target_id # this is *incorrect* - ) - -Mapping a class using the above mixin, we will get an error like:: - - sqlalchemy.exc.InvalidRequestError: this ForeignKey's parent column is not - yet associated with a Table. - -This is because the ``target_id`` :class:`_schema.Column` we've called upon in our -``target()`` method is not the same :class:`_schema.Column` that declarative is -actually going to map to our table. - -The condition above is resolved using a lambda:: - - class RefTargetMixin(object): - @declared_attr - def target_id(cls): - return Column('target_id', ForeignKey('target.id')) - - @declared_attr - def target(cls): - return relationship(Target, - primaryjoin=lambda: Target.id==cls.target_id - ) - -or alternatively, the string form (which ultimately generates a lambda):: - - class RefTargetMixin(object): - @declared_attr - def target_id(cls): - return Column('target_id', ForeignKey('target.id')) - - @declared_attr - def target(cls): - return relationship("Target", - primaryjoin="Target.id==%s.target_id" % cls.__name__ - ) - -Mixing in deferred(), column_property(), and other MapperProperty classes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Like :func:`~sqlalchemy.orm.relationship`, all -:class:`~sqlalchemy.orm.interfaces.MapperProperty` subclasses such as -:func:`~sqlalchemy.orm.deferred`, :func:`~sqlalchemy.orm.column_property`, -etc. ultimately involve references to columns, and therefore, when -used with declarative mixins, have the :class:`.declared_attr` -requirement so that no reliance on copying is needed:: - - class SomethingMixin(object): - - @declared_attr - def dprop(cls): - return deferred(Column(Integer)) - - class Something(SomethingMixin, Base): - __tablename__ = "something" - -The :func:`.column_property` or other construct may refer -to other columns from the mixin. These are copied ahead of time before -the :class:`.declared_attr` is invoked:: - - class SomethingMixin(object): - x = Column(Integer) - - y = Column(Integer) - - @declared_attr - def x_plus_y(cls): - return column_property(cls.x + cls.y) - - -.. versionchanged:: 1.0.0 mixin columns are copied to the final mapped class - so that :class:`.declared_attr` methods can access the actual column - that will be mapped. - -Mixing in Association Proxy and Other Attributes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Mixins can specify user-defined attributes as well as other extension -units such as :func:`.association_proxy`. The usage of -:class:`.declared_attr` is required in those cases where the attribute must -be tailored specifically to the target subclass. An example is when -constructing multiple :func:`.association_proxy` attributes which each -target a different type of child object. Below is an -:func:`.association_proxy` / mixin example which provides a scalar list of -string values to an implementing class:: - - from sqlalchemy import Column, Integer, ForeignKey, String - from sqlalchemy.orm import relationship - from sqlalchemy.ext.associationproxy import association_proxy - from sqlalchemy.ext.declarative import declarative_base, declared_attr - - Base = declarative_base() - - class HasStringCollection(object): - @declared_attr - def _strings(cls): - class StringAttribute(Base): - __tablename__ = cls.string_table_name - id = Column(Integer, primary_key=True) - value = Column(String(50), nullable=False) - parent_id = Column(Integer, - ForeignKey('%s.id' % cls.__tablename__), - nullable=False) - def __init__(self, value): - self.value = value - - return relationship(StringAttribute) - - @declared_attr - def strings(cls): - return association_proxy('_strings', 'value') - - class TypeA(HasStringCollection, Base): - __tablename__ = 'type_a' - string_table_name = 'type_a_strings' - id = Column(Integer(), primary_key=True) - - class TypeB(HasStringCollection, Base): - __tablename__ = 'type_b' - string_table_name = 'type_b_strings' - id = Column(Integer(), primary_key=True) - -Above, the ``HasStringCollection`` mixin produces a :func:`_orm.relationship` -which refers to a newly generated class called ``StringAttribute``. The -``StringAttribute`` class is generated with its own :class:`_schema.Table` -definition which is local to the parent class making usage of the -``HasStringCollection`` mixin. It also produces an :func:`.association_proxy` -object which proxies references to the ``strings`` attribute onto the ``value`` -attribute of each ``StringAttribute`` instance. - -``TypeA`` or ``TypeB`` can be instantiated given the constructor -argument ``strings``, a list of strings:: - - ta = TypeA(strings=['foo', 'bar']) - tb = TypeA(strings=['bat', 'bar']) - -This list will generate a collection -of ``StringAttribute`` objects, which are persisted into a table that's -local to either the ``type_a_strings`` or ``type_b_strings`` table:: - - >>> print(ta._strings) - [<__main__.StringAttribute object at 0x10151cd90>, - <__main__.StringAttribute object at 0x10151ce10>] - -When constructing the :func:`.association_proxy`, the -:class:`.declared_attr` decorator must be used so that a distinct -:func:`.association_proxy` object is created for each of the ``TypeA`` -and ``TypeB`` classes. - -.. _decl_mixin_inheritance: - -Controlling table inheritance with mixins -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The ``__tablename__`` attribute may be used to provide a function that -will determine the name of the table used for each class in an inheritance -hierarchy, as well as whether a class has its own distinct table. - -This is achieved using the :class:`.declared_attr` indicator in conjunction -with a method named ``__tablename__()``. Declarative will always -invoke :class:`.declared_attr` for the special names -``__tablename__``, ``__mapper_args__`` and ``__table_args__`` -function **for each mapped class in the hierarchy, except if overridden -in a subclass**. The function therefore -needs to expect to receive each class individually and to provide the -correct answer for each. - -For example, to create a mixin that gives every class a simple table -name based on class name:: - - from sqlalchemy.ext.declarative import declared_attr - - class Tablename: - @declared_attr - def __tablename__(cls): - return cls.__name__.lower() - - class Person(Tablename, Base): - id = Column(Integer, primary_key=True) - discriminator = Column('type', String(50)) - __mapper_args__ = {'polymorphic_on': discriminator} - - class Engineer(Person): - __tablename__ = None - __mapper_args__ = {'polymorphic_identity': 'engineer'} - primary_language = Column(String(50)) - -Alternatively, we can modify our ``__tablename__`` function to return -``None`` for subclasses, using :func:`.has_inherited_table`. This has -the effect of those subclasses being mapped with single table inheritance -against the parent:: - - from sqlalchemy.ext.declarative import declared_attr - from sqlalchemy.ext.declarative import has_inherited_table - - class Tablename(object): - @declared_attr - def __tablename__(cls): - if has_inherited_table(cls): - return None - return cls.__name__.lower() - - class Person(Tablename, Base): - id = Column(Integer, primary_key=True) - discriminator = Column('type', String(50)) - __mapper_args__ = {'polymorphic_on': discriminator} - - class Engineer(Person): - primary_language = Column(String(50)) - __mapper_args__ = {'polymorphic_identity': 'engineer'} - -.. _mixin_inheritance_columns: - -Mixing in Columns in Inheritance Scenarios -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -In contrast to how ``__tablename__`` and other special names are handled when -used with :class:`.declared_attr`, when we mix in columns and properties (e.g. -relationships, column properties, etc.), the function is -invoked for the **base class only** in the hierarchy. Below, only the -``Person`` class will receive a column -called ``id``; the mapping will fail on ``Engineer``, which is not given -a primary key:: - - class HasId(object): - @declared_attr - def id(cls): - return Column('id', Integer, primary_key=True) - - class Person(HasId, Base): - __tablename__ = 'person' - discriminator = Column('type', String(50)) - __mapper_args__ = {'polymorphic_on': discriminator} - - class Engineer(Person): - __tablename__ = 'engineer' - primary_language = Column(String(50)) - __mapper_args__ = {'polymorphic_identity': 'engineer'} - -It is usually the case in joined-table inheritance that we want distinctly -named columns on each subclass. However in this case, we may want to have -an ``id`` column on every table, and have them refer to each other via -foreign key. We can achieve this as a mixin by using the -:attr:`.declared_attr.cascading` modifier, which indicates that the -function should be invoked **for each class in the hierarchy**, in *almost* -(see warning below) the same way as it does for ``__tablename__``:: - - class HasIdMixin(object): - @declared_attr.cascading - def id(cls): - if has_inherited_table(cls): - return Column(ForeignKey('person.id'), primary_key=True) - else: - return Column(Integer, primary_key=True) - - class Person(HasIdMixin, Base): - __tablename__ = 'person' - discriminator = Column('type', String(50)) - __mapper_args__ = {'polymorphic_on': discriminator} - - class Engineer(Person): - __tablename__ = 'engineer' - primary_language = Column(String(50)) - __mapper_args__ = {'polymorphic_identity': 'engineer'} - -.. warning:: - - The :attr:`.declared_attr.cascading` feature currently does - **not** allow for a subclass to override the attribute with a different - function or value. This is a current limitation in the mechanics of - how ``@declared_attr`` is resolved, and a warning is emitted if - this condition is detected. This limitation does **not** - exist for the special attribute names such as ``__tablename__``, which - resolve in a different way internally than that of - :attr:`.declared_attr.cascading`. - - -.. versionadded:: 1.0.0 added :attr:`.declared_attr.cascading`. - -Combining Table/Mapper Arguments from Multiple Mixins -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -In the case of ``__table_args__`` or ``__mapper_args__`` -specified with declarative mixins, you may want to combine -some parameters from several mixins with those you wish to -define on the class itself. The -:class:`.declared_attr` decorator can be used -here to create user-defined collation routines that pull -from multiple collections:: - - from sqlalchemy.ext.declarative import declared_attr - - class MySQLSettings(object): - __table_args__ = {'mysql_engine':'InnoDB'} - - class MyOtherMixin(object): - __table_args__ = {'info':'foo'} - - class MyModel(MySQLSettings, MyOtherMixin, Base): - __tablename__='my_model' - - @declared_attr - def __table_args__(cls): - args = dict() - args.update(MySQLSettings.__table_args__) - args.update(MyOtherMixin.__table_args__) - return args - - id = Column(Integer, primary_key=True) - -Creating Indexes with Mixins -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -To define a named, potentially multicolumn :class:`.Index` that applies to all -tables derived from a mixin, use the "inline" form of :class:`.Index` and -establish it as part of ``__table_args__``:: - - class MyMixin(object): - a = Column(Integer) - b = Column(Integer) - - @declared_attr - def __table_args__(cls): - return (Index('test_idx_%s' % cls.__tablename__, 'a', 'b'),) - - class MyModel(MyMixin, Base): - __tablename__ = 'atable' - c = Column(Integer,primary_key=True) +See :ref:`orm_mixins_toplevel` for this section.
\ No newline at end of file diff --git a/doc/build/orm/extensions/declarative/relationships.rst b/doc/build/orm/extensions/declarative/relationships.rst index ac2671c52..a5884ef10 100644 --- a/doc/build/orm/extensions/declarative/relationships.rst +++ b/doc/build/orm/extensions/declarative/relationships.rst @@ -4,139 +4,14 @@ Configuring Relationships ========================= -.. seealso:: - - This section describes specifics about how the Declarative system - interacts with SQLAlchemy ORM relationship constructs. For general - information about setting up relationships between mappings, - see :ref:`ormtutorial_toplevel` and :ref:`relationship_patterns`. - -Relationships to other classes are done in the usual way, with the added -feature that the class specified to :func:`~sqlalchemy.orm.relationship` -may be a string name. The "class registry" associated with ``Base`` -is used at mapper compilation time to resolve the name into the actual -class object, which is expected to have been defined once the mapper -configuration is used:: - - class User(Base): - __tablename__ = 'users' - - id = Column(Integer, primary_key=True) - name = Column(String(50)) - addresses = relationship("Address", backref="user") - - class Address(Base): - __tablename__ = 'addresses' - - id = Column(Integer, primary_key=True) - email = Column(String(50)) - user_id = Column(Integer, ForeignKey('users.id')) - -Column constructs, since they are just that, are immediately usable, -as below where we define a primary join condition on the ``Address`` -class using them:: - - class Address(Base): - __tablename__ = 'addresses' - - id = Column(Integer, primary_key=True) - email = Column(String(50)) - user_id = Column(Integer, ForeignKey('users.id')) - user = relationship(User, primaryjoin=user_id == User.id) +This section is covered by :ref:`orm_declarative_properties`. .. _declarative_relationship_eval: Evaluation of relationship arguments ===================================== -In addition to the main argument for :func:`~sqlalchemy.orm.relationship`, -other arguments which depend upon the columns present on an as-yet -undefined class may also be specified as strings. For most of these -arguments except that of the main argument, these strings are -**evaluated as Python expressions using Python's built-in eval() function.** - -The full namespace available within this evaluation includes all classes mapped -for this declarative base, as well as the contents of the ``sqlalchemy`` -package, including expression functions like -:func:`~sqlalchemy.sql.expression.desc` and -:attr:`~sqlalchemy.sql.expression.func`:: - - class User(Base): - # .... - addresses = relationship("Address", - order_by="desc(Address.email)", - primaryjoin="Address.user_id==User.id") - -.. warning:: - - The strings accepted by the following parameters: - - :paramref:`_orm.relationship.order_by` - - :paramref:`_orm.relationship.primaryjoin` - - :paramref:`_orm.relationship.secondaryjoin` - - :paramref:`_orm.relationship.secondary` - - :paramref:`_orm.relationship.remote_side` - - :paramref:`_orm.relationship.foreign_keys` - - :paramref:`_orm.relationship._user_defined_foreign_keys` - - Are **evaluated as Python code expressions using eval(). DO NOT PASS - UNTRUSTED INPUT TO THESE ARGUMENTS.** - - In addition, prior to version 1.3.16 of SQLAlchemy, the main - "argument" to :func:`_orm.relationship` is also evaluated as Python - code. **DO NOT PASS UNTRUSTED INPUT TO THIS ARGUMENT.** - -.. versionchanged:: 1.3.16 - - The string evaluation of the main "argument" no longer accepts an open - ended Python expression, instead only accepting a string class name - or dotted package-qualified name. - -For the case where more than one module contains a class of the same name, -string class names can also be specified as module-qualified paths -within any of these string expressions:: - - class User(Base): - # .... - addresses = relationship("myapp.model.address.Address", - order_by="desc(myapp.model.address.Address.email)", - primaryjoin="myapp.model.address.Address.user_id==" - "myapp.model.user.User.id") - -The qualified path can be any partial path that removes ambiguity between -the names. For example, to disambiguate between -``myapp.model.address.Address`` and ``myapp.model.lookup.Address``, -we can specify ``address.Address`` or ``lookup.Address``:: - - class User(Base): - # .... - addresses = relationship("address.Address", - order_by="desc(address.Address.email)", - primaryjoin="address.Address.user_id==" - "User.id") - -Two alternatives also exist to using string-based attributes. A lambda -can also be used, which will be evaluated after all mappers have been -configured:: - - class User(Base): - # ... - addresses = relationship(lambda: Address, - order_by=lambda: desc(Address.email), - primaryjoin=lambda: Address.user_id==User.id) - -Or, the relationship can be added to the class explicitly after the classes -are available:: - - User.addresses = relationship(Address, - primaryjoin=Address.user_id==User.id) - +This section is moved to :ref:`orm_declarative_relationship_eval`. .. _declarative_many_to_many: @@ -144,37 +19,5 @@ are available:: Configuring Many-to-Many Relationships ====================================== -Many-to-many relationships are also declared in the same way -with declarative as with traditional mappings. The -``secondary`` argument to -:func:`_orm.relationship` is as usual passed a -:class:`_schema.Table` object, which is typically declared in the -traditional way. The :class:`_schema.Table` usually shares -the :class:`_schema.MetaData` object used by the declarative base:: - - keyword_author = Table( - 'keyword_author', Base.metadata, - Column('author_id', Integer, ForeignKey('authors.id')), - Column('keyword_id', Integer, ForeignKey('keywords.id')) - ) - - class Author(Base): - __tablename__ = 'authors' - id = Column(Integer, primary_key=True) - keywords = relationship("Keyword", secondary=keyword_author) - -Like other :func:`~sqlalchemy.orm.relationship` arguments, a string is accepted -as well, passing the string name of the table as defined in the -``Base.metadata.tables`` collection:: - - class Author(Base): - __tablename__ = 'authors' - id = Column(Integer, primary_key=True) - keywords = relationship("Keyword", secondary="keyword_author") - -As with traditional mapping, its generally not a good idea to use -a :class:`_schema.Table` as the "secondary" argument which is also mapped to -a class, unless the :func:`_orm.relationship` is declared with ``viewonly=True``. -Otherwise, the unit-of-work system may attempt duplicate INSERT and -DELETE statements against the underlying table. +this section is moved to :ref:`orm_declarative_relationship_secondary_eval`. diff --git a/doc/build/orm/extensions/declarative/table_config.rst b/doc/build/orm/extensions/declarative/table_config.rst index b35f54d7d..d51fb1831 100644 --- a/doc/build/orm/extensions/declarative/table_config.rst +++ b/doc/build/orm/extensions/declarative/table_config.rst @@ -4,145 +4,19 @@ Table Configuration =================== -.. seealso:: +This section has moved; see :ref:`orm_declarative_table_configuration`. - This section describes specifics about how the Declarative system - defines :class:`_schema.Table` objects that are to be mapped with the - SQLAlchemy ORM. For general information on :class:`_schema.Table` objects - see :ref:`metadata_describing_toplevel`. -Table arguments other than the name, metadata, and mapped Column -arguments are specified using the ``__table_args__`` class attribute. -This attribute accommodates both positional as well as keyword -arguments that are normally sent to the -:class:`~sqlalchemy.schema.Table` constructor. -The attribute can be specified in one of two forms. One is as a -dictionary:: - - class MyClass(Base): - __tablename__ = 'sometable' - __table_args__ = {'mysql_engine':'InnoDB'} - -The other, a tuple, where each argument is positional -(usually constraints):: - - class MyClass(Base): - __tablename__ = 'sometable' - __table_args__ = ( - ForeignKeyConstraint(['id'], ['remote_table.id']), - UniqueConstraint('foo'), - ) - -Keyword arguments can be specified with the above form by -specifying the last argument as a dictionary:: - - class MyClass(Base): - __tablename__ = 'sometable' - __table_args__ = ( - ForeignKeyConstraint(['id'], ['remote_table.id']), - UniqueConstraint('foo'), - {'autoload':True} - ) +.. _declarative_hybrid_table: Using a Hybrid Approach with __table__ ====================================== -As an alternative to ``__tablename__``, a direct -:class:`~sqlalchemy.schema.Table` construct may be used. The -:class:`~sqlalchemy.schema.Column` objects, which in this case require -their names, will be added to the mapping just like a regular mapping -to a table:: - - class MyClass(Base): - __table__ = Table('my_table', Base.metadata, - Column('id', Integer, primary_key=True), - Column('name', String(50)) - ) - -``__table__`` provides a more focused point of control for establishing -table metadata, while still getting most of the benefits of using declarative. -An application that uses reflection might want to load table metadata elsewhere -and pass it to declarative classes:: - - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - Base.metadata.reflect(some_engine) - - class User(Base): - __table__ = metadata.tables['user'] - - class Address(Base): - __table__ = metadata.tables['address'] - -Some configuration schemes may find it more appropriate to use ``__table__``, -such as those which already take advantage of the data-driven nature of -:class:`_schema.Table` to customize and/or automate schema definition. - -Note that when the ``__table__`` approach is used, the object is immediately -usable as a plain :class:`_schema.Table` within the class declaration body itself, -as a Python class is only another syntactical block. Below this is illustrated -by using the ``id`` column in the ``primaryjoin`` condition of a -:func:`_orm.relationship`:: - - class MyClass(Base): - __table__ = Table('my_table', Base.metadata, - Column('id', Integer, primary_key=True), - Column('name', String(50)) - ) +This section has moved; see :ref:`orm_imperative_table_configuration`. - widgets = relationship(Widget, - primaryjoin=Widget.myclass_id==__table__.c.id) - -Similarly, mapped attributes which refer to ``__table__`` can be placed inline, -as below where we assign the ``name`` column to the attribute ``_name``, -generating a synonym for ``name``:: - - from sqlalchemy.ext.declarative import synonym_for - - class MyClass(Base): - __table__ = Table('my_table', Base.metadata, - Column('id', Integer, primary_key=True), - Column('name', String(50)) - ) - - _name = __table__.c.name - - @synonym_for("_name") - def name(self): - return "Name: %s" % _name Using Reflection with Declarative ================================= -It's easy to set up a :class:`_schema.Table` that uses ``autoload=True`` -in conjunction with a mapped class:: - - class MyClass(Base): - __table__ = Table('mytable', Base.metadata, - autoload=True, autoload_with=some_engine) - -However, one improvement that can be made here is to not -require the :class:`_engine.Engine` to be available when classes are -being first declared. To achieve this, use the -:class:`.DeferredReflection` mixin, which sets up mappings -only after a special ``prepare(engine)`` step is called:: - - from sqlalchemy.ext.declarative import declarative_base, DeferredReflection - - Base = declarative_base(cls=DeferredReflection) - - class Foo(Base): - __tablename__ = 'foo' - bars = relationship("Bar") - - class Bar(Base): - __tablename__ = 'bar' - - # illustrate overriding of "bar.foo_id" to have - # a foreign key constraint otherwise not - # reflected, such as when using MySQL - foo_id = Column(Integer, ForeignKey('foo.id')) - - Base.prepare(e) +This section has moved to :ref:`orm_declarative_reflected`. diff --git a/doc/build/orm/inheritance.rst b/doc/build/orm/inheritance.rst index ccda5f20b..12f18c04a 100644 --- a/doc/build/orm/inheritance.rst +++ b/doc/build/orm/inheritance.rst @@ -291,6 +291,108 @@ Note that the mappers for the derived classes Manager and Engineer omit the ``__tablename__``, indicating they do not have a mapped table of their own. +.. _orm_inheritance_column_conflicts: + +Resolving Column Conflicts ++++++++++++++++++++++++++++ + +Note in the previous section that the ``manager_name`` and ``engineer_info`` columns +are "moved up" to be applied to ``Employee.__table__``, as a result of their +declaration on a subclass that has no table of its own. A tricky case +comes up when two subclasses want to specify *the same* column, as below:: + + class Employee(Base): + __tablename__ = 'employee' + id = Column(Integer, primary_key=True) + name = Column(String(50)) + type = Column(String(20)) + + __mapper_args__ = { + 'polymorphic_on':type, + 'polymorphic_identity':'employee' + } + + class Engineer(Employee): + __mapper_args__ = {'polymorphic_identity': 'engineer'} + start_date = Column(DateTime) + + class Manager(Employee): + __mapper_args__ = {'polymorphic_identity': 'manager'} + start_date = Column(DateTime) + +Above, the ``start_date`` column declared on both ``Engineer`` and ``Manager`` +will result in an error:: + + sqlalchemy.exc.ArgumentError: Column 'start_date' on class + <class '__main__.Manager'> conflicts with existing + column 'employee.start_date' + +The above scenario presents an ambiguity to the Declarative mapping system that +may be resolved by using +:class:`.declared_attr` to define the :class:`_schema.Column` conditionally, +taking care to return the **existing column** via the parent ``__table__`` +if it already exists:: + + from sqlalchemy.orm import declared_attr + + class Employee(Base): + __tablename__ = 'employee' + id = Column(Integer, primary_key=True) + name = Column(String(50)) + type = Column(String(20)) + + __mapper_args__ = { + 'polymorphic_on':type, + 'polymorphic_identity':'employee' + } + + class Engineer(Employee): + __mapper_args__ = {'polymorphic_identity': 'engineer'} + + @declared_attr + def start_date(cls): + "Start date column, if not present already." + return Employee.__table__.c.get('start_date', Column(DateTime)) + + class Manager(Employee): + __mapper_args__ = {'polymorphic_identity': 'manager'} + + @declared_attr + def start_date(cls): + "Start date column, if not present already." + return Employee.__table__.c.get('start_date', Column(DateTime)) + +Above, when ``Manager`` is mapped, the ``start_date`` column is +already present on the ``Employee`` class; by returning the existing +:class:`_schema.Column` object, the declarative system recognizes that this +is the same column to be mapped to the two different subclasses separately. + +A similar concept can be used with mixin classes (see :ref:`orm_mixins_toplevel`) +to define a particular series of columns and/or other mapped attributes +from a reusable mixin class:: + + class Employee(Base): + __tablename__ = 'employee' + id = Column(Integer, primary_key=True) + name = Column(String(50)) + type = Column(String(20)) + + __mapper_args__ = { + 'polymorphic_on':type, + 'polymorphic_identity':'employee' + } + + class HasStartDate: + @declared_attr + def start_date(cls): + return cls.__table__.c.get('start_date', Column(DateTime)) + + class Engineer(HasStartDate, Employee): + __mapper_args__ = {'polymorphic_identity': 'engineer'} + + class Manager(HasStartDate, Employee): + __mapper_args__ = {'polymorphic_identity': 'manager'} + Relationships with Single Table Inheritance +++++++++++++++++++++++++++++++++++++++++++ @@ -379,6 +481,7 @@ Above, the ``Manager`` class will have a ``Manager.company`` attribute; loads against the ``employee`` with an additional WHERE clause that limits rows to those with ``type = 'manager'``. + Loading Single Inheritance Mappings +++++++++++++++++++++++++++++++++++ @@ -680,9 +783,6 @@ With a mapping like the above, only instances of ``Manager`` and ``Engineer`` may be persisted; querying against the ``Employee`` class will always produce ``Manager`` and ``Engineer`` objects. -.. seealso:: - - :ref:`declarative_concrete_table` - in the Declarative reference documentation Classical and Semi-Classical Concrete Polymorphic Configuration +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ diff --git a/doc/build/orm/internals.rst b/doc/build/orm/internals.rst index 1a06b73b8..3fdfe500e 100644 --- a/doc/build/orm/internals.rst +++ b/doc/build/orm/internals.rst @@ -56,11 +56,11 @@ sections, are listed here. :members: __get__, __set__, __delete__ :undoc-members: -.. autodata:: sqlalchemy.orm.interfaces.MANYTOONE +.. autodata:: sqlalchemy.orm.MANYTOONE -.. autodata:: sqlalchemy.orm.interfaces.MANYTOMANY +.. autodata:: sqlalchemy.orm.MANYTOMANY -.. autoclass:: sqlalchemy.orm.interfaces.MapperProperty +.. autoclass:: sqlalchemy.orm.MapperProperty :members: .. py:attribute:: info @@ -90,7 +90,7 @@ sections, are listed here. .. autofunction:: sqlalchemy.orm.loading.merge_frozen_result -.. autodata:: sqlalchemy.orm.interfaces.ONETOMANY +.. autodata:: sqlalchemy.orm.ONETOMANY .. autoclass:: sqlalchemy.orm.PropComparator :members: diff --git a/doc/build/orm/loading_objects.rst b/doc/build/orm/loading_objects.rst index 64dce643c..b26b32087 100644 --- a/doc/build/orm/loading_objects.rst +++ b/doc/build/orm/loading_objects.rst @@ -7,7 +7,7 @@ Notes and features regarding the general loading of mapped objects. For an in-depth introduction to querying with the SQLAlchemy ORM, please see the :ref:`ormtutorial_toplevel`. .. toctree:: - :maxdepth: 2 + :maxdepth: 3 loading_columns loading_relationships diff --git a/doc/build/orm/mapper_config.rst b/doc/build/orm/mapper_config.rst index 60ad7f5f9..4de086903 100644 --- a/doc/build/orm/mapper_config.rst +++ b/doc/build/orm/mapper_config.rst @@ -11,9 +11,10 @@ know how to construct and use rudimentary mappers and relationships. .. toctree:: - :maxdepth: 2 + :maxdepth: 3 mapping_styles + declarative_mapping scalar_mapping inheritance nonstandard_mappings diff --git a/doc/build/orm/mapping_api.rst b/doc/build/orm/mapping_api.rst index 250bd26a4..6aa08114d 100644 --- a/doc/build/orm/mapping_api.rst +++ b/doc/build/orm/mapping_api.rst @@ -1,8 +1,23 @@ + .. currentmodule:: sqlalchemy.orm Class Mapping API ================= +.. autoclass:: registry + :members: + +.. autofunction:: declarative_base + +.. autofunction:: as_declarative + +.. autoclass:: declared_attr + :members: + +.. autofunction:: has_inherited_table + +.. autofunction:: synonym_for + .. autofunction:: mapper .. autofunction:: object_mapper diff --git a/doc/build/orm/mapping_columns.rst b/doc/build/orm/mapping_columns.rst index 5423a84eb..29794ce9e 100644 --- a/doc/build/orm/mapping_columns.rst +++ b/doc/build/orm/mapping_columns.rst @@ -1,3 +1,5 @@ +.. _mapping_columns_toplevel: + .. currentmodule:: sqlalchemy.orm Mapping Table Columns @@ -20,8 +22,9 @@ it matches the :attr:`_schema.Column.key` attribute on :class:`_schema.Column`, by default is the same as the :attr:`_schema.Column.name`. The name assigned to the Python attribute which maps to -:class:`_schema.Column` can be different from either :attr:`_schema.Column.name` or :attr:`_schema.Column.key` -just by assigning it that way, as we illustrate here in a Declarative mapping:: +:class:`_schema.Column` can be different from either +:attr:`_schema.Column.name` or :attr:`_schema.Column.key` just by assigning +it that way, as we illustrate here in a Declarative mapping:: class User(Base): __tablename__ = 'user' @@ -39,10 +42,11 @@ can be referenced directly:: id = user_table.c.user_id name = user_table.c.user_name -Or in a classical mapping, placed in the ``properties`` dictionary -with the desired key:: +The corresponding technique for an :term:`imperative` mapping is +to place the desired key in the :paramref:`_orm.mapper.properties` +dictionary with the desired key:: - mapper(User, user_table, properties={ + registry.mapper(User, user_table, properties={ 'id': user_table.c.user_id, 'name': user_table.c.user_name, }) diff --git a/doc/build/orm/mapping_styles.rst b/doc/build/orm/mapping_styles.rst index c156f08f1..29045dbb7 100644 --- a/doc/build/orm/mapping_styles.rst +++ b/doc/build/orm/mapping_styles.rst @@ -1,29 +1,62 @@ -================= -Types of Mappings -================= - -Modern SQLAlchemy features two distinct styles of mapper configuration. -The "Classical" style is SQLAlchemy's original mapping API, whereas -"Declarative" is the richer and more succinct system that builds on top -of "Classical". Both styles may be used interchangeably, as the end -result of each is exactly the same - a user-defined class mapped by the -:func:`.mapper` function onto a selectable unit, typically a :class:`_schema.Table`. +.. _orm_mapping_classes_toplevel: + +======================= +Mapping Python Classes +======================= + +SQLAlchemy historically features two distinct styles of mapper configuration. +The original mapping API is commonly referred to as "classical" style, +whereas the more automated style of mapping is known as "declarative" style. +SQLAlchemy now refers to these two mapping styles as **imperative mapping** +and **declarative mapping**. + +Both styles may be used interchangeably, as the end result of each is exactly +the same - a user-defined class that has a :class:`_orm.Mapper` configured +against a selectable unit, typically represented by a :class:`_schema.Table` +object. + +Both imperative and declarative mapping begin with an ORM :class:`_orm.registry` +object, which maintains a set of classes that are mapped. This registry +is present for all mappings. + +.. versionchanged:: 1.4 Declarative and classical mapping are now referred + to as "declarative" and "imperative" mapping, and are unified internally, + all originating from the :class:`_orm.registry` construct that represents + a collection of related mappings. + +The full suite of styles can be hierarchically organized as follows: + +* :ref:`orm_declarative_mapping` + * Using :func:`_orm.declarative_base` Base class w/ metaclass + * :ref:`orm_declarative_table` + * :ref:`Imperative Table (a.k.a. "hybrid table") <orm_imperative_table_configuration>` + * Using :meth:`_orm.registry.mapped` Declarative Decorator + * Declarative Table + * Imperative Table (Hybrid) + * :ref:`orm_declarative_dataclasses` +* :ref:`Imperative (a.k.a. "classical" mapping) <orm_imperative_mapping>` + * Using :meth:`_orm.registry.map_imperatively` + * :ref:`orm_imperative_dataclasses` + +.. _orm_declarative_mapping: Declarative Mapping =================== -The *Declarative Mapping* is the typical way that -mappings are constructed in modern SQLAlchemy. -Making use of the :ref:`declarative_toplevel` -system, the components of the user-defined class as well as the -:class:`_schema.Table` metadata to which the class is mapped are defined -at once:: +The **Declarative Mapping** is the typical way that +mappings are constructed in modern SQLAlchemy. The most common pattern +is to first construct a base class using the :func:`_orm.declarative_base` +function, which will apply the declarative mapping process to all subclasses +that derive from it. Below features a declarative base which is then +used in a declarative table mapping:: - from sqlalchemy.ext.declarative import declarative_base from sqlalchemy import Column, Integer, String, ForeignKey + from sqlalchemy.orm import declarative_base + # declarative base class Base = declarative_base() + # an example mapping using the base class User(Base): __tablename__ = 'user' @@ -32,64 +65,313 @@ at once:: fullname = Column(String) nickname = Column(String) -Above, a basic single-table mapping with four columns. Additional -attributes, such as relationships to other mapped classes, are also -declared inline within the class definition:: +Above, the :func:`_orm.declarative_base` callable returns a new base class from +which new classes to be mapped may inherit from, as above a new mapped +class ``User`` is constructed. - class User(Base): +The base class refers to a +:class:`_orm.registry` object that maintains a collection of related mapped +classes. The :func:`_orm.declarative_base` function is in fact shorthand +for first creating the registry with the :class:`_orm.registry` +constructor, and then generating a base class using the +:meth:`_orm.registry.generate_base` method:: + + from sqlalchemy.orm import registry + + # equivalent to Base = declarative_base() + + mapper_registry = registry() + Base = mapper_registry.generate_base() + +The :class:`_orm.registry` is used directly in order to access a variety +of mapping styles to suit different use cases: + +* :ref:`orm_declarative_decorator` - declarative mapping using a decorator, + rather than a base class. + +* :ref:`orm_imperative_mapping` - imperative mapping, specifying all mapping + arguments directly rather than scanning a class. + +Documentation for Declarative mapping continues at :ref:`declarative_config_toplevel`. + +.. seealso:: + + :ref:`declarative_config_toplevel` + + +.. _orm_declarative_decorator: + +Declarative Mapping using a Decorator (no declarative base) +------------------------------------------------------------ + +As an alternative to using the "declarative base" class is to apply +declarative mapping to a class explicitly, using either an imperative technique +similar to that of a "classical" mapping, or more succinctly by using +a decorator. The :meth:`_orm.registry.mapped` function is a class decorator +that can be applied to any Python class with no hierarchy in place. The +Python class otherwise is configured in declarative style normally:: + + from sqlalchemy import Column, Integer, String, Text, ForeignKey + + from sqlalchemy.orm import registry + from sqlalchemy.orm import relationship + + mapper_registry = registry() + + @mapper_registry.mapped + class User: __tablename__ = 'user' id = Column(Integer, primary_key=True) name = Column(String) - fullname = Column(String) - nickname = Column(String) - addresses = relationship("Address", backref="user", order_by="Address.id") + addresses = relationship("Address", back_populates="user") - class Address(Base): + @mapper_registry.mapped + class Address: __tablename__ = 'address' id = Column(Integer, primary_key=True) - user_id = Column(ForeignKey('user.id')) + user_id = Column(ForeignKey("user.id")) email_address = Column(String) -The declarative mapping system is introduced in the -:ref:`ormtutorial_toplevel`. For additional details on how this system -works, see :ref:`declarative_toplevel`. + user = relationship("User", back_populates="addresses") + +Above, the same :class:`_orm.registry` that we'd use to generate a declarative +base class via its :meth:`_orm.registry.generate_base` method may also apply +a declarative-style mapping to a class without using a base. When using +the above style, the mapping of a particular class will **only** proceed +if the decorator is applied to that class directly. For inheritance +mappings, the decorator should be applied to each subclass:: + + from sqlalchemy.orm import registry + mapper_registry = registry() + + @mapper_registry.mapped + class Person: + __tablename__ = "person" + + person_id = Column(Integer, primary_key=True) + type = Column(String, nullable=False) + + __mapper_args__ = { + + "polymorphic_on": type, + "polymorphic_identity": "person" + } + + + @mapper_registry.mapped + class Employee(Person): + __tablename__ = "employee" + + person_id = Column(ForeignKey("person.person_id"), primary_key=True) + + __mapper_args__ = { + "polymorphic_identity": "employee" + } + +Both the "declarative table" and "imperative table" styles of declarative +mapping may be used with the above mapping style. + +The decorator form of mapping is particularly useful when combining a +SQLAlchemy declarative mapping with other forms of class declaration, notably +the Python ``dataclasses`` module. See the next section. + +.. _orm_declarative_dataclasses: + +Declarative Mapping with Dataclasses and Attrs +---------------------------------------------- + +The dataclasses_ module, added in Python 3.7, provides a ``@dataclass`` class +decorator to automatically generate boilerplate definitions of ``__init__()``, +``__eq__()``, ``__repr()__``, etc. methods. Another very popular library that does +the same, and much more, is attrs_. Both libraries make use of class +decorators in order to scan a class for attributes that define the class' +behavior, which are then used to generate methods, documentation, and annotations. + +The :meth:`_orm.registry.mapped` class decorator allows the declarative mapping +of a class to occur after the class has been fully constructed, allowing the +class to be processed by other class decorators first. The ``@dataclass`` +and ``@attr.s`` decorators may therefore be applied first before the +ORM mapping process proceeds via the :meth:`_orm.registry.mapped` decorator +or via the :meth:`_orm.registry.map_imperatively` method discussed in a +later section. + +As the attributes set up for ``@dataclass`` or ``@attr.s`` are typically those +which will be matched up to the :class:`_schema.Column` objects that are +mapped, it is usually required that the +:ref:`orm_imperative_table_configuration` style is used in order to configure +the :class:`_schema.Table`, which means that it is defined separately and +associated with the class via the ``__table__``. + + +When attributes are defined using ``dataclasses``, the ``@dataclass`` +decorator consumes them but leaves them in place on the class. +SQLAlchemy's mapping process, when it encounters an attribute that normally +is to be mapped to a :class:`_schema.Column`, checks explicitly if the +attribute is part of a Dataclasses setup, and if so will **replace** +the class-bound dataclass attribute with its usual mapped +properties. The ``__init__`` method created by ``@dataclass`` is left +intact. In contrast, the ``@attr.s`` decorator actually removes its +own class-bound attributes after the decorator runs, so that SQLAlchemy's +mapping process takes over these attributes without any issue. + +.. versionadded:: 1.4 Added support for direct mapping of Python dataclasses, + where the :class:`_orm.Mapper` will now detect attributes that are specific + to the ``@dataclasses`` module and replace them at mapping time, rather + than skipping them as is the default behavior for any class attribute + that's not part of the mapping. + +An example of a mapping using ``@dataclass`` is as follows:: + + from __future__ import annotations + + from dataclasses import dataclass + from dataclasses import field + from typing import List + + from sqlalchemy import Column + from sqlalchemy import ForeignKey + from sqlalchemy import Integer + from sqlalchemy import String + from sqlalchemy import Table + from sqlalchemy.orm import registry + + mapper_registry = registry() + + + @mapper_registry.mapped + @dataclass + class User: + __table__ = Table( + "user", + mapper_registry.metadata, + Column("id", Integer, primary_key=True), + Column("name", String(50)), + Column("fullname", String(50)), + Column("nickname", String(12)), + ) + id: int = field(init=False) + name: str = None + fullname: str = None + nickname: str = None + addresses: List[Address] = field(default_factory=list) + + + @mapper_registry.mapped + @dataclass + class Address: + __table__ = Table( + "address", + mapper_registry.metadata, + Column("id", Integer, primary_key=True), + Column("user_id", Integer, ForeignKey("user.id")), + Column("email_address", String(50)), + ) + id: int = field(init=False) + user_id: int = field(init=False) + email_address: str = None + +In the above example, the ``User.id``, ``Address.id``, and ``Address.user_id`` +attributes are defined as ``field(init=False)``. This means that parameters for +these won't be added to ``__init__()`` methods, but +:class:`.Session` will still be able to set them after getting their values +during flush from autoincrement or other default value generator. To +allow them to be specified in the constructor explicitly, they would instead +be given a default value of ``None``. + +Similarly, a mapping using ``@attr.s``:: + + import attr + + # other imports + + from sqlalchemy.orm import registry + + mapper_registry = registry() + + + @mapper_registry.mapped + @attr.s + class User: + __table__ = Table( + "user", + mapper_registry.metadata, + Column("id", Integer, primary_key=True), + Column("name", String(50)), + Column("fullname", String(50)), + Column("nickname", String(12)), + ) + id = attr.ib() + name = attr.ib() + fullname = attr.ib() + nickname = attr.ib() + addresses = attr.ib() + + # other classes... + +.. sidebar:: Using MyPy with SQLAlchemy models + + If you are using PEP 484 static type checkers for Python, a `MyPy + <http://mypy-lang.org/>`_ plugin is included with `type stubs for + SQLAlchemy <https://github.com/dropbox/sqlalchemy-stubs>`_. The plugin is + tailored towards SQLAlchemy declarative models. SQLAlchemy hopes to include + more comprehensive PEP 484 support in future releases. + + +``@dataclass`` and attrs_ mappings may also be used with classical mappings, i.e. +with the :meth:`_orm.registry.map_imperatively` function. See the section +:ref:`orm_imperative_dataclasses` for a similar example. + +.. _dataclasses: https://docs.python.org/3/library/dataclasses.html +.. _attrs: https://pypi.org/project/attrs/ + +.. _orm_imperative_mapping: .. _classical_mapping: -Classical Mappings -================== +Imperative (a.k.a. Classical) Mappings +====================================== + +An **imperative** or **classical** mapping refers to the configuration of a +mapped class using the :meth:`_orm.registry.map_imperatively` method, +where the target class does not include any declarative class attributes. +The "map imperative" style has historically been achieved using the +:func:`_orm.mapper` function directly, however this function now expects +that a :meth:`_orm.registry` is present. + +.. deprecated:: 1.4 Using the :func:`_orm.mapper` function directly to + achieve a classical mapping directly is deprecated. The + :meth:`_orm.registry.map_imperatively` method retains the identical + functionality while also allowing for string-based resolution of + other mapped classes from within the registry. -A *Classical Mapping* refers to the configuration of a mapped class using the -:func:`.mapper` function, without using the Declarative system. This is -SQLAlchemy's original class mapping API, and is still the base mapping -system provided by the ORM. In "classical" form, the table metadata is created separately with the :class:`_schema.Table` construct, then associated with the ``User`` class via the :func:`.mapper` function:: - from sqlalchemy import Table, MetaData, Column, Integer, String, ForeignKey - from sqlalchemy.orm import mapper + from sqlalchemy import Table, Column, Integer, String, ForeignKey + from sqlalchemy.orm import registry - metadata = MetaData() + mapper_registry = registry() - user = Table('user', metadata, - Column('id', Integer, primary_key=True), - Column('name', String(50)), - Column('fullname', String(50)), - Column('nickname', String(12)) - ) + user_table = Table( + 'user', + mapper_registry.metadata, + Column('id', Integer, primary_key=True), + Column('name', String(50)), + Column('fullname', String(50)), + Column('nickname', String(12)) + ) + + class User: + pass + + mapper_registry.map_imperatively(User, user_table) - class User(object): - def __init__(self, name, fullname, nickname): - self.name = name - self.fullname = fullname - self.nickname = nickname - mapper(User, user) Information about mapped attributes, such as relationships to other classes, are provided via the ``properties`` dictionary. The example below illustrates a second :class:`_schema.Table` @@ -120,39 +402,38 @@ user-defined class, linked together with a :func:`.mapper`. When we talk about "the behavior of :func:`.mapper`", this includes when using the Declarative system as well - it's still used, just behind the scenes. -.. _mapping_dataclasses: - -Mapping dataclasses and attrs ------------------------------ - -The dataclasses_ module, added in Python 3.7, provides a ``dataclass`` class -decorator to automatically generate boilerplate definitions of ``__init__()``, -``__eq__()``, ``__repr()__``, etc. methods. Another very popular library that does -the same, and much more, is attrs_. Classes defined using either of these can -be mapped with the following caveats. -.. versionadded:: 1.4 Added support for direct mapping of Python dataclasses. -The declarative "base" can't be used directly; a mapping function such as -:func:`_declarative.instrument_declarative` or :func:`_orm.mapper` may be -used. -The ``dataclass`` decorator adds class attributes corresponding to simple default values. -This is done mostly as documentation, these attributes are not necessary for the function -of any of the generated methods. Mapping replaces these class attributes with property -descriptors. +.. _orm_imperative_dataclasses: -Mapping of frozen ``dataclass`` and ``attrs`` classes is not possible, because the -machinery used to enforce immutability interferes with loading. +Imperative Mapping with Dataclasses and Attrs +--------------------------------------------- -Example using classical mapping:: +As described in the section :ref:`orm_declarative_dataclasses`, the +``@dataclass`` decorator and the attrs_ library both work as class +decorators that are applied to a class first, before it is passed to +SQLAlchemy for mapping. Just like we can use the +:meth:`_orm.registry.mapped` decorator in order to apply declarative-style +mapping to the class, we can also pass it to the :meth:`_orm.registry.map_imperatively` +method so that we may pass all :class:`_schema.Table` and :class:`_orm.Mapper` +configuration imperatively to the function rather than having them defined +on the class itself as declarative class variables:: from __future__ import annotations - from dataclasses import dataclass, field + + from dataclasses import dataclass + from dataclasses import field from typing import List - from sqlalchemy import Column, ForeignKey, Integer, MetaData, String, Table - from sqlalchemy.orm import mapper, relationship + from sqlalchemy import Column + from sqlalchemy import ForeignKey + from sqlalchemy import Integer + from sqlalchemy import MetaData + from sqlalchemy import String + from sqlalchemy import Table + from sqlalchemy.orm import mapper + from sqlalchemy.orm import relationship @dataclass class User: @@ -193,20 +474,221 @@ Example using classical mapping:: mapper(Address, address) -Note that ``User.id``, ``Address.id``, and ``Address.user_id`` are defined as ``field(init=False)``. -This means that parameters for these won't be added to ``__init__()`` methods, but -:class:`.Session` will still be able to set them after getting their values during flush -from autoincrement or other default value generator. You can also give them a -``None`` default value instead if you want to be able to specify their values in the constructor. +.. _orm_mapper_configuration_overview: -.. _dataclasses: https://docs.python.org/3/library/dataclasses.html -.. _attrs: https://www.attrs.org/en/stable/ +Mapper Configuration Overview +============================= + +With all mapping forms, the mapping of the class can be +configured in many ways by passing construction arguments that become +part of the :class:`_orm.Mapper` object. The function which ultimately +receives these arguments is the :func:`_orm.mapper` function, which are delivered +to it originating from one of the front-facing mapping functions defined +on the :class:`_orm.registry` object. + +There are four general classes of configuration information that the +:func:`_orm.mapper` function looks for: + +The class to be mapped +----------------------- + +This is a class that we construct in our application. +There are generally no restrictions on the structure of this class. [1]_ +When a Python class is mapped, there can only be **one** :class:`_orm.Mapper` +object for the class. [2]_ + +When mapping with the :ref:`declarative <orm_declarative_mapping>` mapping +style, the class to be mapped is either a subclass of the declarative base class, +or is handled by a decorator or function such as :meth:`_orm.registry.mapped`. + +When mapping with the :ref:`imperative <orm_imperative_mapping>` style, the +class is passed directly as the +:paramref:`_orm.registry.map_imperatively.class_` argument. + +the table, or other from clause object +-------------------------------------- + +In the vast majority of common cases this is an instance of +:class:`_schema.Table`. For more advanced use cases, it may also refer +to any kind of :class:`_sql.FromClause` object, the most common +alternative objects being the :class:`_sql.Subquery` and :class:`_sql.Join` +object. + +When mapping with the :ref:`declarative <orm_declarative_mapping>` mapping +style, the subject table is either generated by the declarative system based +on the ``__tablename__`` attribute and the :class:`_schema.Column` objects +presented, or it is established via the ``__table__`` attribute. These +two styles of configuration are presented at +:ref:`orm_declarative_table` and :ref:`orm_imperative_table_configuration`. + +When mapping with the :ref:`imperative <orm_imperative_mapping>` style, the +subject table is passed positionally as the +:paramref:`_orm.registry.map_imperatively.local_table` argument. + +In contrast to the "one mapper per class" requirement of a mapped class, +the :class:`_schema.Table` or other :class:`_sql.FromClause` object that +is the subject of the mapping may be associated with any number of mappings. +The :class:`_orm.Mapper` applies modifications directly to the user-defined +class, but does not modify the given :class:`_schema.Table` or other +:class:`_sql.FromClause` in any way. + +.. _orm_mapping_properties: + +The properties dictionary +-------------------------- + +This is a dictionary of all of the attributes +that will be associated with the mapped class. By default, the +:class:`_orm.Mapper` generates entries for this dictionary derived from the +given :class:`_schema.Table`, in the form of :class:`_orm.ColumnProperty` +objects which each refer to an individual :class:`_schema.Column` of the +mapped table. The properties dictionary will also contain all the other +kinds of :class:`_orm.MapperProperty` objects to be configured, most +commonly instances generated by the :func:`_orm.relationship` construct. + +When mapping with the :ref:`declarative <orm_declarative_mapping>` mapping +style, the properties dictionary is generated by the declarative system +by scanning the class to be mapped for appropriate attributes. See +the section :ref:`orm_declarative_properties` for notes on this process. + +When mapping with the :ref:`imperative <orm_imperative_mapping>` style, the +properties dictionary is passed directly as the ``properties`` argument +to :meth:`_orm.registry.map_imperatively`, which will pass it along to the +:paramref:`_orm.mapper.properties` parameter. + +Other mapper configuration parameters +--------------------------------------- + +These flags are documented at :func:`_orm.mapper`. + +When mapping with the :ref:`declarative <orm_declarative_mapping>` mapping +style, additional mapper configuration arguments are configured via the +``__mapper_args__`` class attribute, documented at +:ref:`orm_declarative_mapper_options` + +When mapping with the :ref:`imperative <orm_imperative_mapping>` style, +keyword arguments are passed to the to :meth:`_orm.registry.map_imperatively` +method which passes them along to the :func:`_orm.mapper` function. + + +.. [1] When running under Python 2, a Python 2 "old style" class is the only + kind of class that isn't compatible. When running code on Python 2, + all classes must extend from the Python ``object`` class. Under + Python 3 this is always the case. + +.. [2] There is a legacy feature known as a "non primary mapper", where + additional :class:`_orm.Mapper` objects may be associated with a class + that's already mapped, however they don't apply instrumentation + to the class. This feature is deprecated as of SQLAlchemy 1.3. + + +Mapped Class Behavior +===================== + +Across all styles of mapping using the :class:`_orm.registry` object, +the following behaviors are common: + +Default Constructor +------------------- + +The :class:`_orm.registry` applies a default constructor, i.e. ``__init__`` +method, to all mapped classes that don't explicitly have their own +``__init__`` method. The behavior of this method is such that it provides +a convenient keyword constructor that will accept as keywords the attributes +that are named. E.g.:: + + from sqlalchemy.orm import declarative_base + + Base = declarative_base() + + class User(Base): + __tablename__ = 'user' + + id = Column(...) + name = Column(...) + fullname = Column(...) + +An object of type ``User`` above will have a constructor which allows +``User`` objects to be created as:: + + u1 = User(name='some name', fullname='some fullname') + +The above constructor may be customized by passing a Python callable to +the :paramref:`_orm.registry.constructor` parameter which provides the +desired default ``__init__()`` behavior. + +The constructor also applies to imperative mappings:: + + from sqlalchemy.orm import registry + + mapper_registry = registry() + + user_table = Table( + 'user', + mapper_registry.metadata, + Column('id', Integer, primary_key=True), + Column('name', String(50)) + ) + + class User: + pass + + mapper_registry.map_imperatively(User, user_table) + +The above class, mapped imperatively as described at :ref:`classical_mapping`, +will also feature the default constructor associated with the :class:`_orm.registry`. + +.. versionadded:: 1.4 classical mappings now support a standard configuration-level + constructor when they are mapped via the :meth:`_orm.registry.map_imperatively` + method. + +Runtime Introspection of Mapped classes and Mappers +--------------------------------------------------- + +A class that is mapped using :class:`_orm.registry` will also feature a few +attributes that are common to all mappings: + +* The ``__mapper__`` attribute will refer to the :class:`_orm.Mapper` that + is associated with the class:: + + mapper = User.__mapper__ + + This :class:`_orm.Mapper` is also what's returned when using the + :func:`_sa.inspect` function against the mapped class:: + + from sqlalchemy import inspect + + mapper = inspect(User) + + .. + +* The ``__table__`` attribute will refer to the :class:`_schema.Table`, or + more generically to the :class:`_schema.FromClause` object, to which the + class is mapped:: + + table = User.__table__ + + This :class:`_schema.FromClause` is also what's returned when using the + :attr:`_orm.Mapper.local_table` attribute of the :class:`_orm.Mapper`:: + + table = inspect(User).local_table + + For a single-table inheritance mapping, where the class is a subclass that + does not have a table of its own, the :attr:`_orm.Mapper.local_table` attribute as well + as the ``.__table__`` attribute will be ``None``. To retrieve the + "selectable" that is actually selected from during a query for this class, + this is available via the :attr:`_orm.Mapper.selectable` attribute:: + + table = inspect(User).selectable + + .. -Runtime Introspection of Mappings, Objects -========================================== +Mapper Inspection Features +-------------------------- -The :class:`_orm.Mapper` object is available from any mapped class, regardless -of method, using the :ref:`core_inspection_toplevel` system. Using the +As illustrated in the previous section, the :class:`_orm.Mapper` object is +available from any mapped class, regardless of method, using the +:ref:`core_inspection_toplevel` system. Using the :func:`_sa.inspect` function, one can acquire the :class:`_orm.Mapper` from a mapped class:: diff --git a/doc/build/orm/nonstandard_mappings.rst b/doc/build/orm/nonstandard_mappings.rst index 94892ffbe..16473e26c 100644 --- a/doc/build/orm/nonstandard_mappings.rst +++ b/doc/build/orm/nonstandard_mappings.rst @@ -2,6 +2,8 @@ Non-Traditional Mappings ======================== +.. _orm_mapping_joins: + .. _maptojoin: Mapping a Class against Multiple Tables @@ -114,26 +116,27 @@ may be used:: that the LEFT OUTER JOIN from "p" to "q" does not have an entry for the "q" side. +.. _orm_mapping_arbitrary_subqueries: -Mapping a Class against Arbitrary Selects -========================================= +Mapping a Class against Arbitrary Subqueries +============================================ -Similar to mapping against a join, a plain :func:`_expression.select` object can be used with a -mapper as well. The example fragment below illustrates mapping a class -called ``Customer`` to a :func:`_expression.select` which includes a join to a -subquery:: +Similar to mapping against a join, a plain :func:`_expression.select` object +can be used with a mapper as well. The example fragment below illustrates +mapping a class called ``Customer`` to a :func:`_expression.select` which +includes a join to a subquery:: from sqlalchemy import select, func subq = select( - func.count(orders.c.id).label('order_count'), - func.max(orders.c.price).label('highest_order'), - orders.c.customer_id - ).group_by(orders.c.customer_id).alias() - - customer_select = select(customers, subq).select_from( - join(customers, subq, customers.c.id == subq.c.customer_id) - ).alias() + func.count(orders.c.id).label('order_count'), + func.max(orders.c.price).label('highest_order'), + orders.c.customer_id + ).group_by(orders.c.customer_id).subquery() + + customer_select = select(customers, subq).join_from( + customers, subq, customers.c.id == subq.c.customer_id + ).subquery() class Customer(Base): __table__ = customer_select diff --git a/doc/build/orm/relationships.rst b/doc/build/orm/relationships.rst index 37f59d345..8a4fe36a1 100644 --- a/doc/build/orm/relationships.rst +++ b/doc/build/orm/relationships.rst @@ -10,7 +10,7 @@ of its usage. For an introduction to relationships, start with the :ref:`ormtutorial_toplevel` and head into :ref:`orm_tutorial_relationship`. .. toctree:: - :maxdepth: 2 + :maxdepth: 3 basic_relationships self_referential diff --git a/doc/build/orm/scalar_mapping.rst b/doc/build/orm/scalar_mapping.rst index e8829af49..001745e9e 100644 --- a/doc/build/orm/scalar_mapping.rst +++ b/doc/build/orm/scalar_mapping.rst @@ -8,7 +8,7 @@ The following sections discuss how table columns and SQL expressions are mapped to individual object attributes. .. toctree:: - :maxdepth: 2 + :maxdepth: 3 mapping_columns mapped_sql_expr diff --git a/doc/build/orm/session.rst b/doc/build/orm/session.rst index db52fd3d1..8e14942c4 100644 --- a/doc/build/orm/session.rst +++ b/doc/build/orm/session.rst @@ -13,7 +13,7 @@ persistence operations is the :class:`.Session`. .. toctree:: - :maxdepth: 2 + :maxdepth: 3 session_basics session_state_management diff --git a/doc/build/orm/tutorial.rst b/doc/build/orm/tutorial.rst index 8c148ac32..c04caf9e6 100644 --- a/doc/build/orm/tutorial.rst +++ b/doc/build/orm/tutorial.rst @@ -104,7 +104,7 @@ application will usually have just one instance of this base in a commonly imported module. We create the base class using the :func:`.declarative_base` function, as follows:: - >>> from sqlalchemy.ext.declarative import declarative_base + >>> from sqlalchemy.orm import declarative_base >>> Base = declarative_base() |