.. _orm_declarative_table_config_toplevel: ============================================= Table Configuration with Declarative ============================================= As introduced at :ref:`orm_declarative_mapping`, the Declarative style includes the ability to generate a mapped :class:`_schema.Table` object at the same time, or to accommodate a :class:`_schema.Table` or other :class:`_sql.FromClause` object directly. The following examples assume a declarative base class as:: from sqlalchemy.orm import DeclarativeBase class Base(DeclarativeBase): pass All of the examples that follow illustrate a class inheriting from the above ``Base``. The decorator style introduced at :ref:`orm_declarative_decorator` is fully supported with all the following examples as well, as are legacy forms of Declarative Base including base classes generated by :func:`_orm.declarative_base`. .. _orm_declarative_table: Declarative Table with ``mapped_column()`` ------------------------------------------ When using Declarative, the body of the class to be mapped in most cases includes an attribute ``__tablename__`` that indicates the string name of a :class:`_schema.Table` that should be generated along with the mapping. The :func:`_orm.mapped_column` construct, which features additional ORM-specific configuration capabilities not present in the plain :class:`_schema.Column` class, is then used within the class body to indicate columns in the table. The example below illustrates the most basic use of this construct within a Declarative mapping:: from sqlalchemy import Integer, String from sqlalchemy.orm import DeclarativeBase from sqlalchemy.orm import mapped_column class Base(DeclarativeBase): pass class User(Base): __tablename__ = "user" id = mapped_column(Integer, primary_key=True) name = mapped_column(String(50), nullable=False) fullname = mapped_column(String) nickname = mapped_column(String(30)) Above, :func:`_orm.mapped_column` constructs are placed inline within the class definition as class level attributes. At the point at which the class is declared, the Declarative mapping process will generate a new :class:`_schema.Table` object against the :class:`_schema.MetaData` collection associated with the Declarative ``Base``; each instance of :func:`_orm.mapped_column` will then be used to generate a :class:`_schema.Column` object during this process, which will become part of the :attr:`.schema.Table.columns` collection of this :class:`_schema.Table` object. In the above example, Declarative will build a :class:`_schema.Table` construct that is equivalent to the following:: # equivalent Table object produced user_table = Table( "user", Base.metadata, Column("id", Integer, primary_key=True), Column("name", String(50)), Column("fullname", String()), Column("nickname", String(30)), ) When the ``User`` class above is mapped, this :class:`_schema.Table` object can be accessed directly via the ``__table__`` attribute; this is described further at :ref:`orm_declarative_metadata`. .. sidebar:: ``mapped_column()`` supersedes the use of ``Column()`` Users of 1.x SQLAlchemy will note the use of the :func:`_orm.mapped_column` construct, which is new as of the SQLAlchemy 2.0 series. This ORM-specific construct is intended first and foremost to be a drop-in replacement for the use of :class:`_schema.Column` within Declarative mappings only, adding new ORM-specific convenience features such as the ability to establish :paramref:`_orm.mapped_column.deferred` within the construct, and most importantly to indicate to typing tools such as Mypy_ and Pylance_ an accurate representation of how the attribute will behave at runtime at both the class level as well as the instance level. As will be seen in the following sections, it's also at the forefront of a new annotation-driven configuration style introduced in SQLAlchemy 2.0. Users of legacy code should be aware that the :class:`_schema.Column` form will always work in Declarative in the same way it always has. The different forms of attribute mapping may also be mixed within a single mapping on an attribute by attribute basis, so migration to the new form can be at any pace. See the section :ref:`whatsnew_20_orm_declarative_typing` for a step by step guide to migrating a Declarative model to the new form. The :func:`_orm.mapped_column` construct accepts all arguments that are accepted by the :class:`_schema.Column` construct, as well as additional ORM-specific arguments. The :paramref:`_orm.mapped_column.__name` field, indicating the name of the database column, is typically omitted, as the Declarative process will make use of the attribute name given to the construct and assign this as the name of the column (in the above example, this refers to the names ``id``, ``name``, ``fullname``, ``nickname``). Assigning an alternate :paramref:`_orm.mapped_column.__name` is valid as well, where the resulting :class:`_schema.Column` will use the given name in SQL and DDL statements, while the ``User`` mapped class will continue to allow access to the attribute using the attribute name given, independent of the name given to the column itself (more on this at :ref:`mapper_column_distinct_names`). .. tip:: The :func:`_orm.mapped_column` construct is **only valid within a Declarative class mapping**. When constructing a :class:`_schema.Table` object using Core as well as when using :ref:`imperative table ` configuration, the :class:`_schema.Column` construct is still required in order to indicate the presence of a database column. .. seealso:: :ref:`mapping_columns_toplevel` - contains additional notes on affecting how :class:`_orm.Mapper` interprets incoming :class:`.Column` objects. .. _orm_declarative_mapped_column: Using Annotated Declarative Table (Type Annotated Forms for ``mapped_column()``) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The :func:`_orm.mapped_column` construct is capable of deriving its column-configuration information from :pep:`484` type annotations associated with the attribute as declared in the Declarative mapped class. These type annotations, if used, **must** be present within a special SQLAlchemy type called :class:`_orm.Mapped`, which is a generic_ type that then indicates a specific Python type within it. Below illustrates the mapping from the previous section, adding the use of :class:`_orm.Mapped`:: from typing import Optional from sqlalchemy import String from sqlalchemy.orm import DeclarativeBase from sqlalchemy.orm import Mapped from sqlalchemy.orm import mapped_column class Base(DeclarativeBase): pass class User(Base): __tablename__ = "user" id: Mapped[int] = mapped_column(primary_key=True) name: Mapped[str] = mapped_column(String(50)) fullname: Mapped[Optional[str]] nickname: Mapped[Optional[str]] = mapped_column(String(30)) Above, when Declarative processes each class attribute, each :func:`_orm.mapped_column` will derive additional arguments from the corresponding :class:`_orm.Mapped` type annotation on the left side, if present. Additionally, Declarative will generate an empty :func:`_orm.mapped_column` directive implicitly, whenever a :class:`_orm.Mapped` type annotation is encountered that does not have a value assigned to the attribute (this form is inspired by the similar style used in Python dataclasses_); this :func:`_orm.mapped_column` construct proceeds to derive its configuration from the :class:`_orm.Mapped` annotation present. .. _orm_declarative_mapped_column_nullability: ``mapped_column()`` derives the datatype and nullability from the ``Mapped`` annotation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The two qualities that :func:`_orm.mapped_column` derives from the :class:`_orm.Mapped` annotation are: * **datatype** - the Python type given inside :class:`_orm.Mapped`, as contained within the ``typing.Optional`` construct if present, is associated with a :class:`_sqltypes.TypeEngine` subclass such as :class:`.Integer`, :class:`.String`, :class:`.DateTime`, or :class:`.Uuid`, to name a few common types. The datatype is determined based on a dictionary of Python type to SQLAlchemy datatype. This dictionary is completely customizable, as detailed in the next section :ref:`orm_declarative_mapped_column_type_map`. The default type map is implemented as in the code example below:: from typing import Any from typing import Dict from typing import Type import datetime import decimal import uuid from sqlalchemy import types # default type mapping, deriving the type for mapped_column() # from a Mapped[] annotation type_map: Dict[Type[Any], TypeEngine[Any]] = { bool: types.Boolean(), bytes: types.LargeBinary(), datetime.date: types.Date(), datetime.datetime: types.DateTime(), datetime.time: types.Time(), datetime.timedelta: types.Interval(), decimal.Decimal: types.Numeric(), float: types.Float(), int: types.Integer(), str: types.String(), uuid.UUID: types.Uuid(), } If the :func:`_orm.mapped_column` construct indicates an explicit type as passed to the :paramref:`_orm.mapped_column.__type` argument, then the given Python type is disregarded. * **nullability** - The :func:`_orm.mapped_column` construct will indicate its :class:`_schema.Column` as ``NULL`` or ``NOT NULL`` first and foremost by the presence of the :paramref:`_orm.mapped_column.nullable` parameter, passed either as ``True`` or ``False``. Additionally , if the :paramref:`_orm.mapped_column.primary_key` parameter is present and set to ``True``, that will also imply that the column should be ``NOT NULL``. In the absence of **both** of these parameters, the presence of ``typing.Optional[]`` within the :class:`_orm.Mapped` type annotation will be used to determine nullability, where ``typing.Optional[]`` means ``NULL``, and the absense of ``typing.Optional[]`` means ``NOT NULL``. If there is no ``Mapped[]`` annotation present at all, and there is no :paramref:`_orm.mapped_column.nullable` or :paramref:`_orm.mapped_column.primary_key` parameter, then SQLAlchemy's usual default for :class:`_schema.Column` of ``NULL`` is used. In the example below, the ``id`` and ``data`` columns will be ``NOT NULL``, and the ``additional_info`` column will be ``NULL``:: from typing import Optional from sqlalchemy.orm import DeclarativeBase from sqlalchemy.orm import Mapped from sqlalchemy.orm import mapped_column class Base(DeclarativeBase): pass class SomeClass(Base): __tablename__ = "some_table" # primary_key=True, therefore will be NOT NULL id: Mapped[int] = mapped_column(primary_key=True) # not Optional[], therefore will be NOT NULL data: Mapped[str] # Optional[], therefore will be NULL additional_info: Mapped[Optional[str]] It is also perfectly valid to have a :func:`_orm.mapped_column` whose nullability is **different** from what would be implied by the annotation. For example, an ORM mapped attribute may be annotated as allowing ``None`` within Python code that works with the object as it is first being created and populated, however the value will ultimately be written to a database column that is ``NOT NULL``. The :paramref:`_orm.mapped_column.nullable` parameter, when present, will always take precedence:: class SomeClass(Base): # ... # will be String() NOT NULL, but can be None in Python data: Mapped[Optional[str]] = mapped_column(nullable=False) Similarly, a non-None attribute that's written to a database column that for whatever reason needs to be NULL at the schema level, :paramref:`_orm.mapped_column.nullable` may be set to ``True``:: class SomeClass(Base): # ... # will be String() NULL, but type checker will not expect # the attribute to be None data: Mapped[str] = mapped_column(nullable=True) .. _orm_declarative_mapped_column_type_map: Customizing the Type Map ~~~~~~~~~~~~~~~~~~~~~~~~ The mapping of Python types to SQLAlchemy :class:`_types.TypeEngine` types described in the previous section defaults to a hardcoded dictionary present in the ``sqlalchemy.sql.sqltypes`` module. However, the :class:`_orm.registry` object that coordinates the Declarative mapping process will first consult a local, user defined dictionary of types which may be passed as the :paramref:`_orm.registry.type_annotation_map` parameter when constructing the :class:`_orm.registry`, which may be associated with the :class:`_orm.DeclarativeBase` superclass when first used. As an example, if we wish to make use of the :class:`_sqltypes.BIGINT` datatype for ``int``, the :class:`_sqltypes.TIMESTAMP` datatype with ``timezone=True`` for ``datetime.datetime``, and then only on Microsoft SQL Server we'd like to use :class:`_sqltypes.NVARCHAR` datatype when Python ``str`` is used, the registry and Declarative base could be configured as:: import datetime from sqlalchemy import BIGINT, Integer, NVARCHAR, String, TIMESTAMP from sqlalchemy.orm import DeclarativeBase from sqlalchemy.orm import Mapped, mapped_column, registry class Base(DeclarativeBase): type_annotation_map = { int: BIGINT, datetime.datetime: TIMESTAMP(timezone=True), str: String().with_variant(NVARCHAR, "mssql"), } class SomeClass(Base): __tablename__ = "some_table" id: Mapped[int] = mapped_column(primary_key=True) date: Mapped[datetime.datetime] status: Mapped[str] Below illustrates the CREATE TABLE statement generated for the above mapping, first on the Microsoft SQL Server backend, illustrating the ``NVARCHAR`` datatype: .. sourcecode:: pycon+sql >>> from sqlalchemy.schema import CreateTable >>> from sqlalchemy.dialects import mssql, postgresql >>> print(CreateTable(SomeClass.__table__).compile(dialect=mssql.dialect())) {printsql}CREATE TABLE some_table ( id BIGINT NOT NULL IDENTITY, date TIMESTAMP NOT NULL, status NVARCHAR(max) NOT NULL, PRIMARY KEY (id) ) Then on the PostgreSQL backend, illustrating ``TIMESTAMP WITH TIME ZONE``: .. sourcecode:: pycon+sql >>> print(CreateTable(SomeClass.__table__).compile(dialect=postgresql.dialect())) {printsql}CREATE TABLE some_table ( id BIGSERIAL NOT NULL, date TIMESTAMP WITH TIME ZONE NOT NULL, status VARCHAR NOT NULL, PRIMARY KEY (id) ) By making use of methods such as :meth:`.TypeEngine.with_variant`, we're able to build up a type map that's customized to what we need for different backends, while still being able to use succinct annotation-only :func:`_orm.mapped_column` configurations. There are two more levels of Python-type configurability available beyond this, described in the next two sections. .. _orm_declarative_mapped_column_type_map_pep593: Mapping Multiple Type Configurations to Python Types ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ As individual Python types may be associated with :class:`_types.TypeEngine` configurations of any variety by using the :paramref:`_orm.registry.type_annotation_map` parameter, an additional capability is the ability to associate a single Python type with different variants of a SQL type based on additional type qualifiers. One typical example of this is mapping the Python ``str`` datatype to ``VARCHAR`` SQL types of different lengths. Another is mapping different varieties of ``decimal.Decimal`` to differently sized ``NUMERIC`` columns. Python's typing system provides a great way to add additional metadata to a Python type which is by using the :pep:`593` ``Annotated`` generic type, which allows additional information to be bundled along with a Python type. The :func:`_orm.mapped_column` construct will correctly interpret an ``Annotated`` object by identity when resolving it in the :paramref:`_orm.registry.type_annotation_map`, as in the example below where we declare two variants of :class:`.String` and :class:`.Numeric`:: from decimal import Decimal from typing_extensions import Annotated from sqlalchemy import Numeric from sqlalchemy import String from sqlalchemy.orm import DeclarativeBase from sqlalchemy.orm import Mapped from sqlalchemy.orm import mapped_column from sqlalchemy.orm import registry str_30 = Annotated[str, 30] str_50 = Annotated[str, 50] num_12_4 = Annotated[Decimal, 12] num_6_2 = Annotated[Decimal, 6] class Base(DeclarativeBase): registry = registry( type_annotation_map={ str_30: String(30), str_50: String(50), num_12_4: Numeric(12, 4), num_6_2: Numeric(6, 2), } ) The Python type passed to the ``Annotated`` container, in the above example the ``str`` and ``Decimal`` types, is important only for the benefit of typing tools; as far as the :func:`_orm.mapped_column` construct is concerned, it will only need perform a lookup of each type object in the :paramref:`_orm.registry.type_annotation_map` dictionary without actually looking inside of the ``Annotated`` object, at least in this particular context. Similarly, the arguments passed to ``Annotated`` beyond the underlying Python type itself are also not important, it's only that at least one argument must be present for the ``Annotated`` construct to be valid. We can then use these augmented types directly in our mapping where they will be matched to the more specific type constructions, as in the following example:: class SomeClass(Base): __tablename__ = "some_table" short_name: Mapped[str_30] = mapped_column(primary_key=True) long_name: Mapped[str_50] num_value: Mapped[num_12_4] short_num_value: Mapped[num_6_2] a CREATE TABLE for the above mapping will illustrate the different variants of ``VARCHAR`` and ``NUMERIC`` we've configured, and looks like: .. sourcecode:: pycon+sql >>> from sqlalchemy.schema import CreateTable >>> print(CreateTable(SomeClass.__table__)) {printsql}CREATE TABLE some_table ( short_name VARCHAR(30) NOT NULL, long_name VARCHAR(50) NOT NULL, num_value NUMERIC(12, 4) NOT NULL, short_num_value NUMERIC(6, 2) NOT NULL, PRIMARY KEY (short_name) ) While variety in linking ``Annotated`` types to different SQL types grants us a wide degree of flexibility, the next section illustrates a second way in which ``Annotated`` may be used with Declarative that is even more open ended. .. _orm_declarative_mapped_column_pep593: Mapping Whole Column Declarations to Python Types ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The previous section illustrated using :pep:`593` ``Annotated`` type instances as keys within the :paramref:`_orm.registry.type_annotation_map` dictionary. In this form, the :func:`_orm.mapped_column` construct does not actually look inside the ``Annotated`` object itself, it's instead used only as a dictionary key. However, Declarative also has the ability to extract an entire pre-established :func:`_orm.mapped_column` construct from an ``Annotated`` object directly. Using this form, we can define not only different varieties of SQL datatypes linked to Python types without using the :paramref:`_orm.registry.type_annotation_map` dictionary, we can also set up any number of arguments such as nullability, column defaults, and constraints in a reusable fashion. A set of ORM models will usually have some kind of primary key style that is common to all mapped classes. There also may be common column configurations such as timestamps with defaults and other fields of pre-established sizes and configurations. We can compose these configurations into :func:`_orm.mapped_column` instances that we then bundle directly into instances of ``Annotated``, which are then re-used in any number of class declarations. Declarative will unpack an ``Annotated`` object when provided in this manner, skipping over any other directives that don't apply to SQLAlchemy and searching only for SQLAlchemy ORM constructs. The example below illustrates a variety of pre-configured field types used in this way, where we define ``intpk`` that represents an :class:`.Integer` primary key column, ``timestamp`` that represents a :class:`.DateTime` type which will use ``CURRENT_TIMESTAMP`` as a DDL level column default, and ``required_name`` which is a :class:`.String` of length 30 that's ``NOT NULL``:: import datetime from typing_extensions import Annotated from sqlalchemy import func from sqlalchemy import String from sqlalchemy.orm import mapped_column intpk = Annotated[int, mapped_column(primary_key=True)] timestamp = Annotated[ datetime.datetime, mapped_column(nullable=False, server_default=func.CURRENT_TIMESTAMP()), ] required_name = Annotated[str, mapped_column(String(30), nullable=False)] The above ``Annotated`` objects can then be used directly within :class:`_orm.Mapped`, where the pre-configured :func:`_orm.mapped_column` constructs will be extracted and copied to a new instance that will be specific to each attribute:: class Base(DeclarativeBase): pass class SomeClass(Base): __tablename__ = "some_table" id: Mapped[intpk] name: Mapped[required_name] created_at: Mapped[timestamp] ``CREATE TABLE`` for our above mapping looks like: .. sourcecode:: pycon+sql >>> from sqlalchemy.schema import CreateTable >>> print(CreateTable(SomeClass.__table__)) {printsql}CREATE TABLE some_table ( id INTEGER NOT NULL, name VARCHAR(30) NOT NULL, created_at DATETIME DEFAULT CURRENT_TIMESTAMP NOT NULL, PRIMARY KEY (id) ) When using ``Annotated`` types in this way, the configuration of the type may also be affected on a per-attribute basis. For the types in the above example that feature explcit use of :paramref:`_orm.mapped_column.nullable`, we can apply the ``Optional[]`` generic modifier to any of our types so that the field is optional or not at the Python level, which will be independent of the ``NULL`` / ``NOT NULL`` setting that takes place in the database:: from typing_extensions import Annotated import datetime from typing import Optional from sqlalchemy.orm import DeclarativeBase timestamp = Annotated[ datetime.datetime, mapped_column(nullable=False), ] class Base(DeclarativeBase): pass class SomeClass(Base): # ... # pep-484 type will be Optional, but column will be # NOT NULL created_at: Mapped[Optional[timestamp]] The :func:`_orm.mapped_column` construct is also reconciled with an explicitly passed :func:`_orm.mapped_column` construct, whose arguments will take precedence over those of the ``Annotated`` construct. Below we add a :class:`.ForeignKey` constraint to our integer primary key and also use an alternate server default for the ``created_at`` column:: import datetime from typing_extensions import Annotated from sqlalchemy import ForeignKey from sqlalchemy import func from sqlalchemy.orm import DeclarativeBase from sqlalchemy.orm import Mapped from sqlalchemy.orm import mapped_column from sqlalchemy.schema import CreateTable intpk = Annotated[int, mapped_column(primary_key=True)] timestamp = Annotated[ datetime.datetime, mapped_column(nullable=False, server_default=func.CURRENT_TIMESTAMP()), ] class Base(DeclarativeBase): pass class Parent(Base): __tablename__ = "parent" id: Mapped[intpk] class SomeClass(Base): __tablename__ = "some_table" # add ForeignKey to mapped_column(Integer, primary_key=True) id: Mapped[intpk] = mapped_column(ForeignKey("parent.id")) # change server default from CURRENT_TIMESTAMP to UTC_TIMESTAMP created_at: Mapped[timestamp] = mapped_column(server_default=func.UTC_TIMESTAMP()) The CREATE TABLE statement illustrates these per-attribute settings, adding a ``FOREIGN KEY`` constraint as well as substituting ``UTC_TIMESTAMP`` for ``CURRENT_TIMESTAMP``: .. sourcecode:: pycon+sql >>> from sqlalchemy.schema import CreateTable >>> print(CreateTable(SomeClass.__table__)) {printsql}CREATE TABLE some_table ( id INTEGER NOT NULL, created_at DATETIME DEFAULT UTC_TIMESTAMP() NOT NULL, PRIMARY KEY (id), FOREIGN KEY(id) REFERENCES parent (id) ) .. note:: The feature of :func:`_orm.mapped_column` just described, where a fully constructed set of column arguments may be indicated using :pep:`593` ``Annotated`` objects that contain a "template" :func:`_orm.mapped_column` object to be copied into the attribute, is currently not implemented for other ORM constructs such as :func:`_orm.relationship` and :func:`_orm.composite`. While this functionality is in theory possible, for the moment attempting to use ``Annotated`` to indicate further arguments for :func:`_orm.relationship` and similar will raise a ``NotImplementedError`` exception at runtime, but may be implemented in future releases. .. _orm_declarative_mapped_column_enums: Using Python ``Enum`` or pep-586 ``Literal`` types in the type map ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. versionadded:: 2.0.0b4 - Added ``Enum`` support .. versionadded:: 2.0.1 - Added ``Literal`` support User-defined Python types which derive from the Python built-in ``enum.Enum`` as well as the ``typing.Literal`` class are automatically linked to the SQLAlchemy :class:`.Enum` datatype when used in an ORM declarative mapping. The example below uses a custom ``enum.Enum`` within the ``Mapped[]`` constructor:: import enum from sqlalchemy.orm import DeclarativeBase from sqlalchemy.orm import Mapped from sqlalchemy.orm import mapped_column class Base(DeclarativeBase): pass class Status(enum.Enum): PENDING = "pending" RECEIVED = "received" COMPLETED = "completed" class SomeClass(Base): __tablename__ = "some_table" id: Mapped[int] = mapped_column(primary_key=True) status: Mapped[Status] In the above example, the mapped attribute ``SomeClass.status`` will be linked to a :class:`.Column` with the datatype of ``Enum(Status)``. We can see this for example in the CREATE TABLE output for the PostgreSQL database: .. sourcecode:: sql CREATE TYPE status AS ENUM ('PENDING', 'RECEIVED', 'COMPLETED') CREATE TABLE some_table ( id SERIAL NOT NULL, status status NOT NULL, PRIMARY KEY (id) ) In a similar way, ``typing.Literal`` may be used instead, using a ``typing.Literal`` that consists of all strings:: from typing import Literal from sqlalchemy.orm import DeclarativeBase from sqlalchemy.orm import Mapped from sqlalchemy.orm import mapped_column class Base(DeclarativeBase): pass Status = Literal["pending", "received", "completed"] class SomeClass(Base): __tablename__ = "some_table" id: Mapped[int] = mapped_column(primary_key=True) status: Mapped[Status] The entries used in :paramref:`_orm.registry.type_annotation_map` link the base ``enum.Enum`` Python type as well as the ``typing.Literal`` type to the SQLAlchemy :class:`.Enum` SQL type, using a special form which indicates to the :class:`.Enum` datatype that it should automatically configure itself against an arbitrary enumerated type. This configuration, which is implicit by default, would be indicated explicitly as:: import enum import typing import sqlalchemy from sqlalchemy.orm import DeclarativeBase class Base(DeclarativeBase): type_annotation_map = { enum.Enum: sqlalchemy.Enum(enum.Enum), typing.Literal: sqlalchemy.Enum(enum.Enum), } The resolution logic within Declarative is able to resolve subclasses of ``enum.Enum`` as well as instances of ``typing.Literal`` to match the ``enum.Enum`` or ``typing.Literal`` entry in the :paramref:`_orm.registry.type_annotation_map` dictionary. The :class:`.Enum` SQL type then knows how to produce a configured version of itself with the appropriate settings, including default string length. If a ``typing.Literal`` that does not consist of only string values is passed, an informative error is raised. Native Enums and Naming +++++++++++++++++++++++ The :paramref:`.sqltypes.Enum.native_enum` parameter refers to if the :class:`.sqltypes.Enum` datatype should create a so-called "native" enum, which on MySQL/MariaDB is the ``ENUM`` datatype and on PostgreSQL is a new ``TYPE`` object created by ``CREATE TYPE``, or a "non-native" enum, which means that ``VARCHAR`` will be used to create the datatype. For backends other than MySQL/MariaDB or PostgreSQL, ``VARCHAR`` is used in all cases (third party dialects may have their own behaviors). Because PostgreSQL's ``CREATE TYPE`` requires that there's an explicit name for the type to be created, special fallback logic exists when working with implicitly generated :class:`.sqltypes.Enum` without specifying an explicit :class:`.sqltypes.Enum` datatype within a mapping: 1. If the :class:`.sqltypes.Enum` is linked to an ``enum.Enum`` object, the :paramref:`.sqltypes.Enum.native_enum` parameter defaults to ``True`` and the name of the enum will be taken from the name of the ``enum.Enum`` datatype. The PostgreSQL backend will assume ``CREATE TYPE`` with this name. 2. If the :class:`.sqltypes.Enum` is linked to a ``typing.Literal`` object, the :paramref:`.sqltypes.Enum.native_enum` parameter defaults to ``False``; no name is generated and ``VARCHAR`` is assumed. To use ``typing.Literal`` with a PostgreSQL ``CREATE TYPE`` type, an explicit :class:`.sqltypes.Enum` must be used, either within the type map:: import enum import typing import sqlalchemy from sqlalchemy.orm import DeclarativeBase Status = Literal["pending", "received", "completed"] class Base(DeclarativeBase): type_annotation_map = { Status: sqlalchemy.Enum("pending", "received", "completed", name="status_enum"), } Or alternatively within :func:`_orm.mapped_column`:: import enum import typing import sqlalchemy from sqlalchemy.orm import DeclarativeBase Status = Literal["pending", "received", "completed"] class Base(DeclarativeBase): pass class SomeClass(Base): __tablename__ = "some_table" id: Mapped[int] = mapped_column(primary_key=True) status: Mapped[Status] = mapped_column( sqlalchemy.Enum("pending", "received", "completed", name="status_enum") ) Altering the Configuration of the Default Enum +++++++++++++++++++++++++++++++++++++++++++++++ In order to modify the fixed configuration of the :class:`.enum.Enum` datatype that's generated implicitly, specify new entries in the :paramref:`_orm.registry.type_annotation_map`, indicating additional arguments. For example, to use "non native enumerations" unconditionally, the :paramref:`.Enum.native_enum` parameter may be set to False for all types:: import enum import typing import sqlalchemy from sqlalchemy.orm import DeclarativeBase class Base(DeclarativeBase): type_annotation_map = { enum.Enum: sqlalchemy.Enum(enum.Enum, native_enum=False), typing.Literal: sqlalchemy.Enum(enum.Enum, native_enum=False), } .. versionchanged:: 2.0.1 Implemented support for overriding parameters such as :paramref:`_sqltypes.Enum.native_enum` within the :class:`_sqltypes.Enum` datatype when establishing the :paramref:`_orm.registry.type_annotation_map`. Previously, this functionality was not working. To use a specific configuration for a specific ``enum.Enum`` subtype, such as setting the string length to 50 when using the example ``Status`` datatype:: import enum import sqlalchemy from sqlalchemy.orm import DeclarativeBase class Status(enum.Enum): PENDING = "pending" RECEIVED = "received" COMPLETED = "completed" class Base(DeclarativeBase): type_annotation_map = { Status: sqlalchemy.Enum(Status, length=50, native_enum=False) } Linking Specific ``enum.Enum`` or ``typing.Literal`` to other datatypes ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ The above examples feature the use of an :class:`_sqltypes.Enum` that is automatically configuring itself to the arguments / attributes present on an ``enum.Enum`` or ``typing.Literal`` type object. For use cases where specific kinds of ``enum.Enum`` or ``typing.Literal`` should be linked to other types, these specific types may be placed in the type map also. In the example below, an entry for ``Literal[]`` that contains non-string types is linked to the :class:`_sqltypes.JSON` datatype:: from typing import Literal from sqlalchemy import JSON from sqlalchemy.orm import DeclarativeBase my_literal = Literal[0, 1, True, False, "true", "false"] class Base(DeclarativeBase): type_annotation_map = {my_literal: JSON} In the above configuration, the ``my_literal`` datatype will resolve to a :class:`._sqltypes.JSON` instance. Other ``Literal`` variants will continue to resolve to :class:`_sqltypes.Enum` datatypes. Dataclass features in ``mapped_column()`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The :func:`_orm.mapped_column` construct integrates with SQLAlchemy's "native dataclasses" feature, discussed at :ref:`orm_declarative_native_dataclasses`. See that section for current background on additional directives supported by :func:`_orm.mapped_column`. .. _orm_declarative_metadata: Accessing Table and Metadata ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ A declaratively mapped class will always include an attribute called ``__table__``; when the above configuration using ``__tablename__`` is complete, the declarative process makes the :class:`_schema.Table` available via the ``__table__`` attribute:: # access the Table user_table = User.__table__ The above table is ultimately the same one that corresponds to the :attr:`_orm.Mapper.local_table` attribute, which we can see through the :ref:`runtime inspection system `:: from sqlalchemy import inspect user_table = inspect(User).local_table The :class:`_schema.MetaData` collection associated with both the declarative :class:`_orm.registry` as well as the base class is frequently necessary in order to run DDL operations such as CREATE, as well as in use with migration tools such as Alembic. This object is available via the ``.metadata`` attribute of :class:`_orm.registry` as well as the declarative base class. Below, for a small script we may wish to emit a CREATE for all tables against a SQLite database:: engine = create_engine("sqlite://") Base.metadata.create_all(engine) .. _orm_declarative_table_configuration: Declarative Table Configuration ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ When using Declarative Table configuration with the ``__tablename__`` declarative class attribute, additional arguments to be supplied to the :class:`_schema.Table` constructor should be provided using the ``__table_args__`` declarative class attribute. This attribute accommodates both positional as well as keyword arguments that are normally sent to the :class:`_schema.Table` constructor. The attribute can be specified in one of two forms. One is as a dictionary:: class MyClass(Base): __tablename__ = "sometable" __table_args__ = {"mysql_engine": "InnoDB"} The other, a tuple, where each argument is positional (usually constraints):: class MyClass(Base): __tablename__ = "sometable" __table_args__ = ( ForeignKeyConstraint(["id"], ["remote_table.id"]), UniqueConstraint("foo"), ) Keyword arguments can be specified with the above form by specifying the last argument as a dictionary:: class MyClass(Base): __tablename__ = "sometable" __table_args__ = ( ForeignKeyConstraint(["id"], ["remote_table.id"]), UniqueConstraint("foo"), {"autoload": True}, ) A class may also specify the ``__table_args__`` declarative attribute, as well as the ``__tablename__`` attribute, in a dynamic style using the :func:`_orm.declared_attr` method decorator. See :ref:`orm_mixins_toplevel` for background. .. _orm_declarative_table_schema_name: Explicit Schema Name with Declarative Table ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The schema name for a :class:`_schema.Table` as documented at :ref:`schema_table_schema_name` is applied to an individual :class:`_schema.Table` using the :paramref:`_schema.Table.schema` argument. When using Declarative tables, this option is passed like any other to the ``__table_args__`` dictionary:: from sqlalchemy.orm import DeclarativeBase class Base(DeclarativeBase): pass class MyClass(Base): __tablename__ = "sometable" __table_args__ = {"schema": "some_schema"} The schema name can also be applied to all :class:`_schema.Table` objects globally by using the :paramref:`_schema.MetaData.schema` parameter documented at :ref:`schema_metadata_schema_name`. The :class:`_schema.MetaData` object may be constructed separately and associated with a :class:`_orm.DeclarativeBase` subclass by assigning to the ``metadata`` attribute directly:: from sqlalchemy import MetaData from sqlalchemy.orm import DeclarativeBase metadata_obj = MetaData(schema="some_schema") class Base(DeclarativeBase): metadata = metadata_obj class MyClass(Base): # will use "some_schema" by default __tablename__ = "sometable" .. seealso:: :ref:`schema_table_schema_name` - in the :ref:`metadata_toplevel` documentation. .. _orm_declarative_column_options: Setting Load and Persistence Options for Declarative Mapped Columns ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The :func:`_orm.mapped_column` construct accepts additional ORM-specific arguments that affect how the generated :class:`_schema.Column` is mapped, affecting its load and persistence-time behavior. Options that are commonly used include: * **deferred column loading** - The :paramref:`_orm.mapped_column.deferred` boolean establishes the :class:`_schema.Column` using :ref:`deferred column loading ` by default. In the example below, the ``User.bio`` column will not be loaded by default, but only when accessed:: class User(Base): __tablename__ = "user" id: Mapped[int] = mapped_column(primary_key=True) name: Mapped[str] bio: Mapped[str] = mapped_column(Text, deferred=True) .. seealso:: :ref:`orm_queryguide_column_deferral` - full description of deferred column loading * **active history** - The :paramref:`_orm.mapped_column.active_history` ensures that upon change of value for the attribute, the previous value will have been loaded and made part of the :attr:`.AttributeState.history` collection when inspecting the history of the attribute. This may incur additional SQL statements:: class User(Base): __tablename__ = "user" id: Mapped[int] = mapped_column(primary_key=True) important_identifier: Mapped[str] = mapped_column(active_history=True) See the docstring for :func:`_orm.mapped_column` for a list of supported parameters. .. seealso:: :ref:`orm_imperative_table_column_options` - describes using :func:`_orm.column_property` and :func:`_orm.deferred` for use with Imperative Table configuration .. _mapper_column_distinct_names: .. _orm_declarative_table_column_naming: Naming Declarative Mapped Columns Explicitly ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ All of the examples thus far feature the :func:`_orm.mapped_column` construct linked to an ORM mapped attribute, where the Python attribute name given to the :func:`_orm.mapped_column` is also that of the column as we see in CREATE TABLE statements as well as queries. The name for a column as expressed in SQL may be indicated by passing the string positional argument :paramref:`_orm.mapped_column.__name` as the first positional argument. In the example below, the ``User`` class is mapped with alternate names given to the columns themselves:: class User(Base): __tablename__ = "user" id: Mapped[int] = mapped_column("user_id", primary_key=True) name: Mapped[str] = mapped_column("user_name") Where above ``User.id`` resolves to a column named ``user_id`` and ``User.name`` resolves to a column named ``user_name``. We may write a :func:`_sql.select` statement using our Python attribute names and will see the SQL names generated: .. sourcecode:: pycon+sql >>> from sqlalchemy import select >>> print(select(User.id, User.name).where(User.name == "x")) {printsql}SELECT "user".user_id, "user".user_name FROM "user" WHERE "user".user_name = :user_name_1 .. seealso:: :ref:`orm_imperative_table_column_naming` - applies to Imperative Table .. _orm_declarative_table_adding_columns: Appending additional columns to an existing Declarative mapped class ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ A declarative table configuration allows the addition of new :class:`_schema.Column` objects to an existing mapping after the :class:`.Table` metadata has already been generated. For a declarative class that is declared using a declarative base class, the underlying metaclass :class:`.DeclarativeMeta` includes a ``__setattr__()`` method that will intercept additional :func:`_orm.mapped_column` or Core :class:`.Column` objects and add them to both the :class:`.Table` using :meth:`.Table.append_column` as well as to the existing :class:`.Mapper` using :meth:`.Mapper.add_property`:: MyClass.some_new_column = mapped_column(String) Using core :class:`_schema.Column`:: MyClass.some_new_column = Column(String) All arguments are supported including an alternate name, such as ``MyClass.some_new_column = mapped_column("some_name", String)``. However, the SQL type must be passed to the :func:`_orm.mapped_column` or :class:`_schema.Column` object explicitly, as in the above examples where the :class:`_sqltypes.String` type is passed. There's no capability for the :class:`_orm.Mapped` annotation type to take part in the operation. Additional :class:`_schema.Column` objects may also be added to a mapping in the specific circumstance of using single table inheritance, where additional columns are present on mapped subclasses that have no :class:`.Table` of their own. This is illustrated in the section :ref:`single_inheritance`. .. note:: Assignment of mapped properties to an already mapped class will only function correctly if the "declarative base" class is used, meaning the user-defined subclass of :class:`_orm.DeclarativeBase` or the dynamically generated class returned by :func:`_orm.declarative_base` or :meth:`_orm.registry.generate_base`. This "base" class includes a Python metaclass which implements a special ``__setattr__()`` method that intercepts these operations. Runtime assignment of class-mapped attributes to a mapped class will **not** work if the class is mapped using decorators like :meth:`_orm.registry.mapped` or imperative functions like :meth:`_orm.registry.map_imperatively`. .. _orm_imperative_table_configuration: Declarative with Imperative Table (a.k.a. Hybrid Declarative) ------------------------------------------------------------- Declarative mappings may also be provided with a pre-existing :class:`_schema.Table` object, or otherwise a :class:`_schema.Table` or other arbitrary :class:`_sql.FromClause` construct (such as a :class:`_sql.Join` or :class:`_sql.Subquery`) that is constructed separately. This is referred to as a "hybrid declarative" mapping, as the class is mapped using the declarative style for everything involving the mapper configuration, however the mapped :class:`_schema.Table` object is produced separately and passed to the declarative process directly:: from sqlalchemy import Column, ForeignKey, Integer, String from sqlalchemy.orm import DeclarativeBase class Base(DeclarativeBase): pass # construct a Table directly. The Base.metadata collection is # usually a good choice for MetaData but any MetaData # collection may be used. user_table = Table( "user", Base.metadata, Column("id", Integer, primary_key=True), Column("name", String), Column("fullname", String), Column("nickname", String), ) # construct the User class using this table. class User(Base): __table__ = user_table Above, a :class:`_schema.Table` object is constructed using the approach described at :ref:`metadata_describing`. It can then be applied directly to a class that is declaratively mapped. The ``__tablename__`` and ``__table_args__`` declarative class attributes are not used in this form. The above configuration is often more readable as an inline definition:: class User(Base): __table__ = Table( "user", Base.metadata, Column("id", Integer, primary_key=True), Column("name", String), Column("fullname", String), Column("nickname", String), ) A natural effect of the above style is that the ``__table__`` attribute is itself defined within the class definition block. As such it may be immediately referred towards within subsequent attributes, such as the example below which illustrates referring to the ``type`` column in a polymorphic mapper configuration:: class Person(Base): __table__ = Table( "person", Base.metadata, Column("id", Integer, primary_key=True), Column("name", String(50)), Column("type", String(50)), ) __mapper_args__ = { "polymorphic_on": __table__.c.type, "polymorhpic_identity": "person", } The "imperative table" form is also used when a non-:class:`_schema.Table` construct, such as a :class:`_sql.Join` or :class:`_sql.Subquery` object, is to be mapped. An example below:: from sqlalchemy import func, select subq = ( select( func.count(orders.c.id).label("order_count"), func.max(orders.c.price).label("highest_order"), orders.c.customer_id, ) .group_by(orders.c.customer_id) .subquery() ) customer_select = ( select(customers, subq) .join_from(customers, subq, customers.c.id == subq.c.customer_id) .subquery() ) class Customer(Base): __table__ = customer_select For background on mapping to non-:class:`_schema.Table` constructs see the sections :ref:`orm_mapping_joins` and :ref:`orm_mapping_arbitrary_subqueries`. The "imperative table" form is of particular use when the class itself is using an alternative form of attribute declaration, such as Python dataclasses. See the section :ref:`orm_declarative_dataclasses` for detail. .. seealso:: :ref:`metadata_describing` :ref:`orm_declarative_dataclasses` .. _orm_imperative_table_column_naming: Alternate Attribute Names for Mapping Table Columns ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The section :ref:`orm_declarative_table_column_naming` illustrated how to use :func:`_orm.mapped_column` to provide a specific name for the generated :class:`_schema.Column` object separate from the attribute name under which it is mapped. When using Imperative Table configuration, we already have :class:`_schema.Column` objects present. To map these to alternate names we may assign the :class:`_schema.Column` to the desired attributes directly:: user_table = Table( "user", Base.metadata, Column("user_id", Integer, primary_key=True), Column("user_name", String), ) class User(Base): __table__ = user_table id = user_table.c.user_id name = user_table.c.user_name The ``User`` mapping above will refer to the ``"user_id"`` and ``"user_name"`` columns via the ``User.id`` and ``User.name`` attributes, in the same way as demonstrated at :ref:`orm_declarative_table_column_naming`. One caveat to the above mapping is that the direct inline link to :class:`_schema.Column` will not be typed correctly when using :pep:`484` typing tools. A strategy to resolve this is to apply the :class:`_schema.Column` objects within the :func:`_orm.column_property` function; while the :class:`_orm.Mapper` already generates this property object for its internal use automatically, by naming it in the class declaration, typing tools will be able to match the attribute to the :class:`_orm.Mapped` annotation:: from sqlalchemy.orm import column_property from sqlalchemy.orm import Mapped class User(Base): __table__ = user_table id: Mapped[int] = column_property(user_table.c.user_id) name: Mapped[str] = column_property(user_table.c.user_name) .. seealso:: :ref:`orm_declarative_table_column_naming` - applies to Declarative Table .. _column_property_options: .. _orm_imperative_table_column_options: Applying Load, Persistence and Mapping Options for Imperative Table Columns ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The section :ref:`orm_declarative_column_options` reviewed how to set load and persistence options when using the :func:`_orm.mapped_column` construct with Declarative Table configuration. When using Imperative Table configuration, we already have existing :class:`_schema.Column` objects that are mapped. In order to map these :class:`_schema.Column` objects along with additional parameters that are specific to the ORM mapping, we may use the :func:`_orm.column_property` and :func:`_orm.deferred` constructs in order to associate additional parameters with the column. Options include: * **deferred column loading** - The :func:`_orm.deferred` function is shorthand for invoking :func:`_orm.column_property` with the :paramref:`_orm.column_property.deferred` parameter set to ``True``; this construct establishes the :class:`_schema.Column` using :ref:`deferred column loading ` by default. In the example below, the ``User.bio`` column will not be loaded by default, but only when accessed:: from sqlalchemy.orm import deferred user_table = Table( "user", Base.metadata, Column("id", Integer, primary_key=True), Column("name", String), Column("bio", Text), ) class User(Base): __table__ = user_table bio = deferred(user_table.c.bio) .. seealso:: :ref:`orm_queryguide_column_deferral` - full description of deferred column loading * **active history** - The :paramref:`_orm.column_property.active_history` ensures that upon change of value for the attribute, the previous value will have been loaded and made part of the :attr:`.AttributeState.history` collection when inspecting the history of the attribute. This may incur additional SQL statements:: from sqlalchemy.orm import deferred user_table = Table( "user", Base.metadata, Column("id", Integer, primary_key=True), Column("important_identifier", String), ) class User(Base): __table__ = user_table important_identifier = column_property( user_table.c.important_identifier, active_history=True ) .. seealso:: The :func:`_orm.column_property` construct is also important for cases where classes are mapped to alternative FROM clauses such as joins and selects. More background on these cases is at: * :ref:`maptojoin` * :ref:`mapper_sql_expressions` For Declarative Table configuration with :func:`_orm.mapped_column`, most options are available directly; see the section :ref:`orm_declarative_column_options` for examples. .. _orm_declarative_reflected: Mapping Declaratively with Reflected Tables -------------------------------------------- There are several patterns available which provide for producing mapped classes against a series of :class:`_schema.Table` objects that were introspected from the database, using the reflection process described at :ref:`metadata_reflection`. A simple way to map a class to a table reflected from the database is to use a declarative hybrid mapping, passing the :paramref:`_schema.Table.autoload_with` parameter to the constructor for :class:`_schema.Table`:: from sqlalchemy import create_engine from sqlalchemy import Table from sqlalchemy.orm import DeclarativeBase engine = create_engine("postgresql+psycopg2://user:pass@hostname/my_existing_database") class Base(DeclarativeBase): pass class MyClass(Base): __table__ = Table( "mytable", Base.metadata, autoload_with=engine, ) A variant on the above pattern that scales for many tables is to use the :meth:`.MetaData.reflect` method to reflect a full set of :class:`.Table` objects at once, then refer to them from the :class:`.MetaData`:: from sqlalchemy import create_engine from sqlalchemy import Table from sqlalchemy.orm import DeclarativeBase engine = create_engine("postgresql+psycopg2://user:pass@hostname/my_existing_database") class Base(DeclarativeBase): pass Base.metadata.reflect(engine) class MyClass(Base): __table__ = Base.metadata.tables["mytable"] One caveat to the approach of using ``__table__`` is that the mapped classes cannot be declared until the tables have been reflected, which requires the database connectivity source to be present while the application classes are being declared; it's typical that classes are declared as the modules of an application are being imported, but database connectivity isn't available until the application starts running code so that it can consume configuration information and create an engine. There are currently two approaches to working around this, described in the next two sections. .. _orm_declarative_reflected_deferred_reflection: Using DeferredReflection ^^^^^^^^^^^^^^^^^^^^^^^^^ To accommodate the use case of declaring mapped classes where reflection of table metadata can occur afterwards, a simple extension called the :class:`.DeferredReflection` mixin is available, which alters the declarative mapping process to be delayed until a special class-level :meth:`.DeferredReflection.prepare` method is called, which will perform the reflection process against a target database, and will integrate the results with the declarative table mapping process, that is, classes which use the ``__tablename__`` attribute:: from sqlalchemy.ext.declarative import DeferredReflection from sqlalchemy.orm import DeclarativeBase class Base(DeclarativeBase): pass class Reflected(DeferredReflection): __abstract__ = True class Foo(Reflected, Base): __tablename__ = "foo" bars = relationship("Bar") class Bar(Reflected, Base): __tablename__ = "bar" foo_id = mapped_column(Integer, ForeignKey("foo.id")) Above, we create a mixin class ``Reflected`` that will serve as a base for classes in our declarative hierarchy that should become mapped when the ``Reflected.prepare`` method is called. The above mapping is not complete until we do so, given an :class:`_engine.Engine`:: engine = create_engine("postgresql+psycopg2://user:pass@hostname/my_existing_database") Reflected.prepare(engine) The purpose of the ``Reflected`` class is to define the scope at which classes should be reflectively mapped. The plugin will search among the subclass tree of the target against which ``.prepare()`` is called and reflect all tables which are named by declared classes; tables in the target database that are not part of mappings and are not related to the target tables via foreign key constraint will not be reflected. Using Automap ^^^^^^^^^^^^^^ A more automated solution to mapping against an existing database where table reflection is to be used is to use the :ref:`automap_toplevel` extension. This extension will generate entire mapped classes from a database schema, including relationships between classes based on observed foreign key constraints. While it includes hooks for customization, such as hooks that allow custom class naming and relationship naming schemes, automap is oriented towards an expedient zero-configuration style of working. If an application wishes to have a fully explicit model that makes use of table reflection, the :ref:`DeferredReflection ` class may be preferable for its less automated approach. .. seealso:: :ref:`automap_toplevel` .. _mapper_automated_reflection_schemes: Automating Column Naming Schemes from Reflected Tables ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ When using any of the previous reflection techniques, we have the option to change the naming scheme by which columns are mapped. The :class:`_schema.Column` object includes a parameter :paramref:`_schema.Column.key` which is a string name that determines under what name this :class:`_schema.Column` will be present in the :attr:`_schema.Table.c` collection, independently of the SQL name of the column. This key is also used by :class:`_orm.Mapper` as the attribute name under which the :class:`_schema.Column` will be mapped, if not supplied through other means such as that illustrated at :ref:`orm_imperative_table_column_naming`. When working with table reflection, we can intercept the parameters that will be used for :class:`_schema.Column` as they are received using the :meth:`_events.DDLEvents.column_reflect` event and apply whatever changes we need, including the ``.key`` attribute but also things like datatypes. The event hook is most easily associated with the :class:`_schema.MetaData` object that's in use as illustrated below:: from sqlalchemy import event from sqlalchemy.orm import DeclarativeBase class Base(DeclarativeBase): pass @event.listens_for(Base.metadata, "column_reflect") def column_reflect(inspector, table, column_info): # set column.key = "attr_" column_info["key"] = "attr_%s" % column_info["name"].lower() With the above event, the reflection of :class:`_schema.Column` objects will be intercepted with our event that adds a new ".key" element, such as in a mapping as below:: class MyClass(Base): __table__ = Table("some_table", Base.metadata, autoload_with=some_engine) The approach also works with both the :class:`.DeferredReflection` base class as well as with the :ref:`automap_toplevel` extension. For automap specifically, see the section :ref:`automap_intercepting_columns` for background. .. seealso:: :ref:`orm_declarative_reflected` :meth:`_events.DDLEvents.column_reflect` :ref:`automap_intercepting_columns` - in the :ref:`automap_toplevel` documentation .. _mapper_primary_key: Mapping to an Explicit Set of Primary Key Columns ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The :class:`.Mapper` construct in order to successfully map a table always requires that at least one column be identified as the "primary key" for that selectable. This is so that when an ORM object is loaded or persisted, it can be placed in the :term:`identity map` with an appropriate :term:`identity key`. In those cases where the a reflected table to be mapped does not include a primary key constraint, as well as in the general case for :ref:`mapping against arbitrary selectables ` where primary key columns might not be present, the :paramref:`.Mapper.primary_key` parameter is provided so that any set of columns may be configured as the "primary key" for the table, as far as ORM mapping is concerned. Given the following example of an Imperative Table mapping against an existing :class:`.Table` object where the table does not have any declared primary key (as may occur in reflection scenarios), we may map such a table as in the following example:: from sqlalchemy import Column from sqlalchemy import MetaData from sqlalchemy import String from sqlalchemy import Table from sqlalchemy import UniqueConstraint from sqlalchemy.orm import DeclarativeBase metadata = MetaData() group_users = Table( "group_users", metadata, Column("user_id", String(40), nullable=False), Column("group_id", String(40), nullable=False), UniqueConstraint("user_id", "group_id"), ) class Base(DeclarativeBase): pass class GroupUsers(Base): __table__ = group_users __mapper_args__ = {"primary_key": [group_users.c.user_id, group_users.c.group_id]} Above, the ``group_users`` table is an association table of some kind with string columns ``user_id`` and ``group_id``, but no primary key is set up; instead, there is only a :class:`.UniqueConstraint` establishing that the two columns represent a unique key. The :class:`.Mapper` does not automatically inspect unique constraints for primary keys; instead, we make use of the :paramref:`.Mapper.primary_key` parameter, passing a collection of ``[group_users.c.user_id, group_users.c.group_id]``, indicating that these two columns should be used in order to construct the identity key for instances of the ``GroupUsers`` class. .. _include_exclude_cols: Mapping a Subset of Table Columns ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Sometimes table reflection may provide a :class:`_schema.Table` with many columns that are not important for our needs and may be safely ignored. For such a table that has lots of columns that don't need to be referenced in the application, the :paramref:`_orm.Mapper.include_properties` or :paramref:`_orm.Mapper.exclude_properties` parameters can indicate a subset of columns to be mapped, where other columns from the target :class:`_schema.Table` will not be considered by the ORM in any way. Example:: class User(Base): __table__ = user_table __mapper_args__ = {"include_properties": ["user_id", "user_name"]} In the above example, the ``User`` class will map to the ``user_table`` table, only including the ``user_id`` and ``user_name`` columns - the rest are not referenced. Similarly:: class Address(Base): __table__ = address_table __mapper_args__ = {"exclude_properties": ["street", "city", "state", "zip"]} will map the ``Address`` class to the ``address_table`` table, including all columns present except ``street``, ``city``, ``state``, and ``zip``. As indicated in the two examples, columns may be referred towards either by string name or by referring to the :class:`_schema.Column` object directly. Referring to the object directly may be useful for explicitness as well as to resolve ambiguities when mapping to multi-table constructs that might have repeated names:: class User(Base): __table__ = user_table __mapper_args__ = { "include_properties": [user_table.c.user_id, user_table.c.user_name] } When columns are not included in a mapping, these columns will not be referenced in any SELECT statements emitted when executing :func:`_sql.select` or legacy :class:`_query.Query` objects, nor will there be any mapped attribute on the mapped class which represents the column; assigning an attribute of that name will have no effect beyond that of a normal Python attribute assignment. However, it is important to note that **schema level column defaults WILL still be in effect** for those :class:`_schema.Column` objects that include them, even though they may be excluded from the ORM mapping. "Schema level column defaults" refers to the defaults described at :ref:`metadata_defaults` including those configured by the :paramref:`_schema.Column.default`, :paramref:`_schema.Column.onupdate`, :paramref:`_schema.Column.server_default` and :paramref:`_schema.Column.server_onupdate` parameters. These constructs continue to have normal effects because in the case of :paramref:`_schema.Column.default` and :paramref:`_schema.Column.onupdate`, the :class:`_schema.Column` object is still present on the underlying :class:`_schema.Table`, thus allowing the default functions to take place when the ORM emits an INSERT or UPDATE, and in the case of :paramref:`_schema.Column.server_default` and :paramref:`_schema.Column.server_onupdate`, the relational database itself emits these defaults as a server side behavior. .. _mypy: https://mypy.readthedocs.io/en/stable/ .. _pylance: https://github.com/microsoft/pylance-release .. _generic: https://peps.python.org/pep-0484/#generics .. _dataclasses: https://docs.python.org/3/library/dataclasses.html