diff options
| author | jonathan vanasco <jonathan@2xlp.com> | 2015-07-10 18:52:46 -0400 |
|---|---|---|
| committer | jonathan vanasco <jonathan@2xlp.com> | 2015-07-10 18:52:46 -0400 |
| commit | 0a5dcdc2c4112478d87e5cd68c187e302f586834 (patch) | |
| tree | dfdb29e7668d4a007bd243805fd38cbccd971ad1 /doc | |
| parent | 6de3d490a2adb0fff43f98e15a53407b46668b61 (diff) | |
| parent | cadc2e0ba00feadf7e860598030bda0fb8bc691c (diff) | |
| download | sqlalchemy-0a5dcdc2c4112478d87e5cd68c187e302f586834.tar.gz | |
Merge branch 'master' of bitbucket.org:zzzeek/sqlalchemy
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/build/Makefile | 2 | ||||
| -rw-r--r-- | doc/build/changelog/changelog_09.rst | 68 | ||||
| -rw-r--r-- | doc/build/changelog/changelog_10.rst | 835 | ||||
| -rw-r--r-- | doc/build/changelog/migration_09.rst | 66 | ||||
| -rw-r--r-- | doc/build/changelog/migration_10.rst | 394 | ||||
| -rw-r--r-- | doc/build/conf.py | 36 | ||||
| -rw-r--r-- | doc/build/core/connections.rst | 68 | ||||
| -rw-r--r-- | doc/build/core/defaults.rst | 40 | ||||
| -rw-r--r-- | doc/build/core/metadata.rst | 14 | ||||
| -rw-r--r-- | doc/build/core/pooling.rst | 2 | ||||
| -rw-r--r-- | doc/build/core/tutorial.rst | 6 | ||||
| -rw-r--r-- | doc/build/faq/sessions.rst | 66 | ||||
| -rw-r--r-- | doc/build/orm/extensions/declarative/mixins.rst | 3 | ||||
| -rw-r--r-- | doc/build/orm/mapping_styles.rst | 2 | ||||
| -rw-r--r-- | doc/build/orm/session_basics.rst | 14 | ||||
| -rw-r--r-- | doc/build/orm/session_transaction.rst | 9 | ||||
| -rw-r--r-- | doc/build/requirements.txt | 2 |
17 files changed, 1582 insertions, 45 deletions
diff --git a/doc/build/Makefile b/doc/build/Makefile index 4de8f98b6..cd39b7288 100644 --- a/doc/build/Makefile +++ b/doc/build/Makefile @@ -2,7 +2,7 @@ # # You can set these variables from the command line. -SPHINXOPTS = +SPHINXOPTS = -v SPHINXBUILD = sphinx-build PAPER = BUILDDIR = output diff --git a/doc/build/changelog/changelog_09.rst b/doc/build/changelog/changelog_09.rst index ab1292f49..2d2964ba4 100644 --- a/doc/build/changelog/changelog_09.rst +++ b/doc/build/changelog/changelog_09.rst @@ -15,6 +15,71 @@ :version: 0.9.10 .. change:: + :tags: feature, sql + :tickets: 3418 + :versions: 1.0.5 + + Added official support for a CTE used by the SELECT present + inside of :meth:`.Insert.from_select`. This behavior worked + accidentally up until 0.9.9, when it no longer worked due to + unrelated changes as part of :ticket:`3248`. Note that this + is the rendering of the WITH clause after the INSERT, before the + SELECT; the full functionality of CTEs rendered at the top + level of INSERT, UPDATE, DELETE is a new feature targeted for a + later release. + + .. change:: + :tags: bug, ext + :tickets: 3408 + :versions: 1.0.4 + + Fixed bug where when using extended attribute instrumentation system, + the correct exception would not be raised when :func:`.class_mapper` + were called with an invalid input that also happened to not + be weak referencable, such as an integer. + + .. change:: + :tags: bug, tests, pypy + :tickets: 3406 + :versions: 1.0.4 + + Fixed an import that prevented "pypy setup.py test" from working + correctly. + + .. change:: + :tags: bug, engine + :tickets: 3375 + :versions: 1.0.1 + + Added the string value ``"none"`` to those accepted by the + :paramref:`.Pool.reset_on_return` parameter as a synonym for ``None``, + so that string values can be used for all settings, allowing + utilities like :func:`.engine_from_config` to be usable without + issue. + + .. change:: + :tags: bug, sql + :tickets: 3362 + :versions: 1.0.0 + + Fixed issue where a :class:`.MetaData` object that used a naming + convention would not properly work with pickle. The attribute was + skipped leading to inconsistencies and failures if the unpickled + :class:`.MetaData` object were used to base additional tables + from. + + .. change:: + :tags: bug, postgresql + :tickets: 3354 + :versions: 1.0.0 + + Fixed a long-standing bug where the :class:`.Enum` type as used + with the psycopg2 dialect in conjunction with non-ascii values + and ``native_enum=False`` would fail to decode return results properly. + This stemmed from when the PG :class:`.postgresql.ENUM` type used + to be a standalone type without a "non native" option. + + .. change:: :tags: bug, orm :tickets: 3349 @@ -77,6 +142,9 @@ Compared to the existing entry for ``"type"``, it will always be a mapped entity, even if extracted from a column expression, or None if the given expression is a pure core expression. + See also :ticket:`3403` which repaired a regression in this feature + which was unreleased in 0.9.10 but was released in the 1.0 version. + .. changelog:: :version: 0.9.9 diff --git a/doc/build/changelog/changelog_10.rst b/doc/build/changelog/changelog_10.rst index 95bb7d0f3..8ac3d5844 100644 --- a/doc/build/changelog/changelog_10.rst +++ b/doc/build/changelog/changelog_10.rst @@ -16,7 +16,821 @@ :start-line: 5 .. changelog:: + :version: 1.0.7 + + .. change:: + :tags: bug, orm + :tickets: 3469 + + Fixed 1.0 regression where value objects that override + ``__eq__()`` to return a non-boolean-capable object, such as + some geoalchemy types as well as numpy types, were being tested + for ``bool()`` during a unit of work update operation, where in + 0.9 the return value of ``__eq__()`` was tested against "is True" + to guard against this. + + .. change:: + :tags: bug, orm + :tickets: 3468 + + Fixed 1.0 regression where a "deferred" attribute would not populate + correctly if it were loaded within the "optimized inheritance load", + which is a special SELECT emitted in the case of joined table + inheritance used to populate expired or unloaded attributes against + a joined table without loading the base table. This is related to + the fact that SQLA 1.0 no longer guesses about loading deferred + columns and must be directed explicitly. + + .. change:: + :tags: bug, orm + :tickets: 3466 + + Fixed 1.0 regression where the "parent entity" of a synonym- + mapped attribute on top of an :func:`.aliased` object would + resolve to the original mapper, not the :func:`.aliased` + version of it, thereby causing problems for a :class:`.Query` + that relies on this attribute (e.g. it's the only representative + attribute given in the constructor) to figure out the correct FROM + clause for the query. + +.. changelog:: + :version: 1.0.6 + :released: June 25, 2015 + + .. change:: + :tags: bug, orm + :tickets: 3465 + + Fixed a major regression in the 1.0 series where the version_id_counter + feature would cause an object's version counter to be incremented + when there was no net change to the object's row, but instead an object + related to it via relationship (e.g. typically many-to-one) + were associated or de-associated with it, resulting in an UPDATE + statement that updates the object's version counter and nothing else. + In the use case where the relatively recent "server side" and/or + "programmatic/conditional" version counter feature were used + (e.g. setting version_id_generator to False), the bug could cause an + UPDATE without a valid SET clause to be emitted. + + .. change:: + :tags: bug, mssql + :tickets: 3464 + + Fixed issue when using :class:`.VARBINARY` type in conjunction with + an INSERT of NULL + pyodbc; pyodbc requires a special + object be passed in order to persist NULL. As the :class:`.VARBINARY` + type is now usually the default for :class:`.LargeBinary` due to + :ticket:`3039`, this issue is partially a regression in 1.0. + The pymssql driver appears to be unaffected. + + .. change:: + :tags: bug, postgresql, pypy + :tickets: 3439 + + Re-fixed this issue first released in 1.0.5 to fix psycopg2cffi + JSONB support once again, as they suddenly + switched on unconditional decoding of JSONB types in version 2.7.1. + Version detection now specifies 2.7.1 as where we should expect + the DBAPI to do json encoding for us. + + .. change:: + :tags: feature, postgresql + :tickets: 3455 + :pullreq: github:179 + + Added support for storage parameters under CREATE INDEX, using + a new keyword argument ``postgresql_with``. Also added support for + reflection to support both the ``postgresql_with`` flag as well + as the ``postgresql_using`` flag, which will now be set on + :class:`.Index` objects that are reflected, as well present + in a new "dialect_options" dictionary in the result of + :meth:`.Inspector.get_indexes`. Pull request courtesy Pete Hollobon. + + .. seealso:: + + :ref:`postgresql_index_storage` + + .. change:: + :tags: bug, orm + :tickets: 3462 + + Fixed 1.0 regression where the enhanced behavior of single-inheritance + joins of :ticket:`3222` takes place inappropriately + for a JOIN along explicit join criteria with a single-inheritance + subclass that does not make use of any discriminator, resulting + in an additional "AND NULL" clause. + + .. change:: + :tags: bug, postgresql + :tickets: 3454 + + Repaired the :class:`.ExcludeConstraint` construct to support common + features that other objects like :class:`.Index` now do, that + the column expression may be specified as an arbitrary SQL + expression such as :obj:`.cast` or :obj:`.text`. + + .. change:: + :tags: feature, postgresql + :pullreq: github:182 + + Added new execution option ``max_row_buffer`` which is interpreted + by the psycopg2 dialect when the ``stream_results`` option is + used, which sets a limit on the size of the row buffer that may be + allocated. This value is also provided based on the integer + value sent to :meth:`.Query.yield_per`. Pull request courtesy + mcclurem. + + .. change:: + :tags: bug, orm + :tickets: 3451 + :pullreq: github:181 + + Fixed bug in new :meth:`.Session.bulk_update_mappings` feature where + the primary key columns used in the WHERE clause to locate the row + would also be included in the SET clause, setting their value to + themselves unnecessarily. Pull request courtesy Patrick Hayes. + + .. change:: + :tags: bug, orm + :tickets: 3448 + + Fixed an unexpected-use regression whereby custom :class:`.Comparator` + objects that made use of the ``__clause_element__()`` method and + returned an object that was an ORM-mapped + :class:`.InstrumentedAttribute` and not explicitly a + :class:`.ColumnElement` would fail to be correctly + handled when passed as an expression to :meth:`.Session.query`. + The logic in 0.9 happened to succeed on this, so this use case is now + supported. + + .. change:: + :tags: bug, sql + :tickets: 3445 + + Fixed a bug where clause adaption as applied to a :class:`.Label` + object would fail to accommodate the labeled SQL expression + in all cases, such that any SQL operation that made use of + :meth:`.Label.self_group` would use the original unadapted + expression. One effect of this would be that an ORM :func:`.aliased` + construct would not fully accommodate attributes mapped by + :obj:`.column_property`, such that the un-aliased table could + leak out when the property were used in some kinds of SQL + comparisons. + + .. change:: + :tags: bug, documentation + :tickets: 2077 + + Fixed an internal "memoization" routine for method types such + that a Python descriptor is no longer used; repairs inspectability + of these methods including support for Sphinx documentation. + +.. changelog:: + :version: 1.0.5 + :released: June 7, 2015 + + .. change:: + :tags: feature, engine + + Added new engine event :meth:`.ConnectionEvents.engine_disposed`. + Called after the :meth:`.Engine.dispose` method is called. + + .. change:: + :tags: bug, postgresql, pypy + :tickets: 3439 + + Repaired some typing and test issues related to the pypy + psycopg2cffi dialect, in particular that the current 2.7.0 version + does not have native support for the JSONB type. The version detection + for psycopg2 features has been tuned into a specific sub-version + for psycopg2cffi. Additionally, test coverage has been enabled + for the full series of psycopg2 features under psycopg2cffi. + + .. change:: + :tags: feature, ext + :pullreq: bitbucket:54 + + Added support for ``*args`` to be passed to the baked query + initial callable, in the same way that ``*args`` are supported + for the :meth:`.BakedQuery.add_criteria` and + :meth:`.BakedQuery.with_criteria` methods. Initial PR courtesy + Naoki INADA. + + .. change:: + :tags: bug, engine + :tickets: 3435 + + Fixed bug where known boolean values used by + :func:`.engine_from_config` were not being parsed correctly; + these included ``pool_threadlocal`` and the psycopg2 argument + ``use_native_unicode``. + + .. change:: + :tags: bug, mssql + :tickets: 3424, 3430 + + Added a new dialect flag to the MSSQL dialect + ``legacy_schema_aliasing`` which when set to False will disable a + very old and obsolete behavior, that of the compiler's + attempt to turn all schema-qualified table names into alias names, + to work around old and no longer locatable issues where SQL + server could not parse a multi-part identifier name in all + circumstances. The behavior prevented more + sophisticated statements from working correctly, including those which + use hints, as well as CRUD statements that embed correlated SELECT + statements. Rather than continue to repair the feature to work + with more complex statements, it's better to just disable it + as it should no longer be needed for any modern SQL server + version. The flag defaults to True for the 1.0.x series, leaving + current behavior unchanged for this version series. In the 1.1 + series, it will default to False. For the 1.0 series, + when not set to either value explicitly, a warning is emitted + when a schema-qualified table is first used in a statement, which + suggests that the flag be set to False for all modern SQL Server + versions. + + .. seealso:: + + :ref:`legacy_schema_rendering` + + .. change:: + :tags: feature, engine + :tickets: 3379 + + Adjustments to the engine plugin hook, such that the + :meth:`.URL.get_dialect` method will continue to return the + ultimate :class:`.Dialect` object when a dialect plugin is used, + without the need for the caller to be aware of the + :meth:`.Dialect.get_dialect_cls` method. + + + .. change:: + :tags: bug, ext + :tickets: 3427 + + Fixed regression in the :mod:`sqlalchemy.ext.mutable` extension + as a result of the bugfix for :ticket:`3167`, + where attribute and validation events are no longer + called within the flush process. The mutable + extension was relying upon this behavior in the case where a column + level Python-side default were responsible for generating the new value + on INSERT or UPDATE, or when a value were fetched from the RETURNING + clause for "eager defaults" mode. The new value would not be subject + to any event when populated and the mutable extension could not + establish proper coercion or history listening. A new event + :meth:`.InstanceEvents.refresh_flush` is added which the mutable + extension now makes use of for this use case. + + .. change:: + :tags: feature, orm + :tickets: 3427 + + Added new event :meth:`.InstanceEvents.refresh_flush`, invoked + when an INSERT or UPDATE level default value fetched via RETURNING + or Python-side default is invoked within the flush process. This + is to provide a hook that is no longer present as a result of + :ticket:`3167`, where attribute and validation events are no longer + called within the flush process. + + .. change:: + :tags: feature, ext + :tickets: 3427 + + Added a new semi-public method to :class:`.MutableBase` + :meth:`.MutableBase._get_listen_keys`. Overriding this method + is needed in the case where a :class:`.MutableBase` subclass needs + events to propagate for attribute keys other than the key to which + the mutable type is associated with, when intercepting the + :meth:`.InstanceEvents.refresh` or + :meth:`.InstanceEvents.refresh_flush` events. The current example of + this is composites using :class:`.MutableComposite`. + + .. change:: + :tags: bug, engine + :tickets: 3421 + + Added support for the case of the misbehaving DBAPI that has + pep-249 exception names linked to exception classes of an entirely + different name, preventing SQLAlchemy's own exception wrapping from + wrapping the error appropriately. + The SQLAlchemy dialect in use needs to implement a new + accessor :attr:`.DefaultDialect.dbapi_exception_translation_map` + to support this feature; this is implemented now for the py-postgresql + dialect. + + .. change:: + :tags: bug, orm + :tickets: 3420 + + The "lightweight named tuple" used when a :class:`.Query` returns + rows failed to implement ``__slots__`` correctly such that it still + had a ``__dict__``. This is resolved, but in the extremely + unlikely case someone was assigning values to the returned tuples, + that will no longer work. + + .. change:: + :tags: bug, engine + :tickets: 3419 + + Fixed bug involving the case when pool checkout event handlers are used + and connection attempts are made in the handler itself which fail, + the owning connection record would not be freed until the stack trace + of the connect error itself were freed. For the case where a test + pool of only a single connection were used, this means the pool would + be fully checked out until that stack trace were freed. This mostly + impacts very specific debugging scenarios and is unlikely to have been + noticable in any production application. The fix applies an + explicit checkin of the record before re-raising the caught exception. + + +.. changelog:: + :version: 1.0.4 + :released: May 7, 2015 + + .. change:: + :tags: bug, orm + :tickets: 3416 + + Fixed unexpected-use regression where in the odd case that the + primaryjoin of a relationship involved comparison to an unhashable + type such as an HSTORE, lazy loads would fail due to a hash-oriented + check on the statement parameters, modified in 1.0 as a result of + :ticket:`3061` to use hashing and modified in :ticket:`3368` + to occur in cases more common than "load on pending". + The values are now checked for the ``__hash__`` attribute beforehand. + + .. change:: + :tags: bug, orm + :tickets: 3412, 3347 + + Liberalized an assertion that was added as part of :ticket:`3347` + to protect against unknown conditions when splicing inner joins + together within joined eager loads with ``innerjoin=True``; if + some of the joins use a "secondary" table, the assertion needs to + unwrap further joins in order to pass. + + .. change:: + :tags: bug, schema + :tickets: 3411 + + Fixed bug in enhanced constraint-attachment logic introduced in + :ticket:`3341` where in the unusual case of a constraint that refers + to a mixture of :class:`.Column` objects and string column names + at the same time, the auto-attach-on-column-attach logic will be + skipped; for the constraint to be auto-attached in this case, + all columns must be assembled on the target table up front. + Added a new section to the migration document regarding the + original feature as well as this change. + + .. seealso:: + + :ref:`change_3341` + + .. change:: + :tags: bug, orm + :tickets: 3409, 3320 + + Repaired / added to tests yet more expressions that were reported + as failing with the new 'entity' key value added to + :attr:`.Query.column_descriptions`, the logic to discover the "from" + clause is again reworked to accommodate columns from aliased classes, + as well as to report the correct value for the "aliased" flag in these + cases. + + +.. changelog:: + :version: 1.0.3 + :released: April 30, 2015 + + .. change:: + :tags: bug, orm, pypy + :tickets: 3405 + + Fixed regression from 0.9.10 prior to release due to :ticket:`3349` + where the check for query state on :meth:`.Query.update` or + :meth:`.Query.delete` compared the empty tuple to itself using ``is``, + which fails on Pypy to produce ``True`` in this case; this would + erronously emit a warning in 0.9 and raise an exception in 1.0. + + .. change:: + :tags: feature, engine + :tickets: 3379 + + New features added to support engine/pool plugins with advanced + functionality. Added a new "soft invalidate" feature to the + connection pool at the level of the checked out connection wrapper + as well as the :class:`._ConnectionRecord`. This works similarly + to a modern pool invalidation in that connections aren't actively + closed, but are recycled only on next checkout; this is essentially + a per-connection version of that feature. A new event + :class:`.PoolEvents.soft_invalidate` is added to complement it. + + Also added new flag + :attr:`.ExceptionContext.invalidate_pool_on_disconnect`. + Allows an error handler within :meth:`.ConnectionEvents.handle_error` + to maintain a "disconnect" condition, but to handle calling invalidate + on individual connections in a specific manner within the event. + + .. change:: + :tags: feature, engine + :tickets: 3355 + + Added new event :class:`.DialectEvents.do_connect`, which allows + interception / replacement of when the :meth:`.Dialect.connect` + hook is called to create a DBAPI connection. Also added + dialect plugin hooks :meth:`.Dialect.get_dialect_cls` and + :meth:`.Dialect.engine_created` which allow external plugins to + add events to existing dialects using entry points. + + .. change:: + :tags: bug, orm + :tickets: 3403, 3320 + + Fixed regression from 0.9.10 prior to release where the new addition + of ``entity`` to the :attr:`.Query.column_descriptions` accessor + would fail if the target entity was produced from a core selectable + such as a :class:`.Table` or :class:`.CTE` object. + + .. change:: + :tags: feature, sql + + Added a placeholder method :meth:`.TypeEngine.compare_against_backend` + which is now consumed by Alembic migrations as of 0.7.6. User-defined + types can implement this method to assist in the comparison of + a type against one reflected from the database. + + .. change:: + :tags: bug, orm + :tickets: 3402 + + Fixed regression within the flush process when an attribute were + set to a SQL expression for an UPDATE, and the SQL expression when + compared to the previous value of the attribute would produce a SQL + comparison other than ``==`` or ``!=``, the exception "Boolean value + of this clause is not defined" would raise. The fix ensures that + the unit of work will not interpret the SQL expression in this way. + + .. change:: + :tags: bug, ext + :tickets: 3397 + + Fixed bug in association proxy where an any()/has() + on an relationship->scalar non-object attribute comparison would fail, + e.g. + ``filter(Parent.some_collection_to_attribute.any(Child.attr == 'foo'))`` + + .. change:: + :tags: bug, sql + :tickets: 3396 + + Fixed bug where the truncation of long labels in SQL could produce + a label that overlapped another label that is not truncated; this + because the length threshhold for truncation was greater than + the portion of the label that remains after truncation. These + two values have now been made the same; label_length - 6. + The effect here is that shorter column labels will be "truncated" + where they would not have been truncated before. + + .. change:: + :tags: bug, orm + :tickets: 3392 + + Fixed unexpected use regression due to :ticket:`2992` where + textual elements placed + into the :meth:`.Query.order_by` clause in conjunction with joined + eager loading would be added to the columns clause of the inner query + in such a way that they were assumed to be table-bound column names, + in the case where the joined eager load needs to wrap the query + in a subquery to accommodate for a limit/offset. + + Originally, the behavior here was intentional, in that a query such + as ``query(User).order_by('name').limit(1)`` + would order by ``user.name`` even if the query was modified by + joined eager loading to be within a subquery, as ``'name'`` would + be interpreted as a symbol to be located within the FROM clauses, + in this case ``User.name``, which would then be copied into the + columns clause to ensure it were present for ORDER BY. However, the + feature fails to anticipate the case where ``order_by("name")`` refers + to a specific label name present in the local columns clause already + and not a name bound to a selectable in the FROM clause. + + Beyond that, the feature also fails for deprecated cases such as + ``order_by("name desc")``, which, while it emits a + warning that :func:`.text` should be used here (note that the issue + does not impact cases where :func:`.text` is used explicitly), + still produces a different query than previously where the "name desc" + expression is copied into the columns clause inappropriately. The + resolution is such that the "joined eager loading" aspect of the + feature will skip over these so-called "label reference" expressions + when augmenting the inner columns clause, as though they were + :func:`.text` constructs already. + + .. change:: + :tags: bug, sql + :tickets: 3391 + + Fixed regression due to :ticket:`3282` where the ``tables`` collection + passed as a keyword argument to the :meth:`.DDLEvents.before_create`, + :meth:`.DDLEvents.after_create`, :meth:`.DDLEvents.before_drop`, and + :meth:`.DDLEvents.after_drop` events would no longer be a list + of tables, but instead a list of tuples which contained a second + entry with foreign keys to be added or dropped. As the ``tables`` + collection, while documented as not necessarily stable, has come + to be relied upon, this change is considered a regression. + Additionally, in some cases for "drop", this collection would + be an iterator that would cause the operation to fail if + prematurely iterated. The collection is now a list of table + objects in all cases and test coverage for the format of this + collection is now added. + + + .. change:: + :tags: bug, orm + :tickets: 3388 + + Fixed a regression regarding the :meth:`.MapperEvents.instrument_class` + event where its invocation was moved to be after the class manager's + instrumentation of the class, which is the opposite of what the + documentation for the event explicitly states. The rationale for the + switch was due to Declarative taking the step of setting up + the full "instrumentation manager" for a class before it was mapped + for the purpose of the new ``@declared_attr`` features + described in :ref:`feature_3150`, but the change was also made + against the classical use of :func:`.mapper` for consistency. + However, SQLSoup relies upon the instrumentation event happening + before any instrumentation under classical mapping. + The behavior is reverted in the case of classical and declarative + mapping, the latter implemented by using a simple memoization + without using class manager. + + .. change:: + :tags: bug, orm + :tickets: 3387 + + Fixed issue in new :meth:`.QueryEvents.before_compile` event where + changes made to the :class:`.Query` object's collection of entities + to load within the event would render in the SQL, but would not + be reflected during the loading process. + +.. changelog:: + :version: 1.0.2 + :released: April 24, 2015 + + .. change:: + :tags: bug, sql + :tickets: 3338, 3385 + + Fixed a regression that was incorrectly fixed in 1.0.0b4 + (hence becoming two regressions); reports that + SELECT statements would GROUP BY a label name and fail was misconstrued + that certain backends such as SQL Server should not be emitting + ORDER BY or GROUP BY on a simple label name at all; when in fact, + we had forgotten that 0.9 was already emitting ORDER BY on a simple + label name for all backends, as described in :ref:`migration_1068`, + even though 1.0 includes a rewrite of this logic as part of + :ticket:`2992`. As far + as emitting GROUP BY against a simple label, even Postgresql has + cases where it will raise an error even though the label to group + on should be apparent, so it is clear that GROUP BY should never + be rendered in this way automatically. + + In 1.0.2, SQL Server, Firebird and others will again emit ORDER BY on + a simple label name when passed a + :class:`.Label` construct that is also present in the columns clause. + Additionally, no backend will emit GROUP BY against the simple label + name only when passed a :class:`.Label` construct. + + .. change:: + :tags: bug, orm, declarative + :tickets: 3383 + + Fixed unexpected use regression regarding the declarative + ``__declare_first__`` and ``__declare_last__`` accessors where these + would no longer be called on the superclass of the declarative base. + +.. changelog:: + :version: 1.0.1 + :released: April 23, 2015 + + .. change:: + :tags: bug, firebird + :tickets: 3380 + :pullreq: github:168 + + Fixed a regression due to :ticket:`3034` where limit/offset + clauses were not properly interpreted by the Firebird dialect. + Pull request courtesy effem-git. + + .. change:: + :tags: bug, firebird + :tickets: 3381 + + Fixed support for "literal_binds" mode when using limit/offset + with Firebird, so that the values are again rendered inline when + this is selected. Related to :ticket:`3034`. + + .. change:: + :tags: bug, sqlite + :tickets: 3378 + + Fixed a regression due to :ticket:`3282`, where due to the fact that + we attempt to assume the availability of ALTER when creating/dropping + schemas, in the case of SQLite we simply said to not worry about + foreign keys at all, since ALTER is not available, when creating + and dropping tables. This meant that the sorting of tables was + basically skipped in the case of SQLite, and for the vast majority + of SQLite use cases, this is not an issue. + + However, users who were doing DROPs on SQLite + with tables that contained data and with referential integrity + turned on would then experience errors, as the + dependency sorting *does* matter in the case of DROP with + enforced constraints, when those tables have data (SQLite will still + happily let you create foreign keys to nonexistent tables and drop + tables referring to existing ones with constraints enabled, as long as + there's no data being referenced). + + In order to maintain the new feature of :ticket:`3282` while still + allowing a SQLite DROP operation to maintain ordering, we now + do the sort with full FKs taken under consideration, and if we encounter + an unresolvable cycle, only *then* do we forego attempting to sort + the tables; we instead emit a warning and go with the unsorted list. + If an environment needs both ordered DROPs *and* has foreign key + cycles, then the warning notes they will need to restore the + ``use_alter`` flag to their :class:`.ForeignKey` and + :class:`.ForeignKeyConstraint` objects so that just those objects will + be omitted from the dependency sort. + + .. seealso:: + + :ref:`feature_3282` - contains an updated note about SQLite. + + .. change:: + :tags: bug, sql + :tickets: 3372 + + Fixed issue where a straight SELECT EXISTS query would fail to + assign the proper result type of Boolean to the result mapping, and + instead would leak column types from within the query into the + result map. This issue exists in 0.9 and earlier as well, however + has less of an impact in those versions. In 1.0, due to :ticket:`918` + this becomes a regression in that we now rely upon the result mapping + to be very accurate, else we can assign result-type processors to + the wrong column. In all versions, this issue also has the effect + that a simple EXISTS will not apply the Boolean type handler, leading + to simple 1/0 values for backends without native boolean instead of + True/False. The fix includes that an EXISTS columns argument + will be anon-labeled like other column expressions; a similar fix is + implemented for pure-boolean expressions like ``not_(True())``. + + .. change:: + :tags: bug, orm + :tickets: 3374 + + Fixed issue where a query of the form + ``query(B).filter(B.a != A(id=7))`` would render the ``NEVER_SET`` + symbol, when + given a transient object. For a persistent object, it would + always use the persisted database value and not the currently + set value. Assuming autoflush is turned on, this usually would + not be apparent for persistent values, as any pending changes + would be flushed first in any case. However, this is inconsistent + vs. the logic used for the non-negated comparison, + ``query(B).filter(B.a == A(id=7))``, which does use the + current value and additionally allows comparisons to transient + objects. The comparison now uses the current value and not + the database-persisted value. + + Unlike the other ``NEVER_SET`` issues that are repaired as regressions + caused by :ticket:`3061` in this release, this particular issue is + present at least as far back as 0.8 and possibly earlier, however it + was discovered as a result of repairing the related ``NEVER_SET`` + issues. + + .. seealso:: + + :ref:`bug_3374` + + .. change:: + :tags: bug, orm + :tickets: 3371 + + Fixed unexpected use regression cause by :ticket:`3061` where + the NEVER_SET + symbol could leak into relationship-oriented queries, including + ``filter()`` and ``with_parent()`` queries. The ``None`` symbol + is returned in all cases, however many of these queries have never + been correctly supported in any case, and produce comparisons + to NULL without using the IS operator. For this reason, a warning + is also added to that subset of relationship queries that don't + currently provide for ``IS NULL``. + + .. seealso:: + + :ref:`bug_3371` + + + .. change:: + :tags: bug, orm + :tickets: 3368 + + Fixed a regression caused by :ticket:`3061` where the + NEVER_SET symbol could leak into a lazyload query, subsequent + to the flush of a pending object. This would occur typically + for a many-to-one relationship that does not use a simple + "get" strategy. The good news is that the fix improves efficiency + vs. 0.9, because we can now skip the SELECT statement entirely + when we detect NEVER_SET symbols present in the parameters; prior to + :ticket:`3061`, we couldn't discern if the None here were set or not. + + +.. changelog:: + :version: 1.0.0 + :released: April 16, 2015 + + .. change:: + :tags: bug, orm + :tickets: 3367 + + Identified an inconsistency when handling :meth:`.Query.join` to the + same target more than once; it implicitly dedupes only in the case of + a relationship join, and due to :ticket:`3233`, in 1.0 a join + to the same table twice behaves differently than 0.9 in that it no + longer erroneously aliases. To help document this change, + the verbiage regarding :ticket:`3233` in the migration notes has + been generalized, and a warning has been added when :meth:`.Query.join` + is called against the same target relationship more than once. + + .. change:: + :tags: bug, orm + :tickets: 3364 + + Made a small improvement to the heuristics of relationship when + determining remote side with semi-self-referential (e.g. two joined + inh subclasses referring to each other), non-simple join conditions + such that the parententity is taken into account and can reduce the + need for using the ``remote()`` annotation; this can restore some + cases that might have worked without the annotation prior to 0.9.4 + via :ticket:`2948`. + + .. change:: + :tags: bug, mssql + :tickets: 3360 + + Fixed a regression where the "last inserted id" mechanics would + fail to store the correct value for MSSQL on an INSERT where the + primary key value was present in the insert params before execution, + as well as in the case where an INSERT from SELECT would state the + target columns as column objects, instead of string keys. + + + .. change:: + :tags: bug, mssql + :pullreq: github:166 + + Using the ``Binary`` constructor now present in pymssql rather than + patching one in. Pull request courtesy Ramiro Morales. + + .. change:: + :tags: bug, tests + :tickets: 3356 + + Fixed the pathing used when tests run; for sqla_nose.py and py.test, + the "./lib" prefix is again inserted at the head of sys.path but + only if sys.flags.no_user_site isn't set; this makes it act just + like the way Python puts "." in the current path by default. + For tox, we are setting the PYTHONNOUSERSITE flag now. + + .. change:: + :tags: feature, sql + :tickets: 3084 + :pullreq: bitbucket:47 + + The topological sorting used to sort :class:`.Table` objects + and available via the :attr:`.MetaData.sorted_tables` collection + will now produce a **deterministic** ordering; that is, the same + ordering each time given a set of tables with particular names + and dependencies. This is to help with comparison of DDL scripts + and other use cases. The tables are sent to the topological sort + sorted by name, and the topological sort itself will process + the incoming data in an ordered fashion. Pull request + courtesy Sebastian Bank. + + .. seealso:: + + :ref:`feature_3084` + + .. change:: + :tags: feature, orm + :pullreq: github:164 + + Added new argument :paramref:`.Query.update.update_args` which allows + kw arguments such as ``mysql_limit`` to be passed to the underlying + :class:`.Update` construct. Pull request courtesy Amir Sadoughi. + +.. changelog:: :version: 1.0.0b5 + :released: April 3, 2015 .. change:: :tags: bug, orm @@ -90,6 +904,9 @@ GROUP BY expressions. The flag is also turned off defensively for the Firebird and Sybase dialects. + .. note:: this resolution was incorrect, please see version 1.0.2 + for a rework of this resolution. + .. change:: :tags: feature, schema :tickets: 3341 @@ -102,6 +919,10 @@ same time the columns are associated with the table. This in particular helps in some edge cases in declarative but is also of general use. + .. seealso:: + + :ref:`change_3341` + .. change:: :tags: bug, sql :tickets: 3340 @@ -150,7 +971,7 @@ courtesy Thomas Grainger. .. change:: - :tags: change, ext, declarative + :tags: change, orm, declarative :tickets: 3331 Loosened some restrictions that were added to ``@declared_attr`` @@ -186,8 +1007,8 @@ :tickets: 3327 :pullreq: github:160 - Fixed 1.0 regression from pullreq github:137 where Py2K unicode - literals (e.g. ``u""``) would not be accepted by the + Fixed unexpected use regression from pullreq github:137 where + Py2K unicode literals (e.g. ``u""``) would not be accepted by the :paramref:`.relationship.cascade` option. Pull request courtesy Julien Castets. @@ -202,7 +1023,7 @@ on compatibility concerns, see :doc:`/changelog/migration_10`. .. change:: - :tags: feature, extensions + :tags: feature, ext :tickets: 3054 Added a new extension suite :mod:`sqlalchemy.ext.baked`. This @@ -277,7 +1098,7 @@ continued after the error raise occurred. .. change:: - :tags: bug, ext + :tags: bug, orm, declarative :tickets: 3219, 3240 Fixed bug where using an ``__abstract__`` mixin in the middle @@ -1030,7 +1851,7 @@ all transactional status and operations. .. change:: - :tags: bug, declarative + :tags: bug, orm, declarative :tickets: 2670 A relationship set up with :class:`.declared_attr` on @@ -1043,7 +1864,7 @@ :ref:`feature_3150` .. change:: - :tags: feature, declarative + :tags: feature, orm, declarative :tickets: 3150 The :class:`.declared_attr` construct has newly improved diff --git a/doc/build/changelog/migration_09.rst b/doc/build/changelog/migration_09.rst index 4904dcfdf..b07aed925 100644 --- a/doc/build/changelog/migration_09.rst +++ b/doc/build/changelog/migration_09.rst @@ -9,7 +9,7 @@ What's New in SQLAlchemy 0.9? and SQLAlchemy version 0.9, which had its first production release on December 30, 2013. - Document last updated: February 28, 2014 + Document last updated: June 10, 2015 Introduction ============ @@ -402,6 +402,70 @@ This is a small change demonstrated as follows:: Behavioral Changes - Core ========================= +Type objects no longer accept ignored keyword arguments +------------------------------------------------------- + +Up through the 0.8 series, most type objects accepted arbitrary keyword +arguments which were silently ignored:: + + from sqlalchemy import Date, Integer + + # storage_format argument here has no effect on any backend; + # it needs to be on the SQLite-specific type + d = Date(storage_format="%(day)02d.%(month)02d.%(year)04d") + + # display_width argument here has no effect on any backend; + # it needs to be on the MySQL-specific type + i = Integer(display_width=5) + +This was a very old bug for which a deprecation warning was added to the +0.8 series, but because nobody ever runs Python with the "-W" flag, it +was mostly never seen:: + + + $ python -W always::DeprecationWarning ~/dev/sqlalchemy/test.py + /Users/classic/dev/sqlalchemy/test.py:5: SADeprecationWarning: Passing arguments to + type object constructor <class 'sqlalchemy.types.Date'> is deprecated + d = Date(storage_format="%(day)02d.%(month)02d.%(year)04d") + /Users/classic/dev/sqlalchemy/test.py:9: SADeprecationWarning: Passing arguments to + type object constructor <class 'sqlalchemy.types.Integer'> is deprecated + i = Integer(display_width=5) + +As of the 0.9 series the "catch all" constructor is removed from +:class:`.TypeEngine`, and these meaningless arguments are no longer accepted. + +The correct way to make use of dialect-specific arguments such as +``storage_format`` and ``display_width`` is to use the appropriate +dialect-specific types:: + + from sqlalchemy.dialects.sqlite import DATE + from sqlalchemy.dialects.mysql import INTEGER + + d = DATE(storage_format="%(day)02d.%(month)02d.%(year)04d") + + i = INTEGER(display_width=5) + +What about the case where we want the dialect-agnostic type also? We +use the :meth:`.TypeEngine.with_variant` method:: + + from sqlalchemy import Date, Integer + from sqlalchemy.dialects.sqlite import DATE + from sqlalchemy.dialects.mysql import INTEGER + + d = Date().with_variant( + DATE(storage_format="%(day)02d.%(month)02d.%(year)04d"), + "sqlite" + ) + + i = Integer().with_variant( + INTEGER(display_width=5), + "mysql" + ) + +:meth:`.TypeEngine.with_variant` isn't new, it was added in SQLAlchemy +0.7.2. So code that is running on the 0.8 series can be corrected to use +this approach and tested before upgrading to 0.9. + ``None`` can no longer be used as a "partial AND" constructor -------------------------------------------------------------- diff --git a/doc/build/changelog/migration_10.rst b/doc/build/changelog/migration_10.rst index f4ead01aa..08e26fd4b 100644 --- a/doc/build/changelog/migration_10.rst +++ b/doc/build/changelog/migration_10.rst @@ -6,9 +6,9 @@ What's New in SQLAlchemy 1.0? This document describes changes between SQLAlchemy version 0.9, undergoing maintenance releases as of May, 2014, - and SQLAlchemy version 1.0, as of yet unreleased. + and SQLAlchemy version 1.0, released in April, 2015. - Document last updated: March 17, 2015 + Document last updated: June 9, 2015 Introduction ============ @@ -609,8 +609,8 @@ than the integer value. .. _feature_3282: -The ``use_alter`` flag on ``ForeignKeyConstraint`` is no longer needed ----------------------------------------------------------------------- +The ``use_alter`` flag on ``ForeignKeyConstraint`` is (usually) no longer needed +-------------------------------------------------------------------------------- The :meth:`.MetaData.create_all` and :meth:`.MetaData.drop_all` methods will now make use of a system that automatically renders an ALTER statement @@ -629,6 +629,16 @@ The :paramref:`.ForeignKeyConstraint.use_alter` and the same effect of establishing those constraints for which ALTER is required during a CREATE/DROP scenario. +As of version 1.0.1, special logic takes over in the case of SQLite, which +does not support ALTER, in the case that during a DROP, the given tables have +an unresolvable cycle; in this case a warning is emitted, and the tables +are dropped with **no** ordering, which is usually fine on SQLite unless +constraints are enabled. To resolve the warning and proceed with at least +a partial ordering on a SQLite database, particuarly one where constraints +are enabled, re-apply "use_alter" flags to those +:class:`.ForeignKey` and :class:`.ForeignKeyConstraint` objects which should +be explicitly omitted from the sort. + .. seealso:: :ref:`use_alter` - full description of the new behavior. @@ -723,6 +733,95 @@ now make use of all CHECK constraint conventions. :ticket:`3299` +.. _change_3341: + +Constraints referring to unattached Columns can auto-attach to the Table when their referred columns are attached +----------------------------------------------------------------------------------------------------------------- + +Since at least version 0.8, a :class:`.Constraint` has had the ability to +"auto-attach" itself to a :class:`.Table` based on being passed table-attached columns:: + + from sqlalchemy import Table, Column, MetaData, Integer, UniqueConstraint + + m = MetaData() + + t = Table('t', m, + Column('a', Integer), + Column('b', Integer) + ) + + uq = UniqueConstraint(t.c.a, t.c.b) # will auto-attach to Table + + assert uq in t.constraints + +In order to assist with some cases that tend to come up with declarative, +this same auto-attachment logic can now function even if the :class:`.Column` +objects are not yet associated with the :class:`.Table`; additional events +are established such that when those :class:`.Column` objects are associated, +the :class:`.Constraint` is also added:: + + from sqlalchemy import Table, Column, MetaData, Integer, UniqueConstraint + + m = MetaData() + + a = Column('a', Integer) + b = Column('b', Integer) + + uq = UniqueConstraint(a, b) + + t = Table('t', m, a, b) + + assert uq in t.constraints # constraint auto-attached + +The above feature was a late add as of version 1.0.0b3. A fix as of +version 1.0.4 for :ticket:`3411` ensures that this logic +does not occur if the :class:`.Constraint` refers to a mixture of +:class:`.Column` objects and string column names; as we do not yet have +tracking for the addition of names to a :class:`.Table`:: + + from sqlalchemy import Table, Column, MetaData, Integer, UniqueConstraint + + m = MetaData() + + a = Column('a', Integer) + b = Column('b', Integer) + + uq = UniqueConstraint(a, 'b') + + t = Table('t', m, a, b) + + # constraint *not* auto-attached, as we do not have tracking + # to locate when a name 'b' becomes available on the table + assert uq not in t.constraints + +Above, the attachment event for column "a" to table "t" will fire off before +column "b" is attached (as "a" is stated in the :class:`.Table` constructor +before "b"), and the constraint will fail to locate "b" if it were to attempt +an attachment. For consistency, if the constraint refers to any string names, +the autoattach-on-column-attach logic is skipped. + +The original auto-attach logic of course remains in place, if the :class:`.Table` +already contains all the target :class:`.Column` objects at the time +the :class:`.Constraint` is constructed:: + + from sqlalchemy import Table, Column, MetaData, Integer, UniqueConstraint + + m = MetaData() + + a = Column('a', Integer) + b = Column('b', Integer) + + + t = Table('t', m, a, b) + + uq = UniqueConstraint(a, 'b') + + # constraint auto-attached normally as in older versions + assert uq in t.constraints + + +:ticket:`3341` +:ticket:`3411` .. _change_2051: @@ -955,6 +1054,117 @@ to by string name as well:: :ticket:`3228` +.. _bug_3371: + +Warnings emitted when comparing objects with None values to relationships +------------------------------------------------------------------------- + +This change is new as of 1.0.1. Some users are performing +queries that are essentially of this form:: + + session.query(Address).filter(Address.user == User(id=None)) + +This pattern is not currently supported in SQLAlchemy. For all versions, +it emits SQL resembling:: + + SELECT address.id AS address_id, address.user_id AS address_user_id, + address.email_address AS address_email_address + FROM address WHERE ? = address.user_id + (None,) + +Note above, there is a comparison ``WHERE ? = address.user_id`` where the +bound value ``?`` is receving ``None``, or ``NULL`` in SQL. **This will +always return False in SQL**. The comparison here would in theory +generate SQL as follows:: + + SELECT address.id AS address_id, address.user_id AS address_user_id, + address.email_address AS address_email_address + FROM address WHERE address.user_id IS NULL + +But right now, **it does not**. Applications which are relying upon the +fact that "NULL = NULL" produces False in all cases run the risk that +someday, SQLAlchemy might fix this issue to generate "IS NULL", and the queries +will then produce different results. Therefore with this kind of operation, +you will see a warning:: + + SAWarning: Got None for value of column user.id; this is unsupported + for a relationship comparison and will not currently produce an + IS comparison (but may in a future release) + +Note that this pattern was broken in most cases for release 1.0.0 including +all of the betas; a value like ``SYMBOL('NEVER_SET')`` would be generated. +This issue has been fixed, but as a result of identifying this pattern, +the warning is now there so that we can more safely repair this broken +behavior (now captured in :ticket:`3373`) in a future release. + +:ticket:`3371` + +.. _bug_3374: + +A "negated contains or equals" relationship comparison will use the current value of attributes, not the database value +------------------------------------------------------------------------------------------------------------------------- + +This change is new as of 1.0.1; while we would have preferred for this to be in 1.0.0, +it only became apparent as a result of :ticket:`3371`. + +Given a mapping:: + + class A(Base): + __tablename__ = 'a' + id = Column(Integer, primary_key=True) + + class B(Base): + __tablename__ = 'b' + id = Column(Integer, primary_key=True) + a_id = Column(ForeignKey('a.id')) + a = relationship("A") + +Given ``A``, with primary key of 7, but which we changed to be 10 +without flushing:: + + s = Session(autoflush=False) + a1 = A(id=7) + s.add(a1) + s.commit() + + a1.id = 10 + +A query against a many-to-one relationship with this object as the target +will use the value 10 in the bound parameters:: + + s.query(B).filter(B.a == a1) + +Produces:: + + SELECT b.id AS b_id, b.a_id AS b_a_id + FROM b + WHERE ? = b.a_id + (10,) + +However, before this change, the negation of this criteria would **not** use +10, it would use 7, unless the object were flushed first:: + + s.query(B).filter(B.a != a1) + +Produces (in 0.9 and all versions prior to 1.0.1):: + + SELECT b.id AS b_id, b.a_id AS b_a_id + FROM b + WHERE b.a_id != ? OR b.a_id IS NULL + (7,) + +For a transient object, it would produce a broken query:: + + SELECT b.id, b.a_id + FROM b + WHERE b.a_id != :a_id_1 OR b.a_id IS NULL + {u'a_id_1': symbol('NEVER_SET')} + +This inconsistency has been repaired, and in all queries the current attribute +value, in this example ``10``, will now be used. + +:ticket:`3374` + .. _migration_3061: Changes to attribute events and other operations regarding attributes that have no pre-existing value @@ -1014,7 +1224,8 @@ INSERT statement in relational databases considers a missing value to be the same as NULL in most cases. Whether SQLAlchemy received a history event for a particular attribute set to None or not would usually not matter; as the difference between sending None/NULL or not wouldn't have an impact. -However, as :ticket:`3060` illustrates, there are some seldom edge cases +However, as :ticket:`3060` (described here in :ref:`migration_3060`) +illustrates, there are some seldom edge cases where we do in fact want to positively have ``None`` set. Also, allowing the attribute event here means it's now possible to create "default value" functions for ORM mapped attributes. @@ -1032,6 +1243,58 @@ symbol, and no change to the object's state occurs. :ticket:`3061` +.. _migration_3060: + +Priority of attribute changes on relationship-bound attributes vs. FK-bound may appear to change +------------------------------------------------------------------------------------------------ + +As a side effect of :ticket:`3060`, setting a relationship-bound attribute to ``None`` +is now a tracked history event which refers to the intention of persisting +``None`` to that attribute. As it has always been the case that setting a +relationship-bound attribute will trump direct assignment to the foreign key +attributes, a change in behavior can be seen here when assigning None. +Given a mapping:: + + class A(Base): + __tablename__ = 'table_a' + + id = Column(Integer, primary_key=True) + + class B(Base): + __tablename__ = 'table_b' + + id = Column(Integer, primary_key=True) + a_id = Column(ForeignKey('table_a.id')) + a = relationship(A) + +In 1.0, the relationship-bound attribute takes precedence over the FK-bound +attribute in all cases, whether or not +the value we assign is a reference to an ``A`` object or is ``None``. +In 0.9, the behavior is inconsistent and +only takes effect if a value is assigned; the None is not considered:: + + a1 = A(id=1) + a2 = A(id=2) + session.add_all([a1, a2]) + session.flush() + + b1 = B() + b1.a = a1 # we expect a_id to be '1'; takes precedence in 0.9 and 1.0 + + b2 = B() + b2.a = None # we expect a_id to be None; takes precedence only in 1.0 + + b1.a_id = 2 + b2.a_id = 2 + + session.add_all([b1, b2]) + session.commit() + + assert b1.a is a1 # passes in both 0.9 and 1.0 + assert b2.a is None # passes in 1.0, in 0.9 it's a2 + +:ticket:`3060` + .. _bug_3139: session.expunge() will fully detach an object that's been deleted @@ -1092,18 +1355,83 @@ joined loader options can still be used:: .. _bug_3233: -Single inheritance join targets will no longer sometimes implicitly alias themselves ------------------------------------------------------------------------------------- +Changes and fixes in handling of duplicate join targets +-------------------------------------------------------- -This is a bug where an unexpected and inconsistent behavior would occur -in some scenarios when joining to a single-table-inheritance entity. The -difficulty this might cause is that the query is supposed to raise an error, -as it is invalid SQL, however the bug would cause an alias to be added which -makes the query "work". The issue is confusing because this aliasing -is not applied consistently and could change based on the nature of the query -preceding the join. +Changes here encompass bugs where an unexpected and inconsistent +behavior would occur in some scenarios when joining to an entity +twice, or to multple single-table entities against the same table, +without using a relationship-based ON clause, as well as when joining +multiple times to the same target relationship. -A simple example is:: +Starting with a mapping as:: + + from sqlalchemy import Integer, Column, String, ForeignKey + from sqlalchemy.orm import Session, relationship + from sqlalchemy.ext.declarative import declarative_base + + Base = declarative_base() + + class A(Base): + __tablename__ = 'a' + id = Column(Integer, primary_key=True) + bs = relationship("B") + + class B(Base): + __tablename__ = 'b' + id = Column(Integer, primary_key=True) + a_id = Column(ForeignKey('a.id')) + +A query that joins to ``A.bs`` twice:: + + print s.query(A).join(A.bs).join(A.bs) + +Will render:: + + SELECT a.id AS a_id + FROM a JOIN b ON a.id = b.a_id + +The query deduplicates the redundant ``A.bs`` because it is attempting +to support a case like the following:: + + s.query(A).join(A.bs).\ + filter(B.foo == 'bar').\ + reset_joinpoint().join(A.bs, B.cs).filter(C.bar == 'bat') + +That is, the ``A.bs`` is part of a "path". As part of :ticket:`3367`, +arriving at the same endpoint twice without it being part of a +larger path will now emit a warning:: + + SAWarning: Pathed join target A.bs has already been joined to; skipping + +The bigger change involves when joining to an entity without using a +relationship-bound path. If we join to ``B`` twice:: + + print s.query(A).join(B, B.a_id == A.id).join(B, B.a_id == A.id) + +In 0.9, this would render as follows:: + + SELECT a.id AS a_id + FROM a JOIN b ON b.a_id = a.id JOIN b AS b_1 ON b_1.a_id = a.id + +This is problematic since the aliasing is implicit and in the case of different +ON clauses can lead to unpredictable results. + +In 1.0, no automatic aliasing is applied and we get:: + + SELECT a.id AS a_id + FROM a JOIN b ON b.a_id = a.id JOIN b ON b.a_id = a.id + +This will raise an error from the database. While it might be nice if +the "duplicate join target" acted identically if we joined both from +redundant relationships vs. redundant non-relationship based targets, +for now we are only changing the behavior in the more serious case where +implicit aliasing would have occurred previously, and only emitting a warning +in the relationship case. Ultimately, joining to the same thing twice without +any aliasing to disambiguate should raise an error in all cases. + +The change also has an impact on single-table inheritance targets. Using +a mapping as follows:: from sqlalchemy import Integer, Column, String, ForeignKey from sqlalchemy.orm import Session, relationship @@ -1151,7 +1479,8 @@ the identical SQL:: WHERE a.type IN (:type_2) The above SQL is invalid, as it renders "a" within the FROM list twice. -The bug however would occur with the second query only and render this instead:: +However, the implicit aliasing bug would occur with the second query only +and render this instead:: SELECT a.id AS a_id, a.type AS a_type FROM a JOIN b ON b.a_id = a.id JOIN a AS a_1 @@ -1173,6 +1502,7 @@ as all the subclasses normally refer to the same table:: print s.query(ASub1).join(B, ASub1.b).join(asub2_alias, B.a.of_type(asub2_alias)) :ticket:`3233` +:ticket:`3367` Deferred Columns No Longer Implicitly Undefer @@ -1290,7 +1620,7 @@ join into a subquery as a join target on SQLite. :ticket:`3008` -.. _change_3429: +.. _change_3249: Subqueries no longer applied to uselist=False joined eager loads ---------------------------------------------------------------- @@ -1339,6 +1669,20 @@ has always emitted a warning here and ignored addtional results for :ticket:`3249` +query.update() / query.delete() raises if used with join(), select_from(), from_self() +-------------------------------------------------------------------------------------- + +A warning is emitted in SQLAlchemy 0.9.10 (not yet released as of +June 9, 2015) when the :meth:`.Query.update` or :meth:`.Query.delete` methods +are invoked against a query which has also called upon :meth:`.Query.join`, +:meth:`.Query.outerjoin`, +:meth:`.Query.select_from` or :meth:`.Query.from_self`. These are unsupported +use cases which silently fail in the 0.9 series up until 0.9.10 where it emits +a warning. In 1.0, these cases raise an exception. + +:ticket:`3349` + + query.update() with ``synchronize_session='evaluate'`` raises on multi-table update ----------------------------------------------------------------------------------- @@ -1810,6 +2154,22 @@ columns regardless of how the object was constructed or its current state. +.. _feature_3084: + +MetaData.sorted_tables accessor is "deterministic" +----------------------------------------------------- + +The sorting of tables resulting from the :attr:`.MetaData.sorted_tables` +accessor is "deterministic"; the ordering should be the same in all cases +regardless of Python hashing. This is done by first sorting the tables +by name before passing them to the topological algorithm, which maintains +that ordering as it iterates. + +Note that this change does **not** yet apply to the ordering applied +when emitting :meth:`.MetaData.create_all` or :meth:`.MetaData.drop_all`. + +:ticket:`3084` + .. _bug_3170: null(), false() and true() constants are no longer singletons diff --git a/doc/build/conf.py b/doc/build/conf.py index 35204b1b9..fa9be2d25 100644 --- a/doc/build/conf.py +++ b/doc/build/conf.py @@ -13,6 +13,25 @@ import sys import os +import traceback + +def force_install_reqs(): + import logging + + log = logging.getLogger("pip") + handler = logging.StreamHandler(sys.stderr) + handler.setFormatter(logging.Formatter("[pip] %(message)s")) + log.addHandler(handler) + log.setLevel(logging.INFO) + + log.info("READTHEDOCS is set, force-installing requirements.txt") + + from pip.commands import install + req = os.path.join(os.path.dirname(__file__), "requirements.txt") + cmd = install.InstallCommand() + options, args = cmd.parse_args(["-v", "-U", "-r", req]) + cmd.run(options, args) + # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the @@ -23,6 +42,19 @@ sys.path.insert(0, os.path.abspath('.')) import sqlalchemy +# attempt to force pip to definitely get the latest +# versions of libraries, see +# https://github.com/rtfd/readthedocs.org/issues/1293 +rtd = os.environ.get('READTHEDOCS', None) == 'True' +if rtd: + try: + force_install_reqs() + except: + traceback.print_exc() + + + + # -- General configuration ----------------------------------------------------- # If your documentation needs a minimal Sphinx version, state it here. @@ -106,9 +138,9 @@ copyright = u'2007-2015, the SQLAlchemy authors and contributors' # The short X.Y version. version = "1.0" # The full version, including alpha/beta/rc tags. -release = "1.0.0b4" +release = "1.0.6" -release_date = "March 29, 2015" +release_date = "June 25, 2015" site_base = os.environ.get("RTD_SITE_BASE", "http://www.sqlalchemy.org") site_adapter_template = "docs_adapter.mako" diff --git a/doc/build/core/connections.rst b/doc/build/core/connections.rst index b6770bb82..72e1d6a61 100644 --- a/doc/build/core/connections.rst +++ b/doc/build/core/connections.rst @@ -368,6 +368,74 @@ the SQL statement. When the :class:`.ResultProxy` is closed, the underlying :class:`.Connection` is closed for us, resulting in the DBAPI connection being returned to the pool with transactional resources removed. +.. _engine_disposal: + +Engine Disposal +=============== + +The :class:`.Engine` refers to a connection pool, which means under normal +circumstances, there are open database connections present while the +:class:`.Engine` object is still resident in memory. When an :class:`.Engine` +is garbage collected, its connection pool is no longer referred to by +that :class:`.Engine`, and assuming none of its connections are still checked +out, the pool and its connections will also be garbage collected, which has the +effect of closing out the actual database connections as well. But otherwise, +the :class:`.Engine` will hold onto open database connections assuming +it uses the normally default pool implementation of :class:`.QueuePool`. + +The :class:`.Engine` is intended to normally be a permanent +fixture established up-front and maintained throughout the lifespan of an +application. It is **not** intended to be created and disposed on a +per-connection basis; it is instead a registry that maintains both a pool +of connections as well as configurational information about the database +and DBAPI in use, as well as some degree of internal caching of per-database +resources. + +However, there are many cases where it is desirable that all connection resources +referred to by the :class:`.Engine` be completely closed out. It's +generally not a good idea to rely on Python garbage collection for this +to occur for these cases; instead, the :class:`.Engine` can be explicitly disposed using +the :meth:`.Engine.dispose` method. This disposes of the engine's +underlying connection pool and replaces it with a new one that's empty. +Provided that the :class:`.Engine` +is discarded at this point and no longer used, all **checked-in** connections +which it refers to will also be fully closed. + +Valid use cases for calling :meth:`.Engine.dispose` include: + +* When a program wants to release any remaining checked-in connections + held by the connection pool and expects to no longer be connected + to that database at all for any future operations. + +* When a program uses multiprocessing or ``fork()``, and an + :class:`.Engine` object is copied to the child process, + :meth:`.Engine.dispose` should be called so that the engine creates + brand new database connections local to that fork. Database connections + generally do **not** travel across process boundaries. + +* Within test suites or multitenancy scenarios where many + ad-hoc, short-lived :class:`.Engine` objects may be created and disposed. + + +Connections that are **checked out** are **not** discarded when the +engine is disposed or garbage collected, as these connections are still +strongly referenced elsewhere by the application. +However, after :meth:`.Engine.dispose` is called, those +connections are no longer associated with that :class:`.Engine`; when they +are closed, they will be returned to their now-orphaned connection pool +which will ultimately be garbage collected, once all connections which refer +to it are also no longer referenced anywhere. +Since this process is not easy to control, it is strongly recommended that +:meth:`.Engine.dispose` is called only after all checked out connections +are checked in or otherwise de-associated from their pool. + +An alternative for applications that are negatively impacted by the +:class:`.Engine` object's use of connection pooling is to disable pooling +entirely. This typically incurs only a modest performance impact upon the +use of new connections, and means that when a connection is checked in, +it is entirely closed out and is not held in memory. See :ref:`pool_switching` +for guidelines on how to disable pooling. + .. _threadlocal_strategy: Using the Threadlocal Execution Strategy diff --git a/doc/build/core/defaults.rst b/doc/build/core/defaults.rst index 1d55cd6c6..4166ac449 100644 --- a/doc/build/core/defaults.rst +++ b/doc/build/core/defaults.rst @@ -325,6 +325,46 @@ executed standalone like a SQL expression, which has the effect of calling its seq = Sequence('some_sequence') nextid = connection.execute(seq) +Associating a Sequence as the Server Side Default +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When we associate a :class:`.Sequence` with a :class:`.Column` as above, +this association is an **in-Python only** association. The CREATE TABLE +that would be generated for our :class:`.Table` would not refer to this +sequence. If we want the sequence to be used as a server-side default, +meaning it takes place even if we emit INSERT commands to the table from +the SQL commandline, we can use the :paramref:`.Column.server_default` +parameter in conjunction with the value-generation function of the +sequence, available from the :meth:`.Sequence.next_value` method:: + + cart_id_seq = Sequence('cart_id_seq') + table = Table("cartitems", meta, + Column( + "cart_id", Integer, cart_id_seq, + server_default=cart_id_seq.next_value(), primary_key=True), + Column("description", String(40)), + Column("createdate", DateTime()) + ) + +The above metadata will generate a CREATE TABLE statement on Postgresql as:: + + CREATE TABLE cartitems ( + cart_id INTEGER DEFAULT nextval('cart_id_seq') NOT NULL, + description VARCHAR(40), + createdate TIMESTAMP WITHOUT TIME ZONE, + PRIMARY KEY (cart_id) + ) + +We place the :class:`.Sequence` also as a Python-side default above, that +is, it is mentioned twice in the :class:`.Column` definition. Depending +on the backend in use, this may not be strictly necessary, for example +on the Postgresql backend the Core will use ``RETURNING`` to access the +newly generated primary key value in any case. However, for the best +compatibility, :class:`.Sequence` was originally intended to be a Python-side +directive first and foremost so it's probably a good idea to specify it +in this way as well. + + Default Objects API ------------------- diff --git a/doc/build/core/metadata.rst b/doc/build/core/metadata.rst index e46217c17..c04de158b 100644 --- a/doc/build/core/metadata.rst +++ b/doc/build/core/metadata.rst @@ -30,10 +30,10 @@ The remaining positional arguments are mostly :class:`~sqlalchemy.schema.Column` objects describing each column:: user = Table('user', metadata, - Column('user_id', Integer, primary_key = True), - Column('user_name', String(16), nullable = False), + Column('user_id', Integer, primary_key=True), + Column('user_name', String(16), nullable=False), Column('email_address', String(60)), - Column('password', String(20), nullable = False) + Column('password', String(20), nullable=False) ) Above, a table called ``user`` is described, which contains four columns. The @@ -151,10 +151,10 @@ will issue the CREATE statements: metadata = MetaData() user = Table('user', metadata, - Column('user_id', Integer, primary_key = True), - Column('user_name', String(16), nullable = False), + Column('user_id', Integer, primary_key=True), + Column('user_name', String(16), nullable=False), Column('email_address', String(60), key='email'), - Column('password', String(20), nullable = False) + Column('password', String(20), nullable=False) ) user_prefs = Table('user_prefs', metadata, @@ -291,7 +291,7 @@ example, MySQL has different table backend types, including "MyISAM" and ``mysql_engine``:: addresses = Table('engine_email_addresses', meta, - Column('address_id', Integer, primary_key = True), + Column('address_id', Integer, primary_key=True), Column('remote_user_id', Integer, ForeignKey(users.c.user_id)), Column('email_address', String(20)), mysql_engine='InnoDB' diff --git a/doc/build/core/pooling.rst b/doc/build/core/pooling.rst index 0dbf835d9..ce6d443f9 100644 --- a/doc/build/core/pooling.rst +++ b/doc/build/core/pooling.rst @@ -56,6 +56,8 @@ queued up - the pool would only grow to that size if the application actually used five connections concurrently, in which case the usage of a small pool is an entirely appropriate default behavior. +.. _pool_switching: + Switching Pool Implementations ------------------------------ diff --git a/doc/build/core/tutorial.rst b/doc/build/core/tutorial.rst index b4f185ac2..cc2a97625 100644 --- a/doc/build/core/tutorial.rst +++ b/doc/build/core/tutorial.rst @@ -364,6 +364,10 @@ statement is compiled against the **first** dictionary in the list, and it's assumed that all subsequent argument dictionaries are compatible with that statement. +The "executemany" style of invocation is available for each of the +:func:`.insert`, :func:`.update` and :func:`.delete` constructs. + + .. _coretutorial_selecting: Selecting @@ -1754,7 +1758,7 @@ that can be specified: COMMIT {stop}<sqlalchemy.engine.result.ResultProxy object at 0x...> -When using :meth:`~.TableClause.update` in an "execute many" context, +When using :meth:`~.TableClause.update` in an "executemany" context, we may wish to also use explicitly named bound parameters in the WHERE clause. Again, :func:`~.expression.bindparam` is the construct used to achieve this: diff --git a/doc/build/faq/sessions.rst b/doc/build/faq/sessions.rst index 300b4bdbc..e3aae00ce 100644 --- a/doc/build/faq/sessions.rst +++ b/doc/build/faq/sessions.rst @@ -7,6 +7,72 @@ Sessions / Queries :backlinks: none +I'm re-loading data with my Session but it isn't seeing changes that I committed elsewhere +------------------------------------------------------------------------------------------ + +The main issue regarding this behavior is that the session acts as though +the transaction is in the *serializable* isolation state, even if it's not +(and it usually is not). In practical terms, this means that the session +does not alter any data that it's already read within the scope of a transaction. + +If the term "isolation level" is unfamiliar, then you first need to read this link: + +`Isolation Level <https://en.wikipedia.org/wiki/Isolation_%28database_systems%29>`_ + +In short, serializable isolation level generally means +that once you SELECT a series of rows in a transaction, you will get +*the identical data* back each time you re-emit that SELECT. If you are in +the next-lower isolation level, "repeatable read", you'll +see newly added rows (and no longer see deleted rows), but for rows that +you've *already* loaded, you won't see any change. Only if you are in a +lower isolation level, e.g. "read committed", does it become possible to +see a row of data change its value. + +For information on controlling the isolation level when using the +SQLAlchemy ORM, see :ref:`session_transaction_isolation`. + +To simplify things dramatically, the :class:`.Session` itself works in +terms of a completely isolated transaction, and doesn't overwrite any mapped attributes +it's already read unless you tell it to. The use case of trying to re-read +data you've already loaded in an ongoing transaction is an *uncommon* use +case that in many cases has no effect, so this is considered to be the +exception, not the norm; to work within this exception, several methods +are provided to allow specific data to be reloaded within the context +of an ongoing transaction. + +To understand what we mean by "the transaction" when we talk about the +:class:`.Session`, your :class:`.Session` is intended to only work within +a transaction. An overview of this is at :ref:`unitofwork_transaction`. + +Once we've figured out what our isolation level is, and we think that +our isolation level is set at a low enough level so that if we re-SELECT a row, +we should see new data in our :class:`.Session`, how do we see it? + +Three ways, from most common to least: + +1. We simply end our transaction and start a new one on next access + with our :class:`.Session` by calling :meth:`.Session.commit` (note + that if the :class:`.Session` is in the lesser-used "autocommit" + mode, there would be a call to :meth:`.Session.begin` as well). The + vast majority of applications and use cases do not have any issues + with not being able to "see" data in other transactions because + they stick to this pattern, which is at the core of the best practice of + **short lived transactions**. + See :ref:`session_faq_whentocreate` for some thoughts on this. + +2. We tell our :class:`.Session` to re-read rows that it has already read, + either when we next query for them using :meth:`.Session.expire_all` + or :meth:`.Session.expire`, or immediately on an object using + :class:`.Session.refresh`. See :ref:`session_expire` for detail on this. + +3. We can run whole queries while setting them to definitely overwrite + already-loaded objects as they read rows by using + :meth:`.Query.populate_existing`. + +But remember, **the ORM cannot see changes in rows if our isolation +level is repeatable read or higher, unless we start a new transaction**. + + "This Session's transaction has been rolled back due to a previous exception during flush." (or similar) --------------------------------------------------------------------------------------------------------- diff --git a/doc/build/orm/extensions/declarative/mixins.rst b/doc/build/orm/extensions/declarative/mixins.rst index 1b3364c2e..917c55f88 100644 --- a/doc/build/orm/extensions/declarative/mixins.rst +++ b/doc/build/orm/extensions/declarative/mixins.rst @@ -138,8 +138,7 @@ point at which the ``User`` class is constructed, and the declarative extension can use the resulting :class:`.Column` object as returned by the method without the need to copy it. -.. versionchanged:: > 0.6.5 - Rename 0.6.5 ``sqlalchemy.util.classproperty`` +.. versionchanged:: 0.6.5 Rename ``sqlalchemy.util.classproperty`` into :class:`~.declared_attr`. Columns generated by :class:`~.declared_attr` can also be diff --git a/doc/build/orm/mapping_styles.rst b/doc/build/orm/mapping_styles.rst index 7571ce650..52c478361 100644 --- a/doc/build/orm/mapping_styles.rst +++ b/doc/build/orm/mapping_styles.rst @@ -120,7 +120,7 @@ user-defined class, linked together with a :func:`.mapper`. When we talk about "the behavior of :func:`.mapper`", this includes when using the Declarative system as well - it's still used, just behind the scenes. -Runtime Intropsection of Mappings, Objects +Runtime Introspection of Mappings, Objects ========================================== The :class:`.Mapper` object is available from any mapped class, regardless diff --git a/doc/build/orm/session_basics.rst b/doc/build/orm/session_basics.rst index 8919864ca..dd1162216 100644 --- a/doc/build/orm/session_basics.rst +++ b/doc/build/orm/session_basics.rst @@ -158,7 +158,7 @@ Session Frequently Asked Questions =================================== By this point, many users already have questions about sessions. -This section presents a mini-FAQ (note that we have also a `real FAQ </faq/index>`) +This section presents a mini-FAQ (note that we have also a :doc:`real FAQ </faq/index>`) of the most basic issues one is presented with when using a :class:`.Session`. When do I make a :class:`.sessionmaker`? @@ -192,9 +192,15 @@ When do I construct a :class:`.Session`, when do I commit it, and when do I clos .. topic:: tl;dr; - As a general rule, keep the lifecycle of the session **separate and - external** from functions and objects that access and/or manipulate - database data. + 1. As a general rule, keep the lifecycle of the session **separate and + external** from functions and objects that access and/or manipulate + database data. This will greatly help with achieving a predictable + and consistent transactional scope. + + 2. Make sure you have a clear notion of where transactions + begin and end, and keep transactions **short**, meaning, they end + at the series of a sequence of operations, instead of being held + open indefinitely. A :class:`.Session` is typically constructed at the beginning of a logical operation where database access is potentially anticipated. diff --git a/doc/build/orm/session_transaction.rst b/doc/build/orm/session_transaction.rst index 24a844650..bca3e944f 100644 --- a/doc/build/orm/session_transaction.rst +++ b/doc/build/orm/session_transaction.rst @@ -484,7 +484,9 @@ everything is rolled back. from sqlalchemy import event + class SomeTest(TestCase): + def setUp(self): # connect to the database self.connection = engine.connect() @@ -502,7 +504,12 @@ everything is rolled back. @event.listens_for(self.session, "after_transaction_end") def restart_savepoint(session, transaction): if transaction.nested and not transaction._parent.nested: - session.begin_nested() + # ensure that state is expired the way + # session.commit() at the top level normally does + # (optional step) + session.expire_all() + + session.begin_nested() # ... the tearDown() method stays the same diff --git a/doc/build/requirements.txt b/doc/build/requirements.txt index 3f87e68ea..d1eb23d0f 100644 --- a/doc/build/requirements.txt +++ b/doc/build/requirements.txt @@ -1,3 +1,3 @@ changelog>=0.3.4 sphinx-paramlinks>=0.2.2 -git+https://bitbucket.org/zzzeek/zzzeeksphinx.git +git+https://bitbucket.org/zzzeek/zzzeeksphinx.git@HEAD#egg=zzzeeksphinx |
