summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/sql/schema.py
Commit message (Collapse)AuthorAgeFilesLines
* Allow duplicate columns in from clauses and selectablesMike Bayer2019-07-111-25/+19
| | | | | | | | | | | | | | | | | | The :func:`.select` construct and related constructs now allow for duplication of column labels and columns themselves in the columns clause, mirroring exactly how column expressions were passed in. This allows the tuples returned by an executed result to match what was SELECTed for in the first place, which is how the ORM :class:`.Query` works, so this establishes better cross-compatibility between the two constructs. Additionally, it allows column-positioning-sensitive structures such as UNIONs (i.e. :class:`.CompoundSelect`) to be more intuitively constructed in those cases where a particular column might appear in more than one place. To support this change, the :class:`.ColumnCollection` has been revised to support duplicate columns as well as to allow integer index access. Fixes: #4753 Change-Id: Ie09a8116f05c367995c1e43623c51e07971d3bf0
* Clear proxy_set cache when creating an annotated columnMike Bayer2019-07-021-1/+1
| | | | | | | | | | | | Fixed an unlikely issue where the "corresponding column" routine for unions and other :class:`.CompoundSelect` objects could return the wrong column in some overlapping column situtations, thus potentially impacting some ORM operations when set operations are in use, if the underlying :func:`.select` constructs were used previously in other similar kinds of routines, due to a cached value not being cleared. Fixes: #4747 Change-Id: I7fb134cac3604f8fe62e220fb24a0945d0a1c56f
* Implement new ClauseElement role and coercion systemMike Bayer2019-05-181-27/+18
| | | | | | | | | | | | | | | | | | | | A major refactoring of all the functions handle all detection of Core argument types as well as perform coercions into a new class hierarchy based on "roles", each of which identify a syntactical location within a SQL statement. In contrast to the ClauseElement hierarchy that identifies "what" each object is syntactically, the SQLRole hierarchy identifies the "where does it go" of each object syntactically. From this we define a consistent type checking and coercion system that establishes well defined behviors. This is a breakout of the patch that is reorganizing select() constructs to no longer be in the FromClause hierarchy. Also includes a rename of as_scalar() into scalar_subquery(); deprecates automatic coercion to scalar_subquery(). Partially-fixes: #4617 Change-Id: I26f1e78898693c6b99ef7ea2f4e7dfd0e8e1a1bd
* - repair param inject to work around code examples to some degreeMike Bayer2019-03-051-5/+5
| | | | | | - link to correct text() construct for CheckConstraint Change-Id: Ic46d1f79670a532f33bd9feb7f568dc3d1d6e38a
* Add resolve_fks=False option for reflectionMike Bayer2019-02-281-1/+52
| | | | | | | | | | | | | | Added new parameters :paramref:`.Table.resolve_fks` and :paramref:`.MetaData.reflect.resolve_fks` which when set to False will disable the automatic reflection of related tables encountered in :class:`.ForeignKey` objects, which can both reduce SQL overhead for omitted tables as well as avoid tables that can't be reflected for database-specific reasons. Two :class:`.Table` objects present in the same :class:`.MetaData` collection can still refer to each other even if the reflection of the two tables occurred separately. Fixes: #4517 Change-Id: I623baed42042a16c5109e4c8af6b2f64d2d00f95
* Remove all remaining text() coercions and ensure identifiers are safeMike Bayer2019-02-061-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fully removed the behavior of strings passed directly as components of a :func:`.select` or :class:`.Query` object being coerced to :func:`.text` constructs automatically; the warning that has been emitted is now an ArgumentError or in the case of order_by() / group_by() a CompileError. This has emitted a warning since version 1.0 however its presence continues to create concerns for the potential of mis-use of this behavior. Note that public CVEs have been posted for order_by() / group_by() which are resolved by this commit: CVE-2019-7164 CVE-2019-7548 Added "SQL phrase validation" to key DDL phrases that are accepted as plain strings, including :paramref:`.ForeignKeyConstraint.on_delete`, :paramref:`.ForeignKeyConstraint.on_update`, :paramref:`.ExcludeConstraint.using`, :paramref:`.ForeignKeyConstraint.initially`, for areas where a series of SQL keywords only are expected.Any non-space characters that suggest the phrase would need to be quoted will raise a :class:`.CompileError`. This change is related to the series of changes committed as part of :ticket:`4481`. Fixed issue where using an uppercase name for an index type (e.g. GIST, BTREE, etc. ) or an EXCLUDE constraint would treat it as an identifier to be quoted, rather than rendering it as is. The new behavior converts these types to lowercase and ensures they contain only valid SQL characters. Quoting is applied to :class:`.Function` names, those which are usually but not necessarily generated from the :attr:`.sql.func` construct, at compile time if they contain illegal characters, such as spaces or punctuation. The names are as before treated as case insensitive however, meaning if the names contain uppercase or mixed case characters, that alone does not trigger quoting. The case insensitivity is currently maintained for backwards compatibility. Fixes: #4481 Fixes: #4473 Fixes: #4467 Change-Id: Ib22a27d62930e24702e2f0f7c74a0473385a08eb
* Fix some long lines that sneaked in with the spelling fixes changeMike Bayer2019-01-251-2/+2
| | | | Change-Id: Icc6d8e564c9d31da2adc39d060e077c3d4811186
* Fix many spell glitchesLele Gaifax2019-01-251-4/+4
| | | | | | | | | | | | This affects mostly docstrings, except in orm/events.py::dispose_collection() where one parameter gets renamed: given that the method is empty, it seemed reasonable to me to fix that too. Closes: #4440 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4440 Pull-request-sha: 779ed75acb6142e1f1daac467b5b14134529bb4b Change-Id: Ic0553fe97853054b09c2453af76d96363de6eb0e
* Add deprecation warnings to all deprecated APIsMike Bayer2019-01-231-34/+44
| | | | | | | | | | | | | | | A large change throughout the library has ensured that all objects, parameters, and behaviors which have been noted as deprecated or legacy now emit ``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now defaults to displaying deprecation warnings, as well as that modern test suites based on tools like tox and pytest tend to display deprecation warnings, this change should make it easier to note what API features are obsolete. See the notes added to the changelog and migration notes for further details. Fixes: #4393 Change-Id: If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b
* Remove version directives for 0.6, 0.7, 0.8Mike Bayer2019-01-151-16/+0
| | | | | | | | | - fix a few "seealso"s - ComparableProprerty's "superseded in 0.7" becomes deprecated in 0.7 Backport to currently maintained doc versions 1.2, 1.1 Change-Id: Ib1fcb2df8673dbe5c4ffc47f3896a60d1dfcb4b2
* Merge "use ..deprecated directive w/ version in all cases"mike bayer2019-01-121-7/+22
|\
| * use ..deprecated directive w/ version in all casesMike Bayer2019-01-111-7/+22
| | | | | | | | | | | | | | | | | | These changes should be ported from 1.3 back to 1.0 or possibly 0.9 to the extent they are relevant in each version. In 1.3 we hope to turn all deprecation documentation into warnings. Change-Id: I205186cde161af9389af513a425c62ce90dd54d8
* | happy new yearMike Bayer2019-01-111-1/+1
|/ | | | Change-Id: I6a71f4924d046cf306961c58dffccf21e9c03911
* Post black reformattingMike Bayer2019-01-061-30/+34
| | | | | | | | | | | | | Applied on top of a pure run of black -l 79 in I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes resolves all remaining flake8 conditions for those codes we have enabled in setup.cfg. Included are resolutions for all remaining flake8 issues including shadowed builtins, long lines, import order, unused imports, duplicate imports, and docstring issues. Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
* Run black -l 79 against all source filesMike Bayer2019-01-061-465/+664
| | | | | | | | | | | | | | This is a straight reformat run using black as is, with no edits applied at all. The black run will format code consistently, however in some cases that are prevalent in SQLAlchemy code it produces too-long lines. The too-long lines will be resolved in the following commit that will resolve all remaining flake8 issues including shadowed builtins, long lines, import order, unused imports, duplicate imports, and docstring issues. Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
* typoRaphael2018-11-291-1/+1
| | | typo
* Merge "Add new "all columns" naming convention tokens"mike bayer2018-11-111-10/+27
|\
| * Add new "all columns" naming convention tokensMike Bayer2018-11-101-10/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added new naming convention tokens ``column_0N_name``, ``column_0_N_name``, etc., which will render the names / keys / labels for all columns referenced by a particular constraint in a sequence. In order to accommodate for the length of such a naming convention, the SQL compiler's auto-truncation feature now applies itself to constraint names as well, which creates a shortened, deterministically generated name for the constraint that will apply to a target backend without going over the character limit of that backend. Additional notes: 1. the SQLite dialect had a format_index method that was apparently not used, removed. 2. the naming convention logic has been applying the foreign key remote column spec to the naming convention, and not the actual column name. In the case where the referenced Table object uses .key inside the columns and these are what ForeignKey() references, the naming convention was doing the wrong thing. The patch here fixes this, however this isn't noted in the migration notes. Fixes: #3989 Change-Id: Ib24f4754b886676096c480fc54b2e5c2463ac99a
* | Implement SQLite ON CONFLICT for constraintsDenis Kataev2018-11-071-6/+7
|/ | | | | | | | | | | Implemented the SQLite ``ON CONFLICT`` clause as understood at the DDL level, e.g. for primary key, unique, and CHECK constraints as well as specified on a :class:`.Column` to satisfy inline primary key and NOT NULL. Pull request courtesy Denis Kataev. Fixes: #4360 Change-Id: I4cd4bafa8fca41e3101c87dbbfe169741bbda3f4 Pull-request: https://github.com/zzzeek/sqlalchemy/pull/431
* Lookup index columns in parent table by key for copyMike Bayer2018-06-141-1/+2
| | | | | | | | | | | | | Fixed regression in 1.2 due to :ticket:`4147` where a :class:`.Table` that has had some of its indexed columns redefined with new ones, as would occur when overriding columns during reflection or when using :paramref:`.Table.extend_existing`, such that the :meth:`.Table.tometadata` method would fail when attempting to copy those indexes as they still referred to the replaced column. The copy logic now accommodates for this condition. Change-Id: I521aa2c9f3baa0e84598bbdd6ffe4bf07b6e3ba8 Fixes: #4279
* happy new yearMike Bayer2018-01-121-1/+1
| | | | Change-Id: I3ef36bfd0cb0ba62b3123c8cf92370a43156cf8f
* Fully copy index expressionsMike Bayer2017-12-141-16/+28
| | | | | | | | | | | | | | | | | Fixed bug where the :meth:`.Table.tometadata` method would not properly accommodate :class:`.Index` objects that didn't consist of simple column expressions, such as indexes against a :func:`.text` construct, indexes that used SQL expressions or :attr:`.func`, etc. The routine now copies expressions fully to a new :class:`.Index` object while substituting all table-bound :class:`.Column` objects for those of the target table. Also refined the means by which tometadata() checks if an Index or UniqueConstraint is generated by a column-level flag, by propagating an attribute "_column_flag=True" to such indexes/constraints. Change-Id: I7ef1b8ea42f9933357ae35f241a5ba9838bac35b Fixes: #4147
* Accommodate tuples for ColumnDefault.__repr__Nicolas CANIART2017-10-311-1/+1
| | | | | | | | | Fixed bug where ``__repr__`` of :class:`.ColumnDefault` would fail if the argument were a tuple. Pull request courtesy Nicolas Caniart. Change-Id: I08aa2448ef91054c43d6068ac54cedbdf7a83d64 Pull-request: https://bitbucket.org/zzzeek/sqlalchemy/pull-requests/1 Fixes: #4126
* Add table / column comments into tometadata()Mike Bayer2017-09-231-0/+2
| | | | | | | | Fixed bug in new SQL comments feature where table and column comment would not be copied when using :meth:`.Table.tometadata`. Change-Id: Ib3112e5e02930245daacb36c8ed38c01fa3e7dbd Fixes: #4087
* Ensure Oracle index w/ col DESC etc. is reflectedMike Bayer2017-08-141-1/+10
| | | | | | | | | | | | | | | | | | | | Fixed bug where an index reflected under Oracle with an expression like "column DESC" would not be returned, if the table also had no primary key, as a result of logic that attempts to filter out the index implicitly added by Oracle onto the primary key columns. Reworked the "filter out the primary key index" logic in oracle get_indexes() to be clearer. This changeset also adds an internal check to ColumnCollection to accomodate for the case of a column being added twice, as well as adding a private _table argument to Index such that reflection can specify the Table explicitly. The _table argument can become part of public API in a later revision or release if needed. Change-Id: I745711e03b3e450b7f31185fc70e10d3823063fa Fixes: #4042
* Add support for CACHE and ORDER to sequencesDavid Moore2017-07-051-2/+17
| | | | | | | | | | | Added new keywords :paramref:`.Sequence.cache` and :paramref:`.Sequence.order` to :class:`.Sequence`, to allow rendering of the CACHE parameter understood by Oracle and PostgreSQL, and the ORDER parameter understood by Oracle. Pull request courtesy David Moore. Change-Id: I082c3f8ef56ef89dbaad5da9d5695be5313b0614 Pull-request: https://bitbucket.org/zzzeek/sqlalchemy/pull-requests/96
* Don't erase reflected comment in _init_existingEloy Felix2017-06-201-2/+3
| | | | | Change-Id: Ie0b78c79367933486528ca0ba686d4a9f16922b1 Pull-request: https://github.com/zzzeek/sqlalchemy/pull/370
* Raise if ForeignKeyConstraint created with different numbers ofMike Bayer2017-05-251-0/+16
| | | | | | | | | | | | | | | | | | local and remote columns. An :class:`.ArgumentError` is now raised if a :class:`.ForeignKeyConstraint` object is created with a mismatched number of "local" and "remote" columns, which otherwise causes the internal state of the constraint to be incorrect. Note that this also impacts the condition where a dialect's reflection process produces a mismatched set of columns for a foreign key constraint. Downstream DB2 dialect has been reported as potentially causing this scenario. Change-Id: Id51c34a6c43749bb582639f9c1dc28723482f0e5 Fixes: #3949 References: #3998
* Allow metadata.reflect() to recover from unreflectable tablesMike Bayer2017-05-221-1/+4
| | | | | | | | | | | | | Added support for views that are unreflectable due to stale table definitions, when calling :meth:`.MetaData.reflect`; a warning is emitted for the table that cannot respond to ``DESCRIBE`` but the operation succeeds. The MySQL dialect now raises UnreflectableTableError which is in turn caught by MetaData.reflect(). Reflecting the view standalone raises this error directly. Change-Id: Id8005219d8e073c154cc84a873df911b4a6cf4d6 Fixes: #3871
* - big rewrite of the Sequence documentation:Mike Bayer2017-05-051-20/+28
| | | | | | | | | | | | | | 1. Sequence should be associated with MetaData always, except in the really weird case someone is sharing a Sequence among multiple metadatas. Make this a "best practice", end the confusion of #3951, #3979 2. "optional" is not a thing people use, trim this way down 3. remove confusing language Change-Id: Iab5aec319da2582092fe2615ee50430f76441aff (cherry picked from commit a1dcf23e371695d8d3be2e1d7875fad10f16a656)
* - add another note re: 339e2c13b0fc8e95a47d00c0f8fc5afc4b6dff9aMike Bayer2017-05-051-0/+12
| | | | | | | | which clarifies that ForeignKey circumvents this logic as a "convenience". issue #3978 is updated to address trying to make this consistent. Change-Id: I089acaa89f11b7a6310c2bf32916e26eb62ab9c0
* - add a note to MetaData.schema indicating that the TableMike Bayer2017-05-051-0/+13
| | | | | | | will be cataloged in the .tables collection with its fully qualified name. Fixes #3978 Change-Id: I65fa063918efc22658e93c39a0680cb83553dec8
* Apply type processing to untyped preexec default clauseMike Bayer2017-03-301-0/+8
| | | | | | | | | | | | | | | | Fixed bug where a SQL-oriented Python-side column default could fail to be executed properly upon INSERT in the "pre-execute" codepath, if the SQL itself were an untyped expression, such as plain text. The "pre- execute" codepath is fairly uncommon however can apply to non-integer primary key columns with SQL defaults when RETURNING is not used. Tests exist here to ensure typing is applied to a typed expression for default, but in the case of an untyped SQL value, we know the type from the column, so apply this. Change-Id: I5d8b391611c137b9f700115a50a2bf5b30abfe94 Fixes: #3923
* Merge branch 'master' of https://bitbucket.org/jalessio/sqlalchemyMike Bayer2017-03-181-2/+2
|\
| * Fix typos ('expicit' -> 'explicit')Jamie Alessio2017-03-141-2/+2
| |
* | New features from python 2.7Катаев Денис2017-03-171-1/+1
| | | | | | | | | | | | | | After bump minimum supported version to 2.7 (1da9d3752160430c91534a8868ceb8c5ad1451d4), we can use new syntax. Change-Id: Ib064c75a00562e641d132f9c57e5e69744200e05 Pull-request: https://github.com/zzzeek/sqlalchemy/pull/347
* | Implement comments for tables, columnsFrazer McLean2017-03-171-2/+22
|/ | | | | | | | | | | | | | Added support for SQL comments on :class:`.Table` and :class:`.Column` objects, via the new :paramref:`.Table.comment` and :paramref:`.Column.comment` arguments. The comments are included as part of DDL on table creation, either inline or via an appropriate ALTER statement, and are also reflected back within table reflection, as well as via the :class:`.Inspector`. Supported backends currently include MySQL, Postgresql, and Oracle. Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Fixes: #1546 Change-Id: Ib90683850805a2b4ee198e420dc294f32f15d35d
* - document ForeignKeyConstraint columns / elements, fixes #2904Mike Bayer2017-03-071-0/+22
| | | | Change-Id: Ia50a3100d1bd88020c30224932b16aeadd7f4c75
* Only use schema_translate_map on SchemaItem subclassesMike Bayer2017-02-281-1/+5
| | | | | | | | | | | | | Fixed bug in new "schema translate" feature where the translated schema name would be invoked in terms of an alias name when rendered along with a column expression; occurred only when the source translate name was "None". The "schema translate" feature now only takes effect for :class:`.SchemaItem` and :class:`.SchemaType` subclasses, that is, objects that correspond to a DDL-creatable structure in a database. Change-Id: Ie8cb35aeaba2c67efec8c8c57c219e4dd346e44a Fixes: #3924
* Better hide engine passwordValery Yundin2017-01-161-2/+2
| | | | | | | | | | Avoid putting engine password in the exception message in `MetaData.reflect` (since exception messages often appear in logs). Use the same redacted `__repr__` implementation in `TLEngine` as in its base class `Engine` Change-Id: Ic0a7baea917a9c8d87dffdd82ef566673ab08e02 Pull-request: https://github.com/zzzeek/sqlalchemy/pull/327
* Support python3.6Mike Bayer2017-01-131-9/+9
| | | | | | | | | | | Corrects some warnings and adds tox config. Adds DeprecationWarning to the error category. Large sweep for string literals w/ backslashes as this is common in docstrings Co-authored-by: Andrii Soldatenko Fixes: #3886 Change-Id: Ia7c838dfbbe70b262622ed0803d581edc736e085 Pull-request: https://github.com/zzzeek/sqlalchemy/pull/337
* update for 2017 copyrightMike Bayer2017-01-041-1/+1
| | | | Change-Id: I4e8c2aa8fe817bb2af8707410fa0201f938781de
* Add _extend_on deduplicating set for metadata.reflect()Mike Bayer2016-11-231-6/+18
| | | | | | | | | | | | | The "extend_existing" option of :class:`.Table` reflection would cause indexes and constraints to be doubled up in the case that the parameter were used with :meth:`.MetaData.reflect` (as the automap extension does) due to tables being reflected both within the foreign key path as well as directly. A new de-duplicating set is passed through within the :meth:`.MetaData.reflect` sequence to prevent double reflection in this way. Change-Id: Ibf6650c1e76a44ccbe15765fd79df2fa53d6bac7 Fixes: #3861
* Update the Column.nullable docstring for the primary_key=True caseMatt Riedemann2016-10-101-4/+8
| | | | | | | | | | | | | | | | | While reviewing a change that created a new table, the primary_key column value was set to True but nullable was not explicitly set, which led to some confusion over the default behavior for the nullable column value when setting a primary_key. Looking at the docs it's not clear, but the code shows that if nullable is not specified, then nullable = not primary_key, so nullable defaults to False when primary_key is True. This patch adds a simple clarification to the docs so people don't have to check the code. Change-Id: I8553339d56fbae11370c7c6af6d8d4723163be1c Pull-request: https://github.com/zzzeek/sqlalchemy/pull/312
* spelling: Postgresql -> PostgreSQLVille Skyttä2016-10-081-3/+3
|
* - some documentation hitsMike Bayer2016-10-011-5/+15
| | | | Change-Id: I468fe70168804b08cf0d5e8b57e235bca904b1ff
* Check for supports_execution at ClauseElement baseMike Bayer2016-08-311-3/+3
| | | | | | | | | | Raise a more descriptive exception / message when ClauseElement or non-SQLAlchemy objects that are not "executable" are erroneously passed to ``.execute()``; a new exception ObjectNotExecutableError is raised consistently in all cases. Change-Id: I2dd393121e2c7e5b6b9e40286a2f25670876e8e4 Fixes: #3786
* Spelling fixespr/299Ville Skyttä2016-08-081-1/+1
|
* Ensure post-__clause_element__() expression are used in IndexMike Bayer2016-07-311-1/+3
| | | | | | | | | | | The change in Index for 1.1 combined with the fix for ref #3763 still fails to deliver the correct object resolved by __clause_element__() to the list of expressions for compilation. Make sure we use the expression that's been unwrapped from __clause_element__(). Change-Id: Ie1df8db5090de665048331786f0024d52851923f Fixes: #3763
* Index should extract __clause_element__() earlyMike Bayer2016-07-301-1/+4
| | | | | | | | | | | | Fixed bug where :class:`.Index` would fail to extract columns from compound SQL expressions if those SQL expressions were wrapped inside of an ORM-style ``__clause_element__()`` construct. This bug exists in 1.0.x as well, however in 1.1 is more noticeable as hybrid_property @expression now returns a wrapped element. Fixes: #3763 Change-Id: I992536386503a1fb3f2305790abe008d72c44c4a
View Lorry Controller Status