summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy
Commit message (Collapse)AuthorAgeFilesLines
* pep-484 for sqlalchemy.event; use future annotationsMike Bayer2022-02-15125-290/+937
| | | | | | | | | | __future__.annotations mode allows us to use non-string annotations for argument and return types in most cases, but more importantly it removes a large amount of runtime overhead that would be spent in evaluating the annotations. Change-Id: I2f5b6126fe0019713fc50001be3627b664019ede References: #6810
* add note reiterating do_orm_execute() is for ORM statements onlyMike Bayer2022-02-151-10/+26
| | | | | Fixes: #7707 Change-Id: I7216f63d9f4269059438e41ad8dd3dffc25a5d03
* remove **kw from AsyncSession.begin() / AsyncSession.begin_nested()Mike Bayer2022-02-141-2/+2
| | | | | | | | | | Removed the unused ``**kw`` arguments from :class:`_asyncio.AsyncSession.begin` and :class:`_asyncio.AsyncSession.begin_nested`. These kw aren't used and appear to have been added to the API in error. Fixes: #7703 Change-Id: I39ff3850929d83e6efeb7f284f2f4d5e4ca120b1
* Merge "establish mypy / typing approach for v2.0" into mainmike bayer2022-02-1360-1883/+3706
|\
| * establish mypy / typing approach for v2.0Mike Bayer2022-02-1360-1883/+3706
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | large patch to get ORM / typing efforts started. this is to support adding new test cases to mypy, support dropping sqlalchemy2-stubs entirely from the test suite, validate major ORM typing reorganization to eliminate the need for the mypy plugin. * New declarative approach which uses annotation introspection, fixes: #7535 * Mapped[] is now at the base of all ORM constructs that find themselves in classes, to support direct typing without plugins * Mypy plugin updated for new typing structures * Mypy test suite broken out into "plugin" tests vs. "plain" tests, and enhanced to better support test structures where we assert that various objects are introspected by the type checker as we expect. as we go forward with typing, we will add new use cases to "plain" where we can assert that types are introspected as we expect. * For typing support, users will be much more exposed to the class names of things. Add these all to "sqlalchemy" import space. * Column(ForeignKey()) no longer needs to be `@declared_attr` if the FK refers to a remote table * composite() attributes mapped to a dataclass no longer need to implement a `__composite_values__()` method * with_variant() accepts multiple dialect names Change-Id: I22797c0be73a8fbbd2d6f5e0c0b7258b17fe145d Fixes: #7535 Fixes: #7551 References: #6810
* | Raise and re-catch NameError from _ModNSMike Bayer2022-02-121-1/+1
| | | | | | | | | | | | | | | | | | | | Fixed issue where using a fully qualified path for the classname in :func:`_orm.relationship` that nonetheless contained an incorrect name for path tokens that were not the first token, would fail to raise an informative error and would instead fail randomly at a later step. Fixes: #7697 Change-Id: I5e1a3aa4c2a6ea5b123be14666f589aec43f4b60
* | Merge "use QueuePool for sqlite file databases" into mainmike bayer2022-02-111-39/+48
|\ \ | |/ |/|
| * use QueuePool for sqlite file databasesMike Bayer2022-02-101-39/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | The SQLite dialect now defaults to :class:`_pool.QueuePool` when a file based database is used. This is set along with setting the ``check_same_thread`` parameter to ``False``. It has been observed that the previous approach of defaulting to :class:`_pool.NullPool`, which does not hold onto database connections after they are released, did in fact have a measurable negative performance impact. As always, the pool class is always customizable via the :paramref:`_sa.create_engine.poolclass` parameter. Fixes: #7490 Change-Id: I5f6c259def0ef43d401c6163dc99f651e519148d
* | Merge "update zimports" into mainmike bayer2022-02-104-27/+22
|\ \ | |/ |/|
| * update zimportsMike Bayer2022-02-094-27/+22
| | | | | | | | | | | | | | | | includes new fix for formatting like black does. also runs black on a few outliers. Change-Id: I67446660a6bc10b73eb710389ae6d3f122af9302
* | doc fixesMike Bayer2022-02-092-13/+36
|/ | | | | | | | | | | | | | | | | | * clarify merge behavior for non-present attributes, references #7687 * fix AsyncSession in async_scoped_session documentation, name the scoped session AsyncScopedSession, fixes: #7671 * Use non-deprecated execute() style in sqltypes JSON examples, fixes: #7633 * Add note regarding mitigation for https://github.com/MagicStack/asyncpg/issues/727, fixes #7245 Fixes: #7671 Fixes: #7633 Fixes: #7245 Change-Id: Ic40b4378ca321367a912864f4eddfdd9714fe217
* Accommodate escaped_bind_names for defaults/insert paramsMike Bayer2022-02-082-7/+28
| | | | | | | | | | Fixed issue in Oracle dialect where using a column name that requires quoting when written as a bound parameter, such as ``"_id"``, would not correctly track a Python generated default value due to the bound-parameter rewriting missing this value, causing an Oracle error to be raised. Fixes: #7676 Change-Id: I5a54426d24f2f9b336e3597d5595fb3e031aad97
* Merge "add slotscheck to CI" into mainmike bayer2022-02-072-4/+10
|\
| * add slotscheck to CIArie Bovenberg2022-02-072-4/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As discussed in #7589, `slotscheck` can prevent slots-related mistakes from creeping back in. Plan for now is to have slotscheck part of the "lint" tests (renamed from pep8) that will run for CI and github actions. To support slotscheck's runtime nature, slotscheck is run twice, first with cython exts enabled and then with them disabled via new environment variable. Also added sqlalchemy[mypy] dependency to support slots checking the mypy plugin. Found and fixed one more `__slots__` issue by disabling C exts. Closes: #7670 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7670 Pull-request-sha: 3e77fe5449615a3c7c61ce9a1e4e79cd6636a89a Change-Id: I90cdd284cdcee316a38856ba94d72ffc98947c5a
* | apply literal value resolution to StringMike Bayer2022-02-071-3/+16
|/ | | | | | | | | | | | | | | | | | | | | | | | | | Python string values for which a SQL type is determined from the type of the value, mainly when using :func:`_sql.literal`, will now apply the :class:`_types.String` type, rather than the :class:`_types.Unicode` datatype, for Python string values that test as "ascii only" using Python ``str.isascii()``. If the string is not ``isascii()``, the :class:`_types.Unicode` datatype will be bound instead, which was used in all string detection previously. This behavior **only applies to in-place detection of datatypes when using ``literal()`` or other contexts that have no existing datatype**, which is not usually the case under normal :class:`_schema.Column` comparison operations, where the type of the :class:`_schema.Column` being compared always takes precedence. Use of the :class:`_types.Unicode` datatype can determine literal string formatting on backends such as SQL Server, where a literal value (i.e. using ``literal_binds``) will be rendered as ``N'<value>'`` instead of ``'value'``. For normal bound value handling, the :class:`_types.Unicode` datatype also may have implications for passing values to the DBAPI, again in the case of SQL Server, the pyodbc driver supports the use of :ref:`setinputsizes mode <mssql_pyodbc_setinputsizes>` which will handle :class:`_types.String` versus :class:`_types.Unicode` differently. Fixes: #7551 Change-Id: I4f8de63e36532ae8ce4c630ee59211349ce95361
* Merge "ensure exception raised for all stream w/ sync result" into mainmike bayer2022-02-045-17/+30
|\
| * ensure exception raised for all stream w/ sync resultMike Bayer2022-02-045-17/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed issue where the :meth:`_asyncio.AsyncSession.execute` method failed to raise an informative exception if the ``stream_results`` execution option were used, which is incompatible with a sync-style :class:`_result.Result` object. An exception is now raised in this scenario in the same way one is already raised when using ``stream_results`` in conjunction with the :meth:`_asyncio.AsyncConnection.execute` method. Additionally, for improved stability with state-sensitive dialects such as asyncmy, the cursor is now closed when this error condition is raised; previously with the asyncmy dialect, the connection would go into an invalid state with unconsumed server side results remaining. Fixes: #7667 Change-Id: I6eb7affe08584889b57423a90258295f8b7085dc
* | Merge "fall back to SHOW VARIABLES for MySQL < 5.6" into mainmike bayer2022-02-042-21/+36
|\ \
| * | fall back to SHOW VARIABLES for MySQL < 5.6Mike Bayer2022-02-042-21/+36
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed regression caused by :ticket:`7518` where changing the syntax "SHOW VARIABLES" to "SELECT @@" broke compatibility with MySQL versions older than 5.6, including early 5.0 releases. While these are very old MySQL versions, a change in compatibility was not planned, so version-specific logic has been restored to fall back to "SHOW VARIABLES" for MySQL server versions < 5.6. includes unrelated orm/test_expire ordering issue , only showing up on 1.4 / py2.7 but seems to be passing by luck otherwise Fixes: #7518 Change-Id: Ia554080af742f2c3437f88cf3f7a4827b5e55da8
* | Fix overlapping slots, base classes without slotsArie Bovenberg2022-02-0112-10/+24
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some `__slots__` were not in order. Fixes #7527 ### Description I'm fixing two types of slots mistakes: - [x] remove overlapping slots (i.e. slots already defined on a base class) - [x] fix broken inheritance (i.e. slots class inheriting from a non-slots class) - [x] slots added to base class `TransactionalContext`. It seemed to use two attributes, which I've added as slots. - [x] empty slots removed from `ORMOption`. Its base class explicitly makes use of `__dict__` so empty slots don't add anything. - [x] empty slots added to `PostLoader`. It doesn't appear to use any slots not already defined on its base classes. - [x] empty slots added to `IterateMappersMixin`. It doesn't appear to use any slots not already defined on its subclasses. - [x] empty slots added to `ImmutableContainer`. It doesn't use any fields. - [x] empty slots added to `OperatorType`. It's a protocol. - [x] empty slots added to `InternalTraversal`, `_HasTraversalDispatch`. They don't seem to use attributes on their own. ### Checklist This pull request is: - [x] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. **Have a nice day!** Closes: #7589 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7589 Pull-request-sha: 70a9c4d46916b7c6907eb1d3ad4f7033ec964191 Change-Id: I6c6e3e69c3c34d0f3bdda7f0684849834fdd1863
* add pg14 to supported implementationsMike Bayer2022-01-311-1/+1
| | | | | Fixes: #7647 Change-Id: I071f1a53714ebb0dc838fddc665640d46666318f
* Fix up Python logging metadataMarkus Gerstel2022-01-272-0/+10
| | | | | | | | | | | | | | | | | Adjusted the logging for key SQLAlchemy components including :class:`_engine.Engine`, :class:`_engine.Connection` to establish an appropriate stack level parameter, so that the Python logging tokens ``funcName`` and ``lineno`` when used in custom logging formatters will report the correct information, which can be useful when filtering log output; supported on Python 3.8 and above. Pull request courtesy Markus Gerstel. Fixes: #7612 Closes: #7615 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7615 Pull-request-sha: cf9567beb06680df320cb12dde1f15baa68e1eb5 Change-Id: Iff23c92ef3453ac93cbd0d190e7efbf8ea4457a2
* Merge "Add compiler support for PostgreSQL "NOT VALID" constraints." into mainmike bayer2022-01-271-2/+74
|\
| * Add compiler support for PostgreSQL "NOT VALID" constraints.Gilbert Gilb's2022-01-261-2/+74
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Added compiler support for the PostgreSQL ``NOT VALID`` phrase when rendering DDL for the :class:`.CheckConstraint`, :class:`.ForeignKeyConstraint` and :class:`.ForeignKey` schema constructs. Pull request courtesy Gilbert Gilb's. Fixes: #7600 Closes: #7601 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7601 Pull-request-sha: 78eecd55fd9fad07030d963f5fd6713c4af60e80 Change-Id: I84bfe84596856eeea2bcca45c04ad23d980a75ec
* | fix typing syntax in python < 3.9Federico Caselli2022-01-261-2/+2
| | | | | | | | Change-Id: If4bca7299a333ff585de6e486ba84276cc75ab6d
* | Merge "restore set-as-superclass for OrderedSet" into mainmike bayer2022-01-252-104/+77
|\ \
| * | restore set-as-superclass for OrderedSetMike Bayer2022-01-252-104/+77
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | OrderedSet again subclasses set, spent some time with the stubs at https://github.com/python/typeshed/blob/master/stdlib/builtins.pyi#L887 to more deeply understand what they are doing here so that we can type check fully. Change-Id: Iec9b5ab43befd30e1f2c5cc40e59ab852dd28e75
* | | join to existing mark expr with "and"Mike Bayer2022-01-251-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ca48f461b2dcac2970829e4e0 considered an existing mark expression plus legacy tags to be an error condition; however these can be joined by "and" and will in our use case do the right thing. The github action scripts make use of legacy tags. We can change that also but I want to just make sure this combination works fully as well. Change-Id: Ifc506de3dd961c01d68d594ec2f5b2c9a0bbad31
* | | Merge "replace test tags with pytest.mark" into mainmike bayer2022-01-256-109/+103
|\ \ \
| * | | replace test tags with pytest.markMike Bayer2022-01-256-109/+103
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | replaced the __tags__ class attribute and the --exclude-tags / --include-tags test runner options with regular pytest.mark names so that we can take advantage of mark expressions. options --nomemory, --notimingintensive, --backend-only, --exclude-tags, --include-tags remain as legacy but make use of pytest mark for implemementation. Added a "mypy" mark for the section of tests that are doing mypy integration tests. The __backend__ and __sparse_backend__ class attributes also use pytest marks for their implementation, which also allows the marks "backend" and "sparse_backend" to be used explicitly. Also removed the no longer used "--cdecimal" option as this was python 2 specific. in theory, the usage of pytest marks could expand such that the whole exclusions system would be based on it, but this does not seem to have any advantage at the moment. Change-Id: Ideeb57d9d49f0efc7fc0b6b923b31207ab783025
* | | replace "e.g.::" with more context regarding attribute events exampleMike Bayer2022-01-251-2/+4
|/ / | | | | | | | | Fixes: #7613 Change-Id: I28a9577587399c41afd53fbd026003667654c3fc
* | mypy: sqlalchemy.utilMike Bayer2022-01-2427-449/+961
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Starting to set up practices and conventions to get the library typed. Key goals for typing are: 1. whole library can pass mypy without any strict turned on. 2. we can incrementally turn on some strict flags on a per-package/ module basis, as here we turn on more strictness for sqlalchemy.util, exc, and log 3. mypy ORM plugin tests work fully without sqlalchemy2-stubs installed 4. public facing methods all have return types, major parameter signatures filled in also 5. Foundational elements like util etc. are typed enough so that we can use them in fully typed internals higher up the stack. Conventions set up here: 1. we can use lots of config in setup.cfg to limit where mypy is throwing errors and how detailed it should be in different packages / modules. We can use this to push up gerrits that will pass tests fully without everything being typed. 2. a new tox target pep484 is added. this links to a new jenkins pep484 job that works across all projects (alembic, dogpile, etc.) We've worked around some mypy bugs that will likely be around for awhile, and also set up some core practices for how to deal with certain things such as public_factory modules (mypy won't accept a module from a callable at all, so need to use simple type checking conditionals). References: #6810 Change-Id: I80be58029896a29fd9f491aa3215422a8b705e12
* | after all that, use pytest warnings pluginMike Bayer2022-01-233-37/+27
|/ | | | | | | | | | | | | | | | | The warnings plugin lets us set the filters up in the config, and as our filter requirements are now simple we can just set this up. additionally pytest now recommends pyproject.toml, since we fully include this now, let's move it there. the pytest logging plugin seems to not be any problem either at the moment, so re-enable that. if it becomes apparent whatever the problem was (which was probably that it was just surprising, or something) we can disable it again and comment what the reason was. Change-Id: Ia9715533b01f72aa5fdcf6a27ce75b76f829fa43
* Merge "dont test squelched warnings against the filter" into mainmike bayer2022-01-231-0/+2
|\
| * dont test squelched warnings against the filterMike Bayer2022-01-231-0/+2
| | | | | | | | | | | | | | | | | | | | | | I spent days on Ibcf09af25228d39ee5a943fda82d8a9302433726 reading it over and over again and noticed this slight inaccuracy 10 seconds after I merged it. the assert_warns_message() and assert_warns() functions should not consider a mismatched warning class as valid for a match. Change-Id: Ib8944dd95bcec1a7e4963917a5f4829e2ba27732
* | fix bullet indentation, clarify insert independent of DDLMike Bayer2022-01-231-8/+12
|/ | | | Change-Id: I3b6f2b7e23044148e837afdbe4fef66773e42777
* Merge "dont use exception catches for warnings; modernize xdist detection" ↵mike bayer2022-01-235-38/+78
|\ | | | | | | into main
| * dont use exception catches for warnings; modernize xdist detectionMike Bayer2022-01-225-38/+78
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Improvements to the test suite's integration with pytest such that the "warnings" plugin, if manually enabled, will not interfere with the test suite, such that third parties can enable the warnings plugin or make use of the ``-W`` parameter and SQLAlchemy's test suite will continue to pass. Additionally, modernized the detection of the "pytest-xdist" plugin so that plugins can be globally disabled using PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 without breaking the test suite if xdist were still installed. Warning filters that promote deprecation warnings to errors are now localized to SQLAlchemy-specific warnings, or within SQLAlchemy-specific sources for general Python deprecation warnings, so that non-SQLAlchemy deprecation warnings emitted from pytest plugins should also not impact the test suite. Fixes: #7599 Change-Id: Ibcf09af25228d39ee5a943fda82d8a9302433726
* | make the autoincrement doc a lot more clearMike Bayer2022-01-231-37/+112
| | | | | | | | | | | | | | | | | | I saw it saying "this parameter has no effect when Identity is in use", which is not accurate, so I've expanded this way out with bullets to make the current situation as clear as possible. Change-Id: I77001cc81cd90a6b867686e2975aed682e539347
* | Merge "Skip PK returned as None for RETURNING, server side default" into mainmike bayer2022-01-221-0/+25
|\ \
| * | Skip PK returned as None for RETURNING, server side defaultMike Bayer2022-01-211-0/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed regression where the ORM exception that is to be raised when an INSERT silently fails to actually insert a row (such as from a trigger) would not be reached, due to a runtime exception raised ahead of time due to the missing primary key value, thus raising an uninformative exception rather than the correct one. For 1.4 and above, a new ``FlushError`` is added for this case that's raised earlier than the previous "null identity" exception was for 1.3, as a situation where the number of rows actually INSERTed does not match what was expected is a more critical situation in 1.4 as it prevents batching of multiple objects from working correctly. This is separate from the case where a newly fetched primary key is fetched as NULL, which continues to raise the existing "null identity" exception. Fixes: #7594 Change-Id: Ie8e181e3472f09f389cca757c5e58e61b15c7d79
* | | Merge "Remove dispose warning on async engines when running tests" into mainmike bayer2022-01-221-1/+6
|\ \ \ | |_|/ |/| |
| * | Remove dispose warning on async engines when running testsFederico Caselli2022-01-211-1/+6
| | | | | | | | | | | | | | | Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Change-Id: Ia3357959ed286dc7d2ce264b5ddcadf309351ff7
* | | Add new infrastructure to support greater use of __slots__Mike Bayer2022-01-219-108/+464
| |/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Changed AliasedInsp to use __slots__ * Migrated all of strategy_options to use __slots__ for objects. Adds new infrastructure to traversals to support shallow copy, to dict and from dict based on internal traversal attributes. Load / _LoadElement then leverage this to provide clone / generative / getstate without the need for __dict__ or explicit attribute lists. Doing this change revealed that there are lots of things that trigger off of whether or not a class has a __visit_name__ attribute. so to suit that we've gone back to having Visitable, which is a better name than Traversible at this point (I think Traversible is mis-spelled too). Change-Id: I13d04e494339fac9dbda0b8e78153418abebaf72 References: #7527
* | Merge "Added support for ``FILESTREAM`` in MSSQL." into mainmike bayer2022-01-211-6/+31
|\ \
| * | Added support for ``FILESTREAM`` in MSSQL.Federico Caselli2022-01-201-6/+31
| | | | | | | | | | | | | | | Fixes: #7243 Change-Id: I99880f429dbaac525bdf7d44438aaab6bc8d0ca6
* | | Merge "re-enable tests for asyncmy; fix Binary" into mainmike bayer2022-01-213-6/+56
|\ \ \
| * | | re-enable tests for asyncmy; fix BinaryMike Bayer2022-01-203-6/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed regression in asyncmy dialect caused by :ticket:`7567` where removal of the PyMySQL dependency broke binary columns, due to the asyncmy dialect not being properly included within CI tests. Also repairs mariadbconnector isolation level for 2.0. basically tox config was failing to include additional drivers. Fixes: #7593 Change-Id: Iefc1061c24c75fcb9ca1a02d0b5e5f43970ade17
* | | | Merge "The parameter `as_uuid` of `UUID` now defaults to `True`." into mainmike bayer2022-01-202-3/+5
|\ \ \ \
| * | | | The parameter `as_uuid` of `UUID` now defaults to `True`.Federico Caselli2022-01-192-3/+5
| | |/ / | |/| | | | | | | | | | | | | | Fixes: #7225 Change-Id: Iddb78bf47ac733300bd12db50e16199cc22e9476
View Lorry Controller Status