summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/dialects/postgresql/psycopg2.py
Commit message (Collapse)AuthorAgeFilesLines
* Result initial introductionMike Bayer2020-02-211-6/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This builds on cc718cccc0bf8a01abdf4068c7ea4f3 which moved RowProxy to Row, allowing Row to be more like a named tuple. - KeyedTuple in ORM is replaced with Row - ResultSetMetaData broken out into "simple" and "cursor" versions for ORM and Core, as well as LegacyCursor version. - Row now has _mapping attribute that supplies full mapping behavior. Row and SimpleRow both have named tuple behavior otherwise. LegacyRow has some mapping features on the tuple which emit deprecation warnings (e.g. keys(), values(), etc). the biggest change for mapping->tuple is the behavior of __contains__ which moves from testing of "key in row" to "value in row". - ResultProxy breaks into ResultProxy and FutureResult (interim), the latter has the newer APIs. Made available to dialects using execution options. - internal reflection methods and most tests move off of implicit Row mapping behavior and move to row._mapping, result.mappings() method using future result - a new strategy system for cursor handling replaces the various subclasses of RowProxy - some execution context adjustments. We will leave EC in but refined things like get_result_proxy() and out parameter handling. Dialects for 1.4 will need to adjust from get_result_proxy() to get_result_cursor_strategy(), if they are using this method - out parameter handling now accommodated by get_out_parameter_values() EC method. Oracle changes for this. external dialect for DB2 for example will also need to adjust for this. - deprecate case_insensitive flag for engine / result, this feature is not used mapping-methods on Row are deprecated, and replaced with Row._mapping.<meth>, including: row.keys() -> use row._mapping.keys() row.items() -> use row._mapping.items() row.values() -> use row._mapping.values() key in row -> use key in row._mapping int in row -> use int < len(row) Fixes: #4710 Fixes: #4878 Change-Id: Ieb9085e9bcff564359095b754da9ae0af55679f0
* happy new yearMike Bayer2020-01-011-1/+1
| | | | Change-Id: I08440dc25e40ea1ccea1778f6ee9e28a00808235
* Use simple growth scale with any max size for BufferedRowResultProxysumau2019-10-301-1/+2
| | | | | | | | | | | | | | | | | The maximum buffer size for the :class:`.BufferedRowResultProxy`, which is used by dialects such as PostgreSQL when ``stream_results=True``, can now be set to a number greater than 1000 and the buffer will grow to that size. Previously, the buffer would not go beyond 1000 even if the value were set larger. The growth of the buffer is also now based on a simple multiplying factor currently set to 5. Pull request courtesy Soumaya Mauthoor. Fixes: #4914 Closes: #4930 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4930 Pull-request-sha: 66841f56e967c784f7078a787cec5129462006c8 Change-Id: I6286220bd9d488027fadc444039421a410e19a19
* Revise psycopg2 execute_values approachMike Bayer2019-08-181-5/+9
| | | | | | | | | | | | Revised the approach for the just added support for the psycopg2 "execute_values()" feature added in 1.3.7 for :ticket:`4623`. The approach relied upon a regular expression that would fail to match for a more complex INSERT statement such as one which had subqueries involved. The new approach matches exactly the string that was rendered as the VALUES clause. Fixes: #4623 Change-Id: Icaae0f7b6bcf87a2cf5c6290a839c8429dd5fac3
* Add new executemany_mode, support for psycopg2.extras.execute_values()Yuval Dinari2019-08-121-29/+149
| | | | | | | | | | | | | | | | | | | Added new dialect flag for the psycopg2 dialect, ``executemany_mode`` which supersedes the previous experimental ``use_batch_mode`` flag. ``executemany_mode`` supports both the "execute batch" and "execute values" functions provided by psycopg2, the latter which is used for compiled :func:`.insert` constructs. Pull request courtesy Yuval Dinari. .. seealso:: :ref:`executemany_mode` Fixes: #4623 Closes: #4764 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4764 Pull-request-sha: c3d3a36f7eb66c86d14ed9c1c31b4b48bd204855 Change-Id: I77e26ca729f9317af1488a6c054c23fa1a6b622b
* psycopg2 NOTICE fixupMike Bayer2019-06-091-5/+26
| | | | | | | | | | | - don't call relatively expensive isEnabledFor(), just call _log_notices - don't reset the list if it's empty - fix the test to use a custom function to definitely create a notice, confirmed that PG seems to no longer create the "implicit sequence" notices - assert that the reset of the notices works too - update the docs to illustrate for folks who haven't worked with logging before Change-Id: I7291e647c177d338e0ad673f3106b4d503e4b3ea
* Black updatesMike Bayer2019-04-111-1/+1
| | | | | | Updating for a few black adjustments Change-Id: I9c8abadd1ccd6173e6d68dd62f14cec208cc304a
* Propagate query-arg-only URL to psycopg2; don't send blank hostMike Bayer2019-04-091-1/+6
| | | | | | | | | | | | | | Fixed regression from release 1.3.2 caused by :ticket:`4562` where a URL that contained only a query string and no hostname, such as for the purposes of specifying a service file with connection information, would no longer be propagated to psycopg2 properly. The change in :ticket:`4562` has been adjusted to further suit psycopg2's exact requirements, which is that if there are any connection parameters whatsoever, the "dsn" parameter is no longer required, so in this case the query string parameters are passed alone. Fixes: #4601 Change-Id: Ic29a8b77bcf50ee996968bab25aaac7ae4bfc26f
* Support DNS-less connections for psycopg2Julian Mehnle2019-03-241-4/+37
| | | | | | | | | | | | | | | | Added support for parameter-less connection URLs for the psycopg2 dialect, meaning, the URL can be passed to :func:`.create_engine` as ``"postgresql+psycopg2://"`` with no additional arguments to indicate an empty DSN passed to libpq, which indicates to connect to "localhost" with no username, password, or database given. Pull request courtesy Julian Mehnle. Fixes: #4562 Closes: #4563 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4563 Pull-request-sha: 8a05c96944a0523b06e9772cfe1832e500a43641 Change-Id: Ib6fca3c3c9eebeaf590d7f7fb0bc8cd4b6e4a55a
* Add deprecation warnings to all deprecated APIsMike Bayer2019-01-231-5/+11
| | | | | | | | | | | | | | | A large change throughout the library has ensured that all objects, parameters, and behaviors which have been noted as deprecated or legacy now emit ``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now defaults to displaying deprecation warnings, as well as that modern test suites based on tools like tox and pytest tend to display deprecation warnings, this change should make it easier to note what API features are obsolete. See the notes added to the changelog and migration notes for further details. Fixes: #4393 Change-Id: If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b
* link the psycopg2 fast execution document to the Core tutorial executemany ↵Mike Bayer2019-01-181-1/+5
| | | | | | | | section Fixes: #4109 Change-Id: I62fcb703b552db842be335c8f9257cc0e8c5cdd5
* Remove version directives for 0.6, 0.7, 0.8Mike Bayer2019-01-151-3/+0
| | | | | | | | | - fix a few "seealso"s - ComparableProprerty's "superseded in 0.7" becomes deprecated in 0.7 Backport to currently maintained doc versions 1.2, 1.1 Change-Id: Ib1fcb2df8673dbe5c4ffc47f3896a60d1dfcb4b2
* happy new yearMike Bayer2019-01-111-1/+1
| | | | Change-Id: I6a71f4924d046cf306961c58dffccf21e9c03911
* Post black reformattingMike Bayer2019-01-061-37/+35
| | | | | | | | | | | | | Applied on top of a pure run of black -l 79 in I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes resolves all remaining flake8 conditions for those codes we have enabled in setup.cfg. Included are resolutions for all remaining flake8 issues including shadowed builtins, long lines, import order, unused imports, duplicate imports, and docstring issues. Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
* Run black -l 79 against all source filesMike Bayer2019-01-061-76/+114
| | | | | | | | | | | | | | This is a straight reformat run using black as is, with no edits applied at all. The black run will format code consistently, however in some cases that are prevalent in SQLAlchemy code it produces too-long lines. The too-long lines will be resolved in the following commit that will resolve all remaining flake8 issues including shadowed builtins, long lines, import order, unused imports, duplicate imports, and docstring issues. Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
* Assorted pre-Black fixesMike Bayer2019-01-051-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes to the test suite, a few errant imports, and setup.py: - mysql and postgresql have unused 'json' imports; remove - postgresql is exporting the 'json' symbol, remove - make sure setup.py can find __version__ using " or ' - retry logic in provision create database for postgresql fixed - refactor test_magazine to use cls.tables rather than globals - remove unused class in test_scoping - add a comment to test_deprecations that this test suite itself is deprecated - don't use mapper() and orm_mapper() in test_unitofwork, just use mapper() - remove dupe test_scalar_set_None test in test_attributes - Python 2.7 and above includes unittest.SkipTest, remove pre-2.7 fallback - use imported SkipTest in profiling - declarative test_reflection tests with "reflectable_autoincrement" already don't run on oracle or firebird; remove conditional logic for these, which also removes an "id" symbol - clean up test in test_functions, remove print statement - remove dupe test_literal_processor_coercion_native_int_out_of_range in test/sql/test_types.py - fix psycopg2_hstore ref Change-Id: I7b3444f8546aac82be81cd1e7b6d8b2ad6834fe6
* Add ssl "operation timed out" message for psycopg2André Cruz2018-02-151-0/+1
| | | | | | | | | Added "SSL SYSCALL error: Operation timed out" to the list of messages that trigger a "disconnect" scenario for the psycopg2 driver. Pull request courtesy André Cruz. Change-Id: Ie1a8fc97e74b6906ccacf53dad70fed973c42b7f Pull-request: https://bitbucket.org/zzzeek/sqlalchemy/pull-requests/3
* happy new yearMike Bayer2018-01-121-1/+1
| | | | Change-Id: I3ef36bfd0cb0ba62b3123c8cf92370a43156cf8f
* Fix as many RST parse warnings as possible.Mike Bayer2017-11-031-7/+7
| | | | | | | Still a few I can't get. Also 0.9 is EOL so hide the unreleased notes. Change-Id: If0e44d4a0b3e78e211f32d5c33b51b1a007c9c69
* Add fast execution helper support.Mike Bayer2017-10-101-1/+45
| | | | | | | | | | | | Added a new flag ``use_batch_mode`` to the psycopg2 dialect. This flag enables the use of psycopg2's ``psycopg2.extras.execute_batch`` extension when the :class:`.Engine` calls upon ``cursor.executemany()``. This extension provides a critical performance increase by over an order of magnitude when running INSERT statements in batch. The flag is False by default as it is considered to be experimental for now. Change-Id: Ib88d28bc792958d47109f644ff1d08c897db4ff7 Fixes: #4109
* Make a common approach for "emulated" typesMike Bayer2017-09-251-1/+1
| | | | | | | | | | | | | | Internal refinements to the :class:`.Enum`, :class:`.Interval`, and :class:`.Boolean` types, which now extend a common mixin :class:`.Emulated` that indicates a type that provides Python-side emulation of a DB native type, switching out to the DB native type when a supporting backend is in use. The Postgresql :class:`.INTERVAL` type when used directly will now include the correct type coercion rules for SQL expressions that also take effect for :class:`.sqltypes.Interval` (such as adding a date to an interval yields a datetime). Change-Id: Ifb9f9d7cbd9f5990dcb2abb583193e9e92b789ad Fixes: #4088
* Double percent signs based on paramstyle, not dialectMike Bayer2017-04-051-9/+2
| | | | | | | | | | | | This patch moves the "doubling" of percent signs into the base compiler and makes it completely a product of whether or not the paramstyle is format/pyformat or not. Without this paramstyle, percent signs are not doubled across text(), literal_column(), and column(). Change-Id: Ie2f278ab1dbb94b5078f85c0096d74dbfa049197 Fixes: #3740
* update for 2017 copyrightMike Bayer2017-01-041-1/+1
| | | | Change-Id: I4e8c2aa8fe817bb2af8707410fa0201f938781de
* Add support for server side cursors to mysqldb and pymysqlRoman Podoliaka2016-11-101-37/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows to skip buffering of the results on the client side, e.g. the following snippet: table = sa.Table( 'testtbl', sa.MetaData(), sa.Column('id', sa.Integer, primary_key=True), sa.Column('a', sa.Integer), sa.Column('b', sa.String(512)) ) table.create(eng, checkfirst=True) with eng.connect() as conn: result = conn.execute(table.select().limit(1)).fetchone() if result is None: for _ in range(1000): conn.execute( table.insert(), [{'a': random.randint(1, 100000), 'b': ''.join(random.choice(string.ascii_letters) for _ in range(100))} for _ in range(1000)] ) with eng.connect() as conn: for row in conn.execution_options(stream_results=True).execute(table.select()): pass now uses ~23 MB of memory instead of ~327 MB on CPython 3.5.2 and PyMySQL 0.7.9. psycopg2 implementation and execution options (stream_results, server_side_cursors) are reused. Change-Id: I4dc23ce3094f027bdff51b896b050361991c62e2
* spelling: Postgresql -> PostgreSQLVille Skyttä2016-10-081-8/+8
|
* Handle `SSL error: decryption failed or bad record mac`Iuri de Silvio2016-05-131-0/+1
| | | | | | | | | This is another psycopg2 error message representing an unusable connection. Fixes #3715 Change-Id: Ida6e212963e9c7336bf2944e7ef928619ac3a0e7 Pull-request: https://bitbucket.org/zzzeek/sqlalchemy/pull-requests/85
* - happy new yearMike Bayer2016-01-291-1/+1
|
* - Added :class:`.mysql.JSON` for MySQL 5.7. The JSON type providesMike Bayer2016-01-061-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | persistence of JSON values in MySQL as well as basic operator support of "getitem" and "getpath", making use of the ``JSON_EXTRACT`` function in order to refer to individual paths in a JSON structure. fixes #3547 - Added a new type to core :class:`.types.JSON`. This is the base of the PostgreSQL :class:`.postgresql.JSON` type as well as that of the new :class:`.mysql.JSON` type, so that a PG/MySQL-agnostic JSON column may be used. The type features basic index and path searching support. fixes #3619 - reorganization of migration docs etc. to try to refer both to the fixes to JSON that helps Postgresql while at the same time indicating these are new features of the new base JSON type. - a rework of the Array/Indexable system some more, moving things that are specific to Array out of Indexable. - new operators for JSON indexing added to core so that these can be compiled by the PG and MySQL dialects individually - rename sqltypes.Array to sqltypes.ARRAY - as there is no generic Array implementation, this is an uppercase type for now, consistent with the new sqltypes.JSON type that is also not a generic implementation. There may need to be some convention change to handle the case of datatypes that aren't generic, rely upon DB-native implementations, but aren't necessarily all named the same thing.
* - merge of ticket_3499 indexed access branchMike Bayer2015-08-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - The "hashable" flag on special datatypes such as :class:`.postgresql.ARRAY`, :class:`.postgresql.JSON` and :class:`.postgresql.HSTORE` is now set to False, which allows these types to be fetchable in ORM queries that include entities within the row. fixes #3499 - The Postgresql :class:`.postgresql.ARRAY` type now supports multidimensional indexed access, e.g. expressions such as ``somecol[5][6]`` without any need for explicit casts or type coercions, provided that the :paramref:`.postgresql.ARRAY.dimensions` parameter is set to the desired number of dimensions. fixes #3487 - The return type for the :class:`.postgresql.JSON` and :class:`.postgresql.JSONB` when using indexed access has been fixed to work like Postgresql itself, and returns an expression that itself is of type :class:`.postgresql.JSON` or :class:`.postgresql.JSONB`. Previously, the accessor would return :class:`.NullType` which disallowed subsequent JSON-like operators to be used. part of fixes #3503 - The :class:`.postgresql.JSON`, :class:`.postgresql.JSONB` and :class:`.postgresql.HSTORE` datatypes now allow full control over the return type from an indexed textual access operation, either ``column[someindex].astext`` for a JSON type or ``column[someindex]`` for an HSTORE type, via the :paramref:`.postgresql.JSON.astext_type` and :paramref:`.postgresql.HSTORE.text_type` parameters. also part of fixes #3503 - The :attr:`.postgresql.JSON.Comparator.astext` modifier no longer calls upon :meth:`.ColumnElement.cast` implicitly, as PG's JSON/JSONB types allow cross-casting between each other as well. Code that makes use of :meth:`.ColumnElement.cast` on JSON indexed access, e.g. ``col[someindex].cast(Integer)``, will need to be changed to call :attr:`.postgresql.JSON.Comparator.astext` explicitly. This is part of the refactor in references #3503 for consistency in operator use.
* - add test cases for pullreq github:182, where we add a newMike Bayer2015-06-141-2/+11
| | | | | | | | "max_row_buffer" execution option for BufferedRowResultProxy - also add documentation, changelog and version notes - rework the max_row_buffer argument to be interpreted from the execution options upfront when the BufferedRowResultProxy is first initialized.
* - Repaired some typing and test issues related to the pypyMike Bayer2015-06-051-5/+18
| | | | | | | | | psycopg2cffi dialect, in particular that the current 2.7.0 version does not have native support for the JSONB type. The version detection for psycopg2 features has been tuned into a specific sub-version for psycopg2cffi. Additionally, test coverage has been enabled for the full series of psycopg2 features under psycopg2cffi. fixes #3439
* - Fixed bug where known boolean values used byMike Bayer2015-05-261-0/+4
| | | | | | | | :func:`.engine_from_config` were not being parsed correctly; these included ``pool_threadlocal`` and the psycopg2 argument ``use_native_unicode``. fixes #3435 - add legacy_schema_aliasing config parsing for mssql - move use_native_unicode config arg to the psycopg2 dialect
* - Fixed a long-standing bug where the :class:`.Enum` type as usedMike Bayer2015-04-041-1/+1
| | | | | | | | | | with the psycopg2 dialect in conjunction with non-ascii values and ``native_enum=False`` would fail to decode return results properly. This stemmed from when the PG :class:`.postgresql.ENUM` type used to be a standalone type without a "non native" option. fixes #3354 - corrected the assertsql comparison rule to expect a non-ascii SQL string
* - copyright 2015Mike Bayer2015-03-101-1/+1
|
* - Repaired support for Postgresql UUID types in conjunction withMike Bayer2015-02-011-3/+35
| | | | | | | | | | the ARRAY type when using psycopg2. The psycopg2 dialect now employs use of the psycopg2.extras.register_uuid() hook so that UUID values are always passed to/from the DBAPI as UUID() objects. The :paramref:`.UUID.as_uuid` flag is still honored, except with psycopg2 we need to convert returned UUID objects back into strings when this is disabled. fixes #2940
* - Added support for the :class:`postgresql.JSONB` datatype whenMike Bayer2015-01-311-10/+16
| | | | | | | | | | | | | | | | | using psycopg2 2.5.4 or greater, which features native conversion of JSONB data so that SQLAlchemy's converters must be disabled; additionally, the newly added psycopg2 extension ``extras.register_default_jsonb`` is used to establish a JSON deserializer passed to the dialect via the ``json_deserializer`` argument. Also repaired the Postgresql integration tests which weren't actually round-tripping the JSONB type as opposed to the JSON type. Pull request courtesy Mateusz Susik. - Repaired the use of the "array_oid" flag when registering the HSTORE type with older psycopg2 versions < 2.4.3, which does not support this flag, as well as use of the native json serializer hook "register_default_json" with user-defined ``json_deserializer`` on psycopg2 versions < 2.5, which does not include native json.
* Merge remote-tracking branch 'origin/pr/145' into pr145Mike Bayer2015-01-311-2/+14
|\
| * Psycopg2 JSONB support.pr/145Mateusz Susik2014-10-241-2/+14
| |
* | Added psycopg2cffi dialectShaun Stanworth2015-01-261-3/+14
| |
* | - correctionsMike Bayer2014-12-271-4/+5
| | | | | | | | - attempt to add a script to semi-automate the fixing of links
* | - in lieu of adding a new system of translating bound parameter namesMike Bayer2014-11-101-0/+49
|/ | | | | | | for psycopg2 and others, encourage users to take advantage of positional styles by documenting "paramstyle". A section is added to psycopg2 specifically as this is a pretty common spot for named parameters that may be unusually named. fixes #3246.
* - clean up unicode docs and clarify that client_encodingMike Bayer2014-10-071-13/+53
| | | | | at the engine level is not the same thing as at the connect args level.
* - A revisit to this issue first patched in 0.9.5, apparentlyMike Bayer2014-09-291-5/+9
| | | | | | | | | | psycopg2's ``.closed`` accessor is not as reliable as we assumed, so we have added an explicit check for the exception messages "SSL SYSCALL error: Bad file descriptor" and "SSL SYSCALL error: EOF detected" when detecting an is-disconnect scenario. We will continue to consult psycopg2's connection.closed as a first check. fixes #3021
* - apply pep8 formatting to sqlalchemy/sql, sqlalchemy/util, sqlalchemy/dialects,Brian Jarrett2014-07-201-56/+71
| | | | sqlalchemy/orm, sqlalchemy/event, sqlalchemy/testing
* - break up the <authors> copyright comment as part of a passMike Bayer2014-07-091-1/+2
| | | | to get all flake8 passing
* - changelog + docs for pg8000 transaction isolation levelMike Bayer2014-05-301-0/+6
|
* update commentsMike Bayer2014-05-301-4/+6
|
* Merge branch 'psycopg_disconnect' of https://github.com/dirkmueller/sqlalchemyMike Bayer2014-05-301-1/+5
|\
| * Another Variant for detecting if a connection is closedpr/87Dirk Mueller2014-05-111-1/+5
| | | | | | | | | | | | | | If there is a closed attribute on the connection and it is true, return true. Implements a todo in the code and helps in one specific disconnect case where it previously did not match because the error message was "unknown error".
* | Merged in mineo/sqlalchemy/mineo/posgtresql-postgresql-1399232068499 (pull ↵Mike Bayer2014-05-121-1/+1
|\ \ | | | | | | | | | | | | | | | request #17) Posgtresql -> PostgreSQL
View Lorry Controller Status