summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/dialects/postgresql/psycopg2.py
Commit message (Collapse)AuthorAgeFilesLines
...
* Support for multiple hosts in PostgreSQL connection stringRamonWill2020-09-231-0/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Provide support for multiple hosts in the PostgreSQL connection string. A user requested for SQLAlchemy to support multiple hosts within a PostgreSQL URL string. The proposed fix allows this. In the event that the url contains multiple hosts the proposed code will convert the query["hosts"] tuple into a single string. This allows the hosts to then get converted into a valid dsn variable in the psycopg2 connect function. This pull request is: - [ ] A documentation / typographical error fix - Good to go, no issue or tests are needed - [X ] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. - [ ] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. **Have a nice day!** Fixes: #4392 Closes: #5554 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5554 Pull-request-sha: 3f7a0ab8df9f1411a9f1ac0e152583bc7bf0c365 Change-Id: I3f3768d51b8331de786ffdc025b7ecfc662eafe5
* Deprecate engine-wise ss cursors; repair mariadbconnectorMike Bayer2020-09-131-16/+1
| | | | | | | | | | | | | | | | | | | The server_side_cursors engine-wide feature relies upon regexp parsing of statements a well as general guessing as to when the feature should be used. This is not within the 2.0 way of doing things and should be removed. Additionally, mariadbconnector defaults to unbuffered cursors; add new cursor hooks so that mariadbconnector can specify buffered or unbuffered cursors without too much difficulty. This will also correctly default mariadbconnector to buffered cursors which should repair the segfaults we've been getting. Try to restore the assert_raises that was removed in 5b6dfc0c38bf1f01da4b8 to see if mariadbconnector segfaults are resolved. Change-Id: I77f1c972c742e40694972f578140bb0cac8c39eb
* Create connection characteristics API; implement postgresql flagsMike Bayer2020-09-081-0/+12
| | | | | | | | | | | Added support for PostgreSQL "readonly" and "deferrable" flags for all of psycopg2, asyncpg and pg8000 dialects. This takes advantage of a newly generalized version of the "isolation level" API to support other kinds of session attributes set via execution options that are reliably reset when connections are returned to the connection pool. Fixes: #5549 Change-Id: I0ad6d7a095e49d331618274c40ce75c76afdc7dd
* Clean python UUID importsVlastimil Zíma2020-07-301-5/+1
| | | | | | | | | | | Fixes: #5482 All supported python versions provide 'uuid' module. Closes: #5483 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5483 Pull-request-sha: fc32498a8b639ff21d5898100592782826d2c6dd Change-Id: I8b41b811da7576f724353425dad5d6f581641b4b
* ORM executemany returningMike Bayer2020-06-271-7/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Build on #5401 to allow the ORM to take advanage of executemany INSERT + RETURNING. Implemented the feature updated tests to support INSERT DEFAULT VALUES, needed to come up with a new syntax for compiler INSERT INTO table (anycol) VALUES (DEFAULT) which can then be iterated out for executemany. Added graceful degrade to plain executemany for PostgreSQL <= 8.2 Renamed EXECUTEMANY_DEFAULT to EXECUTEMANY_PLAIN Fix issue where unicode identifiers or parameter names wouldn't work with execute_values() under Py2K, because we have to encode the statement and therefore have to encode the insert_single_values_expr too. Correct issue from #5401 to support executemany + return_defaults for a PK that is explicitly pre-generated, meaning we aren't actually getting RETURNING but need to return it from compiled_parameters. Fixes: #5263 Change-Id: Id68e5c158c4f9ebc33b61c06a448907921c2a657
* Default psycopg2 executemany mode to "values_only"Mike Bayer2020-06-251-123/+128
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The psycopg2 dialect now defaults to using the very performant ``execute_values()`` psycopg2 extension for compiled INSERT statements, and also impements RETURNING support when this extension is used. This allows INSERT statements that even include an autoincremented SERIAL or IDENTITY value to run very fast while still being able to return the newly generated primary key values. The ORM will then integrate this new feature in a separate change. Implements RETURNING for insert with executemany Adds support to return_defaults() mode and inserted_primary_key to support mutiple INSERTed rows, via return_defauls_rows and inserted_primary_key_rows accessors. within default execution context, new cached compiler getters are used to fetch primary keys from rows inserted_primary_key now returns a plain tuple. this is not yet a row-like object however this can be added. Adds distinct "values_only" and "batch" modes, as "values" has a lot of benefits but "batch" breaks cursor.rowcount psycopg2 minimum version 2.7 so we can remove the large number of checks for very old versions of psycopg2 simplify tests to no longer distinguish between native and non-native json Fixes: #5401 Change-Id: Ic08fd3423d4c5d16ca50994460c0c234868bd61c
* callcount reductions and refinement for cached queriesMike Bayer2020-05-281-5/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit includes that we've removed the "_orm_query" attribute from compile state as well as query context. The attribute created reference cycles and also added method call overhead. As part of this change, the interface for ORMExecuteState changes a bit, as well as the interface for the horizontal sharding extension which now deprecates the "query_chooser" callable in favor of "execute_chooser", which receives the contextual object. This will also work more nicely when we implement the new execution path for bulk updates and deletes. Pre-merge execution options for statement, connection, arguments all up front in Connection. that way they can be passed to the before_execute / after_execute events, and the ExecutionContext doesn't have to merge as second time. Core execute is pretty close to 1.3 now. baked wasn't using the new one()/first()/one_or_none() methods, fixed that. Convert non-buffered cursor strategy to be a stateless singleton. inline all the paths by which the strategy gets chosen, oracle and SQL Server dialects make use of the already-invoked post_exec() hook to establish the alternate strategies, and this is actually much nicer than it was before. Add caching to mapper instance processor for getters. Identified a reference cycle per query that was showing up as a lot of gc cleanup, fixed that. After all that, performance not budging much. Even test_baked_query now runs with significantly fewer function calls than 1.3, still 40% slower. Basically something about the new patterns just makes this slower and while I've walked a whole bunch of them back, it hardly makes a dent. that said, the performance issues are relatively small, in the 20-40% time increase range, and the new caching feature does provide for regular ORM and Core queries that are cached, and they are faster than non-cached. Change-Id: I7b0b0d8ca550c05f79e82f75cd8eff0bbfade053
* Add immutabledict C codeMike Bayer2020-05-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Start trying to convert fundamental objects to C as we now rely on a fairly small core of things, and 1.4 is having problems with complexity added being slower than the performance gains we are trying to build in. immutabledict here does seem to bench as twice as fast as the Python one, see below. However, it does not appear to be used prominently enough to make any dent in the performance tests. at the very least it may provide us some more lift-and-copy code for more C extensions. import timeit from sqlalchemy.util._collections import not_immutabledict, immutabledict def run(dict_cls): for i in range(1000000): d1 = dict_cls({"x": 5, "y": 4}) d2 = d1.union({"x": 17, "new key": "some other value"}, None) assert list(d2) == ["x", "y", "new key"] print( timeit.timeit( "run(d)", "from __main__ import run, not_immutabledict as d", number=1 ) ) print( timeit.timeit( "run(d)", "from __main__ import run, immutabledict as d", number=1 ) ) output: python: 1.8799766399897635 C code: 0.8880784640205093 Change-Id: I29e7104dc21dcc7cdf895bf274003af2e219bf6d
* Documentation updates for ResultProxy -> ResultMike Bayer2020-05-011-2/+2
| | | | | | | | | This is based off of I8091919d45421e3f53029b8660427f844fee0228 and includes all documentation-only changes as a separate merge, once the parent is merged. Change-Id: I711adea23df0f9f0b1fe7c76210bd2de6d31842d
* Set up absolute references for create_engine and relatedMike Bayer2020-04-141-9/+12
| | | | | | | includes more replacements for create_engine(), Connection, disambiguation of Result from future/baked Change-Id: Icb60a79ee7a6c45ea9056c211ffd1be110da3b5e
* Run search and replace of symbolic module namesMike Bayer2020-04-141-12/+23
| | | | | | | | Replaces a wide array of Sphinx-relative doc references with an abbreviated absolute form now supported by zzzeeksphinx. Change-Id: I94bffcc3f37885ffdde6238767224296339698a2
* Enable zzzeeksphinx module prefixesMike Bayer2020-04-141-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | zzzeeksphinx 1.1.2 in git can now convert short prefix names in a configured lookup to fully qualified module names, so that we can have succinct and portable pyrefs that still resolve absolutely. It also includes a formatter that will format all pyrefs in a fully consistent way regardless of the package path, by unconditionally removing all package tokens but always leaving class names in place including for methods, which means we no longer have to deal with tildes in pyrefs. The most immediate goal of the absolute prefixes is that we have lots of "ambiguous" names that appear in muliple places, like select(), ARRAY, ENUM etc. With the incoming future packages there is going to be lots of name overlap so it is necessary that all names eventually use absolute package paths when Sphinx receives them. In multiple stages, pyrefs will be converted using the zzzeeksphinx tools/fix_xrefs.py tool so that doclinks can be made absolute using symbolic prefixes. For this review, the actual search and replace of symbols is not performed, instead some general cleanup to prepare the docs as well as a lookup file used by the tool to do the conversion. this relatively small patch will be backported with appropriate changes to 1.3, 1.2, 1.1 and the tool can then be run on each branch individually. We are shooting for almost no warnings at all for master (still a handful I can't figure out which don't seem to have any impact) , very few for 1.3, and for 1.2 / 1.1 we hope for a significant reduction in warnings. Overall for all versions pyrefs should always point to the correct target, if they are in fact hyperlinked. it's better for a ref to go nowhere and be plain text than go to the wrong thing. Right now, hundreds of API links are pointing to the wrong thing as they are ambiguous names such as refresh(), insert(), update(), select(), join(), JSON etc. and Sphinx sends these all to essesntially random destinations among as many as five or six possible choices per symbol. A shorthand system that allows us to use absolute refs without having to type out a full blown absoulte module is the only way this is going to work, and we should ultimately seek to abandon any use of prefix dot for lookups. Everything should be on an underscore token so at the very least the module spaces can be reorganized without having to search and replace the entire documentation every time. Change-Id: I484a7329034af275fcdb322b62b6255dfeea9151
* Correct ambiguous func / class linksMike Bayer2020-03-251-2/+2
| | | | | | | | | :func:`.sql.expression.select`, :func:`.sql.expression.insert` and :class:`.sql.expression.Insert` were hitting many ambiguous symbol errors, due to future.select, as well as the PG/MySQL variants of Insert. Change-Id: Iac862bfc172a7f7f0cbba5353a83dc203bed376c
* Ensure all nested exception throws have a causeMike Bayer2020-03-021-5/+8
| | | | | | | | | | | | | | | Applied an explicit "cause" to most if not all internally raised exceptions that are raised from within an internal exception catch, to avoid misleading stacktraces that suggest an error within the handling of an exception. While it would be preferable to suppress the internally caught exception in the way that the ``__suppress_context__`` attribute would, there does not as yet seem to be a way to do this without suppressing an enclosing user constructed context, so for now it exposes the internally caught exception as the cause so that full information about the context of the error is maintained. Fixes: #4849 Change-Id: I55a86b29023675d9e5e49bc7edc5a2dc0bcd4751
* Result initial introductionMike Bayer2020-02-211-6/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This builds on cc718cccc0bf8a01abdf4068c7ea4f3 which moved RowProxy to Row, allowing Row to be more like a named tuple. - KeyedTuple in ORM is replaced with Row - ResultSetMetaData broken out into "simple" and "cursor" versions for ORM and Core, as well as LegacyCursor version. - Row now has _mapping attribute that supplies full mapping behavior. Row and SimpleRow both have named tuple behavior otherwise. LegacyRow has some mapping features on the tuple which emit deprecation warnings (e.g. keys(), values(), etc). the biggest change for mapping->tuple is the behavior of __contains__ which moves from testing of "key in row" to "value in row". - ResultProxy breaks into ResultProxy and FutureResult (interim), the latter has the newer APIs. Made available to dialects using execution options. - internal reflection methods and most tests move off of implicit Row mapping behavior and move to row._mapping, result.mappings() method using future result - a new strategy system for cursor handling replaces the various subclasses of RowProxy - some execution context adjustments. We will leave EC in but refined things like get_result_proxy() and out parameter handling. Dialects for 1.4 will need to adjust from get_result_proxy() to get_result_cursor_strategy(), if they are using this method - out parameter handling now accommodated by get_out_parameter_values() EC method. Oracle changes for this. external dialect for DB2 for example will also need to adjust for this. - deprecate case_insensitive flag for engine / result, this feature is not used mapping-methods on Row are deprecated, and replaced with Row._mapping.<meth>, including: row.keys() -> use row._mapping.keys() row.items() -> use row._mapping.items() row.values() -> use row._mapping.values() key in row -> use key in row._mapping int in row -> use int < len(row) Fixes: #4710 Fixes: #4878 Change-Id: Ieb9085e9bcff564359095b754da9ae0af55679f0
* happy new yearMike Bayer2020-01-011-1/+1
| | | | Change-Id: I08440dc25e40ea1ccea1778f6ee9e28a00808235
* Use simple growth scale with any max size for BufferedRowResultProxysumau2019-10-301-1/+2
| | | | | | | | | | | | | | | | | The maximum buffer size for the :class:`.BufferedRowResultProxy`, which is used by dialects such as PostgreSQL when ``stream_results=True``, can now be set to a number greater than 1000 and the buffer will grow to that size. Previously, the buffer would not go beyond 1000 even if the value were set larger. The growth of the buffer is also now based on a simple multiplying factor currently set to 5. Pull request courtesy Soumaya Mauthoor. Fixes: #4914 Closes: #4930 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4930 Pull-request-sha: 66841f56e967c784f7078a787cec5129462006c8 Change-Id: I6286220bd9d488027fadc444039421a410e19a19
* Revise psycopg2 execute_values approachMike Bayer2019-08-181-5/+9
| | | | | | | | | | | | Revised the approach for the just added support for the psycopg2 "execute_values()" feature added in 1.3.7 for :ticket:`4623`. The approach relied upon a regular expression that would fail to match for a more complex INSERT statement such as one which had subqueries involved. The new approach matches exactly the string that was rendered as the VALUES clause. Fixes: #4623 Change-Id: Icaae0f7b6bcf87a2cf5c6290a839c8429dd5fac3
* Add new executemany_mode, support for psycopg2.extras.execute_values()Yuval Dinari2019-08-121-29/+149
| | | | | | | | | | | | | | | | | | | Added new dialect flag for the psycopg2 dialect, ``executemany_mode`` which supersedes the previous experimental ``use_batch_mode`` flag. ``executemany_mode`` supports both the "execute batch" and "execute values" functions provided by psycopg2, the latter which is used for compiled :func:`.insert` constructs. Pull request courtesy Yuval Dinari. .. seealso:: :ref:`executemany_mode` Fixes: #4623 Closes: #4764 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4764 Pull-request-sha: c3d3a36f7eb66c86d14ed9c1c31b4b48bd204855 Change-Id: I77e26ca729f9317af1488a6c054c23fa1a6b622b
* psycopg2 NOTICE fixupMike Bayer2019-06-091-5/+26
| | | | | | | | | | | - don't call relatively expensive isEnabledFor(), just call _log_notices - don't reset the list if it's empty - fix the test to use a custom function to definitely create a notice, confirmed that PG seems to no longer create the "implicit sequence" notices - assert that the reset of the notices works too - update the docs to illustrate for folks who haven't worked with logging before Change-Id: I7291e647c177d338e0ad673f3106b4d503e4b3ea
* Black updatesMike Bayer2019-04-111-1/+1
| | | | | | Updating for a few black adjustments Change-Id: I9c8abadd1ccd6173e6d68dd62f14cec208cc304a
* Propagate query-arg-only URL to psycopg2; don't send blank hostMike Bayer2019-04-091-1/+6
| | | | | | | | | | | | | | Fixed regression from release 1.3.2 caused by :ticket:`4562` where a URL that contained only a query string and no hostname, such as for the purposes of specifying a service file with connection information, would no longer be propagated to psycopg2 properly. The change in :ticket:`4562` has been adjusted to further suit psycopg2's exact requirements, which is that if there are any connection parameters whatsoever, the "dsn" parameter is no longer required, so in this case the query string parameters are passed alone. Fixes: #4601 Change-Id: Ic29a8b77bcf50ee996968bab25aaac7ae4bfc26f
* Support DNS-less connections for psycopg2Julian Mehnle2019-03-241-4/+37
| | | | | | | | | | | | | | | | Added support for parameter-less connection URLs for the psycopg2 dialect, meaning, the URL can be passed to :func:`.create_engine` as ``"postgresql+psycopg2://"`` with no additional arguments to indicate an empty DSN passed to libpq, which indicates to connect to "localhost" with no username, password, or database given. Pull request courtesy Julian Mehnle. Fixes: #4562 Closes: #4563 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4563 Pull-request-sha: 8a05c96944a0523b06e9772cfe1832e500a43641 Change-Id: Ib6fca3c3c9eebeaf590d7f7fb0bc8cd4b6e4a55a
* Add deprecation warnings to all deprecated APIsMike Bayer2019-01-231-5/+11
| | | | | | | | | | | | | | | A large change throughout the library has ensured that all objects, parameters, and behaviors which have been noted as deprecated or legacy now emit ``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now defaults to displaying deprecation warnings, as well as that modern test suites based on tools like tox and pytest tend to display deprecation warnings, this change should make it easier to note what API features are obsolete. See the notes added to the changelog and migration notes for further details. Fixes: #4393 Change-Id: If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b
* link the psycopg2 fast execution document to the Core tutorial executemany ↵Mike Bayer2019-01-181-1/+5
| | | | | | | | section Fixes: #4109 Change-Id: I62fcb703b552db842be335c8f9257cc0e8c5cdd5
* Remove version directives for 0.6, 0.7, 0.8Mike Bayer2019-01-151-3/+0
| | | | | | | | | - fix a few "seealso"s - ComparableProprerty's "superseded in 0.7" becomes deprecated in 0.7 Backport to currently maintained doc versions 1.2, 1.1 Change-Id: Ib1fcb2df8673dbe5c4ffc47f3896a60d1dfcb4b2
* happy new yearMike Bayer2019-01-111-1/+1
| | | | Change-Id: I6a71f4924d046cf306961c58dffccf21e9c03911
* Post black reformattingMike Bayer2019-01-061-37/+35
| | | | | | | | | | | | | Applied on top of a pure run of black -l 79 in I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes resolves all remaining flake8 conditions for those codes we have enabled in setup.cfg. Included are resolutions for all remaining flake8 issues including shadowed builtins, long lines, import order, unused imports, duplicate imports, and docstring issues. Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
* Run black -l 79 against all source filesMike Bayer2019-01-061-76/+114
| | | | | | | | | | | | | | This is a straight reformat run using black as is, with no edits applied at all. The black run will format code consistently, however in some cases that are prevalent in SQLAlchemy code it produces too-long lines. The too-long lines will be resolved in the following commit that will resolve all remaining flake8 issues including shadowed builtins, long lines, import order, unused imports, duplicate imports, and docstring issues. Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
* Assorted pre-Black fixesMike Bayer2019-01-051-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes to the test suite, a few errant imports, and setup.py: - mysql and postgresql have unused 'json' imports; remove - postgresql is exporting the 'json' symbol, remove - make sure setup.py can find __version__ using " or ' - retry logic in provision create database for postgresql fixed - refactor test_magazine to use cls.tables rather than globals - remove unused class in test_scoping - add a comment to test_deprecations that this test suite itself is deprecated - don't use mapper() and orm_mapper() in test_unitofwork, just use mapper() - remove dupe test_scalar_set_None test in test_attributes - Python 2.7 and above includes unittest.SkipTest, remove pre-2.7 fallback - use imported SkipTest in profiling - declarative test_reflection tests with "reflectable_autoincrement" already don't run on oracle or firebird; remove conditional logic for these, which also removes an "id" symbol - clean up test in test_functions, remove print statement - remove dupe test_literal_processor_coercion_native_int_out_of_range in test/sql/test_types.py - fix psycopg2_hstore ref Change-Id: I7b3444f8546aac82be81cd1e7b6d8b2ad6834fe6
* Add ssl "operation timed out" message for psycopg2André Cruz2018-02-151-0/+1
| | | | | | | | | Added "SSL SYSCALL error: Operation timed out" to the list of messages that trigger a "disconnect" scenario for the psycopg2 driver. Pull request courtesy André Cruz. Change-Id: Ie1a8fc97e74b6906ccacf53dad70fed973c42b7f Pull-request: https://bitbucket.org/zzzeek/sqlalchemy/pull-requests/3
* happy new yearMike Bayer2018-01-121-1/+1
| | | | Change-Id: I3ef36bfd0cb0ba62b3123c8cf92370a43156cf8f
* Fix as many RST parse warnings as possible.Mike Bayer2017-11-031-7/+7
| | | | | | | Still a few I can't get. Also 0.9 is EOL so hide the unreleased notes. Change-Id: If0e44d4a0b3e78e211f32d5c33b51b1a007c9c69
* Add fast execution helper support.Mike Bayer2017-10-101-1/+45
| | | | | | | | | | | | Added a new flag ``use_batch_mode`` to the psycopg2 dialect. This flag enables the use of psycopg2's ``psycopg2.extras.execute_batch`` extension when the :class:`.Engine` calls upon ``cursor.executemany()``. This extension provides a critical performance increase by over an order of magnitude when running INSERT statements in batch. The flag is False by default as it is considered to be experimental for now. Change-Id: Ib88d28bc792958d47109f644ff1d08c897db4ff7 Fixes: #4109
* Make a common approach for "emulated" typesMike Bayer2017-09-251-1/+1
| | | | | | | | | | | | | | Internal refinements to the :class:`.Enum`, :class:`.Interval`, and :class:`.Boolean` types, which now extend a common mixin :class:`.Emulated` that indicates a type that provides Python-side emulation of a DB native type, switching out to the DB native type when a supporting backend is in use. The Postgresql :class:`.INTERVAL` type when used directly will now include the correct type coercion rules for SQL expressions that also take effect for :class:`.sqltypes.Interval` (such as adding a date to an interval yields a datetime). Change-Id: Ifb9f9d7cbd9f5990dcb2abb583193e9e92b789ad Fixes: #4088
* Double percent signs based on paramstyle, not dialectMike Bayer2017-04-051-9/+2
| | | | | | | | | | | | This patch moves the "doubling" of percent signs into the base compiler and makes it completely a product of whether or not the paramstyle is format/pyformat or not. Without this paramstyle, percent signs are not doubled across text(), literal_column(), and column(). Change-Id: Ie2f278ab1dbb94b5078f85c0096d74dbfa049197 Fixes: #3740
* update for 2017 copyrightMike Bayer2017-01-041-1/+1
| | | | Change-Id: I4e8c2aa8fe817bb2af8707410fa0201f938781de
* Add support for server side cursors to mysqldb and pymysqlRoman Podoliaka2016-11-101-37/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows to skip buffering of the results on the client side, e.g. the following snippet: table = sa.Table( 'testtbl', sa.MetaData(), sa.Column('id', sa.Integer, primary_key=True), sa.Column('a', sa.Integer), sa.Column('b', sa.String(512)) ) table.create(eng, checkfirst=True) with eng.connect() as conn: result = conn.execute(table.select().limit(1)).fetchone() if result is None: for _ in range(1000): conn.execute( table.insert(), [{'a': random.randint(1, 100000), 'b': ''.join(random.choice(string.ascii_letters) for _ in range(100))} for _ in range(1000)] ) with eng.connect() as conn: for row in conn.execution_options(stream_results=True).execute(table.select()): pass now uses ~23 MB of memory instead of ~327 MB on CPython 3.5.2 and PyMySQL 0.7.9. psycopg2 implementation and execution options (stream_results, server_side_cursors) are reused. Change-Id: I4dc23ce3094f027bdff51b896b050361991c62e2
* spelling: Postgresql -> PostgreSQLVille Skyttä2016-10-081-8/+8
|
* Handle `SSL error: decryption failed or bad record mac`Iuri de Silvio2016-05-131-0/+1
| | | | | | | | | This is another psycopg2 error message representing an unusable connection. Fixes #3715 Change-Id: Ida6e212963e9c7336bf2944e7ef928619ac3a0e7 Pull-request: https://bitbucket.org/zzzeek/sqlalchemy/pull-requests/85
* - happy new yearMike Bayer2016-01-291-1/+1
|
* - Added :class:`.mysql.JSON` for MySQL 5.7. The JSON type providesMike Bayer2016-01-061-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | persistence of JSON values in MySQL as well as basic operator support of "getitem" and "getpath", making use of the ``JSON_EXTRACT`` function in order to refer to individual paths in a JSON structure. fixes #3547 - Added a new type to core :class:`.types.JSON`. This is the base of the PostgreSQL :class:`.postgresql.JSON` type as well as that of the new :class:`.mysql.JSON` type, so that a PG/MySQL-agnostic JSON column may be used. The type features basic index and path searching support. fixes #3619 - reorganization of migration docs etc. to try to refer both to the fixes to JSON that helps Postgresql while at the same time indicating these are new features of the new base JSON type. - a rework of the Array/Indexable system some more, moving things that are specific to Array out of Indexable. - new operators for JSON indexing added to core so that these can be compiled by the PG and MySQL dialects individually - rename sqltypes.Array to sqltypes.ARRAY - as there is no generic Array implementation, this is an uppercase type for now, consistent with the new sqltypes.JSON type that is also not a generic implementation. There may need to be some convention change to handle the case of datatypes that aren't generic, rely upon DB-native implementations, but aren't necessarily all named the same thing.
* - merge of ticket_3499 indexed access branchMike Bayer2015-08-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - The "hashable" flag on special datatypes such as :class:`.postgresql.ARRAY`, :class:`.postgresql.JSON` and :class:`.postgresql.HSTORE` is now set to False, which allows these types to be fetchable in ORM queries that include entities within the row. fixes #3499 - The Postgresql :class:`.postgresql.ARRAY` type now supports multidimensional indexed access, e.g. expressions such as ``somecol[5][6]`` without any need for explicit casts or type coercions, provided that the :paramref:`.postgresql.ARRAY.dimensions` parameter is set to the desired number of dimensions. fixes #3487 - The return type for the :class:`.postgresql.JSON` and :class:`.postgresql.JSONB` when using indexed access has been fixed to work like Postgresql itself, and returns an expression that itself is of type :class:`.postgresql.JSON` or :class:`.postgresql.JSONB`. Previously, the accessor would return :class:`.NullType` which disallowed subsequent JSON-like operators to be used. part of fixes #3503 - The :class:`.postgresql.JSON`, :class:`.postgresql.JSONB` and :class:`.postgresql.HSTORE` datatypes now allow full control over the return type from an indexed textual access operation, either ``column[someindex].astext`` for a JSON type or ``column[someindex]`` for an HSTORE type, via the :paramref:`.postgresql.JSON.astext_type` and :paramref:`.postgresql.HSTORE.text_type` parameters. also part of fixes #3503 - The :attr:`.postgresql.JSON.Comparator.astext` modifier no longer calls upon :meth:`.ColumnElement.cast` implicitly, as PG's JSON/JSONB types allow cross-casting between each other as well. Code that makes use of :meth:`.ColumnElement.cast` on JSON indexed access, e.g. ``col[someindex].cast(Integer)``, will need to be changed to call :attr:`.postgresql.JSON.Comparator.astext` explicitly. This is part of the refactor in references #3503 for consistency in operator use.
* - add test cases for pullreq github:182, where we add a newMike Bayer2015-06-141-2/+11
| | | | | | | | "max_row_buffer" execution option for BufferedRowResultProxy - also add documentation, changelog and version notes - rework the max_row_buffer argument to be interpreted from the execution options upfront when the BufferedRowResultProxy is first initialized.
* - Repaired some typing and test issues related to the pypyMike Bayer2015-06-051-5/+18
| | | | | | | | | psycopg2cffi dialect, in particular that the current 2.7.0 version does not have native support for the JSONB type. The version detection for psycopg2 features has been tuned into a specific sub-version for psycopg2cffi. Additionally, test coverage has been enabled for the full series of psycopg2 features under psycopg2cffi. fixes #3439
* - Fixed bug where known boolean values used byMike Bayer2015-05-261-0/+4
| | | | | | | | :func:`.engine_from_config` were not being parsed correctly; these included ``pool_threadlocal`` and the psycopg2 argument ``use_native_unicode``. fixes #3435 - add legacy_schema_aliasing config parsing for mssql - move use_native_unicode config arg to the psycopg2 dialect
* - Fixed a long-standing bug where the :class:`.Enum` type as usedMike Bayer2015-04-041-1/+1
| | | | | | | | | | with the psycopg2 dialect in conjunction with non-ascii values and ``native_enum=False`` would fail to decode return results properly. This stemmed from when the PG :class:`.postgresql.ENUM` type used to be a standalone type without a "non native" option. fixes #3354 - corrected the assertsql comparison rule to expect a non-ascii SQL string
* - copyright 2015Mike Bayer2015-03-101-1/+1
|
* - Repaired support for Postgresql UUID types in conjunction withMike Bayer2015-02-011-3/+35
| | | | | | | | | | the ARRAY type when using psycopg2. The psycopg2 dialect now employs use of the psycopg2.extras.register_uuid() hook so that UUID values are always passed to/from the DBAPI as UUID() objects. The :paramref:`.UUID.as_uuid` flag is still honored, except with psycopg2 we need to convert returned UUID objects back into strings when this is disabled. fixes #2940
* - Added support for the :class:`postgresql.JSONB` datatype whenMike Bayer2015-01-311-10/+16
| | | | | | | | | | | | | | | | | using psycopg2 2.5.4 or greater, which features native conversion of JSONB data so that SQLAlchemy's converters must be disabled; additionally, the newly added psycopg2 extension ``extras.register_default_jsonb`` is used to establish a JSON deserializer passed to the dialect via the ``json_deserializer`` argument. Also repaired the Postgresql integration tests which weren't actually round-tripping the JSONB type as opposed to the JSON type. Pull request courtesy Mateusz Susik. - Repaired the use of the "array_oid" flag when registering the HSTORE type with older psycopg2 versions < 2.4.3, which does not support this flag, as well as use of the native json serializer hook "register_default_json" with user-defined ``json_deserializer`` on psycopg2 versions < 2.5, which does not include native json.
View Lorry Controller Status