summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/pool.py
Commit message (Collapse)AuthorAgeFilesLines
* - do a broken zimports run on libMike Bayer2019-01-051-4/+9
|
* step 1, straight black -l 79 runMike Bayer2019-01-051-140/+196
| | | | includes adjustment to setup.py to recognize __version__ correctly
* re-word recycle parameter documentationDavid Poggi2018-10-301-4/+4
| | | | | Change-Id: I80cabcd9fa3f3b45e5355bf6c774a8eee02e7f1b (cherry picked from commit 83864ffe634bb1e3e9e530fbe51c03a186799110)
* Fix links and update verbiage for reset_on_returnMike Bayer2018-08-281-14/+11
| | | | | Change-Id: I642056b78570b44fdefe7f0874a8e4c720389f14 (cherry picked from commit d8bb208a85f6366c58426a85b3d4ec1d6e43ac6c)
* Prevent double-checkins and guard during reset-on-return invalidationsMike Bayer2018-05-151-3/+6
| | | | | | | | | | | | | | | | | Fixed connection pool issue whereby if a disconnection error were raised during the connection pool's "reset on return" sequence in conjunction with an explicit transaction opened against the enclosing :class:`.Connection` object (such as from calling :meth:`.Session.close` without a rollback or commit, or calling :meth:`.Connection.close` without first closing a transaction declared with :meth:`.Connection.begin`), a double-checkin would result, which could then lead towards concurrent checkouts of the same connection. The double-checkin condition is now prevented overall by an assertion, as well as the specific double-checkin scenario has been fixed. Change-Id: If5bb6941e36326846b14918c33ebfdd5604f642e Fixes: #4252 (cherry picked from commit dada909a1009ad2f77063752ac8c13a7808dd916)
* Invalidate on failed connect handlerMike Bayer2018-03-281-4/+8
| | | | | | | | | | | | | | Fixed bug in connection pool where a connection could be present in the pool without all of its "connect" event handlers called, if a previous "connect" handler threw an exception; note that the dialects themselves have connect handlers that emit SQL, such as those which set transaction isolation, which can fail if the database is in a non-available state, but still allows a connection. The connection is now invalidated first if any of the connect handlers fail. Change-Id: I61d6f4827a98ab8455f1c3e1c55d046eeccec09a Fixes: #4225 (cherry picked from commit c5437296713288cbfcd323032ca48a75a97e10e5)
* Ensure weakref finalize_fairy operates upon the current connectionMike Bayer2018-02-081-6/+9
| | | | | | | | | | | | | | | Fixed a fairly serious connection pool bug where a connection that is acquired after being refreshed as a result of a user-defined :class:`.DisconnectionError` or due to the 1.2-released "pre_ping" feature would not be correctly reset if the connection were returned to the pool by weakref cleanup (e.g. the front-facing object is garbage collected); the weakref would still refer to the previously invalidated DBAPI connection which would have the reset operation erroneously called upon it instead. This would lead to stack traces in the logs and a connection being checked into the pool without being reset, which can cause locking issues. Change-Id: Iabd9f3a63a1d0207d0de0054a6ced3560818cf9c Fixes: #4184
* happy new yearMike Bayer2018-01-121-1/+1
| | | | Change-Id: I3ef36bfd0cb0ba62b3123c8cf92370a43156cf8f
* Implement an error lookupMike Bayer2017-12-271-1/+1
| | | | | | | | Add codes to commonly raised error messages and classes that link back to fixed documentation sections giving background on these messages. Change-Id: I78d0660add7026bb662e20305a59283b20616954
* Fix as many RST parse warnings as possible.Mike Bayer2017-11-031-1/+2
| | | | | | | Still a few I can't get. Also 0.9 is EOL so hide the unreleased notes. Change-Id: If0e44d4a0b3e78e211f32d5c33b51b1a007c9c69
* Don't do recovery operations under Empty/AttributeErrorMike Bayer2017-08-251-16/+20
| | | | | | | | | | | | Made some adjustments to :class:`.Pool` and :class:`.Connection` such that recovery logic is not run underneath exception catches for ``pool.Empty``, ``AttributeError``, since when the recovery operation itself fails, Python 3 creates a misleading stack trace referring to the ``Empty`` / ``AttributeError`` as the cause, when in fact these exception catches are part of control flow. Change-Id: Id3ed9a8f96ce4ccb4009c94af30ddc2ddb9818b9 Fixes: #4028
* Integrate "pre-ping" into connection pool.Mike Bayer2017-03-201-8/+51
| | | | | | | | | | | | | | | Added native "pessimistic disconnection" handling to the :class:`.Pool` object. The new parameter :paramref:`.Pool.pre_ping`, available from the engine as :paramref:`.create_engine.pool_pre_ping`, applies an efficient form of the "pre-ping" recipe featured in the pooling documentation, which upon each connection check out, emits a simple statement, typically "SELECT 1", to test the connection for liveness. If the existing connection is no longer able to respond to commands, the connection is transparently recycled, and all other connections made prior to the current timestamp are invalidated. Change-Id: I89700d0075e60abd2250e54b9bd14daf03c71c00 Fixes: #3919
* Support python3.6Mike Bayer2017-01-131-3/+3
| | | | | | | | | | | Corrects some warnings and adds tox config. Adds DeprecationWarning to the error category. Large sweep for string literals w/ backslashes as this is common in docstrings Co-authored-by: Andrii Soldatenko Fixes: #3886 Change-Id: Ia7c838dfbbe70b262622ed0803d581edc736e085 Pull-request: https://github.com/zzzeek/sqlalchemy/pull/337
* update for 2017 copyrightMike Bayer2017-01-041-1/+1
| | | | Change-Id: I4e8c2aa8fe817bb2af8707410fa0201f938781de
* Spelling fixesVille Skyttä2016-10-081-1/+1
|
* Handle BaseException in all _handle_dbapi_errorMike Bayer2016-09-211-0/+1
| | | | | | | | | | | | | | | | | Tests illustrate that exceptions like GreenletExit and even KeyboardInterrupt can corrupt the state of a DBAPI connection like that of pymysql and mysqlclient. Intercept BaseException errors within the handle_error scheme and invalidate just the connection alone in this case, but not the whole pool. The change is backwards-incompatible with a program that currently intercepts ctrl-C within a database transaction and wants to continue working on that transaction. Ensure the event hook can be used to reverse this behavior. Change-Id: Ifaa013c13826d123eef34e32b7e79fff74f1b21b Fixes: #3803
* Additions to support HAAlchemy pluginMike Bayer2016-09-161-11/+80
| | | | | | | | | | | | | | | | | | | - add a connect=True key to connection record to support pre-loading of _ConnectionRecord objects - ensure _ConnectionRecord.close() leaves the record in a good state for reopening - add _ConnectionRecord.record_info for persistent storage - add "in_use" accessor based on fairy_ref being present or not - allow for the exclusions system and SuiteRequirements to be usable without the full plugin_base setup. - move some Python-env requirements to the importable requirements.py module. - allow starttime to be queried - add additional events for engine plugins - have "dialect" be a first-class parameter to the pool, ensure the engine strategy supplies it up front Change-Id: Ibf549f7a1766e49d335cd6f5e26bacfaef9a8229
* - remember to not reassign self.connection now that __connect()Mike Bayer2016-03-251-2/+2
| | | | has no return value
* - Added connection pool events :meth:`ConnectionEvents.close`,Mike Bayer2016-03-251-22/+30
| | | | | :meth:`.ConnectionEvents.detach`, :meth:`.ConnectionEvents.close_detached`.
* - happy new yearMike Bayer2016-01-291-1/+1
|
* - Fixed critical issue whereby the pool "checkout" event handlerMike Bayer2015-07-221-0/+5
| | | | | | | | | | | | may be called against a stale connection without the "connect" event handler having been called, in the case where the pool attempted to reconnect after being invalidated and failed; the stale connection would remain present and would be used on a subsequent attempt. This issue has a greater impact in the 1.0 series subsequent to 1.0.2, as it also delivers a blanked-out ``.info`` dictionary to the event handler; prior to 1.0.2 the ``.info`` dictionary is still the previous one. fixes #3497
* - Fixed bug where in the case that a pool checkout event handler is usedMike Bayer2015-05-141-1/+7
| | | | | | | | | | | | | | | | | and the database can no longer be connected towards, that the checkout handler failure is caught, the attempt to re-acquire the connection also raises an exception, but the underlying connection record is not immediately re-checked in before the exception is propagated outwards, having the effect that the checked-out record does not close itself until the stack trace it's associated with is garbage collected, preventing that record from being used for a new checkout until we leave the scope of the stack trace. This can lead to confusion in the specific case of when the number of current stack traces in memory exceeds the number of connections the pool can return, as the pool will instead begin to raise errors about no more checkouts available, rather than attempting a connection again. The fix applies a checkin of the record before re-raising. fixes #3419
* - revise the last commit with a more traditional approachMike Bayer2015-04-301-29/+14
| | | | using descriptors; ensure that mock.patch() honors descriptor setters
* - work the wrapping of the "creator" to be as resilient toMike Bayer2015-04-301-8/+31
| | | | | old / new style, direct access, and ad-hoc patching and unpatching as possible
* - New features added to support engine/pool plugins with advancedMike Bayer2015-04-301-13/+73
| | | | | | | | | | | | | | | | | | | | | | | | | functionality. Added a new "soft invalidate" feature to the connection pool at the level of the checked out connection wrapper as well as the :class:`._ConnectionRecord`. This works similarly to a modern pool invalidation in that connections aren't actively closed, but are recycled only on next checkout; this is essentially a per-connection version of that feature. A new event :class:`.PoolEvents.soft_invalidate` is added to complement it. fixes #3379 - Added new flag :attr:`.ExceptionContext.invalidate_pool_on_disconnect`. Allows an error handler within :meth:`.ConnectionEvents.handle_error` to maintain a "disconnect" condition, but to handle calling invalidate on individual connections in a specific manner within the event. - Added new event :class:`.DialectEvents.do_connect`, which allows interception / replacement of when the :meth:`.Dialect.connect` hook is called to create a DBAPI connection. Also added dialect plugin hooks :meth:`.Dialect.get_dialect_cls` and :meth:`.Dialect.engine_created` which allow external plugins to add events to existing dialects using entry points. fixes #3355
* - add a warning to SingletonThreadPool that it isn't for general useMike Bayer2015-04-211-0/+13
|
* - Added the string value ``"none"`` to those accepted by theMike Bayer2015-04-211-1/+5
| | | | | | | | :paramref:`.Pool.reset_on_return` parameter as a synonym for ``None``, so that string values can be used for all settings, allowing .ini file utilities like :func:`.engine_from_config` to be usable without issue. fixes #3375
* - copyright 2015Mike Bayer2015-03-101-1/+1
|
* - Fixed bug in :class:`.Connection` and pool where theMike Bayer2015-02-041-0/+1
| | | | | | | | | :meth:`.Connection.invalidate` method, or an invalidation due to a database disconnect, would fail if the ``isolation_level`` parameter had been used with :meth:`.Connection.execution_options`; the "finalizer" that resets the isolation level would be called on the no longer opened connection. fixes #3302
* - wip - start factoring events so that we aren't using descriptors for dispatch,Mike Bayer2015-01-041-0/+1
| | | | allowing us to move to __slots__
* - correctionsMike Bayer2014-12-271-3/+3
| | | | - attempt to add a script to semi-automate the fixing of links
* cleanup exception handling - use new exception hierarchy (since python 2.5)pr/140ndparker2014-10-021-8/+4
|
* improve exception vs. exit handlingndparker2014-09-231-4/+4
|
* Fix doc typo 'conjunection'pr/130Gunnlaugur Þór Briem2014-08-211-1/+1
|
* - Fixed bug in connection pool logging where the "connection checked out"Mike Bayer2014-08-201-10/+12
| | | | | | | | debug logging message would not emit if the logging were set up using ``logging.setLevel()``, rather than using the ``echo_pool`` flag. Tests to assert this logging have been added. This is a regression that was introduced in 0.9.0. fixes #3168
* PEP8 style fixesBrian Jarrett2014-07-131-99/+115
|
* - break up the <authors> copyright comment as part of a passMike Bayer2014-07-091-1/+2
| | | | to get all flake8 passing
* - Fixed some "double invalidate" situations were detected whereMike Bayer2014-05-101-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | a connection invalidation could occur within an already critical section like a connection.close(); ultimately, these conditions are caused by the change in :ticket:`2907`, in that the "reset on return" feature calls out to the Connection/Transaction in order to handle it, where "disconnect detection" might be caught. However, it's possible that the more recent change in :ticket:`2985` made it more likely for this to be seen as the "connection invalidate" operation is much quicker, as the issue is more reproducible on 0.9.4 than 0.9.3. Checks are now added within any section that an invalidate might occur to halt further disallowed operations on the invalidated connection. This includes two fixes both at the engine level and at the pool level. While the issue was observed with highly concurrent gevent cases, it could in theory occur in any kind of scenario where a disconnect occurs within the connection close operation. fixes #3043 ref #2985 ref #2907 - add some defensive checks during an invalidate situation: 1. _ConnectionRecord.invalidate might be called twice within finalize_fairy if the _reset() raises an invalidate condition, invalidates, raises and then goes to invalidate the CR. so check for this. 2. similarly within Conneciton, anytime we do handle_dbapi_error(), we might become invalidated. so a following finally must check self.__invalid before dealing with the connection any futher.
* - work on fixing some race-condition failures:Mike Bayer2014-03-261-2/+5
| | | | | | | | | | | 1. make sure pool._invalidate() sets the timestamp up before invalidating the target connection. we can otherwise show how the conn.invalidate() + pool._invalidate() can lead to an extra connection being made. 2. to help with that, soften up the check on connection.invalidate() when connection is already closed. a warning is fine here 3. add a mutex to test_max_overflow() when we connect, because the way we're using mock depends on an iterator, that needs to be synchronized
* - A major improvement made to the mechanics by which the :class:`.Engine`Mike Bayer2014-03-221-19/+27
| | | | | | | | | | | | recycles the connection pool when a "disconnect" condition is detected; instead of discarding the pool and explicitly closing out connections, the pool is retained and a "generational" timestamp is updated to reflect the current time, thereby causing all existing connections to be recycled when they are next checked out. This greatly simplifies the recycle process, removes the need for "waking up" connect attempts waiting on the old pool and eliminates the race condition that many immediately-discarded "pool" objects could be created during the recycle operation. fixes #2985
* - Fixed small issue in :class:`.SingletonThreadPool` where the currentMike Bayer2014-03-051-3/+3
| | | | | connection to be returned might get inadvertently cleaned out during the "cleanup" process. Patch courtesy jd23.
* - don't duplicate docs for Pool within QueuePoolMike Bayer2014-01-311-71/+68
| | | | - add huge warning regarding how use_threadlocal probably not what you want
* - :class:`.Connection` now associates a newMike Bayer2014-01-121-12/+43
| | | | | | | | | | | | | | | | | | | | | | :class:`.RootTransaction` or :class:`.TwoPhaseTransaction` with its immediate :class:`._ConnectionFairy` as a "reset handler" for the span of that transaction, which takes over the task of calling commit() or rollback() for the "reset on return" behavior of :class:`.Pool` if the transaction was not otherwise completed. This resolves the issue that a picky transaction like that of MySQL two-phase will be properly closed out when the connection is closed without an explicit rollback or commit (e.g. no longer raises "XAER_RMFAIL" in this case - note this only shows up in logging as the exception is not propagated within pool reset). This issue would arise e.g. when using an orm :class:`.Session` with ``twophase`` set, and then :meth:`.Session.close` is called without an explicit rollback or commit. The change also has the effect that you will now see an explicit "ROLLBACK" in the logs when using a :class:`.Session` object in non-autocommit mode regardless of how that session was discarded. Thanks to Jeff Dairiki and Laurence Rowe for isolating the issue here. [ticket:2907]
* - add new event PoolEvents.invalidate(). allows interception of invalidationMike Bayer2014-01-121-17/+118
| | | | | | | | | | | | | | | events including auto-invalidation, which is useful both for tests here as well as detecting failure conditions within the "reset" or "close" cases. - rename the argument for PoolEvents.reset() to dbapi_connection and connection_record to be consistent with everything else. - add new documentation sections on invalidation, including auto-invalidation and the invalidation process within the pool. - add _ConnectionFairy and _ConnectionRecord to the pool documentation. Establish docs for common _ConnectionFairy/_ConnectionRecord methods and accessors and have PoolEvents docs refer to _ConnectionRecord, since it is passed to all events. Rename a few _ConnectionFairy methods that are actually private to pool such as _checkout(), _checkin() and _checkout_existing(); there should not be any external code calling these
* - happy new yearMike Bayer2014-01-051-1/+1
|
* - restore having the connection closed before decrementing the overflow counterMike Bayer2013-12-061-2/+4
|
* - The :class:`.QueuePool` has been enhanced to not block new connectionMike Bayer2013-12-061-31/+33
| | | | | | | | attempts when an existing connection attempt is blocking. Previously, the production of new connections was serialized within the block that monitored overflow; the overflow counter is now altered within it's own critical section outside of the connection process itself. [ticket:2880]
* move FAQ to the docs, [ticket:2133]Mike Bayer2013-08-211-4/+4
|
* - apply an import refactoring to the ORM as wellMike Bayer2013-08-141-3/+1
| | | | | | | | | - rework the event system so that event modules load after their targets, dependencies are reversed - create an improved strategy lookup system for the ORM - rework the ORM to have very few import cycles - move out "importlater" to just util.dependency - other tricks to cross-populate modules in as clear a way as possible
* - assorted fixes raised by pypy 2.1beta2, but all of which are goodMike Bayer2013-08-011-1/+2
| | | | | | | | | | | | | | | ideas in general: - pypy2.1 w/ sqlite3 is the first DBAPI we're seeing returning unicode in cursor.description without being py3k. add a new on-connect check for this, if we get back a u"", just don't do description decoding, should be OK for now. - the set tests in test_collection were assuming the two sets would be ordered the same when it tested pop(), can't really assume that. - test_serializer gets worse and worse, pickle is just not really viable here, ding out pypy - pypy2.1b2 seems to allow cursor.lastrowid to work (or we changed something?) - pool._threadconns.current() is a weakref, it can be None - another one of those logging.handlers imports
View Lorry Controller Status