delta/python-packages/sqlalchemy.git/test/aaa_profiling/test_memusage.py, branch fix_mariadb102_default github.com: zzzeek/sqlalchemy.git Isolate memory tests in forks 2017-08-14T16:41:58+00:00 Mike Bayer mike_mp@zzzcomputing.com 2017-08-14T16:04:26+00:00 2c594da2148bf15bcb8e10fc9616bbacc70b61a3 Swing the biggest hammer, run multiprocessing.Process() for each memusage test individually so that they are fully isolated from the parent process and any side effects of pytest-xdist Also add --nomemory as a shortcut to exclude_tags=memory-intensive and add this to the setup.py test runner as the memory tests should not be running for quick runs Change-Id: I3c16c781e21b33deb939a64e77a6e0e41fb86922 
Swing the biggest hammer, run multiprocessing.Process() for
each memusage test individually so that they are fully isolated
from the parent process and any side effects of pytest-xdist

Also add --nomemory as a shortcut to exclude_tags=memory-intensive
and add this to the setup.py test runner as the memory tests
should not be running for quick runs

Change-Id: I3c16c781e21b33deb939a64e77a6e0e41fb86922
- try adding the prints back in and re-test 2017-08-11T13:54:16+00:00 Mike Bayer mike_mp@zzzcomputing.com 2017-08-11T13:54:16+00:00 f1fd7f1e01bd9901fab7b5f33183e310bf0c0fd6 Change-Id: I3a82b06ecc3c065240cc05871dee2881f20e414e 
Change-Id: I3a82b06ecc3c065240cc05871dee2881f20e414e
- take more print statements out, maybe this is not 2017-08-10T19:46:37+00:00 Mike Bayer mike_mp@zzzcomputing.com 2017-08-10T19:44:54+00:00 3568e508d065d3ca9a17b11363249e8cf7c2a107 the issue - unblock pytest-xdist now that the upstream issue is fixed, maybe this old version is the issue Change-Id: I28dd7ae0872948a188651d42e2f4af60bcbafe81 
the issue
- unblock pytest-xdist now that the upstream issue is fixed,
maybe this old version is the issue

Change-Id: I28dd7ae0872948a188651d42e2f4af60bcbafe81
- dont print samples, this appears like it may be 2017-08-08T21:24:23+00:00 Mike Bayer mike_mp@zzzcomputing.com 2017-08-08T21:24:23+00:00 725298ea4bdd874990940ca93c406ccb533c732d itself causing the memory leak in conjunction with pytest-xdist Change-Id: Ia8704e54186e6dd60ea0e32a246fcf1419686663 
itself causing the memory leak in conjunction with pytest-xdist

Change-Id: Ia8704e54186e6dd60ea0e32a246fcf1419686663
- allow the shrink phase for memusage to go until zeroed, 2017-08-08T18:02:16+00:00 Mike Bayer mike_mp@zzzcomputing.com 2017-08-08T18:02:16+00:00 0a23654649e9b121001b24213bfa946409774e55 some MySQL-env element is causing memory growth that goes very far before stopping Change-Id: Ic0882dd78636067980fceba4e3a969de78d5b26a 
some MySQL-env element is causing memory growth that goes very
far before stopping

Change-Id: Ic0882dd78636067980fceba4e3a969de78d5b26a
Use baked lazyloading by default 2017-04-13T18:22:59+00:00 Mike Bayer mike_mp@zzzcomputing.com 2017-04-07T18:18:22+00:00 b7644319e85ce38c1a576802317a9058a6aed82d The ``lazy="select"`` loader strategy now makes used of the :class:`.BakedQuery` query caching system in all cases. This removes most overhead of generating a :class:`.Query` object and running it into a :func:`.select` and then string SQL statement from the process of lazy-loading related collections and objects. The "baked" lazy loader has also been improved such that it can now cache in most cases where query load options are used. Change-Id: Ic96792fffaa045ae9aa0a4657d6d29235d3efb85 Fixes: #3954 
The ``lazy="select"`` loader strategy now makes used of the
:class:`.BakedQuery` query caching system in all cases.  This
removes most overhead of generating a :class:`.Query` object and
running it into a :func:`.select` and then string SQL statement from
the process of lazy-loading related collections and objects.  The
"baked" lazy loader has also been improved such that it can now
cache in most cases where query load options are used.

Change-Id: Ic96792fffaa045ae9aa0a4657d6d29235d3efb85
Fixes: #3954
- try one more test, then we're likely going to give up on cx_Oracle 2017-04-12T19:28:05+00:00 Mike Bayer mike_mp@zzzcomputing.com 2017-04-12T19:28:05+00:00 bfaef1eaf54e0901b964ee163e179f6ec6cbd6e5 Change-Id: I7f9a1265664b0368ee7a771d01c7ca1612156d1f 
Change-Id: I7f9a1265664b0368ee7a771d01c7ca1612156d1f
- move a few memusage tests out of "backend". something is up w/ cx_Oracle 2017-04-12T19:15:16+00:00 Mike Bayer mike_mp@zzzcomputing.com 2017-04-12T19:15:16+00:00 b1f8d46fca951467ce0c05fe648f9f34de95c2ed when the suite runs, such as a background thread or something like that, which is affecting these tests a bit. Change-Id: I52d50a44778ec1eecb8e335ae59b1a4773e80a79 
when the suite runs, such as a background thread or something like that,
which is affecting these tests a bit.

Change-Id: I52d50a44778ec1eecb8e335ae59b1a4773e80a79
Warn on _compiled_cache growth 2017-04-12T16:53:40+00:00 Mike Bayer mike_mp@zzzcomputing.com 2017-04-12T15:37:19+00:00 cef4e5ff38dc7d2200800837c110ab6beec10d8a Added warnings to the LRU "compiled cache" used by the :class:`.Mapper` (and ultimately will be for other ORM-based LRU caches) such that when the cache starts hitting its size limits, the application will emit a warning that this is a performance-degrading situation that may require attention. The LRU caches can reach their size limits primarily if an application is making use of an unbounded number of :class:`.Engine` objects, which is an antipattern. Otherwise, this may suggest an issue that should be brought to the SQLAlchemy developer's attention. Additionally, adjusted the test_memusage algorithm again as the previous one could still allow a growing memory size to be missed. Change-Id: I020d1ceafb7a08f6addfa990a1e7acd09f933240 
Added warnings to the LRU "compiled cache" used by the :class:`.Mapper`
(and ultimately will be for other ORM-based LRU caches) such that
when the cache starts hitting its size limits, the application will
emit a warning that this is a performance-degrading situation that
may require attention.   The LRU caches can reach their size limits
primarily if an application is making use of an unbounded number
of :class:`.Engine` objects, which is an antipattern.  Otherwise,
this may suggest an issue that should be brought to the SQLAlchemy
developer's attention.

Additionally, adjusted the test_memusage algorithm again as the
previous one could still allow a growing memory size to be missed.

Change-Id: I020d1ceafb7a08f6addfa990a1e7acd09f933240
Don't cache savepoint identifiers 2017-03-06T22:20:06+00:00 Mike Bayer mike_mp@zzzcomputing.com 2017-03-06T17:26:01+00:00 f4c4f784cde8e51301b09f187d2f086bfae47453 Fixed bug in compiler where the string identifier of a savepoint would be cached in the identifier quoting dictionary; as these identifiers are arbitrary, a small memory leak could occur if a single :class:`.Connection` had an unbounded number of savepoints used, as well as if the savepoint clause constructs were used directly with an unbounded umber of savepoint names. The memory leak does **not** impact the vast majority of cases as normally the :class:`.Connection`, which renders savepoint names with a simple counter starting at "1", is used on a per-transaction or per-fixed-number-of-transactions basis before being discarded. The savepoint name in virtually all cases does not require quoting at all, however to support potential third party use cases the "check for quotes needed" logic is retained, at a small performance cost. Uncondtionally quoting the name is another option, but this would turn the name into a case sensitive name which runs the risk of poor interactions with existing deployments that may be looking at these names in other contexts. Change-Id: I6b53c96abf7fdf1840592bbca5da81347911844c Fixes: #3931 
Fixed bug in compiler where the string identifier of a savepoint would
be cached in the identifier quoting dictionary; as these identifiers
are arbitrary, a small memory leak could occur if a single
:class:`.Connection` had an unbounded number of savepoints used,
as well as if the savepoint clause constructs were used directly
with an unbounded umber of savepoint names.   The memory leak does
**not** impact the vast majority of cases as normally the
:class:`.Connection`, which renders savepoint names with a simple
counter starting at "1", is used on a per-transaction or
per-fixed-number-of-transactions basis before being discarded.

The savepoint name in virtually all cases does not require quoting
at all, however to support potential third party use cases
the "check for quotes needed" logic is retained, at a small
performance cost.   Uncondtionally quoting the name is another
option, but this would turn the name into a case sensitive name
which runs the risk of poor interactions with existing deployments
that may be looking at these names in other contexts.

Change-Id: I6b53c96abf7fdf1840592bbca5da81347911844c
Fixes: #3931