<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/python-packages/sqlalchemy.git/test/aaa_profiling, branch rel_2_0_6</title>
<subtitle>github.com: zzzeek/sqlalchemy.git
</subtitle>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/'/>
<entry>
<title>ensure single import per line</title>
<updated>2023-02-28T16:50:54+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2023-02-28T16:05:48+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=da70478eb2eafe9c76b836217371e029c3c820e3'/>
<id>da70478eb2eafe9c76b836217371e029c3c820e3</id>
<content type='text'>
This adds the very small plugin flake8-import-single which
will prevent us from having an import with more than one symbol
on a line.

Flake8 by itself prevents this pattern with E401:

import collections, os, sys

However does not do anything with this:

from sqlalchemy import Column, text

Both statements have the same issues generating merge artifacts
as well as presenting a manual decision to be made.   While
zimports generally cleans up such imports at the top level, we
don't enforce zimports / pre-commit use.

the plugin finds the same issue for imports that are inside of
test methods.   We shouldn't usually have imports in test methods
so most of them here are moved to be top level.

The version is pinned at 0.1.5; the project seems to have no
activity since 2019, however there are three 0.1.6dev releases
on pypi which stopped in September 2019, they seem to be
experiments with packaging.  The source for 0.1.5
is extremely simple and only reveals one method to flake8
(the run() method).

Change-Id: Icea894e43bad9c0b5d4feb5f49c6c666d6ea6aa1
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This adds the very small plugin flake8-import-single which
will prevent us from having an import with more than one symbol
on a line.

Flake8 by itself prevents this pattern with E401:

import collections, os, sys

However does not do anything with this:

from sqlalchemy import Column, text

Both statements have the same issues generating merge artifacts
as well as presenting a manual decision to be made.   While
zimports generally cleans up such imports at the top level, we
don't enforce zimports / pre-commit use.

the plugin finds the same issue for imports that are inside of
test methods.   We shouldn't usually have imports in test methods
so most of them here are moved to be top level.

The version is pinned at 0.1.5; the project seems to have no
activity since 2019, however there are three 0.1.6dev releases
on pypi which stopped in September 2019, they seem to be
experiments with packaging.  The source for 0.1.5
is extremely simple and only reveals one method to flake8
(the run() method).

Change-Id: Icea894e43bad9c0b5d4feb5f49c6c666d6ea6aa1
</pre>
</div>
</content>
</entry>
<entry>
<title>move more Session weakref tests to test_memusage</title>
<updated>2022-12-07T19:23:45+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2022-12-07T19:23:45+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=59f5beff1928e752b33d65a541cd68295ae0a5f1'/>
<id>59f5beff1928e752b33d65a541cd68295ae0a5f1</id>
<content type='text'>
py311 may be more sensitive here, or maybe the machines are
acting differently these days, in any case move
memory  / GC sensitive tests to test_memusage.

Change-Id: I218295150efc2f7ea88da9960ff10fda63dc60b1
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
py311 may be more sensitive here, or maybe the machines are
acting differently these days, in any case move
memory  / GC sensitive tests to test_memusage.

Change-Id: I218295150efc2f7ea88da9960ff10fda63dc60b1
</pre>
</div>
</content>
</entry>
<entry>
<title>adjustments for unreliable gc</title>
<updated>2022-12-05T04:31:20+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2022-12-05T04:25:14+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=ca46caede4b8d846f3cd48e642922ae821d0be2b'/>
<id>ca46caede4b8d846f3cd48e642922ae821d0be2b</id>
<content type='text'>
sporadic (and at the moment persistent) test failures
related to aiosqlite seem to have in common that Python
gc stops working fully when we run a lot of tests with
aiosqlite.  The failures are not limited to aiosqlite
as they are more involving places where we assume or
expect gc.collect() to get rid of things, and it doesn't.

Identify (based on reproducible case on the d3 CI runner)
the spots where this happens and add fixes.

test/orm/test_transaction.py test_gced_delete_on_rollback
has always been a very sensitive test with a lot of issues,
so here we move it to the test_memusage suite and limit
it only to when the memusage suite is running.

Change-Id: I683412d0effe8732c45980b40722e5bb63431177
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
sporadic (and at the moment persistent) test failures
related to aiosqlite seem to have in common that Python
gc stops working fully when we run a lot of tests with
aiosqlite.  The failures are not limited to aiosqlite
as they are more involving places where we assume or
expect gc.collect() to get rid of things, and it doesn't.

Identify (based on reproducible case on the d3 CI runner)
the spots where this happens and add fixes.

test/orm/test_transaction.py test_gced_delete_on_rollback
has always been a very sensitive test with a lot of issues,
so here we move it to the test_memusage suite and limit
it only to when the memusage suite is running.

Change-Id: I683412d0effe8732c45980b40722e5bb63431177
</pre>
</div>
</content>
</entry>
<entry>
<title>Try running pyupgrade on the code</title>
<updated>2022-11-16T22:03:04+00:00</updated>
<author>
<name>Federico Caselli</name>
<email>cfederico87@gmail.com</email>
</author>
<published>2022-11-03T19:52:21+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=4eb4ceca36c7ce931ea65ac06d6ed08bf459fc66'/>
<id>4eb4ceca36c7ce931ea65ac06d6ed08bf459fc66</id>
<content type='text'>
command run is "pyupgrade --py37-plus --keep-runtime-typing --keep-percent-format &lt;files...&gt;"
pyupgrade will change assert_ to assertTrue. That was reverted since assertTrue does not
exists in sqlalchemy fixtures

Change-Id: Ie1ed2675c7b11d893d78e028aad0d1576baebb55
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
command run is "pyupgrade --py37-plus --keep-runtime-typing --keep-percent-format &lt;files...&gt;"
pyupgrade will change assert_ to assertTrue. That was reverted since assertTrue does not
exists in sqlalchemy fixtures

Change-Id: Ie1ed2675c7b11d893d78e028aad0d1576baebb55
</pre>
</div>
</content>
</entry>
<entry>
<title>perf improvements related to corresponding_column (2)</title>
<updated>2022-11-15T19:16:06+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2022-11-14T02:48:53+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=d5be2cc1391d0ff4b21557b036eba4713fde7bcf'/>
<id>d5be2cc1391d0ff4b21557b036eba4713fde7bcf</id>
<content type='text'>
commit two of two. this reorganizes ColumnCollection
to build a new index up front that's used to optimize
the corresponding_column() method.

Additional performance enhancements within ORM-enabled SQL statements,
specifically targeting callcounts within the construction of ORM
statements, using combinations of :func:`_orm.aliased` with
:func:`_sql.union` and similar "compound" constructs, in addition to direct
performance improvements to the ``corresponding_column()`` internal method
that is used heavily by the ORM by constructs like :func:`_orm.aliased` and
similar.

Fixes: #8796
Change-Id: I4a76788007d5a802b9a4081e6a0f6e4b52497b50
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
commit two of two. this reorganizes ColumnCollection
to build a new index up front that's used to optimize
the corresponding_column() method.

Additional performance enhancements within ORM-enabled SQL statements,
specifically targeting callcounts within the construction of ORM
statements, using combinations of :func:`_orm.aliased` with
:func:`_sql.union` and similar "compound" constructs, in addition to direct
performance improvements to the ``corresponding_column()`` internal method
that is used heavily by the ORM by constructs like :func:`_orm.aliased` and
similar.

Fixes: #8796
Change-Id: I4a76788007d5a802b9a4081e6a0f6e4b52497b50
</pre>
</div>
</content>
</entry>
<entry>
<title>don't invoke fromclause.c when creating an annotated</title>
<updated>2022-11-15T18:46:02+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2022-11-13T16:49:43+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=93dc7ea1502c37793011b094447641361aff5aba'/>
<id>93dc7ea1502c37793011b094447641361aff5aba</id>
<content type='text'>
The ``aliased()`` constructor calls upon ``__clause_element__()``,
which internally annotates a ``FromClause``, like a subquery.
This became expensive as ``AnnotatedFromClause`` has for
many years called upon ``element.c`` so that the full ``.c``
collection is transferred to the Annotated.

Taking this out proved to be challenging. A straight remove
seemed to not break any tests except for the one that
tested the exact condition.  Nevertheless this seemed
"spooky" so I instead moved the get of ``.c`` to be in a
memoized proxy method.   However, that then exposed
a recursion issue related to loader_criteria; so the
source of that behavior, which was an accidental behavioral
artifact, is now made into an explcicit option that
loader_criteria uses directly.

The accidental behavioral artifact in question is still
kind of strange since I was not able to fully trace out
how it works, but the end result is that fixing the
artifact to be "correct" causes loader_criteria, within
the particular test for #7491, creates a select/
subquery structure with a cycle in it, so compilation fails
with recursion overflow.
The "solution" is to cause the artifact to occur in this
case, which is that the ``AnnotatedFromClause`` will have a
different ``.c`` collection than its element, which is a
subquery.  It's not totally clear how a cycle is generated
when this is not done.

This is commit one of two, which goes through
some hoops to make essentially a one-line change.

The next commit will rework ColumnCollection to optimize
the corresponding_column() method significantly.

Fixes: #8796
Change-Id: Id58ae6554db62139462c11a8be7313a3677456ad
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The ``aliased()`` constructor calls upon ``__clause_element__()``,
which internally annotates a ``FromClause``, like a subquery.
This became expensive as ``AnnotatedFromClause`` has for
many years called upon ``element.c`` so that the full ``.c``
collection is transferred to the Annotated.

Taking this out proved to be challenging. A straight remove
seemed to not break any tests except for the one that
tested the exact condition.  Nevertheless this seemed
"spooky" so I instead moved the get of ``.c`` to be in a
memoized proxy method.   However, that then exposed
a recursion issue related to loader_criteria; so the
source of that behavior, which was an accidental behavioral
artifact, is now made into an explcicit option that
loader_criteria uses directly.

The accidental behavioral artifact in question is still
kind of strange since I was not able to fully trace out
how it works, but the end result is that fixing the
artifact to be "correct" causes loader_criteria, within
the particular test for #7491, creates a select/
subquery structure with a cycle in it, so compilation fails
with recursion overflow.
The "solution" is to cause the artifact to occur in this
case, which is that the ``AnnotatedFromClause`` will have a
different ``.c`` collection than its element, which is a
subquery.  It's not totally clear how a cycle is generated
when this is not done.

This is commit one of two, which goes through
some hoops to make essentially a one-line change.

The next commit will rework ColumnCollection to optimize
the corresponding_column() method significantly.

Fixes: #8796
Change-Id: Id58ae6554db62139462c11a8be7313a3677456ad
</pre>
</div>
</content>
</entry>
<entry>
<title>Domain type</title>
<updated>2022-06-21T14:17:40+00:00</updated>
<author>
<name>David Baumgold</name>
<email>david@davidbaumgold.com</email>
</author>
<published>2022-02-11T17:30:24+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=017fd9ae0645eaf2a0fbdd067d10c721505b018c'/>
<id>017fd9ae0645eaf2a0fbdd067d10c721505b018c</id>
<content type='text'>
Added a new Postgresql :class:`_postgresql.DOMAIN` datatype, which follows
the same CREATE TYPE / DROP TYPE behaviors as that of PostgreSQL
:class:`_postgresql.ENUM`. Much thanks to David Baumgold for the efforts on
this.

Fixes: #7316
Closes: #7317
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7317
Pull-request-sha: bc9a82f010e6ca2f70a6e8a7620b748e483c26c3

Change-Id: Id8d7e48843a896de17d20cc466b115b3cc065132
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Added a new Postgresql :class:`_postgresql.DOMAIN` datatype, which follows
the same CREATE TYPE / DROP TYPE behaviors as that of PostgreSQL
:class:`_postgresql.ENUM`. Much thanks to David Baumgold for the efforts on
this.

Fixes: #7316
Closes: #7317
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7317
Pull-request-sha: bc9a82f010e6ca2f70a6e8a7620b748e483c26c3

Change-Id: Id8d7e48843a896de17d20cc466b115b3cc065132
</pre>
</div>
</content>
</entry>
<entry>
<title>update flake8 noqa skips with proper syntax</title>
<updated>2022-04-11T20:48:23+00:00</updated>
<author>
<name>Federico Caselli</name>
<email>cfederico87@gmail.com</email>
</author>
<published>2022-04-11T20:21:20+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=11bf82447438a9d5d9df9d613bf245694d6120dc'/>
<id>11bf82447438a9d5d9df9d613bf245694d6120dc</id>
<content type='text'>
Change-Id: I42ed77f559e3ee5b8c600d98457ee37803ef0ea6
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Change-Id: I42ed77f559e3ee5b8c600d98457ee37803ef0ea6
</pre>
</div>
</content>
</entry>
<entry>
<title>maintain complete cloned_set for BindParameter</title>
<updated>2022-04-06T14:01:43+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2022-04-06T13:41:11+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=2168a64affb2e299b9a37079af7b2a8d4ae0ff64'/>
<id>2168a64affb2e299b9a37079af7b2a8d4ae0ff64</id>
<content type='text'>
Fixed regression caused by :ticket:`7823` which impacted the caching
system, such that bound parameters that had been "cloned" within ORM
operations, such as polymorphic loading, would in some cases not acquire
their correct execution-time value leading to incorrect bind values being
rendered.

Fixes: #7903
Change-Id: I61c802749b859bebeb127d24e66d6e77d13ce57a
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Fixed regression caused by :ticket:`7823` which impacted the caching
system, such that bound parameters that had been "cloned" within ORM
operations, such as polymorphic loading, would in some cases not acquire
their correct execution-time value leading to incorrect bind values being
rendered.

Fixes: #7903
Change-Id: I61c802749b859bebeb127d24e66d6e77d13ce57a
</pre>
</div>
</content>
</entry>
<entry>
<title>pep484 - sql.selectable</title>
<updated>2022-04-04T13:26:43+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2022-03-30T22:01:58+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=3b4d62f4f72e8dfad7f38db192a6a90a8551608c'/>
<id>3b4d62f4f72e8dfad7f38db192a6a90a8551608c</id>
<content type='text'>
the pep484 task becomes more intense as there is mounting
pressure to come up with a consistency in how data moves
from end-user to instance variable.

current thinking is coming into:

1. there are _typing._XYZArgument objects that represent "what the
   user sent"
2. there's the roles, which represent a kind of "filter" for different
   kinds of objects.   These are mostly important as the argument
   we pass to coerce().
3. there's the thing that coerce() returns, which should be what the
   construct uses as its internal representation of the thing.
   This is _typing._XYZElement.

but there's some controversy over whether or
not we should pass actual ClauseElements around by their role
or not.   I think we shouldn't at the moment, but this makes the
"role-ness" of something a little less portable. Like, we have
to set DMLTableRole for TableClause, Join, and Alias, but then
also we have to repeat those three types in order to set up
_DMLTableElement.

Other change introduced here, there was a deannotate=True
for the left/right of a sql.join().    All tests pass without that.
I'd rather not have that there as if we have a join(A, B) where
A, B are mapped classes, we want them inside of the _annotations.
The rationale seems to be performance, but this performance can
be illustrated to be on the compile side which we hope is cached
in the normal case.

CTEs now accommodate for text selects including recursive.

Get typing to accommodate "util.preloaded" cleanly; add "preloaded"
as a real module.  This seemed like we would have needed
pep562 `__getattr__()` but we don't, just set names in
globals() as we import them.

References: #6810
Change-Id: I34d17f617de2fe2c086fc556bd55748dc782faf0
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
the pep484 task becomes more intense as there is mounting
pressure to come up with a consistency in how data moves
from end-user to instance variable.

current thinking is coming into:

1. there are _typing._XYZArgument objects that represent "what the
   user sent"
2. there's the roles, which represent a kind of "filter" for different
   kinds of objects.   These are mostly important as the argument
   we pass to coerce().
3. there's the thing that coerce() returns, which should be what the
   construct uses as its internal representation of the thing.
   This is _typing._XYZElement.

but there's some controversy over whether or
not we should pass actual ClauseElements around by their role
or not.   I think we shouldn't at the moment, but this makes the
"role-ness" of something a little less portable. Like, we have
to set DMLTableRole for TableClause, Join, and Alias, but then
also we have to repeat those three types in order to set up
_DMLTableElement.

Other change introduced here, there was a deannotate=True
for the left/right of a sql.join().    All tests pass without that.
I'd rather not have that there as if we have a join(A, B) where
A, B are mapped classes, we want them inside of the _annotations.
The rationale seems to be performance, but this performance can
be illustrated to be on the compile side which we hope is cached
in the normal case.

CTEs now accommodate for text selects including recursive.

Get typing to accommodate "util.preloaded" cleanly; add "preloaded"
as a real module.  This seemed like we would have needed
pep562 `__getattr__()` but we don't, just set names in
globals() as we import them.

References: #6810
Change-Id: I34d17f617de2fe2c086fc556bd55748dc782faf0
</pre>
</div>
</content>
</entry>
</feed>
