<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/python-packages/sqlalchemy.git/examples/performance/__init__.py, branch 2020_tutorial</title>
<subtitle>github.com: zzzeek/sqlalchemy.git
</subtitle>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/'/>
<entry>
<title>callcount reductions and refinement for cached queries</title>
<updated>2020-05-28T18:38:56+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2020-05-26T02:36:44+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=77f1b7d236dba6b1c859bb428ef32d118ec372e6'/>
<id>77f1b7d236dba6b1c859bb428ef32d118ec372e6</id>
<content type='text'>
This commit includes that we've removed the "_orm_query"
attribute from compile state as well as query context.
The attribute created reference cycles and also added
method call overhead.    As part of this change,
the interface for ORMExecuteState changes a bit, as well
as the interface for the horizontal sharding extension
which now deprecates the "query_chooser" callable
in favor of "execute_chooser", which receives the contextual
object.  This will also work more nicely when we implement
the new execution path for bulk updates and deletes.

Pre-merge execution options for statement, connection,
arguments all up front in Connection.  that way they
can be passed to the before_execute / after_execute events,
and the ExecutionContext doesn't have to merge as second
time.   Core execute is pretty close to 1.3 now.

baked wasn't using the new one()/first()/one_or_none() methods,
fixed that.

Convert non-buffered cursor strategy to be a stateless
singleton.  inline all the paths by which the strategy
gets chosen, oracle and SQL Server dialects make use of the
already-invoked post_exec() hook to establish the alternate
strategies, and this is actually much nicer than it was before.

Add caching to mapper instance processor for getters.

Identified a reference cycle per query that was showing
up as a lot of gc cleanup, fixed that.

After all that, performance not budging much.  Even
test_baked_query now runs with significantly fewer function
calls than 1.3, still 40% slower.

Basically something about the new patterns just makes
this slower and while I've walked a whole bunch of them
back, it hardly makes a dent.  that said, the performance
issues are relatively small, in the 20-40% time increase
range, and the new caching feature
does provide for regular ORM and Core queries that
are cached, and they are faster than non-cached.

Change-Id: I7b0b0d8ca550c05f79e82f75cd8eff0bbfade053
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This commit includes that we've removed the "_orm_query"
attribute from compile state as well as query context.
The attribute created reference cycles and also added
method call overhead.    As part of this change,
the interface for ORMExecuteState changes a bit, as well
as the interface for the horizontal sharding extension
which now deprecates the "query_chooser" callable
in favor of "execute_chooser", which receives the contextual
object.  This will also work more nicely when we implement
the new execution path for bulk updates and deletes.

Pre-merge execution options for statement, connection,
arguments all up front in Connection.  that way they
can be passed to the before_execute / after_execute events,
and the ExecutionContext doesn't have to merge as second
time.   Core execute is pretty close to 1.3 now.

baked wasn't using the new one()/first()/one_or_none() methods,
fixed that.

Convert non-buffered cursor strategy to be a stateless
singleton.  inline all the paths by which the strategy
gets chosen, oracle and SQL Server dialects make use of the
already-invoked post_exec() hook to establish the alternate
strategies, and this is actually much nicer than it was before.

Add caching to mapper instance processor for getters.

Identified a reference cycle per query that was showing
up as a lot of gc cleanup, fixed that.

After all that, performance not budging much.  Even
test_baked_query now runs with significantly fewer function
calls than 1.3, still 40% slower.

Basically something about the new patterns just makes
this slower and while I've walked a whole bunch of them
back, it hardly makes a dent.  that said, the performance
issues are relatively small, in the 20-40% time increase
range, and the new caching feature
does provide for regular ORM and Core queries that
are cached, and they are faster than non-cached.

Change-Id: I7b0b0d8ca550c05f79e82f75cd8eff0bbfade053
</pre>
</div>
</content>
</entry>
<entry>
<title>Updates for examples / performance</title>
<updated>2020-05-26T03:22:48+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2020-05-26T03:22:48+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=6b3513f56c2a5d3ba45215b2438c47dba4336740'/>
<id>6b3513f56c2a5d3ba45215b2438c47dba4336740</id>
<content type='text'>
Change-Id: I55fc60258075f6f3efb7f738b900f60b2d2994c4
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Change-Id: I55fc60258075f6f3efb7f738b900f60b2d2994c4
</pre>
</div>
</content>
</entry>
<entry>
<title>Performance fixes for new result set</title>
<updated>2020-05-21T18:16:03+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2020-05-20T17:41:44+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=4550983e0ce2f35b3585e53894c941c23693e71d'/>
<id>4550983e0ce2f35b3585e53894c941c23693e71d</id>
<content type='text'>
A few small mistakes led to huge callcounts.   Additionally,
the warn-on-get behavior which is attempting to warn for
deprecated access in SQLAlchemy 2.0 is very expensive; it's not clear
if its feasible to have this warning or to somehow alter how it
works.

Fixes: #5340
Change-Id: I73bdd2d7b6f1b25cc0222accabd585cf761a5af4
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
A few small mistakes led to huge callcounts.   Additionally,
the warn-on-get behavior which is attempting to warn for
deprecated access in SQLAlchemy 2.0 is very expensive; it's not clear
if its feasible to have this warning or to somehow alter how it
works.

Fixes: #5340
Change-Id: I73bdd2d7b6f1b25cc0222accabd585cf761a5af4
</pre>
</div>
</content>
</entry>
<entry>
<title>Enable F841</title>
<updated>2019-06-20T17:50:41+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2019-05-30T15:31:03+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=190e0139e834e4271268652e058c280787ae69eb'/>
<id>190e0139e834e4271268652e058c280787ae69eb</id>
<content type='text'>
This is a very useful assertion which prevents unused variables
from being set up allows code to be more readable and sometimes
even more efficient.  test suites seem to be where the most
problems are and there do not seem to be documentation examples
that are using this, or at least the linter is not taking effect
within rst blocks.

Change-Id: I2b3341d8dd14da34879d8425838e66a4b9f8e27d
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This is a very useful assertion which prevents unused variables
from being set up allows code to be more readable and sometimes
even more efficient.  test suites seem to be where the most
problems are and there do not seem to be documentation examples
that are using this, or at least the linter is not taking effect
within rst blocks.

Change-Id: I2b3341d8dd14da34879d8425838e66a4b9f8e27d
</pre>
</div>
</content>
</entry>
<entry>
<title>Post black reformatting</title>
<updated>2019-01-06T23:23:11+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2019-01-06T06:19:47+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=1e278de4cc9a4181e0747640a960e80efcea1ca9'/>
<id>1e278de4cc9a4181e0747640a960e80efcea1ca9</id>
<content type='text'>
Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9,  this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.

Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.

Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9,  this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.

Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.

Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
</pre>
</div>
</content>
</entry>
<entry>
<title>Run black -l 79 against all source files</title>
<updated>2019-01-06T17:34:50+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2019-01-06T06:14:26+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=1e1a38e7801f410f244e4bbb44ec795ae152e04e'/>
<id>1e1a38e7801f410f244e4bbb44ec795ae152e04e</id>
<content type='text'>
This is a straight reformat run using black as is, with no edits
applied at all.

The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines.   The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.

Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This is a straight reformat run using black as is, with no edits
applied at all.

The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines.   The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.

Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
</pre>
</div>
</content>
</entry>
<entry>
<title>- Added a new extension suite :mod:`sqlalchemy.ext.baked`.  This</title>
<updated>2015-03-12T00:31:11+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2015-03-12T00:22:42+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=e3b46bd62405b6ff57119e164718118f3e3565e0'/>
<id>e3b46bd62405b6ff57119e164718118f3e3565e0</id>
<content type='text'>
simple but unusual system allows for a dramatic savings in Python
overhead for the construction and processing of orm :class:`.Query`
objects, from query construction up through rendering of a string
SQL statement.
fixes #3054
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
simple but unusual system allows for a dramatic savings in Python
overhead for the construction and processing of orm :class:`.Query`
objects, from query construction up through rendering of a string
SQL statement.
fixes #3054
</pre>
</div>
</content>
</entry>
<entry>
<title>- add a callers option</title>
<updated>2015-03-09T19:19:49+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2015-03-09T19:19:49+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=611d88d234319b730b521734418ff7e9aafb1716'/>
<id>611d88d234319b730b521734418ff7e9aafb1716</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>- pick around gaining modest dings in callcounts here and there</title>
<updated>2015-03-09T18:36:23+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2015-03-09T18:36:23+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=9afe585f4599d8114abe8d11f924c206a8962cda'/>
<id>9afe585f4599d8114abe8d11f924c206a8962cda</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>- random performance whacking vs. 0.9, in particular we have to watch</title>
<updated>2015-03-08T19:32:21+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2015-03-08T19:32:21+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=a421106c9d1d660af7c5d9aba5928dda20c950e1'/>
<id>a421106c9d1d660af7c5d9aba5928dda20c950e1</id>
<content type='text'>
for the slots-based __getattr__ thing getting hit
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
for the slots-based __getattr__ thing getting hit
</pre>
</div>
</content>
</entry>
</feed>
