<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/python-packages/sqlalchemy.git/lib/sqlalchemy/testing/profiling.py, branch review/mike_bayer/tutorial20</title>
<subtitle>github.com: zzzeek/sqlalchemy.git
</subtitle>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/'/>
<entry>
<title>Create a framework to allow all SQLALCHEMY_WARN_20 to pass</title>
<updated>2020-09-16T16:31:05+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2020-08-17T21:24:27+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=7e864fc7b1b950760cbf02e6dcd5aa5aac267400'/>
<id>7e864fc7b1b950760cbf02e6dcd5aa5aac267400</id>
<content type='text'>
As the test suite has widespread use of many patterns
that are deprecated, enable SQLALCHEMY_WARN_20 globally
for the test suite but then break the warnings filter
out into a whole list of all the individual warnings
we are looking for.  this way individual changesets
can target a specific class of warning, as many of these
warnings will indivdidually affect dozens of files
and potentially hundreds of lines of code.

Many warnings are also resolved here as this
patch started out that way.   From this point
forward there should be changesets that target a
subset of the warnings at a time.

For expediency, updates some migration 2.0 docs
for ORM as well.

Change-Id: I98b8defdf7c37b818b3824d02f7668e3f5f31c94
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
As the test suite has widespread use of many patterns
that are deprecated, enable SQLALCHEMY_WARN_20 globally
for the test suite but then break the warnings filter
out into a whole list of all the individual warnings
we are looking for.  this way individual changesets
can target a specific class of warning, as many of these
warnings will indivdidually affect dozens of files
and potentially hundreds of lines of code.

Many warnings are also resolved here as this
patch started out that way.   From this point
forward there should be changesets that target a
subset of the warnings at a time.

For expediency, updates some migration 2.0 docs
for ORM as well.

Change-Id: I98b8defdf7c37b818b3824d02f7668e3f5f31c94
</pre>
</div>
</content>
</entry>
<entry>
<title>Add complete platform data to profiling data</title>
<updated>2020-08-07T16:05:45+00:00</updated>
<author>
<name>Federico Caselli</name>
<email>cfederico87@gmail.com</email>
</author>
<published>2020-07-27T19:57:32+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=24a9e6974c177e90d2213080b1f6d8270d78f69b'/>
<id>24a9e6974c177e90d2213080b1f6d8270d78f69b</id>
<content type='text'>
Initially to distinsuish between arm and x86_64
architecture, expand out the profile key to include
machine, system, python impl in all cases.

Ref: #5436
Change-Id: I7e48f0462ba7d9c680b2dac45ce7b0cf709b9b22
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Initially to distinsuish between arm and x86_64
architecture, expand out the profile key to include
machine, system, python impl in all cases.

Ref: #5436
Change-Id: I7e48f0462ba7d9c680b2dac45ce7b0cf709b9b22
</pre>
</div>
</content>
</entry>
<entry>
<title>Make call-count profiling tests on osx have their own platform key</title>
<updated>2020-07-11T17:10:26+00:00</updated>
<author>
<name>Federico Caselli</name>
<email>cfederico87@gmail.com</email>
</author>
<published>2020-07-11T17:10:26+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=cc638994aedf0ff7c0c4ab09bbb0a22196c9a716'/>
<id>cc638994aedf0ff7c0c4ab09bbb0a22196c9a716</id>
<content type='text'>
They previously would use the linux profiles, but recently some discrepancies in
the function call count on osx would make the tests fail.

Change-Id: Ifdfdca1676972de4179f59cdaae196f6805d4a21
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
They previously would use the linux profiles, but recently some discrepancies in
the function call count on osx would make the tests fail.

Change-Id: Ifdfdca1676972de4179f59cdaae196f6805d4a21
</pre>
</div>
</content>
</entry>
<entry>
<title>Fix a wide variety of typos and broken links</title>
<updated>2020-06-25T23:42:28+00:00</updated>
<author>
<name>aplatkouski</name>
<email>5857672+aplatkouski@users.noreply.github.com</email>
</author>
<published>2020-06-22T15:34:39+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=2a1a9f5f5a9723f757439657d2bdf224baed8748'/>
<id>2a1a9f5f5a9723f757439657d2bdf224baed8748</id>
<content type='text'>
Note the PR has a few remaining doc linking issues
listed in the comment that must be addressed separately.

Signed-off-by: aplatkouski &lt;5857672+aplatkouski@users.noreply.github.com&gt;
Closes: #5371
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5371
Pull-request-sha: 7e7d233cf3a0c66980c27db0fcdb3c7d93bc2510

Change-Id: I9c36e8d8804483950db4b42c38ee456e384c59e3
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Note the PR has a few remaining doc linking issues
listed in the comment that must be addressed separately.

Signed-off-by: aplatkouski &lt;5857672+aplatkouski@users.noreply.github.com&gt;
Closes: #5371
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5371
Pull-request-sha: 7e7d233cf3a0c66980c27db0fcdb3c7d93bc2510

Change-Id: I9c36e8d8804483950db4b42c38ee456e384c59e3
</pre>
</div>
</content>
</entry>
<entry>
<title>Add full profile sort, dumping to profile results</title>
<updated>2020-05-28T22:16:32+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2020-05-28T22:16:32+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=5e1d11573350f8035ed607e9c97b9f8896ab3132'/>
<id>5e1d11573350f8035ed607e9c97b9f8896ab3132</id>
<content type='text'>
Change-Id: Ib256ae34de15d29ee9a48e3be86073610f8d1a65
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Change-Id: Ib256ae34de15d29ee9a48e3be86073610f8d1a65
</pre>
</div>
</content>
</entry>
<entry>
<title>Convert execution to move through Session</title>
<updated>2020-05-25T17:56:37+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2020-04-27T16:58:12+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=6930dfc032c3f9f474e71ab4e021c0ef8384930e'/>
<id>6930dfc032c3f9f474e71ab4e021c0ef8384930e</id>
<content type='text'>
This patch replaces the ORM execution flow with a
single pathway through Session.execute() for all queries,
including Core and ORM.

Currently included is full support for ORM Query,
Query.from_statement(), select(), as well as the
baked query and horizontal shard systems.  Initial
changes have also been made to the dogpile caching
example, which like baked query makes use of a
new ORM-specific execution hook that replaces the
use of both QueryEvents.before_compile() as well
as Query._execute_and_instances() as the central
ORM interception hooks.

select() and Query() constructs alike can be passed to
Session.execute() where they will return ORM
results in a Results object.   This API is currently
used internally by Query.   Full support for
Session.execute()-&gt;results to behave in a fully
2.0 fashion will be in later changesets.

bulk update/delete with ORM support will also
be delivered via the update() and delete()
constructs, however these have not yet been adapted
to the new system and may follow in a subsequent
update.

Performance is also beginning to lag as of this
commit and some previous ones.   It is hoped that
a few central functions such as the coercions
functions can be rewritten in C to re-gain
performance.  Additionally, query caching
is now available and some subsequent patches
will attempt to cache more of the per-execution
work from the ORM layer, e.g. column getters
and adapters.

This patch also contains initial "turn on" of the
caching system enginewide via the query_cache_size
parameter to create_engine(). Still defaulting at
zero for "no caching".   The caching system still
needs adjustments in order to gain adequate performance.

Change-Id: I047a7ebb26aa85dc01f6789fac2bff561dcd555d
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This patch replaces the ORM execution flow with a
single pathway through Session.execute() for all queries,
including Core and ORM.

Currently included is full support for ORM Query,
Query.from_statement(), select(), as well as the
baked query and horizontal shard systems.  Initial
changes have also been made to the dogpile caching
example, which like baked query makes use of a
new ORM-specific execution hook that replaces the
use of both QueryEvents.before_compile() as well
as Query._execute_and_instances() as the central
ORM interception hooks.

select() and Query() constructs alike can be passed to
Session.execute() where they will return ORM
results in a Results object.   This API is currently
used internally by Query.   Full support for
Session.execute()-&gt;results to behave in a fully
2.0 fashion will be in later changesets.

bulk update/delete with ORM support will also
be delivered via the update() and delete()
constructs, however these have not yet been adapted
to the new system and may follow in a subsequent
update.

Performance is also beginning to lag as of this
commit and some previous ones.   It is hoped that
a few central functions such as the coercions
functions can be rewritten in C to re-gain
performance.  Additionally, query caching
is now available and some subsequent patches
will attempt to cache more of the per-execution
work from the ORM layer, e.g. column getters
and adapters.

This patch also contains initial "turn on" of the
caching system enginewide via the query_cache_size
parameter to create_engine(). Still defaulting at
zero for "no caching".   The caching system still
needs adjustments in order to gain adequate performance.

Change-Id: I047a7ebb26aa85dc01f6789fac2bff561dcd555d
</pre>
</div>
</content>
</entry>
<entry>
<title>Rename py.test to pytest</title>
<updated>2020-04-16T22:06:36+00:00</updated>
<author>
<name>Gord Thompson</name>
<email>gord@gordthompson.com</email>
</author>
<published>2020-04-16T22:06:36+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=405fc9717048b0adc852a72da540048df7a8142a'/>
<id>405fc9717048b0adc852a72da540048df7a8142a</id>
<content type='text'>
Change-Id: I431e1ef41e26d490343204a75a5c097768749768
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Change-Id: I431e1ef41e26d490343204a75a5c097768749768
</pre>
</div>
</content>
</entry>
<entry>
<title>Fix tests failing for SQLite file databases; repair provisioning</title>
<updated>2020-03-13T19:04:33+00:00</updated>
<author>
<name>Gord Thompson</name>
<email>gord@gordthompson.com</email>
</author>
<published>2020-02-26T19:50:01+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=79a53645ba8e6c12aff33d9ba1318ea4328ab7de'/>
<id>79a53645ba8e6c12aff33d9ba1318ea4328ab7de</id>
<content type='text'>
1. ensure provision.py loads dialect implementations when running
reap_dbs.py.   Reapers haven't been working since
598f2f7e557073f29563d4d567f43931fc03013f .

2. add some exclusion rules to allow the sqlite_file target to work;
add to tox.

3. add reap dbs target for SQLite, repair SQLite drop_db routine
which also wasn't doing the right thing for memory databases
etc.

4. Fix logging in provision files, as the main provision logger
is the one that's enabled by reap_dbs and maybe others, have all
the provision files use the provision logger.

Fixes: #5180
Fixes: #5168

Change-Id: Ibc1b0106394d20f5bcf847f37b09d185f26ac9b5
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
1. ensure provision.py loads dialect implementations when running
reap_dbs.py.   Reapers haven't been working since
598f2f7e557073f29563d4d567f43931fc03013f .

2. add some exclusion rules to allow the sqlite_file target to work;
add to tox.

3. add reap dbs target for SQLite, repair SQLite drop_db routine
which also wasn't doing the right thing for memory databases
etc.

4. Fix logging in provision files, as the main provision logger
is the one that's enabled by reap_dbs and maybe others, have all
the provision files use the provision logger.

Fixes: #5180
Fixes: #5168

Change-Id: Ibc1b0106394d20f5bcf847f37b09d185f26ac9b5
</pre>
</div>
</content>
</entry>
<entry>
<title>Result initial introduction</title>
<updated>2020-02-21T22:53:33+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2019-06-04T21:29:20+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=f559f378c47811b5528ad1769cb86925e85fd1e5'/>
<id>f559f378c47811b5528ad1769cb86925e85fd1e5</id>
<content type='text'>
This builds on cc718cccc0bf8a01abdf4068c7ea4f3 which moved
RowProxy to Row, allowing Row to be more like a named tuple.

- KeyedTuple in ORM is replaced with Row

- ResultSetMetaData broken out into "simple" and "cursor" versions
  for ORM and Core, as well as LegacyCursor version.

- Row now has _mapping attribute that supplies full mapping behavior.
Row and SimpleRow both have named tuple behavior otherwise.
LegacyRow has some mapping features on the tuple which emit
deprecation warnings (e.g. keys(), values(), etc).   the biggest
change for mapping-&gt;tuple is the behavior of __contains__ which
moves from testing of "key in row" to "value in row".

- ResultProxy breaks into ResultProxy and FutureResult (interim),
the latter has the newer APIs.   Made available to dialects
using execution options.

- internal reflection methods and most tests move off of implicit
Row mapping behavior and move to row._mapping, result.mappings()
method using future result

- a new strategy system for cursor handling replaces the various
subclasses of RowProxy

- some execution context adjustments. We will leave EC in but
refined things like get_result_proxy() and out parameter handling.
Dialects for 1.4 will need to adjust from get_result_proxy()
to get_result_cursor_strategy(), if they are using this method

- out parameter handling now accommodated by get_out_parameter_values()
EC method.   Oracle changes for this.  external dialect for
DB2 for example will also need to adjust for this.

- deprecate case_insensitive flag for engine / result, this
feature is not used

mapping-methods on Row are deprecated, and replaced with
Row._mapping.&lt;meth&gt;, including:

   row.keys()  -&gt; use row._mapping.keys()
   row.items()  -&gt; use row._mapping.items()
   row.values() -&gt; use row._mapping.values()
   key in row  -&gt; use key in row._mapping
   int in row  -&gt; use int &lt; len(row)

Fixes: #4710
Fixes: #4878
Change-Id: Ieb9085e9bcff564359095b754da9ae0af55679f0
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This builds on cc718cccc0bf8a01abdf4068c7ea4f3 which moved
RowProxy to Row, allowing Row to be more like a named tuple.

- KeyedTuple in ORM is replaced with Row

- ResultSetMetaData broken out into "simple" and "cursor" versions
  for ORM and Core, as well as LegacyCursor version.

- Row now has _mapping attribute that supplies full mapping behavior.
Row and SimpleRow both have named tuple behavior otherwise.
LegacyRow has some mapping features on the tuple which emit
deprecation warnings (e.g. keys(), values(), etc).   the biggest
change for mapping-&gt;tuple is the behavior of __contains__ which
moves from testing of "key in row" to "value in row".

- ResultProxy breaks into ResultProxy and FutureResult (interim),
the latter has the newer APIs.   Made available to dialects
using execution options.

- internal reflection methods and most tests move off of implicit
Row mapping behavior and move to row._mapping, result.mappings()
method using future result

- a new strategy system for cursor handling replaces the various
subclasses of RowProxy

- some execution context adjustments. We will leave EC in but
refined things like get_result_proxy() and out parameter handling.
Dialects for 1.4 will need to adjust from get_result_proxy()
to get_result_cursor_strategy(), if they are using this method

- out parameter handling now accommodated by get_out_parameter_values()
EC method.   Oracle changes for this.  external dialect for
DB2 for example will also need to adjust for this.

- deprecate case_insensitive flag for engine / result, this
feature is not used

mapping-methods on Row are deprecated, and replaced with
Row._mapping.&lt;meth&gt;, including:

   row.keys()  -&gt; use row._mapping.keys()
   row.items()  -&gt; use row._mapping.items()
   row.values() -&gt; use row._mapping.values()
   key in row  -&gt; use key in row._mapping
   int in row  -&gt; use int &lt; len(row)

Fixes: #4710
Fixes: #4878
Change-Id: Ieb9085e9bcff564359095b754da9ae0af55679f0
</pre>
</div>
</content>
</entry>
<entry>
<title>Limit non-backend critical profiling tests to SQLite</title>
<updated>2020-02-17T19:41:21+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2020-02-17T16:51:33+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=386012597b9e7aa2a7f987930d00b892ed54121d'/>
<id>386012597b9e7aa2a7f987930d00b892ed54121d</id>
<content type='text'>
issues with backend-specific profiling should be limited
to tests that are explcitly against resultset, compiler, etc.

MySQL in particular has an often varying callcount that isn't
worth running these tests against nor is it worth profiling
them for other backends like Oracle and SQL Server.

Also add the REQUIRE_SQLALCHEMY_CEXT flag to
the regen_callcounts.tox.ini script, which is part of some review
somewhere but is needed here to generate callcounts correctly.

Add a "warmup" phase for some of the ORM tests for join conditions
that have varying profile counts based on whether mappings have been
used already or not; profiling should always be against the
"warmed up" version of a function.

Change-Id: If483820235fa4cc4360cbd067a9b68d83512d587
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
issues with backend-specific profiling should be limited
to tests that are explcitly against resultset, compiler, etc.

MySQL in particular has an often varying callcount that isn't
worth running these tests against nor is it worth profiling
them for other backends like Oracle and SQL Server.

Also add the REQUIRE_SQLALCHEMY_CEXT flag to
the regen_callcounts.tox.ini script, which is part of some review
somewhere but is needed here to generate callcounts correctly.

Add a "warmup" phase for some of the ORM tests for join conditions
that have varying profile counts based on whether mappings have been
used already or not; profiling should always be against the
"warmed up" version of a function.

Change-Id: If483820235fa4cc4360cbd067a9b68d83512d587
</pre>
</div>
</content>
</entry>
</feed>
