<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/python-packages/sqlalchemy.git/test/orm/test_bulk.py, branch main</title>
<subtitle>github.com: zzzeek/sqlalchemy.git
</subtitle>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/'/>
<entry>
<title>ORM bulk insert via execute</title>
<updated>2022-09-24T15:18:01+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2022-08-07T16:14:19+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=a8029f5a7e3e376ec57f1614ab0294b717d53c05'/>
<id>a8029f5a7e3e376ec57f1614ab0294b717d53c05</id>
<content type='text'>
* ORM Insert now includes "bulk" mode that will run
  essentially the same process as session.bulk_insert_mappings;
  interprets the given list of values as ORM attributes for
  key names
* ORM UPDATE has a similar feature, without RETURNING support,
  for session.bulk_update_mappings
* Added support for upserts to do RETURNING ORM objects as well
* ORM UPDATE/DELETE with list of parameters + WHERE criteria
  is a not implemented; use connection
* ORM UPDATE/DELETE defaults to "auto" synchronize_session;
  use fetch if RETURNING is present, evaluate if not, as
  "fetch" is much more efficient (no expired object SELECT problem)
  and less error prone if RETURNING is available
  UPDATE: howver this is inefficient!   please continue to
  use evaluate for simple cases, auto can move to fetch
  if criteria not evaluable
* "Evaluate" criteria will now not preemptively
  unexpire and SELECT attributes that were individually
  expired. Instead, if evaluation of the criteria indicates that
  the necessary attrs were expired, we expire the object
  completely (delete) or expire the SET attrs unconditionally
  (update). This keeps the object in the same unloaded state
  where it will refresh those attrs on the next pass, for
  this generally unusual case.  (originally #5664)
* Core change! update/delete rowcount comes from len(rows)
  if RETURNING was used.  SQLite at least otherwise did not
  support this.  adjusted test_rowcount accordingly
* ORM DELETE with a list of parameters at all is also a not
  implemented as this would imply "bulk", and there is no
  bulk_delete_mappings (could be, but we dont have that)
* ORM insert().values() with single or multi-values translates
  key names based on ORM attribute names
* ORM returning() implemented for insert, update, delete;
  explcit returning clauses now interpret rows in an ORM
  context, with support for qualifying loader options as well
* session.bulk_insert_mappings() assigns polymorphic identity
  if not set.
* explicit RETURNING + synchronize_session='fetch' is now
  supported with UPDATE and DELETE.
* expanded return_defaults() to work with DELETE also.
* added support for composite attributes to be present
  in the dictionaries used by bulk_insert_mappings and
  bulk_update_mappings, which is also the new ORM bulk
  insert/update feature, that will expand the composite
  values into their individual mapped attributes the way they'd
  be on a mapped instance.
* bulk UPDATE supports "synchronize_session=evaluate", is the
  default.  this does not apply to session.bulk_update_mappings,
  just the new version
* both bulk UPDATE and bulk INSERT, the latter with or without
  RETURNING, support *heterogenous* parameter sets.
  session.bulk_insert/update_mappings did this, so this feature
  is maintained.  now cursor result can be both horizontally
  and vertically spliced :)

This is now a long story with a lot of options, which in
itself is a problem to be able to document all of this
in some way that makes sense.  raising exceptions for
use cases we haven't supported is pretty important here
too, the tradition of letting unsupported things just not work
is likely not a good idea at this point, though there
are still many cases that aren't easily avoidable

Fixes: #8360
Fixes: #7864
Fixes: #7865
Change-Id: Idf28379f8705e403a3c6a937f6a798a042ef2540
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* ORM Insert now includes "bulk" mode that will run
  essentially the same process as session.bulk_insert_mappings;
  interprets the given list of values as ORM attributes for
  key names
* ORM UPDATE has a similar feature, without RETURNING support,
  for session.bulk_update_mappings
* Added support for upserts to do RETURNING ORM objects as well
* ORM UPDATE/DELETE with list of parameters + WHERE criteria
  is a not implemented; use connection
* ORM UPDATE/DELETE defaults to "auto" synchronize_session;
  use fetch if RETURNING is present, evaluate if not, as
  "fetch" is much more efficient (no expired object SELECT problem)
  and less error prone if RETURNING is available
  UPDATE: howver this is inefficient!   please continue to
  use evaluate for simple cases, auto can move to fetch
  if criteria not evaluable
* "Evaluate" criteria will now not preemptively
  unexpire and SELECT attributes that were individually
  expired. Instead, if evaluation of the criteria indicates that
  the necessary attrs were expired, we expire the object
  completely (delete) or expire the SET attrs unconditionally
  (update). This keeps the object in the same unloaded state
  where it will refresh those attrs on the next pass, for
  this generally unusual case.  (originally #5664)
* Core change! update/delete rowcount comes from len(rows)
  if RETURNING was used.  SQLite at least otherwise did not
  support this.  adjusted test_rowcount accordingly
* ORM DELETE with a list of parameters at all is also a not
  implemented as this would imply "bulk", and there is no
  bulk_delete_mappings (could be, but we dont have that)
* ORM insert().values() with single or multi-values translates
  key names based on ORM attribute names
* ORM returning() implemented for insert, update, delete;
  explcit returning clauses now interpret rows in an ORM
  context, with support for qualifying loader options as well
* session.bulk_insert_mappings() assigns polymorphic identity
  if not set.
* explicit RETURNING + synchronize_session='fetch' is now
  supported with UPDATE and DELETE.
* expanded return_defaults() to work with DELETE also.
* added support for composite attributes to be present
  in the dictionaries used by bulk_insert_mappings and
  bulk_update_mappings, which is also the new ORM bulk
  insert/update feature, that will expand the composite
  values into their individual mapped attributes the way they'd
  be on a mapped instance.
* bulk UPDATE supports "synchronize_session=evaluate", is the
  default.  this does not apply to session.bulk_update_mappings,
  just the new version
* both bulk UPDATE and bulk INSERT, the latter with or without
  RETURNING, support *heterogenous* parameter sets.
  session.bulk_insert/update_mappings did this, so this feature
  is maintained.  now cursor result can be both horizontally
  and vertically spliced :)

This is now a long story with a lot of options, which in
itself is a problem to be able to document all of this
in some way that makes sense.  raising exceptions for
use cases we haven't supported is pretty important here
too, the tradition of letting unsupported things just not work
is likely not a good idea at this point, though there
are still many cases that aren't easily avoidable

Fixes: #8360
Fixes: #7864
Fixes: #7865
Change-Id: Idf28379f8705e403a3c6a937f6a798a042ef2540
</pre>
</div>
</content>
</entry>
<entry>
<title>Modernize tests - session_query_get</title>
<updated>2021-10-29T00:31:09+00:00</updated>
<author>
<name>Gord Thompson</name>
<email>gord@gordthompson.com</email>
</author>
<published>2021-09-30T23:39:05+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=6a786297014c7b2a381084299ac3b4aabcf3c318'/>
<id>6a786297014c7b2a381084299ac3b4aabcf3c318</id>
<content type='text'>
Co-authored-by: Mike Bayer &lt;mike_mp@zzzcomputing.com&gt;
Change-Id: I92013aad471baf32df1b51b756e86d95449b5cfd
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Co-authored-by: Mike Bayer &lt;mike_mp@zzzcomputing.com&gt;
Change-Id: I92013aad471baf32df1b51b756e86d95449b5cfd
</pre>
</div>
</content>
</entry>
<entry>
<title>Modernize tests - calling_mapper_directly</title>
<updated>2021-09-30T14:10:16+00:00</updated>
<author>
<name>Gord Thompson</name>
<email>gord@gordthompson.com</email>
</author>
<published>2021-08-12T19:04:28+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=257f9130c321aaa948690d0e49c7352ad1188abd'/>
<id>257f9130c321aaa948690d0e49c7352ad1188abd</id>
<content type='text'>
a few changes for py2k:

* map_imperatively() includes the check that a class
  is being sent, this was only working for mapper() before

* the test suite didn't place the py2k "autouse" workaround
  in the correct order, seemingly, tried to adjust the
  per-test ordering setup in pytestplugin.py

Change-Id: I4cc39630724e810953cfda7b2afdadc8b948e3c2
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
a few changes for py2k:

* map_imperatively() includes the check that a class
  is being sent, this was only working for mapper() before

* the test suite didn't place the py2k "autouse" workaround
  in the correct order, seemingly, tried to adjust the
  per-test ordering setup in pytestplugin.py

Change-Id: I4cc39630724e810953cfda7b2afdadc8b948e3c2
</pre>
</div>
</content>
</entry>
<entry>
<title>implement cache key for return_defaults token</title>
<updated>2021-07-21T17:57:22+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2021-07-21T15:18:01+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=a34a4af8a80f4edd12b022753b69065025818e20'/>
<id>a34a4af8a80f4edd12b022753b69065025818e20</id>
<content type='text'>
Fixed critical caching issue where the ORM's persistence feature using
INSERT..RETURNING would cache an incorrect query when mixing the "bulk
save" and standard "flush" forms of INSERT.

Fixes: #6793
Change-Id: Ifeb61c1226d3fa6d5e1c2e29b6f5ff77a27d6a2d
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Fixed critical caching issue where the ORM's persistence feature using
INSERT..RETURNING would cache an incorrect query when mixing the "bulk
save" and standard "flush" forms of INSERT.

Fixes: #6793
Change-Id: Ifeb61c1226d3fa6d5e1c2e29b6f5ff77a27d6a2d
</pre>
</div>
</content>
</entry>
<entry>
<title>fix mapper._primary_key_propkeys</title>
<updated>2021-04-29T18:20:27+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2021-04-29T16:03:47+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=7892cc30ad60ae450471b62f351b3beb66911892'/>
<id>7892cc30ad60ae450471b62f351b3beb66911892</id>
<content type='text'>
Fixed issue in :meth:`_orm.Session.bulk_save` when used with persistent
objects which would fail to track the primary key of mappings where the
column name of the primary key were different than the attribute name.

Fixes: #6392
Change-Id: I9b89bf00f900b7d2287517249e4f9356f20f0bdb
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Fixed issue in :meth:`_orm.Session.bulk_save` when used with persistent
objects which would fail to track the primary key of mappings where the
column name of the primary key were different than the attribute name.

Fixes: #6392
Change-Id: I9b89bf00f900b7d2287517249e4f9356f20f0bdb
</pre>
</div>
</content>
</entry>
<entry>
<title>remove more bound metadata</title>
<updated>2021-01-06T04:46:02+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2021-01-04T20:18:25+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=6fccdf4a285d5332ef49f23dc18c3ce45501d78b'/>
<id>6fccdf4a285d5332ef49f23dc18c3ce45501d78b</id>
<content type='text'>
in Iae6ab95938a7e92b6d42086aec534af27b5577d3 I missed
that the "bind" was being stuck onto the MetaData in
TablesTest, which led thousands of ORM tests to still use
bound metadata.  Keep looking for bound metadata.

standardize all ORM tests on a single means of getting a
Session when the Session API isn't the thing we are directly
testing, using a new function fixture_session() that replaces
create_session() and uses modern defaults.

Change-Id: Iaf71206e9ee568151496d8bc213a069504bf65ef
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
in Iae6ab95938a7e92b6d42086aec534af27b5577d3 I missed
that the "bind" was being stuck onto the MetaData in
TablesTest, which led thousands of ORM tests to still use
bound metadata.  Keep looking for bound metadata.

standardize all ORM tests on a single means of getting a
Session when the Session API isn't the thing we are directly
testing, using a new function fixture_session() that replaces
create_session() and uses modern defaults.

Change-Id: Iaf71206e9ee568151496d8bc213a069504bf65ef
</pre>
</div>
</content>
</entry>
<entry>
<title>ORM executemany returning</title>
<updated>2020-06-28T01:30:37+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2020-04-17T14:55:08+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=08c46eea924d23a234bf3feea1a928eb8ae8a00a'/>
<id>08c46eea924d23a234bf3feea1a928eb8ae8a00a</id>
<content type='text'>
Build on #5401 to allow the ORM to take advanage
of executemany INSERT + RETURNING.

Implemented the feature

updated tests

to support INSERT DEFAULT VALUES, needed to come up with
a new syntax for compiler INSERT INTO table (anycol) VALUES (DEFAULT)
which can then be iterated out for executemany.

Added graceful degrade to plain executemany for PostgreSQL &lt;= 8.2

Renamed EXECUTEMANY_DEFAULT to EXECUTEMANY_PLAIN

Fix issue where unicode identifiers or parameter names wouldn't
work with execute_values() under Py2K, because we have to
encode the statement and therefore have to encode the
insert_single_values_expr too.

Correct issue from #5401 to support executemany + return_defaults
for a PK that is explicitly pre-generated, meaning we aren't actually
getting RETURNING but need to return it from compiled_parameters.

Fixes: #5263
Change-Id: Id68e5c158c4f9ebc33b61c06a448907921c2a657
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Build on #5401 to allow the ORM to take advanage
of executemany INSERT + RETURNING.

Implemented the feature

updated tests

to support INSERT DEFAULT VALUES, needed to come up with
a new syntax for compiler INSERT INTO table (anycol) VALUES (DEFAULT)
which can then be iterated out for executemany.

Added graceful degrade to plain executemany for PostgreSQL &lt;= 8.2

Renamed EXECUTEMANY_DEFAULT to EXECUTEMANY_PLAIN

Fix issue where unicode identifiers or parameter names wouldn't
work with execute_values() under Py2K, because we have to
encode the statement and therefore have to encode the
insert_single_values_expr too.

Correct issue from #5401 to support executemany + return_defaults
for a PK that is explicitly pre-generated, meaning we aren't actually
getting RETURNING but need to return it from compiled_parameters.

Fixes: #5263
Change-Id: Id68e5c158c4f9ebc33b61c06a448907921c2a657
</pre>
</div>
</content>
</entry>
<entry>
<title>Add pyproject</title>
<updated>2020-01-27T21:56:25+00:00</updated>
<author>
<name>Federico Caselli</name>
<email>cfederico87@gmail.com</email>
</author>
<published>2020-01-27T21:51:05+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=bfddff4c9f72adc5d326cee9783c77d358254837'/>
<id>bfddff4c9f72adc5d326cee9783c77d358254837</id>
<content type='text'>
- Added pyproject.toml with black arguments
- Updated black version in precommit hook
- Reformatted the code

Fixes: #5100
Closes: #5103
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5103
Pull-request-sha: 795fd5f896be4a07a2b18e6525674b815ac17593

Change-Id: I14eedbaa51fb531cbf90fcefe6a1e07c8a565625
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- Added pyproject.toml with black arguments
- Updated black version in precommit hook
- Reformatted the code

Fixes: #5100
Closes: #5103
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5103
Pull-request-sha: 795fd5f896be4a07a2b18e6525674b815ac17593

Change-Id: I14eedbaa51fb531cbf90fcefe6a1e07c8a565625
</pre>
</div>
</content>
</entry>
<entry>
<title>Post black reformatting</title>
<updated>2019-01-06T23:23:11+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2019-01-06T06:19:47+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=1e278de4cc9a4181e0747640a960e80efcea1ca9'/>
<id>1e278de4cc9a4181e0747640a960e80efcea1ca9</id>
<content type='text'>
Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9,  this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.

Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.

Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9,  this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.

Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.

Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
</pre>
</div>
</content>
</entry>
<entry>
<title>Run black -l 79 against all source files</title>
<updated>2019-01-06T17:34:50+00:00</updated>
<author>
<name>Mike Bayer</name>
<email>mike_mp@zzzcomputing.com</email>
</author>
<published>2019-01-06T06:14:26+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/python-packages/sqlalchemy.git/commit/?id=1e1a38e7801f410f244e4bbb44ec795ae152e04e'/>
<id>1e1a38e7801f410f244e4bbb44ec795ae152e04e</id>
<content type='text'>
This is a straight reformat run using black as is, with no edits
applied at all.

The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines.   The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.

Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This is a straight reformat run using black as is, with no edits
applied at all.

The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines.   The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.

Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
</pre>
</div>
</content>
</entry>
</feed>
