summaryrefslogtreecommitdiff
path: root/examples/sharding
Commit message (Collapse)AuthorAgeFilesLines
* add set_shard_id() loader option for horizontal shardMike Bayer2023-01-254-230/+643
| | | | | | | | | | | | | | Added new option to horizontal sharding API :class:`_horizontal.set_shard_id` which sets the effective shard identifier to query against, for both the primary query as well as for all secondary loaders including relationship eager loaders as well as relationship and column lazy loaders. Modernize sharding examples with new-style mappings, add new asyncio example. Fixes: #7226 Fixes: #7028 Change-Id: Ie69248060c305e8de04f75a529949777944ad511
* reorganize pre_session_exec around do_orm_executeMike Bayer2022-12-263-17/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | Allow do_orm_execute() events to both receive the complete state of bind_argments, load_options, update_delete_options as they do already, but also allow them to *change* all those things via new execution options. Options like autoflush, populate_existing etc. can now be updated within a do_orm_execute() hook and those changes will take effect all the way through. Took a few tries to get something that covers every case here, in particular horizontal sharding which is consuming those options as well as using context.invoke(), without excess complexity. The good news seems to be that a simple reorg and replacing the "reentrant" boolean with "is this before do_orm_execute is invoked" was all that was needed. As part of this we add a new "identity_token" option allowing this option to be controlled from do_orm_execute() as well as from the outside. WIP Fixes: #7837 Change-Id: I087728215edec8d1b1712322ab389e3f52ff76ba
* fixes for doc buildsMike Bayer2022-10-133-3/+0
| | | | | | | * requirements needs typing_extensions * update all "future=True" references to be appropriate to 2.0 Change-Id: I2eeb0ae65afdb587f21aeb0020f1d8a292f67c21
* use begin() for settting up first PKMike Bayer2022-03-212-2/+2
| | | | Change-Id: I227bbb46fbcbae1f60d3f5bb4dd2b9f41ca3dd0c
* note that horizontal sharding supports multi schema translatesMike Bayer2022-03-213-26/+269
| | | | | | | | | | | | | the horizontal sharding API needs some work as it is still exposing some legacy details, but in any case illustrate how we can, for the moment, to use multiple schema translate maps in a single session. A lot more cleanup is needed in horizontal sharding, see #7837 Change-Id: Ia925e2226ecee9d747a8c4fc1772917f10bc505f References: #7832 References: #7837
* Replace all http:// links to https://Federico Caselli2021-07-041-1/+1
| | | | | | Also replace http://pypi.python.org/pypi with https://pypi.org/project Change-Id: I84b5005c39969a82140706472989f2a30b0c7685
* Documentation updates for 1.4Mike Bayer2020-08-053-53/+379
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * major additions to 1.4 migration doc; removed additional verbosity regarding caching methodology and reorganized the doc to present itself more as a "what's changed" guide * as we now have a path for asyncio, update that doc so that we aren't spreading obsolete information * updates to the 2.0 migration guide with latest info, however this is still an architecture doc and not a migration guide yet, will need further rework. * start really talking about 1.x vs. 2.0 style everywhere. Querying is most of the docs so this is going to be a prominent theme, start getting it to fit in * Add introductory documentation for ORM example sections as these are too sparse * new documentation for do_orm_execute(), many separate sections, adding deprecation notes to before_compile() and similar * new example suites to illustrate do_orm_execute(), with_loader_criteria() * modernized horizontal sharding examples and added a separate example to distinguish between multiple databases and single database w/ multiple tables use case * introducing DEEP ALCHEMY, will use zzzeeksphinx 1.1.6 * no name for the alchemist yet however the dragon's name is Flambé Change-Id: Id6b5c03b1ce9ddb7b280f66792212a0ef0a1c541
* Use expanding IN for all literal value IN expressionsMike Bayer2019-12-221-18/+2
| | | | | | | | | | | | | | | | | | | | | | | | The "expanding IN" feature, which generates IN expressions at query execution time which are based on the particular parameters associated with the statement execution, is now used for all IN expressions made against lists of literal values. This allows IN expressions to be fully cacheable independently of the list of values being passed, and also includes support for empty lists. For any scenario where the IN expression contains non-literal SQL expressions, the old behavior of pre-rendering for each position in the IN is maintained. The change also completes support for expanding IN with tuples, where previously type-specific bind processors weren't taking effect. As part of this change, a more explicit separation between "literal execute" and "post compile" bound parameters is being made; as the "ansi bind rules" feature is rendering bound parameters inline, as we now support "postcompile" generically, these should be used here, however we have to render literal values at execution time even for "expanding" parameters. new test fixtures etc. are added to assert everything goes to the right place. Fixes: #4645 Change-Id: Iaa2b7bfbfaaf5b80799ee17c9b8507293cba6ed1
* Post black reformattingMike Bayer2019-01-062-16/+17
| | | | | | | | | | | | | Applied on top of a pure run of black -l 79 in I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes resolves all remaining flake8 conditions for those codes we have enabled in setup.cfg. Included are resolutions for all remaining flake8 issues including shadowed builtins, long lines, import order, unused imports, duplicate imports, and docstring issues. Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
* Run black -l 79 against all source filesMike Bayer2019-01-061-48/+69
| | | | | | | | | | | | | | This is a straight reformat run using black as is, with no edits applied at all. The black run will format code consistently, however in some cases that are prevalent in SQLAlchemy code it produces too-long lines. The too-long lines will be resolved in the following commit that will resolve all remaining flake8 issues including shadowed builtins, long lines, import order, unused imports, duplicate imports, and docstring issues. Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
* Add Query.lazy_load_from attribute for shardingMike Bayer2018-06-041-1/+7
| | | | | | | | | | | | | | | | | | Added new attribute :attr:`.Query.lazy_loaded_from` which is populated with an :class:`.InstanceState` that is using this :class:`.Query` in order to lazy load a relationship. The rationale for this is that it serves as a hint for the horizontal sharding feature to use, such that the identity token of the state can be used as the default identity token to use for the query within id_chooser(). Also repaired an issue in the :meth:`.Result.with_post_criteria` method added in I899808734458e25a023142c2c5bb37cbed869479 for :ticket:`4128` where the "unbake subquery loaders" version was calling the post crtieria functions given the :class:`.Result` as the argument rather than applying them to the :class:`.Query`. Change-Id: I3c0919ce7fd151b80fe2f9b5f99f60df31c2d73d Fixes: #4243
* Add an identity_token to the identity keyMike Bayer2017-12-221-99/+106
| | | | | | | | | For the purposes of assisting with sharded setups, add a new member to the identity key that can be customized. this allows sharding across databases where the primary key space is shared. Change-Id: Iae3909f5d4c501b62c10d0371fbceb01abda51db Fixes: #4137
* Maul the evaulate & friends typoPriit Laes2014-12-191-1/+1
|
* Fix many typos throughout the codebasepr/85Alex Gaynor2014-04-261-1/+1
| | | | Found using: https://github.com/intgr/topy
* - add a new sphinx extension "viewsource". takes advantage of part of the ↵Mike Bayer2013-11-091-0/+2
| | | | | | | | | viewcode extension to allow ad-hoc display of the source of any file, as well as a "directory listing" structure. - reorganize examples/ to take advantage of new extension. in particular, keep moving all the descriptive text for files etc. into module docstrings, taking more advantage of self-documentation.
* -whitespace bonanza, contdMike Bayer2012-07-281-4/+4
|
* trailing whitespace bonanzaMike Bayer2012-07-281-17/+17
|
* - Fixed the attribute shard example to checkMike Bayer2011-06-261-11/+13
| | | | | for bind param callable correctly in 0.7 style.
* add example usage for shard_id in context feature [ticket:2031]Mike Bayer2011-01-231-2/+15
|
* - whitespace removal bonanzaMike Bayer2011-01-021-14/+14
|
* - gently push users towards entity name for simple casesMike Bayer2010-12-071-3/+13
|
* - NullPool is now used by default for SQLite file-Mike Bayer2010-11-281-2/+5
| | | | | | based databases. :memory: databases will continue to select SingletonThreadPool by default. [ticket:1921]
* - remove remaining create_session() calls from examples, replace with SessionMike Bayer2010-10-241-3/+1
| | | | - replace all flush()/expunge_all() with commit()
* - Updated attribute_shard.py example to use a more robustMike Bayer2010-04-021-32/+107
| | | | | method of searching a Query for binary expressions which compare columns against literal values.
* - The sqlalchemy.orm.shard module now becomes an extension,Mike Bayer2010-03-281-1/+1
| | | | | sqlalchemy.ext.horizontal_shard. The old import works with a deprecation warning.
* - The official name for the relation() function is nowMike Bayer2010-03-171-2/+2
| | | | | | relationship(), to eliminate confusion over the relational algebra term. relation() however will remain available in equal capacity for the foreseeable future. [ticket:1740]
* - mega example cleanupMike Bayer2010-01-192-18/+20
| | | | | | - added READMEs to all examples in each __init__.py and added to sphinx documentation - added versioning example - removed vertical/vertical.py, the dictlikes are more straightforward
* - Fixed a slight inaccuracy in the sharding example.Mike Bayer2009-10-251-1/+7
| | | | | | Comparing equivalence of columns in the ORM is best accomplished using col1.shares_lineage(col2). [ticket:1491]
* Added in Examples into the test suite so they get exercised regularly. ↵Michael Trier2009-07-271-0/+0
| | | | Cleaned up some deprecation warnings in the examples.
* Cleaned up the deprecation problems with the examples.Michael Trier2009-04-131-2/+2
|
* Lots of fixes to the code examples to specify imports explicitly.Michael Trier2009-03-311-7/+8
| | | | | | | Explicit imports make it easier for users to understand the examples. Additionally a lot of the examples were fixed to work with the changes in the 0.5.x code base. One small correction to the Case expression. Thanks a bunch to Adam Lowry! Fixes #717.
* - fixes to ShardedSession to work with deferred columns [ticket:771].Mike Bayer2007-09-081-1/+2
| | | | | | | - user-defined shard_chooser() function must accept "clause=None" argument; this is the ClauseElement passed to session.execute(statement) and can be used to determine correct shard id (since execute() doesn't take an instance)
* - modified SQL operator functions to be module-level operators, allowingMike Bayer2007-08-181-4/+4
| | | | | | | SQL expressions to be pickleable [ticket:735] - small adjustment to mapper class.__init__ to allow for Py2.6 object.__init__() behavior
* #725 add query arg to id_chooser()Mike Bayer2007-08-101-1/+1
|
* - added hooks for alternate session classes into sessionmakerMike Bayer2007-08-031-7/+12
| | | | - moved shard example/unittest over to sessionmaker
* merging 0.4 branch to trunk. see CHANGES for details. 0.3 moves to ↵Mike Bayer2007-07-271-0/+194
maintenance branch in branches/rel_0_3.
View Lorry Controller Status