This is kind of a mixed bag of all kinds to help get us
to 1.4 betas.    The documentation stuff is a work in
progress.    Lots of other relatively small changes to
APIs and things.    More commits will follow to continue
improving the documentation and transitioning to the
1.4/2.0 hybrid documentation.  In particular some refinements
to Session usage models so that it can match Engine's
scoping / transactional patterns, and a decision to
start moving away from "subtransactions" completely.

* add select().from_statement() to produce FromStatement in an
  ORM context

* begin referring to select() that has "plugins" for the few edge
  cases where select() will have ORM-only behaviors

* convert dynamic.AppenderQuery to its own object that can use
  select(), though at the moment it uses Query to support legacy
  join calling forms.

* custom query classes for AppenderQuery are replaced by
  do_orm_execute() hooks for custom actions, a separate gerrit
  will document this

* add Session.get() to replace query.get()

* Deprecate session.begin->subtransaction.  propose within the
  test suite a hypothetical recipe for apps that rely on this
  pattern

* introduce Session construction level context manager,
  sessionmaker context manager, rewrite the whole top of the
  session_transaction.rst documentation.   Establish context manager
  patterns for Session that are identical to engine

* ensure same begin_nested() / commit() behavior as engine

* devise all new "join into an external transaction" recipe,
  add test support for it, add rules into Session so it
  just works, write new docs.  need to ensure this doesn't
  break anything

* vastly reduce the verbosity of lots of session docs as
  I dont think people read this stuff and it's difficult
  to keep current in any case

* constructs like case(), with_only_columns() really need
  to move to *columns, add a coercion rule to just change
  these.

* docs need changes everywhere I look.  in_() is not in
  the Core tutorial?  how do people even know about it?
  Remove tons of cruft from Select docs, etc.

* build a system for common ORM options like populate_existing
  and autoflush to populate from execution options.

* others?

Change-Id: Ia4bea0f804250e54d90b3884cf8aab8b66b82ecf

loader options can now make a deterministic cache key based
 on the structure they are given, and this accommodates for
aliased classes as well so that these cache keys are now
"safe".     Have baked query call upon
the regular cache key method.

Change-Id: Iaa2ef4064cfb16146f415ca73080f32003dd830d

Convert Query to do virtually all compile state computation
in the _compile_context() phase, and organize it all
such that a plain select() construct may also be used as the
source of information in order to generate ORM query state.
This makes it such that Query is not needed except for
its additional methods like from_self() which are all to
be deprecated.

The construction of ORM state will occur beyond the
caching boundary when the new execution model is integrated.

future select() gains a working join() and filter_by() method.
as we continue to rebase and merge each commit in the steps,
callcounts continue to bump around.  will have to look at
the final result when it's all in.

References: #5159
References: #4705
References: #4639
References: #4871
References: #5010

Change-Id: I19e05b3424b07114cce6c439b05198ac47f7ac10

Start trying to convert fundamental objects to
C as we now rely on a fairly small core of things,
and 1.4 is having problems with complexity added being
slower than the performance gains we are trying to build in.

immutabledict here does seem to bench as twice as fast as the
Python one, see below.  However, it does not appear to be
used prominently enough to make any dent in the performance
tests.

at the very least it may provide us some more lift-and-copy
code for more C extensions.

import timeit

from sqlalchemy.util._collections import not_immutabledict, immutabledict

def run(dict_cls):
    for i in range(1000000):
        d1 = dict_cls({"x": 5, "y": 4})

        d2 = d1.union({"x": 17, "new key": "some other value"}, None)

        assert list(d2) == ["x", "y", "new key"]

print(
    timeit.timeit(
        "run(d)", "from __main__ import run, not_immutabledict as d", number=1
    )
)
print(
    timeit.timeit(
        "run(d)", "from __main__ import run, immutabledict as d", number=1
    )
)

output:

python: 1.8799766399897635
C code: 0.8880784640205093

Change-Id: I29e7104dc21dcc7cdf895bf274003af2e219bf6d

Enhanced logic that tracks if relationships will be conflicting with each
other when they write to the same column to include simple cases of two
relationships that should have a "backref" between them.   This means that
if two relationships are not viewonly, are not linked with back_populates
and are not otherwise in an inheriting sibling/overriding arrangement, and
will populate the same foreign key column, a warning is emitted at mapper
configuration time warning that a conflict may arise.  A new parameter
:paramref:`.relationship.overlaps` is added to suit those very rare cases
where such an overlapping persistence arrangement may be unavoidable.

Fixes: #5171
Change-Id: Ifae5998fc1c7e49ce059aec8a67c80cabee768ad

Added test support and repaired a wide variety of unnecessary reference
cycles created for short-lived objects, mostly in the area of ORM queries.

Fixes: #5056
Change-Id: Ifd93856eba550483f95f9ae63d49f36ab068b85a

Created new visitor system called "internal traversal" that
applies a data driven approach to the concept of a class that
defines its own traversal steps, in contrast to the existing
style of traversal now known as "external traversal" where
the visitor class defines the traversal, i.e. the SQLCompiler.

The internal traversal system now implements get_children(),
_copy_internals(), compare() and _cache_key() for most Core elements.
Core elements with special needs like Select still implement
some of these methods directly however most of these methods
are no longer explicitly implemented.

The data-driven system is also applied to ORM elements that
take part in SQL expressions so that these objects, like mappers,
aliasedclass, query options, etc. can all participate in the
cache key process.

Still not considered is that this approach to defining traversibility
will be used to create some kind of generic introspection system
that works across Core / ORM.  It's also not clear if
real statement caching using the _cache_key() method is feasible,
if it is shown that running _cache_key() is nearly as expensive as
compiling in any case.    Because it is data driven, it is more
straightforward to optimize using inlined code, as is the case now,
as well as potentially using C code to speed it up.

In addition, the caching sytem now accommodates for anonymous
name labels, which is essential so that constructs which have
anonymous labels can be cacheable, that is, their position
within a statement in relation to other anonymous names causes
them to generate an integer counter relative to that construct
which will be the same every time.   Gathering of bound parameters
from any cache key generation is also now required as there is
no use case for a cache key that does not extract bound parameter
values.

Applies-to: #4639
Change-Id: I0660584def8627cad566719ee98d3be045db4b8d

Added new loader option method :meth:`.Load.options` which allows loader
options to be constructed hierarchically, so that many sub-options can be
applied to a particular path without needing to call :func:`.defaultload`
many times.  Thanks to Alessio Bogon for the idea.

Also applies a large pass to the loader option documentation which
needed improvement.

Fixes: #4736

Change-Id: I93c453e30a20c074f27e87cf7e95b13dd3f2b494

This is a very useful assertion which prevents unused variables
from being set up allows code to be more readable and sometimes
even more efficient.  test suites seem to be where the most
problems are and there do not seem to be documentation examples
that are using this, or at least the linter is not taking effect
within rst blocks.

Change-Id: I2b3341d8dd14da34879d8425838e66a4b9f8e27d

Fixed regression where a new error message that was supposed to raise when
attempting to link a relationship option to an AliasedClass without using
:meth:`.PropComparator.of_type` would instead raise an ``AttributeError``.
Note that in 1.3, it is no longer valid to create an option path from a
plain mapper relationship to an :class:`.AliasedClass` without using
:meth:`.PropComparator.of_type`.

Fixes: #4566
Change-Id: Ic547a1c8408e41aec66ef9644aac7f76f50dd064