Applied an explicit "cause" to most if not all internally raised exceptions
that are raised from within an internal exception catch, to avoid
misleading stacktraces that suggest an error within the handling of an
exception.  While it would be preferable to suppress the internally caught
exception in the way that the ``__suppress_context__`` attribute would,
there does not as yet seem to be a way to do this without suppressing an
enclosing user constructed context, so for now it exposes the internally
caught exception as the cause so that full information about the context
of the error is maintained.

Fixes: #4849
Change-Id: I55a86b29023675d9e5e49bc7edc5a2dc0bcd4751

In Ia63a510f9c1d08b055eef62cf047f1f427f0450c we introduced
"lambda combinations" which use a bit of function closure inspection
in order to allow for testing combinations that make use of symbols that
come from test fixtures, or from the test itself.

Two problems.  One is that we can't use F821 flake8 rule without either
adding lots of noqas, skipping the file, or adding arguments to the
lambdas themselves that are then populated, which makes for a very
verbose system.  The other is that the system is already verbose
with all those lambdas and the magic in use is a non-explicit kind,
hence F821 reminds us that if we can improve upon this, we should.

So let's improve upon it by making it so that the "lambda" is just
once and up front for the whole thing, and let it accept the arguments
directly.   This still requires magic, because these test cases need
to resolve at test collection time, not test runtime.  But we will
instead substitute a namespace up front that can be coerced into
its desired form within the tests.

Additionally, there's a little bit of py2k compatible type annotations
present; f821 is checking these, so we have to add those imports
also using the TYPE_CHECKING boolean so they don't take place in
py2k.

Change-Id: Idb7e7a0c8af86d9ab133f548511306ef68cdba14

Change-Id: I08440dc25e40ea1ccea1778f6ee9e28a00808235

A few new formatting errors are caught by this version.

Change-Id: I737b33267a00f400b7ba7696a03ddb07a4c95bc0

Created new visitor system called "internal traversal" that
applies a data driven approach to the concept of a class that
defines its own traversal steps, in contrast to the existing
style of traversal now known as "external traversal" where
the visitor class defines the traversal, i.e. the SQLCompiler.

The internal traversal system now implements get_children(),
_copy_internals(), compare() and _cache_key() for most Core elements.
Core elements with special needs like Select still implement
some of these methods directly however most of these methods
are no longer explicitly implemented.

The data-driven system is also applied to ORM elements that
take part in SQL expressions so that these objects, like mappers,
aliasedclass, query options, etc. can all participate in the
cache key process.

Still not considered is that this approach to defining traversibility
will be used to create some kind of generic introspection system
that works across Core / ORM.  It's also not clear if
real statement caching using the _cache_key() method is feasible,
if it is shown that running _cache_key() is nearly as expensive as
compiling in any case.    Because it is data driven, it is more
straightforward to optimize using inlined code, as is the case now,
as well as potentially using C code to speed it up.

In addition, the caching sytem now accommodates for anonymous
name labels, which is essential so that constructs which have
anonymous labels can be cacheable, that is, their position
within a statement in relation to other anonymous names causes
them to generate an integer counter relative to that construct
which will be the same every time.   Gathering of bound parameters
from any cache key generation is also now required as there is
no use case for a cache key that does not extract bound parameter
values.

Applies-to: #4639
Change-Id: I0660584def8627cad566719ee98d3be045db4b8d

generation is to be enhanced to include caching
functionality, so ensure that Query and all generative in Core
(e.g. select, DML etc) are using the same generations system.

Additionally, deprecate Select.append methods and state
Select methods independently of their append versions.
Mutability of expression objects is a special case only when
generating new objects during a visit.

Fixes: #4637
Change-Id: I3dfac00d5e0f710c833b236f7a0913e1ca24dde4

The :func:`.select` construct and related constructs now allow for
duplication of column labels and columns themselves in the columns clause,
mirroring exactly how column expressions were passed in.   This allows
the tuples returned by an executed result to match what was SELECTed
for in the first place, which is how the ORM :class:`.Query` works, so
this establishes better cross-compatibility between the two constructs.
Additionally, it allows column-positioning-sensitive structures such as
UNIONs (i.e. :class:`.CompoundSelect`) to be more intuitively constructed
in those cases where a particular column might appear in more than one
place.   To support this change, the :class:`.ColumnCollection` has been
revised to support duplicate columns as well as to allow integer index
access.

Fixes: #4753
Change-Id: Ie09a8116f05c367995c1e43623c51e07971d3bf0

As part of the SQLAlchemy 2.0 migration project, a conceptual change has
been made to the role of the :class:`.SelectBase` class hierarchy,
which is the root of all "SELECT" statement constructs, in that they no
longer serve directly as FROM clauses, that is, they no longer subclass
:class:`.FromClause`.  For end users, the change mostly means that any
placement of a :func:`.select` construct in the FROM clause of another
:func:`.select` requires first that it be wrapped in a subquery first,
which historically is through the use of the :meth:`.SelectBase.alias`
method, and is now also available through the use of
:meth:`.SelectBase.subquery`.    This was usually a requirement in any
case since several databases don't accept unnamed SELECT subqueries
in their FROM clause in any case.

See the documentation in this change for lots more detail.

Fixes: #4617
Change-Id: I0f6174ee24b9a1a4529168e52e855e12abd60667

A major refactoring of all the functions handle all detection of
Core argument types as well as perform coercions into a new class hierarchy
based on "roles", each of which identify a syntactical location within a
SQL statement.  In contrast to the ClauseElement hierarchy that identifies
"what" each object is syntactically, the SQLRole hierarchy identifies
the "where does it go" of each object syntactically.   From this we define
a consistent type checking and coercion system that establishes well
defined behviors.

This is a breakout of the patch that is reorganizing select()
constructs to no longer be in the FromClause hierarchy.

Also includes a rename of as_scalar() into scalar_subquery(); deprecates
automatic coercion to scalar_subquery().

Partially-fixes: #4617
Change-Id: I26f1e78898693c6b99ef7ea2f4e7dfd0e8e1a1bd

Added accessors for execution options to Core and ORM, via
:meth:`.Query.get_execution_options`,
:meth:`.Connection.get_execution_options`,
:meth:`.Engine.get_execution_options`, and
:meth:`.Executable.get_execution_options`.  PR courtesy Daniel Lister.

Fixes: #4406
Closes: #4465
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4465
Pull-request-sha: 9674688bb5e80471a6a421bac06f995c2e64f8f7

Change-Id: I93ba51d7a2d687e255edd6938db15615e56dd237