summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorMike Bayer <mike_mp@zzzcomputing.com>2016-01-06 12:40:34 -0500
committerMike Bayer <mike_mp@zzzcomputing.com>2016-01-06 12:47:48 -0500
commita80bb4e5aabc4850a202f3a4d114c543357e37d5 (patch)
tree203dbaaa9002a2b9372dcee0976d7e1c3e0baf6b /doc
parent16746dd1a63198e3c27422517fa22ec76f441ceb (diff)
downloadsqlalchemy-a80bb4e5aabc4850a202f3a4d114c543357e37d5.tar.gz
- Added :class:`.mysql.JSON` for MySQL 5.7. The JSON type provides
persistence of JSON values in MySQL as well as basic operator support of "getitem" and "getpath", making use of the ``JSON_EXTRACT`` function in order to refer to individual paths in a JSON structure. fixes #3547 - Added a new type to core :class:`.types.JSON`. This is the base of the PostgreSQL :class:`.postgresql.JSON` type as well as that of the new :class:`.mysql.JSON` type, so that a PG/MySQL-agnostic JSON column may be used. The type features basic index and path searching support. fixes #3619 - reorganization of migration docs etc. to try to refer both to the fixes to JSON that helps Postgresql while at the same time indicating these are new features of the new base JSON type. - a rework of the Array/Indexable system some more, moving things that are specific to Array out of Indexable. - new operators for JSON indexing added to core so that these can be compiled by the PG and MySQL dialects individually - rename sqltypes.Array to sqltypes.ARRAY - as there is no generic Array implementation, this is an uppercase type for now, consistent with the new sqltypes.JSON type that is also not a generic implementation. There may need to be some convention change to handle the case of datatypes that aren't generic, rely upon DB-native implementations, but aren't necessarily all named the same thing.
Diffstat (limited to 'doc')
-rw-r--r--doc/build/changelog/changelog_11.rst35
-rw-r--r--doc/build/changelog/migration_11.rst231
-rw-r--r--doc/build/core/type_basics.rst23
-rw-r--r--doc/build/dialects/mysql.rst2
-rw-r--r--doc/build/dialects/postgresql.rst3
5 files changed, 185 insertions, 109 deletions
diff --git a/doc/build/changelog/changelog_11.rst b/doc/build/changelog/changelog_11.rst
index 83a57ba7c..63e0ca472 100644
--- a/doc/build/changelog/changelog_11.rst
+++ b/doc/build/changelog/changelog_11.rst
@@ -22,6 +22,33 @@
:version: 1.1.0b1
.. change::
+ :tags: feature, mysql
+ :tickets: 3547
+
+ Added :class:`.mysql.JSON` for MySQL 5.7. The JSON type provides
+ persistence of JSON values in MySQL as well as basic operator support
+ of "getitem" and "getpath", making use of the ``JSON_EXTRACT``
+ function in order to refer to individual paths in a JSON structure.
+
+ .. seealso::
+
+ :ref:`change_3547`
+
+ .. change::
+ :tags: feature, sql
+ :tickets: 3619
+
+ Added a new type to core :class:`.types.JSON`. This is the
+ base of the PostgreSQL :class:`.postgresql.JSON` type as well as that
+ of the new :class:`.mysql.JSON` type, so that a PG/MySQL-agnostic
+ JSON column may be used. The type features basic index and path
+ searching support.
+
+ .. seealso::
+
+ :ref:`change_3619`
+
+ .. change::
:tags: bug, sql
:tickets: 3616
@@ -275,7 +302,7 @@
:tickets: 3132
Added support for the SQL-standard function :class:`.array_agg`,
- which automatically returns an :class:`.Array` of the correct type
+ which automatically returns an :class:`.postgresql.ARRAY` of the correct type
and supports index / slice operations, as well as
:func:`.postgresql.array_agg`, which returns a :class:`.postgresql.ARRAY`
with additional comparison features. As arrays are only
@@ -292,8 +319,8 @@
:tags: feature, sql
:tickets: 3516
- Added a new type to core :class:`.types.Array`. This is the
- base of the PostgreSQL :class:`.ARRAY` type, and is now part of Core
+ Added a new type to core :class:`.types.ARRAY`. This is the
+ base of the PostgreSQL :class:`.postgresql.ARRAY` type, and is now part of Core
to begin supporting various SQL-standard array-supporting features
including some functions and eventual support for native arrays
on other databases that have an "array" concept, such as DB2 or Oracle.
@@ -399,7 +426,7 @@
:tickets: 3514
Additional fixes have been made regarding the value of ``None``
- in conjunction with the Postgresql :class:`.JSON` type. When
+ in conjunction with the Postgresql :class:`.postgresql.JSON` type. When
the :paramref:`.JSON.none_as_null` flag is left at its default
value of ``False``, the ORM will now correctly insert the Json
"'null'" string into the column whenever the value on the ORM
diff --git a/doc/build/changelog/migration_11.rst b/doc/build/changelog/migration_11.rst
index b5889c763..70182091c 100644
--- a/doc/build/changelog/migration_11.rst
+++ b/doc/build/changelog/migration_11.rst
@@ -502,33 +502,138 @@ UNIONs with parenthesized SELECT statements is much less common than the
:ticket:`2528`
+.. _change_3619:
+
+JSON support added to Core
+--------------------------
+
+As MySQL now has a JSON datatype in addition to the Postgresql JSON datatype,
+the core now gains a :class:`sqlalchemy.types.JSON` datatype that is the basis
+for both of these. Using this type allows access to the "getitem" operator
+as well as the "getpath" operator in a way that is agnostic across Postgresql
+and MySQL.
+
+The new datatype also has a series of improvements to the handling of
+NULL values as well as expression handling.
+
+.. seealso::
+
+ :ref:`change_3547`
+
+ :class:`.types.JSON`
+
+ :class:`.postgresql.JSON`
+
+ :class:`.mysql.JSON`
+
+:ticket:`3619`
+
+.. _change_3514:
+
+JSON "null" is inserted as expected with ORM operations, regardless of column default present
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The :class:`.types.JSON` type and its descendant types :class:`.postgresql.JSON`
+and :class:`.mysql.JSON` have a flag :paramref:`.types.JSON.none_as_null` which
+when set to True indicates that the Python value ``None`` should translate
+into a SQL NULL rather than a JSON NULL value. This flag defaults to False,
+which means that the column should *never* insert SQL NULL or fall back
+to a default unless the :func:`.null` constant were used. However, this would
+fail in the ORM under two circumstances; one is when the column also contained
+a default or server_default value, a positive value of ``None`` on the mapped
+attribute would still result in the column-level default being triggered,
+replacing the ``None`` value::
+
+ obj = MyObject(json_value=None)
+ session.add(obj)
+ session.commit() # would fire off default / server_default, not encode "'none'"
+
+The other is when the :meth:`.Session.bulk_insert_mappings`
+method were used, ``None`` would be ignored in all cases::
+
+ session.bulk_insert_mappings(
+ MyObject,
+ [{"json_value": None}]) # would insert SQL NULL and/or trigger defaults
+
+The :class:`.types.JSON` type now implements the
+:attr:`.TypeEngine.should_evaluate_none` flag,
+indicating that ``None`` should not be ignored here; it is configured
+automatically based on the value of :paramref:`.types.JSON.none_as_null`.
+Thanks to :ticket:`3061`, we can differentiate when the value ``None`` is actively
+set by the user versus when it was never set at all.
+
+If the attribute is not set at all, then column level defaults *will*
+fire off and/or SQL NULL will be inserted as expected, as was the behavior
+previously. Below, the two variants are illustrated::
+
+ obj = MyObject(json_value=None)
+ session.add(obj)
+ session.commit() # *will not* fire off column defaults, will insert JSON 'null'
+
+ obj = MyObject()
+ session.add(obj)
+ session.commit() # *will* fire off column defaults, and/or insert SQL NULL
+
+The feature applies as well to the new base :class:`.types.JSON` type
+and its descendant types.
+
+:ticket:`3514`
+
+.. _change_3514_jsonnull:
+
+New JSON.NULL Constant Added
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To ensure that an application can always have full control at the value level
+of whether a :class:`.types.JSON`, :class:`.postgresql.JSON`, :class:`.mysql.JSON`,
+or :class:`.postgresql.JSONB` column
+should receive a SQL NULL or JSON ``"null"`` value, the constant
+:attr:`.types.JSON.NULL` has been added, which in conjunction with
+:func:`.null` can be used to determine fully between SQL NULL and
+JSON ``"null"``, regardless of what :paramref:`.types.JSON.none_as_null` is set
+to::
+
+ from sqlalchemy import null
+ from sqlalchemy.dialects.postgresql import JSON
+
+ obj1 = MyObject(json_value=null()) # will *always* insert SQL NULL
+ obj2 = MyObject(json_value=JSON.NULL) # will *always* insert JSON string "null"
+
+ session.add_all([obj1, obj2])
+ session.commit()
+
+The feature applies as well to the new base :class:`.types.JSON` type
+and its descendant types.
+
+:ticket:`3514`
+
.. _change_3516:
Array support added to Core; new ANY and ALL operators
------------------------------------------------------
-Along with the enhancements made to the Postgresql :class:`.ARRAY`
-type described in :ref:`change_3503`, the base class of :class:`.ARRAY`
-itself has been moved to Core in a new class :class:`.types.Array`.
+Along with the enhancements made to the Postgresql :class:`.postgresql.ARRAY`
+type described in :ref:`change_3503`, the base class of :class:`.postgresql.ARRAY`
+itself has been moved to Core in a new class :class:`.types.ARRAY`.
Arrays are part of the SQL standard, as are several array-oriented functions
such as ``array_agg()`` and ``unnest()``. In support of these constructs
for not just PostgreSQL but also potentially for other array-capable backends
in the future such as DB2, the majority of array logic for SQL expressions
-is now in Core. The :class:`.Array` type still **only works on
+is now in Core. The :class:`.types.ARRAY` type still **only works on
Postgresql**, however it can be used directly, supporting special array
use cases such as indexed access, as well as support for the ANY and ALL::
mytable = Table("mytable", metadata,
- Column("data", Array(Integer, dimensions=2))
+ Column("data", ARRAY(Integer, dimensions=2))
)
expr = mytable.c.data[5][6]
expr = mytable.c.data[5].any(12)
-In support of ANY and ALL, the :class:`.Array` type retains the same
-:meth:`.Array.Comparator.any` and :meth:`.Array.Comparator.all` methods
+In support of ANY and ALL, the :class:`.types.ARRAY` type retains the same
+:meth:`.types.ARRAY.Comparator.any` and :meth:`.types.ARRAY.Comparator.all` methods
from the PostgreSQL type, but also exports these operations to new
standalone operator functions :func:`.sql.expression.any_` and
:func:`.sql.expression.all_`. These two functions work in more
@@ -541,7 +646,7 @@ as::
For the PostgreSQL-specific operators "contains", "contained_by", and
"overlaps", one should continue to use the :class:`.postgresql.ARRAY`
-type directly, which provides all functionality of the :class:`.Array`
+type directly, which provides all functionality of the :class:`.types.ARRAY`
type as well.
The :func:`.sql.expression.any_` and :func:`.sql.expression.all_` operators
@@ -564,7 +669,7 @@ such as::
New Function features, "WITHIN GROUP", array_agg and set aggregate functions
----------------------------------------------------------------------------
-With the new :class:`.Array` type we can also implement a pre-typed
+With the new :class:`.types.ARRAY` type we can also implement a pre-typed
function for the ``array_agg()`` SQL function that returns an array,
which is now available using :class:`.array_agg`::
@@ -767,8 +872,9 @@ As described in :ref:`change_3499`, the ORM relies upon being able to
produce a hash function for column values when a query's selected entities
mixes full ORM entities with column expressions. The ``hashable=False``
flag is now correctly set on all of PG's "data structure" types, including
-:class:`.ARRAY` and :class:`.JSON`. The :class:`.JSONB` and :class:`.HSTORE`
-types already included this flag. For :class:`.ARRAY`,
+:class:`.postgresql.ARRAY` and :class:`.postgresql.JSON`.
+The :class:`.JSONB` and :class:`.HSTORE`
+types already included this flag. For :class:`.postgresql.ARRAY`,
this is conditional based on the :paramref:`.postgresql.ARRAY.as_tuple`
flag, however it should no longer be necessary to set this flag
in order to have an array value present in a composed ORM row.
@@ -840,7 +946,7 @@ The JSON cast() operation now requires ``.astext`` is called explicitly
As part of the changes in :ref:`change_3503`, the workings of the
:meth:`.ColumnElement.cast` operator on :class:`.postgresql.JSON` and
:class:`.postgresql.JSONB` no longer implictly invoke the
-:attr:`.JSON.Comparator.astext` modifier; Postgresql's JSON/JSONB types
+:attr:`.postgresql.JSON.Comparator.astext` modifier; Postgresql's JSON/JSONB types
support CAST operations to each other without the "astext" aspect.
This means that in most cases, an application that was doing this::
@@ -852,88 +958,6 @@ Will now need to change to this::
expr = json_col['somekey'].astext.cast(Integer)
-
-.. _change_3514:
-
-Postgresql JSON "null" is inserted as expected with ORM operations, regardless of column default present
------------------------------------------------------------------------------------------------------------
-
-The :class:`.JSON` type has a flag :paramref:`.JSON.none_as_null` which
-when set to True indicates that the Python value ``None`` should translate
-into a SQL NULL rather than a JSON NULL value. This flag defaults to False,
-which means that the column should *never* insert SQL NULL or fall back
-to a default unless the :func:`.null` constant were used. However, this would
-fail in the ORM under two circumstances; one is when the column also contained
-a default or server_default value, a positive value of ``None`` on the mapped
-attribute would still result in the column-level default being triggered,
-replacing the ``None`` value::
-
- obj = MyObject(json_value=None)
- session.add(obj)
- session.commit() # would fire off default / server_default, not encode "'none'"
-
-The other is when the :meth:`.Session.bulk_insert_mappings`
-method were used, ``None`` would be ignored in all cases::
-
- session.bulk_insert_mappings(
- MyObject,
- [{"json_value": None}]) # would insert SQL NULL and/or trigger defaults
-
-The :class:`.JSON` type now implements the
-:attr:`.TypeEngine.should_evaluate_none` flag,
-indicating that ``None`` should not be ignored here; it is configured
-automatically based on the value of :paramref:`.JSON.none_as_null`.
-Thanks to :ticket:`3061`, we can differentiate when the value ``None`` is actively
-set by the user versus when it was never set at all.
-
-If the attribute is not set at all, then column level defaults *will*
-fire off and/or SQL NULL will be inserted as expected, as was the behavior
-previously. Below, the two variants are illustrated::
-
- obj = MyObject(json_value=None)
- session.add(obj)
- session.commit() # *will not* fire off column defaults, will insert JSON 'null'
-
- obj = MyObject()
- session.add(obj)
- session.commit() # *will* fire off column defaults, and/or insert SQL NULL
-
-:ticket:`3514`
-
-.. seealso::
-
- :ref:`change_3250`
-
- :ref:`change_3514_jsonnull`
-
-.. _change_3514_jsonnull:
-
-New JSON.NULL Constant Added
-----------------------------
-
-To ensure that an application can always have full control at the value level
-of whether a :class:`.postgresql.JSON` or :class:`.postgresql.JSONB` column
-should receive a SQL NULL or JSON ``"null"`` value, the constant
-:attr:`.postgresql.JSON.NULL` has been added, which in conjunction with
-:func:`.null` can be used to determine fully between SQL NULL and
-JSON ``"null"``, regardless of what :paramref:`.JSON.none_as_null` is set
-to::
-
- from sqlalchemy import null
- from sqlalchemy.dialects.postgresql import JSON
-
- obj1 = MyObject(json_value=null()) # will *always* insert SQL NULL
- obj2 = MyObject(json_value=JSON.NULL) # will *always* insert JSON string "null"
-
- session.add_all([obj1, obj2])
- session.commit()
-
-.. seealso::
-
- :ref:`change_3514`
-
-:ticket:`3514`
-
.. _change_2729:
ARRAY with ENUM will now emit CREATE TYPE for the ENUM
@@ -975,6 +999,25 @@ emits::
Dialect Improvements and Changes - MySQL
=============================================
+.. _change_3547:
+
+MySQL JSON Support
+------------------
+
+A new type :class:`.mysql.JSON` is added to the MySQL dialect supporting
+the JSON type newly added to MySQL 5.7. This type provides both persistence
+of JSON as well as rudimentary indexed-access using the ``JSON_EXTRACT``
+function internally. An indexable JSON column that works across MySQL
+and Postgresql can be achieved by using the :class:`.types.JSON` datatype
+common to both MySQL and Postgresql.
+
+.. seealso::
+
+ :ref:`change_3619`
+
+:ticket:`3547`
+
+
.. _change_mysql_3216:
No more generation of an implicit KEY for composite primary key w/ AUTO_INCREMENT
diff --git a/doc/build/core/type_basics.rst b/doc/build/core/type_basics.rst
index ec3c14dd6..9edba0061 100644
--- a/doc/build/core/type_basics.rst
+++ b/doc/build/core/type_basics.rst
@@ -38,9 +38,6 @@ database column type available on the target database when issuing a
type is emitted in ``CREATE TABLE``, such as ``VARCHAR`` see `SQL
Standard Types`_ and the other sections of this chapter.
-.. autoclass:: Array
- :members:
-
.. autoclass:: BigInteger
:members:
@@ -101,12 +98,19 @@ Standard Types`_ and the other sections of this chapter.
.. _types_sqlstandard:
-SQL Standard Types
-------------------
+SQL Standard and Multiple Vendor Types
+--------------------------------------
+
+This category of types refers to types that are either part of the
+SQL standard, or are potentially found within a subset of database backends.
+Unlike the "generic" types, the SQL standard/multi-vendor types have **no**
+guarantee of working on all backends, and will only work on those backends
+that explicitly support them by name. That is, the type will always emit
+its exact name in DDL with ``CREATE TABLE`` is issued.
-The SQL standard types always create database column types of the same
-name when ``CREATE TABLE`` is issued. Some types may not be supported
-on all databases.
+
+.. autoclass:: ARRAY
+ :members:
.. autoclass:: BIGINT
@@ -140,6 +144,9 @@ on all databases.
.. autoclass:: INT
+.. autoclass:: JSON
+ :members:
+
.. autoclass:: sqlalchemy.types.INTEGER
diff --git a/doc/build/dialects/mysql.rst b/doc/build/dialects/mysql.rst
index 33a0d783b..100f2d2e3 100644
--- a/doc/build/dialects/mysql.rst
+++ b/doc/build/dialects/mysql.rst
@@ -74,6 +74,8 @@ construction arguments, are as follows:
.. autoclass:: INTEGER
:members: __init__
+.. autoclass:: JSON
+ :members:
.. autoclass:: LONGBLOB
:members: __init__
diff --git a/doc/build/dialects/postgresql.rst b/doc/build/dialects/postgresql.rst
index 7e2a20ef7..e158cacbf 100644
--- a/doc/build/dialects/postgresql.rst
+++ b/doc/build/dialects/postgresql.rst
@@ -78,9 +78,6 @@ construction arguments, are as follows:
.. autoclass:: JSONB
:members:
-.. autoclass:: JSONElement
- :members:
-
.. autoclass:: MACADDR
:members: __init__