Add more docs on `OpenType`improve-open-type-docs

This change adds more explanations and examples on ASN.1 ANY
DEFINED BY syntax (OpenType).

8 files changed, 139 insertions, 42 deletions

diff --git a/docs/source/pyasn1/type/opentype/contents.rst b/docs/source/pyasn1/type/opentype/contents.rst
index 9ae10d0..034870b 100644
--- a/docs/source/pyasn1/type/opentype/contents.rst
+++ b/docs/source/pyasn1/type/opentype/contents.rst
@@ -1,16 +1,67 @@
 
 .. _type.opentype:
 
-Untyped fields of constructed types
------------------------------------
+Dynamic or open type
+--------------------
 
-To aid data structures flexibility, ASN.1 allows the designer to
-leave incomplete field type specification in the
-:ref:`Sequence <univ.Sequence>` and :ref:`Set <univ.Set>` types.
+ASN.1 allows data structure designer to leave "holes" in field type
+specification of :ref:`Sequence <univ.Sequence>` or
+:ref:`Set <univ.Set>` types.
 
-To figure out field's type at the run time, a type selector field
-must accompany the open type field. The open type situation can
-be captured by the :ref:`OpenType <opentype.OpenType>` object.
+The idea behind that feature is that there can be times, when the
+exact field type is not known at the design time, or it is anticipated
+that new field types may come up in the future.
+
+This "hole" type is manifested in the data structure by :ref:`Any <univ.Any>`
+type. Technically, the actual type is serialized into an octet stream
+and then put into :ref:`Any <univ.Any>` "container", which is in fact an
+(untagged, by default) specialization of ASN.1
+:ref:`OctetString <univ.OctetString>` type.
+
+.. code-block:: bash
+
+       Algorithm ::= SEQUENCE {
+               algorithm OBJECT IDENTIFIER,
+               parameters ANY DEFINED BY algorithm OPTIONAL
+       }
+
+On the receiving end, to know how to interpret the open type
+serialization, the receiver can rely on the supplied value in
+the other field of the data structure. That other field is
+semantically linked with the open type field. This link
+is expressed in ASN.1 by the *DEFINE BY* clause.
+
+From ASN.1 perspective, it is not an error if the decoder does
+not know a type selector value it receives. In that case pyasn1 decoder
+just leaves serialized blob in the open type field.
+
+.. note::
+
+   By default, ASN.1 ANY type has no tag. That makes it an
+   "invisible" in serialization. However, like any other ASN.1 type,
+   ANY type can be subtyped by :ref:`tagging <type.tag>`.
+
+Besides scalar open type fields, ASN.1 allows using *SET OF*
+or *SEQUENCE OF* containers holding zero or more of *ANY*
+scalars.
+
+.. code-block:: bash
+
+   AttributeTypeAndValues ::= SEQUENCE {
+      type OBJECT IDENTIFIER,
+      values SET OF ANY DEFINED BY type
+   }
+
+.. note::
+
+   A single type selector field is used to guide the decoder
+   of potentially many elements of a *SET OF* or *SEQUENCE OF* container
+   all at once. That implies that all *ANY* elements must be of the same
+   type in any given instance of a data structure.
+
+When expressing ASN.1 type "holes" in pyasn1, the
+:ref:`OpenType <opentype.OpenType>` object should be used to establish
+a semantic link between type selector field and open type field.
 
 .. code-block:: python
 
@@ -24,15 +75,38 @@ be captured by the :ref:`OpenType <opentype.OpenType>` object.
        """
        Algorithm ::= SEQUENCE {
                algorithm OBJECT IDENTIFIER,
-               parameters ANY DEFINED BY algorithm OPTIONAL
+               parameters ANY DEFINED BY algorithm
        }
        """
        componentType = NamedTypes(
            NamedType('algorithm', ObjectIdentifier()),
-           OptionalNamedType('parameters', Any(),
+           NamedType('parameters', Any(),
                      openType=OpenType('algorithm', algo_map))
        )
 
+Similarly for `SET OF ANY DEFINED BY` or `SEQUENCE OF ANY DEFINED BY`
+constructs:
+
+.. code-block:: python
+
+   algo_map = {
+       ObjectIdentifier('1.2.840.113549.1.1.1'): rsaEncryption(),
+       ObjectIdentifier('1.2.840.113549.1.1.2'): md2WithRSAEncryption()
+   }
+
+
+   class Algorithm(Sequence):
+       """
+       Algorithm ::= SEQUENCE {
+               algorithm OBJECT IDENTIFIER,
+               parameters SET OF ANY DEFINED BY algorithm
+       }
+       """
+       componentType = NamedTypes(
+           NamedType('algorithm', ObjectIdentifier()),
+           NamedType('parameters', SetOf(componentType=Any()),
+                     openType=OpenType('algorithm', algo_map))
+       )
 
 .. toctree::
    :maxdepth: 2
diff --git a/docs/source/pyasn1/type/opentype/opentype.rst b/docs/source/pyasn1/type/opentype/opentype.rst
index dc2fa47..8ce9303 100644
--- a/docs/source/pyasn1/type/opentype/opentype.rst
+++ b/docs/source/pyasn1/type/opentype/opentype.rst
@@ -9,9 +9,4 @@
 .. autoclass:: pyasn1.type.opentype.OpenType
    :members:
 
-   .. note::
-
-        The |OpenType| class models an untyped field of a constructed ASN.1
-        type. In ASN.1 syntax it is usually represented by the
-        `ANY DEFINED BY` clause. Typically used with :ref:`Any <univ.Any>`
-        type.
+More information on open type use can be found :ref:`here <type.opentype>`.
\ No newline at end of file
diff --git a/docs/source/pyasn1/type/univ/sequence.rst b/docs/source/pyasn1/type/univ/sequence.rst
index 6d2c6b5..f48f497 100644
--- a/docs/source/pyasn1/type/univ/sequence.rst
+++ b/docs/source/pyasn1/type/univ/sequence.rst
@@ -6,7 +6,7 @@
 |ASN.1| type
 ------------
 
-.. autoclass:: pyasn1.type.univ.Sequence(componentType=None, tagSet=tagSet(), subtypeSpec=ConstraintsIntersection(), sizeSpec=ConstraintsIntersection())
+.. autoclass:: pyasn1.type.univ.Sequence(componentType=NamedTypes(), tagSet=tagSet(), subtypeSpec=ConstraintsIntersection(), sizeSpec=ConstraintsIntersection())
    :members: isValue, isSameTypeWith, isSuperTypeOf, tagSet, effectiveTagSet, tagMap, componentType, subtypeSpec, sizeSpec, getComponentByPosition,
              setComponentByPosition, getComponentByName, setComponentByName, setDefaultComponents
 
@@ -15,5 +15,5 @@
         The |ASN.1| type models a collection of named ASN.1 components.
         Ordering of the components **is** preserved upon de/serialisation.
 
-   .. automethod:: pyasn1.type.univ.Sequence.clone(componentType=None, tagSet=tagSet(), subtypeSpec=ConstraintsIntersection())
-   .. automethod:: pyasn1.type.univ.Sequence.subtype(componentType=None, implicitTag=Tag(), explicitTag=Tag(),subtypeSpec=ConstraintsIntersection())
+   .. automethod:: pyasn1.type.univ.Sequence.clone(componentType=NamedTypes(), tagSet=tagSet(), subtypeSpec=ConstraintsIntersection())
+   .. automethod:: pyasn1.type.univ.Sequence.subtype(componentType=NamedTypes(), implicitTag=Tag(), explicitTag=Tag(),subtypeSpec=ConstraintsIntersection())
diff --git a/docs/source/pyasn1/type/univ/sequenceof.rst b/docs/source/pyasn1/type/univ/sequenceof.rst
index dadef67..ed6a781 100644
--- a/docs/source/pyasn1/type/univ/sequenceof.rst
+++ b/docs/source/pyasn1/type/univ/sequenceof.rst
@@ -6,7 +6,7 @@
 |ASN.1| type
 ------------
 
-.. autoclass:: pyasn1.type.univ.SequenceOf(componentType=None, tagSet=TagSet(), subtypeSpec=ConstraintsIntersection(), sizeSpec=ConstraintsIntersection())
+.. autoclass:: pyasn1.type.univ.SequenceOf(componentType=NoValue(), tagSet=TagSet(), subtypeSpec=ConstraintsIntersection(), sizeSpec=ConstraintsIntersection())
    :members: isValue, isSameTypeWith, isSuperTypeOf, tagSet, effectiveTagSet, tagMap, componentType, subtypeSpec, sizeSpec,
              getComponentByPosition, setComponentByPosition
 
@@ -15,5 +15,5 @@
        The |ASN.1| type models a collection of elements of a single ASN.1 type.
        Ordering of the components **is** preserved upon de/serialisation.
         
-   .. automethod:: pyasn1.type.univ.SequenceOf.clone(componentType=None, tagSet=TagSet(), subtypeSpec=ConstraintsIntersection())
-   .. automethod:: pyasn1.type.univ.SequenceOf.subtype(componentType=None, implicitTag=Tag(), explicitTag=Tag(),subtypeSpec=ConstraintsIntersection())
+   .. automethod:: pyasn1.type.univ.SequenceOf.clone(componentType=NoValue(), tagSet=TagSet(), subtypeSpec=ConstraintsIntersection())
+   .. automethod:: pyasn1.type.univ.SequenceOf.subtype(componentType=NoValue(), implicitTag=Tag(), explicitTag=Tag(),subtypeSpec=ConstraintsIntersection())
diff --git a/docs/source/pyasn1/type/univ/set.rst b/docs/source/pyasn1/type/univ/set.rst
index 5c75938..229525a 100644
--- a/docs/source/pyasn1/type/univ/set.rst
+++ b/docs/source/pyasn1/type/univ/set.rst
@@ -6,7 +6,7 @@
 |ASN.1| type
 ------------
 
-.. autoclass:: pyasn1.type.univ.Set(componentType=None, tagSet=TagSet(), subtypeSpec=ConstraintsIntersection(), sizeSpec=ConstraintsIntersection())
+.. autoclass:: pyasn1.type.univ.Set(componentType=NamedTypes(), tagSet=TagSet(), subtypeSpec=ConstraintsIntersection(), sizeSpec=ConstraintsIntersection())
    :members: isValue, isSameTypeWith, isSuperTypeOf, tagSet, effectiveTagSet, tagMap, componentType, subtypeSpec, sizeSpec,
              getComponentByPosition, setComponentByPosition, getComponentByName, setComponentByName, setDefaultComponents,
              getComponentByType, setComponentByType
@@ -16,5 +16,5 @@
         The |ASN.1| type models a collection of named ASN.1 components.
         Ordering of the components **is not** preserved upon de/serialisation.
 
-   .. automethod:: pyasn1.type.univ.Set.clone(componentType=None, tagSet=TagSet(), subtypeSpec=ConstraintsIntersection())
-   .. automethod:: pyasn1.type.univ.Set.subtype(componentType=None, implicitTag=Tag(), explicitTag=Tag(),subtypeSpec=ConstraintsIntersection())
+   .. automethod:: pyasn1.type.univ.Set.clone(componentType=NamedTypes(), tagSet=TagSet(), subtypeSpec=ConstraintsIntersection())
+   .. automethod:: pyasn1.type.univ.Set.subtype(componentType=NamedTypes(), implicitTag=Tag(), explicitTag=Tag(),subtypeSpec=ConstraintsIntersection())
diff --git a/docs/source/pyasn1/type/univ/setof.rst b/docs/source/pyasn1/type/univ/setof.rst
index 0317f4a..2745677 100644
--- a/docs/source/pyasn1/type/univ/setof.rst
+++ b/docs/source/pyasn1/type/univ/setof.rst
@@ -6,7 +6,7 @@
 |ASN.1| type
 ------------
 
-.. autoclass:: pyasn1.type.univ.SetOf(componentType=None, tagSet=TagSet(), subtypeSpec=ConstraintsIntersection(), sizeSpec=ConstraintsIntersection())
+.. autoclass:: pyasn1.type.univ.SetOf(componentType=NoValue(), tagSet=TagSet(), subtypeSpec=ConstraintsIntersection(), sizeSpec=ConstraintsIntersection())
    :members: isValue, isSameTypeWith, isSuperTypeOf, tagSet, effectiveTagSet, tagMap, componentType, subtypeSpec, sizeSpec,
              getComponentByPosition, setComponentByPosition
 
@@ -15,5 +15,5 @@
         The |ASN.1| type models a collection of elements of a single ASN.1 type.
         Ordering of the components **is not** preserved upon de/serialisation.
 
-   .. automethod:: pyasn1.type.univ.SetOf.clone(componentType=None, tagSet=TagSet(), subtypeSpec=ConstraintsIntersection())
-   .. automethod:: pyasn1.type.univ.SetOf.subtype(componentType=None, implicitTag=Tag(), explicitTag=Tag(),subtypeSpec=ConstraintsIntersection())
+   .. automethod:: pyasn1.type.univ.SetOf.clone(componentType=NoValue(), tagSet=TagSet(), subtypeSpec=ConstraintsIntersection())
+   .. automethod:: pyasn1.type.univ.SetOf.subtype(componentType=NoValue(), implicitTag=Tag(), explicitTag=Tag(),subtypeSpec=ConstraintsIntersection())
diff --git a/pyasn1/type/base.py b/pyasn1/type/base.py
index b231482..61ec35e 100644
--- a/pyasn1/type/base.py
+++ b/pyasn1/type/base.py
@@ -12,7 +12,8 @@ from pyasn1.type import constraint
 from pyasn1.type import tag
 from pyasn1.type import tagmap
 
-__all__ = ['Asn1Item', 'Asn1ItemBase', 'AbstractSimpleAsn1Item', 'AbstractConstructedAsn1Item']
+__all__ = ['Asn1Item', 'Asn1ItemBase', 'AbstractSimpleAsn1Item',
+           'AbstractConstructedAsn1Item']
 
 
 class Asn1Item(object):
@@ -535,8 +536,7 @@ class AbstractConstructedAsn1Item(Asn1ItemBase):
         Note
         ----
         Due to the mutable nature of the |ASN.1| object, even if no arguments
-        are supplied, new |ASN.1| object will always be created as a shallow
-        copy of `self`.
+        are supplied, a new |ASN.1| object will be created and returned.
         """
         cloneValueFlag = kwargs.pop('cloneValueFlag', False)
 
@@ -588,9 +588,8 @@ class AbstractConstructedAsn1Item(Asn1ItemBase):
 
         Note
         ----
-        Due to the immutable nature of the |ASN.1| object, if no arguments
-        are supplied, no new |ASN.1| object will be created and `self` will
-        be returned instead.
+        Due to the mutable nature of the |ASN.1| object, even if no arguments
+        are supplied, a new |ASN.1| object will be created and returned.
         """
 
         initializers = self.readOnly.copy()
diff --git a/pyasn1/type/opentype.py b/pyasn1/type/opentype.py
index d37a533..29645f0 100644
--- a/pyasn1/type/opentype.py
+++ b/pyasn1/type/opentype.py
@@ -11,11 +11,22 @@ __all__ = ['OpenType']
 class OpenType(object):
     """Create ASN.1 type map indexed by a value
 
-    The *DefinedBy* object models the ASN.1 *DEFINED BY* clause which maps
-    values to ASN.1 types in the context of the ASN.1 SEQUENCE/SET type.
-
-    OpenType objects are duck-type a read-only Python :class:`dict` objects,
-    however the passed `typeMap` is stored by reference.
+    The *OpenType* object models an untyped field of a constructed ASN.1
+    type. In ASN.1 syntax it is usually represented by the
+    `ANY DEFINED BY` for scalars or `SET OF ANY DEFINED BY`,
+    `SEQUENCE OF ANY DEFINED BY` for container types clauses. Typically
+    used together with :class:`~pyasn1.type.univ.Any` object.
+
+    OpenType objects duck-type a read-only Python :class:`dict` objects,
+    however the passed `typeMap` is not copied, but stored by reference.
+    That means the user can manipulate `typeMap` at run time having this
+    reflected on *OpenType* object behavior.
+
+    The |OpenType| class models an untyped field of a constructed ASN.1
+    type. In ASN.1 syntax it is usually represented by the
+    `ANY DEFINED BY` for scalars or `SET OF ANY DEFINED BY`,
+    `SEQUENCE OF ANY DEFINED BY` for container types clauses. Typically
+    used with :class:`~pyasn1.type.univ.Any` type.
 
     Parameters
     ----------
@@ -28,12 +39,14 @@ class OpenType(object):
 
     Examples
     --------
+
+    For untyped scalars:
+
     .. code-block:: python
 
         openType = OpenType(
-            'id',
-            {1: Integer(),
-             2: OctetString()}
+            'id', {1: Integer(),
+                   2: OctetString()}
         )
         Sequence(
             componentType=NamedTypes(
@@ -41,6 +54,22 @@ class OpenType(object):
                 NamedType('blob', Any(), openType=openType)
             )
         )
+
+    For untyped `SET OF` or `SEQUENCE OF` vectors:
+
+    .. code-block:: python
+
+        openType = OpenType(
+            'id', {1: Integer(),
+                   2: OctetString()}
+        )
+        Sequence(
+            componentType=NamedTypes(
+                NamedType('id', Integer()),
+                NamedType('blob', SetOf(componentType=Any()),
+                          openType=openType)
+            )
+        )
     """
 
     def __init__(self, name, typeMap=None):