started to update docs for new qface v2

7 files changed, 191 insertions, 154 deletions

diff --git a/docs/grammar.rst b/docs/grammar.rst
index a4c2663..ee6fa06 100644
--- a/docs/grammar.rst
+++ b/docs/grammar.rst
@@ -2,15 +2,16 @@
 Grammar
 =======
 
-QFace (Qt interface language) is an Interface Description Language (IDL). While it is primarily designed to define an interface between Qt, QML and C++, it is intended to be flexible enough also to be used in other contexts.
+QFace (Quick Interface Language) is an Interface Description Language (IDL). While it is primarily designed to define an interface between Qt, QML and C++, it is intended to be flexible enough also to be used in other contexts.
 
 The grammar of QFace is well defined and is based on the concept of modules as larger collections of information.
 
-A module can have several interfaces, structs and/or enums/flags.
+A module can have several interfaces, structs and/or enums/flags. Here a not complete unformal grammar.
 
 .. code-block:: html
 
     module <module> <version>
+
     import <module> <version>
 
     interface <Identifier> {
@@ -20,7 +21,7 @@ A module can have several interfaces, structs and/or enums/flags.
     }
 
     struct <Identifier> {
-        <type> <identifier>;
+        <type> <identifier>
     }
 
     enum <Identifier> {
@@ -31,7 +32,7 @@ A module can have several interfaces, structs and/or enums/flags.
         <name> = <value>,
     }
 
-A QFace document always describes one module. Each document can contain one or more interfaces, structs, flags or enums. Each document can import other modules using the import statement.
+A QFace document always describes one module. It is convention that a qface document is named after the module. If the same module appears in different documents the behavior is not defined currently. Each document can contain one or more interfaces, structs, flags or enums. Each document can import other modules using the import statement.
 
 
 Module
@@ -47,6 +48,8 @@ A module is identified by its name. A module should be normally a URI where all
     import org.common 1.0
 
 
+.. note:: The parser will not validate if the module exists yet. It will just provide the reference to the module and will try to resolve the module on code-generation runtime.
+
 Interface
 =========
 
@@ -62,6 +65,9 @@ An interface is a collection of properties, operations and signals. Properties c
 
 QFace allows to extend interfaces using the ``extends`` keyword after the interface name.
 
+.. note:: It is in the responsibility of the author to ensure the order of interfaces are correct. E.g. the base interface should come before the depending interface. QFace does not try to re-order interfaces based on dependency. The order they appear in the document is the order they are passed to the code generator.
+
+
 .. code-block:: js
 
     interface Station {
@@ -75,12 +81,12 @@ QFace allows to extend interfaces using the ``extends`` keyword after the interf
 
 .. note::
 
-    For the sake of simplicity, as an API designer you should carefully evaluate if this is required. The typical way in QFace to allow extensions is normally to write your own code-generator and use type annotations.
+    For the sake of simplicity, as an API designer you should carefully evaluate if an extension is required. The typical way in QFace to allow extensions is normally to write your own code-generator and use type annotations for kind of interfaces.
 
 
     .. code-block:: js
 
-        @extends: Station
+        @kind: Station
         interface WeatherStation {
             real temperature;
         }
@@ -88,11 +94,10 @@ QFace allows to extend interfaces using the ``extends`` keyword after the interf
     The API reader does not need to know the internals of the API. The station behavior would be automatically attached by the custom generator.
 
 
-
 Struct
 ======
 
-The struct resembles a data container. It consist of a set of fields where each field has a data type and a name.
+The struct resembles a data container. It consist of a set of fields where each field has a name and a data type. Data types can be primitive of complex types.
 
 .. code-block:: js
 
@@ -112,29 +117,31 @@ Structs can also be nested. A struct can be used everywhere where a type can be
         signal error(Error error);
     }
 
+.. note:: When you nest structs, ensure the used struct comes before the using structs and there are no circular dependencies. The order struct appear is the same order they are passed to the code generator.
+
 
 
 Enum/Flag
 =========
 
-An enum and flag is an enumeration type. The value of each member is automatically assigned if missing.
+An enum and flag is an enumeration type. The value of each member is automatically assigned if missing and starts with 0.
 
 .. code-block:: js
 
     enum State {
-        Null,
-        Loading,
-        Ready,
-        Error
+        Null,       // implicit 0
+        Loading,    // will be one
+        Ready,      // will be two
+        Error       // will be three
     }
 
-The value assignment for the enum type is sequential beginning from 0. To specify the exact value you can assign a value to the member.
+The value assignment for the enum type is sequential beginning from 0. To specify the exact value you can assign a value to the member. The value can also be written in hex form (e.f. 0xN).
 
 .. code-block:: js
 
     enum State {
         Null = 0,
-        Loading = 1,
+        Loading = 0x1,
         Ready = 2,
         Error = 3
     }
@@ -144,14 +151,13 @@ The flag type defines an enumeration type where different values are treated as
 .. code-block:: js
 
     flag Cell {
-        Null,
-        Box,
-        Wall,
-        Figure
+        Null,   // starting value is one
+        Box,    // value is two
+        Wall,   // value is four
+        Figure  // value is eight
     }
 
 
-
 Types
 =====
 
diff --git a/docs/index.rst b/docs/index.rst
index 12a05b7..e7a6e42 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -2,6 +2,8 @@
 QFace
 =====
 
+* see https://github.com/pelagicore/qface
+
 QFace is a flexible API generator inspired by the Qt API idioms. It uses a common IDL format (called QFace interface document) to define an API. QFace is optimized to write a custom generator based on the common IDL format.
 
 Several code generators for common use cases have already been implemented. These can be used as is or can be used as a base for a custom generator.
@@ -11,30 +13,30 @@ Several code generators for common use cases have already been implemented. Thes
 
    motivation
    usage
-   builtin
+   reference
    grammar
    annotations
    yaml
    json
    domain
    extending
-   script
+   rules
    api
 
 
 Features
 ========
 
-The list fo features is plit between features which are based on the chosen IDL and features which are provided by the generator itself.
+The list fo features is split between features which are based on the chosen IDL and features which are provided by the generator itself.
 
 .. rubric:: IDL Features
 
 - Common modern IDL
 - Scalable through modules
-- Structure through structs, enums, flags
+- Structured data through structs, enums, flags
 - Interface API with properties, operations and signals
 - Annotations using YAML syntax
-- Documentable IDL
+- Fully documentable
 
 .. rubric:: Generator Features
 
@@ -62,25 +64,19 @@ You can verify that you have qface installed with
 
 .. code-block:: sh
 
-    python3
-
-and then
-
-.. code-block:: python3
-
-    import qface
+    qface --help
 
 
 Custom Generator
 ----------------
 
-For your own generator you need several ingredients. A QFace interface file (here called "sample.qface"), a small script which loads and parses the document (generator.py) and one or more template files, used by the script to generate the resulting code.
+To write a custom generator it is normally enough to write a generator rules and the used templates. We use a QFace interface file (here called "sample.qface") as an example.
 
 The QFace document could look like this
 
 .. code-block:: thrift
 
-    // sample.qface
+    // interfaces/sample.qface
     module org.example 1.0
 
     interface Echo {
@@ -88,35 +84,35 @@ The QFace document could look like this
     }
 
 
-Your generator can now parse the documents and call the templates
+We need now to write our templates for the code generation. In our example we would simple print out for each module the interfaces and it's operations.
+
+.. code-block:: jinja
 
-.. code-block:: python3
+    {# templates/module.tpl #}
+    {% for interface in module.interfaces %}
+    {{module}}.{{interface}}
+    {% endfor %}
 
-    // generator.py
-    from qface.generator import FileSystem, Generator
+This will write for each interface in the module the text ``<module>.<interface>``. The rules file will define what shall be generated and where.
 
-    system = FileSystem.parse('sample.qface')
-    generator = Generator('./templates')
-    for module in system.modules:
-        ctx = { 'module' : module }
-        generator.write('{{module}}.txt', 'module.tpl', ctx)
+.. code-block:: yaml
 
-The final piece is the template to parameterize the output in this case called "module.tpl"
+    # qface-rules.yaml
 
-.. code-block:: jinja
+    project:
+      module:
+        documents:
+          - {{module}}.csv: module.tpl
 
-    // templates/module.tpl
-    {% for interface in module.interfaces %}
-    {{module.name}}
-    {% endfor %}
+The first entry defined a scope (e.g. project). Then for each module we geenrated documents. We use the module.tpl document from the templates folder and generate a CSV document based on the module name.
 
-Now you can simple call your script
+Now you can simple call your rules document
 
 .. code-block:: bash
 
-    python3 generator.py
+    qface --rules qface-rules.yaml --target output interfaces
 
-And a "org.example.txt" file named after the module should be generated.
+And a "org.example.csv" file named after the module should be generated.
 
 .. rubric:: See Also
 
@@ -125,15 +121,14 @@ And a "org.example.txt" file named after the module should be generated.
 * :doc:`domain`
 * :doc:`api`
 
-Bundled Generators
-------------------
+Generators
+----------
 
-QFace has some generators which are bundled with the QFace library. They live in their own repositories. These generators are documented in their respective repositories.
+QFace has several generator maintained by the qface team. They are maintained and documented in their own repositories.
 
-.. rubric:: See Also
+* https://github.com/pelagicore/qface-qtcpp
+* https://github.com/pelagicore/qface-qtqml
 
-* :doc:`qtcpp`
-* :doc:`qtqml`
 
 
 
diff --git a/docs/motivation.rst b/docs/motivation.rst
index 299c160..b121cfb 100644
--- a/docs/motivation.rst
+++ b/docs/motivation.rst
@@ -2,12 +2,12 @@
 Motivation
 ==========
 
-QFace is an attempt to establish one interface description language with an easy to use code generator framework. While QFace as an interface description language is Qt friendly, it is not limited to Qt. The hope is that many projects can agree on this interface language and many interesting generators will be created and we all can learn from how other projects generate code from the same IDL.
+QFace is an attempt to establish a common interface description language with an easy to use code generator framework. While QFace as an interface description language which is Qt friendly, it is not limited to Qt usage. The vision is that many projects can agree on this interface language and many different generators will be created. In the end we all can learn from how other projects generate code from the same IDL.
 
 The IDL
 =======
 
-The IDL uses common API concept such as modules, interfaces, properties, structs and enums/flags. Additionally it knows about lists, maps and models. A list is an array of primitive or complex types. A map is an associative array of key/value pairs. A model is an indicator for large data sets which are typically used via a defined interface or via pagination.
+The IDL uses common API concept such as ``modules``, ``interfaces``, ``properties``, ``structs`` and ``enums``/``flags``. Additionally it knows about ``lists``, ``maps`` and ``models``. A list is an array of primitive or complex types. A map is an associative array of key/value pairs. A model is an indicator for large data sets which are typically used using a streaming concept, e.g. via a defined interface or via pagination.
 
 .. code-block:: js
 
@@ -27,51 +27,57 @@ The IDL uses common API concept such as modules, interfaces, properties, structs
 
 The data types provided by QFace can be divided into primitive and complex types:
 
-Primitive Types
+.. rubric:: Primitive Types
 
-* bool
-* int
-* real
-* string
-* var
+The primiteve types are mostly modeled after the JSON data types (see https://www.json.org/).
 
-Complex Types
+All exact data types (e.g. 32bit, 64bit, etc depend on the generator the IDL).
 
-* Interface
-* Struct
-* Enum
-* Flag
-* List
-* Map
-* Model
+* ``bool`` - true/false
+* ``int`` - represents integer type
+* ``real`` - floating point number
+* ``string`` - unicode string
+* ``var`` - placeholder for a data type
+
+.. rubric:: Complex Types
+
+A complex type is a composition of other types and add specific semantic to the type.
+
+* ``Interface`` - collection of properties, operations and signals
+* ``Struct`` - typed data package
+* ``Enum`` - enumeration of integer values
+* ``Flag`` - enumeration with n^2 values
+* ``List`` - array of primitive or complex data types
+* ``Map`` - collection of primitive or complex value. Key type is defined by generator. E.g. string is recommended.
+* ``Model`` - A stream of primitive or complex data values.
 
 
 Why another IDL
 ===============
 
-Many IDLs are already in existence. Most of them are bound to a certain technology or library or are limited for a specific use. Only a few IDLs exist which are independent from a technology. From these few which are known to the author none satisfied the requirement from the author to be Qt compatible and easy to use. Also the IDL should be easy to install and be extendable. The unique mix of technologies used in QFace allows it to provide a solid stable IDL with a powerful generation framework.
+Many IDLs are already in existence. But most of them are bound to a certain technology or library or are limited for a specific use. Only a few IDLs exist which are independent from a technology. From these few technology independent IDLs which are known to the author satisfied the requirement to be Qt compatible and easy to use. Also the IDL should be easy to install and be extendable. The unique mix of technologies used in QFace allows it to provide a solid stable IDL with a powerful generation framework. - The base for your own code generator.
 
 
 Defining APIs
 =============
 
-There are many opinions how to define APIs and what the best way is. The idea of QFace is that many projects find it useful and use the same IDL. Consequently, there will be a large set of generators and finally APIs can be compared and unified also if they will be used with different technologies.
+There are many opinions how to define APIs and what would be the best way. The idea of QFace is that many projects find the IDL useful and use it to create their own code generator. Consequently, there will be a large set of generators and finally APIs can be compared and unified, even if they will be used with different technologies.
 
-Even inside one technology there are often discussions about how an interface shall be coded. QFace allows the different parties to create their own generators based on the same API. Ideally at the end the knowledge how an interface shall be coded will reside in the provided generator.
+Inside one technology area there are often discussions between developers or teams about how an interface shall be coded. QFace allows the different parties to create their own generators based on the same API. Ideally at the end the knowledge how an interface shall be best coded will reside in the provided generator.
 
 Large Projects
 ==============
 
-In larger projects there is the need to make a large set of operating services available to QML. It is less about defining new visual items in C++, more about creating an abstraction of a service and make it available to the HMI developer inside QML.
+In larger projects there is the need to make a large set of operating services available to an user interface layer. It is less about defining new visual items in C++, more about creating an abstraction of a service and make it available to the UI developer.
 
-This can be a challenge when you create many of these plugins and in the middle of the project you figure out that you have issues with your current design or if the customer in the next project wants to use a different HMI technology. All the knowledge is inside these plugins.
+This can be a challenge when you create many plugins and in the middle of the project you figure out that you have issues with your current design or if the customer in the next project wants to use a different HMI technology. All the knowledge is inside these plugins.
 
-With QFace these companies can be certain that QFace does not lock them into the HMI technology and smaller design issues can be fixed by fixing the generator.
+With QFace these companies can be ensured that QFace does not lock them into the UI technology and smaller API design issues can be fixed by fixing the used code generator.
 
 Remote Services
 ===============
 
-Some projects use network communication to communicate from the HMI to the services, which might run on a different process or even a networked device. QFace was not designed for remote services as it does not define any storage types (e.g. int32, int16, int64), it  only knows an int and does not define how large the int shall be. For this QFace needs to rely on the author of the generators to have a defined protocol to exchange data.
+Some projects use network communication to communicate from the HMI to the services, which might run on a different process or even a networked device. QFace was not designed for remote services as it does not define any storage types (e.g. int32, int16, int64), it  only knows an ``int`` and does not define how large the ``int`` shall be. For this QFace needs to rely on the author of the generators to have a defined protocol to exchange data using QFace common data types.
 
 Complex APIs
 ============
@@ -83,6 +89,6 @@ QFace does not support unions or structs that extend other structs. If you look
 Limitations
 ===========
 
-Like other code generation tools, QFace is limited by how much information you can place inside your interface files. In excessive cases code generation might not make sense and hence QFace will also not help.
+Like other code generation tools, QFace is limited by how much information you can place inside your interface files. In excessive cases code generation might not make sense and hence QFace will also not help you.
 
-QFace allows you to use annotations which can add meta information to the interface files. But the generator needs to be designed to understand this meta information. Only the structure of these annotations are defined not the information they carry. Annotations might help to add information to an interface document to better control the code generation process.
+QFace allows you to use annotations which can add meta information to the interface files. But the generator needs to be designed to understand this meta information. QFace only defined the the structure of these annotations not the information and semantic they carry. Annotations might help you to add information to an interface document to better control the code generation process.
diff --git a/docs/builtin.rst b/docs/reference.rst
index b0a339b..4c87a83 100644
--- a/docs/builtin.rst
+++ b/docs/reference.rst
@@ -1,7 +1,7 @@
-Generator Examples
-==================
+Reference Generators
+====================
 
-QFace does provide some real world generators which are hosted as separated projects. Their purpose is merely to showcase how to write a code generator with QFace. They are working and complete examples of general purpose generators.
+QFace does provide some real world reference generators which are hosted as separated projects. Their purpose is merely to showcase how to write a code generator using QFace. They are working and complete examples of general purpose generators.
 
 `qface-qtcpp`_
 
diff --git a/docs/rules.rst b/docs/rules.rst
new file mode 100644
index 0000000..0d4cd02
--- /dev/null
+++ b/docs/rules.rst
@@ -0,0 +1,52 @@
+**********
+Rules Mode
+**********
+
+In the rules mode, qface is used as the qface exectuable. In this mode the code generator consits of a rule document and template files and optionally a filter module.
+
+Whereas normally the generator writer create an own python package in this module only some documents are needed and the qface rules is used.
+
+Setup
+=====
+
+To get started create a `qface-rules.yml` document and a templates folder::
+
+    qface-rules.yml
+    templates/
+
+
+In the rules file you provide the code generation rules according to the rule generator documentation. The templates folder will contain the required templates.
+
+Filters
+=======
+
+To provide extra filder you need to create a `filters.py` document with the declaration of your filters::
+
+    # a filter takes in a domain element or string
+    # and returns a string
+    def echo(s):
+        return '{} World!'.format(s)
+
+    def get_filters():
+        # returns a dict of new filters
+        return {
+            'echo': echo
+        }
+
+The filters module will be loaded by qface and all entries to the filters dictionary are added to the global lists of Jinja filters. You can now use it like any other Jinja filter.
+
+.. code-block:: jinja
+
+    {{ "Hello" | echo }}
+
+Will resolve to ``Hello World!``.
+
+
+Running
+=======
+
+To run now the generator you can simply call::
+
+    qface --rules qface-rules.yml --target out counter.qface
+
+This will take your rules and generate the files inside the out folder based on the `counter.qface` interface file.
diff --git a/docs/script.rst b/docs/script.rst
deleted file mode 100644
index fb3def2..0000000
--- a/docs/script.rst
+++ /dev/null
@@ -1,40 +0,0 @@
-***********
-Script Mode
-***********
-
-In the script mode, qface is used as the qface exectuable. In this mode the code generator consits of a rule document and template files and optionally a filter module.
-
-Whereas normally the generator writer create an own python package in this module only some documents are needed and the qface script is used.
-
-Setup
-=====
-
-To get started create a `qface-rules.yml` document and a templates folder::
-
-    qface-rules.yml
-    templates/
-
-
-In the rules file you provide the code generation rules accoring to the rule generator documentation. The templates folder will contain the required templates.
-
-Filters
-=======
-
-To provide extra filder you need to create a `filters.py` document with the declaration of your filters::
-
-    def echo(s):
-        print(s)
-        return "World"
-
-    filters['echo'] = echo
-
-The filters module will be loaded by qface and all entries to the filters dictionary are added to the global lists of Jinja filters.
-
-Running
-=======
-
-To run now the generator you can simply call::
-
-    qface --rules qface-rules.yml --target out counter.qface
-
-This will take your rules and generate the files inside the out folder based on the `counter.qface` interface file.
diff --git a/docs/usage.rst b/docs/usage.rst
index 87216c7..4256012 100644
--- a/docs/usage.rst
+++ b/docs/usage.rst
@@ -5,56 +5,48 @@ Usage
 Concept
 =======
 
-QFace requires one or more IDL files as input file and a generator to produce output files. The IDL files are named QFace interface documents.
+QFace requires one or more IDL files as input file and a custom generator to produce output files. The IDL files are named QFace interface documents.
 
 .. figure:: qface_concept.jpg
 
-To use QFace you need to write your own generator. A generator is a small python script which reads the QFace document and writes code using template files.
+To use QFace you need to write your own generator. A generator is a small rules document which reads the QFace document and writes code using template files.
 
-.. code-block:: python
+.. code-block:: yaml
 
-    # gen.py
-    from qface.generator import FileSystem, Generator
+    # rules-qface.yaml
+    project:
+        interface:
+            - {{interface}}.h: interface.h
+            - {{interface}}.h: interface.cpp
+            - Makefile: Makefile
 
-    def generate(input, output):
-        # parse the interface files
-        system = FileSystem.parse(input)
-        # setup the generator
-        generator = Generator(search_path='templates')
-        # create a context object
-        ctx = {'output': output, 'system': system}
-        # apply the context on the template and write the output to file
-        generator.write('{{output}}/modules.csv', 'modules.csv', ctx)
-
-    # call the generation function
-    generate('sample.qface', 'out')
 
+You then call the script using the qface executable.
 
 .. code-block:: sh
 
-    python3 gen.py
+    qface --rules rules-qface.yaml --target output echo.qface
 
 
 Code Generation Principle
 =========================
 
-The code generation is driven by a small script which iterates over the domain model and writes files using the Python Jinja template language. Refer to http://jinja.pocoo.org and particularly the template designer documentation at http://jinja.pocoo.org/docs/dev/templates/.
+The code generation is driven by a rules document which applies the domain model and writes files using the Python Jinja template language.
 
-.. code-block:: python
+.. note:: Refer to http://jinja.pocoo.org and particularly the template designer documentation at http://jinja.pocoo.org/docs/dev/templates/.
 
-    from qface.generator import FileSystem, Generator
+.. code-block:: yaml
 
-    def generate(input, output):
-        system = FileSystem.parse(input)
-        generator = Generator(searchpath='templates')
-        ctx = {'output': output, 'system': system}
-        generator.write('{{output}}/modules.csv', 'modules.csv', ctx)
+    project:
+        system:
+            - project_report.csv: report.tpl
 
 
-This script reads the input directory and returns a system object from the domain model. This is used as the root object for the code generation inside the template language.
+The qface executable reads the input qface files and converts them into a domain model. The domain model is then passed into the rules document. Inside the rules document you specify scopes and matches. If a ``system`` is specified as the match the ``system`` is passed into the given template documents.
 
 .. code-block:: jinja
 
+    {# templates/report.tpl #}
     {% for module in system.modules %}
         {%- for interface in module.interfaces -%}
         INTERFACE, {{module}}.{{interface}}
@@ -67,4 +59,30 @@ This script reads the input directory and returns a system object from the domai
         {% endfor -%}
     {% endfor %}
 
-The template iterates over the domain objects and generates text which is written into the output file. Using the generator write method ``generator.write(path, template, context)`` the output file path can be specified.
+The template iterates over the domain objects and generates text which is written into the output file in the given target folder.
+
+You call the yaml document by calling the qface executable and provide the rules document as also the output document. The domain model is created based on the given input files.
+
+.. code-block:: sh
+
+    qface --rules rules-qface.yaml --target output echo.qface
+
+
+To know more about the different options just ask the help of qface.
+
+.. code-block:: sh
+
+    qface --help
+
+    Usage: qface [OPTIONS] [SOURCE]...
+
+    Options:
+      --rules PATH
+      --target DIRECTORY
+      --reload / --no-reload      Auto reload script on changes
+      --scaffold / --no-scaffold  Add extrac scaffolding code
+      --watch DIRECTORY
+      --feature TEXT
+      --run TEXT                  run script after generation
+      --force / --no-force        forces overwriting of files
+      --help                      Show this message and exit.