DOC: Update documentation, closer to 1.0 release

13 files changed, 346 insertions, 248 deletions

diff --git a/doc/compiling b/doc/compilation.mdwn
index 32ac930..37876db 100644
--- a/doc/compiling
+++ b/doc/compilation.mdwn
@@ -1,70 +1,68 @@
-The nitty-gritty of Lace's compilation process
-==============================================
+Lace - The nitty-gritty of the compilation process
+==================================================
 
 When you construct a Lace engine, you give it a compilation callback
 set.  That set is used to call you back when Lace encounters something it
 needs help compiling.  The structure of it is:
 
-{ _lace = {
-   loader = function(compcontext, nametoload) ... end,
-   commands = {
-      ... = function(compcontext, words...) ... end,
-   },
-   controltype = {
-      ... = function(compcontext, type, words...) ... end,
-   }
-} }
-
-Anything outside of the "_lace" entry in the context is considered
+    { _lace = {
+       loader = function(compcontext, nametoload) ... end,
+       commands = {
+          ... = function(compcontext, words...) ... end,
+       },
+       controltype = {
+          ... = function(compcontext, type, words...) ... end,
+       }
+    } }
+
+Anything outside of the `_lace` entry in the context is considered
 fair game structure-wise and can be used by the functions called back
 to acquire internal pointers etc.  Note however that the compilation
 context will not be passed around during execution so if you need to
 remember some of it in order to function properly, then it's up to
-you.  Also note that anything not defined above in the "_lace" entry
+you.  Also note that anything not defined above in the `_lace` example
 is considered "private" to Lace and should not be touched by non-Lace
-code.  Lace will usually be considerate and will not create arbitrary
-entries in that table which do not start with a dot.  This allows for
-forward/backward compatibility to some extent.
+code.
 
-In addition, Lace will maintain a 'source' entry in the _lace table
+In addition, Lace will maintain a `source` entry in the _lace table
 with the lexed source which is being compiled and, if we're compiling
 an included source, a parent entry with the compilation context of the
 parent.  The toplevel field is the compilation context of the top
 level compilation.  If parent is nil, then toplevel will equal
-compcontext.  Lace also maintains a 'linenr' entry with the
+compcontext.  Lace also maintains a `linenr` entry with the
 currently-being-compiled line number, so that commands and control
 types can use that in error reports if necessary.
 
-If loader is absent then the include statement refuses to include
+If `loader` is absent then the `include` statement refuses to include
 anything mandatory.  If it is present but returns nil when called then
-the include statement fails any mandatory includes which do so.
+the `include` statement fails any mandatory includes which do so.
 
 Otherwise the loader is expected to return the 'real' name of the
 source and the content of it.  This allows for symbolic lookups.
 
 If Lace encounters a command it does not recognise then it will call
-context.commands[cmdname] passing in the words representing the line
+`context._lace.commands[cmdname]` passing in the words representing the line
 in question.  It's up to that function to compile the line or else to
-return an error.  (More later)
+return an error.
 
-If Lace encounters a control type during a define command which it
-does not understand, then it calls the context.controltype[name]
+If Lace encounters a control type during a `define` command which it
+does not understand, then it calls the `context._lace.controltype[name]`
 function passing in all the remaining arguments.  The control type
 function is expected to return a compiled set for the define or else
-an error.  (More later)
+an error.
 
 To start a Lace engine compiling a ruleset, simply do (pseudocode):
 
     rules, err = lace.compiler.compile(compcontext, sourcename[, sourcecontent])
 
-If sourcecontent is not given, Lace will use the loader in the
-compcontext to load the source.
+If `sourcecontent` is not given, Lace will use the loader in the
+`compcontext` to load the source.
 
-If rules is nil, err is a Lua error.
-If rules is false, err is a nice error from compilation
-Otherwise, rules should be a table for the ruleset.
+If `rules` is `nil`, then `err` is a Lua error.
+If `rules` is `false`, then `err` is a formatted error from compilation
+Otherwise, `rules` should be a table containing the ruleset.
 
-Internally, once compiled, Lace rulesets are a list of tables.  Each
+Internally, once compiled, Lace rulesets are a table of tables.  Each
 rule entry has a reference to its source and line number.  It then has
 a function pointer for executing this rule, and a set of arguments to
 give the rule.  Lace automatically passes the execution context as the
@@ -74,31 +72,31 @@ the arguments to the function used to run the rule.
 Loader
 ======
 
-When Lace wishes to load an entry, it calls the loader function.  This
+When Lace wishes to load an entry, it calls the `loader` function.  This
 is to allow rulesets to be loadable from arbitrary locations such as
 files on disk, HTTP URLs, random bits of memory or even out of version
 control repositories directly.
 
-The loader is given the compilation context and the name of the source
-to load.  Note that while it has the compilation context, the loader
-function must be sensitive to the case of the initial load.  Under
-that circumstance, the source information in the compilation context
-will be unavailable.  The loader function is required to fit the
-following pseudocode definition:
+The `loader` function is given the compilation context and the name of the
+source to load.  Note that while it has the compilation context, the loader
+function must be sensitive to the case of the initial load.  Under that
+circumstance, the source information in the compilation context will be
+unavailable.  The loader function is required to fit the following pseudocode
+definition:
 
     realname, content = loader(compcontext, nametoload)
 
-If realname is not a string then content is expected to be an
-"internal" error message (see below) which will to prefixed with the
-calling source position etc and assembled into an error to return to
-the caller of lace.compiler.compile.
+If `realname` is not a string then content is expected to be an
+"internal" error message (see below) which will be augmented with the
+calling source position etc and rendered into an error to return to
+the caller of `lace.compiler.compile()`.
 
-If realname is a string then it is taken to be the real name of the
+If `realname` is a string then it is taken to be the real name of the
 loaded content (at worst you should return nametoload here) and
-content is a string representing the contents of that file.
+`content` is a string representing the contents of that file.
 
-Once loaded with the loader, Lace will then compile that sub-ruleset
-before continuing with the current ruleset.
+Once it has been loaded by the `loader` function, Lace will compile that
+sub-ruleset before continuing with the current ruleset.
 
 Commands
 ========
@@ -112,12 +110,12 @@ The command functions must fit the following pseudocode definition:
 
     cmdtab, msg = command_func(compcontext, words...)
 
-If cmdtab is not a table, msg should be an "internal" error message
-(see below) which will be prefixed with the calling source position
-etc and assembled into an error to return to the caller of
-lace.compiler.compile.
+If `cmdtab` is not a table, msg should be an "internal" error message
+(see below) which will be augmented with the calling source position
+etc and rendered into an error to return to the caller of
+`lace.compiler.compile()`.
 
-If cmdtab is a table, it is taken to be the compiled table
+If `cmdtab` is a table, it is taken to be the compiled table
 representing the command to run at ruleset execution time.  It should
 have the form:
 
@@ -126,12 +124,12 @@ have the form:
 Lace will automatically augment that with the source information which
 led to the compiled rule for use later.
 
-The exec_function is expected to fit the following pseudocode
+The `exec_function` is expected to fit the following pseudocode
 definition:
 
     result, msg = exec_function(exec_context, unpack(args))
 
-See execution for notes on how these exec_function functions are meant
+See [[execution]] for notes on how these `exec_function` functions are meant
 to behave.
 
 Control Types
@@ -141,53 +139,52 @@ When Lace is compiling a definition rule with a control type it has
 not got internally, Lace will call the controltype function associated
 with it (or report an error if no such control type is found).
 
-The control type functions must fir the following pseudocode
+The control type functions must fit the following pseudocode
 definition:
 
     ctrltab, msg = controltype_func(compcontext, type, words...)
 
-If ctrltab is not a table, msg should be an "internal" error message
-(see below) which will be prefixed with the calling source position
-etc and assembled into an error to return to the caller of
-lace.compiler.compile.
+If `ctrltab` is not a table, msg should be an "internal" error message
+(see below) which will be augmented with the calling source position
+etc and rendered into an error to return to the caller of
+`lace.compiler.compile()`.
 
-If ctrltab is a table, it is taken to be the compiled table
+If `ctrltab` is a table, it is taken to be the compiled table
 representing the control type to run at ruleset execution time.  It
 should have the form:
 
     { fn = ct_function, args = {...} }
 
-The exec_function is expected to fit the following pseudocode
+The `ct_function` is expected to fit the following pseudocode
 definition:
 
     result, msg = ct_function(exec_context, unpack(args))
 
-See execution for notes on how these ct_function functions are meant
+See [[execution]] for notes on how these `ct_function` functions are meant
 to behave.
 
 Compiler internal errors
 ========================
 
-Error messages during compilation are of the form:
+Error messages during compilation are generated by calling:
 
-{
-   msg = "some string with no newlines",
-   words = { ... }
-}
+    return lace.error.error("my message", { x, y })
 
-Where words is the numeric index of the words which caused the error.
-If words is empty (or nil) then the error is considered to be the
+Where the table is a list of numeric indices of the words which caused the
+error.  If words is empty (or nil) then the error is considered to be the
 entire line.
 
 Lace will use this information to construct meaningful long error
 messages which point at the words in question.  Such as:
 
-myruleset:6: Unknown command name: 'go_fish'
-             go_fish "I have no bananas"
-             ^^^^^^^
+    Unknown command name: 'go_fish'
+    myruleset :: 6
+    go_fish "I have no bananas"
+    ^^^^^^^
 
-In the case of control type compilation, the words will automatically
-be offset by the appropriate number to account for the define words.
-This means you should always 1-index from your arguments.
+In the case of control type compilation, the words will automatically be offset
+by the appropriate number to account for the define words.  This means you
+should always 1-index from your arguments where index 1 is the control type
+word index.
 
-The same kind of situation occurs during execution.
+The same kind of situation occurs during [[execution]].
diff --git a/doc/developing.mdwn b/doc/developing.mdwn
new file mode 100644
index 0000000..f82a593
--- /dev/null
+++ b/doc/developing.mdwn
@@ -0,0 +1,55 @@
+Lace - Helping with Development
+===============================
+
+The Lace codebase is divided up in to directories:
+
+    lace/
+         lib/
+             ... The Lace libraries live here
+         test/
+             ... All the tests and their data live here
+         example/
+             ... The example and its data lives here
+         extras/
+             ... The Lua coverage tool lives here
+         doc/
+             ... All the documentation lives here
+
+The codebase has a top level `Makefile` which defaults to running the test
+suite.  The test suite requires Lua be present and as well as running all the
+tests, also runs the Lua coverage tool to produce `luacov.report.out` which
+details the coverage of the Lace codebase.
+
+It is a policy that releases must have 100% coverage from the test suite.
+Ideally 100% coverage would be attained by the test cases for the given modules
+but sometimes cross-module usage is required in order to best provide 100%
+coverage.
+
+Any line not covered by the tests will be marked the `***0` in the
+`luacov.report.out` file.
+
+If you make substantive non-backward-compatible changes to the API of Lace then
+you should increment the ABI number in the main `lib/lace.lua` file.  If you
+make bug fixes or backward-compatible improvements then don't worry, the
+version number in that file will be incremented during the release process.
+
+If you add more modules to Lace, you should note that you need to update:
+
+1. The `Makefile`'s `MODULES` variable
+2. The `test/` directory will need a `test-lace.NEWMODULE.lua` file
+3. You will need to alter `lib/lace.lua` to pull it in and update
+   the `test/test-lace.lua` test to include a check for the new module
+4. You should ensure your new tests in 2 cover the module fully.
+5. You should ensure that any changes are shown in the example if possible.
+6. You should ensure any changes are reflected in the `docs/` files.
+
+You can check individual test suite coverage by running:
+
+    make MODULES=lace.SOMEMODULE
+
+This will cause the test suite for the named modules *only* to be run, and for
+the coverage data for that module to be generated.  It is not policy that a
+given module's individual tests MUST cover the module 100%, but if possible it
+SHOULD.  It is, as stated before, policy that the full test suite should cover
+100% of the modules when run as a whole though.
+
diff --git a/doc/execution b/doc/execution.mdwn
index 9b89590..50049a9 100644
--- a/doc/execution
+++ b/doc/execution.mdwn
@@ -1,29 +1,30 @@
-Execution of Lace rulesets
-==========================
+Lace - Execution of rulesets
+============================
 
-Once compiled, a ruleset is essentially a sequence of functions to
-call on the execution context.  The simplest execution context is an
-empty table.  If Lace is going to store anything it will use a "_lace"
-prefix as with compilation contexts.
+Once compiled, a ruleset is essentially a sequence of functions to call on the
+execution context.  The simplest execution context is an empty table.  If Lace
+is going to store anything it will use a `_lace` prefix as with [[compilation]]
+contexts.  As with compilation, the caller is not permitted to put anything
+inside `_lace` nor to rely on its layout.
 
 A few important functions make up the execution engine.  The top level
 function is simply:
 
     result, msg = lace.engine.run(ruleset, exec_context)
 
-This will run the ruleset with the given execution context and return
+This will run `ruleset` with the given execution context and return
 a simple result.
 
-If the result is nil, then msg is a long-form string error explaining
+If `result` is `nil`, then `msg` is a long-form string error explaining
 what went wrong.  It represents a Lua error being caught and as such
 you may not want to report it to your users.
 
-If the result is false, then msg is a long-form string error
+If `result` is `false`, then `msg` is a long-form string error
 explaining that something returned an error during execution which it
-would be reasonable to report to users.
+would be reasonable to report to users under most circumstances.
 
-If the result is "allow", then msg is an optional string saying why
-the ruleset resulted in an allow.  Ditto for "deny".  Essentially any
+If `result` is `"allow"`, then `msg` is an optional string saying why
+the ruleset resulted in an allow.  Ditto for `"deny"`.  Essentially any
 string might be a reason.  This is covered below in Commands.
 
 Commands
@@ -33,20 +34,20 @@ When a command is being run, it is called as:
 
     result, msg = command_fn(exec_context, unpack(args))
 
-where args are the arguments it returned when being compiled.
+where `args` are the arguments returned during the compilation of the command.
 
 If the function throws an error, that will be caught and processed by
 the execution engine.
 
-If result is falsehood (nil, false) then the command is considered to
-have failed for some reason and msg contains an "internal" error
+If `result` is falsehood (`nil`, `false`) then the command is considered to
+have failed for some reason and `msg` contains an "internal" error
 message to report to the user.  This aborts the execution of the
 ruleset.
 
-If result is true, then the command successfully ran, and execution
+If `result` is `true`, then the command successfully ran, and execution
 continues at the next rule.
 
-If result is a string, then the command returned a result.  This
+If `result` is a string, then the command returned a result.  This
 ceases execution of the ruleset and the result and message (which must
 be a string explanation) are returned to the caller.  Typically such
 results would be "allow" or "deny" although there's nothing forcing
@@ -59,15 +60,15 @@ When a control type function is being run, it is called as:
 
     result, msg = ct_fn(exec_context, unpack(args))
 
-where args are the arguments it returned when being compiled.
+where `args` are the arguments returned when the definition was compiled.
 
 If the function throws an error, it will be caught and processed by
 the execution engine.
 
-If result is nil then msg is an "internal" error, execution will be
+If `result` is `nil` then msg is an "internal" error, execution will be
 stopped and the issue reported to the caller.
 
-If result is false, the control call failed and returned falsehood.
+If `result` is `false`, the control call failed and returned falsehood.
 Anything else and the control call succeeded and returns truth.
 
 Control type functions are called at the point of test, not at the
@@ -78,10 +79,10 @@ needed to ensure suitably performant behaviour.
 Helper functions
 ================
 
-Since sometimes you need to know if a given define rule passes, Lace
-provides a function to do this.  It is bound up in the behaviour of
-Lace's internal 'define' command and as such, you should treat it as a
-black box.
+Since sometimes when writing command functions, you need to know if a given
+define rule passes, Lace provides a function to do this.  It is bound up in the
+behaviour of Lace's internal `define` command and as such, you should treat it
+as a black box.
 
     result, msg = lace.engine.test(exec_context, name)
 
@@ -89,5 +90,5 @@ This, via the magic of the execution context calls through to the
 appropriate control type functions, returning their results directly.
 
 This means that it can throw an error in the case of a Lua error,
-otherwise it returns the two values as above.
+otherwise it returns the two values as detailed above.
 
diff --git a/doc/index.mdwn b/doc/index.mdwn
new file mode 100644
index 0000000..2b7f485
--- /dev/null
+++ b/doc/index.mdwn
@@ -0,0 +1,25 @@
+Lace - Lua Access Control Engine
+================================
+
+Lace is the core of an access control engine designed to be embedded into other
+applications.  It is also designed to be extended by the very applications it
+is embedded into.
+
+As such, Lace provides only the core lexing, parsing, error handling and
+related functionality of an access control engine, along with some initial
+semantics to help the application developer along.
+
+All rules and mechanisms of deciding if access is to be permitted or not are up
+to the application author to define.  As such, while this documentation for
+Lace will be useful for the application developer; it is recommended that the
+applications do not refer their users to the Lace documentation except to
+augment that provided in the application specific documentation.
+
+The Lace codebase provides an example of using the library which should be
+referred to for getting started with Lace.  However, there is also extensive
+documentation on the [[syntax]] of Lace rulesets and also on the
+[[compilation]] and [[execution]] phases of access control.
+
+If you wish to assist with Lace development, then see the [[developing]]
+document for pointers around the codebase and the test suite.
+
diff --git a/doc/syntax-allow-deny b/doc/syntax-allow-deny
deleted file mode 100644
index 094352b..0000000
--- a/doc/syntax-allow-deny
+++ /dev/null
@@ -1,17 +0,0 @@
-Syntax of Allow and Deny statements
------------------------------------
-
-These access control statements start 'allow' or 'deny', followed by a
-single token reason to be returned to the caller, followed by zero or
-more definitions, all of which must pass for the statement to execute.
-
-allow "Administrators can do anything" is-admin
-deny "Plebs may do nothing" is-pleb
-
-allow ''
-
-As with definitions, includes, etc, if rule names are prefixed with an
-exclamation point then their sense is inverted, e.g.
-
-deny "Only admins may alter hooks" altering-hooks !is-admin
-
diff --git a/doc/syntax-allow-deny.mdwn b/doc/syntax-allow-deny.mdwn
new file mode 100644
index 0000000..c227abf
--- /dev/null
+++ b/doc/syntax-allow-deny.mdwn
@@ -0,0 +1,20 @@
+Lace - Syntax of Allow and Deny statements
+==========================================
+
+These access control statements start `allow` or `deny`, followed by a single
+token reason to be returned to the caller (thus the reason should be quoted if
+it contains spaces), followed by zero or more definition names, all of which
+must pass for the statement to execute.
+
+For example:
+
+    allow "Administrators can do anything" is-admin
+    deny "Plebs may do nothing" is-pleb
+    
+    allow ''
+
+As with definitions, includes, etc, if rule names are prefixed with an
+exclamation point then their sense is inverted, e.g.
+
+    deny "Only admins may alter hooks" altering-hooks !is-admin
+
diff --git a/doc/syntax-default b/doc/syntax-default
deleted file mode 100644
index 91b9c0d..0000000
--- a/doc/syntax-default
+++ /dev/null
@@ -1,32 +0,0 @@
-The syntax of the default statement
------------------------------------
-
-The default statement is interesting.  It has no behaviour at runtime,
-but at compile time it alters the behaviour of the compiler with
-respect to the end of the ruleset.
-
-If, when the ruleset has finished compiling, the last allow or deny
-was not unconditional, then the compiler will, in the absence of a
-'default' statement, inject a terminal allow/deny of the opposite
-sense of the last explicit operation, unconditionally and with a
-reason of the empty string.
-
-If a 'default' statement was encountered during processing then it
-will be used instead.
-
-The syntax of the 'default' statement is:
-
-'default' 'allow' <reason>?
-or
-'default' 'deny' <reason>?
-
-If reasons are not provided, the string "Default behaviour" is
-substituted.
-
-Once a single 'default' statement has been encountered during
-compilation it is an error, and the compiler WILL cease, if it
-encounters an additional 'default' statement.
-
-Since it's common for rulesets to stem from a single core point, it's
-recommended that the application define a policy at the start of these
-core statements.
diff --git a/doc/syntax-default.mdwn b/doc/syntax-default.mdwn
new file mode 100644
index 0000000..2304ed5
--- /dev/null
+++ b/doc/syntax-default.mdwn
@@ -0,0 +1,33 @@
+Lace - The syntax of the default statement
+==========================================
+
+The `default` statement is unusual in that it has no behaviour at runtime.  At
+compile time the `default` statement alters the behaviour of the compiler with
+respect to what happens at the end of the ruleset parse.
+
+If, when Lace has finished parsing the ruleset, the last allow or deny was not
+unconditional, then the compiler will, in the absence of a `default` statement,
+inject a terminal `allow`/`deny` of the opposite sense of the last explicit
+operation, unconditionally and with a reason of the empty string.
+
+If a `default` statement was encountered during processing then its chosen
+behaviour will be used instead.
+
+The syntax of the `default` statement is:
+
+    default 'allow' <reason>?
+
+or
+
+    default 'deny' <reason>?
+
+If reasons are not provided, the string "Default behaviour" is
+substituted.
+
+Once a single `default` statement has been encountered during
+compilation; it is an error, and the compiler WILL cease, if it
+encounters an additional `default` statement.
+
+Since it's common for rulesets to stem from a single core point, therefore it
+is recommended that the application define a `default` policy at the start of
+these core statements.
diff --git a/doc/syntax-define b/doc/syntax-define
deleted file mode 100644
index 277fb54..0000000
--- a/doc/syntax-define
+++ /dev/null
@@ -1,34 +0,0 @@
-Syntax of definition statements
--------------------------------
-
-Definition statements start with one of 'def' or 'define'.  They may
-also start 'acl' to look more squiddish.
-
-The rough syntax of a definition statement is:
-
-'def' <name> <controltype> <0-or-more-args>
-
-There are some loose constraints on the name and controltype
-arguments.  The name SHOULD NOT contain quote characters and MUST NOT
-start with an exclamation point.  The controltype simply SHOULD NOT
-contain quotes.
-
-The control types are typically provided by the program which is using
-Lace.  However Lace does provide some simple control types which can
-be useful and callers are welcome to add them to their engines.
-
-The two simplest control types Lace provides are the allof and anyof
-controls.  They, as their arguments, take two or more rule names and
-match if all or any (respectively) of their arguments resolve to true.
-As with other parts of Lace, if the rule names are prefixed by
-exclamation points then their result is inverted before being tested.
-
-These give you ways to produce common subexpressions of the 'AND' and
-'OR' forms to be used in later rules.
-
-Lace also provides a match control type which does a string comparison
-on a context value.  The match control type takes two arguments.  The
-key to match and the value to test against.  If the value starts with
-a tilde then Lace will use Lua pattern matches.  This is not a
-terribly useful control type and is provided purely for simple cases
-where the caller does not need more complex control types.
diff --git a/doc/syntax-define.mdwn b/doc/syntax-define.mdwn
new file mode 100644
index 0000000..579fe44
--- /dev/null
+++ b/doc/syntax-define.mdwn
@@ -0,0 +1,31 @@
+Lace - Syntax of definition statements
+======================================
+
+Definition statements start with one of `def` or `define`.  They may
+also start `acl` to look more squiddish.
+
+The rough syntax of a definition statement is:
+
+    define <name> <controltype> <0-or-more-args>
+
+for example:
+
+    define is-admin is-in-group administrators
+
+There are some loose constraints on the name and controltype
+arguments.  The name SHOULD NOT contain quote characters and MUST NOT
+start with an exclamation point.  The controltype simply SHOULD NOT
+contain quotes.
+
+The control types are typically provided by the program which is using
+Lace.  However Lace does provide some simple control types which can
+be useful and callers are welcome to add them to their engines.
+
+The two simple control types Lace provides are the `allof` and `anyof`
+controls.  They, as their arguments, take two or more rule names and match if
+all or any (respectively) of their arguments resolve to true.  As with other
+parts of Lace, if the rule names are prefixed by exclamation points then their
+result is inverted before being tested.
+
+These give you ways to produce common subexpressions of the 'AND' and
+'OR' forms to be used in later rules.
diff --git a/doc/syntax-include b/doc/syntax-include
deleted file mode 100644
index 8cd452c..0000000
--- a/doc/syntax-include
+++ /dev/null
@@ -1,26 +0,0 @@
-Syntax of include statements
-----------------------------
-
-Include statements take a source token to include and an optional list
-of definitions which must all be true before the include will take
-place.
-
-'include' <sourcename> <0-or-more-definitions>
-
-If the include ends with a question mark then should the sourcename
-not be available at the time, it will be silently ignored.
-
-Nominally an included ruleset is linearly inserted into the execution
-stream there and then.  In practice, while include statements result
-in compilation of the rulesets at the same time, the contents of the
-ruleset will not be run unless the definitions all pass.  This means
-that the defines will not happen unless the definitions all pass.  As
-such, use conditional includes carefully.
-
-If an "optional" include source isn't available, it is as though the
-definitions did not pass.  If a mandatory include source isn't
-available then compilation of the ruleset will fail immediately.
-
-Circular includes are not acceptable.  Not even if there's a set of
-conditions which mean that it might never happen.  This is critical
-because rulesets are entirely loaded at compile time.
diff --git a/doc/syntax-include.mdwn b/doc/syntax-include.mdwn
new file mode 100644
index 0000000..a95f489
--- /dev/null
+++ b/doc/syntax-include.mdwn
@@ -0,0 +1,27 @@
+Lace - Syntax of include statements
+===================================
+
+Include statements take a source token to include and an optional list
+of definitions which must all be true before the include will take
+place.
+
+    include <sourcename> <0-or-more-definitions>
+
+If the include ends with a question mark (`include?`) then should the
+sourcename not be available at the time, it will be silently ignored.
+
+Nominally an included ruleset is linearly inserted into the execution stream
+there and then.  In practice, while include statements result in compilation of
+the rulesets at the same time, the contents of the ruleset will not be run
+unless the supplied definitions all pass.  This means that any rules (including
+`define`s) in the included ruleset will not be executed unless the definitions
+all pass.  As such, use conditional includes carefully.
+
+If an "optional" include source isn't available, it is as though the
+definitions did not pass.  If a mandatory include source isn't
+available then compilation of the ruleset will fail immediately.
+
+Circular includes are not acceptable.  Not even if there's a set of conditions
+which mean that it might never happen.  Any attempt at circular includes will
+result in a critical error at compile time because that is when all included
+rulesets are loaded in their entirety.
diff --git a/doc/syntax b/doc/syntax.mdwn
index 5ac4115..75a2b1e 100644
--- a/doc/syntax
+++ b/doc/syntax.mdwn
@@ -1,8 +1,9 @@
-# Syntax of Lace
+Lace - Syntax of rulesets
+=========================
 
 Lace rule files are parsed line-by-line.  There is no provision at
 this time for rules to be split across multiple lines.  If you require
-such, please submit a patch.
+such, please [submit a patch][developing].
 
 Lace splits each line into a series of tokens.  Tokens are always
 strings and they can be optionally quoted to allow for spaces in them.
@@ -17,26 +18,46 @@ differentiate between single and double quotes.  Backslash escaped
 characters sometimes have special meaning if inside quotes.  For
 example, the following strings lex in the following ways:
 
-1. hello world
-   * Two tokens, one of each word.
-2. hello   world
-   * The same as 1.
-3. "hello world"
-   * One token with both words separated by one space
-4. 'hello world'
-   * Same as 3.
-5. hello\ world
-   * Same as 3 and 4.
-6. up\town
-   * One token, consisting of the letters: uptown
-6. "up\town"
-   * One token, consisting of the letters: up TAB own
-7. \"
-   * One token, a double-quote character
-8. '"'
-   * The same as 7
-9. "\""
-   * The same as 7 and 8.
+1. Two tokens, one of each word.
+
+          hello world
+
+2. The same as 1.
+
+          hello   world
+
+3. One token with both words separated by one space
+
+          "hello world"
+
+4. Same as 3.
+
+          'hello world'
+
+5. Same as 3 and 4.
+
+          hello\ world
+
+6. One token, consisting of the letters: `uptown`
+
+          up\town
+
+6. One token, consisting of the letters: `up TAB own`
+
+          "up\town"
+
+7. One token, a double-quote character
+
+          \"
+
+8. The same as 7
+
+          '"'
+
+9. The same as 7 and 8.
+
+          "\""
+
 
 As you can see, the lexing rules are not trivial, but should not come
 as a surprise to anyone used to standard command-line lexing
@@ -46,20 +67,20 @@ Comments in Lace are prefixed by a hash '#', a double-slash '//' or a
 double-dash '--'.  The first word of the line must start with one of
 these markers (but may contain other text also), for example:
 
-# This is a command
-// As is this
--- And this
-
-#But also this
-//And this
---And also this
+    # This is a comment
+    // As is this
+    -- And this
+    
+    #But also this
+    //And this
+    --And also this
 
 Blank lines are permitted and ignored (except for counting), any
 prefixed or postfixed whitespace is deleted before lexing begins.  As
 such:
 
-# This is a comment
-  # So is this
+    # This is a comment
+      # So is this
 
 This allows for sub-rulesets to be indented as the author pleases,
 without altering the meaning of the rules.
@@ -68,27 +89,24 @@ The first word of a rule defines its type.  You can think of it as the
 command being run.  Lace, by default, provides a small number of rule
 types:
 
-1. Definitions
+1. [Definitions](../syntax-define)
    * This define access control stanzas.  The definitions produced are
      used in further rules to control access.  Lace does not allow any
      name to be reused.
-2. Includes
+2. [Includes](../syntax-include)
    * Lace can include further rules at any point during a ruleset.  If
      the rules are to be optionally run then Lace cannot perform static
      analysis of the definitions within the ruleset.  Instead it will
      rely on runtime catching of multiple-definitions etc.
-3. Access control statements
+3. [Access control statements](../syntax-allow-deny)
    * These are the core functions of Lace.  Namely the allow and deny
      statements.  These use control definitions from earlier in a ruleset
      to determine whether to allow or deny access.  The first allow
      or deny statement which passes will stop execution of the ruleset.
-4. Default statement
+4. [Default statement](../syntax-default)
   * The 'default' statement can only be run once and provides Lace with
     information on what to do in the case of no allow or deny rule passing.
 
-For further information on the syntax of each rule type, see the
-corresponding file syntax-<ruletype>
-
 In those files, if you encounter the words WILL, MAY, SHOULD, MUST, or
 their negatives, specifically in all-caps, then the meaning of the
 words is taken in the spirit of the RFC usage of them.