This can be replaced by nm_utils_escaped_tokens_split().

Note that nm_utils_escaped_tokens_split() does not behave exactly
the same. For example, nm_utils_str_simpletokens_extract_next() would
remove all backslashes and leave only the following character.
nm_utils_escaped_tokens_split() instead only strips backslashes
that preceed a delimiter, whitespace or another backslash.

But we should have one preferred way of tokenizing, and I find this
preferable, because it allows for most backslashes to appear verbatim.

Optimally, all list types properties in nmcli support proper escaping.
For that, we should use nm_utils_escaped_tokens_*() API.

For now, just change that for policy routes. They were just added recently,
so no change in behavior of released API.

Replace nm_utils_str_simpletokens_extract_next() by
nm_utils_escaped_tokens_split().

nm_utils_escaped_tokens_split() should become our first choice for
parsing and tokenizing.

Note that both nm_utils_str_simpletokens_extract_next() and
nm_utils_escaped_tokens_split() need to strdup the string once,
and tokenizing takes O(n). So, they are roughtly the same performance
wise. The only difference is, that as we iterate through the tokens,
we might abort early on error with nm_utils_str_simpletokens_extract_next()
and not parse the entire string. But that is a small benefit, since we
anyway always strdup() the string (being O(n) already).

Note that to-string will no longer escape ',' and ';'. This is a change
in behavior, of unreleased API. Also note, that escaping these is no
longer necessary, because nmcli soon will also use nm_utils_escaped_tokens_*().

Another change in behavior is that nm_utils_str_simpletokens_extract_next()
treated invalid escape sequences (backslashes followed by an arbitrary
character), buy stripping the backslash. nm_utils_escaped_tokens_*()
leaves such backslashes as is, and only honors them if they are followed
by a whitespace (the delimiter) or another backslash. The disadvantage
of the new approach is that backslashes are treated differently
depending on the following character. The benefit is, that most
backslashes can now be written verbatim, not requiring them to escape
them with a double-backslash.

Yes, there is a problem with these nested escape schemes:

  - the caller may already need to escape backslash in shell.

  - then nmcli will use backslash escaping to split the rules at ','.

  - then nm_ip_routing_rule_from_string() will honor backslash escaping
    for spaces.

  - then iifname and oifname use backslash escaping for nm_utils_buf_utf8safe_escape()
    to express non-UTF-8 characters (because interface names are not
    necessarily UTF-8).

This is only redeamed because escaping is really only necessary for very
unusual cases, if you want to embed a backslash, a space, a comma, or a
non-UTF-8 character. But if you have to, now you will be able to express
that.

The other upside of these layers of escaping is that they become all
indendent from each other:

  - shell can accept quoted/escaped arguments and will unescape them.

  - nmcli can do the tokenizing for ',' (and escape the content
    unconditionally when converting to string).

  - nm_ip_routing_rule_from_string() can do its tokenizing without
    special consideration of utf8safe escaping.

  - NMIPRoutingRule takes iifname/oifname as-is and is not concerned
    about nm_utils_buf_utf8safe_escape(). However, before configuring
    the rule in kernel, this utf8safe escape will be unescaped to get
    the interface name (which is non-UTF8 binary).

nmcli supports list options (optlist and multilist properties).
These commonly are individual items, concatenated by a delimiter.

It should be generally possibly to express every value. That means, we
need some for of escaping mechanism for delimiters.

Currently this is all inconsistent or no escaping is supported. I intend
to fix that (which will be a change in behavior).

For now, just add yet another style of tokenzing/concatenating list
items in nmcli. This is the style to replace all other styles.

This escapes strings so that they can be concatenated with a delimiter
and without loss tokenized with nm_utils_escaped_tokens_split().

Note that this is similar to _nm_utils_escape_plain() and
_nm_utils_escape_spaces(). The difference is that
nm_utils_escaped_tokens_escape() also escapes the last trailing
whitespace. That means, if delimiters contains all NM_ASCII_SPACES, then
it is identical to _nm_utils_escape_spaces(). Otherwise, the trailing
space is treated specially. That is, because
nm_utils_escaped_tokens_split() uses NM_UTILS_STRSPLIT_SET_FLAGS_STRSTRIP,
to strip leading and trailing whitespace. To still express a trailing
whitespace, the last whitespace must be escaped. Note that
NM_UTILS_STRSPLIT_SET_FLAGS_STRSTRIP also honors escaping any whitespace
(not only at the last position), but when escaping we don't need to
escape them, because unescaped (non-trailing) whitespace are taken just
fine.

The pair nm_utils_escaped_tokens_split() and
nm_utils_escaped_tokens_escape() are proposed as default way of
tokenizing a list of items. For example, with

  $ nmcli connection modify "$PROFILE" +ipv4.routing-rules 'priority 5 from 192.168.7.5/32 table 5, priority 6 iif a\, from 192.168.7.5/32 table 6'

Here we implement a to/from string function to handle one item
(nm_ip_routing_rule_{from,to}_string()). When such elements are combined with ',',
then we need to support an additional layer of escaping on top of that.
The advantage is that the indvidual to/from string functions are agnostic
to this second layer of escaping/tokenizing that nmcli employs to handle
a list of these items.
The disadvantage is that we possibly get multiple layers of backslash
escapings. That is only mitigated by the fact that nm_utils_escaped_tokens_*()
supports a syntax for which *most* characters don't need any special escaping.
Only delimiters, backslash, and the trailing space needs escaping, and
these are cases are expected to be few.

Add a new flag that will remove escape characters after splitting
the string.

This implements a special kind of backslash escaping. It's not C escape
sequences (like '\n' or '\020'), but simply to take the special character
following the backslash verbatim. Note that the backslash is only
considered special, if it's followed by a delimiter, another backslash,
or a whitespace (in combination with %NM_UTILS_STRSPLIT_SET_FLAGS_STRSTRIP).

The main purpose of this form of escaping is nmcli's list options, like

  $ nmcli connection modify "$PROFILE" +ipv4.routing-rules 'priority 5 from 192.168.7.5/32 table 5, priority 6 iif a\, from 192.168.7.5/32 table 6'

It's a contrieved example, but the list options are a list of IP
addresses, rules, etc. They implement their own syntax for one element,
and are concatenated by ','. To support that one element may have
arbitrary characters (including the delimiter and whitespaces), nmcli
employs a tokenization with this special kind of escaping.

This will essentially call g_strstrip() on each token.

There are some specialties:

 - if the resulting word is empty after stripping, then according to
   %NM_UTILS_STRSPLIT_SET_FLAGS_PRESERVE_EMPTY, the empty token will be
   removed. If that results in an emptry string array, %NULL will be
   returned.

 - if %NM_UTILS_STRSPLIT_SET_FLAGS_ALLOW_ESCAPING is set, then
   whitespace that is backslash escaped is not removed.

Since this is a post-operation that happens after tokeninzing, it
could be done as a separate function. And we already have this function:
_nm_utils_unescape_plain() and _nm_utils_unescape_spaces().
However, that is ugly for several reasons:

 - the stripping should be part of the tokenizing, you shouldn't need
   several steps.

 - nm_utils_strsplit_set_full() returns a "const char **" which
   indicates the strings must not be freed. However, it is perfectly
   valid to modify the string inplace. Hence, the post-op function
   would need to cast the strings to "char *", which seems ugly
   (although we do that on many places, and it's guaranteed to work).

 - _nm_utils_unescape_plain()/_nm_utils_unescape_spaces() is indeed
   already used together with nm_utils_strsplit_set_full(). However,
   it requires to initialize the cb_lookup buffer again. I would expect
   that initializing the cb_lookup buffer is a large portion of what
   the function does already.
   This issue will be solved in the next commit by adding yet another flag
   which allows to unescape.

Usually, obj_to_str_fcn() should not fail and always add something.
If not, remove the delimiter again.

https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests/108