Teach fast-import to ignore lines starting with '#'

Several frontend developers have asked that some form of stream
comments be permitted within a fast-import data stream.  This way
they can include information from their own frontend program about
where specific data was taken from in the source system, or about
a decision that their frontend may have made while creating the
fast-import data stream.

This change introduces comments in the Bourne-shell/Tcl/Perl style.
Lines starting with '#' are ignored, up to and including the LF.
Unlike the above mentioned three languages however we do not look for
and ignore leading whitespace.  This just simplifies the definition
of the comment format and the code that parses them.

To make comments work we had to stop using read_next_command() within
cmd_data() and directly invoke read_line() during the inline variant
of the function.  This is necessary to retain any lines of the
input data that might otherwise look like a comment to fast-import.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

1 files changed, 15 insertions, 2 deletions

diff --git a/fast-import.c b/fast-import.c
index d7fa2b7baa..98ebe4770d 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -122,6 +122,17 @@ Format of STDIN stream:
   email ::= # valid GIT author/committer email;
   ts    ::= # time since the epoch in seconds, ascii base10 notation;
   tz    ::= # GIT style timezone;
+
+     # note: comments may appear anywhere in the input, except
+     # within a data command.  Any form of the data command
+     # always escapes the related input from comment processing.
+     #
+     # In case it is not clear, the '#' that starts the comment
+     # must be the first character on that the line (an lf have
+     # preceeded it).
+     #
+  comment ::= '#' not_lf* lf;
+  not_lf  ::= # Any byte that is not ASCII newline (LF);
 */
 
 #include "builtin.h"
@@ -1454,7 +1465,9 @@ static void dump_marks(void)
 
 static void read_next_command(void)
 {
-	read_line(&command_buf, stdin, '\n');
+	do {
+		read_line(&command_buf, stdin, '\n');
+	} while (!command_buf.eof && command_buf.buf[0] == '#');
 }
 
 static void cmd_mark(void)
@@ -1481,7 +1494,7 @@ static void *cmd_data (size_t *size)
 		length = 0;
 		buffer = xmalloc(sz);
 		for (;;) {
-			read_next_command();
+			read_line(&command_buf, stdin, '\n');
 			if (command_buf.eof)
 				die("EOF in data (terminator '%s' not found)", term);
 			if (term_len == command_buf.len