Generate manpages directly from source code.

We generate manpages for autoconf’s installed programs (autoconf,
autoheader, etc.) using help2man, which runs each program in order to
learn its --help output.  Each manpage therefore has a dependency on
the existence of the corresponding program, but this dependency is
intentionally left out of the Makefile so that one can build from a
tarball release (which will include prebuilt manpages) without having
help2man installed.

But when building from a git checkout with high levels of
parallelism (-j20 or so), the missing dependency can lead to build
failures, because help2man will try to run the program before it
exists.  In an earlier patch I tried to work around this with a
recursive make invocation in the ‘.x.1’ rule, to ensure the existence
of the program.  That only traded one concurrency bug for another, now
we could have two jobs trying to build the same program simultaneously
and they would clobber each other’s work and the build would still
fail.

Instead, this patch introduces a utility script ‘help-extract.pl’ that
reads --help and --version information directly from the source code
for each program.  This utility, wrapped appropriately for each
program, is what help2man now runs.  Usage is a little weird because
help2man doesn’t let you specify any arguments to the “executable”
that it runs, but it works, and lets us write all of the true
dependencies of each manpage into the Makefile without naming any file
that would be created during a build from a tarball.  help-extract.pl
is a Perl script, so it introduces no new build-time requirements.

A downside is that we have to make sure each of the script sources in
bin/, and also part of lib/Autom4te/ChannelDefs.pm, are parseable by
help-extract.  The most important constraints are that the text output
by --help must be defined in a global variable named ‘help’, and its
definition has to be formatted just the way these definitions are
currently formatted.  Similarly for --version.  Furthermore, only some
non-literal substitutions are possible in these texts; each has to be
explicitly supported in help-extract.pl.  The current list of supported
substitutions is $0, @PACKAGE_NAME@, @VERSION@, @RELEASE_YEAR@, and
Autom4te::ChannelDefs::usage.

The generated manpages themselves are character-for-character
identical before and after this patch.

 * build-aux/help-extract.pl: New build script that extracts --help
  and --version output from manpages.

 * man/autoconf.w, man/autoheader.w, man/autom4te.w, man/autoreconf.w
 * man/autoscan.w, man/autoupdate.w, man/ifnames.w: New shell scripts
   which wrap build-aux/help-extract.pl.

 * man/local.mk: Generate each manpage by running help2man on the
   corresponding .w script, not on the built utility itself.
   Revise all dependencies to match.

 * bin/autoconf.as: Rename ‘usage’ variable to ‘help’ and
   ‘help’ variable to ‘usage_err’.
 * bin/autoheader.in: Call Autom4te::ChannelDefs::usage with no
   function-call parentheses, matching all the other scripts.
 * bin/autom4te.in: Initialize $version with a regular double-quoted
   string, not a heredoc, matching all the other scripts.
 * bin/autoscan.in: Remove global variable $configure_scan.

9 files changed, 99 insertions, 40 deletions

diff --git a/man/autoconf.w b/man/autoconf.w
new file mode 100755
index 00000000..4bb2eb63
--- /dev/null
+++ b/man/autoconf.w
@@ -0,0 +1,4 @@
+#! /bin/sh
+# help-extract wrapper, see build-aux/help-extract.pl and man/local.mk
+exec "${PERL-perl}" "${top_srcdir}/build-aux/help-extract.pl" \
+   bin/autoconf.as "$@"
diff --git a/man/autoheader.w b/man/autoheader.w
new file mode 100755
index 00000000..821d30ba
--- /dev/null
+++ b/man/autoheader.w
@@ -0,0 +1,4 @@
+#! /bin/sh
+# help-extract wrapper, see build-aux/help-extract.pl and man/local.mk
+exec "${PERL-perl}" "${top_srcdir}/build-aux/help-extract.pl" \
+   bin/autoheader.in "$@"
diff --git a/man/autom4te.w b/man/autom4te.w
new file mode 100755
index 00000000..03375042
--- /dev/null
+++ b/man/autom4te.w
@@ -0,0 +1,4 @@
+#! /bin/sh
+# help-extract wrapper, see build-aux/help-extract.pl and man/local.mk
+exec "${PERL-perl}" "${top_srcdir}/build-aux/help-extract.pl" \
+   bin/autom4te.in "$@"
diff --git a/man/autoreconf.w b/man/autoreconf.w
new file mode 100755
index 00000000..ab98a4f2
--- /dev/null
+++ b/man/autoreconf.w
@@ -0,0 +1,4 @@
+#! /bin/sh
+# help-extract wrapper, see build-aux/help-extract.pl and man/local.mk
+exec "${PERL-perl}" "${top_srcdir}/build-aux/help-extract.pl" \
+   bin/autoreconf.in "$@"
diff --git a/man/autoscan.w b/man/autoscan.w
new file mode 100755
index 00000000..214fc794
--- /dev/null
+++ b/man/autoscan.w
@@ -0,0 +1,4 @@
+#! /bin/sh
+# help-extract wrapper, see build-aux/help-extract.pl and man/local.mk
+exec "${PERL-perl}" "${top_srcdir}/build-aux/help-extract.pl" \
+   bin/autoscan.in "$@"
diff --git a/man/autoupdate.w b/man/autoupdate.w
new file mode 100755
index 00000000..0b75208d
--- /dev/null
+++ b/man/autoupdate.w
@@ -0,0 +1,4 @@
+#! /bin/sh
+# help-extract wrapper, see build-aux/help-extract.pl and man/local.mk
+exec "${PERL-perl}" "${top_srcdir}/build-aux/help-extract.pl" \
+   bin/autoupdate.in "$@"
diff --git a/man/common.w b/man/common.w
new file mode 100755
index 00000000..ed981db4
--- /dev/null
+++ b/man/common.w
@@ -0,0 +1,4 @@
+#! /bin/sh
+# help-extract wrapper, see build-aux/help-extract.pl and man/local.mk
+exec "${PERL-perl}" "${top_srcdir}/build-aux/help-extract.pl" \
+   bin/common.in "$@"
diff --git a/man/ifnames.w b/man/ifnames.w
new file mode 100755
index 00000000..7812018a
--- /dev/null
+++ b/man/ifnames.w
@@ -0,0 +1,4 @@
+#! /bin/sh
+# help-extract wrapper, see build-aux/help-extract.pl and man/local.mk
+exec "${PERL-perl}" "${top_srcdir}/build-aux/help-extract.pl" \
+   bin/ifnames.in "$@"
diff --git a/man/local.mk b/man/local.mk
index 68cf5a8d..d594448d 100644
--- a/man/local.mk
+++ b/man/local.mk
@@ -15,8 +15,6 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <https://www.gnu.org/licenses/>.
 
-binsrcdir = $(srcdir)/bin
-
 dist_man_MANS = \
   man/autoconf.1 \
   man/autoheader.1 \
@@ -26,57 +24,86 @@ dist_man_MANS = \
   man/autoupdate.1 \
   man/ifnames.1
 
-EXTRA_DIST += $(dist_man_MANS:.1=.x) man/common.x
+EXTRA_DIST += $(dist_man_MANS:.1=.w) $(dist_man_MANS:.1=.x) man/common.x
 
-# Depend on .version to get version number changes.
-# Don't depend on the generated scripts, because then we would try to
-# regenerate the manpages after building the scripts, which would
-# defeat the purpose of shipping the manpages in the tarball.
-# (Instead we have recursive makes in the .x.1 rule below, which is
-# not ideal, but at least it prevents us from generating a manpage
-# from the *installed* utility.)
-common_dep = $(srcdir)/.version $(srcdir)/man/common.x
-man/autoconf.1:   $(common_dep) $(binsrcdir)/autoconf.as
-man/autoheader.1: $(common_dep) $(binsrcdir)/autoheader.in
-man/autom4te.1:   $(common_dep) $(binsrcdir)/autom4te.in
-man/autoreconf.1: $(common_dep) $(binsrcdir)/autoreconf.in
-man/autoscan.1:   $(common_dep) $(binsrcdir)/autoscan.in
-man/autoupdate.1: $(common_dep) $(binsrcdir)/autoupdate.in
-man/ifnames.1:    $(common_dep) $(binsrcdir)/ifnames.in
+# Each manpage depends on:
+# - its .w and .x files and its source script in bin/
+# - common.x for the SEE ALSO list
+# - lib/Autom4te/ChannelDefs.pm which contains additional --help text
+#   (not included in _all_ the manpages, but it's easier to have them
+#   all depend on it)
+# - .version and configure.ac for version information
+#
+# We ship the manpages in tarball releases so people can build from
+# them without having help2man installed.  For this to work correctly,
+# the manpages cannot have declared dependencies on any file that is
+# not also shipped in the tarball.  To avoid concurrency bugs, those
+# files plus the Makefile must in fact be sufficient to generate the
+# manpages.  See the automake manual, section 'Errors with distclean',
+# for further discussion.
 
-remove_time_stamp = 's/^\(\.TH[^"]*"[^"]*"[^"]*\)"[^"]*"/\1/'
+binsrcdir      = $(top_srcdir)/bin
+channeldefs_pm = lib/Autom4te/ChannelDefs.pm
+man_common_dep = $(top_srcdir)/man/common.x \
+		 $(top_srcdir)/$(channeldefs_pm) \
+		 $(top_srcdir)/.version \
+		 $(top_srcdir)/configure.ac
 
-MOSTLYCLEANFILES     += $(dist_man_MANS:=.t) $(dist_man_MANS:=a.t) \
-			$(dist_man_MANS:=.tmp)
-MAINTAINERCLEANFILES += $(dist_man_MANS)
+man/autoconf.1:   $(common_dep) man/autoconf.w   man/autoconf.x   $(binsrcdir)/autoconf.as
+man/autoheader.1: $(common_dep) man/autoheader.w man/autoheader.x $(binsrcdir)/autoheader.in
+man/autom4te.1:   $(common_dep) man/autom4te.w   man/autom4te.x   $(binsrcdir)/autom4te.in
+man/autoreconf.1: $(common_dep) man/autoreconf.w man/autoreconf.x $(binsrcdir)/autoreconf.in
+man/autoscan.1:   $(common_dep) man/autoscan.w   man/autoscan.x   $(binsrcdir)/autoscan.in
+man/autoupdate.1: $(common_dep) man/autoupdate.w man/autoupdate.x $(binsrcdir)/autoupdate.in
+man/ifnames.1:    $(common_dep) man/ifnames.w    man/ifnames.x    $(binsrcdir)/ifnames.in
 
-# To satisfy 'distcleancheck', we need to delete built manpages in
-# 'distclean' when the build and source directories are not the same.
-# We know we are in this case when 'man/common.x' doesn't exist.
-distclean-local: distclean-local-man
-distclean-local-man:
-	test -f man/common.x || rm -f $(dist_man_MANS)
+# To generate the manpages, we use help2man, but we don't have it run
+# the built script that corresponds to the manpage.  Instead it runs
+# the .w file listed above, which is a wrapper around
+# build-aux/help-extract.pl, which parses the *source* of the script
+# and extracts the help and version text.  This means that the built
+# script doesn't need to exist to create the manpage.  If it did,
+# we would have a concurrency bug, since we can't declare a dependency
+# on the built script, as discussed above.
+# We use a suffix rule describing the manpage as built from its .w file
+# so that we can use $(<F) to name the executable to be run, which avoids
+# an extra subshell and sed invocation.
 
+remove_time_stamp = 's/^\(\.TH[^"]*"[^"]*"[^"]*\)"[^"]*"/\1/'
+SUFFIXES += .w .1
 
-SUFFIXES += .x .1
-
-.x.1:
-	@set -e; cmd=`basename $*`; \
-	test -x bin/$$cmd || $(MAKE) bin/$$cmd; \
-	test -x tests/$$cmd || $(MAKE) tests/$$cmd;
+.w.1:
 	@echo "Updating man page $@"
-	@test -d $(@D) || mkdir $(@D)
-	PATH="./tests$(PATH_SEPARATOR)$(top_srcdir)/build-aux$(PATH_SEPARATOR)$$PATH"; \
-	export PATH; \
+	$(MKDIR_P) $(@D)
+	PATH="$(top_srcdir)/man$(PATH_SEPARATOR)$$PATH"; \
+	PERL="$(PERL)"; \
+	PACKAGE_NAME="$(PACKAGE_NAME)"; \
+	VERSION="$(VERSION)"; \
+	RELEASE_YEAR="$(RELEASE_YEAR)"; \
+	top_srcdir="$(top_srcdir)"; \
+	channeldefs_pm="$(channeldefs_pm)"; \
+	export PATH PERL PACKAGE_NAME VERSION RELEASE_YEAR; \
+	export top_srcdir channeldefs_pm; \
 	$(HELP2MAN) \
 	    --include=$(srcdir)/$*.x \
 	    --include=$(srcdir)/man/common.x \
 	    --source='$(PACKAGE_STRING)' \
-	    --output=$@.t `echo '$*' | sed 's,.*/,,'`
-	if sed $(remove_time_stamp) $@ >$@a.t 2>/dev/null && \
-	   sed $(remove_time_stamp) $@.t | cmp $@a.t - >/dev/null 2>&1; then \
+	    --output=$@.t $(<F)
+	if $(SED) $(remove_time_stamp) $@ >$@a.t 2>/dev/null && \
+	   $(SED) $(remove_time_stamp) $@.t | cmp $@a.t - >/dev/null 2>&1; then \
 		touch $@; \
 	else \
 		mv $@.t $@; \
 	fi
 	rm -f $@.t $@a.t
+
+
+MOSTLYCLEANFILES     += $(dist_man_MANS:=.t) $(dist_man_MANS:=a.t)
+MAINTAINERCLEANFILES += $(dist_man_MANS)
+
+# To satisfy 'distcleancheck', we need to delete built manpages in
+# 'distclean' when the build and source directories are not the same.
+# We know we are in this case when 'man/common.x' doesn't exist.
+distclean-local: distclean-local-man
+distclean-local-man:
+	test -f man/common.x || rm -f $(dist_man_MANS)