Revert to 2.69-compatible behavior in AC_PROG_LEX (#110346)

Commit 29ede6b96feee29c0c477d1659081bbdb82cd8b3 caused AC_PROG_LEX to
stop looking for a library that provides yywrap.  This broke several
packages in a Debian archive rebuild.

Revert all the way to the 2.69 behavior, which was to set LEXLIB to
-ll or -lfl if that library defines yywrap, but allow AC_PROG_LEX to
succeed if neither -ll nor -lfl exists on the system, even if a lex
program that doesn't define yywrap would need it.
(This behavior was a bug, but people have come to depend on it.
See https://savannah.gnu.org/support/index.php?110269 and the
thread starting from
https://lists.gnu.org/r/autoconf-patches/2020-07/msg00013.html
for gory details.)

To provide a path away from bug-compatibility, AC_PROG_LEX now takes
one argument, documented as a whitespace-separated list of options.
Two options are defined: ‘yywrap’ means to look for yywrap and behave
as if lex is unavailable if it isn’t found; ‘noyywrap’ means to not
look for yywrap at all.  These are mutually exclusive.

Fixes bug #110346.

* lib/autoconf/programs.m4 (AC_PROG_LEX): Add an argument which
  can be either ‘yywrap’, meaning to look for yywrap in -ll, or
  ‘noyywrap’, meaning to not look for yywrap at all.  In the
  absence of either option, issue an obsoletion warning and
  revert to behavior bug-compatible with 2.69.

* tests/semantics.at: Add more tests of AC_PROG_LEX.
* tests/mktests.sh: Exclude AC_PROG_LEX from autogenerated tests.

* doc/autoconf.texi: Update documentation of AC_PROG_LEX.
* NEWS: Update notes on AC_PROG_LEX.

2 files changed, 126 insertions, 0 deletions

diff --git a/tests/mktests.sh b/tests/mktests.sh
index 849e4c12..3ce0c2b4 100755
--- a/tests/mktests.sh
+++ b/tests/mktests.sh
@@ -134,6 +134,10 @@ ac_exclude_list='
 	/^AC_(PROG_CC|C_CONST|C_VOLATILE)$/ {next}
 	/^AC_PATH_XTRA$/ {next}
 
+        # Use without an argument is obsolete.
+        # Checked in semantics.
+        /^AC_PROG_LEX$/ {next}
+
 	# Requires a working C++ compiler, which is not a given.
 	/^AC_PROG_CXX_C_O$/ {next}
 
diff --git a/tests/semantics.at b/tests/semantics.at
index ab3cea7c..d14e5713 100644
--- a/tests/semantics.at
+++ b/tests/semantics.at
@@ -1044,3 +1044,125 @@ entered d_blurf/configure
 ]])
 
 AT_CLEANUP
+
+
+## ------------------------------------- ##
+## AC_PROG_LEX with and without yywrap.  ##
+## ------------------------------------- ##
+
+# We don't currently have a good way of verifying that each mode
+# does what it's supposed to, but we can at least put them through
+# their paces as much as the autogenerated AT_CHECK_MACRO invocation
+# used to, back when AC_PROG_LEX took no arguments.
+
+AT_CHECK_MACRO([AC_PROG_LEX with yywrap], [AC_PROG_LEX([yywrap])])
+AT_CHECK_MACRO([AC_PROG_LEX with noyywrap], [AC_PROG_LEX([noyywrap])])
+
+AT_SETUP([AC_PROG_LEX in legacy mode])
+
+AT_CONFIGURE_AC([[AC_PROG_LEX]])
+AT_CHECK_AUTOHEADER([], [ignore])
+AT_CHECK_AUTOCONF([], [], [],
+[[configure.ac:4: warning: AC_PROG_LEX without either yywrap or noyywrap is obsolete
+programs.m4: _AC_PROG_LEX is expanded from...
+programs.m4: AC_PROG_LEX is expanded from...
+configure.ac:4: the top level
+]])
+
+AT_CHECK_CONFIGURE
+
+AT_CLEANUP
+
+
+## ---------------------------------- ##
+## Invalid arguments to AC_PROG_LEX.  ##
+## ---------------------------------- ##
+
+AT_SETUP([Invalid arguments to AC_PROG_LEX])
+
+AT_CONFIGURE_AC(
+[[AC_PROG_LEX([nonsense])
+]])
+AT_CHECK_AUTOCONF([], [1], [],
+[[configure.ac:4: error: AC_PROG_LEX: unrecognized argument: nonsense
+programs.m4: _AC_PROG_LEX is expanded from...
+programs.m4: AC_PROG_LEX is expanded from...
+configure.ac:4: the top level
+autom4te: error: m4 failed with exit status: 1
+]])
+
+AT_CONFIGURE_AC(
+[[AC_PROG_LEX([too],[many])
+]])
+AT_CHECK_AUTOCONF([], [1], [],
+[[configure.ac:4: error: too many arguments to AC_PROG_LEX
+programs.m4: AC_PROG_LEX is expanded from...
+configure.ac:4: the top level
+autom4te: error: m4 failed with exit status: 1
+]])
+
+AT_CONFIGURE_AC(
+[[AC_PROG_LEX([yywrap noyywrap])
+]])
+AT_CHECK_AUTOCONF([], [1], [],
+[[configure.ac:4: error: AC_PROG_LEX: yywrap and noyywrap are mutually exclusive
+programs.m4: _AC_PROG_LEX is expanded from...
+programs.m4: AC_PROG_LEX is expanded from...
+configure.ac:4: the top level
+autom4te: error: m4 failed with exit status: 1
+]])
+
+AT_CONFIGURE_AC(
+[[AC_PROG_LEX([noyywrap yywrap])
+]])
+AT_CHECK_AUTOCONF([], [1], [],
+[[configure.ac:4: error: AC_PROG_LEX: yywrap and noyywrap are mutually exclusive
+programs.m4: _AC_PROG_LEX is expanded from...
+programs.m4: AC_PROG_LEX is expanded from...
+configure.ac:4: the top level
+autom4te: error: m4 failed with exit status: 1
+]])
+
+AT_CONFIGURE_AC(
+[[AC_PROG_LEX([yywrap nonsense])
+]])
+AT_CHECK_AUTOCONF([], [1], [],
+[[configure.ac:4: error: AC_PROG_LEX: unrecognized argument: yywrap nonsense
+programs.m4: _AC_PROG_LEX is expanded from...
+programs.m4: AC_PROG_LEX is expanded from...
+configure.ac:4: the top level
+autom4te: error: m4 failed with exit status: 1
+]])
+
+AT_CONFIGURE_AC(
+[[AC_PROG_LEX([nonsense noyywrap])
+]])
+AT_CHECK_AUTOCONF([], [1], [],
+[[configure.ac:4: error: AC_PROG_LEX: unrecognized argument: nonsense noyywrap
+programs.m4: _AC_PROG_LEX is expanded from...
+programs.m4: AC_PROG_LEX is expanded from...
+configure.ac:4: the top level
+autom4te: error: m4 failed with exit status: 1
+]])
+
+# A double invocation with matching arguments should be accepted
+# without complaint.  FIXME: verify that it runs the test only once.
+AT_CONFIGURE_AC(
+[[AC_PROG_LEX([noyywrap])
+AC_PROG_LEX([noyywrap])
+]])
+AT_CHECK_AUTOCONF
+
+# A double invocation with matching arguments should trigger a warning.
+AT_CONFIGURE_AC(
+[[AC_PROG_LEX([yywrap])
+AC_PROG_LEX([noyywrap])
+]])
+AT_CHECK_AUTOCONF([], [0], [],
+[[configure.ac:5: warning: AC_PROG_LEX used twice with mismatched options
+programs.m4: _AC_PROG_LEX is expanded from...
+programs.m4: AC_PROG_LEX is expanded from...
+configure.ac:5: the top level
+]])
+
+AT_CLEANUP