AC_LANG_CALL(C++): Use ‘int’ for return type of conftest::$2.

Commit 326c9a547423d25c621bc5c0ef76edbf6eda8c92 introduced a custom
AC_LANG_CALL for C++.  Jani Välimaa reports in
https://lists.gnu.org/archive/html/bug-autoconf/2020-10/msg00054.html
that the new code does not handle AC_CHECK_LIB([foo], [main])
correctly.  This is not the recommended way to use AC_CHECK_LIB, but
it’s what you get if you autoupdate from AC_HAVE_LIBRARY, and some
people may not have bothered replacing main with a more appropriate
symbol.

This patch changes the return type of the fake function declaration
for AC_CHECK_LIB’s second argument to be ‘int’, which is sufficient to
make g++ 10.2.0 happy again.  We’re still on thin ice, unfortunately;
the code generated by AC_LANG_CALL *always* has undefined behavior, in
both C and C++, unless by chance the real prototype of the function
we’re probing for happens to match our fake declaration.  The only
permanent cure is to stop faking declarations, and that’s going to be
a challenge.

* lib/autoconf/c.m4 (AC_LANG_CALL(C++)): Use ‘int’ for the return
  type of the fake function declaration, to avoid problems when
  the function whose declaration we’re faking is ‘main’.

1 files changed, 12 insertions, 7 deletions

diff --git a/lib/autoconf/c.m4 b/lib/autoconf/c.m4
index de40b8ca..53217077 100644
--- a/lib/autoconf/c.m4
+++ b/lib/autoconf/c.m4
@@ -240,16 +240,21 @@ ac_compiler_gnu=$ac_cv_cxx_compiler_gnu
 # AC_LANG_CALL(C++)(PROLOGUE, FUNCTION)
 # -------------------------------------
 m4_define([AC_LANG_CALL(C++)],
+dnl We do not know the function signature of the real $2.
+dnl Declare it in a namespace so the compiler doesn't recognize it
+dnl (with, most likely, a clashing prototype); the 'extern "C"' hides
+dnl the namespace from the linker.
+dnl Use 'int' for the return type, because some C++ compilers consider
+dnl 'namespace conftest { extern "C" void main (); }' to be an
+dnl erroneous redeclaration of ::main, namespace notwithstanding.
+dnl The logic they're applying could be extended in the future to
+dnl other built-in extern "C" functions, but let's worry about that
+dnl when it actually happens.
 [AC_LANG_PROGRAM([[$1
-// We do not know the function signature of the real $2.
-// Declare it in a namespace so the compiler doesn't recognize it
-// (with, most likely, a clashing prototype); the 'extern "C"' will
-// hide the namespace from the linker, so it will still look for the
-// real (global) $2.
 namespace conftest {
-  extern "C" void $2 ();
+  extern "C" int $2 ();
 }]],
-[[conftest::$2 (); return 0;]])])
+[[return conftest::$2 ();]])])
 
 
 # AC_LANG_CPLUSPLUS