Make the compiler warnings work with clang, gcc, and others

Make the compiler warnings work with clang in addition to gcc by
test compiling all warning options before actually using them.

This came up when the pedantic build tests failed on FreeBSD 13.

To achieve this, we add -Werror=unknown-warning-option to
the CFLAGS for clang to abort immediately on encountering
an unknown warning option. This means we now only add warnings
which the compiler (clang, gcc, or whatever) actually supports.

As running configure now checks whether compiling is possible
for each warning flag, this should not introduce compatibility
issues where builds which used to work now fail to build.

However, you can always opt to not use the generated CFLAGS
by setting CFLAGS to empty or whatever else you want when
running your build, e.g. like

    make all check CFLAGS=

2 files changed, 159 insertions, 20 deletions

diff --git a/configure.ac b/configure.ac
index c7b73f050..20a6a0045 100644
--- a/configure.ac
+++ b/configure.ac
@@ -103,6 +103,10 @@ dnl Determine the most pedantic compiler flags we can find for both
 dnl C and C++, so that we can test that the libgphoto2 headers compile
 dnl with those flags.
 dnl --------------------------------------------------------------------
+dnl
+dnl
+GP_WITH_EMPTY_CONFDEFS_H([dnl
+dnl
 GP_PEDANTIC_COMPILER_FLAGS([C90],   [C],   [-std=c90])
 GP_PEDANTIC_COMPILER_FLAGS([C99],   [C],   [-std=c99])
 GP_PEDANTIC_COMPILER_FLAGS([C11],   [C],   [-std=c11])
@@ -113,13 +117,65 @@ GP_PEDANTIC_COMPILER_FLAGS([CXX11], [C++], [-std=c++11])
 GP_PEDANTIC_COMPILER_FLAGS([CXX14], [C++], [-std=c++14])
 GP_PEDANTIC_COMPILER_FLAGS([CXX17], [C++], [-std=c++17])
 GP_PEDANTIC_COMPILER_FLAGS([CXX20], [C++], [-std=c++20])
+dnl
+
+dnl --------------------------------------------------------------------
+dnl The goals here are twofold:
+dnl
+dnl   * Produce as many compiler warnings as possible to find
+dnl     potential problems in our own code.
+dnl
+dnl   * Make sure our API header files are as broadly compatible as
+dnl     possible.
+dnl
+dnl We want our library and driver code to compile with the lowest
+dnl possible number of warnings, and the pedantic compilation tests to
+dnl compile with zero warnings (-Werror) so we do not limit the users
+dnl of our API in their use of compiler warnings.
+dnl
+dnl If, for some reason, you need to build without all these compiler
+dnl warnings, run the build while setting CFLAGS to an empty string:
+dnl
+dnl     $ make all check CFLAGS=
+dnl --------------------------------------------------------------------
 
-GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS],[-Wall])
-GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS],[-Wextra])
-GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS],[-Wno-unused-parameter])
-GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS],[-Wno-format-overflow])
-GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS],[-Wno-format-truncation])
-GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS],[-Wno-stringop-truncation])
+dnl * gcc exits non-zero when it encounters an unknown warning option
+dnl * clang by default ignores unknown warning option
+dnl * clang with -Werror=unknown-warning-option will exit non-zero
+dnl   when it encounters an unknown warning option
+GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS], [-Werror=unknown-warning-option])
+
+GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS], [-Wall])
+GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS], [-Wextra])
+GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS], [-Weverything])
+
+dnl We used to have these two defined for GCC instead of the whole -Wextra
+dnl GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS], [-Wmissing-declarations])
+dnl GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS], [-Wmissing-prototypes])
+
+dnl clang finds the underscores in all the "struct _Foo" definitions
+dnl which means they fall into the reserved category. Before we can
+dnl think about renaming those idenifiers, we need to verify they are
+dnl not part of our public API.
+GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS], [-Wno-error=reserved-identifier])
+
+dnl clang finds "/** comments using \deprecated */" without a
+dnl corresponding __attribute__(( deprecated )). We need a good header
+dnl file defining such compiler and standard library specific
+dnl attributes purely as C preprocessor macros without configure
+dnl involvement before we can change anything about this in the public
+dnl API's include files.
+GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS], [-Wno-error=documentation-deprecated-sync])
+
+dnl We used to have these four... no idea why, though. Stop reporting
+dnl unused parameters for now, as those are VERY numerous and not much
+dnl of a security risk, unlike the format and string warnings.
+GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS], [-Wno-unused-parameter])
+dnl GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS], [-Wno-format-overflow])
+dnl GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS], [-Wno-format-truncation])
+dnl GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS], [-Wno-stringop-truncation])
+
+])dnl GP_WITH_EMPTY_CONFDEFS_H
 
 dnl Every compile example after here will be using the C language
 AC_LANG([C])
@@ -129,9 +185,6 @@ dnl ---------------------------------------------------------------------------
 dnl Turn on (almost) all warnings when using gcc
 dnl ---------------------------------------------------------------------------
 if test "x$GCC" = "xyes"; then
-	# CFLAGS="$CFLAGS --std=c99 -Wall -Wextra -Werror -pedantic"
-	CFLAGS="$CFLAGS -Wall -Wmissing-declarations -Wmissing-prototypes"
-	LDFLAGS="$LDFLAGS"
 	AC_SUBST([NO_UNUSED_CFLAGS], [-Wno-unused])
 fi
 AM_CONDITIONAL([HAVE_GCC], [test "x$GCC" = "xyes"])
diff --git a/gphoto-m4/gp-pedantic-compiler-flags.m4 b/gphoto-m4/gp-pedantic-compiler-flags.m4
index b84aa0b51..1b690428c 100644
--- a/gphoto-m4/gp-pedantic-compiler-flags.m4
+++ b/gphoto-m4/gp-pedantic-compiler-flags.m4
@@ -7,14 +7,30 @@ dnl ####################################################################
 dnl GP_CONDITIONAL_COMPILE_FLAGS(FLAG_VAR, FLAGS)
 dnl ####################################################################
 AC_DEFUN([GP_CONDITIONAL_COMPILE_FLAGS], [dnl
+dnl
 # BEGIN $0($@)
-AS_VAR_IF([$1], [], [dnl
-  $1="$2"
-], [dnl
-  $1="[$]{$1} $2"
-])
-AC_MSG_CHECKING([whether $1="[$]$1" compiles])
-AC_COMPILE_IFELSE([m4_case([$1], [CFLAGS], [dnl
+dnl
+m4_pushdef([the_lang],
+           [m4_case([$1],
+                    [CFLAGS], [C],
+                    [CXXFLAGS], [C++],
+                    [m4_fatal([unhandled compiler flags variable: ][$1])])])dnl
+dnl
+AC_LANG_PUSH(the_lang)dnl
+dnl
+dnl If $2 is a warning option, normalize it, as -Wno-foo will compile
+dnl successfully, regardless of whether the compiler even knows the
+dnl -Wfoo warning. FIXME: Split $2 and operate on each part?
+m4_pushdef([normalized_option],
+           [m4_bpatsubst([$2],
+                         [^-W\(no-\)?\(error=\)?\([-_0-9a-zA-Z=]+\)$],
+			 [-W\3])])dnl
+dnl
+saved_$1="${$1}"
+$1="${saved_$1} normalized_option -Werror"
+AC_MSG_CHECKING([whether $1 can append $2])
+AC_COMPILE_IFELSE([dnl
+m4_case(the_lang, [C], [dnl
 AC_LANG_SOURCE([[
 #include <stdio.h>
 int main(int argc, char *argv[])
@@ -27,7 +43,7 @@ int main(int argc, char *argv[])
   return 0;
 }
 ]])dnl
-], [CXXFLAGS], [dnl
+], [C++], [dnl
 AC_LANG_SOURCE([[
 #include <iostream>
 int main(int argc, char *argv[])
@@ -40,14 +56,18 @@ int main(int argc, char *argv[])
   return 0;
 }
 ]])dnl
-], [m4_fatal([wrong compiler flag])])], [dnl
-  : "Added flag $2 to $1 and it works"
+], [m4_fatal([unhandled language ]the_lang)])dnl
+], [dnl
   AC_MSG_RESULT([yes])
+  $1="${saved_$1} $2"
 ], [dnl
-  : "Added flag $2 to $1 and it does not work"
   AC_MSG_RESULT([no])
   gp_have_pedantic_compiler=no
+  $1="$saved_$1"
 ])
+m4_popdef([normalized_option])dnl
+AC_LANG_POP(the_lang)dnl
+m4_popdef([the_lang])dnl
 # END $0($@)
 ])dnl
 dnl
@@ -117,6 +137,72 @@ dnl
 dnl
 dnl ####################################################################
 dnl
+dnl GP_WITH_EMPTY_CONFDEFS_H
+dnl
+dnl Usage:
+dnl     GP_WITH_EMPTY_CONFDEFS_H([
+dnl       GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS], [-Wno-foo])
+dnl       GP_CONDITIONAL_COMPILE_FLAGS([CFLAGS], [-Wno-bar])
+dnl     ])dnl
+dnl
+dnl Since autoconf 2.63b, the macros defined by AC_DEFINE so far
+dnl (mostly PACKAGE, VERSION, etc.) are in confdefs.h by now and are
+dnl included using AC_LANG_SOURCE or AC_LANG_PROGRAM.
+dnl
+dnl We cannot use a confdefs.h with macro definitions for the pedantic
+dnl compilation tests as that pedantic compilation might complain
+dnl about macros being defined but not used.
+dnl
+dnl We also cannot just put AC_LANG_DEFINES_PROVIDED inside the
+dnl program, as that will make sure the program starts with the
+dnl current content of confdefs.h.
+dnl
+dnl In order to be able to test compile programs without any macro
+dnl definitions from confdefs.ht, we save the original confdefs.h file
+dnl and use an empty confdefs.h for our checks, then restore the
+dnl original confdefs.h after our checks are done.
+dnl
+dnl Of course, we must not append to confdefs.h by e.g. calling
+dnl AC_DEFINE or AC_DEFINE_UNQUOTED until the original confdefs.h has
+dnl been restored, as those definitions would then be lost.
+dnl
+dnl ####################################################################
+AC_DEFUN([GP_WITH_EMPTY_CONFDEFS_H], [dnl
+m4_pushdef([AC_DEFINE],
+           [m4_fatal([Must not use AC_DEFINE inside $0])])dnl
+m4_pushdef([AC_DEFINE_UNQUOTED],
+           [m4_fatal([Must not use AC_DEFINE_UNQUOTED inside $0])])dnl
+cat confdefs.h
+AC_MSG_NOTICE([saving original confdefs.h and creating empty confdefs.h])
+rm -f confdefs.h.saved
+mv confdefs.h confdefs.h.saved
+cat <<EOF >confdefs.h
+/* confdefs.h forced empty for the pedantic compile tests */
+EOF
+dnl
+AC_MSG_CHECKING([CFLAGS value before])
+AC_MSG_RESULT([$CFLAGS])
+AC_MSG_CHECKING([CXXFLAGS value before])
+AC_MSG_RESULT([$CXXFLAGS])
+dnl
+$1
+dnl
+AC_MSG_CHECKING([CFLAGS value after])
+AC_MSG_RESULT([$CFLAGS])
+AC_MSG_CHECKING([CXXFLAGS value after])
+AC_MSG_RESULT([$CXXFLAGS])
+dnl
+AC_MSG_NOTICE([restoring original confdefs.h])
+rm -f confdefs.h
+mv confdefs.h.saved confdefs.h
+cat confdefs.h
+m4_popdef([AC_DEFINE_UNQUOTED])dnl
+m4_popdef([AC_DEFINE])dnl
+])dnl
+dnl
+dnl
+dnl ####################################################################
+dnl
 dnl Local Variables:
 dnl mode: autoconf
 dnl End: