open: detect FreeBSD bug

open("link-to-file/", O_RDONLY) mistakenly succeeds.  The previous
patch was enough to fix utimens when no fd is involved, but this is
necessary for futimens to pass.

* m4/open.m4 (gl_FUNC_OPEN): Also detect FreeBSD bug with slash on
symlink.
* doc/posix-functions/open.texi (open): Document the bug.
* doc/posix-functions/utimes.texi (utimes): Likewise.
* tests/test-open.h (test_open): Add parameters, and test symlink
handling.
* tests/test-open.c (main): Adjust caller.
* tests/test-fcntl-safer.c (main): Likewise.
* modules/open-tests (Depends-on): Add stdbool, symlink.
* modules/fcntl-safer-tests (Depends-on): Likewise.
* tests/test-openat.c (main): Add test-open tests.

Signed-off-by: Eric Blake <ebb9@byu.net>

1 files changed, 13 insertions, 4 deletions

diff --git a/m4/open.m4 b/m4/open.m4
index c0eb8e8623..ba7d8761b1 100644
--- a/m4/open.m4
+++ b/m4/open.m4
@@ -1,4 +1,4 @@
-# open.m4 serial 7
+# open.m4 serial 8
 dnl Copyright (C) 2007-2009 Free Software Foundation, Inc.
 dnl This file is free software; the Free Software Foundation
 dnl gives unlimited permission to copy and/or distribute it,
@@ -13,10 +13,15 @@ AC_DEFUN([gl_FUNC_OPEN],
       ;;
     *)
       dnl open("foo/") should not create a file when the file name has a
-      dnl trailing slash.
+      dnl trailing slash.  FreeBSD only has the problem on symlinks.
+      AC_CHECK_FUNCS_ONCE([lstat])
       AC_CACHE_CHECK([whether open recognizes a trailing slash],
         [gl_cv_func_open_slash],
-        [
+        [# Assume that if we have lstat, we can also check symlinks.
+          if test $ac_cv_func_lstat = yes; then
+            touch conftest.tmp
+            ln -s conftest.tmp conftest.lnk
+          fi
           AC_TRY_RUN([
 #include <fcntl.h>
 #if HAVE_UNISTD_H
@@ -24,18 +29,22 @@ AC_DEFUN([gl_FUNC_OPEN],
 #endif
 int main ()
 {
+#if HAVE_LSTAT
+  if (open ("conftest.lnk/", O_RDONLY) != -1) return 2;
+#endif
   return open ("conftest.sl/", O_CREAT, 0600) >= 0;
 }], [gl_cv_func_open_slash=yes], [gl_cv_func_open_slash=no],
             [
 changequote(,)dnl
              case "$host_os" in
+               freebsd*)        gl_cv_func_open_slash="guessing no" ;;
                solaris2.[0-9]*) gl_cv_func_open_slash="guessing no" ;;
                hpux*)           gl_cv_func_open_slash="guessing no" ;;
                *)               gl_cv_func_open_slash="guessing yes" ;;
              esac
 changequote([,])dnl
             ])
-          rm -f conftest.sl
+          rm -f conftest.sl conftest.tmp conftest.lnk
         ])
       case "$gl_cv_func_open_slash" in
         *no)