path: root/STATUS

APACHE PORTABLE RUNTIME (APR) LIBRARY STATUS:			-*-text-*-
Last modified at [$Date: 2002/07/15 01:25:05 $]

Release:

    0.9.0-dev : in progress

    2.0a9   : released December 12, 2000
    2.0a8   : released November 20, 2000
    2.0a7   : released October 8, 2000
    2.0a6   : released August 18, 2000
    2.0a5   : released August 4, 2000
    2.0a4   : released June 7, 2000
    2.0a3   : released April 28, 2000
    2.0a2   : released March 31, 2000
    2.0a1   : released March 10, 2000

RELEASE SHOWSTOPPERS:

    * Must namespace protect all include/apr_foo.h headers.  Jon Travis
      has especially observed these including apr and Apache-1.3.
        Message-ID: <20020128100116.A4288@covalent.net>
      (Those problems have been fixed, but it is a good example of
      what to look for.)
      Some headers with issues: 
        apr_fnmatch.h         (FNM_foo)
        apr_general.h         (MAXIMUM_WAIT_OBJECTS)
        apr_md5.h             (MD5_DIGESTSIZE)
        apr_network_io.h      (MAX_SECONDS_TO_LINGER)
        apr.hnw               (READDIR_IS_THREAD_SAFE, ENUM_BITFIELD, 
                              _POSIX_THREAD_SAFE_FUNCTIONS (?),
                              nuint8, nuint16, NGetLo8, NGetHi8,
                              HIBYTE, LOBYTE)
        apr.hw                (NO_USE_SIGACTION)

    * complete the efforts started by DougM for cleaner fn naming
      conventions: see proposed name changes in renames_pending
      and offer up any additions/vetos/clarifications.
      DougM offered to complete the work with his nifty perl rename
      script at the hackathon. 

    * When Win32 apr_proc_create was fixed, the apr_proc_t hproc
      member was added for that platform.  That's a problem (and
      was when pid was abused as well) since nobody goes and cleans
      up hproc after the process is dead.  Can't do a pool cleanup,
      since apr_proc_create didn't allocate the apr_proc_t storage.
      (Aren't transparent types swell?)  Suggestions?

    * Almost every API in APR depends on pools, but pool semantics
      aren't a good match for a lot of applications.  We need to find
      a way to support alternate allocators polymorphically without
      a significant performance penalty.

    * extract the MAJOR version from apr_version.h and pass it to
      libtool for use in applying version numbers to the shared
      libraries.


CURRENT VOTES:

    * apr_time_t will change to use binary microseconds based on
      profiling.  The last remaining question on the table is keeping
      the apr_time_t designation, or changing the symbol name.

      1) Keeping the existing apr_time_t names, in spite of confusion
         with ANSI/C99 time_t's units, and prior decimal usec definition.
         +1: rbb, jerenkrantz, striker, dreid, jim, jwoolley, brane
         +0: wrowe [apr_types don't promise to map to C99/ANSI units]
         +0: brianp
         -1: aaron [veto for reusing the apr_time_t identifier for a new use]
             fielding [if they don't map to system types, then don't mimic
               the system types --- give me back all of my ap_time functions
               that were converted to microsecond arguments even though none
               of them do anything useful with microseconds. Confusion is
               demonstrated by dozens of bug fixes since it was introduced.]
             ianh [me too]

      2) Renaming the function to get rid of apr_time_t vs time_t confusion,
         but keep it ambigious and make no contract with the user about the
         units represented.  Needs a better suggestion than apr_timeval_t.
         +1: aaron, brianp, ianh,
             fielding [prefers apr_utime_t and apr_utimediff_t (64bit)]
	 +0.5: jim
         -0: jerenkrantz, dreid
         -0.5: rbb, jwoolley, striker, brane
               wrowe [prefers apr_utime_t and apr_uspan_t where u==undefined]

      3) Renaming the function to get rid of apr_time_t vs time_t confusion,
         and strongly identify the type as apr_busec_t or apr_butime_t, with
         an ongoing contract with users about the type's units.
         +0.5: wrowe,  [prefers apr_time_busec_t and apr_span_busec_t]
               brianp, [can live with apr_time_busec_t and apr_span_busec_t]
               fielding [me too]
         +0: brane
         -0: jerenkrantz
         -0.5: rbb, ianh, dreid, jim, jwoolley, striker

      4) Using time_t and struct timeval/tm
         +1: fielding (if apr_time is not an ADT)
         -1: brianp, wrowe, jwoolley, brane

      [fielding: Is APR time guaranteed to be a scalar quantity?  If so,
       then we must include units as part of the definition of the
       type in order to let developers make use of that quarantee.  
       In that case, the units should be in the type name [e.g., apr_busec]

      [brianp: I think that apr_time_t is really a "struct with an
       compact representation that we can pass around easily and
       add/subtract efficiently," rather than a scalar.  It's probably
       worth noting that I look at it this way because it often is
       populated from the struct timeval produced by gettimeofday().
       Because I think of the scalar representation as an implementation
       detail, rather than an feature of the time API, I'd prefer to
       use a name that doesn't advertise the binary microseconds concept.
       But I've changed my -1 on the binary microsecond name to a -0.5.]

      [wrowe: deltas require NO definition of the scale.]

        [fielding: That's nonsense. What does overflow mean?  What are you
         going to do when you print?  How do you interface with other library
         routines?  Scale always matters for scalars.]

          [wrowe: We have apr_time formatting and math routines.
           But I've always favored an explicit contract.]

      [fielding: If not, then we should be storing time in a 
       structure with separate fields in order to have better precision 
       with less code.]

      [wrowe: dean argued that away a very, very long time ago.  That is
       a dead horse... compositing and breaking apart for each simple deltas 
       (the most common case) is too costly.  Scalars are the only clean
       answer - and you do not need to know scale to do addition/subtraction.]

        [fielding: Dean argued that in general.  I argue that httpd never
         does time arithmetic other than in seconds and second-comparisons.
         Microseconds are therefore harmful to httpd.]
       
      [fielding: In any case, time_t ==> seconds because time_t is guaranteed
       by POSIX to be a scalar quantity for arithmetic operations.
       Saying that apr_time_t doesn't imply seconds is to ignore the
       fact that all of those httpd functions used to create APR were
       defined in terms of seconds and make no use of microseconds.
       Meanwhile, the only reason we have this debate is 
       because wrowe insists that time_t is 32 bits and therefore dies in 2038.
       In fact, time_t is 64 bits on 64bit NT, Linux, OSF, and probably
       others that I haven't checked.  In any case, since we use the
       system's time_t time() function to get the time value everywhere
       except Win32 w/SYSTEMTIME, we only ever have a resolution of
       seconds or milliseconds.  So, why the hell are we storing usecs?

           [brianp: This is incorrect.  We use gettimeofday() in place
            of time.  It's faster than time(), and it gives us microseconds
            in addition to seconds.  Why do you want to throw away the
            microseconds?!!]

              [fielding: Sorry, I missed them:
                  86 calls to apr_time_now()
                  32 calls to time()
               +1 to making time consistent.]

       We don't use them.  We don't even have display functions for them.
       We have to do a stupid conversion every time we actually do something
       useful with time in order to support somebody's wet dream of a
       potentially useful feature?  That's crap!  This is exactly why
       I hate portability libraries that aren't based on the demonstrated
       needs of a specific application.]

      [wrowe: 1. no, time_t is undefined.  Sometimes 32, sometimes 64 bits.
       2. no, on 64 bit WinNT time_t remains 32 bits, as do all ints.
       3. several apps include flood and ab require usec resolution.
       4. Posix timeval structures use sec/usec resolution.]

     [fielding: 1. POSIX requires it to be long, so largest native int.
       2. Microsoft claims otherwise, but it is still vaporware anyway.
       3. Benchmarks are meaningless unless they average over hundreds
          of requests, which requires double floats (not time intervals).
       4. POSIX always stores them as separate integers. +1 for that.]

      [fielding; Cliff says he has a sample app.  I still don't know how
       he uses them without making implementation assumptions about
       apr_time_t everywhere (there is no print routine for microsecond
       resolution), but I'll accept the need for microsecond resolution
       in addition to second resolution.]

RELEASE NON-SHOWSTOPPERS BUT WOULD BE REAL NICE TO WRAP THESE UP:

    * The return type of a thread function (void *) is inconsistent with
      the type used in apr_thread_exit()/apr_thread_join() (apr_status_t).
      The thread function's return type should be changed to apr_status_t
      so that a return from the thread main function has the same effect
      as apr_thread_exit().
      See Message-Id: <E16JjZA-0007hg-00@zakath.apana.org.au> for thread
      discussing this.
        +1: BrianH, Aaron

    * Need some architecture/OS specific versions of the atomic operations.
	progress: generic, solaris Sparc, FreeBSD5, linux, and OS/390 done
	need: AIX, AS400, HPUX

    * The new lock API is a full replacement for the old API, but is
      not yet complete on all platforms. Components that are incomplete
      or missing include:
      Netware: apr_proc_mutex_*() (Is proc_mutex unnecessary on Netware?)
      OS/2: apr_thread_cond_*(), apr_proc_mutex_*()

      Less critical components that we may wish to add at some point:
      Beos: apr_thread_rwlock_try*lock()
            apr_proc_mutex_trylock()
      Unix: apr_thread_rwlock_*() for platforms w/o rwlocks in pthread
      Win32: apr_thread_rwlock_try*lock(), apr_thread_cond_timedwait(),
             apr_proc_mutex_*() (Is proc_mutex unnecessary on Win32?)

    * Need to contemplate apr_strftime... platforms vary.  OtherBill
      suggested this solution (but has no time to implement):
        Document our list of 'supported' escapes.
        Run some autoconf/m4 magic against the complete list we support.
        Move the strftime re-implementation from time/win32 to time/unix.
        Add some APR_HAVE_STRFTIME magic to use the system fn, or fail
        over to time/unix/strftime.c.
        Message-ID: <025e01c1a891$bf41f660$94c0b0d0@v505>

    * Using reentrant libraries with non-threaded APR
        - Anecdotal evidence exists that suggests it is bad to
          mix reentrant and non-reentrant libraries and therefore
          we should always use the reentrant versions.
        - Unfortunately, on some platforms (AIX 4.2.1) defining
          the reentrant flag (-D_THREAD_SAFE) causes builds to fail,
          and so one would expect --disable-threads to fix this.
          Although this has been fixed for that particular version
          of AIX, it may be useful to only enable the reentrant
          versions when threads are enabled.
        How will we deal with this issue once APR becomes a standalone
        library? It is perfectly legitimate to have apps needing
        both versions (threaded/reentrant and non-threaded/non-reentrant)
        on the same machine.

    * Pools debugging
        - Find a way to do check if a pool is used in multiple
          threads, while the creation flags say it isn't.  IOW,
          when the pool was created with APR_POOL_FNEWALLOCATOR,
          but without APR_POOL_FLOCK.
          Currently, no matter what the creation flags say, we always
          create a lock.  Without it integrity_check() and
          apr_pool_num_bytes() blow up (because they traverse pools
          child lists that possibly belong to another thread, in
          combination with the pool having no lock).  However,
          this might actually hide problems like creating a child pool
          of a pool belonging to another thread.
          Maybe a debug function apr_pool_set_owner(apr_thread_t *) in
          combination with extra checks in integrity_check() will point
          out these problems.  apr_pool_set_owner() would need to be called
          everytime the owner(the thread the pool is being used in) of
          the pool changes.

        - Implement apr_pool_join and apr_pool_lock.  Those functions
          are noops at the moment.

        - Add stats to the pools code.  We already have basic stats
          in debug mode.  Stats that tell us about wasted memory
          in the production code require more thought.
            Status: Sander Striker is looking into this (low priority)

    * Deal with largefiles properly on those platforms that support it.

      Justin says: Linux refuses to have sendfile support and largefiles
                   at the same time.  Largefile autoconf patch:
                   http://www.apache.org/~jerenkrantz/apr_largefile.m4

    * Get OTHER_CHILD support into Win32
        Status: Bill S. is looking into this

    * Win32 apr_proc_create fails to create 16 bit apps detached
      (a win32 bug.)  The question - test in advance (slow) or
      recover gracefully from failure and try again?  Only the test
      method will work on Win9x, since it will appear to work, only
      to encounter mangled pipes.  Win2K (NT?) simply fails.

    * SysV semaphore support isn't usable by Apache when started as
      root because we don't have a way to allow the semaphore to be
      used by the configured User and Group.  Current work-around:
      change the initial permissions to 0666.  Needed code:  See
      1.3's http_main.c, SysV sem flavor of accept_mutex_init().
      Status: Jim will look into this
      Update: Apache deals with this itself, though it might be nice
              if APR could do something.

    * Build scripts do not recognise AIX 4.2.1 pthreads
      Justin says: "Is this still true?"

    * FirstBill says we need a new procattr, APR_CREATE_SUSPENDED (or
      something similar) to direct ap_create_process to create the
      process suspended. We also need a call to wake up the suspended 
      process.  This may not be able to be implemented everywhere though.
        Status: OtherBill asks, why?  What is the benefit, how is it
        portably implemented?  Unless this creates some tangible that
        mirrors another platform, then I'm -1.

    * Replace tables with a proper opaque ADT that has pluggable
      implementations (including something like the existing data type,
      plus hash tables for speed, with options for more later).
	Status: fanf is working on this.

    * add a version number to apr_initialize() as an extra failsafe against
      (APR) library version skew.
      MsgID: <Pine.LNX.4.10.10005231712380.31927-100000@nebula.lyra.org>
      Status: Greg +1 (volunteers), Jeff +1, Ryan +1, Tony -0(?), david +1

    * add apr_crypt() and APR_HAS_CRYPT for apps to determine whether the
      crypt() function is available, and a way to call it (whether it is
      located in libc, libcrypt, or libufc)
      Justin says: Should apr_crypt() be in apr-util?

        Status: Greg +1 (volunteers)

    * configure.in does post-processing on the AC_OUTPUT files (for
      VPATH support). This means that config.status doesn't do the
      right thing when you re-run it. We ought to revamp the makefiles 
      to do the right AC_SUBST stuff rather than depend upon rewriting.

      Sascha: As the rewriter is a crude hack, I would not worry too
              much about it.  It is designed to go away once we have
              a proper build system in place.
              
              One of the perceived deficiencies of automake is that it
              uses AC_SUBST too often, thereby slowing down the task of
              generating Makefiles significantly, because it applies
              dozens of substitutions to each Makefile.  And why?  Make's
              built-in macro processing is much more powerful, and
              combined with the include facility, generating Makefiles
              becomes simpler and faster.
      Justin says: "I think this got fixed with Roy's build changes."

    * use os_(un)cork in network_io/unix/sendrecv.c for FreeBSD's
      sendfile implementation.

      david: The socket options stuff is now in and using it should
             reduce the number of syscalls that are required for 
             os_cork and uncork, so the code should be reviewed to
             make use of the new calls.  If no-one beats me to it I'll
             get around to it soonish...

    * toss the per-Makefile setup of INCLUDES; shift to rules.mk.in
        rbb: This is a bad thing IMHO.  If we do this, then we
             can't use these makefiles for anything else.  For example,
             apr-util

    * add the rest of the pool accessor declare/impl macros.
      Justin says: Both thread and file have the accessors now.  Any others?
      Status: Greg volunteers

    * I think apr_open_stderr() and friends dup() the descriptor. That
      would allow the new/returned file to be closed (via pool cleanup
      or manually) without accidentally closing stderr/out.
      Justin says: Is this "I think it should?"

    * need to export the shared library extension (e.g. ".so") for the
      platform. clients need to use this to construct filenames to
      pass to apr_dso_load()
      -- note on Win32 we distinguish 'apache module' names from other 
         'loadable module' names, so be careful with Apache's directive.

    * Possible gmtime_r replacement in explode_time
      On Solaris (and possibly others), the gmtime_r libc function obtains
      a mutex.  We have seen 21/25 threads being blocked in this mutex on 
      a threaded httpd MPM when requesting static pages.  It may be worth 
      it to hand optimize this since there is no real need for a mutex at 
      the system level (straight arithmetic from what I can tell).  If you 
      have access to the Solaris source code:
        osnet_volume/usr/src/lib/libc/port/gen/time_comm.c.

    * Add a way to query APR for what features it has at runtime (i.e. 
      threads).
      Justin says: I'm not completely sold on this, but it has been mentioned 
                   before and at least added to STATUS.

    * apr_xlate.h generates a bunch of compiler warnings.
      Jeff asks: which platform?
      Justin says: Solaris with Forte 6.1.

    * fcntl() oddness on Solaris.  Under high loads, fcntl() decides to
      return error code 46 (ENOLCK).

      httpd (prefork MPM) error log says (predictably):

      (46)No record locks available: couldn't grab the accept mutex
 
      All of the children report this and subsequently exits.  httpd is now 
      hosed.  AFAICT, this does not look to be an out-of-fds error.
 
      Solaris's man page says:
        ENOLCK
           The  cmd argument is F_SETLK, F_SETLK64, F_SETLKW,  or
           F_SETLKW64  and  satisfying the lock or unlock request
           would result in the number of locked  regions  in  the
           system exceeding a system-imposed limit.

      Justin says: What is this system-imposed limit and how do we change it?
                   This gives me more rationale for switching the default
                   interprocess lock mechanism to pthread (if available).

    * Generate a good bug report to send to the FreeBSD hackers that details
      the problems we have seen with threads and system calls (specifically
      sendfile data is corrupted).  From our analysis so far, we don't think 
      that this is an APR issue, but rather a FreeBSD kernel issue.  Our
      current solution is to just disable threads across the board on
      FreeBSD.

      MsgID: <20010828091959.D17570@ebuilt.com>
        Status: Fixed in -CURRENT.  MFC in about a week.  Continuing
                testing with threads on FreeBSD.

                FreeBSD PR kern/32684:
                  http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/32684

    * There are some optimizations that can be done to the new
      apr_proc_*() functions (on UNIX). One that may reduce pointer
      indirection would be to make the apr_proc_mutex_unix_lock_methods_t
      first-class members of the apr_proc_mutex_t structure.

    * Condition variables are tricky enough to use, and even trickier
      to implement properly. We could really use a better test case
      for those subtle quirks that sometimes creep into CV implementations.

    * Once we are fully satisfied with the new lock API, we can
      begin to migrate the old API to be implemented on top of the
      new one, or just decide to get rid of it altogether.
      
    * FreeBSD returns 45 (EOPNOTSUPP) when the lockfile is on a NFS
      partition when you call fcntl(F_SETLKW).  It may be good if we 
      can somehow detect this and error out when creating the lock
      rather than waiting for the error to occur when acquiring lock.

    * Fix autoconf tests for strerror_r on BeOS and remove the hack in
      misc/unix/errorcodes.c to get error reporting working.  Committed as
      the solution is elusive at present.

    * implement APR_PROGRAM_ENV and APR_PROGRAM_PATH on BeOS, OS/2,
      Netware, and Win32.

    * stat() on a few platforms (notably Solaris and AIX) succeeds for
      a non-directory even if a trailing '/' was specified in the
      name.  APR should perhaps simulate the normal -1/ENOTDIR
      behavior in APR routines which retrieve information about the
      file.  Note: Win2K fails GetFileAttributesEx in this scenario.
      See OtherBill's comments in this message to dev@httpd.apache.org:
      Message-Id: <5.1.0.14.2.20020315080852.00bce168@localhost>

Documentation that needs writing:

    * API documentation
        Ian Says: APR Stuff in now in Doxygen format, which is the first step.

    * apr-site needs to be revamped with Anakia/XHTML.

Stuff waiting for code thawing after Beta 1:

    * Identify and implement those protection bits that have general 
        usefulness, perhaps hidden, generic read-only [immutable],
        effective current user permissions, etc.

    * Maybe make the following functions return void instead of
      apr_status_t, as they cannot ever error:

         apr_md5_init()
         apr_md5_update()
         apr_md5_final()
         apr_md5_encode()
         apr_md5()       /* plus make the obvious code tweak in this one */

      (Volunteer: Karl Fogel <kfogel@collab.net>.)

      However, don't do this until after apr and apr-util offer
      library version numbers, and httpd uses those numbers to
      indicate which version it needs.  Until the libraries are
      versioned, this api change is [somewhat] painful for httpd.
      Status: Still in discussion, current leanings appear to be
        Bill Stoddard -0.5 (?)
        Sander Striker +1
        Greg Stein +1
        Karl Fogel +1
      (Not sure if the negatives would stay negative given that the
      change would now wait for the library versioning thing described
      above, though.)
        Justin says: If you do this, please move it into apr-util where md5
                     belongs!  You'll have to address the random issue in
                     misc/unix/getuuid.c that forces md5 to be in APR.
        Sander: +1 for the move.