Security implications of tmpreaper
----------------------------------

Below is a message that was forwarded to me, concerning vulnerabilities in
tmpwatch, of which tmpreaper is a fork and may thus share vulnerabilities
with.

IMO there are a couple of things wrong about the points in the text, which I
could not resolve in discussion (my direct questions were avoided by
responding that I clearly didn't understand the issues -- which is (a) faulty
argumentation leading me to believe that they themselves were incapable of
explaining, and (b) wrong). For example, the text speaks of "creation time";
I responded that there is no such thing in POSIX, only the inode change time
(which is also changed if you link the file, or rename it, or change the
permissions, or even when you delete it).  I never got an answer on that.

Here's one thing that was said:

   Suppose a victim program is setuid, and you want to attack it to
   get root. So you run it, let it open a temp file, and SIGSTOP it.
   Now let is sit, suspended, for x days. Tmpreaper then removes the
   temp file, since mtime, atime, and ctime are all x days. Now replace
   the tmp file with something crafted to buffer overflow or otherwise 
   exploit the program, and SIGCONT it so it runs, accesses the tmp file
   it thinks it just made, and is exploited.

   There is no known fix to this hole. You can play with fuser and
   limit it to a certian smaller class of victim programs, but not
   entirely close it. It's also pretty easy to exploit on any machine
   with a large uptime and lots of processes to hide in.

This calls for the setuid program to close the file, and then open it again
without checking the owner, inode number, etc.  This also means that fuser
won't help a bit (the file must be closed, remember? So why mention it). That
in my book is a problem of the setuid program, not of tmpreaper, as this is
nothing that won't happen either if the admin does "rm -rf /tmp/*" now and
again, despite following claim that that is a preferable alternative to
tmpreaper:

   At least admins running rm -rf is something that happens
   unpredictably ...


Another thing that was said:

   tmp{reaper,watch} use lstat() to look at file creation time. Then they
   use unlink() to delete it. Two system calls, therefore there is a
   time period between them in which an attacker may do the very unlikely
   trick described in the paper of removing its decoy file and letting
   mkstemp put in a file by the same name, which is then deleted.
   At this point the attacker can replace the temp file with their own
   creation, and mess with the program that expects its own temp file there.

There's that mythical "file creation time" thing again...
Besides that, he's right in saying that it's very unlikely to manage to squeeze
inbetween those two system calls. Bookies will give you _very_ large odds on
that...

1. you need to be able to predict days in advance what pseudo-random name
   mkstemp will generate at that precise moment in time. From the message
   below:
     ... While mkstemp() names are guaranteed to be unique, they shouldn't be
     expected to be unpredictable - in most implementations, the name is a
     function of process ID and time - so it is possible for the attacker to
     guess it and create a decoy in advance. ...
   Guessing what process ID a victim process will have a number of days in
   advance on a multiuser system is a pretty neat trick.

2. you then need to be able to remove that file at exactly the moment
   tmpreaper after tmpreaper has used lstat(), but before it does the unlink().

3. mkstemp (in the daemon/whatever that you're trying to attack) will
   have to create the file with the correct name, still before tmpreaper's
   unlink() call.

4. now tmpreaper has to execute the unlink() call, before the
   daemon/whatever has gone on and used the tmpfile for its intended purpose.

5. you now have to create the file again, with the correct name and
   contents.

6. the daemon/whatever has to close and reopen the file without checking
   its owner, permission, inode number, etc. and then use the data therein.

This would be an incredible feat to accomplish, needing unbelievable amounts
of luck in timing (skill is not an issue; timing is), requiring also an app
running with sufficient privileges to do harm that uses its tempfiles
insecurely in the first place.  In such a scenario, is the problem with
tmpreaper? I don't think so.


The scenario described is so inprobable that if you're worried about that, you
should buy a lottery ticket every week: the chance that *someone* wins the
lottery is 100% (well, with a lot of lotteries :-); the chance that this will
ever happen is less.


There's also a discussion below about the problems when two tmpwatch processes
are running simultaneously (hence would also apply to tmpreaper as well).
Tmpreaper avoids this by not allowing itself to run for more than one minute.
There's also a random delay before processing (when invoked with --delay as is
done in the default cron.daily script) to make it harder to determine when
tmpreaper runs exactly.
See also the tmpreaper manpage for other precautions taken.

Of course, if you still are worried, you are of course free to purge tmpreaper
from your system, and do a regular "rm -rf /tmp/*" if you think that's
safer...


Below is the message that started this discussion, and judge for yourself.



Paul Slootman 2003-05-21, updated 2003-06-02


From: Michal Zalewski <lcamtuf@ghettot.org>
Date: Fri, 20 Dec 2002 09:30:30 -0800 (PST)
To: bugtraq@securityfocus.com, <vulnwatch@vulnwatch.org>,
    <full-disclosure@netsys.com>
Subject: [RAZOR] Problems with mkstemp()

  Common use of 'tmpwatch' utility and its counterparts triggers race
  conditions in many applications

  Michal Zalewski <lcamtuf@razor.bindview.com>, 12/05/2002
  Copyright (C) 2002 by Bindview Corporation


1) Scope and exposure info
--------------------------

  A common practice of installing 'tmpwatch' utility or similar software
  configured to sweep the /tmp directory on Linux and unix systems can
  compromise secure temporary file creation mechanisms in certain applications,
  creating a potential privilege escalation scenario. This document briefly
  discusses the exposure, providing some examples, and suggesting possible
  workarounds.

  It is believed that many unix operating systems using 'tmpwatch' or an
  equivalent are affected. Numerous Linux systems, such as Red Hat, that ship
  with cron daemon running and 'tmpwatch' configured to sweep /tmp are
  susceptible to the attack.


2) Application details
----------------------

  'Tmpwatch' is a handy utility that removes files which haven't been
  accessed for a period of time. It was developed by Erik Troan and
  Preston Brown of Red Hat Software, and, with time, has become a
  component of many Linux distributions, also ported to platforms
  such as Solaris, *BSD or HP/UX. By default, it is installed with a
  crontab entry that sweeps /tmp directory on a daily basis, deleting
  files that have not been accessed for the past few days.

  An alternative program, called 'stmpclean' and authored by Stanislav
  Shalunov, is shipped with *BSD systems and some Linux distributions
  to perform the same task, and some administrators deploy other tools or
  scripts for this purpose.


3) Vulnerability details
------------------------

  Numerous applications rely either on mkstemp() or custom O_EXCL file
  creation mechanisms to store temporary data in the /tmp directory
  in a secure manner. Of those, certain programs run with elevated
  privileges, or simply at a different privilege level than the caller.

  The exposure is a result of a common misconception, promoted by almost
  all secure programming tutorials and manpages, that /tmp files created
  with mkstemp(), granted that umask() settings were proper, are
  safe against hijacking and common races. The file, since it is created
  in a sticky-bit directory, indeed cannot be removed or replaced by
  the attacker running with different non-root privileges, but since
  many operating systems feature 'tmpwatch'-alike solutions, the only
  thing that can and should be considered safe in /tmp is the descriptor
  returned by mkstemp() - the filename should not be relied upon. There
  are two major reasons for this:

  1) unlink() races

     It is very difficult to remove a file without risking a potential
     race (see section 4). 'Tmpwatch' does not take any extra measures to
     prevent races, and probes file creation time using lstat(). Based on this
     data, it calls unlink() as root. Problem is, on a multitasking system,it
     is possible for the attacker to get some CPU time between those two system
     calls, remove the old "decoy" file that has been probed with lstat(), and
     let the application of his choice create its own temporary file under this
     name. While mkstemp() names are guaranteed to be unique, they shouldn't be
     expected to be unpredictable - in most implementations, the name is a
     function of process ID and time - so it is possible for the attacker to
     guess it and create a decoy in advance. Once the tmpwatch process is
     resumed, the file is immediately removed, based on the result of
     earlier lstat() on the old, no longer existing file.

     While this three-component race requires very precise timing, it
     is possible to try a number of times in a single 'tmpwatch' run if
     enough decoy files are created by the attacker. Additionally, since
     each step of the attack would result in a corresponding filesystem
     change, it is fairly easy to carefully measure timings and
     coordinate the attack.

     If the attacker cannot make the application run at the same time
     as 'tmpwatch' - for example, if the application is executed by
     hand by the administrator, or is running from cron - 'tmpwatch'
     itself can be artificially delayed for almost an arbitrary amount
     of time by creating and continuously expending an elaborate directory
     structure in /tmp using hard links (to preserve access times of
     files) and running other processes that demand disk access and
    cache space to slow down the process.

     'Stmpclean' offers additional protection against races by not removing
     root-owned files and temporarily dropping privileges when removing
     the file to match the owner of lstat()ed resource. Unfortunately,
     not removing root files is a considerable drawback, and there is still
     a potential for a race using carefully crafted hard links to a file
     owned by the victim and two concurrent 'stmpclean' processes:

       - the attacker links /tmp/foo to ~victim/.bash_profile
       - tmpwatch #1 does lstat() on /tmp/foo and setuid victim
       - tmpwatch #2 does lstat() on /tmp/foo and setuid victim
       - tmpwatch #1 does unlink("/tmp/foo")
       - victim application creates /tmp/foo at uid==victim
       - tmpwatch #2 does unlink("/tmp/foo") and succeeds
       - the attacker creates /tmp/foo
       - victim application proceeds

     On certain systems such as Owl Linux, the attack will be not possible
     due to hardlink limits imposed on sticky-bit directories.

  2) suspended processes and 'legitimate' file removal

     Here, all conventional measures that could be exercised by /tmp cleaners
     fail miserably. A vulnerable application can be often delayed or suspended
     after mkstemp() / open() - for example, a setuid program can be
     stopped with SIGSTOP and resumed with SIGCONT. If the application is
     suspended for long enough, its temporary files are likely to be
     removed. This method requires much less precision, but is also
     more time-consuming and has a more limited scope (interactive
     applications only).

     Note that it is sometimes possible to delay the execution of
     a daemon - client wait, considerable I/O or CPU loads, and subsequent
     mkstemp() calls can be all used to achieve the effect. The
     feasibility and efficiency is low, but the potential issue
     exists. Some client applications that are often left unattended
     and create temporary files - such as mail/news clients, web
     browsers, irc clients, etc - can also be used to compromise
     other accounts on the machine.

  Not all applications are prone to the problem just because mkstemp()
  is used to create files in /tmp; if the file name is not used to perform
  any sensitive operations with some extra privileges afterward (read,
  write, chown, chmod, link/rename, etc), and only the descriptor is
  being used, the application is safe. This practice is often exercised by
  programmers who want to avoid leaving dangling temporary files in case
  the program is aborted or crashes. Similarly, if the application uses
  temporary files improperly, but does not rely on their contents and does
  not attempt to access them with higher privileges, the application is
  secure in that regard.

  Applications that run with higher privileges and reopen their
  /tmp temporary files for reading or writing, call chown(), chmod() on
  them, rename or link the file to replace some sensitive information, and
  so on, are exposed. It is worth mentioning that a popular 'mktemp'
  utility coming from OpenBSD passes only the filename to the
  caller shell script, thus rendering almost all scripts using it
  fundamentally flawed. If the script is being run as a cron job or
  other administrative task, and mktemp is used, the system can be likely
  compromised by replacing the file after mktemp and prior to any write
  to the file. In the example quoted in the documentation for mktemp(1):

    TMPFILE=`mktemp /tmp/$0.XXXXXX` || exit 1
    echo "program output" >> $TMPFILE

  ...the attacker would want to replace temporary file right before
  'echo', causing the text "program output" to be appended to a target
  file of his choice using symlinks or hardlinks; or, if it is more
  desirable, he'd spoof file contents to cause the program to misbehave.

  Another example of the problem is a popular logrotate utility,
  coded - ironically - by Erik Troan, one of co-authors of 'tmpwatch'
  itself. The program suffered /tmp races in the past, but later
  switched to mkstemp(). The following sequence is used to handle
  post-rotation shell commands specified in config files:

  open("/tmp/logrotate.wvpNmP", O_WRONLY|O_CREAT|O_EXCL, 0700) = 6
  ...
  write(6, "#!/bin/sh\n\n", 11)     = 11
  write(6, "\n\t/bin/kill -HUP `cat /var/lock/"..., 79) = 79
  close(6)                          = 0
  ... fork, etc ...
  execve("/bin/sh", ["sh", "-c", "/bin/sh /tmp/logrotate.wvpNmP" ...

  Obviously, if the attacker can have /tmp/logrotate.* replaced in
  between mkstemp() (represented as open() syscall above) and the
  point where another process is spawned, a shell interpreter is invoked,
  then executes another copy of the shell interpreter (apparent
  programmer's mistake) and finally reads the input file - which is
  a considerable chunk of time - the shell will be called with
  attacker-supplied commands to be executed with root privileges.

  On Red Hat, logrotate is executed from crontab on a daily basis, in
  a sequence before 'tmpwatch', and the easiest option for the attacker
  is to maintain a still-running tmpwatch process from the previous day
  to exploit the condition. On systems where those programs are not
  executed sequentially - for example, when both programs are listed
  directly in /etc/crontab - the attack requires less precision.


4) Workarounds and fixes:
-------------------------

  Recommended immediate workaround is to discontinue the use of 'tmpwatch'
  or equivalent to sweep /tmp directory if this service is not necessary.

  For applications that rely on TMPDIR or a similar environment
  variable, setting it to a separate, not publicly writable directory
  is often a viable solution. Note that not all applications honor
  this setting.

  In terms of a permanent solution, two different attack vectors have
  to be addressed, as discussed in section 3:

  1) unlink() race

     The proper way to remove files in sticky-bit directories while
     minimizing the risk is as follows:

       a) lstat() the file to be removed
       b) if owned by root, do not remove
       c) if st_nlink > 1, do not remove
       d) if owned by user, temporarily change privileges to this user
       e) attempt unlink()
       f) if failed, warn about a possible race condition
       g) switch privileges back to root

     With the exception of step c, this is implemented in 'stmpclean'.
     Unfortunately, step c is crucial on systems that do not have
     restricted /tmp kernel patches from Openwall (http://www.openwall.com),
     otherwise, there is a potential for fooling the algorithm by supplying
     a hard link to a file owned by the victim, as discussed in section 3.
     This approach has several drawbacks - such as the fact root-owned files
     will not be removed. Other solution is to modify applications that
     generate filenames on their own, and to modify mkstemp(), to generate
     names that are not only unique, but not feasible to predict.

     Another suggestion is to implement a funlink() capability in the kernel
     of the operating system in question, to allow race-free file removal,
     thus removing the non-root ownership requirement for the method described
     above, and simplifying the approach. A skeleton patch to implement
     funlink() semantics and make sure the file being removed is the file
     opened and fstat()ed previously is available at:
     http://lcamtuf.coredump.cx/soft/linux-2.4-funlink.diff (this and
     other patches are not endorsed by RAZOR in any way).

  2) suspended process and 'legitimate' file removal

     This issue is fairly difficult to address. The most basic idea is
     to use a special naming scheme for temporary files to avoid deletion -
     unfortunately, this seems to defeat the purpose of using tmpwatch-alike
     solutions in the first place.

     An alternative approach, which is to enforce separate temporary
     directories for certain applications, either process-, session- or uid-
     based, is generally fairly controversial, and raises some concerns.
     Advisory separation is generally acceptable, but there are a number of
     applications that do not accept TMPDIR setting, and a widespread practice
     of using /tmp in in-house applications. Mandatory separation (kernel
     modification) raises compatibility concerns and is generally approached
     with skepticism - no implementation has become particularly popular.

  Ideally, implementators should carefully audit their sources. It is
  recommended for privileged applications to use private temporary
  directories for sensitive files, if possible; if using /tmp is necessary,
  extra caution has to be exercised to avoid referencing the file by name.
  Note that comparing the descriptor and a reopened file to verify inode
  numbers, creation times or file ownership is not sufficient - please refer
  to "Symlinks and Cryogenic Sleep" by Olaf Kirch, available at
  http://www.opennet.ru/base/audit/17.txt.html .

  It's worth noticing that 'tmpwatch' offers a -s option, which causes the
  program to run the 'fuser' command to prevent removal of files that are
  currently open. At first sight, this could be an effective way to solve the
  problem. Unfortunately, this is not true, since many applications close the
  file for a period of time before reopening (including logrotate and
  mktemp(1)).


5) Credits and thanks
---------------------

  Thanks to Solar Designer for interesting discussions on the subject,
  to Matt Power for useful feedback, and to RAZOR team in general for making
  this publication possible.