# phpman > man > safecat(1) [safecat(1)](https://www.chedong.com/phpMan.php/man/safecat/1/markdown) General Commands Manual [safecat(1)](https://www.chedong.com/phpMan.php/man/safecat/1/markdown) ## NAME safecat - safely write data to a file ## SYNOPSIS **safecat** _tempdir_ _destdir_ ## INTRODUCTION _safecat_ is a program which implements Professor Daniel Bernstein's _maildir_ algorithm to copy _stdin_ safely to a file in a specified directory. With _safecat_, the user is offered two as‐ surances. First, if _safecat_ returns a successful exit status, then all data is guaranteed to be saved in the destination directory. Second, if a file exists in the destination direc‐ tory, placed there by _safecat_, then the file is guaranteed to be complete. When saving data with _safecat_, the user specifies a destination directory, but not a file name. The file name is selected by _safecat_ to ensure that no filename collisions occur, even if many _safecat_ processes and other programs implementing the _maildir_ algorithm are writing to the directory simultaneously. If particular filenames are desired, then the user should rename the file after _safecat_ completes. In general, when spooling data with _safecat_, a sin‐ gle, separate process should handle naming, collecting, and deleting these files. Examples of such a process are daemons, cron jobs, and mail readers. ## RELIABILITY ISSUES A machine may crash while data is being written to disk. For many programs, including many mail delivery agents, this means that the data will be silently truncated. Using Professor Bernstein's _maildir_ algorithm, every file is guaranteed complete or nonexistent. Many people or programs may write data to a common "spool" directory. Systems like _mh-mail_ store files using numeric names in a directory. Incautious writing to files can result in a collision, in which one write succeeds and the other appears to succeed but fails. Common strategies to resolve this problem involve creation of lock files or other synchronizing mechanisms, but such mechanisms are subject to failure. Anyone who has deleted $HOME/.net‐ scape/lock in order to start netscape can attest to this. The _maildir_ algorithm is immune to this problem because it uses no locks at all. ## THE MAILDIR ALGORITHM As described in [maildir(5)](https://www.chedong.com/phpMan.php/man/maildir/5/markdown), _safecat_ applies the _maildir_ algorithm by writing data in six steps. First, it **stat()s** the two directories _tempdir_ and _destdir_, and exits unless both di‐ rectories exist and are writable. Second, it **stat()s** the name **tempdir/**_time.pid.host_, where _time_ is the number of seconds since the beginning of 1970 GMT, _pid_ is the program's process ID, and _host_ is the host name. Third, if **stat()** returned anything other than ENOENT, the program sleeps for two seconds, updates _time_, and tries the **stat()** again, a limited number of times. Fourth, the program creates **tempdir/**_time.pid.host_. Fifth, the program _NFS-writes_ the message to the file. Sixth, the program **link()**s the file to **destdir/**_time.pid.host_. At that instant the data has been successfully written. In addition, _safecat_ starts a 24-hour timer before creating **tempdir/**_time.pid.host_, and aborts the write if the timer expires. Upon error, timeout, or normal completion, _safecat_ attempts to **unlink()** **tempdir/**_time.pid.host_. ## EXIT STATUS An exit status of 0 (success) implies that all data has been safely committed to disk. A non-zero exit status should be considered to mean failure, though there is an outside chance that _safecat_ wrote the data successfully, but didn't think so. Note again that if a file appears in the destination directory, then it is guaranteed to be complete. If _safecat_ completes successfully, then it will print the name of the newly created file (without its path) to standard output. ## SUGGESTED APPLICATIONS Exciting uses for _safecat_ abound, obviously, but a word may be in order to suggest what they are. If you run Linux and use qmail instead of sendmail, you should consider converting your inbox to _maildir_ for its superior reliability. If your home directory is NFS mounted, qmail forces you to use _maildir_. If you write CGI applications to collect data over the World Wide Web, you might find _safecat_ useful. Web applications suffer from two major problems. Their performance suffers from ev‐ ery stoppage or bottleneck in the internet; they cannot afford to introduce performance prob‐ lems of their own. Additionally, web applications should NEVER leave the server and database in an inconsistent state. This is likely, however, if CGI scripts directly frob some data‐ base--particularly if the database is overloaded or slow. What happens when users get bored and click "Stop" or "Back"? Maybe the database activity completes. Maybe the CGI script is killed, leaving the DB in an inconsistent state. Consider the following strategy. Make your CGI script dump its request to a spool directory using _safecat_. Immediately return a receipt to the browser. Now the browser has a complete guarantee that their submission is received, and the perceived performance of your web appli‐ cation is optimal. Meanwhile, a spooler daemon notices the fresh request, snatches it and updates the database. Browsers can be informed that their request will be fulfilled in X minutes. The result is optimal performance despite a capricious internet. In addition, users can be offered nearly 100% reliability. ## EXAMPLES To convince sendmail to use _maildir_ for message delivery, add the following line to your .forward file: ### |SAFECAT HOME/Maildir/tmp HOME/Maildir/new || exit 75 #USERNAME where **SAFECAT** is the complete path of the _safecat_ program, **HOME** is the complete path to your home directory, and **USERNAME** is your login name. Making this change is likely to pay off; many campuses and companies mount user home directories with NFS. Using _maildir_ to deliver to your inbox folder helps ensure that your mail will not be lost due to some NFS error. Of course, if you are a System Administrator, you should consider switching to qmail. To run a program and catch its output safely into some directory, you can use a shell script like the following. #!/bin/bash MYPROGRAM=cat # The program you want to run TEMPDIR=/tmp # The name of a temporary directory DESTDIR=$HOME/work/data # The directory for storing information try() { $* 2>/dev/null || echo NO 1>&2 } set `( try $MYPROGRAM | try safecat $TEMPDIR $DESTDIR ) 2>&1` test "$?" = "0" || exit -1 test "$1" = "NO" && { rm -f $DESTDIR/$2; exit -1; } This script illustrates the pitfalls of writing secure programs with the shell. The script assumes that your program might generate some output, but then fail to complete. There is no way for _safecat_ to know whether your program completed successfully or not, because of the semantics of the shell. As a result, safecat might create a file in the data directory which is "complete" but not useful. The shell script deletes the file in that case. More generally, the safest way to use _safecat_ is from within a C program which invokes safe‐ cat with _fork()_ and _execve()_. The parent process can the simply _kill()_ the _safecat_ process if any problems develop, and optionally can try again. Whether to go to this trouble depends upon how serious you are about protecting your data. Either way, _safecat_ will not be the weak link in your data flow. ## BUGS In order to perform the last step and _link()_ the temporary file into the destination direc‐ tory, both directories must reside in the same file system. If they do not, _safecat_ will quietly fail every time. In Professor Bernstein's implementation of _maildir_, the temporary and destination directories are required to belong to the same parent directory, which essen‐ tially avoids this problem. We relax this requirement to provide some flexibility, at the cost of some risk. Caveat emptor. Although _safecat_ cleans up after itself, it may sometimes fail to delete the temporary file located in _tempdir_. Since safecat times out after 24 hours, you may freely delete any tempo‐ rary files older than 36 hours. Files newer than 36 hours should be left alone. A system of data flow involving safecat should include a cron job to clean up temporary files, or should obligate consumers of the data to do the cleanup, or both. In the case of qmail, mail read‐ ers using _maildir_ are expected to scan and clean up the temporary directory. The guarantee of safe delivery of data is only "as certain as UNIX will allow." In particu‐ lar, a disk hardware failure could result in _safecat_ concluding that the data was safe, when it was not. Similarly, a successful exit status from _safecat_ is of no value if the computer, its disks and backups all explode at some subsequent time. In other words, if your data is vital to you, then you won't just use _safecat_. You'll also invest in good equipment (possibly including a RAID disk), a UPS for the server and drives, a regular backup schedule, and competent system administration. For many purposes, however, _safecat_ can be considered 100% reliable. Also note that _safecat_ was designed for spooling email messages; it is not the right tool for spooling large files--files larger than 2GB, for example. Some operating systems have a bug which causes safecat to fail silently when spooling files larger than 2GB. When building _safecat_, you can take advantage of conditional support for large files on Linux; see _conf-cc_ for further information. ## CREDITS The _maildir_ algorithm was devised by Professor Daniel Bernstein, the author of qmail. Parts of this manpage borrow directly from [maildir(5)](https://www.chedong.com/phpMan.php/man/maildir/5/markdown) by Professor Bernstein. In particular, the section "THE MAILDIR ALGORITHM" transplants his explanation of the _maildir_ algorithm in order to illustrate that _safecat_ complies with it. The original code for _safecat_ was written by the present author, but was since augmented with heavy borrowings from qmail code. However, under no circumstances should the author of qmail be contacted concerning safecat bugs; all are the fault, and the responsibility, of the present author. Copyright (c) 2000, Len Budney. All rights reserved. ## SEE ALSO [mbox(5)](https://www.chedong.com/phpMan.php/man/mbox/5/markdown), [qmail-local(8)](https://www.chedong.com/phpMan.php/man/qmail-local/8/markdown), [maildir(5)](https://www.chedong.com/phpMan.php/man/maildir/5/markdown) [safecat(1)](https://www.chedong.com/phpMan.php/man/safecat/1/markdown)