{ "content": [ { "type": "text", "text": "# safecat (man)\n\n## NAME\n\nsafecat - safely write data to a file\n\n## SYNOPSIS\n\nsafecat tempdir destdir\n\n## Sections\n\n- **NAME**\n- **SYNOPSIS**\n- **INTRODUCTION**\n- **RELIABILITY ISSUES**\n- **THE MAILDIR ALGORITHM**\n- **EXIT STATUS**\n- **SUGGESTED APPLICATIONS**\n- **EXAMPLES** (1 subsections)\n- **BUGS**\n- **CREDITS**\n- **SEE ALSO**\n\nUse structuredContent.sections for detailed options, examples, and full documentation.\n" } ], "structuredContent": { "command": "safecat", "section": "", "mode": "man", "summary": "safecat - safely write data to a file", "synopsis": "safecat tempdir destdir", "tldr_summary": null, "tldr_examples": [], "tldr_source": null, "flags": [], "examples": [ "To convince sendmail to use maildir for message delivery, add the following line to your", ".forward file:", "where SAFECAT is the complete path of the safecat program, HOME is the complete path to your", "home directory, and USERNAME is your login name. Making this change is likely to pay off;", "many campuses and companies mount user home directories with NFS. Using maildir to deliver", "to your inbox folder helps ensure that your mail will not be lost due to some NFS error. Of", "course, if you are a System Administrator, you should consider switching to qmail.", "To run a program and catch its output safely into some directory, you can use a shell script", "like the following.", "#!/bin/bash", "MYPROGRAM=cat # The program you want to run", "TEMPDIR=/tmp # The name of a temporary directory", "DESTDIR=$HOME/work/data # The directory for storing information", "try() { $* 2>/dev/null || echo NO 1>&2 }", "set `( try $MYPROGRAM | try safecat $TEMPDIR $DESTDIR ) 2>&1`", "test \"$?\" = \"0\" || exit -1", "test \"$1\" = \"NO\" && { rm -f $DESTDIR/$2; exit -1; }", "This script illustrates the pitfalls of writing secure programs with the shell. The script", "assumes that your program might generate some output, but then fail to complete. There is no", "way for safecat to know whether your program completed successfully or not, because of the", "semantics of the shell. As a result, safecat might create a file in the data directory which", "is \"complete\" but not useful. The shell script deletes the file in that case.", "More generally, the safest way to use safecat is from within a C program which invokes safe‐", "cat with fork() and execve(). The parent process can the simply kill() the safecat process", "if any problems develop, and optionally can try again. Whether to go to this trouble depends", "upon how serious you are about protecting your data. Either way, safecat will not be the", "weak link in your data flow." ], "see_also": [ { "name": "mbox", "section": "5", "url": "https://www.chedong.com/phpMan.php/man/mbox/5/json" }, { "name": "qmail-local", "section": "8", "url": "https://www.chedong.com/phpMan.php/man/qmail-local/8/json" }, { "name": "maildir", "section": "5", "url": "https://www.chedong.com/phpMan.php/man/maildir/5/json" } ], "section_outline": [ { "name": "NAME", "lines": 3, "subsections": [] }, { "name": "SYNOPSIS", "lines": 3, "subsections": [] }, { "name": "INTRODUCTION", "lines": 15, "subsections": [] }, { "name": "RELIABILITY ISSUES", "lines": 13, "subsections": [] }, { "name": "THE MAILDIR ALGORITHM", "lines": 15, "subsections": [] }, { "name": "EXIT STATUS", "lines": 11, "subsections": [] }, { "name": "SUGGESTED APPLICATIONS", "lines": 27, "subsections": [] }, { "name": "EXAMPLES", "lines": 3, "subsections": [ { "name": "|SAFECAT HOME/Maildir/tmp HOME/Maildir/new || exit 75 #USERNAME", "lines": 34 } ] }, { "name": "BUGS", "lines": 31, "subsections": [] }, { "name": "CREDITS", "lines": 13, "subsections": [] }, { "name": "SEE ALSO", "lines": 5, "subsections": [] } ], "sections": { "NAME": { "content": "safecat - safely write data to a file\n\n", "subsections": [] }, "SYNOPSIS": { "content": "safecat tempdir destdir\n\n", "subsections": [] }, "INTRODUCTION": { "content": "safecat is a program which implements Professor Daniel Bernstein's maildir algorithm to copy\nstdin safely to a file in a specified directory. With safecat, the user is offered two as‐\nsurances. First, if safecat returns a successful exit status, then all data is guaranteed to\nbe saved in the destination directory. Second, if a file exists in the destination direc‐\ntory, placed there by safecat, then the file is guaranteed to be complete.\n\nWhen saving data with safecat, the user specifies a destination directory, but not a file\nname. The file name is selected by safecat to ensure that no filename collisions occur, even\nif many safecat processes and other programs implementing the maildir algorithm are writing\nto the directory simultaneously. If particular filenames are desired, then the user should\nrename the file after safecat completes. In general, when spooling data with safecat, a sin‐\ngle, separate process should handle naming, collecting, and deleting these files. Examples\nof such a process are daemons, cron jobs, and mail readers.\n\n", "subsections": [] }, "RELIABILITY ISSUES": { "content": "A machine may crash while data is being written to disk. For many programs, including many\nmail delivery agents, this means that the data will be silently truncated. Using Professor\nBernstein's maildir algorithm, every file is guaranteed complete or nonexistent.\n\nMany people or programs may write data to a common \"spool\" directory. Systems like mh-mail\nstore files using numeric names in a directory. Incautious writing to files can result in a\ncollision, in which one write succeeds and the other appears to succeed but fails. Common\nstrategies to resolve this problem involve creation of lock files or other synchronizing\nmechanisms, but such mechanisms are subject to failure. Anyone who has deleted $HOME/.net‐\nscape/lock in order to start netscape can attest to this. The maildir algorithm is immune to\nthis problem because it uses no locks at all.\n\n", "subsections": [] }, "THE MAILDIR ALGORITHM": { "content": "As described in maildir(5), safecat applies the maildir algorithm by writing data in six\nsteps. First, it stat()s the two directories tempdir and destdir, and exits unless both di‐\nrectories exist and are writable. Second, it stat()s the name tempdir/time.pid.host, where\ntime is the number of seconds since the beginning of 1970 GMT, pid is the program's process\nID, and host is the host name. Third, if stat() returned anything other than ENOENT, the\nprogram sleeps for two seconds, updates time, and tries the stat() again, a limited number of\ntimes. Fourth, the program creates tempdir/time.pid.host. Fifth, the program NFS-writes the\nmessage to the file. Sixth, the program link()s the file to destdir/time.pid.host. At that\ninstant the data has been successfully written.\n\nIn addition, safecat starts a 24-hour timer before creating tempdir/time.pid.host, and aborts\nthe write if the timer expires. Upon error, timeout, or normal completion, safecat attempts\nto unlink() tempdir/time.pid.host.\n\n", "subsections": [] }, "EXIT STATUS": { "content": "An exit status of 0 (success) implies that all data has been safely committed to disk. A\nnon-zero exit status should be considered to mean failure, though there is an outside chance\nthat safecat wrote the data successfully, but didn't think so.\n\nNote again that if a file appears in the destination directory, then it is guaranteed to be\ncomplete.\n\nIf safecat completes successfully, then it will print the name of the newly created file\n(without its path) to standard output.\n\n", "subsections": [] }, "SUGGESTED APPLICATIONS": { "content": "Exciting uses for safecat abound, obviously, but a word may be in order to suggest what they\nare.\n\nIf you run Linux and use qmail instead of sendmail, you should consider converting your inbox\nto maildir for its superior reliability. If your home directory is NFS mounted, qmail forces\nyou to use maildir.\n\nIf you write CGI applications to collect data over the World Wide Web, you might find safecat\nuseful. Web applications suffer from two major problems. Their performance suffers from ev‐\nery stoppage or bottleneck in the internet; they cannot afford to introduce performance prob‐\nlems of their own. Additionally, web applications should NEVER leave the server and database\nin an inconsistent state. This is likely, however, if CGI scripts directly frob some data‐\nbase--particularly if the database is overloaded or slow. What happens when users get bored\nand click \"Stop\" or \"Back\"? Maybe the database activity completes. Maybe the CGI script is\nkilled, leaving the DB in an inconsistent state.\n\nConsider the following strategy. Make your CGI script dump its request to a spool directory\nusing safecat. Immediately return a receipt to the browser. Now the browser has a complete\nguarantee that their submission is received, and the perceived performance of your web appli‐\ncation is optimal.\n\nMeanwhile, a spooler daemon notices the fresh request, snatches it and updates the database.\nBrowsers can be informed that their request will be fulfilled in X minutes. The result is\noptimal performance despite a capricious internet. In addition, users can be offered nearly\n100% reliability.\n\n", "subsections": [] }, "EXAMPLES": { "content": "To convince sendmail to use maildir for message delivery, add the following line to your\n.forward file:\n", "subsections": [ { "name": "|SAFECAT HOME/Maildir/tmp HOME/Maildir/new || exit 75 #USERNAME", "content": "where SAFECAT is the complete path of the safecat program, HOME is the complete path to your\nhome directory, and USERNAME is your login name. Making this change is likely to pay off;\nmany campuses and companies mount user home directories with NFS. Using maildir to deliver\nto your inbox folder helps ensure that your mail will not be lost due to some NFS error. Of\ncourse, if you are a System Administrator, you should consider switching to qmail.\n\nTo run a program and catch its output safely into some directory, you can use a shell script\nlike the following.\n\n#!/bin/bash\n\nMYPROGRAM=cat # The program you want to run\nTEMPDIR=/tmp # The name of a temporary directory\nDESTDIR=$HOME/work/data # The directory for storing information\n\ntry() { $* 2>/dev/null || echo NO 1>&2 }\n\nset `( try $MYPROGRAM | try safecat $TEMPDIR $DESTDIR ) 2>&1`\ntest \"$?\" = \"0\" || exit -1\ntest \"$1\" = \"NO\" && { rm -f $DESTDIR/$2; exit -1; }\n\nThis script illustrates the pitfalls of writing secure programs with the shell. The script\nassumes that your program might generate some output, but then fail to complete. There is no\nway for safecat to know whether your program completed successfully or not, because of the\nsemantics of the shell. As a result, safecat might create a file in the data directory which\nis \"complete\" but not useful. The shell script deletes the file in that case.\n\nMore generally, the safest way to use safecat is from within a C program which invokes safe‐\ncat with fork() and execve(). The parent process can the simply kill() the safecat process\nif any problems develop, and optionally can try again. Whether to go to this trouble depends\nupon how serious you are about protecting your data. Either way, safecat will not be the\nweak link in your data flow.\n\n" } ] }, "BUGS": { "content": "In order to perform the last step and link() the temporary file into the destination direc‐\ntory, both directories must reside in the same file system. If they do not, safecat will\nquietly fail every time. In Professor Bernstein's implementation of maildir, the temporary\nand destination directories are required to belong to the same parent directory, which essen‐\ntially avoids this problem. We relax this requirement to provide some flexibility, at the\ncost of some risk. Caveat emptor.\n\nAlthough safecat cleans up after itself, it may sometimes fail to delete the temporary file\nlocated in tempdir. Since safecat times out after 24 hours, you may freely delete any tempo‐\nrary files older than 36 hours. Files newer than 36 hours should be left alone. A system of\ndata flow involving safecat should include a cron job to clean up temporary files, or should\nobligate consumers of the data to do the cleanup, or both. In the case of qmail, mail read‐\ners using maildir are expected to scan and clean up the temporary directory.\n\nThe guarantee of safe delivery of data is only \"as certain as UNIX will allow.\" In particu‐\nlar, a disk hardware failure could result in safecat concluding that the data was safe, when\nit was not. Similarly, a successful exit status from safecat is of no value if the computer,\nits disks and backups all explode at some subsequent time.\n\nIn other words, if your data is vital to you, then you won't just use safecat. You'll also\ninvest in good equipment (possibly including a RAID disk), a UPS for the server and drives, a\nregular backup schedule, and competent system administration. For many purposes, however,\nsafecat can be considered 100% reliable.\n\nAlso note that safecat was designed for spooling email messages; it is not the right tool for\nspooling large files--files larger than 2GB, for example. Some operating systems have a bug\nwhich causes safecat to fail silently when spooling files larger than 2GB. When building\nsafecat, you can take advantage of conditional support for large files on Linux; see conf-cc\nfor further information.\n\n", "subsections": [] }, "CREDITS": { "content": "The maildir algorithm was devised by Professor Daniel Bernstein, the author of qmail. Parts\nof this manpage borrow directly from maildir(5) by Professor Bernstein. In particular, the\nsection \"THE MAILDIR ALGORITHM\" transplants his explanation of the maildir algorithm in order\nto illustrate that safecat complies with it.\n\nThe original code for safecat was written by the present author, but was since augmented with\nheavy borrowings from qmail code. However, under no circumstances should the author of qmail\nbe contacted concerning safecat bugs; all are the fault, and the responsibility, of the\npresent author.\n\nCopyright (c) 2000, Len Budney. All rights reserved.\n\n", "subsections": [] }, "SEE ALSO": { "content": "mbox(5), qmail-local(8), maildir(5)\n\n\n\nsafecat(1)", "subsections": [] } } } }