# phpman > man > ncftpspooler(1) [ncftpspooler(1)](https://www.chedong.com/phpMan.php/man/ncftpspooler/1/markdown) General Commands Manual [ncftpspooler(1)](https://www.chedong.com/phpMan.php/man/ncftpspooler/1/markdown) ## NAME ncftpspooler - Global batch FTP job processor daemon ## SYNOPSIS **ncftpspooler** -d [_options_] **ncftpspooler** -l [_options_] ## OPTIONS ### Command line flags: ### -d ### -q fault directory, /var/spool/ncftp. ### -o rather inappropriately) the program simply uses a file called _log_ in the job queue directory. If you don't want a log, use this option to specify /dev/null. ### -l ### -s see if a new job has been submitted. Use this option to change the number of seconds used for this delay. ## DESCRIPTION The _ncftpspooler_ program evolved from the _ncftpbatch_ program. The _ncftpbatch_ program was originally designed as a ``personal FTP spooler'' which would process a single background job a particular user and exit when it finished; the _ncftpspooler_ program is a ``global FTP spooler'' which stays running and processes background jobs as they are submitted. The job queue directory is monitored for specially-named and formatted text files. Each file serves as a single FTP job. The name of the job file contains the type of FTP job (_get_ or _put_), a timestamp indicating the earliest the job should be processed, and optionally some additional information to make it easier to create unique job files (i.e. a sequence number). The contents of the job files have information such as the remote server machine to FTP to, username, password, remote pathname, etc. Your job queue directory must be readable and writable by the user that you plan to run _ncftpspooler_ as, so that jobs can be removed or renamed within the queue. More importantly, the user that is running the program will need adequate privileges to ac‐ cess the local files that are involved in the FTPing. I.e., if your spooler is going to be processing jobs which upload files to remote servers, then the user will need read permission on the local files that will be uploaded (and directory access permission the parent directo‐ ries). Likewise, if your spooler is going to be processing jobs which download files, then the user would need to be able to write to the local directories. Once you have created your spool directory with appropriate permissions and ownerships, you can run _ncftpspooler_ _-d_ to launch the spooler daemon. You can run additional spoolers if you want to process more than FTP job from the same job queue directory simultaneously. You can then monitor the log file (i.e., using _tail_ _-f_ ) to track the progress of the spooler. Most of the time it won't be doing anything, unless job files have appeared in the job queue di‐ rectory. ## JOB FILE NAMES When the _ncftpspooler_ program monitors the job queue directory, it ignores any files that do not follow the naming convention for job files. The job files must be prefixed in the format of _X-YYYYMMDD-hhmmss_ where _X_ denotes a job type, _YYYY_ is the four-digit year, _MM_ is the two- digit month number, _DD_ is the two-digit day of the month, _hh_ is the two-digit hour of the day (00-23), _mm_ is the two-digit minute, and _ss_ is the two-digit second. The date and time rep‐ resent the earliest time you want the job to be run. The job type can be _g_ for a get (download from remote host), or _p_ for aput (upload to remote host). As an example, if you wanted to schedule an upload to occur at 11:45 PM on December 7, 2001, a job file could be named p-20011207-234500 In practice, the job files include additional information such as a sequence number or process ID. This makes it easier to create unique job file names. Here is the same example, with a process ID and a sequence number: p-20011207-234500-1234-2 When submitting job files to the queue directory, be sure to use a dash character after the _hhmmss_ field if you choose to append any additional data to the job file name. ## JOB FILE CONTENTS Job files are ordinary text files, so that they can be created by hand. Each line of the file is a key-pair in the format _variable_=_value_, or is a comment line beginning with an oc‐ tothorpe character (_#_), or is a blank line. Here is an example job file: # This is a NcFTP spool file entry. job-name=g-20011016-100656-008299-1 op=get hostname=ftp.freebsd.org xtype=I passive=1 remote-dir=pub/FreeBSD local-dir=/tmp remote-file=README.TXT local-file=readme.txt Job files are flexible since they follow an easy-to-use format and do not have many require‐ ments, but there are a few mandatory parameters that must appear for the spooler to be able to process the job. _op_ The operation (job type) to perform. Valid values are _get_ and _put_. _hostname_ The remote host to FTP to. This may be an IP address or a DNS name (i.e. ftp.exam‐ ple.com). For a regular _get_ job, these parameters are required: _remote-file_ The pathname of the file to download from the remote server. _local-file_ The pathname to use on the local server for the downloaded file. For a regular _put_ job, these parameters are required: _local-file_ The pathname of the file to upload to the remote server. _remote-file_ The pathname to use on the remote server for the uploaded file. For a recursive _get_ job, these parameters are required: _remote-file_ The pathname of the file or directory to download from the remote server. _local-dir_ The directory pathname to use on the local server to contain the downloaded items. For a recursive _put_ job, these parameters are required: _local-file_ The pathname of the file or directory to upload to the remote server. _remote-dir_ The directory pathname to use on the remote server to contain the uploaded items. The rest of the parameters are optional. The spooler will attempt to use reasonable defaults for these parameters if necessary. _user_ The username to use to login to the remote server. Defaults to ``anonymous'' for guest access. _pass_ The password to use in conjunction with the username to login to the remote server. _acct_ The account to use in conjunction with the username to login to the remote server. The need to specify this parameter is extremely rare. _port_ The port number to use in conjunction with the remote hostname to connect to the re‐ mote server. Defaults to the standard FTP port number, 21. _host-ip_ The IP address to use in conjunction with the remote hostname to connect to the re‐ mote server. This parameter can be used in place of the _hostname_ parameter, but one or the other must be used. This parameter is commonly included along with the _host__‐ _name_ parameter as supplemental information. _xtype_ The transfer type to use. Defaults to binary transfer type (TYPE I). Valid values are _I_ for binary, _A_ for ASCII text. _passive_ Whether to use FTP passive data connections (PASV) or FTP active data connections (PORT). Valid values are _0_ for active, _1_ for passive, or _2_ to try passive, then fallback to active. The default is _2_. _recursive_ This can be used to transfer entire directory trees. By default, only a single file is transferred. Valid values are _yes_ or _no_. _delete_ This can be used to delete the source file on the source machine after successfully transferring the file to the destination machine. By default, source files are not deleted. Valid values are _yes_ or _no_. _job-name_ This isn't used by the program, but can be used by an entity which is automatically generating job files. As an example, when using the _-bbb_ flag with _ncftpput_, it cre‐ ates a job file on stdout with a _job-name_ parameter so you can easily copy the file to the job queue directory with the suggested job name as the job file name. _pre-ftp-command_ _post-ftp-command_ These parameters correspond to the _-W_, and _-Y_ options of _ncftpget_ and _ncftpput_. It is important to note that these refer to RFC959 File Transfer Protocol commands and **not** shell commands, nor commands used from within /usr/bin/ftp or ncftp. _pre-shell-command_ _post-shell-command_ These parameters provide hooks so you can run a custom program when an item is pro‐ cessed by the spooler. Valid values are pathnames to scripts or executable programs. Note that the value must not contain any command-line arguments -- if you want to do that, create a shell script and have it run your program with the command-line argu‐ ments it requires. Generally speaking, _post-shell-command_ is much more useful than _pre-shell-command_ since if you need to use these options you're more likely to want to do something after the FTP trans‐ fer has completed rather than before. For example, you might want to run a shell script which pages an administrator to notify her that her 37 gigabyte file download has completed. When your custom program is run, it receives on standard input the contents of the job file (i.e. several lines of _variable_=_value_ key-pairs), as well as additional data the spooler may provide, such as a _result_ key-pair with a textual description of the job's completion status. _post-shell-command_ update a log file named /var/log/ncftp_spooler. #!/usr/bin/perl -w my ($line); my (%params) = (); while (defined($line = )) { $params{$1} = $2 if ($line =~ /^([^=\#\s]+)=(.*)/); } if ((defined($params{"result"})) && ($params{"result"} =~ /^Succeeded/)) { open(LOG, ">> /var/log/ncftp_spooler.log") or [exit(1)](https://www.chedong.com/phpMan.php/man/exit/1/markdown); print LOG "DOWNLOAD" if ($params{"op"} eq "get"); print LOG "UPLOAD" if ($params{"op"} eq "put"); print LOG " ", $params{"local-file"}, "\n"; close(LOG); } ## DIAGNOSTICS The log file should be examined to determine if any _ncftpspooler_ processes are actively work‐ ing on jobs. The log contains copious amounts of useful information, including the entire FTP control connection conversation between the FTP client and server. ## BUGS The _recursive_ option may not be reliable since _ncftpspooler_ depends on functionality which may or may not be present in the remote server software. Additionally, even if the function‐ ality is available, _ncftpspooler_ may need to use heuristics which cannot be considered 100% accurate. Therefore it is best to create individual jobs for each file in the directory tree, rather than a single recursive directory job. For resumption of downloads to work, the remote server must support the FTP _SIZE_ and _MDTM_ primitives. Most modern FTP server software can do this, but there are still a number of bare-bones _ftpd_ implementations which do not. In these cases, _ncftpspooler_ will re-download the file in entirety each time until the download succeeds. The program needs to be improved to detect jobs that have no chance of ever completing suc‐ cessfully. There are still a number of cases where jobs can get spooled but get retried over and over again until a vigilant sysadmin manually removes the jobs. The spool files may contain usernames and passwords stored in cleartext. These files should not be readable by any user except the user running the program! ## AUTHOR Mike Gleason, NcFTP Software (). ## SEE ALSO [_ncftpbatch(1)](https://www.chedong.com/phpMan.php/man/ncftpbatch/1/markdown),_ [_ncftp_(1)](https://www.chedong.com/phpMan.php/man/ncftp/1/markdown), [_ncftpput_(1)](https://www.chedong.com/phpMan.php/man/ncftpput/1/markdown), [_ncftpget_(1)](https://www.chedong.com/phpMan.php/man/ncftpget/1/markdown), [_uucp_(1)](https://www.chedong.com/phpMan.php/man/uucp/1/markdown). ncftpspooler NcFTP Software [ncftpspooler(1)](https://www.chedong.com/phpMan.php/man/ncftpspooler/1/markdown)