9/23/2004
Eric Low

In order to automatically transfer files to one of our clients on a nightly basis, I had to create a simple perl script to first fetch the file via HTTP and then deliver it via SSH.

First thing I did was download and install the correct modules. I installed the modules in the following order (and of course, if there were any other prerequisites that I didn't already have, I would have installed those as well):

If you want to install these modules via the .tar.gz like I did, simply extract the archive, then go into the directory and run the following commands:

Keep in mind that in order to use Math::Pari, you must first have PARI/GP installed on your system. This can be uptained from http://pari.math.u-bordeaux.fr/download.html and installed with the following commands:

You can also install these modules (MUCH more easily, too) by using cpan, a perl module and script that is most likely installed on your system by default. To invoke it, simply type cpan. The first time it is run, it will ask you a bunch of questions to set its configuration. Once you're in, you can install a module by typing install Module::Name. It's probably a good idea, the first time cpan is run, to type install Bundle::CPAN in order to update to the newest modules and add any that may be missing.

As it turned out, we needed to connect through Implicit SSL. I looked through the documentation for both Net::SFTP and Net::SSH::Perl, which Net::SFTP uses, but found no mention of this connection method at all. After a little bit of Googling, I ran across a perl module interface, WWW::Curl::easy, for the libcurl C library (obtained from http://curl.haxx.se/libcurl/).

Before I could install the perl module, I had to install curl and curl-devel, the latter of which is required for the Perl bindings. This required a lot of prequisites, which I installed in the following order:

After that was done, I installed the WWW::Curl perl module, which includes WWW::Curl::easy, from the .tar.gz.

After reading more, trying to figure out how to do implicit SSL with Curl, I found out that the SSL part of curl was really just thrown together. Furthermore, it appeared that maybe it didn't do Implicit SSL at all, but rather only support TLS/SSL, and did not encrypt the actual transmission of data. This is fundamental for our purposes, and to be honest, I did not know if SFTP over implicit SSL did this at all! (for the record, in the ftps man page, it says that in implicit SSL connections, all data connections are implicitly secure. It is in TLS/SSL where they are not).

I tried looking for yet another program that could handle SFTP with Implicit SSL - specifically, a command line utilitiy. I found an open source FTP server (with an available client) called BSDFTPD-SSL which can be obtained here (an alternative may also be LFTP, which can be obtained here). I download the binary RPM's and installed it. The command for the client is simply ftps (as opposed to sftp, which is BSD's FTP-over-SSH program that comes with OpenSSH (I think).) I typed man ftps to get help.

-z ssl

As it turned out, I ended up using a product called Secure FTP 2 (www.glub.com/products/secureftp/download.shtml), which is a command line ftps client that requires java. It is very good, and allows scripts to be easily run and logged.

This program requires Java 1.3 or above, which can be checked by typing java -version. I did not have Java installed, so I downloaded the self-extracting rpm from sun.com and installed it by typing ./jre-1_5_0-linux-i586-rpm.bin, which even ran rpm so that I didn't have to. Now that I had the newest version of Java, I installed Secure FTP 2 by typing ./secureftp2_cli_setup.bin and followed the prompts. I told it to set the prefix to /usr, which is a preference of mine. Java had also installed its executable to /usr/java/jre1.5.0/bin/java, so I had to tell it where that was located. Presto, Secure FTP 2 was installed! Note that I could not install SecureFTP2 at first because it could not run uncompress. This can be found in the ncompress RPM.

When you generate a secureftp2 script, you simply run it by typing /usr/secureftp/ftps.sh -script Australia.

log /programs/Australia/currdate.log
open 207.5.95.12
Name
password
PASV
send local-file [remote-file]
close
quit

I then set up a cron job to run every day at 4pm, which would run my script to automatically retrieve and send the file. I set up the cron job by typing crontab -e, and they entering the following line:

The 0 16 * * * tells the script to run every day at 4pm (1600 hours). The next part is the commands to run, which simply changes directories and then runs the script, sending all output to a log file.

It was just that simple. Enjoy!

More to come...