Snort Intrusion Detection (IDS)

4/13/2006
Eric Low

Snort is the leading Open Source IDS, providing valuable logs of malicious activity across your firewall.

First, download and extract Snort. You must also have libpcap and libpcre, so install those if needed.

Now, install Snort. I used the following commands:

./configure --prefix=/usr --mandir=/usr/share/man
make
make check
make install

Then, create a directory called /etc/snort. Copy all files from the archive's etc directory to this directory. When you do start Snort, it will look for all of these configuration files, in the same directory that snort.conf is located in. So keep them together! Edit the configuration file, /etc/snort/snort.conf, to reflect your network setup. This file is very well commented, so it should be pretty straightforward. In the conf file, I set my rules files path to /etc/snort/rules (var RULE_PATH /etc/snort/rules), so I created that directory for it to use.

In order to download any of the recent rulesets, you will have to register at the Snort site. Once you have the account set up, go to the download rules page and download "Sourcefire VRT Certified Rules - The Official Snort Rules (registered user release)." These are labeled, "VRT Certified Rules for Snort CURRENT." I then extracted this archive into my /etc/snort directory. The archive contains two directories, doc and rules, so extracting to /etc/snort put my rules in the /etc/snort/rules directory as I specified in my snort.conf file.

 

Logging options can be defined either in snort.conf or at the command line, although there are less options available at the command line. For both security and convenience, we are logging in two places. The first is on a remote logserver (through syslog), so that we can analyze the logs with snortsnarf and serve them on an HTTP server for easy viewing. To do this, add the following line to snort.conf:

output alert_syslog: LOG_AUTH LOG_ALERT LOG_PID

Note that the LOG_PID option is important for SnortSnarf to work correctly.

The second method of logging is in ascii files in the /var/log/snort directory, which is the default. You must create this directory. This also produces packet dumps of any packets that triggered alerts. These are stored in subdirectories, named according to their source IP address, off of /var/log/snort. To turn this option on, add the following line to snort.conf:

output alert_full: /var/log/snort/alert
(or, simply put output alert_full: alert)

Keep in mind that this slows down snort considerably, and could actually cause some packets to be missed. If this becomes a problem, use "alert_fast" instead, which doesn't print all of the packet headers.

 

Snort can now be run by typing snort -D -d -e -i bridge -c /etc/snort.conf, in which the "-c" flag tells it where to find the configuration file. The "-D" option tells it to run as a daemon (background process), the "-d" option tells it to display the packet data (not just the headers), while "-e" tells it to also log the data link layer headers. To make it log to syslog, run with the "-s" option as well. There is also an implied switch, "-A full" that tells it to log in a more detailed format (the -s option, however, may override this and set it to "-A fast"). This should be started upon bootup at the end of the rc.local script file.

For some reason, when I left off the -i option for our bridge interface (which made it default to our external interface, eth0), it seemed to cause a lot of problems with false alarms - something to do with the interface not having an IP address, it seems. These were false alarms coming from our firewall IP to external that should have been ignored. Once I added the interface, which does have the IP, these went away.

 

To help alleviate high levels of false alarms:

For the most part, the default Snort rules (as specified at the end of snort.conf) are used. Another variable, WEBONLY_SERVERS, should be created in snort.conf. The following rules' destination should be changed from $HTTP_SERVERS to $WEBONLY_SERVERS to help eliminate numerous false alarms caused by traffic to Outlook Web Access:
In web-iis.rules file: WEB-IIS view source via translate header
In web-cgi.rules file: WEB-CGI calendar access
In web-misc.rules file: WEB-MISC webdav search access

 

The following rules' destination should be changed from $HOME_NET to $HTTP_SERVERS in order to prevent numerous false alarms:
In dos.rules file: DOS MSDTC attempt

 

In order to prevent high frequencies of false alarms resulting from BOTH the web and email server, the following lines should be commented out:
In web-iis.rules file: WEB-IIS ISAPI .ida attempt
In web-misc.rules file: WEB-MISC cat%20 access (this only affects systems running the Irix OS)
In experimental.rules file: EXPERIMENTAL WEB-CLIENT javascript URL host spoofing attempt

 

=============================================================================================
=============================================================================================

 

Installing Snort, for use with Base

I heavily referenced this document for the following procedures.

First, Base requires Snort to use a database for its logging. If it is not already, install MySQL, and compile it to run with OpenSSL. I did not investigate this, but I'm assuming that you actually have to compile it to do this, rather than installing an RPM. I also set it to build without libgcc and zlib. I downloaded the source package (it's way down at the bottom of the downloads page, surrounded by big warnings), then extracted it and then installed it with the following commands:

LDFLAGS="-R/usr/lib" ./configure --prefix=/usr --mandir=/usr/share/man --localstatedir=/var --with-openssl --without-docs --without-libgcc --with-named-z-libs=z
make
make install

------------------------------------

Because I had originally compiled Snort (version 2.4.4) without MySQL support, I had to recompile it (actually, I had to compile it against OpenSSL as well). To ensure I had a clean copy, I removed the old directory that I compiled Snort from and unarchived a new copy. I then recompiled and installed it with the following commands:

LDFLAGS="-L/usr/lib/mysql" ./configure --prefix=/usr --mandir=/usr/share/man --localstatedir=/var --with-mysql=/usr --with-openssl=/usr
make
make check
make install

Note that I had to stick that LDFLAGS environment variable in there to avoid a "ld: cannot find -lmysqlclient" error that I kept getting (although I can't remember if it was while I was running configure or actually compiling with make). This was due to the fact that I had kept the mysql-client rpm on there, and compiled the mysql server package seperately - so libraries were in two different locations (mysql-client libraries were in /usr/lib/mysql while mysql libraries were in /usr/include/mysql).

In fact, when I finally did try starting snort with mysql support, I got the following error: snort: error while loading shared libraries: libmysqlclient.so.15: cannot open shared object file: no such file or directory. To fix this, I added a line to /etc/ld.so.conf that read /usr/lib/mysql and then ran ldconfig. Snort stopped complaining and started running.

------------------------------------

Before I could run MySQL, I had to set up an unprivileged user:

groupadd -r mysqld
useradd -g mysqld -r -c 'MySQL User' mysqld
passwd mysqld
mkdir /usr/mysql
mkdir /usr/mysql/data
chown -R mysqld:mysqld /usr/mysql

I then copied a default config file to the /etc directory for the MySQL daemon to use by typing cp support-files/my-small.cnf /etc/my.cnf. I modified this file, uncommenting all of the lines regarding InnoDB, since I definitely wanted to use InnoDB tables. I then changed the following lines to make the MySQL daemon always start with my newly created user:

[mysqld]
datadir = /usr/mysql/data/
user = mysqld
pid-file = /usr/mysql/mysqld.pid
innodb_data_home_dir = /usr/mysql/data/
innodb_log_group_home_dir = /usr/mysql/data/
innodb_log_arch_dir = /usr/mysql/data/

MySQL also needs a default table named mysql installed to start. I created this by running the following script:

scripts/mysql_install_db --user=mysqld

I tried starting mysqld with its output sent to the screen, so I could see what was going on:

/usr/libexec/mysqld --console

It looked good, so I changed the password for MySQL's root user (this password is blank by default), as well as deleted any anonymous accounts:

# mysql -u root mysql
> SET PASSWORD FOR root@localhost=PASSWORD('newpassword');
> DELETE FROM mysql.user WHERE User='';
> FLUSH PRIVILEGES;
> EXIT

Check to make sure that the root user does not exist in any other host context as well! Type SELECT Host,User FROM mysql.user; and change the password for any other root user as well!

Now, I created the database for Snort:

mysqladmin -u root -p create snort

The appropriate tables for snort must then be created; Snort provides a script to do this with the following command (run from my unarchived snort directory):

mysql -u root -p <schemas/create_mysql snort

The snort user must also be added, and permissions set for its database and tables:

# mysql -u root -p snort
> grant CREATE,INSERT,SELECT,DELETE,UPDATE on snort.* to snort@localhost;
> SET PASSWORD FOR snort@localhost=PASSWORD('snortuserpass');
> FLUSH PRIVILEGES;
> EXIT

I then copied the included startup script to my startup script directory (note that I am on SuSE and therefore all the scripts are in /etc/rc.d/ rather than /etc/rc.d/init.d/):

cp support-files/mysql.server /etc/rc.d/mysqld
chmod 755 /etc/rc.d/mysqld

Next, I modified my /etc/snort/snort.conf file to output both log and alert data to MySQL:

output database: log, mysql, user=snort password=snortuserpass dbname=snort host=localhost

I also created a Snort user, using the same commands that I used to create the mysqld user, and made sure the config files were only readable or writeable by this user:

groupadd -r mysqld
useradd -g mysqld -r -c 'MySQL User' mysqld
passwd mysqld
chown -R snort:snort /etc/snort/*
chmod -R 600 /etc/snort/*

The rest of the instructions for configuring Snort, getting rules, etc. are in my section up above, installing plain old Snort.

------------------------------------

PHP will require the gd libraries (and be compiled with them) once you want to start adding image extensions, so I had to install those first. I downloaded the source (version 2.0.33), unarchived it, and installed it with the following commands:

ln -sv /usr/lib/libjpeg.so.6 /usr/lib/libjpeg.so
ln -sv /usr/lib/libpng.so.3.1.2.6 /usr/lib/libpng.so
./configure --prefix=/usr --mandir=/usr/share/man
make
make install

I had to create those symbolic links in there so that PHP wouldn't complain when I tried configuring it with the --with-gd= option.

------------------------------------

Now, I had to install PHP. I downloaded and unarchived the source code (version 5.1.2), then installed it with the following commands:

LDFLAGS="-L/usr/lib/mysql" ./configure --prefix=/usr/local/php --mandir=/usr/share/man --enable-memory-limit=yes --with-apxs2=/usr/bin/apxs --with-gettext=/usr/bin/gettext --with-exif --without-mm --with-mysql=/usr --with-openssl=/usr --with-zlib --with-jpeg-dir=/usr --with-png-dir=/usr --with-exec-dir=/usr/local/php/libexec --enable-cli --enable-sockets --with-gd=/usr --with-jpeg-dir=/usr --with-png-dir=/usr
make
make install

I'm not entirely sure why I used the --prefix=/usr/local/php rather than my usual --prefix=/usr option... I figured there must've been a pretty good reason why the author of the previous document didn't simply use /usr/local, so... I followed suit!

I did get the following error on my first try running configure: "configure: error: xml2-config not found. Please check your libxml2 installation." I installed libxml2-devel, and that was solved.

I also originally got an error that read "configure: error: mysql configure failed. Please check config.log for more information." This is why I ended up sticking that -L/usr/lib/mysql into the LDFLAGS variable. Upon running make, I once again got the following error about finding mysqlclient:

/usr/lib/gcc-lib/i586-suse-linux/3.3.4/../../../../i586-suse-linux/bin/ld: cannot find -lmysqlclient
collect2: ld returned 1 exit status
make: *** [libphp5.la] Error 1

I tried everything to get it to compile - modifying /etc/ld.so.conf (along with running ldconfig) was no use, and the options specified with ./configure would work either (including changing the directory on --with-mysql=). What finally worked was to specify LDFLAGS while running make! Here is my final make line:

LDFLAGS="-L/usr/lib/mysql" make

When specifying the --with-png-dir option (this is required for --with-gd), I kept getting the following error: "configure: error: png.h not found." I already had libpng (the RPM) installed, but it did not include png.h! I downloaded the libpng .tar.gz source, then installed it OVER the top of my RPM. I did not want to uninstall the RPM, seeing as it had ten million dependencies, and I knew that YaST would throw a fit next time I used it to install software. I installed with the following commands:

./configure --prefix=/usr --mandir=/usr/share/man
make
make install

This overwrote all my existing libpng library files, and dumped png.h into the /usr/include directory.


Finally, it compiled!

Once PHP was installed, I had to put a php.ini file in place. PHP looks for this file in the following locations, in order:

1. The directory from which the PHP script was called
2. The root of your Web directory (typically public_html)
3. The Web server's default php.ini

I took one of the default ini files and copied it /etc/php.ini, hoping for the best (other docs have mentioned copying it to the /usr/local/php/lib/php.ini for a similar install to mine). As it turned out, both /etc/php.ini or /usr/local/php/lib/php.ini worked for me. So take your pick!:

cp php.ini-recommended /etc/php.ini

I also added the following line to that file:

include_path = ".:/usr/local/php/lib/php"

You may also need to useapxs to add the libphp5.so modules to Apache. I did not have to do so in my installation, because for some reason, "LoadModule php5_module modules/libphp5.so" was already in my httpd.conf file (I don't know if it was always there or what, but I never added it). Anyhow, if you have to do so, run the following commands (I *think*):

/usr/bin/apxs -i -a /usr/modules/libphp5.so
apachectl restart

 

------------------------------------

BASE also requires ADOdb for talking to MySQL on the backend. Installation is simple - I simply downloaded it (version 4.80) and extracted them to a directory that Apache can access (I used /usr/share/adodb).

------------------------------------

BASE also needs a couple of PHP extensions, installed using PEAR (similar to CPAN). Image_Color requires the PHP to be compiled with gd, so if you didn't install it above, you must now do some backtracking. I installed these with the following commands:

/usr/local/php/bin/pear install Image_Color
/usr/local/php/bin/pear install Log
/usr/local/php/bin/pear install Numbers_Roman
/usr/local/php/bin/pear install http://pear.php.net/get/Numbers_Words-0.14.0.tgz
/usr/local/php/bin/pear install http://pear.php.net/get/Image_Canvas-0.3.0.tgz
/usr/local/php/bin/pear install http://pear.php.net/get/Image_Graph-0.7.2.tgz

Those last three packages will not automatically be found by PEAR, which is why I had to specify the URL. Finding the URL is easy, however. Simply go to http://pear.php.net and search for the package name, then copy the link location for the current release.

------------------------------------

Believe it or not, those are the ONLY prerequisites! That was a pain in the ass, I know. Now for the reason we all started this journey.

To install BASE, I downloaded and unarchived the latest version (1.2.4, aka Melissa). The documentation says to unpack it into Apache's document root (usually the htdocs directory, although not in my case), but I don't like that idea for many reasons. I unarchived it into its own directory (/programs/base-1.2.4) and then created a symbolic link inside of the htdocs directory by typing ln -sv /programs/base-1.2.4 /usr/apache/www/default/base.

I then created the BASE database in MySQL by using the supplied script:

mysql -u root -p < sql/create_base_tbls_mysql.sql snort

Next, I copied the stock configuration file and customized it for my environment. This is in base' root dir, and can be named correctly, and permissions set, by typing the following commands:

cp base_conf.php.dist base_conf.php
chown snort:snort base_conf.php
chmod 600 base_conf.php

Make sure to set the permissions on this file, because it contains the password for mysql's snort user in plain text!

Now, edit this base_conf.php file and make sure that the following lines are correct:

$DBlib_path = '/usr/share/adodb';
$DBtype = 'mysql';
$Use_Auth_System = 0; /* Set this to 1 to force users to authenticate in order to access BASE. */
$BASE_urlpath = '/base';
$alert_dbname = 'snort';
$alert_host = 'localhost';
$alert_port = ''; /* This is the port on which the database is accessed. Leave it blank, however, if MySQL is not running on a network socket. */
$alert_user = 'snort';
$alert_password = 'snortuserpass';
$db_connect_method = 2;

That $db_connect_method option is important - left at the default (1 = persistant connection), Snort will be unable to obtain a connection with the MySQL server and therefore be able to add any alerts!

You may also need to add an ExecCGI directive to your httpd.conf file for the base directory (especially if you get a "Options ExecCGI is off in this directory" error). This, of course, can also be done by creating an .htaccess file, with the following lines:

Options ExecCGI
DirectoryIndex index.cgi index.pl index.html index.htm default.htm default.html home.html index.shtml index.sht index.php index.php3
AuthType Basic
AuthUserFile /pathto/.htpasswd
AuthName "BASE Home"
require valid-user

You will need the following lines in httpd.conf as well:

LoadModule php5_module modules/libphp5.so
AddType text/html php AddHandler php5-script php

<Files *.php>
SetOutputFilter PHP SetInputFilter PHP
LimitRequestBody 9524288
</Files>

Those first two lines are actually a replacement for one line that you normally see in the documentation, "AddType application/x-httpd-php .php" The reasons for these alternative lines can be read here. I actually put them in because the normal line didn't do anything for me. :)

When I tried to load the index.php page, I got the following error:

Warning: mysql_pconnect() [function.mysql-pconnect]: Access denied for user 'nobody'@'localhost' (using password: NO) in /usr/share/adodb/drivers/adodb-mysql.inc.php on line 373

Error (p)connecting to DB : @

Check the DB connection variables in base_conf.php

= $alert_dbname : MySQL database name where the alerts are stored
= $alert_host : host where the database is stored
= $alert_port : port where the database is stored
= $alert_user : username into the database
= $alert_password : password for the username

Database ERROR:Access denied for user 'nobody'@'localhost' (using password: NO)

However, all of the settings appeared correct in my base_conf.php file! I looked around a bit, and discovered a setup page called setup1.php in the base directory. I ran it in my browser, filled in all the appropriate values, and this is the result:

Step 4 of 5

Operation Description Status
BASE tables Adds tables to extend the Snort DB to support the BASE functionality DONE
Search Indexes
(Optional) Adds indexes to the Snort DB to optimize the speed of the queries
DONE

The underlying Alert DB is configured for usage with BASE.

Additional DB permissions
In order to support Alert purging (the selective ability to permanently delete alerts from the database) and DNS/whois lookup caching, the DB user "snort" must have the DELETE and UPDATE privilege on the database "snort@localhost"

Ok, so it modified my stort database, adding tables and what not! Guess someone had to do it! :) It could not write to base_conf.php to set the values, but it printed out a conf file for me to cut and paste. I did so, into a new base_conf.php file, and I no longer got the error! I then proceeded to delete my base/setup directory.

 

Of course, it did let me in without authentication! I had not yet set the authentication through Apache at this point though, so I simply set it through there (with the .htaccess file I printed above). To generate or add to the .htpasswd file, run the htpasswd program supplied with Apache (/usr/bin/htpasswd -d /path/to/.htpasswd username) Voila!

You should now have a working Snort/Mysql/Base installation. Enjoy!

 

If you get a bunch of alerts in your syslog similar to the following:

Apr 20 13:46:19 firewall snort[14012]: database: mysql_error: Duplicate entry '2-4576' for key 1 SQL=INSERT INTO event (sid,cid,signature,timestamp) VALUES ('2', '4576', '5', '2006-04-20 13:46:19.116+-04')

It is most likely because you have more than one "output database" line in your snort.conf file, both going to the same database! Older documentation for installing Snort/BASE usually told you to add both of these lines:

output database: log, mysql, user=snort password=snortuserpass dbname=snort host=localhost
output database: alert, mysql, user=snort password=snortuserpass dbname=snort host=localhost

However, it appears that only the first one ("output database: log, ...") is actually required. Looking through my alerts through BASE, it appears that all of the information is still there. Take out that second line and your problems will disappear!

 

It appeared that most of my alerts were coming from my own network to external IP's, even though I had my $HOME_NET variable set correctly. From what I read, many of the scanners have problems with IP lists in $HOME_NET (as opposed to just a single IP or subnet in that variable). As the majority of these alerts were "portscans," identified by the sfPortscan module, I added my $HOME_NET variable to the list of ignore_scanners to sfPortscanner's options list, as defined in snort.conf. That section then looked like this:

preprocessor sfportscan: proto { all } \
memcap { 10000000 } \
sense_level { low } \
ignore_scanners { $HOME_NET }

You could, of course, stick a list in there without the variable - such as, "ignore_scanners { 192.168.16.0/24,10.0.7.0/24 }".

 

In fact, the easiest, most complete way to avoid logging (false) alerts from within your internal network is with a BPF filter. This is done by creating a small file containing a little exception rule, then specifying this file on the command line when running Snort.

I created a file called /etc/snort/rules/bpf-filters.conf, which contains the following line:

not src net 192.168.16.0/24 and not src net 10.0.7.0/24

Then, I added -F /etc/snort/bpf-filters.conf to the command line when running snort. (The full command that I ran to run snort was now snort -D -d -e -i bridge -c /etc/snort/snort.conf -u snort -g snort -F /etc/snort/bpf-filters.conf). No more alerts from inside my home network!

 

Should you wish to create a database for archiving Snort alerts, here are the commands (which are almost exactly like those used to create the original database):

# mysqladmin -u root -p create snortarchive
# mysql -u root -p <schemas/create_mysql snortarchive
# mysql -u root -p snortarchive
> grant CREATE,INSERT,SELECT,DELETE,UPDATE on snortarchive.* to snortarchive@localhost;
> FLUSH PRIVILEGES;
> EXIT
# mysql -u root -p < sql/create_base_tbls_mysql.sql snortarchive

Then, run http://base.ip/base/setup/setup1.php again to extend the database tables and create a new base_conf.php file.

 

 

 

Install Barnyard (processor for Snort's Unified File Format - increases speed) and Oinkmaster (keeps rules up to date). Check out this documentation.

A Snort/ACID database archiving/cleaning perl script can be found here.

Set up archive database for BASE?

Set appropriate permissions?


More to come...

 

Links:

Basic Analysis and Security Engine (BASE) project - This is the successor to ACID.

 

zlib-1.1.4-10.i386.rpm
openssl-0.9.7b.tar.gz
openssh-3.7.1p1.tar.gz

sshd

putkeys
keyconfig

dosemu-1.1.0.tgz ***This is NOT a stable release!***
dosemu-1.1.4.tgz ***This is NOT a stable release!***
dosemu-1.1.5.gz ***This is the most recent DEVELOPER'S version (8/20/03), but many problems are fixed here. ***
dosemu-freedos-b8p-bin.gz

flex-2.5.4a-29.i386.rpm
bison-1.35-6.i386.rpm

ipxutils-2.2.1-1.i386.rpm

screen-3.9.13-5.i386.rpm

linspy.tar
linspy2beta2.tgz

btee.c
btee (compiled version)

XFree86-devel-4.3.0-2.i386.rpm
XFree86-libs-4.3.0-2.i386.rpm
fontconfig-devel-2.1-9.i386.rpm