Offline pcap NSM Analysis Script

#!/bin/sh
# - - -
# Offline pcap NSM Analysis Script - Tested on Mr Bejtlich FreeBSD 5.4 VM ( www.SGUIL.net )
# I made this script for personal use primarily, nice if you find it useful.
# nillepill[at]gmail.com
# TODO: Fix working Tethereal options in script variable
# - - -
# Would like to recommend TaoSecurity Training, http://www.taosecurity.com/training.html
# I really enjoyed attending the 4 day Network Security Operations Course
# - - -

# Description
# Takes input from specified tcpdump file
# Creates new folders in users home directory, puts all analysis output in those directorys
#

# Enviroment
WORK_DIR=$HOME
FILE=$1
CASE_FOLDER=$(date +%H-%M-%S)-$1
ANALYSIS_FOLDER=NSM_Analysis
DATE_FOLDER=$(date +%Y-%m-%d)
HASH_DIR=HASHES
# Tools
DATE=/bin/date
TCPDSTAT=/usr/local/bin/tcpdstat
CAPINFOS=/usr/X11R6/bin/capinfos
TCPDUMP=/usr/sbin/tcpdump
TETHEREAL=/usr/X11R6/bin/tethereal
# Should work? TETHEREAL_OPTIONS=`-qzio,phs -nr`
P0F=/usr/local/bin/p0f
SNORT=/usr/local/bin/snort
SNORT_CONF=/usr/local/etc/nsm/snort.conf
ARGUS=/usr/local/sbin/argus
RAGATOR=/usr/local/bin/ragator
RACOUNT=/usr/local/bin/racount
RAHOSTS=/usr/local/bin/rahosts
RA=/usr/local/bin/ra
MD5=/sbin/md5
SHA1=/sbin/sha1
# Usage description
if [ -z "$1" ]; then
echo ""
echo "-----------------------------------"
echo "Offline pcap NSM Analysis Script"
echo "-----------------------------------"
echo ""
echo Usage: $0 interface.pcap
echo ""
exit
fi

# Start Analysis
echo ""
echo "Starting Offline pcap NSM Analysis Script"
$DATE
echo ""

echo "# Create folders, copy file and set to read only"
mkdir $WORK_DIR/$ANALYSIS_FOLDER
mkdir $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER
mkdir $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER
mkdir $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$HASH_DIR
cp $1 $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/
chmod 400 $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$1
echo "# Create hashes"
$MD5 $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$1 > $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$HASH_DIR/$1.sha1
$SHA1 $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$1 > $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$HASH_DIR/$1.md5
echo "# Set hash mod"
chmod 400 $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$HASH_DIR/$1.sha1
chmod 400 $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$HASH_DIR/$1.md5
echo "# Set dir mod"
chmod 700 $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$HASH_DIR/
echo "# Run statistical tools"
$CAPINFOS $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE > $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE.txt.capinfos
$TCPDSTAT $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE > $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE.txt.tcpdstat
# Should work? Just works (TM) below tho.. $TETEHEREAL $TETHEREAL_OPTIONS $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE > $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE.txt.tethereal
tethereal -qzio,phs -nr $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE > $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE.txt.tethereal
echo "# Run Session tools"
$ARGUS -r $FILE -w $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE.arg.bin
$RAGATOR -r $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE.arg.bin -w $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE.arg.bin.ragator
$RACOUNT -ar $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE.arg.bin.ragator > $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE.arg.txt.racount
$RAHOSTS -nr $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE.arg.bin.ragator > $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE.arg.txt.rahosts
$RA -nn -r $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE.arg.bin.ragator -s saddr daddr dport proto | sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4 | uniq -c > $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE.arg.txt.ra.session
echo "# Fingerprint identifiable hosts"
$P0F -U -s $FILE -o $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE.txt.p0f
echo "# Run Alert tools"
$SNORT -c $SNORT_CONF -l . -b -y -r $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE -l $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/
echo "# Copy hostinfo to case"
uname -a >> $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE.txt.analysis.host
date >> $WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/$FILE.txt.analysis.date
echo "# Finish!"
echo ""
echo "------------------------------------------------------"
$DATE
echo "Analysis finished, data are ready for review in"
echo "$WORK_DIR/$ANALYSIS_FOLDER/$DATE_FOLDER/$CASE_FOLDER/"
echo ""
echo "------------------------------------------------------"
echo ""

Save a tree each day using Windows :)

Assume files and folders in drive F

Batch.tree.f.bat Put directory tree into new file each day

@echo off for /f "tokens=1-3 delims=- " %%g in ('date /t') do ( set yy=%%g set mm=%%h set dd=%%i ) tree /a f:\ > "f:\Daily Tree Backup\%yy%-%mm%-%dd%.tree.f.txt" if exist "f:\Daily Tree Backup\%yy%-%mm%-%dd%.tree.f.txt" ( rd /S /Q "f:\Daily Tree Backup\%yy%-%mm%-%dd%.tree.f.txt" )

Send Exchange SMTP Logs To Syslog Server

"Log parser is a powerful, versatile tool that provides universal query access to text-based data such as log files, XML files and CSV files, as well as key data sources on the Windows® operating system such as the Event Log, the Registry, the file system, and Active Directory®."

http://www.microsoft.com/downloads/details.aspx?FamilyID=890cd06b-abf8-4c25-91b2-f8d975cf8c07&displaylang=en


Run in batchfile from Windows Scheduled tasks every (day/hour/other)

Exchange.Syslog.smtp.bat
LogParser.exe file:Queryexchange.sql -i:IISW3C -iCheckPoint:cp.txt -o:SYSLOG

Queryexchange.sql
SELECT TO_TIMESTAMP(date, time),c-ip,cs-username,s-sitename,s-computername,s-ip,s-port,cs-method,cs-uri-stem,cs-uri-query,sc-status,sc-win32-status,sc-bytes,cs-bytes,time-taken,cs-version,cs-host,cs(User-Agent),cs(Cookie),cs(Referer)
INTO @syslogserver.com:514
FROM L:\log\SMTPSVC1\*.log

Looks nice in Splunk ;)

FreeBSD 6.2 Raid-1

How to setup software raid-1 during installation of FreeBSD 6.2.

Pre req; Server with two sata harddrives prefered same brand and make.
Assume,
Hdd 1 = ad0
Hdd 2 = ad1

Start installation, select ad0 as hdd, choose packages add some accounts etc. But DO NOT REBOOT when done!
Go back to installation and start "Fix it" shell (ALT+F4)

# sysctl kern.geom.debugflags=16
# gmirror label -v -b round-robin gm0 /dev/ad0
# echo geom_mirror_load="YES" >> /boot/loader.conf
# vi /etc/fstab
Edit : i

add /mirror/ after dev and change ad0 to gm0

Real world example below

Device Mountpoint FStype Options Dump Pass
/dev/mirror/gm0s1b none swap sw 0 0
/dev/mirror/gm0s1a / ufs rw 1 1
/dev/mirror/gm0s1e /tmp ufs rw 2 2
/dev/mirror/gm0s1f /usr ufs rw 2 2
/dev/mirror/gm0s1d /var ufs rw 2 2
/dev/acd0 /cdrom cd9660 ro,noauto 0 0


Check config and reboot
Continue below if reboot successful

# gmirror insert gm0 /dev/ad1
# gmirror status
Check status, ad1 (hdd 2) should start rebuilding from ad0 (hdd 1)


Source
http://www.onlamp.com/pub/a/bsd/2005/11/10/FreeBSD_Basics.html

Splunk Base 2.2.3 - FreeBSD installation

Splunk Install on FreeBSD 6.2

You have to install ports before proceeding

cd /usr/ports/misc/compat5x
#make install clean


#mkdir /usr/local/src
cd /usr/local/src

#pkg_add -r wget


#wget 'http://www.splunk.com/index.php/download_track?file=/2.2.3/freebsd/splunk-2.2.3-18173-freebsd-5.4-intel.tgz&ac=&wget=true&name=wget'

Override default installation directory (/opt/splunk) I think nonstandard stuff "should" go to /usr/local/.. ..

#pkg_add -v -p /usr/local/src/ splunk-2.1-freebsd-5.4-intel.tgz

Start Splunk
/usr/local/src/splunk/bin/splunk
..
License shows
..
Accept? y

Splunk now listens on

TCP
8000
8001
8089


If you want Splunk to listen for syslog,
be certain to keep it form starting on reboot.

# vi /etc/rc.conf

Add

syslogd_enable="NO"


Then setup Splunk to run on startup

#crontab -e

@reboot /usr/local/src/splunk/bin/./splunk start


Done!

Still, you need to enable local firewall for protection

PADS - Passive Asset Detection System

"..It will listen to a network and attempt to provide an up-to-date look at the hosts and services running on the network."

http://passive.sourceforge.net/

Simple to run, maybe nohup or screen. Why not rc.d :)

- - -

$cat pads.em0

#!/bin/sh
#A small PADS script, change values to match enviroment
#Define interface we use
INTERFACE=em0
#Define main directory we use, each interface gets a directory within
DIRECTORY=/nsm/pads
#Define local path to our binary
BINARY=/usr/local/bin/pads
#Define output
OUTPUT=assets.csv

$BINARY -D -i $INTERFACE -w $DIRECTORY/$INTERFACE/$OUTPUT

- - -

Tethereal Ringbuffer

Simple loggerscript, easy for running with nohup or screen.

- - -

#!/bin/sh
FILESIZE=1000000
FILENUMBER=200
INTERFACE=em0
/usr/local/bin/tethereal -n -i $INTERFACE -s 1515 -q -a filesize:$FILESIZE -b files:$FILENUMBER -w /nsm/em0/tethereal.fullcontent.ringbuffer.em0

- - -

Other tools

Mr Bejtlich shows Deamonlogger, running ringbuffer using nonroot user/group.
http://taosecurity.blogspot.com/2007/04/daemonlogger-in-ring-buffer-mode.html

OpenBSD provides tcpdump builtin priv sep
http://ftp.bg.openbsd.org/OpenBSD/src/usr.sbin/tcpdump/privsep.c

SGUIL Setup Part 1

Step one, acquire hardware!

What do I have?

Today
Fujitsu Siemens RX100 1U 2.4/512/80+320/3 NIC

Future
1 HP Proliant DL380G3 2U 2.8/2048/72 + 72/2 NIC
0SEK
2 Fujitsu Siemens RX100 S2 1U 3.0/1500/250+250/3 NIC
0SEK

Needs

TAP -
2 NetOptics 10/100BaseT Tap
P/N: TP-CU (96430)
3800SEK pcs

NIC -
2 INTEL PRO/1000PT DUAL PORT SERVER ADAPTER PCI-E
P/N: EXPI9402PTBLK
1400SEK pcs

HDD -
4 SEAGATE BARRACUDA ES 250GB SATA/300 16MB

P/N: ST3250620NS

600SEK pcs

2 HP Universal Hard Drive - Hårddisk - 146.8 GB - hot-swap - 3.5" - Ultra320 SCSI - 10000 rpm
P/N: 286716-B22
2800SEK pcs