DNS Visibility

Doug Burks wrote up a 2nd post at the SecurityOnion Blog about DNS Visibility.
http://securityonion.blogspot.com/2013/10/got-dns-visibility.html

Some of the original ideas come from a post by Johannes Ullrich
 https://isc.sans.edu/diary/A+Poor+Man%27s+DNS+Anomaly+Detection+Script/13918

Doug Comments:
For those running Bro [1] on Security Onion [2], I've modified the script [3].
and he posts the code at: 
[3] - http://code.google.com/p/security-onion/wiki/DNSAnomalyDetection

#!/bin/bash

BRO_LOGS="/nsm/bro/logs"
TODAY=`date +%Y-%m-%d`
YESTERDAY=`date -d yesterday +%Y-%m-%d`
OLD_DIRS=`ls $BRO_LOGS |egrep -v "current|stats|$TODAY|$YESTERDAY"`
TMPDIR=/tmp
OLDLOG=$TMPDIR/oldlog
NEWLOG=$TMPDIR/newlog
SUSPECTS=$TMPDIR/suspects
for DIR in $OLD_DIRS; do zcat $BRO_LOGS/$DIR/dns* |bro-cut query; done | sort | uniq -c | sort -k2 > $OLDLOG
zcat $BRO_LOGS/$YESTERDAY/dns* |bro-cut query | sort | uniq -c | sort -k2 > $NEWLOG
join -1 2 -2 2  -a 2 $OLDLOG $NEWLOG | egrep -v '.* [0-9]+ [0-9]+$' | sort -nr -k2 | head -10 > $SUSPECTS

This script will run through *ALL* of your DNS/bro logs on your Security Onion install.  I have modified the script to allow you to look back over any number of days and compare that summary to yesterday's DNS logs. 

Examples:

last week compared to yesterday ($sh DNSAnomalyDetection 7)

2 days ago compared to yesterday ($sh DNSAnomalyDetection 2)

-----------------
#!/bin/bash

BRO_LOGS="/nsm/bro/logs"

TIMEFRAME=$1
TODAY=`date +%Y-%m-%d`
YESTERDAY=`date -d yesterday +%Y-%m-%d`

TMPDIR=/tmp
OLDLOG=$TMPDIR/oldlog
NEWLOG=$TMPDIR/newlog
SUSPECTS=$TMPDIR/suspects

while [ $TIMEFRAME -ne 1 ];
do
 PROCESSING_DAY=`date -d "-$TIMEFRAME Day" +%Y-%m-%d`
 zcat $BRO_LOGS/$PROCESSING_DAY/dns* |bro-cut query;
 TIMEFRAME=$((TIMEFRAME - 1))
done | sort | uniq -c | sort -k2 > $OLDLOG

zcat $BRO_LOGS/$YESTERDAY/dns* | bro-cut query | sort | uniq -c | sort -k2 > $NEWLOG

join -1 2 -2 2 -a 2 $OLDLOG $NEWLOG | egrep -v '.* [0-9]+ [0-9]+$' | sort -nr -k2 | head -10 > $SUSPECTS

-Ross Warren