Importing Nessus CSV reports to SPLUNK from the Command Line!

Problem Solved!  Hurricane Labs provides a nice Splunk App to consume Nessus CSV files.  But, I did not want to manually download a new CSV from the Nessus web interface and then move it to my Splunk server. I could have made a samba share from my Splunk server to my PC and just saved the output from the Nessus web interface to the share.. Still too much manual work!

After a lot of searching I found some good information on the Nessus discussion pages

https://discussions.nessus.org/message/17812#17812
cmerchant@responsys.com answers their own question:

#!/bin/bash

AUTH=$(wget --no-check-certificate --post-data 'login=nessus&password=password' https://server:8834/login -O -| grep -Po '(?<=token\>)[^\<]+(?=\<\/token)')
FILE=$(wget --no-check-certificate --post-data 'token='$AUTH'&report=XXXXXX&xslt=csv.xsl' https://server:8834/file/xslt -O - | grep -Po '(?<=/file/xslt/download/\?)[^\"]+(?=\"\>)')

wget --no-check-certificate --post-data 'token='$AUTH'&'$FILE'&step=2' https://server:8834/file/xslt/download -O test.csv

This got me moving toward a solution. I had never done any web page parsing with wget and javascripts, so it was about time to learn...

My requirements were:

• No interaction - must be able to be run in cron
• Grab all completed Nessus results
• Save the file with the Friendly Report name so Splunk can use the file name as the Report Name

Here is the results. This needs some clean up and more documentation, but it is completely usable as is. Except you will need to replace xxxxxx with your password and x.x.x.x with your nessus server IP.
(word wrap didnt play nice here, carefull with your cut and paste)


#!/bin/bash

#Variables
SPLUNK_NESSUS=/mnt/nessus

#Retrive AUTH Token
token="$(/usr/bin/wget -q --no-check-certificate --post-data 'login=nessus&password=xxxxxx' https://x.x.x.x:8834/login -O - | grep -Po '(?<=token\>)[^\<]+(?=\<\/token)')"

#Get list of reports
/usr/bin/wget -q --no-check-certificate --post-data "token=$token" https://x.x.x.x:8834/report/list -O - | grep -Po '(?<=name\>)[^\<]+(?=\<\/name)' > /tmp/reports

#Get Friendly Names
/usr/bin/wget -q --no-check-certificate --post-data "token=$token" https://x.x.x.x4:8834/report/list -O - | grep -Po '(?<=readableName\>)[^\<]+(?=\<\/readableName)' > /tmp/names

#Merge two files
/usr/bin/pr -tmJ --sep-string=" " /tmp/reports /tmp/names > /tmp/named.reports

for i in $(cut -d' ' -f1 /tmp/named.reports);
do
#Get Filenames for reports
FILENAME=$(/usr/bin/wget -q --no-check-certificate --post-data 'token='$token'&report='$i'&xslt=csv.xsl' https://x.x.x.x:8834/file/xslt -O - | grep -Po '(?<=/file/xslt/download/\?fileName=)[^\"]+(?=\"\>)')

#Get files
#build Readable name to report number match
READABLENAME=$(grep $i /tmp/named.reports | cut -d' ' -f2- --output-delimiter='')
sleep 5
/usr/bin/wget -q --no-check-certificate --post-data 'token='$token'&fileName='$FILENAME'&step=2' https://x.x.x.x:8834/file/xslt/download -O $SPLUNK_NESSUS/$READABLENAME.csv;
done;

#Cleanup
rm /tmp/reports
rm /tmp/names
rm /tmp/named.reports

#note
# Remove files in /opt/nessus/var/nessus/users/nessus/files on nessus server

If you use this please send me an email rossw@woodhome.com