In this post, we will leverage Splunk – which I installed previously – to build a dashboard that allows us to get a quick overview of our Palo Alto “Threats” Logs.
In the previous post we looked at parsing the “TRAFFIC” Logs
In this post we look at parsing the “THREAT” logs. Once again, before we can
parse a log we need to understand the structure of the log file. From below we
can see that this log event is “,” delimited. So looking at this log, let’s do
our thing. If you looked at the previous post, the "THREAT" logs is structured basically the same as the "TRAFFIC" logs except for a few fields where the value differs such as the "sub type"
Jul 8 07:00:45
192.168.1.5 Jul 7 18:57:46 1,2016/07/07
18:57:46,001606042988,THREAT,vulnerability,1,2016/07/07
18:57:41,192.168.1.29,107.xx.xx.21,99.xx.xx.63,107.21.226.21,Web
Traffic,,,web-browsing,vsys1,TRUSTED_LAN_L3,INTERNET,vlan,ethernet1/1,Security
Monitoring,2016/07/07 18:57:46,7691,1,60316,80,30074,80,0x400000,tcp,alert,"",HTTP
OPTIONS
Method(30520),any,informational,client-to-server,825,0x0,192.168.0.0-192.168.255.255,United
States,0,
using the search query below we can parse the Threat logs
host="192.168.1.5" THREAT | rex field=_raw ".*,THREAT,(?<fw_threat_subtype>\w+),.*?,(?<fw_srcIP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),(?<fw_dstIP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,(?<fw_dstPort>\w+),.*?,.*?,.*?,.*?,(?<fw_protocol>\w+),(?<fw_action>\w+),.*?,(?<fw_msg>.*?),.*?,.*?,(?<fw_direction>\w+),.*?,.*?,(?<fw_geolocation>.*?)," | stats count by fw_threat_subtype, fw_srcIP, fw_dstIP, fw_dstPort, fw_protocol, fw_action, fw_msg, fw_direction, fw_geolocation | sort count | reverse
host="192.168.1.5" THREAT | rex field=_raw ".*,THREAT,(?<fw_threat_subtype>\w+),.*?,(?<fw_srcIP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),(?<fw_dstIP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,(?<fw_dstPort>\w+),.*?,.*?,.*?,.*?,(?<fw_protocol>\w+),(?<fw_action>\w+),.*?,(?<fw_msg>.*?),.*?,.*?,(?<fw_direction>\w+),.*?,.*?,(?<fw_geolocation>.*?)," | stats count by fw_threat_subtype, fw_srcIP, fw_dstIP, fw_dstPort, fw_protocol, fw_action, fw_msg, fw_direction, fw_geolocation | sort count | reverse
Taking another example, let’s look at vulnerabilities which
have been seen over the past … however long we got some logs for …
host="192.168.1.5" THREAT | rex field=_raw ".*,THREAT,(?<fw_threat_subtype>\w+),.*?,(?<fw_srcIP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),(?<fw_dstIP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,(?<fw_dstPort>\w+),.*?,.*?,.*?,.*?,(?<fw_protocol>\w+),(?<fw_action>\w+),.*?,(?<fw_msg>.*?),.*?,.*?,(?<fw_direction>\w+),.*?,.*?,(?<fw_geolocation>.*?)," | stats count by fw_msg | sort count | reverse
Ok enough of this. In this series we built a monitoring solution and then used Splunk to prase some Palo Alto logs.
Hope you enjoyed and do leave a comment if you find this series useful.
All Posts In This Series.
Thanks Nik for sharing.
ReplyDeleteOne thing that has changed in the recent Palo Alto threat log syntax is "Threat Content Name (threatid)", which is equivalent to the field name "fw_msg" in the above SPL query you mentioned.
I also able to write a SPL which works in a bit different way than yours, and extracts all the important fields. I hope this will be helpful for someone.
Here is the SPL:
index=* sourcetype="pan:threat"
| eval field=split(_raw, ",")
| eval type=mvindex(field,3), subtype=mvindex(field,4), src_ip=mvindex(field,7), dst_ip=mvindex(field,8), nat_src=mvindex(field,9), nat_dst=mvindex(field,10), rule_name=mvindex (field,11), app=mvindex(field,14),src_zone=mvindex(field,16),dst_zone=mvindex(field,17), ingress_if=mvindex(field,18), egress_if=mvIndex(field,19), log_action=mvindex(field ,20), src_port=mvindex(field,24), dst_port=mvindex(field,25), proto=mvindex(field,29), action=mvindex(field,30), url=mvindex(field,31), threat_id=mvindex(field,32), cat =mvindex(field,33), sev=mvindex(field,34), direction=mvindex(field,45)
| table _time type subtype src_ip dst_ip nat_src nat_dst rule_name app src_zone dst_zone ingress_if egress_if log_action src_port dst_port proto action url threat_id cat sev direction index
Field extraction is based on https://docs.paloaltonetworks.com/pan-os/8-1/pan-os-admin/monitoring/use-syslog-for-monitoring/syslog-field-descriptions/threat-log-fields