Sunday, July 10, 2016

Building a Splunk Dashboard by parsing Palo Alto events – “THREAT” Logs

In this post, we will leverage Splunk – which I installed previously – to build a dashboard that allows us to get a quick overview of our Palo Alto “Threats” Logs.

In the previous post we looked at parsing the “TRAFFIC” Logs In this post we look at parsing the “THREAT” logs. Once again, before we can parse a log we need to understand the structure of the log file. From below we can see that this log event is “,” delimited. So looking at this log, let’s do our thing. If you looked at the previous post, the "THREAT" logs is structured basically the same as the "TRAFFIC" logs except for a few fields where the value differs such as the "sub type"

Jul  8 07:00:45 Jul  7 18:57:46 1,2016/07/07 18:57:46,001606042988,THREAT,vulnerability,1,2016/07/07 18:57:41,,107.xx.xx.21,99.xx.xx.63,,Web Traffic,,,web-browsing,vsys1,TRUSTED_LAN_L3,INTERNET,vlan,ethernet1/1,Security Monitoring,2016/07/07 18:57:46,7691,1,60316,80,30074,80,0x400000,tcp,alert,"",HTTP OPTIONS Method(30520),any,informational,client-to-server,825,0x0,,United States,0,

using the search query below we can parse the Threat logs

host="" THREAT | rex field=_raw ".*,THREAT,(?<fw_threat_subtype>\w+),.*?,(?<fw_srcIP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),(?<fw_dstIP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,(?<fw_dstPort>\w+),.*?,.*?,.*?,.*?,(?<fw_protocol>\w+),(?<fw_action>\w+),.*?,(?<fw_msg>.*?),.*?,.*?,(?<fw_direction>\w+),.*?,.*?,(?<fw_geolocation>.*?)," | stats count by fw_threat_subtype, fw_srcIP, fw_dstIP, fw_dstPort, fw_protocol, fw_action, fw_msg, fw_direction, fw_geolocation | sort count | reverse

Taking another example, let’s look at vulnerabilities which have been seen over the past … however long we got some logs for …

host="" THREAT | rex field=_raw ".*,THREAT,(?<fw_threat_subtype>\w+),.*?,(?<fw_srcIP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),(?<fw_dstIP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,(?<fw_dstPort>\w+),.*?,.*?,.*?,.*?,(?<fw_protocol>\w+),(?<fw_action>\w+),.*?,(?<fw_msg>.*?),.*?,.*?,(?<fw_direction>\w+),.*?,.*?,(?<fw_geolocation>.*?)," | stats count by fw_msg | sort count | reverse

Ok enough of this. In this series we built a monitoring solution and then used Splunk to prase some Palo Alto logs.

Hope you enjoyed and do leave a comment if you find this series useful.

All Posts In This Series.

Building a monitoring solution – Hardening the OS - CentOS 7 (Linux)
Building a monitoring solution – Forwarding Palo Alto Logs
Building a monitoring solution - Parsing Palo Alto 200 - TRAFFIC (Firewall) Logs
Building a Splunk Dashboard by parsing Palo Alto events – “THREAT” Logs

1 comment:

  1. Thanks Nik for sharing.

    One thing that has changed in the recent Palo Alto threat log syntax is "Threat Content Name (threatid)", which is equivalent to the field name "fw_msg" in the above SPL query you mentioned.

    I also able to write a SPL which works in a bit different way than yours, and extracts all the important fields. I hope this will be helpful for someone.

    Here is the SPL:
    index=* sourcetype="pan:threat"
    | eval field=split(_raw, ",")
    | eval type=mvindex(field,3), subtype=mvindex(field,4), src_ip=mvindex(field,7), dst_ip=mvindex(field,8), nat_src=mvindex(field,9), nat_dst=mvindex(field,10), rule_name=mvindex (field,11), app=mvindex(field,14),src_zone=mvindex(field,16),dst_zone=mvindex(field,17), ingress_if=mvindex(field,18), egress_if=mvIndex(field,19), log_action=mvindex(field ,20), src_port=mvindex(field,24), dst_port=mvindex(field,25), proto=mvindex(field,29), action=mvindex(field,30), url=mvindex(field,31), threat_id=mvindex(field,32), cat =mvindex(field,33), sev=mvindex(field,34), direction=mvindex(field,45)
    | table _time type subtype src_ip dst_ip nat_src nat_dst rule_name app src_zone dst_zone ingress_if egress_if log_action src_port dst_port proto action url threat_id cat sev direction index

    Field extraction is based on