As mentioned in my previous posts, no matter which tool you use for SIEM, there will be times when this information is not readily available. Just as this was for the FireEye, Imperva SecureSphere and Sourcefire device in the last 3 posts, it is the same for WebSense events. Do remember also, ensuring that you can have access to the raw events that is received by your SIEM is extremely important. Let's look at examples for WebSense permitted and blocked transactions.
sample event
<159>Jul 21 09:03:16 172.25.30.101 LEEF:1.0|Websense|Security|7.7.3|transaction:blocked|sev=7 cat=9 usrName=LDAP://securitynik.lab/user1 src=10.0.0.1 srcPort=50459 srcBytes=268 dstBytes=0 dst=10.0.0.2 dstPort=80 proxyStatus-code=302 serverStatus-code=0 duration=3 method=GET disposition=1061 contentType=- reason=- policy=ALWAYS BLOCK role=8 userAgent=Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) url=http://somewebsite.org/
<159>Jul 17 14:46:38 172.27.30.68 LEEF:1.0|Websense|Security|7.7.3|transaction:permitted|sev=1 cat=9 usrName=LDAP://securitynik.lab/user src=10.0.0.1 srcPort=51917 srcBytes=408 dstBytes=8852 dst=10.0.0.2 dstPort=80 proxyStatus-code=200 serverStatus-code=200 duration=3 method=GET disposition=1048 contentType=text/html reason=- policy=security-nik-default-policy role=8 userAgent=Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; MS-RTC LM 8; InfoPath.3) url=http://somewebsite.org/view/outlook/outlook/outlook/outlook.html
Let's look at extracting the following fields:
• sev
• cat
• srcBytes
• dstBytes
• proxyStatus-code
• serverStatus-code
• duration
• method
• disposition
• contentType
• reason
• policy
• role
• userAgent
• url
The following fiields were extracted from the WebSense Events
Property Type: Regex Based
New Property Name: Sev
Field Type: Numeric
Description: Extracts the sev key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: \|sev=([0-9])(\s) - Capture Group: 1
Enabled
Property Type: Regex Based
New Property Name: Cat
Field Type: Numeric
Description: Extracts the cat key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\scat=)([0-9]*)(\s) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: srcBytes
Field Type: Numeric
Description: Extracts the srcBytes key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\ssrcBytes=)([0-9]*)(\s) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: dstBytes
Field Type: Numeric
Description: Extracts the dstBytes key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\sdstBytes=)([0-9]*)(\s) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: proxyStatus-code
Field Type: Numeric
Description: Extracts the proxyStatus-code key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\sproxyStatus-code=)([0-9]*)(\s) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: serverStatus-code
Field Type: Numeric
Description: Extracts the serverStatus-code key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\sserverStatus-code=)([0-9]*)(\s) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: method
Field Type: Numeric
Description: Extracts the method key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\smethod=)([a-zA-Z]*)(\s) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: disposition
Field Type: Numeric
Description: Extracts the disposition key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\smethod=)([a-zA-Z]*)(\s) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: contentType
Field Type: AlphaNumeric
Description: Extracts the contentType key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\scontentType=)(.*)(\sreason) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: reason
Field Type: AlphaNumeric
Description: Extracts the reason key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\sreason=)(.*)(\spolicy) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: policy
Field Type: AlphaNumeric
Description: Extracts the policy key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex:(\spolicy=)(.*)(\srole) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: role
Field Type: Numeric
Description: Extracts the role key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex:(\srole=)([0-9]*)(\s) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: userAgent
Field Type: AlphaNumeric
Description: Extracts the userAgent key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex:(\suserAgent=)(.*)(\surl) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: proxy-url
Field Type: AlphaNumeric
Description: Extracts the url key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex:(\surl=)(.*)(\s) - Capture Group: 2
Enabled
Once again, we have successfully extracted additional fields from WebSense
As was shown in the previous 3 posts, while a SIEM can be very helpful, it does not always give you everything you want, whenever you want, however you want. Maybe one of these days we will be able to have it our way ;-)
Regex Refernces:
http://www.autohotkey.com/docs/misc/RegEx-QuickRef.htm
https://www.tcl.tk/man/tcl8.5/tutorial/Tcl20.html
http://www.adobe.com/devnet/dreamweaver/articles/regular_expressions_pt1.html
http://www.rexegg.com/
As mentioned in the previous 2 posts in this series, we don't always get the fields parsed and displayed as we would like from the SIEM. In addition, as previously mentioned also, ensuring the payload is accessible is critical. In this post we will extract fields from Sourcefire logs.
Sample Payload Data:
DeviceType=Estreamer DeviceAddress=10.0.0.1 CurrentTime=1406554255965 recordType=INTRUSION_EVENT_RECORD recordLength=60 timestamp=28 Jun 2014 08:30:54 detectionEngineRef=2 eventId=458907 eventSecond=1406554254 eventMicrosecond=552218 rule.generatorId=1 rule.ruleId=16606 rule.ruleRevision=9 rule.renderedSignatureId=16606 rule.message=IPS Alert From SourceFire rule.ruleUUID=2FEB420CE5684FC5A90610F645E268C7 rule.ruleRevisionUUID=2B1A93DA34E811E3B791848F69E36DD2 classification.classificationId=9 classification.name=attempted-user classification.description=Attempted User Privilege Gain classification.classificationUUID=9D0A6F5ECBA211D9925A005056040501 classification.classificationRevisionUUID=00000000000000000000000000000000 priority.priorityId=1 priority.name=high sourceAddress=10.0.0.2 destinationAddress=10.0.0.3 sourcePortOrICMPType=49555 destinationPortOrICMPCode=443 ipProtocolId=6 impactFlags=00000000 impact=5 blocked=0
The fields to be extracted in this case are:
• DeviceType
• DeviceAddress
• recordType
• detectionEngineRef
• eventId
• rule.generatorId
• rule.ruleId
• rule.ruleRevision
• rule.renderedSignatureId
• rule.message
• rule.ruleUUID
• rule.ruleRevisionUUID
• classification.classificationId
• classification.name
• classification.description
• classification.classificationUUID
• classification.classificationRevisionUUID
• priority.priorityId
• priority.name
• sourceAddress
• destinationAddress
• sourcePortOrICMPType
• destinationPortOrICMPCode
• ipProtocolId
• impactFlags
• impact
• blocked
Without further ado, let's "Extract Property".
As always the prroperty type will be "Regex Based"
New Property Name: DeviceType
Field Type: AlphaNumeric
Description: Device Type as extracted from SourceFire Payload
Log Source Type: Sourcefire Defense Center
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: DeviceType=([a-zA-Z]*) - Capture Group: 1
Enabled
New Property Name: DeviceAddress
Field Type: AlphaNumeric
Description: Device Address as extracted from SourceFire Payload
Log Source Type: Sourcefire Defense Center
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: DeviceAddress=(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}) - Capture Group: 1
Enabled
New Property Name: RecordType
Field Type: AlphaNumeric
Description: Record Type as extracted from SourceFire Payload
Log Source Type: Sourcefire Defense Center
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: recordType=(.*?)(\s) - Capture Group: 1
Enabled
New Property Name: DetectionEngineRef
Field Type: Numeric
Description: Detection Refernece as extracted from SourceFire Payload
Log Source Type: Sourcefire Defense Center
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: detectionEngineRef=([0-9]*) - Capture Group: 1
Enabled
New Property Name: EventID
Field Type: Numeric
Description: Event ID as extracted from SourceFire Payload
Log Source Type: Sourcefire Defense Center
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: eventId=([0-9]*) - Capture Group: 1
Enabled
New Property Name: GID
Field Type: Numeric
Description: GeneratorID as extracted from SourceFire Payload
Log Source Type: Sourcefire Defense Center
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: GeneratorID=([0-9]*) - Capture Group: 1
Enabled
New Property Name: RuleID
Field Type: Numeric
Description: Rule ID as extracted from SourceFire Payload
Log Source Type: Sourcefire Defense Center
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: rule.ruleId=([0-9]*) - Capture Group: 1
Enabled
New Property Name: RuleRevision
Field Type: Numeric
Description: Rule Revision Number as extracted from SourceFire Payload
Log Source Type: Sourcefire Defense Center
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: rule.ruleId=([0-9]*) - Capture Group: 1
Enabled
New Property Name: RuleRenderedSignatureID
Field Type: Numeric
Description: Rule Rendered Signature ID Number as extracted from SourceFire Payload
Log Source Type: Sourcefire Defense Center
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: rule.renderedSignatureId=([0-9]*) - Capture Group: 1
Enabled
New Property Name: RuleUUID
Field Type: AlphaNumeric
Description: Rule UUID ID Number as extracted from SourceFire Payload
Log Source Type: Sourcefire Defense Center
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: rule.ruleUUID=([a-zA-Z0-9]*) - Capture Group: 1
Enabled
New Property Name: RuleRevisionUUID
Field Type: AlphaNumeric
Description: Rule Revision UUID ID Number as extracted from SourceFire Payload
Log Source Type: Sourcefire Defense Center
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: ruleRevisionUUID=([a-zA-Z0-9]*) - Capture Group: 1
Enabled
New Property Name: Classification
Field Type: AlphaNumeric
Description: Rule classification as extracted from SourceFire Payload
Log Source Type: Sourcefire Defense Center
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: classification.description=(.*)\sclassification.classificationUUID=(.*) - Capture Group: 1
Enabled
Regex Refernces:
http://www.autohotkey.com/docs/misc/RegEx-QuickRef.htm
https://www.tcl.tk/man/tcl8.5/tutorial/Tcl20.html
http://www.adobe.com/devnet/dreamweaver/articles/regular_expressions_pt1.html
http://www.rexegg.com/
As mentioned in my previous post, no matter which tool you use for SIEM, there will be times when this information is not readily available. Just as this was for the FireEye device in the last post, it is the same for Imperva's SecureSphere. Do remember also, ensuring that you can have access to the raw events that is received by your SIEM is extremely important.
Sample Event:
<6>LEEF:1.0|Imperva|SecureSphere|10.0.0|Firewall None|Alert ID=912905|devTimeFormat=yyyy-MM-dd HH:mm:ss.S|devTime=2014-07-22 06:59:58.0|Alert type=Firewall|src=10.0.0.1|usrName=n/a|Application name=${Alert.applicationName}|Service name=${Alert.serviceName}|Alert Description=TCP - TCP Unexpected SYN|Severity=High|Simulation Mode=false|Immediate Action=None|Event ID=4238139139125767123|dst=10.0.0.2|dp=443|Server Group=securitynik_servers|Affected Application=|Affected Application (violation)=$item.alert.applicationName|HTTP Method=|HTTP Host=|Query=
Let's try to extract the following fields.
Alert Description
Alert ID
Severity
Similarly to the previous post, we will "Extract Property" from the events
Property Type: Regex based
Property Name: Alert Description
Field Type: AlphaNumeric
Description: Alert Description as extracted from the raw Imperva Log
Log Source Type: Imperva SecureSphere
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: Alert Description=(.*?\|) - Capture Group 1
Enabled
Property Name: Alert ID:
Field Type: AlphaNumeric
Description: Alert ID Field
Log Source Type: Imperva SecureSphere
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: Alert\sID=([0-9]*) - Capture Group 1
Enabled
Property Name: Severity
OPtimized parsing for rules, reports and searches
Field Type: AlphaNumeric
Description: Severity
Log Source Type: Imperva SecureSphere
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (Severity\=)([a-zA-Z]*) - Capture Group 2
Enabled
Voila!!! Just like that, we've extracted data from Imperva's SecureSphere which was not readily parsed by QRadar.
Regex Refernces:
http://www.autohotkey.com/docs/misc/RegEx-QuickRef.htm
https://www.tcl.tk/man/tcl8.5/tutorial/Tcl20.html
http://www.adobe.com/devnet/dreamweaver/articles/regular_expressions_pt1.html
http://www.rexegg.com/
One thing I've learnt that I know will remain true. No matter which tool you use for SIEM, there will be times when this information is not readily available. When I say readily available, I mean parsed by default. However, I do believe once the raw data is in the event, it can be extracted and this is what becomes important. Ensuring that you can have access to the raw events that is received by your SIEM is extremely important.
Considering the amount of time I spend working with QRadar, I am surprised that I havent done any posts on working with QRadar as yet. Anyhow, for this posts, I will extract some information from FireEye's Malware-Object and Infection-Match events.
There are lots of similarities between these two objects. However, there are also lots of differences. As a result, I will use the Infection-Match as the main event from extraction
Sample Malware-Object event
<164>fenotify-129166.alert: LEEF:1.0|FireEye|MPS|7.2.0.224371|malware-object|osinfo=Microsoft Windows7 64-bit 6.1 sp1 14.0528;Microsoft WindowsXP 32-bit 5.1 sp3 14.0528^src=10.0.0.1^sname=Trojan.Asprox^shost=host.securitynik.lab^fileHash=1e5e39f8691b50377769690625efb172^filePath=/someTypeOfExe.exe^dst=10.0.0.2^proto=tcp^dvchost=FireEye^dvc=10.0.0.3^cncHost=cnc.securitynik.lab^externalId=129166^devTime=Jul 01 2014 18:27:32 UTC^sid=33351728^cncPort=8080^link=https://FireEye.securitynik.lab/event_stream/events_for_bot?ma_id\=129166^cncChannel=POST /D552F7C0BB0949631E52BEED25BA191DA4C6182356 HTTP/1.1::~~Accept: */*::~~Content-Type: application/x-www-form-urlencoded::~~User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0::~~Host: cnc.securitynik.lab:8080::~~Content-Length: 312::~~Cache-Control: no-cache::~~::~~\\200^vlan=0^
Sample Infection-Match event
<164>fenotify-29625.1.alert: LEEF:1.0|FireEye|MPS|7.2.0.224371|infection-match|src=10.0.0.1^sname=Local.Infection^shost=host.securitynik.lab^dstMAC=00:15:17:ef:dd:3a^proto=tcp^dvchost=FireEye^dst=10.0.0.2^vlan=0^srcPort=50297^request=hxxp://www.securitynik.labhttp://www.securitynik.lab/sites/default/files/css/css_bae06db3942ff213d9081182d8d659be.css^dvc=10.0.0.3^cncHost=10.0.0.2^externalId=29625^devTime=Jun 09 2014 12:29:47 UTC^sid=502048^cncPort=9119^link=https://FireEye.securitynik.lab/event_stream/events_for_bot?ev_id\=29625^dstPort=9119^cncChannel=GET http://www.securitynik.lab/sites/default/files/css/css_bae06db3942ff213d9081182d8d659be.css HTTP/1.1::~~Accept: text/css::~~Referer: http://www.securitynik.lab/menu::~~Accept-Language: en-US::~~User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)::~~Accept-Encoding: gzip, deflate::~~Host: www.securitynik.lab::~~Proxy-Connection: Keep-Alive::~~Cookie: SESScee87b0f0ae3e3341e2b18fd993a57c3\=8g9c1rd?
Now that we have the information to be extracted there are a couple of ways to do this. However, let's assume we already have one of these events open. If we do, we can then "Extract Property" from this event. We will use the following information below to complete this task.
Property Type: Regex based
Property Definition:
field: sname
New Property: sname
Optimize for parsing for rules, reports and searches
Fieldtype: Alphanumeric
Description: This field extract the "sname" key value pair from FireEye's malware-object
Log source type: FireEye
Log Source: All
High Level Category: Any
Low Level Category: Any
Regex: sname=(.*?)(\^) - Capture Group 1
Enabled
Property Definition:
field: shost
New Property: shost
Optimize for parsing for rules, reports and searches
Fieldtype: Alphanumeric
Description: This field extract the "shost" key value pair from FireEye
Log source type: FireEye
Log Source: All
Event Name: infection-match
High Level Category: Any
Low Level Category: Any
Regex: shost=(.*?)(\^) - Capture Group 1
Enabled
Property Definition:
field: fileHash
New Property: fileHash
Optimize for parsing for rules, reports and searches
Fieldtype: Alphanumeric
Description: This field extract the "fileHash" key value pair from FireEye
Log source type: FireEye
Log Source: All
High Level Category: Any
Low Level Category: Any
Regex: fileHash=(.*?)(\^) - Capture Group 1
Enabled
Property Definition:
field: filePath
New Property: filePath
Optimize for parsing for rules, reports and searches
Fieldtype: Alphanumeric
Description: This field extract the "filePath" key value pair from FireEye
Log source type: FireEye
Log Source: All
High Level Category: Any
Low Level Category: Any
Regex: filePath=(.*?)(\^) - Capture Group 1
Enabled
Property Definition:
field: proto
New Property: proto
Optimize for parsing for rules, reports and searches
Fieldtype: Alphanumeric
Description: This field extract the "proto" key value pair from FireEye
Log source type: FireEye
Log Source: All
High Level Category: Any
Low Level Category: Any
Regex: proto=(.*?)(\^) - Capture Group 1
Enabled
Property Definition:
field: dvchost
New Property: dvchost
Fieldtype: Alphanumeric
Description: This field extract the "dvchost" key value pair from FireEye
Log source type: FireEye
Log Source: All
High Level Category: Any
Low Level Category: Any
Regex: dvchost=(.*?)(\^) - Capture Group 1
Enabled
Property Definition:
field: dvc
New Property: dvc
Optimize for parsing for rules, reports and searches
Fieldtype: Alphanumeric
Description: This field extract the "dvc" key value pair from FireEye
Log source type: FireEye
Log Source: All
High Level Category: Any
Low Level Category: Any
Regex: dvc=(.*?)(\^) - Capture Group 1
Enabled
Property Definition:
field: cncHost
New Property: cncHost
Optimize for parsing for rules, reports and searches
Fieldtype: Alphanumeric
Description: This field extract the "cncHost" key value pair from FireEye's malware-object
Log source type: FireEye
Log Source: All
High Level Category: Any
Low Level Category: Any
Regex: cncHost=(.*?)(\^) - Capture Group 1
Enabled
Property Definition:
field: externalId
New Property: externalId
Fieldtype: Alphanumeric
Description: This field extract the "externalId" key value pair from FireEye
Log source type: FireEye
Log Source: All
High Level Category: Any
Low Level Category: Any
Regex: externalId=(.*?)(\^) - Capture Group 1
Enabled
Property Definition:
field: devTime
New Property: devTime
Optimize for parsing for rules, reports and searches
Fieldtype: Alphanumeric
Description: This field extract the "devTime" key value pair from FireEye
Log source type: FireEye
Log Source: All
High Level Category: Any
Low Level Category: Any
Regex: devTime=(.*?)(\^) - Capture Group 1
Enabled
Property Definition:
field: sid
New Property: FireEye-sid
Fieldtype: Alphanumeric
Description: This field extract the "sid" key value pair from FireEye
Log source type: FireEye
Log Source: All
High Level Category: Any
Low Level Category: Any
Regex: sid=(.*?)(\^) - Capture Group 1
Enabled
Property Definition:
field: sid
New Property: cncPort
Fieldtype: Alphanumeric
Description: This field extract the "cncPort" key value pair from FireEye
Log source type: FireEye
Log Source: All
High Level Category: Any
Low Level Category: Any
Regex: cncPort=(.*?)(\^) - Capture Group 1
Enabled
Property Definition:
field: link
New Property: link
Optimize for parsing for rules, reports and searches
Fieldtype: Alphanumeric
Description: This field extract the "link" key value pair from FireEye
Log source type: FireEye
Log Source: All
High Level Category: Any
Low Level Category: Any
Regex: link=(.*?)(\^) - Capture Group 1
Enabled
Property Definition:
field: link
New Property: cncChannel
Optimize for parsing for rules, reports and searches
Fieldtype: Alphanumeric
Description: This field extract the "cncChannel" key value pair from FireEye
Log source type: FireEye
Log Source: All
High Level Category: Any
Low Level Category: Any
Regex: cncChannel=(.*?)(\^) - Capture Group 1
Enabled
There it is, we have successfully extracted the information deemed pertinent for us at this time, which FireEye did not provide by default.
Regex Refernces:
http://www.autohotkey.com/docs/misc/RegEx-QuickRef.htm
https://www.tcl.tk/man/tcl8.5/tutorial/Tcl20.html
http://www.adobe.com/devnet/dreamweaver/articles/regular_expressions_pt1.html
http://www.rexegg.com/
