sample event
<159>Jul 21 09:03:16 172.25.30.101 LEEF:1.0|Websense|Security|7.7.3|transaction:blocked|sev=7 cat=9 usrName=LDAP://securitynik.lab/user1 src=10.0.0.1 srcPort=50459 srcBytes=268 dstBytes=0 dst=10.0.0.2 dstPort=80 proxyStatus-code=302 serverStatus-code=0 duration=3 method=GET disposition=1061 contentType=- reason=- policy=ALWAYS BLOCK role=8 userAgent=Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) url=http://somewebsite.org/
<159>Jul 17 14:46:38 172.27.30.68 LEEF:1.0|Websense|Security|7.7.3|transaction:permitted|sev=1 cat=9 usrName=LDAP://securitynik.lab/user src=10.0.0.1 srcPort=51917 srcBytes=408 dstBytes=8852 dst=10.0.0.2 dstPort=80 proxyStatus-code=200 serverStatus-code=200 duration=3 method=GET disposition=1048 contentType=text/html reason=- policy=security-nik-default-policy role=8 userAgent=Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; MS-RTC LM 8; InfoPath.3) url=http://somewebsite.org/view/outlook/outlook/outlook/outlook.html
Let's look at extracting the following fields:
• sev
• cat
• srcBytes
• dstBytes
• proxyStatus-code
• serverStatus-code
• duration
• method
• disposition
• contentType
• reason
• policy
• role
• userAgent
• url
The following fiields were extracted from the WebSense Events
Property Type: Regex Based
New Property Name: Sev
Field Type: Numeric
Description: Extracts the sev key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: \|sev=([0-9])(\s) - Capture Group: 1
Enabled
Property Type: Regex Based
New Property Name: Cat
Field Type: Numeric
Description: Extracts the cat key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\scat=)([0-9]*)(\s) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: srcBytes
Field Type: Numeric
Description: Extracts the srcBytes key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\ssrcBytes=)([0-9]*)(\s) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: dstBytes
Field Type: Numeric
Description: Extracts the dstBytes key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\sdstBytes=)([0-9]*)(\s) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: proxyStatus-code
Field Type: Numeric
Description: Extracts the proxyStatus-code key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\sproxyStatus-code=)([0-9]*)(\s) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: serverStatus-code
Field Type: Numeric
Description: Extracts the serverStatus-code key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\sserverStatus-code=)([0-9]*)(\s) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: method
Field Type: Numeric
Description: Extracts the method key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\smethod=)([a-zA-Z]*)(\s) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: disposition
Field Type: Numeric
Description: Extracts the disposition key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\smethod=)([a-zA-Z]*)(\s) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: contentType
Field Type: AlphaNumeric
Description: Extracts the contentType key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\scontentType=)(.*)(\sreason) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: reason
Field Type: AlphaNumeric
Description: Extracts the reason key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\sreason=)(.*)(\spolicy) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: policy
Field Type: AlphaNumeric
Description: Extracts the policy key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex:(\spolicy=)(.*)(\srole) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: role
Field Type: Numeric
Description: Extracts the role key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex:(\srole=)([0-9]*)(\s) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: userAgent
Field Type: AlphaNumeric
Description: Extracts the userAgent key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex:(\suserAgent=)(.*)(\surl) - Capture Group: 2
Enabled
Property Type: Regex Based
New Property Name: proxy-url
Field Type: AlphaNumeric
Description: Extracts the url key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex:(\surl=)(.*)(\s) - Capture Group: 2
Enabled
Once again, we have successfully extracted additional fields from WebSense
As was shown in the previous 3 posts, while a SIEM can be very helpful, it does not always give you everything you want, whenever you want, however you want. Maybe one of these days we will be able to have it our way ;-)
Regex Refernces:
http://www.autohotkey.com/docs/misc/RegEx-QuickRef.htm
https://www.tcl.tk/man/tcl8.5/tutorial/Tcl20.html
http://www.adobe.com/devnet/dreamweaver/articles/regular_expressions_pt1.html
http://www.rexegg.com/
copy and past error for disposition field.
ReplyDeleteShould be:
Regex: (\sdisposition=)([a-zA-Z]*)(\s) - Capture Group: 2
Thanks for the update! Very much appreciated!!
DeleteThanks it helps a lot. I need to extract fields from sysmon logs.
ReplyDeleteGlad you found it helpful. However, for sysmon you will at least need to develop a UDSM as I don't think there is any DSM for this.
DeleteTake a look at my post on building your first UDSM. This should help you.
Created uDSM for it,it actually receives logs,need RegEx to extract usefule fields,like for ProcessGuid i have used \sLogonGuid:\s(.*?)\s\w+:
Deletepayload contains: LogonGuid: {8CD23D7C-B703-5922-0000-0020E7030000} LogonId: 0x3e7
Like this what are imp fields to extract that i don't understand
Send me a few sanitize sample logs, maybe about 10 entries in a file and I will put together a quick post on parsing SYSMON. Send them to my email nikalleyne at gmail dot com.
DeleteIn my case,my wincollect Payload gets truncated at qradar side.How to solve this?
ReplyDeleteAre you using the latest version of the WinCollect software?
Deleteyes 7.2
DeleteIf you haven't as yet, I suggest you open a ticket with IBM to see what's going on.
Delete