Friday, August 1, 2014

QRadar - Extracting fields from WebSense events

As mentioned in my previous posts, no matter which tool you use for SIEM, there will be times when this information is not readily available. Just as this was for the FireEye, Imperva SecureSphere and Sourcefire device in the last 3 posts, it is the same for WebSense events. Do remember also, ensuring that you can have access to the raw events that is received by your SIEM is extremely important. Let's look at examples for WebSense permitted and blocked transactions.

sample event
<159>Jul 21 09:03:16 172.25.30.101 LEEF:1.0|Websense|Security|7.7.3|transaction:blocked|sev=7    cat=9    usrName=LDAP://securitynik.lab/user1    src=10.0.0.1    srcPort=50459    srcBytes=268    dstBytes=0    dst=10.0.0.2    dstPort=80    proxyStatus-code=302    serverStatus-code=0    duration=3    method=GET    disposition=1061    contentType=-    reason=-    policy=ALWAYS BLOCK    role=8    userAgent=Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)    url=http://somewebsite.org/ 

<159>Jul 17 14:46:38 172.27.30.68 LEEF:1.0|Websense|Security|7.7.3|transaction:permitted|sev=1    cat=9    usrName=LDAP://securitynik.lab/user    src=10.0.0.1    srcPort=51917    srcBytes=408    dstBytes=8852    dst=10.0.0.2    dstPort=80    proxyStatus-code=200    serverStatus-code=200    duration=3    method=GET    disposition=1048    contentType=text/html    reason=-    policy=security-nik-default-policy    role=8    userAgent=Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; MS-RTC LM 8; InfoPath.3)    url=http://somewebsite.org/view/outlook/outlook/outlook/outlook.html 



Let's look at extracting the following fields:
sev 
cat 
srcBytes 
dstBytes 
proxyStatus-code 
serverStatus-code 
duration 
method 
disposition 
contentType 
reason 
policy 
role 
userAgent 
url


The following fiields were extracted from the WebSense Events

Property Type: Regex Based
New Property Name: Sev
Field Type: Numeric
Description: Extracts the sev key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: \|sev=([0-9])(\s) - Capture Group: 1
Enabled


Property Type: Regex Based
New Property Name: Cat
Field Type: Numeric
Description: Extracts the cat key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\scat=)([0-9]*)(\s) - Capture Group: 2
Enabled

Property Type: Regex Based
New Property Name: srcBytes
Field Type: Numeric
Description: Extracts the srcBytes key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\ssrcBytes=)([0-9]*)(\s) - Capture Group: 2
Enabled


Property Type: Regex Based
New Property Name: dstBytes
Field Type: Numeric
Description: Extracts the dstBytes key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\sdstBytes=)([0-9]*)(\s) - Capture Group: 2
Enabled

Property Type: Regex Based
New Property Name: proxyStatus-code
Field Type: Numeric
Description: Extracts the proxyStatus-code key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\sproxyStatus-code=)([0-9]*)(\s) - Capture Group: 2
Enabled

Property Type: Regex Based
New Property Name: serverStatus-code
Field Type: Numeric
Description: Extracts the serverStatus-code key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\sserverStatus-code=)([0-9]*)(\s) - Capture Group: 2
Enabled

Property Type: Regex Based
New Property Name: method
Field Type: Numeric
Description: Extracts the method key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\smethod=)([a-zA-Z]*)(\s) - Capture Group: 2
Enabled


Property Type: Regex Based
New Property Name: disposition
Field Type: Numeric
Description: Extracts the disposition key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\smethod=)([a-zA-Z]*)(\s) - Capture Group: 2
Enabled

Property Type: Regex Based
New Property Name: contentType
Field Type: AlphaNumeric
Description: Extracts the contentType key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\scontentType=)(.*)(\sreason) - Capture Group: 2
Enabled

Property Type: Regex Based
New Property Name: reason
Field Type: AlphaNumeric
Description: Extracts the reason key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex: (\sreason=)(.*)(\spolicy) - Capture Group: 2
Enabled

Property Type: Regex Based
New Property Name: policy
Field Type: AlphaNumeric
Description: Extracts the policy key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex:(\spolicy=)(.*)(\srole) - Capture Group: 2
Enabled

Property Type: Regex Based
New Property Name: role
Field Type: Numeric
Description: Extracts the role key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex:(\srole=)([0-9]*)(\s) - Capture Group: 2
Enabled

Property Type: Regex Based
New Property Name: userAgent
Field Type: AlphaNumeric
Description: Extracts the userAgent key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex:(\suserAgent=)(.*)(\surl) - Capture Group: 2
Enabled


Property Type: Regex Based
New Property Name: proxy-url
Field Type: AlphaNumeric
Description: Extracts the url key value pair from the WebSense log
Log Source Type: Websense VSeries
Log Source: All
Category: High Level: Any
Low Level Category Any
Regex:(\surl=)(.*)(\s) - Capture Group: 2
Enabled

Once again, we have successfully extracted additional fields from WebSense

As was shown in the previous 3 posts, while a SIEM can be very helpful, it does not always give you everything you want, whenever you want, however you want. Maybe one of these days we will be able to have it our way ;-)


Regex Refernces:
http://www.autohotkey.com/docs/misc/RegEx-QuickRef.htm
https://www.tcl.tk/man/tcl8.5/tutorial/Tcl20.html
http://www.adobe.com/devnet/dreamweaver/articles/regular_expressions_pt1.html
http://www.rexegg.com/

10 comments:

  1. copy and past error for disposition field.

    Should be:
    Regex: (\sdisposition=)([a-zA-Z]*)(\s) - Capture Group: 2

    ReplyDelete
  2. Thanks it helps a lot. I need to extract fields from sysmon logs.

    ReplyDelete
    Replies
    1. Glad you found it helpful. However, for sysmon you will at least need to develop a UDSM as I don't think there is any DSM for this.

      Take a look at my post on building your first UDSM. This should help you.

      Delete
    2. Created uDSM for it,it actually receives logs,need RegEx to extract usefule fields,like for ProcessGuid i have used \sLogonGuid:\s(.*?)\s\w+:

      payload contains: LogonGuid: {8CD23D7C-B703-5922-0000-0020E7030000} LogonId: 0x3e7

      Like this what are imp fields to extract that i don't understand

      Delete
    3. Send me a few sanitize sample logs, maybe about 10 entries in a file and I will put together a quick post on parsing SYSMON. Send them to my email nikalleyne at gmail dot com.

      Delete
  3. In my case,my wincollect Payload gets truncated at qradar side.How to solve this?

    ReplyDelete
    Replies
    1. Are you using the latest version of the WinCollect software?

      Delete
    2. If you haven't as yet, I suggest you open a ticket with IBM to see what's going on.

      Delete