Had a little issue today, where the team could not connect to an IBM QRadar appliance via SSH or HTTPS. This is somewhat strange as we expect these services to be available for us to be able to do our job. I Assigned the task to an Analyst to take a quick look. here is what we got.
1. I am unable to access QRadar via SSH
2. I am unable to access QRadar via HTTPS
Current solution:
Report to the team that supports QRadar, that we are unable to access the environment via SSH or HTTPS.
What is the QRadar team’s solution?
Allow the IPs (and I assume ports) to the QRadar device. In my opinion, this was not needed, unless they recently blocked it. However, with that step done, the problem still exists.
Working as an Analyst, you are always expected to put your supporting evidence of the issue. This is true whether it is a security issue you are reporting or connectivity issue. Think device up/down (via Nagios, Solarwinds, etc.), monitoring as an example outside of security.
Considering the above, let’s use packet analysis with TShark to see what we can learn about these two issue.
Starting with TCP Port 22 associated with SSH.
Loading up the first 7 frames of the PCAP in TShark, we see:
kali@securitynik:~$ tshark -r ssh.pcapng -c 7 1 0.000000 10.0.0.1 → 10.0.0.2 TCP 66 57157 → 22 [SYN, ECN, CWR] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1 2 0.029507 10.0.0.2 → 10.0.0.1 TCP 66 22 → 57157 [SYN, ACK] Seq=0 Ack=1 Win=42340 Len=0 MSS=1380 SACK_PERM=1 WS=4096 3 0.029570 10.0.0.1 → 10.0.0.2 TCP 54 57157 → 22 [ACK] Seq=1 Ack=1 Win=66048 Len=0 4 0.029698 10.0.0.1 → 10.0.0.2 SSH 96 Client: Protocol (SSH-2.0-SecureCRT_8.5.4 (x64 build 1942)) 5 0.057867 10.0.0.2 → 10.0.0.1 TCP 60 22 → 57157 [ACK] Seq=1 Ack=43 Win=45056 Len=0 6 58.098216 10.0.0.1 → 10.0.0.2 TCP 54 57157 → 22 [FIN, ACK] Seq=43 Ack=1 Win=66048 Len=0 7 58.166203 10.0.0.2 → 10.0.0.1 TCP 60 22 → 57157 [ACK] Seq=1 Ack=44 Win=45056 Len=0
Breaking this down further below, we see frames 1 to 3 showing the 3-way handshake completed successfully. Don't let's worry about ECN and CWR in this post. Focus on the SYN -> SYN/ACK -> ACK.
kali@securitynik:~$ tshark -r ssh.pcapng -c 3 1 0.000000 10.0.0.1 → 10.0.0.2 TCP 66 57157 → 22 [SYN, ECN, CWR] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1 2 0.029507 10.0.0.2 → 10.0.0.1 TCP 66 22 → 57157 [SYN, ACK] Seq=0 Ack=1 Win=42340 Len=0 MSS=1380 SACK_PERM=1 WS=4096 3 0.029570 10.0.0.1 → 10.0.0.2 TCP 54 57157 → 22 [ACK] Seq=1 Ack=1 Win=66048 Len=0
Looking at above, we conclude the host is available and that port TCP 22, typically associated with SSH is available.
Looking closer at the SSH communication in frame 4, we see our client begins communication at the application layer, sending its SSH identification information.
kali@securitynik:~$ tshark -n -r ssh.pcap -Y 'frame.number==4' 4 0.029698 10.0.0.1 → 10.0.0.2 SSH 96 Client: Protocol (SSH-2.0-SecureCRT_8.5.4 (x64 build 1942))
Transitioning to the response from the server, we see:
kali@securitynik:~$ tshark -n -r ssh.pcapng -Y '(frame.number==5) || (frame.number==6) || (frame.number==7)' 5 0.057867 10.0.0.2 → 10.0.0.1 TCP 60 22 → 57157 [ACK] Seq=1 Ack=43 Win=45056 Len=0 6 58.098216 10.0.0.1 → 10.0.0.2 TCP 54 57157 → 22 [FIN, ACK] Seq=43 Ack=1 Win=66048 Len=0 7 58.166203 10.0.0.2 → 10.0.0.1 TCP 60 22 → 57157 [ACK] Seq=1 Ack=44 Win=45056 Len=0
What is interesting about above, is the SSH Client starts the SSH connection with the client’s QRadar, sending its protocol and client information (Identification information).
The Client’s QRadar responds with “ACK”, then our SSH Client initiates a graceful teardown of the communication by setting its “FIN,
ACK” flags. Looking at this, we can see our SSH Client is responsible for
initiating the teardown of the communication. But why is this so?
Looking at RFC 4253 (RFC 4253 - The Secure Shell (SSH) Transport Layer Protocol (ietf.org) ) it states …
When the connection has been established, both sides MUST send an
identification string. This identification string MUST be
SSH-protoversion-softwareversion SP comments CR LF
Note the MUST in the wording above. Looking back above, we see our SSH client sends its identification string as “SSH-2.0-SecureCRT_8.5.4 (x64 build 1942)”. However, the Client’s QRadar, rather than sending its identification string, simply responded with “ACK”. We don’t see its identification string. I believe this is the reason why our SSH client requested to close the connection as it is unable to transition to the Key Exchange phase of the SSH connection. I believe if we run the SSH client in Debug mode if available, we may see this as the issue.
Issue Number 2: Looking at HTTPS. This is less complicated to see what the issue is.
kali@securitynik:~$ tshark -r 443-4.pcapng -Y "(tcp.port==57129) && (tcp.port==443)" 23 6.960410 10.0.0.1 → 10.0.0.2 TCP 66 57129 → 443 [SYN, ECN, CWR] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1 24 6.989644 10.0.0.2 → 10.0.0.1 TCP 60 443 → 57129 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0 29 7.489974 10.0.0.1 → 10.0.0.2 TCP 66 [TCP Retransmission] 57129 → 443 [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1 30 7.519994 10.0.0.2 → 10.0.0.1 TCP 60 443 → 57129 [RST, ACK] Seq=4184603348 Ack=1 Win=0 Len=0 35 8.019747 10.0.0.1 → 10.0.0.2 TCP 62 [TCP Retransmission] 57129 → 443 [SYN] Seq=0 Win=8192 Len=0 MSS=1460 SACK_PERM=1 36 8.049017 10.0.0.2 → 10.0.0.1 TCP 60 443 → 57129 [RST, ACK] Seq=739404838 Ack=1 Win=0 Len=0
As you look above, you see the HTTPS client sends its “SYN”. However, it is unable to complete the 3-way handshake. How do we know this? Look at the session above where we discussed SSH. However, staying with the HTTPS connection, we see the QRadar responds with “RST, ACK”. This tells me it is not listening on port 443.
With the above analysis, what have we concluded:
1. The QRadar devce is UP. We know this because SSH 3-way handshake completed successfully on TCP port 22 and the QRadar also sent “RST” stating that HTTPS on TCP port 443 is not “listening”
2. SSH does not seems to be fully available on the QRadar device as it did not return its identification string.
3. HTTPS is not available/listening on the QRadar device.
Recommended solutions based on the analysis above:
1. Work with the remote team to restart the SSH and HTTPS services on the QRadar device.
2. If recommendation 1 fails, there is going to be a need to restart the QRadar console.
Takeaway for you the reader.
As a Security Analyst, ensure you have enough evidence to support your positions. This is true whether you are reporting on a security incident or addressing a device down.
If you wish to learn more about the importance of being able to analyze packets, come hangout with us at an upcoming SEC503 - Intrusion Detection in Depth or at an upcoming SEC582 - Mastering TShark Packet Analysis class. Alternatively grab a copy of my book Mastering TShark Network Forensics.
Additional read to help you get going in the meantime.
Learning
by practicing: Windump basics by examples (securitynik.com)
Learning by practicing: a few not so basic windump examples
(securitynik.com)
Man
page of TCPDUMP
RFC 4253 - SSH
No comments:
Post a Comment