One of the most important skills in anyone's armory responsible for looking after the security of a corporation's networks should be how to analyze network capture files (PCAP files) obtained from sniffers. Putting a sniffer on the network can not only help you investigate network issues, but also give you a great insight into the "unseeable" security vulnerabilities that are occurring on a daily basis.
This is probably one of the cheapest security tools you can use on the network, as it's free, and can find a multitude of potential issues. This being the case, it is also one of the easiest attack vectors an attacker or disgruntled employee can use on your internal network to extract data, and not get noticed.
Now, on a switched network you can't just fire up your sniffer, and expect to capture all the traffic. You'll see traffic, but mostly only broadcast traffic. You could set up a SPAN or mirrored port on your switch, or set up a TAP, but that's assuming you manage your network. What an attacker would do would involves creating a man-in-the-middle attack using ARP poisoning techniques.
The majority of networks we test, there tends to be little or no ARP poisoning defenses in place, and a one-line command, using something like the tool ettercap soon allows you to redirect traffic on the local network via your host to capture all the traffic traversing the network. We can then sniff the data and write it to a PCAP file for analysis. Unless you have some Intrusion Detection System (IDS) or Network Access Control (NAC) system in place, you won't even notice this is going on. As a result, it is often used post exploitation on a PC or server by the bad guys, to just sit there and capture traffic, and retrieve the PCAP file for off-line analysis.
Now, if you are managing the security of the network and are not aware of what's travelling over your networks, then you should be aware! If on the other hand you are doing this, then great, good job :-)! Either way, you should be aware whether anyone can perform ARP poisoning on your network and capture this data gold mine. I'm not going to go over how to ARP poison, just know that it is easy and anyone can do it and you should know if you're vulnerable or not.
So, assuming you have captured network traffic into a PCAP file ready for analysis, I thought I'd collate some of my favourite methods of searching PCAP files for "hidden gems" you may find useful.
Wireshark is probably one of the best tools out there for this job (usually). It gives you a great number of options to analyze captured traffic on the network. One of its biggest failings however is it can't handle large PCAP files....computer says no! It just crashes and hangs. In this case I
always resort to the command line. Tools that can analyze PCAP files via the CLI have several advantages over the Wireshark GUI: firstly, you don't need a GUI so you can run them via a shell and can do it remotely. Secondly, they can handle large files, and thirdly they are generally scriptable. I'm sure there are other pros and cons, but that's enough for now :-)
So, below are my top 10 favorites I tend to use on a daily basis. These should be run to just get an idea of what the traffic is 'like' and to get you more information to delve in further. Note I used different CLI tools for different tasks, either for personal preference or just because it's easier!
There is no doubt different ways to achieve the same task! I would advise also exploring the tools used further to show how powerful they can be.
No 1) List the hosts in the pcap file using wireshark's CLI, tshark
This is useful to generate a quick target list of IP's.
tshark -r file.pcap -q -z hosts,ipv4
e.g.
# Host data gathered from file.pcap
192.168.100.11 so-dc.domain.co.uk
192.168.101.87 name
No 2) Show hierarchy stats using tshark (-z option is for listing stats)
This is useful to get an overview of all the protocols and services running on the network to see what has been captured.
tshark -r file.pcap -q -z io,phs
e.g.
Protocol Hierarchy Statistics
Filter:
eth frames:87837 bytes:34609121
ip frames:84316 bytes:34404858
tcp frames:81502 bytes:34009978
ssh frames:446 bytes:54908
tpkt frames:3083 bytes:361497
http frames:3936 bytes:2713999
data-text-lines frames:574 bytes:309642
tcp.segments frames:121 bytes:76011
json frames:29 bytes:22741
tcp.segments frames:2 bytes:820
ocsp frames:36 bytes:24755
tcp.segments frames:3 bytes:1775
image-gif frames:278 bytes:150491
tcp.segments frames:59 bytes:27149
malformed frames:3 bytes:4302
png frames:42 bytes:28379
tcp.segments frames:27 bytes:17360
media frames:69 bytes:49723
tcp.segments frames:33 bytes:13579
image-jfif frames:39 bytes:37590
tcp.segments frames:18 bytes:14910
tcp.segments frames:36 bytes:16771
malformed frames:21 bytes:1243
xml frames:8 bytes:3362
tcp.segments frames:2 bytes:1430
msnms frames:645 bytes:164923
ftp frames:153 bytes:15210
ftp-data frames:5788 bytes:8277176
ssl frames:2500 bytes:1413771
tcp.segments frames:126 bytes:115216
data frames:132 bytes:40583
nbss frames:130 bytes:24246
smb frames:50 bytes:9766
pipe frames:10 bytes:1768
lanman frames:10 bytes:1768
smb2 frames:66 bytes:13178
tcp.segments frames:2 bytes:570
icmp frames:1528 bytes:212944
udp frames:1286 bytes:181936
nbns frames:552 bytes:51136
db-lsp-disc frames:430 bytes:75588
data frames:85 bytes:12014
nbdgm frames:95 bytes:21185
smb frames:95 bytes:21185
mailslot frames:95 bytes:21185
browser frames:95 bytes:21185
http frames:52 bytes:15140
dns frames:64 bytes:5397
cldap frames:6 bytes:1296
ntp frames:2 bytes:180
arp frames:2052 bytes:109746
arp.duplicate-address-detected frames:650 bytes:29892
arp.duplicate-address-detected frames:452 bytes:18984
arp frames:452 bytes:18984
arp frames:198 bytes:10908
llc frames:736 bytes:46022
stp frames:729 bytes:45200
ipx frames:7 bytes:822
nbipx frames:6 bytes:588
nmpi frames:1 bytes:234
smb frames:1 bytes:234
mailslot frames:1 bytes:234
browser frames:1 bytes:234
ipv6 frames:58 bytes:17103
udp frames:58 bytes:17103
dhcpv6 frames:40 bytes:7200
http frames:18 bytes:9903
lldp frames:25 bytes:1500
3) Show conversations and destination ports using tshark
This is useful to see what ports are open on hosts without scanning, and
see who is talking to it.
tshark -o column.format:'"Source", "%s", "Destination",
"%d", "dstport", "%uD"' -r file.pcap |sort|uniq
e.g.
192.168.101.31 -> 192.168.101.255 138
192.168.101.3 -> 192.168.101.255 137
192.168.101.3 -> 192.168.101.255 138
192.168.101.3 -> 192.168.101.87 3389
192.168.101.3 -> 224.0.0.252 5355
192.168.101.4 -> 192.168.101.255 137
192.168.101.4 -> 192.168.101.255 138
192.168.101.4 -> 224.0.0.252 5355
4) Show top talkers for all packets using tcpdump
This is useful to see who are the noisiest hosts on the network. A noisy
host is a busy host and therefore requires further investigation.
tcpdump -tnr $FILE |awk -F '.' '{print $1"."$2"."$3"."$4}' | sort | uniq -c | sort -n | tail
e.g.
# of Pkts IP Addr
========= =======
333 IP 192.168.101.112
361 IP 192.168.101.7
520 ARP, Reply 192.168.101.200 is-at 00:11:22:33:44:55, length 28
521 IP 192.168.100.2
636 STP 802.1d, Config, Flags [none], bridge-id 8065.00:11:22:33:44:55.8002, length 42
637 STP 802.1d, Config, Flags [none], bridge-id 8000.66:77:88:99:11:22:db.8011, length 43
770 IP 192.168.101.88
2049 ARP, Request who-has 192.168.101.200 tell 192
2588 IP 192.168.101.3
4902 IP 192.168.101.87
5) Search for text strings using ngrep
This is useful to look for any specific string or regex you want e.g look for "password", "card","username" etc....This will normally find FTP, HTTP, or POP passwords as examples. Although this is a simple example, ngrep can be used for complex regex's.
ngrep -q -I file.pcap|grep -i user
e.g
..........< TRUNCATED>..........
en" id="secure_username" name="username" value="" />...<input type="hidden"
id="secure_password" name="password" value="" />...<input type="hidden"
id="secure_key" name="key" value="d8a7
..........< TRUNCATED>..........
6) Find emails using ngrep
As an example of ngrep's regular expression power, you can make up your regex, and then use grep to display it in a nice format. In this example, I'm searching anything that looks like an email address and output the result
ngrep -q -I file.pcap '[a-zA-Z0-9.]+\.?@[a-zA-Z0-9.]+\.[a-zA-Z0-9]+' |grep -Eo '[a-zA-Z0-9.]+\.?@[a-zA-Z0-9.]+\.[a-zA-Z0-9]+'|sort|uniq
NOTE - you will also get false positives using the above, so best to check manually.
e.g.
foo@hotmail.com
name@outlook.com
joe@blogs.co.uk....Rel
..e.E..1H.@.......e
E....z.1..j..F.WHP0@..U...90705.3.1
7) Finding Credit Card Numbers using ccsrch
This tool will look at all files in a directory, including PCAP files. You just need to give it the directory or file name. If you're doing PCI assessments this is a great tool for finding PAN data, credit card numbers and track data on windows and UNIX operating systems.
Again this can have a large number of false positives which will require further investigation. If it is a valid number then you will need to determine what is leaking the data. Its normally a database, when a client makes searches, to the server, but can also be found in emails, web application traffic traversing the network in clear text.
ccsrch /home
e.g.
/home/file.pcap VISA 469xxxxxxxxxx320
/home/file.pcap AMEX 3444xxxxxxxx444
......<TRUNCATED>...................
Files searched -> 15
Search time (seconds) -> 0
Credit card matches-> 12
8) Show all TCP streams using tshark
This is useful to show all the TCP sessions on the network
tshark -r file.pcap -q -z conv,tcp
e.g.
<......TRUNCATED.......>
192.168.101.3:23445 <-> 192.168.101.87:ms-wbt-server 4683 452766 2572 274418 7255
727184 0.070916000 1271.9423
192.168.101.88:42299 <-> 204.13.200.136:ssh 521 59058 769 66850 1290 125908 0.001257000 1271.0156
192.168.101.87:62467 <-> 192.168.100.11:netbios-ssn 12 1579 15 4707 27 6286 1135.718589000 13.7333
<......TRUNCATED.......>
9) Just show specific TCP data using tcpick
This is a great tool to easily look at TCP streams. It can be used as a sniffer or to analyze PCAP files. I tend to use it to look at specific port data to find ascii data from emails, ftp data,www traffic, telnet etc, but it can do much of the queries already discussed. Just another great command line tool.
e.g. to show all ftp payloads
tcpick -r file.pcap -yP "port 21"
(-yP means payload printable)
Timeout for connections is 600
tcpick: reading from file.pcap
setting filter: "port 21"
PORT 192,168,101,6,16,251
PORT 192,168,101,6,16,251
PORT 192,168,101,6,16,251
200 PORT command successful.
200 PORT command successful.
200 PORT command successful.
RETR data01.csv
RETR data01.csv
RETR data01.csv
10) See the connection status using tcpick
Again another simple command to quickly assess who is talking to who (brought to you in pretty Technicolor in the terminal!)
e.g.
tcpick -r file.pcap -C
1 SYN-SENT 192.168.101.87:62478 >192.168.100.11:loc-srv
1 SYN-RECEIVED 192.168.101.87:62478 >192.168.100.11:loc-srv
1 ESTABLISHED 192.168.101.87:62478 >192.168.100.11:loc-srv
2 SYN-SENT 192.168.101.87:62479 >192.168.100.11:49155
2 SYN-RECEIVED 192.168.101.87:62479 >192.168.100.11:49155
2 ESTABLISHED 192.168.101.87:62479 > 192.168.100.11:49155
3 SYN-SENT 192.168.101.87:62480 >192.168.100.11:ldap
3 SYN-RECEIVED 192.168.101.87:62480 >192.168.100.11:ldap
3 ESTABLISHED 192.168.101.87:62480 >192.168.100.11:ldap
4 SYN-SENT 192.168.101.87:62481 > 192.168.100.12:kerberos
4 SYN-RECEIVED 192.168.101.87:62481 >192.168.100.12:kerberos
Obviously the commands above should just be the start of your investigation and analysis of the PCAP file. It will give you an idea of potential issues, and will help direct you to delve deeper into the data as required....and yes there will be other ways of achieving the same results :-)
So, it time honoured fashion, here's a wee script I made earlier just pulling all of the above together....
#!/bin/sh
# Script to take a PCAP network capture file and try to extract useful
# information such as clear text, passwords, users, emails, CC data etc
# Ensure ngrep,tcpick are installed.
# written by D Kirkpatrick
ARGS=1
FILE="$1"
###################################################
if [ $# -ne "$ARGS" ]; then
printf "Usage: `basename $0` file.pcap \n "
echo "Searches a pcap file for useful information"
exit 0
fi
#########################################
# Search Criteria
########################################
# show hosts of IP traffic
echo "Show hosts"
echo "=========="
tshark -r $FILE -q -z hosts,ipv4
# Show Hierarcy stats using tshark (-z option is for listing stats)
echo "Show hierarchy"
echo "=============="
tshark -r $FILE -q -z io,phs
# Show all traffic and dest port info
echo "Show all sessions and destination ports"
echo "======================================="
tshark -o column.format:'"Source", "%s", "Destination", "%d", "dstport", "%uD"' -r $FILE |sort|uniq
# List the top talkers, and sort by number of bytes sent by each
echo "Top Talkers For All Traffic"
echo "============================="
echo
echo -e '# of Pkts \t IP Addr'
echo -e '========= \t ======='
tcpdump -tnr $FILE |awk -F '.' '{print $1"."$2"."$3"."$4}' | sort | uniq -c | sort -n | tail
# Show all conversations and port detail using tshark filters"
echo "Show all conversation"
echo "====================="
tshark -o column.format:'"Source", "%s", "Destination", "%d", "dstport", "%uD"' -r $FILE |sort|uniq
# Show all TCP streams
echo "Show TCP Streams"
echo "================"
tshark -r $FILE -q -z conv,tcp
# Search for any word with user
echo "Show any packes with "User" in it"
echo "================================="
ngrep -q -I $FILE|grep -i user
# Search for any word with pass
echo "Show any words with "Pass" in it"
echo "================================"
ngrep -q -I $FILE|grep -i pass
# Search for emails
echo "Find anything looking like an email address"
echo "==========================================="
ngrep -q -I $FILE '[a-zA-Z0-9.]+\.?@[a-zA-Z0-9.]+\.[a-zA-Z0-9]+' |grep -Eo '[a-zA-Z0-9.]+\.?@[a-zA-Z0-9.]+\.[a-zA-Z0-9]+'|sort|uniq
# Look at TCP data in printable form
echo "Show Port 25 data"
echo "================="
tcpick -r $FILE -yP "port 25"
echo "Show Port 21 data"
echo "================="
tcpick -r $FILE -yP "port 21"
echo "Show Port 23 data"
echo "================="
tcpick -r $FILE -yP "port 23"
echo "Show Port 80 data"
echo "================="
tcpick -r $FILE -yP "port 80"
echo "Show all TCP flows"
echo "=================="
tcpick -r $FILE -C
