Trustwave SpiderLabs Uncovers Ov3r_Stealer Malware Spread via Phishing and Facebook Advertising. Learn More

Trustwave SpiderLabs Uncovers Ov3r_Stealer Malware Spread via Phishing and Facebook Advertising. Learn More

Services
Capture
Managed Detection & Response

Eliminate active threats with 24/7 threat detection, investigation, and response.

twi-managed-portal-color
Co-Managed SOC (SIEM)

Maximize your SIEM investment, stop alert fatigue, and enhance your team with hybrid security operations support.

twi-briefcase-color-svg
Advisory & Diagnostics

Advance your cybersecurity program and get expert guidance where you need it most.

tw-laptop-data
Penetration Testing

Test your physical locations and IT infrastructure to shore up weaknesses before exploitation.

twi-database-color-svg
Database Security

Prevent unauthorized access and exceed compliance requirements.

twi-email-color-svg
Email Security

Stop email threats others miss and secure your organization against the #1 ransomware attack vector.

tw-officer
Digital Forensics & Incident Response

Prepare for the inevitable with 24/7 global breach response in-region and available on-site.

tw-network
Firewall & Technology Management

Mitigate risk of a cyberattack with 24/7 incident and health monitoring and the latest threat intelligence.

Solutions
BY TOPIC
Microsoft Exchange Server Attacks
Stay protected against emerging threats
Rapidly Secure New Environments
Security for rapid response situations
Securing the Cloud
Safely navigate and stay protected
Securing the IoT Landscape
Test, monitor and secure network objects
Why Trustwave
About Us
Awards and Accolades
Trustwave SpiderLabs Team
Trustwave Fusion Security Operations Platform
Trustwave Security Colony
Partners
Technology Alliance Partners
Key alliances who align and support our ecosystem of security offerings
Trustwave PartnerOne Program
Join forces with Trustwave to protect against the most advance cybersecurity threats
SpiderLabs Blog

How To Decrypt Ruby SSL Communications with Wireshark

Debugging a program that communicates with a remote endpoint usually involves analyzing the network communications. A common method is capturing the traffic using a packet analyzer tool such as tcpdump or Wireshark. However, this process can be tricky when the communication is encrypted. Our team, responsible for the Trustwave network vulnerability scanning system, regularly encounters this situation - especially when scanning systems we don't control (mostly those of customers).

In this blog post, I will explain how to decrypt SSL/TLS communications to allow for the analysis of that traffic with Wireshark. I will focus on Ruby and the binding for OpenSSL. Please note that the topic of this post is not methods for breaking crypto-systems. Instead, it's about how to retrieve key material for decryption.

Decryption using Wireshark

Wireshark is one of the more famous packet analyzers out there, and it's capable of decrypting SSL/TLS communications. All it needs is the Master Key used in encrypting the data.

Here is a high level explanation of how Wireshark can retrieve this key:

  1. First, Wireshark checks whether any cached session matches the Session ID or the Session Ticket from the Server Hello handshake message. If so, it retrieves the Master Key from the cached session.
  2. If the key exchange algorithm is PSK, you can setup the path to the clear text Pre-Shared Key that was used during the key exchange. Wireshark will use it to calculate the Master Key.
  3. If the key exchange algorithm is RSA, you can provide the server Private Key (in PEM format) that was used for encryption during the key exchange step. Wireshark will try to decrypt the encrypted Pre-Master Key, which is retrieved from the Client Key Exchange handshake message. If successful, it uses the decrypted Pre-Master Key to calculate the Master Key.
  4. Finally, you can provide a key log file that contains a list of Master Keys and/or Pre-Master Keys (used to calculate the Master Key - the same way explained above).

Only step four is relevant in our case. Step one is mostly an internal Wireshark process and only possible if a previous session with the correct key material has been cached. This isn't relevant to our objective. We want to decrypt new SSL sessions. Steps two and three are limited to PSK and RSA key-exchange algorithms, which also require keys from the remote server (hard to capture in the context of vulnerability scanning).

The key log file

Without going deeper in the cryptographic process, Wireshark can decrypt the SSL communication using the Master Key exchanged during the handshake. Since a network capture (pcap) will likely contain many SSL sessions, it needs to be able to map the key with the corresponding SSL traffic. To do so, it provides a unique identifier for each key you put in the key log file. Each time a new SSL session begins, Wireshark will review the key log file entries to look for the identifier corresponding to the current session. If the identifier is found, it retrieves the corresponding key and decrypts the whole session.

The general format is "<id> <key>". Here is the definition from Wireshark source code documentation:

  1. "RSA xxxx yyyy": Where xxxx are the first eight bytes of the encrypted Pre-Master Secret (hex-encoded) and yyyy is the cleartext Pre-Master Secret (hex-encoded).
  2. "RSA Session-ID:xxxx Master-Key:yyyy": Where xxxx is the SSL session ID (hex-encoded) and yyyy is the cleartext Master Secret (hex-encoded).
  3. "CLIENT_RANDOM xxxx yyyy": Where xxxx is the Client Random from the ClientHello (hex-encoded) and yyyy is the cleartext Master Secret (hex-encoded).

For the first item, you need to provide the first eight bytes of the encrypted Pre-Master Secret, which will serve as an identifier.

In that case, Wireshark will do the following: extract the encrypted Pre-Master Secret from the Client Key Exchange handshake message in the pcap, look for the first eight bytes in the key log file, retrieve the plain text Pre-Master Secret and finally calculate the Master Key. The only issue with this method is it doesn't work with the Diffie-Hellman (DH) key exchange algorithm because the Pre-Master Secret is not kept anywhere.

The last two options work similarly: a unique identifier followed by the Master Key. One uses the Session ID ("RSA Session-ID: …" format) and the other the Client Random value ("CLIENT_RANDOM …" format). According to RFC 5246, the Session ID is "a value generated by a server that identifies a particular session" and the Client Random is "a 32-byte value provided by the client". The Session ID is used to resume a cached session and the Client Random is part of the key material needed to compute the Master Key (along with the Server Random and the Pre-Master Secret). Since both are generated for each SSL session, they can be used as an identifier.

Setting up Wireshark

You will find the SSL-related configuration in Edit menu under Preferences/Protocols (on the left hand side)/SSL (after expanding the tree menu).

10687_94ab5c47-d815-45fa-b168-126559631224

Here are the interesting fields:

  • RSA keys list: this is where you can provide server private keys to decrypt the Pre-Master Secrets, but only if it was encrypted with the server public key (i.e., via the RSA key exchange algorithm).

  • SSL debug file: you can provide the path of a text file that will be used by Wireshark to output useful debugging information when decrypting SSL (recommended for troubleshooting).

  • Pre-Shared-Key: here is where the plain text Pre-Shared Key goes... duh! (but only when the PSK key exchange algorithm is used).

  • (Pre)-Master-Secret log filename: this is where the key log file path goes which we discussed in the previous section. We will go into more detail later.

Export Session ID in Ruby file

On the client side, a basic Ruby SSL connection is usually initiated as follows:

require 'socket'
require 'openssl'
socket = TCPSocket.open("192.168.1.214", 443)
ssl_context = OpenSSL::SSL::SSLContext.new
ssl_socket = OpenSSL::SSL::SSLSocket.new(socket, ssl_context)
ssl_socket.connect

Here we are using the default SSL context, and we connect to the port 443/TCP on the remote host (192.168.1.241).

Ruby offers SSL session information in three formats: DER, PEM and plaintext. The output of the #to_text method gives you the same output that "openssl s_client" will display after a successful connection:

> puts ssl_socket.session.to_text
SSL-Session:
Protocol : TLSv1
Cipher : AES256-SHA
Session-ID: EF306F66879F1FDE10EEA29E3D30BED1AED6A6894B7439A7F21829DE9DD2D9CB
Session-ID-ctx:
Master-Key: E6822B0C718654FDA14F2635689DD586D52AC686A0FFA31F6D3B309DEE5B9B8708107ED450C25E963DDCFE57F64CEF2E
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 300 (seconds)
TLS session ticket:
0000 - 92 79 20 40 4b ce 7b 9b-09 8b 82 40 8b 28 1e 7c .y @K.{....@.(.|
0010 - c5 08 f6 70 ac 99 30 c7-36 bc 48 dd 20 23 b0 b9 ...p..0.6.H. #..
0020 - 4d 40 40 a6 ed a6 e1 6a-75 2b b4 2a fb c5 72 01 M@@....ju+.*..r.
0030 - 15 fe 05 fa e2 70 db 37-c9 c6 ac 30 63 c0 6e b2 .....p.7...0c.n.
0040 - a4 84 2e 22 e0 e3 67 3f-9e 00 53 0f 21 76 d2 f5 ..."..g?..S.!v..
0050 - eb 72 c0 78 37 8a 4e 8b-1b 95 ee 12 80 0d 93 fb .r.x7.N.........
0060 - 54 73 90 bc bc 94 2d 23-7a 0b b2 e2 ba 3d 61 ec Ts....-#z....=a.
0070 - 45 06 4f bc c2 00 b0 67-ea 71 79 65 a3 5f 53 98 E.O....g.qye._S.
0080 - 05 ce 5f 96 5c c4 f7 55-30 22 b9 59 57 0f 01 cc .._.\..U0".YW...
0090 - 77 ed 54 8d eb 74 27 a8-3c bd 69 e0 bb c6 ad a7 w.T..t'.

As can be seen, the Session ID and the Master Key are displayed and already in the correct format. This is all we need to fill our key log file that Wireshark will use to decrypt this communication.

Here is a simple proof of concept:

socket = TCPSocket.open("192.168.1.214", 443)
ssl_context = OpenSSL::SSL::SSLContext.new
ssl_socket = OpenSSL::SSL::SSLSocket.new(socket, ssl_context)
ssl_socket.connect

session_id = ""
master_key = ""
ssl_socket.session.to_text.each_line do |line|
if match = line.match(/Session-ID\s*: (?<session_id>.*)/)
session_id = match[:session_id]
end
if match = line.match(/Master-Key\s*: (?<master_key>.*)/)
master_key = match[:master_key]
end
end

data = "GET / HTTP/1.1\r\n"
data << "Host: #{host}\r\n"
data << "Random: Foo/#{rand(36**10).to_s(36)}\r\n"
data << "\r\n"

ssl_socket.puts(data)
puts "Request:"
puts data
puts "----------- Response: #{ssl_socket.gets()}"

ssl_socket.close

File.open("keys_dump_ruby", "a") do |file|
file.write("RSA Session-ID:#{session_id} Master-Key:#{master_key}\n")
end

Here we'll run this and capture the traffic with Wireshark or tcpdump:

$ ruby ssl_key_export_session-ID_only.rb
Request:
GET / HTTP/1.1
Host: 192.168.1.214
Random: Foo/cl5dx8et9r

----------- Response: HTTP/1.1 200 OK
$ cat keys_dump_ruby
RSA Session-ID:DA6DDA7502BE043CE9ACAD482C994CCEF9282F6CAE7DCF48BC7314952E9D1F6B Master-Key:89C9F116F9C4778C234100858EFC05CC5E1F58D155E5A94CD10D4CF08D4473341019F1115B95820C2C3C7EE311BD5D5D

As expected, the traffic appears encrypted when opening the pcap with Wireshark:

11760_c8741109-ad67-445a-8a5a-e20ee454e597

Here we'll configure the key log file path in the SSL preferences and look at the packets again:

10114_7afd30c9-96a3-46d8-b73b-65a751a4129d

9720_6892cfc5-5a65-41ee-b0dd-1cc1412fc905

A new tab appears at the bottom of the window ("Decrypted SSL data"), which shows the data in the Packet Bytes section.

You can also display the full decrypted SSL session by right clicking on an SSL frame and selecting "Follow SSL Stream":

10243_7f6f4040-4d92-4d17-8cc1-58b592948b7d

8473_2aec1c5d-7cd5-43ff-bdf2-20b5cef69465

This is awesome so far, but we can increase the awesomeness by telling Wireshark which dissectors should be used to parse the decrypted data. For that, you need to use the first field of the SSL preferences: "RSA keys list." As we saw, this is the field where you list the server private keys to decrypt communications using the RSA key exchange algorithm. We are going to use it anyway to create an association and inform Wireshark which protocol needs to be displayed. I know, it is a little odd, but it is the only way I found to make it work.

Click on Edit, New and fill the form with the remote host IP address, the port number and the protocol to use (HTTP in this case). You also need to provide a private key in PEM format. Any key will do the job; it just has to be a private key in the right format. I have no idea why it has been designed this way, but it won't work without it. You can use the openssl command to generate a fake key, and it will work just fine: openssl genrsa -out private.pem

8513_2d51ab96-8df9-4d0f-9d91-549b2ddd355c

Now our packets are correctly parsed:

11384_b645c9ad-5333-4d61-bb2c-1e1fddde3ddb

Export Client Random in Ruby file

This method works great, but sometimes the server may not return a Session ID. Usually, the server being configured to use Session Tickets is to blame:

8051_14ddd0c7-a9cb-4e9d-97e8-db396fac58fd

As can be seen, the Session ID length is 0 in the Server Hello handshake message, and Wireshark won't be able to find the correct Master Key.

Each time a new session is initiated, both the client and the server create a random value as defined in RFC 5246. The client sends this value during the first step of the SSL handshake (Client Hello). Since this value is unique and generated for each session, Wireshark can also use it to identify the Master Key in the key log file:

12847_fc19c447-3390-4825-a810-66392bd51fa2

Unfortunately, the Ruby binding for OpenSSL does not expose this value to the end user and, unless we patch the native code (a topic for another post), it requires a little bit of work to retrieve the Client Random. I used ffi-pcap, a Ruby FFI binding for libpcap, to extract the value directly from a network capture in the Ruby code.

Here is the how I did that (I just added the changes in the code of the last section for simplicity):

...[SNIP]...
require 'ffi/pcap'
...[SNIP]...
pcap = FFI::PCap::Live.new(:device => "en0",
:timeout => 1,
:handler => nil)
pcap.setfilter("dst host 192.168.1.214 and tcp dst port 443 and tcp[13]&8!=0 and tcp[32]==22 and tcp[37]==1")
pcap.non_blocking = true

ssl_socket.connect # 1) do |this, pkt|
pkt.body[77,32].each_byte { |byte| client_random << "%02X" % byte.ord }
end
...[SNIP]...
ssl_socket.close
pcap.close

File.open("keys_dump_ruby", "a") do |file|
file.write("CLIENT_RANDOM #{client_random} #{master_key}\n")
file.write("RSA Session-ID:#{session_id} Master-Key:#{master_key}\n")
end

The pcap filter will only capture outbound packets on port 443/TCP (dst host 192.168.1.214 and tcp dst port 443) with the PSH flag (tcp[13]&8!=0), SSL Handshake Content type (tcp[32]==22) and Client Hello Handshake Type (tcp[37]==1).

After calling OpenSSL#connect, the #dispatch block will extract the 32-byte Client Random (RFC5246 A.6) directly from the captured packet.

Let's try it now:

$ ruby ./ssl_key_export_session-ID_and_client_random.rb
Request:
GET / HTTP/1.1
Host: 192.168.1.214
Random: Foo/wet78yp4zs

----------- Response: HTTP/1.1 200 OK
$ cat keys_dump_ruby
CLIENT_RANDOM 38AB4EBBDE18601C0558EF9F8E8B00CBFB745D1F6BD01AA08A433F5C9E87AD22 12EA2924180E97C94CE8729A0AB178EE95EE8AEBADEC96A42A2016AAA7D3C330E46B37178137D3FFFA2118A295475A6D
RSA Session-ID:7C0DD8780BF5445B03A35BEEF47AD3F612FD6373F3002BED2F5766F01314923C Master-Key:12EA2924180E97C94CE8729A0AB178EE95EE8AEBADEC96A42A2016AAA7D3C330E46B37178137D3FFFA2118A295475A6D

We are now able to decrypt the SSL communication again:

11863_cd1545d6-8c54-40fa-8425-b6b469e75cf9

Bonus: Monkey patching Ruby binding of OpenSSL

As a bonus, you can monkey patch the OpenSSL#connect method to be used with any Ruby code that uses OpenSSL.

require 'openssl'
require 'ffi/pcap'

module SSLKeyExport
def connect
pcap = FFI::PCap::Live.new(:device => "en0",
:timeout => 1,
:handler => nil)
# filtering on only tcp packets with PSH flag, SSL Handshake Content type and the Client Hello Handshake Type
pcap.setfilter("dst host 192.168.1.214 and tcp dst port 443 and tcp[13]&8!=0 and tcp[32]==22 and tcp[37]==1")
pcap.non_blocking = true

super

client_random = ""
pcap.dispatch(:count => 1) do |this, pkt|
pkt.body[77,32].each_byte { |byte| client_random << "%02X" % byte.ord }
end
pcap.close

session_id = ""
master_key = ""
self.session.to_text.each_line do |line|
if match = line.match(/Session-ID\s*: (?<session_id>.*)/)
session_id = match[:session_id]
end
if match = line.match(/Master-Key\s*: (?<master_key>.*)/)
master_key = match[:master_key]
end
end

File.open("keys_dump_ruby", "a") do |file|
file.write("CLIENT_RANDOM #{client_random} #{master_key}\n") unless client_random.empty?
file.write("RSA Session-ID:#{session_id} Master-Key:#{master_key}\n") unless session_id.empty?
end
end
end

class OpenSSL::SSL::SSLSocket
prepend SSLKeyExport
end

You just need to require this code using "-r":

ruby -r ./ssl_key_export_monkey_patch ./ssl_http_get.rb

Parting Thoughts

Thanks to this great Wireshark feature, you can now decrypt the encrypted network traffic generated by your Ruby code. This technique can be easily adapted to other languages too. Another approach would be to create a library that hooks the low level OpenSSL functions directly and dump the Session ID, Client Random and Master Key to a file. You would need to load your library before OpenSSL is loaded by using the LD_PRELOAD environment variable for example. Here is an implementation of that technique: https://git.lekensteyn.nl/peter/wireshark-notes/tree/src/sslkeylog.c.

Latest SpiderLabs Blogs

Welcome to Adventures in Cybersecurity: The Defender Series

I’m happy to say I’m done chasing Microsoft certifications (AZ104/AZ500/SC100), and as a result, I’ve had the time to put some effort into a blog series that hopefully will entertain and inform you...

Read More

Trustwave SpiderLabs: Insights and Solutions to Defend Educational Institutions Against Cyber Threats

Security teams responsible for defending educational institutions at higher education and primary school levels often find themselves facing harsh lessons from threat actors who exploit the numerous...

Read More

Breakdown of Tycoon Phishing-as-a-Service System

Just weeks after Trustwave SpiderLabs reported on the Greatness phishing-as-a-service (PaaS) framework, SpiderLabs’ Email Security team is tracking another PaaS called Tycoon Group.

Read More