Trustwave's 2024 Financial Services Threat Reports Highlight Alarming Trends in Insider Threats & Phishing-as-a-Service. Learn More

Trustwave's 2024 Financial Services Threat Reports Highlight Alarming Trends in Insider Threats & Phishing-as-a-Service. Learn More

Services
Managed Detection & Response

Eliminate active threats with 24/7 threat detection, investigation, and response.

Co-Managed SOC (SIEM)

Maximize your SIEM investment, stop alert fatigue, and enhance your team with hybrid security operations support.

Advisory & Diagnostics

Advance your cybersecurity program and get expert guidance where you need it most.

Penetration Testing

Test your physical locations and IT infrastructure to shore up weaknesses before exploitation.

Database Security

Prevent unauthorized access and exceed compliance requirements.

Email Security

Stop email threats others miss and secure your organization against the #1 ransomware attack vector.

Digital Forensics & Incident Response

Prepare for the inevitable with 24/7 global breach response in-region and available on-site.

Firewall & Technology Management

Mitigate risk of a cyberattack with 24/7 incident and health monitoring and the latest threat intelligence.

Solutions
BY TOPIC
Microsoft Security
Unlock the full power of Microsoft Security
Offensive Security
Solutions to maximize your security ROI
Rapidly Secure New Environments
Security for rapid response situations
Securing the Cloud
Safely navigate and stay protected
Securing the IoT Landscape
Test, monitor and secure network objects
Why Trustwave
About Us
Awards and Accolades
Trustwave SpiderLabs Team
Trustwave Fusion Security Operations Platform
Trustwave Security Colony
Partners
Technology Alliance Partners
Key alliances who align and support our ecosystem of security offerings
Trustwave PartnerOne Program
Join forces with Trustwave to protect against the most advance cybersecurity threats

Windows Debugging & Exploiting Part 3: WinDBG Time Travel Debugging

Introduction

Hi, my fellow friends! How are you? Hopefully, you had a terrific holiday and much love for everyone! Time to start 2020? No better time for writing about the TTD (Time Travel Debugging) feature from WinDBGFor those that are not familiar with this series, you can check the first part where I set up the environment for Windows remote kernel debugging and also the second part where I wrote about some basic commands on WinDBG. This will be a quick post but since this is a very useful resource on the tool, I thought indispensable for our introduction to Windows exploitation.

Overview

As the name suggests, Microsoft implemented a feature on the WinDBG Preview that allows the debugger to travel in time. From Microsoft:

"Time Travel Debugging, is a tool that allows you to record an execution of your process running, then replay it later both forwards and backwards. Time Travel Debugging (TTD) can help you debug issues easier by letting you "rewind" your debugger session, instead of having to reproduce the issue until you find the bug."

So what that mean? Normally we can start debugging an application and from there we are able to set breakpoints and then stopping the execution for analysis, but we don't really know where it ends. Utilizing this utility, the entire execution will be recorded (in a file on the disk) and we will be able to browse at the execution from the end to the beginning. 

For example, if there is a crash on the execution, you will be able to navigate on the execution to discover what was wrong and caused the unexpected stop from the crash. Normally we can identify the last function executed before crashing, however, it does not mean that this execution is the root of the problem. Utilizing the TTD, we can go backwards and check where the data that caused the issue came from.

Demonstration

Using the binary from the H2HC CTF that we already mentioned in the previous post to work with TTD - actually, I did use the TTD for solving this challenge.

Start the WinDBG as Administrator (mandatory for using TTD), go to Start debugging and then Launch executable (advanced):

 

Picture1

Figure 1 - Launching the executable

 

Browse where the binary is and mark the box "Record with Time Travel Debugging". It will be asked where you want to save your recording.

 

Picture2

Figure 2 - Location to save the TTD file

 

You could also attach to an existing process and enable the TTD.

Launching the H2HC binary challenge:

[+] H2HC - 16th Edition challenge
[+] Server listening
[+] Waiting for H2HC evil connections

I will let the reverse engineering for another blog post, for now, you need to know that this service is waiting for a connection on port 54345/TCP. Using the right data, we will crash the binary and then we can click on "Stop and Debug". Now we can open the file (Start debugging > Open trace file) and we will have the entire execution flow for our analysis.

Microsoft (R) Windows Debugger Version 10.0.19528.1000 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.

Loading Dump File [C:\Users\mphx2\Documents\h2hc01.run]
JavaScript script successfully loaded from 'C:\Program Files\WindowsApps\Microsoft.WinDbg_1.1912.11001.0_neutral__8wekyb3d8bbwe\amd64\TTD\Analyzers\HeapAnalysis.js'

JavaScript script successfully loaded from 'C:\Program Files\WindowsApps\Microsoft.WinDbg_1.1912.11001.0_neutral__8wekyb3d8bbwe\amd64\TTD\Analyzers\TtdAnalyze.js'

************* Path validation summary **************
Response                         Time (ms) Location
Deferred                                       srv*
Symbol search path is: srv*
Executable search path is: 
ModLoad: 00007ff7`3f9f0000 00007ff7`3fa04000   C:\Users\mphx2\Desktop\H2HC_CTF_2019\xpl_to_distribute\h2hc.exe
ModLoad: 00007ffc`ac840000 00007ffc`aca1a000   C:\ProgramData\Microsoft\Windbg\1-1912-11001\TTD\TTDRecordCPU.dll
ModLoad: 00007ffc`cb970000 00007ffc`cb984000   C:\Program Files\AVAST Software\Avast\aswhook.dll
ModLoad: 00007ffc`dd6e0000 00007ffc`dd76f000   C:\WINDOWS\SYSTEM32\apphelp.dll
ModLoad: 00007ffc`df750000 00007ffc`df9f3000   C:\WINDOWS\System32\KERNELBASE.dll
ModLoad: 00007ffc`e1280000 00007ffc`e13a0000   C:\WINDOWS\System32\RPCRT4.dll
ModLoad: 00007ffc`e1e30000 00007ffc`e1ee2000   C:\WINDOWS\System32\KERNEL32.DLL
ModLoad: 00007ffc`e2310000 00007ffc`e237f000   C:\WINDOWS\System32\WS2_32.dll
ModLoad: 00007ffc`e2540000 00007ffc`e2730000   C:\WINDOWS\SYSTEM32\ntdll.dll
.........
(1f20.fc0): Break instruction exception - code 80000003 (first/second chance not available)
Time Travel Position: F:0 [Unindexed] Index
!index
Indexed 2/2 keyframes
Successfully created the index in 379ms.

Microsoft (R) Windows Debugger Version 10.0.19528.1000 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.

Loading Dump File [C:\Users\mphx2\Documents\h2hc01.run]
JavaScript script successfully loaded from 'C:\Program Files\WindowsApps\Microsoft.WinDbg_1.1912.11001.0_neutral__8wekyb3d8bbwe\amd64\TTD\Analyzers\HeapAnalysis.js'

JavaScript script successfully loaded from 'C:\Program Files\WindowsApps\Microsoft.WinDbg_1.1912.11001.0_neutral__8wekyb3d8bbwe\amd64\TTD\Analyzers\TtdAnalyze.js'

************* Path validation summary **************
Response                         Time (ms) Location
Deferred                                       srv*
Symbol search path is: srv*
Executable search path is: 
ModLoad: 00007ff7`3f9f0000 00007ff7`3fa04000   C:\Users\mphx2\Desktop\H2HC_CTF_2019\xpl_to_distribute\h2hc.exe
ModLoad: 00007ffc`ac840000 00007ffc`aca1a000   C:\ProgramData\Microsoft\Windbg\1-1912-11001\TTD\TTDRecordCPU.dll
ModLoad: 00007ffc`cb970000 00007ffc`cb984000   C:\Program Files\AVAST Software\Avast\aswhook.dll
ModLoad: 00007ffc`dd6e0000 00007ffc`dd76f000   C:\WINDOWS\SYSTEM32\apphelp.dll
ModLoad: 00007ffc`df750000 00007ffc`df9f3000   C:\WINDOWS\System32\KERNELBASE.dll
ModLoad: 00007ffc`e1280000 00007ffc`e13a0000   C:\WINDOWS\System32\RPCRT4.dll
ModLoad: 00007ffc`e1e30000 00007ffc`e1ee2000   C:\WINDOWS\System32\KERNEL32.DLL
ModLoad: 00007ffc`e2310000 00007ffc`e237f000   C:\WINDOWS\System32\WS2_32.dll
ModLoad: 00007ffc`e2540000 00007ffc`e2730000   C:\WINDOWS\SYSTEM32\ntdll.dll
.........
(1f20.fc0): Break instruction exception - code 80000003 (first/second chance not available)
Time Travel Position: F:0ntdll!LdrInitializeThunk:
00007ffc`e25b17f0 4053            push rbx
0:000>

 

It starts at the beginning of the execution, then we can use all the commands within WinDBG adding the resources from Time Travel Debugging. Going backwards we can use the same commands for going forwards as (g, t, p, etc) adding a hyphen. Going to the end of the execution:

 

0:000> g
ModLoad: 00007ffc`dec20000 00007ffc`dec87000   C:\WINDOWS\system32\mswsock.dll
(1f20.fc0): Access violation - code c0000005 (first/second chance not available)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
Time Travel Position: 979:0
00000000`00000000 ??              ???

 

We can see that the execution finishes at a NULL pointer. From the end of the flow now we can move back and analyze:

 

0:000> t-
Time Travel Position: 951:24
00000000`00000000 ??              ???
0:000> 
*** WARNING: Unable to verify checksum for h2hc.exe
Time Travel Position: 951:23
h2hc+0x1476:
00007ff7`3f9f1476 c3              ret
0:000> 

Time Travel Position: 951:22
h2hc+0x146f:
00007ff7`3f9f146f 4881c448010000  add rsp,148h
0:000> dqs rsp+0x148
00000000`001ff978  00000000`00000000
00000000`001ff980  00000000`0000010c
00000000`001ff988  00000000`00000100
[..]

At H2HC+0x146f, it will be added 0x148 bytes to the register RSP and this address will be the return point, now we know where it will be directed to the NULL pointer that crashes our binary. We can also identify the stack address 00000000`001ff978. This is where the problem happens, let's keep our attention on this address:

 

0:000> t-
Time Travel Position: 951:21
h2hc+0x1098:
00007ff7`3f9f1098 c3              ret

0:000> 
Time Travel Position: 951:20
h2hc+0x1091:
00007ff7`3f9f1091 4881c418020000  add rsp,218h

0:000> 
Time Travel Position: 951:1F
h2hc+0x108b:
00007ff7`3f9f108b 880523d50000    mov byte ptr [h2hc+0xe5b4 (00007ff7`3f9fe5b4)],al ds:00007ff7`3f9fe5b4=00

0:000> 
Time Travel Position: 951:1E
h2hc+0x1089:
00007ff7`3f9f1089 fec8            dec al

0:000> 
Time Travel Position: 951:1D
h2hc+0x1082:
00007ff7`3f9f1082 0fb6052bd50000  movzx eax,byte ptr [h2hc+0xe5b4 (00007ff7`3f9fe5b4)] ds:00007ff7`3f9fe5b4=00

0:000> 
Time Travel Position: 951:1C
h2hc+0x107e:
00007ff7`3f9f107e 488948f8        mov qword ptr [rax-8],rcx ds:00000000`001ff978=0000000000000000

0:000> dqs rcx
00000000`00000000  ????????`????????

 

Here the application is moving the data from RCX (0x0) to the address at RAX-0x8, the noted address in the stack. Now we know that the NULL pointer comes from this instruction.

To debug furthermore, we still can go backwards:

 

0:000> t-
Time Travel Position: 951:1B
h2hc+0x107a:
00007ff7`3f9f107a 488b0c24        mov rcx,qword ptr [rsp] ss:00000000`001ff610=0000000000000000

0:000> t-
Time Travel Position: 951:1A
h2hc+0x1072:
00007ff7`3f9f1072 488b842420020000 mov     rax,qword ptr [rsp+220h] ss:00000000`001ff830=00000000001ff980

0:000> 
Time Travel Position: 951:19
h2hc+0x106e:
00007ff7`3f9f106e 48890424        mov qword ptr [rsp],rax ss:00000000`001ff610=0000000000000000

0:000> 
Time Travel Position: 951:18
h2hc+0x106a:
00007ff7`3f9f106a 488b04c1        mov rax,qword ptr [rcx+rax*8] ds:00007ff7`3f9fe5b8=0000000000000000

0:000> dqs 00007ff7`3f9fe5b8
00007ff7`3f9fe5b8  00000000`00000000
00007ff7`3f9fe5c0  00007ff7`3f9f1a6b h2hc+0x1a6b
00007ff7`3f9fe5c8  00007ff7`3f9f165d h2hc+0x165d

 

So observing the data, we can see that the NULL pointer really originated at 00007ff7`3f9fe5b8.

For testing purposes, if we breakpoint h2hc+0x1063 and modify the data on the specific address:

 

Breakpoint 0 hit
h2hc+0x1063:
00007ff7`3f9f1063 488d0d4ed50000 lea rcx,[h2hc+0xe5b8 (00007ff7`3f9fe5b8)]
0:000> eq 00007ff7`3f9fe5b8 0x4141414141414141

 

And we continue the execution:

 

0:000> g
(1418.2bc8): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
h2hc+0x1476:
00007ff7`3f9f1476 c3 ret

 

We have a different exception and checking the stack frame:

 

00 41414141`41414141 : 00000000`000000d8 00000000`00000100 00000000`00eff3d8 00000000`00eff450 : h2hc+0x1476
01 00000000`000000d8 : 00000000`00000100 00000000`00eff3d8 00000000`00eff450 00000000`00000008 : 0x41414141`41414141
02 00000000`00000100 : 00000000`00eff3d8 00000000`00eff450 00000000`00000008 00000100`43483248 : 0xd8

 

The return point is now our defined value.

It needs more debugging to discover the real root of the problem, but we already can discriminate the data at the execution. I won't solve this here, because I will utilize an entire post for that in the future.

Conclusion

As demonstrated, this tool could be a very useful and important tool during an analysis. This also could be very helpful for a team contribution, since you can share the saved file with anyone for analysis. The function is pretty straight-forward and no much mystery involved, but if you have any questions, let me know! 

ABOUT TRUSTWAVE

Trustwave is a globally recognized cybersecurity leader that reduces cyber risk and fortifies organizations against disruptive and damaging cyber threats. Our comprehensive offensive and defensive cybersecurity portfolio detects what others cannot, responds with greater speed and effectiveness, optimizes client investment, and improves security resilience. Learn more about us.

Latest Intelligence

Discover how our specialists can tailor a security program to fit the needs of
your organization.

Request a Demo