Trustwave SpiderLabs Uncovers Unique Cybersecurity Risks in Today's Tech Landscape. Learn More

Trustwave SpiderLabs Uncovers Unique Cybersecurity Risks in Today's Tech Landscape. Learn More

Services
Capture
Managed Detection & Response

Eliminate active threats with 24/7 threat detection, investigation, and response.

twi-managed-portal-color
Co-Managed SOC (SIEM)

Maximize your SIEM investment, stop alert fatigue, and enhance your team with hybrid security operations support.

twi-briefcase-color-svg
Advisory & Diagnostics

Advance your cybersecurity program and get expert guidance where you need it most.

tw-laptop-data
Penetration Testing

Test your physical locations and IT infrastructure to shore up weaknesses before exploitation.

twi-database-color-svg
Database Security

Prevent unauthorized access and exceed compliance requirements.

twi-email-color-svg
Email Security

Stop email threats others miss and secure your organization against the #1 ransomware attack vector.

tw-officer
Digital Forensics & Incident Response

Prepare for the inevitable with 24/7 global breach response in-region and available on-site.

tw-network
Firewall & Technology Management

Mitigate risk of a cyberattack with 24/7 incident and health monitoring and the latest threat intelligence.

Solutions
BY TOPIC
Microsoft Exchange Server Attacks
Stay protected against emerging threats
Rapidly Secure New Environments
Security for rapid response situations
Securing the Cloud
Safely navigate and stay protected
Securing the IoT Landscape
Test, monitor and secure network objects
Why Trustwave
About Us
Awards and Accolades
Trustwave SpiderLabs Team
Trustwave Fusion Security Operations Platform
Trustwave Security Colony
Partners
Technology Alliance Partners
Key alliances who align and support our ecosystem of security offerings
Trustwave PartnerOne Program
Join forces with Trustwave to protect against the most advance cybersecurity threats
SpiderLabs Blog

ModSecurity Advanced Topic of the Week: Mitigation of 'Slow Read' Denial of Service Attack

Slow-Read DoS Attack Background

Another tweak in the ongoing "Slow" DoS attacks has emerged this week. The previous Slow DoS attacks focuses on methods of slowing down the rate of request data sent to the target web server. This time, the attack centers on a method of slowing down the rate at which the client (attacker) is able to consume the response data sent back by the web server - hence the name "Slow Read" DoS.

It is rather serendipitous as I actually had the same attack scenario in mind for some time but never got around to develop working exploit code. So a hat tip to Sergey Shekyan as we have seen in the security space that theoretical exploits fall on deaf ears and that you need working proof of concept code to get people to listen.

For those of you familiar with the old LaBrea Tarpit app for slowing down network based worms, this is somewhat of a reverse approach. Instead of the defender (LaBrea) sending back a TCP Window size of 0 to the attacker (worm) which would force the TCP client to wait for a period of time before resubmitting, in this scenario the attacker is the one forcing the web server to wait. After sending in the request, the client responds with TCP window sizes that are much smaller than normal. This forces the web server to queue the response a break it up into smaller chunks that the client will accept.

Slow-Read DoS Attack Example

You can download the slowhttptest tool from the project site and then follow the steps on this page to test out a Slow-Read attack. Here is an example test command that I modified to more closely mimic the LaBrea concept as it uses TCP window sizes of only 1 or 2:

./slowhttptest -c 1000 -X -g -o slow_read_stats -r 200 -w 1 -y 2 -n 5 -z 32 -k 3 -u http://localhost/target_file -p 3 

When running this command against an Apache server, the tool shows this data:

$ ./slowhttptest -c 1000 -X -g -o slow_read_stats -r 200 -w 512 -y 1024 -n 5 -z 32 -k 3 -u http://localhost/target_file -p 3 Fri Jan  6 10:07:09 2012:set open files limit to 1010Fri Jan  6 10:07:09 2012:Using:test type:                        SLOW READnumber of connections:            1000URL:                              http://localhost/target_fileverb:                             GETreceive window range:             512 - 1024pipeline factor:                  3read rate from receive buffer:    32 bytes / 5 secconnections per seconds:          200probe connection timeout:         3 secondstest duration:                    240 secondsFri Jan  6 10:07:09 2012:slow HTTP test status on 0th second:initializing:        0pending:             1connected:           0error:               0closed:              0service available:   YESFri Jan  6 10:07:14 2012:slow HTTP test status on 5th second:initializing:        0pending:             541connected:           383error:               0closed:              0service available:   NOFri Jan  6 10:07:19 2012:slow HTTP test status on 10th second:initializing:        0pending:             617connected:           383error:               0closed:              0service available:   NO

You can see that after a 5 seconds, the server is no longer able to service new requests as all of the existing threads are stuck in a WRITE state.

The following tcpdump data shows where the tool changes the TCP window size to 1:

10:56:06.872895 IP (tos 0x0, ttl 64, id 5652, offset 0, flags [DF], proto TCP (6), length 64, bad cksum 0 (->a081)!)    192.168.1.105.http > 192.168.1.105.57011: Flags [S.], cksum 0x8455 (incorrect -> 0x6be3), seq 1528097122, ack 3885912325, win 65535, options [mss 16344,nop,wscale 1,nop,nop,TS val 317572454 ecr 317572454,sackOK,eol], length 0        0x0000:  4500 0040 1614 4000 4006 0000 c0a8 0169  E..@..@.@......i        0x0010:  c0a8 0169 0050 deb3 5b14 e962 e79e 5105  ...i.P..[..b..Q.        0x0020:  b012 ffff 8455 0000 0204 3fd8 0103 0301  .....U....?.....        0x0030:  0101 080a 12ed c566 12ed c566 0402 0000  .......f...f....10:56:06.872902 IP (tos 0x0, ttl 64, id 32564, offset 0, flags [DF], proto TCP (6), length 52, bad cksum 0 (->376d)!)    192.168.1.105.57010 > 192.168.1.105.http: Flags [.], cksum 0x8449 (incorrect -> 0x7a07), ack 1, win 65328, options [nop,nop,TS val 317572454 ecr 317572454], length 0        0x0000:  4500 0034 7f34 4000 4006 0000 c0a8 0169  E..4.4@.@......i        0x0010:  c0a8 0169 deb2 0050 6a91 b7b5 5fc3 67ad  ...i...Pj..._.g.        0x0020:  8010 ff30 8449 0000 0101 080a 12ed c566  ...0.I.........f        0x0030:  12ed c566                                ...f10:56:06.872906 IP (tos 0x0, ttl 64, id 1603, offset 0, flags [DF], proto TCP (6), length 52, bad cksum 0 (->b05e)!)    192.168.1.105.57011 > 192.168.1.105.http: Flags [.], cksum 0x8449 (incorrect -> 0xe6a1), ack 1, win 65328, options [nop,nop,TS val 317572454 ecr 317572454], length 0        0x0000:  4500 0034 0643 4000 4006 0000 c0a8 0169  E..4.C@.@......i        0x0010:  c0a8 0169 deb3 0050 e79e 5105 5b14 e963  ...i...P..Q.[..c        0x0020:  8010 ff30 8449 0000 0101 080a 12ed c566  ...0.I.........f        0x0030:  12ed c566                                ...f10:56:06.877862 IP (tos 0x0, ttl 64, id 42608, offset 0, flags [DF], proto TCP (6), length 64, bad cksum 0 (->1025)!)    192.168.1.105.57012 > 192.168.1.105.http: Flags [S], cksum 0x8455 (incorrect -> 0x4db9), seq 1017984763, win 1, options [mss 16344,nop,wscale 0,nop,nop,TS val 317572454 ecr 0,sackOK,eol], length 0        0x0000:  4500 0040 a670 4000 4006 0000 c0a8 0169  E..@.p@.@......i        0x0010:  c0a8 0169 deb4 0050 3cad 36fb 0000 0000  ...i...P<.6.....        0x0020:  b002 0001 8455 0000 0204 3fd8 0103 0300  .....U....?.....        0x0030:  0101 080a 12ed c566 0000 0000 0402 0000  .......f........

For further visual evidence, here is a screen shot of the Server-Status page right when the attack is stopped:

11403_b6ee0a9d-9b60-4c85-90fb-fb58a7e4e2c2
Notice that all of the threads are in a "W" Sending Reply state.

Mitigating Slow-Read DoS Attacks with ModSecurity

ModSecurity v2.6 introduced a new directive called SecWriteStateLimit, which will place a limit on the concurrent number of threads (per IP address) in a SERVER_BUSY_WRITE state. This was originally created to help mitigate the Slow Request Body DoS attacks as Apache moves these threads into this state when it is reading request body payloads. Well, as it turns out, when Apache is sending response body data back to a client, it too moves the threads into the SERVER_BUSY_WRITE state. This means that we can use the ModSecurity SecWriteStateLimit to set an upper threshold of concurrent threads (per IP address) in this state. If this threshold is met, then new threads over this limit will be terminated. The end result is that an attacker will not be able to tie up all available threads and other clients will be able to access the web server.

For testing purposes, the following ModSecurity directive was added to the configuration:

SecWriteStateLimit 100

The same slowhttptest test was then run and the Apache Server-Status page looked like this during the attack:

BSL_10707_95b79b0e-62c1-4eef-b1bb-28490b21c6ce

Notice that there is now a limit of ~100 concurrent connections (per IP address) in the W state and that the other threads are open and able to process new requests.

During the attack, the Apache error_log showed these error message from ModSecurity's SecLimitWriteState directive:

[Fri Jan 06 10:39:38 2012] [warn] ModSecurity: Access denied with code 400. Too many threads [101] of 100 allowed in WRITE state from 192.168.1.105 - Possible DoS Consumption Attack [Rejected][Fri Jan 06 10:39:38 2012] [warn] ModSecurity: Access denied with code 400. Too many threads [101] of 100 allowed in WRITE state from 192.168.1.105 - Possible DoS Consumption Attack [Rejected][Fri Jan 06 10:39:38 2012] [warn] ModSecurity: Access denied with code 400. Too many threads [101] of 100 allowed in WRITE state from 192.168.1.105 - Possible DoS Consumption Attack [Rejected][Fri Jan 06 10:39:38 2012] [warn] ModSecurity: Access denied with code 400. Too many threads [101] of 100 allowed in WRITE state from 192.168.1.105 - Possible DoS Consumption Attack [Rejected][Fri Jan 06 10:39:38 2012] [warn] ModSecurity: Access denied with code 400. Too many threads [101] of 100 allowed in WRITE state from 192.168.1.105 - Possible DoS Consumption Attack [Rejected][Fri Jan 06 10:39:38 2012] [warn] ModSecurity: Access denied with code 400. Too many threads [101] of 100 allowed in WRITE state from 192.168.1.105 - Possible DoS Consumption Attack [Rejected][Fri Jan 06 10:39:38 2012] [warn] ModSecurity: Access denied with code 400. Too many threads [101] of 100 allowed in WRITE state from 192.168.1.105 - Possible DoS Consumption Attack [Rejected][Fri Jan 06 10:39:38 2012] [warn] ModSecurity: Access denied with code 400. Too many threads [101] of 100 allowed in WRITE state from 192.168.1.105 - Possible DoS Consumption Attack [Rejected][Fri Jan 06 10:39:38 2012] [warn] ModSecurity: Access denied with code 400. Too many threads [101] of 100 allowed in WRITE state from 192.168.1.105 - Possible DoS Consumption Attack [Rejected]

Latest SpiderLabs Blogs

Zero Trust Essentials

This is Part 5 in my ongoing project to cover 30 cybersecurity topics in 30 weekly blog posts. The full series can be found here.

Read More

Why We Should Probably Stop Visually Verifying Checksums

Hello there! Thanks for stopping by. Let me get straight into it and start things off with what a checksum is to be inclusive of all audiences here, from Wikipedia [1]:

Read More

Agent Tesla's New Ride: The Rise of a Novel Loader

Malware loaders, critical for deploying malware, enable threat actors to deliver and execute malicious payloads, facilitating criminal activities like data theft and ransomware. Utilizing advanced...

Read More