Trustwave SpiderLabs Uncovers Unique Cybersecurity Risks in Today's Tech Landscape. Learn More

Trustwave SpiderLabs Uncovers Unique Cybersecurity Risks in Today's Tech Landscape. Learn More

Services
Capture
Managed Detection & Response

Eliminate active threats with 24/7 threat detection, investigation, and response.

twi-managed-portal-color
Co-Managed SOC (SIEM)

Maximize your SIEM investment, stop alert fatigue, and enhance your team with hybrid security operations support.

twi-briefcase-color-svg
Advisory & Diagnostics

Advance your cybersecurity program and get expert guidance where you need it most.

tw-laptop-data
Penetration Testing

Test your physical locations and IT infrastructure to shore up weaknesses before exploitation.

twi-database-color-svg
Database Security

Prevent unauthorized access and exceed compliance requirements.

twi-email-color-svg
Email Security

Stop email threats others miss and secure your organization against the #1 ransomware attack vector.

tw-officer
Digital Forensics & Incident Response

Prepare for the inevitable with 24/7 global breach response in-region and available on-site.

tw-network
Firewall & Technology Management

Mitigate risk of a cyberattack with 24/7 incident and health monitoring and the latest threat intelligence.

Solutions
BY TOPIC
Microsoft Exchange Server Attacks
Stay protected against emerging threats
Rapidly Secure New Environments
Security for rapid response situations
Securing the Cloud
Safely navigate and stay protected
Securing the IoT Landscape
Test, monitor and secure network objects
Why Trustwave
About Us
Awards and Accolades
Trustwave SpiderLabs Team
Trustwave Fusion Security Operations Platform
Trustwave Security Colony
Partners
Technology Alliance Partners
Key alliances who align and support our ecosystem of security offerings
Trustwave PartnerOne Program
Join forces with Trustwave to protect against the most advance cybersecurity threats
SpiderLabs Blog

Optimizing Regular Expressions

As many of you have noticed, the Core Rule Set contains very complex regular expressions. For example:

(?:\b(?:(?:s(?:elect\b(?:.{1,100}?\b(?:(?:length|count|top)\b.{1,100}
?\bfrom|from\b.{1,100}?\bwhere)|.*?\b(?:d(?:ump\b.*\bfrom|ata_type)|
(?:to_(?:numbe|cha)|inst)r))|p_(?:(?:addextendedpro|sqlexe)c|...

These regular expressions are assembled from a list of simpler regular expressions for efficiency reasons. A single optimized regular expression test takes much less time than a series of simpler regular expression tests. The downside is readability and ease of editing. A future version of ModSecurity will overcome this limitation, but meanwhile, in order to optimize performance you have to think about optimization yourself.

Manual assembly and optimization is both hard and error prone, so for the Core Rule Set we use a clever Perl Module: Regexp::Assemble. As the name suggests, Regexp::Assemble knows how to assemble a number of regular expressions into one optimized regular expression.

Since Regexp::Assemble is not a program, but rather a Perl module, you will need some glue code to use it. The following instructions will help you if you are not Perl Wizards.

If you don't have Perl, you will need to install it. The easiest Perl distribution to install, especially if you use Windows, is ActivePerl.

Now install Regexp::Assemble. If you used ActivePerl, you can use the following command:

ppm install regexp-assemble

If you use another Perl distribution, you will need to download the module and use the normal Perl module installation procedure as outlined in the README file.

Once you have Perl and Regexp::Assemble installed, all you need is this little script:

#!/usr/local/bin/perl
use strict;
use Regexp::Assemble;

my $ra = Regexp::Assemble->new;
while (<>)
{
$ra->add($_);
}
print $ra->as_string() . "\n";

The script will take either standard input or an input file with each line containing a regular expression and print out the optimized expression:

regexp_builder.pl simple_regexps.txt > optimized_regexp.txt

On a Unix system you might need to change the fist line to point to the local Perl interpreter. On Windows you may need to precede the script name with the command 'perl'.

And if all this is too complex, you can just download the pre-compiled version for Windows.

Latest SpiderLabs Blogs

Zero Trust Essentials

This is Part 5 in my ongoing project to cover 30 cybersecurity topics in 30 weekly blog posts. The full series can be found here.

Read More

Why We Should Probably Stop Visually Verifying Checksums

Hello there! Thanks for stopping by. Let me get straight into it and start things off with what a checksum is to be inclusive of all audiences here, from Wikipedia [1]:

Read More

Agent Tesla's New Ride: The Rise of a Novel Loader

Malware loaders, critical for deploying malware, enable threat actors to deliver and execute malicious payloads, facilitating criminal activities like data theft and ransomware. Utilizing advanced...

Read More