This week's feature is the effective use of Real-time Blacklist lookups (@rbl).

rbl

Description: Look up the parameter in the RBL given as parameter. Parameter can be an IPv4 address, or a hostname.

Example:

SecRule REMOTE_ADDR "@rbl sc.surbl.org"

OWASP ModSecurity CRS

The OWASP ModSecurity CRS includes limited use of the @rbl operator within the optional_rules/modsecurity_crs_42_comments_spam.conf file:

# # Comment spam is an attack against blogs, guestbooks, wikis and other types of # interactive web sites that accept and display hyperlinks submitted by # visitors. The spammers automatically post specially crafted random comments # which include links that point to the spammer's web site. The links # artificially increas the site's search engine ranking and may make the site # more noticable in search results. #  SecRule IP:PREVIOUS_RBL_CHECK "@eq 1" "phase:1,t:none,pass,nolog,skipAfter:END_RBL_LOOKUP"  SecRule REMOTE_ADDR "@rbl sbl-xbl.spamhaus.org" "phase:1,t:none,pass,nolog,auditlog,msg:'RBL Match for SPAM Source',tag:'AUTOMATION/MALICIOUS',severity:'2',setvar:'tx.msg=%{rule.msg}',setvar:tx.automation_score=+%{tx.warning_anomaly_score},setvar:tx.anomaly_score=+%{tx.warning_anomaly_score},setvar:tx.%{rule.id}-AUTOMATION/MALICIOUS-%{matched_var_name}=%{matched_var},setvar:ip.spammer=1,expirevar:ip.spammer=86400,setvar:ip.previous_rbl_check=1,expirevar:ip.previous_rbl_check=86400,skipAfter:END_RBL_CHECK"   SecAction "phase:1.t:none,nolog,pass,setvar:ip.previous_rbl_check=1,expirevar:ip.previous_rbl_check=86400" SecMarker END_RBL_LOOKUP  SecRule IP:SPAMMER "@eq 1" "phase:1,t:none,pass,nolog,auditlog,msg:'Request from Known SPAM Source (Previous RBL Match)',tag:'AUTOMATION/MALICIOUS',severity:'2',setvar:'tx.msg=%{rule.msg}',setvar:tx.automation_score=+%{tx.warning_anomaly_score},setvar:tx.anomaly_score=+%{tx.warning_anomaly_score},setvar:tx.%{rule.id}-AUTOMATION/MALICIOUS-%{matched_var_name}=%{matched_var}"  SecMarker END_RBL_CHECK 

The goal of this ruleset is to run an @rbl check once for each IP address and then save the response in a TX variable for 1 day. This is used to limit the number of @rbl lookups that the web server needs to do as there is a latency hit for executing the DNS queries.

So What?

Why use Real-time Blacklist Lookups anyways? What we are talking about here is IP Reputation. Has this client been identified as bad by other web sites? It is sort of like the "No Fly" lists that the Department of Homeland Security makes available to airlines. It is a method of sharing information about clients so that you can decided if you want to allow this client access to your site at all or perhaps treat them differently (such as with increased logging). Real-time block lists (RBL) are community-based, central repositories for IP Reputation. RBLs are most commonly used to identify web-based comment spam. If you run a blog or user-forum site, wouldn't you like to know if the current client has already been identified as a spammer?

While @rbl is a useful feature, there is a caution with its usage - it is a severe performance hit and can cause increased latency for clients. Whereas the @geoLookup operator accessed a local DB, @rbl checks occur in real-time over the network and utilize the DNS infrastructure. For the same reason that most web admins disable real-time client resolution in logging, running a DNS lookup on each client request can cause severe delays.

@rbl Tips

Here are a few recommended tips for using @rbl.

DNS Caching

Implement a local caching DNS server like rbldnsd so that your @rbl checks issue DNS queries to the local system first.

Use ModSecurity Persistent Storage

Alternatively, you can use ModSecurity to save rbl responses in the IP persistent storage collection. This is what the CRS modsecurity_crs_42_comment_spam.conf file does. The persistent data is cached for 1 day.

Choose your RBL carefully

Make sure that you choose your RBL carefully. You not only want to ensure that the RBL category is appropriate for your site but also that the accuracy of the list is good.