Dynamic DAST/WAF Integration: Realtime Virtual Patching

At the recent OWASP AppSecDC conference, I presented on this topic. I received a lot of feedback from the attendees afterwards so I wanted to get this out to a wider community.

Limitations of Manual DAST -> WAF Data Sharing

While this process does have great value for implementing compensating controls and mitigations for identified web application vulnerabilities until the code can be fixed, it still has some limitations.

Time-to-Fix vs. Time-to-Hack

There are many tasks that must be completed as part of this data sharing:

All of these steps take time. How much time? We asked this question to the community as part of a "Virtual Patching Survey":

Let's now compare these timeframes against the Time-to-Hack metrics we gathered as part of the ModSecurity SQL Injection Challenge:

As you can see, the remediation times usually take much longer than it takes an attacker to find and exploit the vulnerability.

We need a faster method of mitigating these vulnerabilities in production...

Dynamic DAST/WAF Integration

Our goal is to decrease the Time-to-Fix metrics for web application vulnerabilities, as we must beat the Time-to-Hack metric for attackers. In order to accomplish this task, we need to utilize automation and directly integrate our DAST/WAF tools. Attacker use automation, so should defenders!

Automation/Integration Steps

Here are some general integration steps that we must complete:

Now we need to figure out how to integrate the WAF and DAST tools.

Arachni Scanner + ModSecurity Lua API

Arachni scanner is an absolutely awesome web application scanner framework devleoped by Tasos Laskos (@Zap0tek). It is written in Ruby and has just the feature we need for our integration: RPC API. We need to use the RPC architecture so that ModSecurity can be the one to initate assessments rather than a human sitting at the command line terminal. By starting the Arachni RPC service listening on a local IP address, we allow remote ModSecurity installations to contact it and share data.

Arachni RPC Service

Here is an example command line view of initiating the arachni_rpcd process:

# arachni_rpcd --address=192.168.168.128Arachni - Web Application Security Scanner Framework       Author: Tasos "Zapotek" Laskos                                                      (With the support of the community and the Arachni Team.)       Website:       http://github.com/Zapotek/arachni       Documentation: http://github.com/Zapotek/arachni/wikiArachni - Web Application Security Scanner Framework v0.4.1 [0.2.5]       Author: Tasos "Zapotek" Laskos                                                      (With the support of the community and the Arachni Team.)       Website:       http://github.com/Zapotek/arachni       Documentation: http://github.com/Zapotek/arachni/wikiI, [2012-04-05T11:11:35.605542 #2985]  INFO -- System: RPC Server started.I, [2012-04-05T11:11:35.605931 #2985]  INFO -- System: Listening on 192.168.168.128:39127

At this point, the RPC service is ready to receive remote calls.

Arachni RPC Lua Client

In order to have ModSecurity be able to communicate with the Arachni RPC service, Tasos created a Lua RPC client and supporting scripts which are available from his Github repo. These scripts would allow a Lua client to communicate with the RPC service, however the scripts needed to be updated so that it could be initiated from within ModSecurity itself. The script is called arachni_integration.lus and is now available in the ModSecurity CRS SVN trunk. Before using it, you will need to update the following variable to the proper IP address of your Arachni RPC host:

---- Set the remote Arachni RPC host--arachni_host = '192.168.168.128'

Initiating On-Demand Resource Scans

With these tools in place, we are now ready to use them. The OWASP ModSecurity CRS SVN trunk also includes an experimental rules file called modsecurity_crs_16_scanner_integration.conf that has the following entries:

## -=[ You must be using the Resource Profiling Rules to track this data ]=-## modsecurity_crs_40_appsensor_detection_point_2.0_setup.conf# modsecurity_crs_40_appsensor_detection_point_3.0_end.conf                                         ### -=[ Disable ModSecurity For Arachni Scans ]=-## Update the remote IP address for your Arachni RPC host#SecRule REMOTE_ADDR "@ipMatch 192.168.168.128" "chain,phase:1,t:none,nolog,pass"       SecRule REQUEST_HEADERS:User-Agent "@beginsWith Arachni/" "ctl:ruleEngine=Off"## -=[ Initiate Arachni Scan on 1st URL Access ]=-# # Update the path to the arachni_integration.lua script#SecRule &RESOURCE:ARACHNI_SCAN_COMPLETED "@eq 0" "chain,phase:5,t:none,log,pass"       SecRule &ARGS "@gt 0" "exec:/etc/apache2/modsecurity-crs/lua/arachni_integration.lua"

The final rule is what is actually triggered the first time that a client access a specific URL resource. The rule will check the RESOURCE collection to see if we have previously scanned the URL. If not, then it will trigger our arachni_integration.lua script. Here is some of the Lua debug log data showing the processing and some data that is passed to the Arachni RPC service:

Lua: Executing script: /etc/apache2/modsecurity-crs/base_rules/arachni_integration.luaArachni: Host: 192.168.168.128Arachni: Filename: /vicnum/vicnum5.phpArachni: URL to scan is: http://192.168.168.128/vicnum/vicnum5.phpArachni: Request Method is: POSTArachni: Arg Name: player and Value: test.Arachni: Updated ARGS table is: ---player: testArachni: Updated Cookies table is: --- {}Arachni: Yaml output of vectors is: ---- inputs:    player: test  type: form  method: POST  action: http://192.168.168.128/vicnum/vicnum5.php

As you can see, we have extracted the key data points for a focused Arachni assessment of this particular resource. Back in the Arachni RPC service, it shows that it has received the request and initiated a scan:

I, [2012-04-05T11:33:32.006918 #3771]  INFO -- System: RPC Server started.I, [2012-04-05T11:33:32.007164 #3771]  INFO -- System: Listening on 192.168.168.128:44604I, [2012-04-05T11:35:47.390623 #3746]  INFO -- Call: dispatcher.dispatch [192.168.168.128]I, [2012-04-05T11:35:47.419363 #3748]  INFO -- Call: modules.load [192.168.168.128]Arachni - Web Application Security Scanner Framework v0.4.1 [0.2.5]       Author: Tasos "Zapotek" Laskos                                                      (With the support of the community and the Arachni Team.)       Website:       http://github.com/Zapotek/arachni       Documentation: http://github.com/Zapotek/arachni/wikiI, [2012-04-05T11:35:47.451187 #3748]  INFO -- Call: plugins.load [192.168.168.128]I, [2012-04-05T11:35:47.447358 #3837]  INFO -- System: RPC Server started.I, [2012-04-05T11:35:47.453383 #3837]  INFO -- System: Listening on 192.168.168.128:61420I, [2012-04-05T11:35:47.459832 #3748] INFO -- Call: opts.set [192.168.168.128]I, [2012-04-05T11:35:47.487119 #3748] INFO -- Call: framework.run [192.168.168.128]

Once a scan has been initiated, ModSecurity will then save the appropriate scan instance data within the RESOURCE collection:

Re-retrieving collection prior to store: resourceWrote variable: name "__expire_KEY", value "1333644233".Wrote variable: name "KEY", value "192.168.168.128_/vicnum/vicnum5.php".Wrote variable: name "TIMEOUT", value "3600".Wrote variable: name "__key", value "192.168.168.128_/vicnum/vicnum5.php".Wrote variable: name "__name", value "resource".Wrote variable: name "CREATE_TIME", value "1333640632".Wrote variable: name "UPDATE_COUNTER", value "1".Wrote variable: name "min_pattern_threshold", value "50".Wrote variable: name "min_traffic_threshold", value "100".Wrote variable: name "arachni_scan_initiated", value "1".Wrote variable: name "arachni_instance_info_port", value "30118".Wrote variable: name "arachni_instance_info_token", value "c5ab2feb9072ed8e7737f7d526e7b254".Wrote variable: name "traffic_counter", value "1".Wrote variable: name "request_method_counter_POST", value "1".Wrote variable: name "NumOfArgs_counter_1", value "1".Wrote variable: name "args_names_counter_player", value "1".Wrote variable: name "ARGS:player_length_4_counter", value "1".Wrote variable: name "ARGS:player_alpha_counter", value "1".Wrote variable: name "LAST_UPDATE_TIME", value "1333640633".Persisted collection (name "resource", key "192.168.168.128_/vicnum/vicnum5.php").

This data is used so that we don't initiate multiple scans and that we know which Arachni Dispatcher to contact and the proper report hash token to request. At this point, Arachni will have started scanning this particular resource with the plugins you have specified. From the previous ModSecurity rules file, when we see Arachni traffic coming from an authorized source IP address and it has the "Arachni" data in the User-Agent field, we will disable ModSecurity so that we don't interfere with its assessment or create needless alerts. Here is an example from the Apache log file showing the original client request that initiated the Arachni scan and the subsequent scan traffic itself:

192.168.168.1 - - [05/Apr/2012:11:35:47 -0400] "POST /vicnum/vicnum5.php HTTP/1.1" 200 1022 "http://192.168.168.128/vicnum/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:11.0) Gecko/20100101 Firefox/11.0"192.168.168.128 - - [05/Apr/2012:11:35:48 -0400] "POST /vicnum/vicnum5.php HTTP/1.1" 200 1107 "-" "Arachni/0.4.1"192.168.168.128 - - [05/Apr/2012:11:35:48 -0400] "POST /vicnum/vicnum5.php HTTP/1.1" 200 1022 "-" "Arachni/0.4.1"192.168.168.128 - - [05/Apr/2012:11:35:48 -0400] "POST /vicnum/vicnum5.php HTTP/1.1" 200 1022 "-" "Arachni/0.4.1"192.168.168.128 - - [05/Apr/2012:11:35:48 -0400] "POST /vicnum/vicnum5.php HTTP/1.1" 200 1116 "-" "Arachni/0.4.1"192.168.168.128 - - [05/Apr/2012:11:35:48 -0400] "POST /vicnum/vicnum5.php HTTP/1.1" 200 1100 "-" "Arachni/0.4.1"192.168.168.128 - - [05/Apr/2012:11:35:48 -0400] "POST /vicnum/vicnum5.php HTTP/1.1" 200 1081 "-" "Arachni/0.4.1"192.168.168.128 - - [05/Apr/2012:11:35:48 -0400] "POST /vicnum/vicnum5.php HTTP/1.1" 200 1082 "-" "Arachni/0.4.1"…

Keep in mind here that this process is much faster than normal assessments as Arachni no longer has to first crawl and enumerate resources. We are telling it exactly what data to focus on. After Arachni has completed its scan, it saves the report data for pickup by ModSecurity. The next time a client access the same URL, ModSecurity will check with Arachni and pull down the assessment report. Here is an example of the report data obtained by ModSecurity from Arachni:

Arachni: Previous scan was initiated, checking scan status.Arachni: Port info: 30118 and Token info: c5ab2feb9072ed8e7737f7d526e7b254Arachni: Scan completed - calling for report.Arachni: Yaml Results: ---- cwe: '79'  description: "Client-side code (like JavaScript) can\n                    be injected    into the web application which is then returned to the user's browser.\n                    This    can lead to a compromise of the client's system or serve as a pivoting point for    other attacks."  references:    ha.ckers: http://ha.ckers.org/xss.html    Secunia: http://secunia.com/advisories/9716/  variations: []  _hash: d241855ec9dd4694f6eaf28e28a0913f  mod_name: XSS var: player  elem: form  url: http://192.168.168.128/vicnum/vicnum5.php  cvssv2: '9.0'  method: POST

As we can see from this beginning section of the report, Arachni has found that there is an XSS vulnerability within the parameter called "player" on this page. With this new found vulnerability data, ModSecurity can now add new variables to the RESOURCE collection:

Wrote variable: name "min_pattern_threshold", value "50".Wrote variable: name "min_traffic_threshold", value "100".Wrote variable: name "arachni_scan_initiated", value "1".Wrote variable: name "arachni_instance_info_port", value "30118".Wrote variable: name "arachni_instance_info_token", value "c5ab2feb9072ed8e7737f7d526e7b254".Wrote variable: name "traffic_counter", value "2".Wrote variable: name "request_method_counter_POST", value "2".Wrote variable: name "NumOfArgs_counter_1", value "2".Wrote variable: name "args_names_counter_player", value "2".Wrote variable: name "ARGS:player_length_4_counter", value "2".Wrote variable: name "ARGS:player_alpha_counter", value "2".Wrote variable: name "LAST_UPDATE_TIME", value "1333640642".Wrote variable: name "xss_vulnerable_params", value "player".Wrote variable: name "sqli_vulnerable_params", value "player".Wrote variable: name "arachni_scan_completed", value "1".Persisted collection (name "resource", key "192.168.168.128_/vicnum/vicnum5.php").

We now know the vulnerability injection points for this particular resource. In order to use this data for blocking/virtual patching purposes, we need to use one more rules file from the OWASP ModSecurity CRS called modsecurity_crs_46_scanner_integration.conf: 

## -=[ You must be using the Resource Profiling Rules to track this data ]=-## modsecurity_crs_40_appsensor_detection_point_2.0_setup.conf# modsecurity_crs_40_appsensor_detection_point_3.0_end.conf #SecRule TX:/XSS-ARGS:/ ".*" "id:'999003',chain,phase:2,t:none,msg:'XSS Attack Against Known Vulnerable Parameter.',logdata:'%{matched_var}'"        SecRule MATCHED_VARS_NAMES "-ARGS:(.*)$" "chain,capture"                SecRule TX:1 "@within %{resource.xss_vulnerable_params}"SecRule TX:/SQL_INJECTION-ARGS:/ ".*" "id:'999004',chain,phase:2,t:none,msg:'SQLi Attack Against Known Vulnerable Parameter.',logdata:'%{matched_var}'"        SecRule MATCHED_VARS_NAMES "-ARGS:(.*)$" "chain,capture"                SecRule TX:1 "@within %{resource.sqli_vulnerable_params}"

These rules work in collaboration with the generic attack detection capabilities of the OWASP ModSecurity CRS. When the CRS identifies SQLi or XSS payloads, it creates temporary TX variables that hold meta-data about the injection points and attack category. These rules simply correlate any existing TX varaible data with the vulnerability data identifies by Arachni. For example, if a client were to send an SQLi attack to this particular resource, here are some example alerts that would be generated:

[Thu Apr 05 11:44:39 2012] [error] [client 192.168.168.1] ModSecurity: Warning. Pattern match "(?i:(?:(\\"|'|`|\\xc2\\xb4|\\xe2\\x80\\x99|\\xe2\\x80\\x98)\\\\s*\\\\*.+(?:x?or|div|like|between|and|id)\\\\W*(\\"|'|`|\\xc2\\xb4|\\xe2\\x80\\x99|\\xe2\\x80\\x98)\\\\d)|(?:\\\\^(\\"|'|`|\\xc2\\xb4|\\xe2\\x80\\x99|\\xe2\\x80\\x98))|(?:^[\\\\w\\\\s(\\"|'|`|\\xc2\\xb4|\\xe2\\x80\\x99|\\xe2\\x80\\x98)-]+( ..." at ARGS:player. [file "/etc/apache2/modsecurity-crs/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "573"] [id "981243"] [msg "Detects classic SQL injection probings 2/2"] [data "' or"] [severity "CRITICAL"] [tag "WEB_ATTACK/SQLI"] [tag "WEB_ATTACK/ID"] [tag "WEB_ATTACK/LFI"] [hostname "192.168.168.128"] [uri "/vicnum/vicnum5.php"] [unique_id "T329538AAQEAAA-3DtwAAAAJ"][Thu Apr 05 11:44:39 2012] [error] [client 192.168.168.1] ModSecurity: Warning. String match within "player" at TX:1. [file "/etc/apache2/modsecurity-crs/base_rules/modsecurity_crs_46_known_vulns.conf"] [line "5"] [id "999004"] [msg "SQLi Attack Against Known Vulnerable Parameter."] [data "player"] [hostname "192.168.168.128"] [uri "/vicnum/vicnum5.php"] [unique_id "T329538AAQEAAA-3DtwAAAAJ"]

The 2nd alert is the new data obtained through this Arachni scanner integration. With this new vulnerability data correlated with the attack payloads, a ModSecurity user may have a greater confidence in blocking the transaction.

Conclusion

This proof of concept integration is an important step forward for automated virtual patching as it reduces the Time-to-Fix metrics from hours or days to merely seconds. We are also researching options to integrate into other DAST tools such as OWASP ZAP and Burp Suite as they both have APIs.

Caveats

While this is certainly exciting, there are however some caveats and areas for more testing:

• Choose your DAST tool carefully - each tool has different levels of false positives.
• Tune your DAST tool - make sure that you understand all of the scanning options and choose testing plugins appropriately.
• Test with Detection Only Mode initially - then move to a blocking mode once you see how the integration works.

Call for Testing

We encourage community testing. If you do test out the integration, please sign up for the project mail-list and and let us know how it goes!

Integration Support

Trustwave SpiderLabs offers professional services for integrating ModSecurity into your environment. If you need commercial help with this, let us know.