Advanced Topic of the Week: Validating Byte Ranges

We are starting a new blog post series here on the ModSecurity site called "Advanced Feature of the Week" where we will be highlighting many of ModSecurity's really cool capabilities. These are the features that seldom used or fully understood by the average ModSecurity user however can provide detection of sophisticated attacks if used properly. It is our goal with these blog posts to help shed light on these unique features and to provide some real-world, in-the-trenches gotchas for successful usage of these features.

This blog post series will have the following major topic sections -

1) ModSecurity Reference Manual Information

Provide reference manual data.

2) Use Within the OWASP Core Rule Set (CRS)

Outline if/when/how the CRS is utilizing this feature.

3) So What?

Will provide some context as to why you as a user should even care about this capability. What advanced attack/vulnerability is this attempting to catch.

This week's feature is on the use of the @validateByteRange operator.

Reference Manual

validateByteRange

Description: Validates the byte range used in the variable falls into the specified range.

Example:

SecRule ARGS:text "@validateByteRange 10, 13, 32-126"

Note

You can force requests to consist only of bytes from a certain byte range. This can be useful to avoid stack overflow attacks (since they usually contain "random" binary content). Default range values are 0 and 255, i.e. all byte values are allowed. This directive does not check byte range in a POST payload when multipart/form-data encoding (file upload) is used. Doing so would prevent binary files from being uploaded. However, after the parameters are extracted from such request they are checked for a valid range.

validateByteRange is similar to the ModSecurity 1.X SecFilterForceByteRange Directive however since it works in a rule context, it has the following differences:

  • You can specify a different range for different variables.

  • It has an "event" context (id, msg....)

  • It is executed in the flow of rules rather than being a built in pre-check.

ASCII Byte Range Chart

æ







backspace
tab
linefeed


c return


















space
!
"
#
$
%
&
'
(
)
*
+
,
-
.
/
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
0
1
2
3
4
5
6
7
8
9
:
;
<
=
>
?
@
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
X
Y
Z
[
\
]
^
_
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
`
a
b
c
d
e
f
g
h
i
j
k
l
m
n
o
p
q
r
s
t
u
v
w
x
y
z
{
|
}
~




ƒ




ˆ

Š

Œ

Ž

96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143

'
'
"
"



˜

š

œ

ž
Ÿ

¡
¢
£

¥
|
§
¨
©
ª
«
¬
¯
®
¯
°
±
²
³
´
µ

·
¸
¹
º
»
¼
½
¾
¿
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
À
Á
Â
Ã
Ä
Å
Æ
Ç
È
É
Ê
Ë
Ì
Í
Î
Ï
Ð
Ñ
Ò
Ó
Ô
Õ
Ö

Ø
Ù
Ú
Û
Ü
Ý
Þ
ß
à
á
â
ã
ä
å
æ
ç
è
é
ê
ë
ì
í
î
ï
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
ð
ñ
ò
ó
ô
õ
ö
÷
ø
ù
ú
û
ü
ý
þ
ÿ

240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255


OWASP ModSecurity CRS

Use of @validateByteRange in the OWASP ModSecurity CRS (from the end of the modsecurity_crs_20_protocol_violations.conf file) -

# # Restrict type of characters sent # NOTE In order to be broad and support localized applications this rule # only validates that NULL Is not used. # # The strict policy version also validates that protocol and application  # generated fields are limited to printable ASCII.  # # -=[ Rule Logic ]=- # This rule uses the @validateByteRange operator to look for Nul Bytes. # If you set Paranoid Mode - it will check if your application use the range 32-126 for parameters. # # -=[ References ]=- # http://i-technica.com/whitestuff/asciichart.html #  SecRule ARGS|ARGS_NAMES|REQUEST_HEADERS|!REQUEST_HEADERS:Referer "@validateByteRange 1-255" \  "phase:2,rev:'2.0.8',pass,nolog,auditlog,msg:'Invalid character in request',id:'960901',tag:'PROTOCOL_VIOLATION/EVASION',tag:'WASCTC/WASC-28',tag:'OWASP_TOP_10/A1',tag:'OWASP_AppSensor/CIE3',tag:'PCI/6.5.2',severity:'4',t:none,t:urlDecodeUni,setvar:'tx.msg=%{rule.msg}',tag:'http://i-technica.com/whitestuff/asciichart.html',setvar:tx.anomaly_score=+%{tx.notice_anomaly_score},setvar:tx.protocol_violation_score=+%{tx.notice_anomaly_score},setvar:tx.%{rule.id}-PROTOCOL_VIOLATION/EVASION-%{matched_var_name}=%{matched_var}"  SecRule TX:PARANOID_MODE "@eq 1" "chain,phase:2,rev:'2.0.8',pass,nolog,auditlog,msg:'Invalid character in request',id:'960018',tag:'PROTOCOL_VIOLATION/EVASION',tag:'WASCTC/WASC-28',tag:'OWASP_TOP_10/A1',tag:'OWASP_AppSensor/CIE3',tag:'PCI/6.5.2',severity:'4',t:none,t:urlDecodeUni,tag:'http://i-technica.com/whitestuff/asciichart.html'"  SecRule REQUEST_URI|REQUEST_BODY|REQUEST_HEADERS_NAMES|REQUEST_HEADERS|!REQUEST_HEADERS:Referer|TX:HPP_DATA \  "@validateByteRange 32-126" \  "t:none,t:urlDecodeUni,setvar:'tx.msg=%{rule.msg}',setvar:tx.anomaly_score=+%{tx.notice_anomaly_score},setvar:tx.protocol_violation_score=+%{tx.notice_anomaly_score},setvar:tx.%{rule.id}-PROTOCOL_VIOLATION/EVASION-%{matched_var_name}=%{matched_var}"

As you can see, the CRS is, by default, only restricting the existence of Nul Bytes. If the user initiates the PARANOID_MODE variable, however, the CRS will restrict down the allowed byte range to only allow 32-126 which are the normal printable characters for US ASCII (space character through the tilde character).

So What?

Why would you need to use @validateByteRange to restrict anything more than Nul Bytes? The short answer is because of the potential of Impedance Mismatches between a security inspection system (IDS, IPS or WAF) and the target web application. The process of data normalization or canonicalization and how the destination web application handles best-fit mappings can cause issues with bypasses. Here is a great recent references for this issue.

  • Lost In Translation - Giorgia Maone (of the NoScript FF extension fame) outlines how ASP classic web apps attempt to do best-fit mappings of non-ASCII Unicode characters. One example issue is the following XSS payload

%u3008scr%u0131pt%u3009%u212fval(%uFF07al%u212Frt(%22XSS%22)%u02C8)%u2329/scr%u0131pt%u2A
This payload should be correctly decoded to this -

〈scrıpt〉ℯval('alℯrt("XSS")ˈ)〈/scrıpt〉

ASP classic, however, will try and do best-fit mapping and actually will normalize the data into working JS code -

<script>eval('alert("XSS")')</script>
The issue that this raises, for security inspection, is that the inbound payload will most likely not match most XSS regular expression payloads however the application itself will modify it into executable code!

So, this brings us to today's advanced ModSecurity feature - @validateByteRange. By restricting the allowed character byte ranges, you can help to identify when unexpected character code points are used. If the payload above is sent, you should receive an alert message similar to the following -

Message: Found 3 byte(s) in REQUEST_URI outside range: 32-126. [file "/usr/local/apache/conf/modsec_current/base_rules/modsecurity_crs_20_protocol_violations.conf"] [line "251"] [id "960018"] [rev "2.0.6"] [msg "Invalid character in request"] [severity "WARNING"] [tag "PROTOCOL_VIOLATION/EVASION"] [tag "WASCTC/WASC-28"] [tag "OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/CIE3"] [tag "PCI/6.5.2"] [tag "http://i-technica.com/whitestuff/asciichart.html"] 


Trustwave reserves the right to review all comments in the discussion below. Please note that for security and other reasons, we may not approve comments containing links.