ARGS_NAMES not treated as a collection

[michaelgranzow-avi]

In version 3 of libmodsec, some variables that according to the documentation should be 'collections' are treated as scalars. This affects the following variables:

ARGS_NAMES
ARGS_GET_NAMES
ARGS_POST_NAMES
REQUEST_HEADERS_NAMES
RESPONSE_HEADERS_NAMES

In the code these are implemented as instances of class AnchoredVariable. These variables are constructed from the individual args (or headers) by concatenation (with a single whitespace as the separator).

A rule like the following (from https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual#ARGS_NAMES) therefore doesn't work as intended:

SecRule ARGS_NAMES "!^(p|a)$" "id:13"

Instead of rejecting all requests that contain an argument other than 'p' and 'a', it rejects all requests that have more than one argument (and the one argument has to be called either 'a' or 'p'). For a query string 'a=v1&p=v2', the match will be made against the string
'a p'. This won't match the regex '^(p|a)$' and the request will be rejected.

Similarly, the following rule from version 3.0.2 of the CRS doesn't work because it rejects all requests that have more than one argument:

SecRule ARGS|ARGS_NAMES|REQUEST_BODY "@validateByteRange
38,44-46,48-58,61,65-90,95,97-122"
"phase:request,
rev:'2',
ver:'OWASP_CRS/3.0.0',
maturity:'9',
accuracy:'9',
block,
msg:'Invalid character in request (outside of very strict set)',
id:920273,
severity:'CRITICAL',
t:none,t:urlDecodeUni,
tag:'application-multi',
tag:'language-multi',
tag:'platform-multi',
tag:'attack-protocol',
tag:'OWASP_CRS/PROTOCOL_VIOLATION/EVASION',
tag:'paranoia-level/4',
setvar:'tx.msg=%{rule.msg}',
setvar:tx.anomaly_score=+%{tx.critical_anomaly_score},
setvar:tx.%{rule.id
}-OWASP_CRS/PROTOCOL_VIOLATION/EVASION-%{matched_var_name}=%{matched_var}"

Interestingly, this seems to be entirely intentional in the code: there are even tests for exactly this behaviour: In file 'variable-ARGS_NAMES', a request with 'param1=value1&param2=value2' and a rule
SecRule ARGS_NAMES "@contains test " "id:1,phase:3,pass,t:trim"
is expected to produce the following debug log line:
Target value: "param1 param2"

Of all the variables ending in '_NAMES', only REQUEST_COOKIES_NAMES is actually treated as a collection.

[michaelgranzow-avi]

Hi Victor, we have a fix which is currently going through our own QA.

In the meantime, could you please provide your opinion on this issue? What confuses me is the clear contradiction between the handbook and the code.
Thanks!

[victorhora]

Hi @michaelgranzow-avi,

I've looked into this issue yesterday and I think you're correct. The code shows that these variables doesn't seem like collections such as REQUEST_COOKIES_NAMES.
There could be more cases like this which worth investigating.

I think the reason for the contradiction between the docs and the v3 code was a simple overlook :)

You see the code for these variables in v2 are set as collections as you can see here, here and here so that's why the reference manual suggests they are collections. As we are in the phase of feature completion between v2 and v3 these things can happen :)

That said I'll add your issue to the pipeline for a fix. In the meantime, if you are working on a fix on your side it would be great if you could submit a pull request with your suggested fix for us :)

Thanks!

[zimmerle]

ARGS_NAMES changes was held here: 1d3c4c6

[abhijoglekar]

@michaelgranzow-avi 's PR for this issue, he is on PTO, so submitting for him.

cc @victorhora @zimmerle

[abhijoglekar]

These set of commits were based off Modsecurity v3/master July 31st top-of-tree

0508395

[zimmerle]

Thanks @abhijoglekar and @michaelgranzow-avi