Description
We are facing issue with POST request which contain binary files in body with Content-Type as "application/x-www-form-urlencoded"
we are using below versions
Apache :- apache-2.4.41
mod_security :- 2.9.3
CRS :- 3.0
ModSec Directive setting at Vhost level
SecRuleEngine On
SecRequestBodyAccess On
SecRequestBodyLimitAction ProcessPartial
SecRequestBodyNoFilesLimit 90000000
SecRequestBodyLimit 90000000
SecRequestBodyInMemoryLimit 5000000
Our is hosting Environment and many application send big files in the body(using curl or Client java code) with Content-Type: application/x-www-form-urlencoded. So following i have noticed
-
Sent a 83864080 bytes file with
curl $CURL_OPTS -H "Expect: 100-continue" --data-binary @test.txt https://example.com
where $CURL_OPTS contain TLS, cert etc
- As binary file is embedded in body, seems mod_sec apply SecRequestBodyNoFilesLimit not SecRequestBodyLimit. If we set SecRequestBodyNoFilesLimit limit anything smaller than the file limit e.g 20 MB, then only 20 MB file is being transferred to backend server ( we use mod proxy balancer) and rest body request is truncated. Is this expected behavior with content-Type: application/x-www-form-urlencoded ? Though DOS is another concern with such higher values.
3.Setting SecRequestBodyNoFilesLimit to such higherlimit adding high latency for upload request. If i set modsec in detection mode , this request just take 13-14 seconed for its completion ( end to end), but turning ON mod_sec with SecRequestBodyNoFilesLimit (90000000) taking 9-11 Minutes .
-
No improvement noticed by Increasing SecRequestBodyInMemoryLimit to 5 MB . I think this limit is applicable only for Content-Type: application/multipart/form-data
-
Debug logs show that during REQUEST_BODY phase, input filter always read data in 8192 bytes
+++++++++++++++
29/Apr/2020:22:26:54.317475 +0200] [test1.example.com/sid#3680840][rid#f76e890][/endpoints/web][4] Second phase starting (dcfg 22ec680).
[29/Apr/2020:22:26:54.317492 +0200] [test1.example.com/sid#3680840][rid#f76e890][/endpoints/web][4] Input filter: Reading request body.
[29/Apr/2020:22:26:54.324645 +0200] [test1.example.com/sid#3680840][rid#f76e890][/endpoints/web][9] Input filter: Bucket type TRANSIENT contains 8192 bytes.
[29/Apr/2020:22:26:54.324673 +0200] [test1.example.com/sid#3680840][rid#f76e890][/endpoints/web][9] Input filter: Bucket type TRANSIENT contains 8192 bytes.
........Above lines are repeated 10868 time
I think its taking long time to read body and is cause of high latency. How Can we increase this 8192 limit? In our apache we are not using any mod_buffer though we have set RLimitMEM which anyway is on higher side.