Skip to content

Issue with POST request which contain binary files in body with Content-Type as "application/x-www-form-urlencoded"  #2303

Closed
@gullu44

Description

@gullu44

We are facing issue with POST request which contain binary files in body with Content-Type as "application/x-www-form-urlencoded"

we are using below versions

Apache :- apache-2.4.41
mod_security :- 2.9.3
CRS :- 3.0

ModSec Directive setting at Vhost level

SecRuleEngine On

SecRequestBodyAccess On
SecRequestBodyLimitAction ProcessPartial

SecRequestBodyNoFilesLimit 90000000
SecRequestBodyLimit 90000000
SecRequestBodyInMemoryLimit 5000000

Our is hosting Environment and many application send big files in the body(using curl or Client java code) with Content-Type: application/x-www-form-urlencoded. So following i have noticed

  1. Sent a 83864080 bytes file with

    curl $CURL_OPTS -H "Expect: 100-continue" --data-binary @test.txt https://example.com

where $CURL_OPTS contain TLS, cert etc

  1. As binary file is embedded in body, seems mod_sec apply SecRequestBodyNoFilesLimit not SecRequestBodyLimit. If we set SecRequestBodyNoFilesLimit limit anything smaller than the file limit e.g 20 MB, then only 20 MB file is being transferred to backend server ( we use mod proxy balancer) and rest body request is truncated. Is this expected behavior with content-Type: application/x-www-form-urlencoded ? Though DOS is another concern with such higher values.

3.Setting SecRequestBodyNoFilesLimit to such higherlimit adding high latency for upload request. If i set modsec in detection mode , this request just take 13-14 seconed for its completion ( end to end), but turning ON mod_sec with SecRequestBodyNoFilesLimit (90000000) taking 9-11 Minutes .

  1. No improvement noticed by Increasing SecRequestBodyInMemoryLimit to 5 MB . I think this limit is applicable only for Content-Type: application/multipart/form-data

  2. Debug logs show that during REQUEST_BODY phase, input filter always read data in 8192 bytes

     +++++++++++++++
    

29/Apr/2020:22:26:54.317475 +0200] [test1.example.com/sid#3680840][rid#f76e890][/endpoints/web][4] Second phase starting (dcfg 22ec680).
[29/Apr/2020:22:26:54.317492 +0200] [test1.example.com/sid#3680840][rid#f76e890][/endpoints/web][4] Input filter: Reading request body.
[29/Apr/2020:22:26:54.324645 +0200] [test1.example.com/sid#3680840][rid#f76e890][/endpoints/web][9] Input filter: Bucket type TRANSIENT contains 8192 bytes.
[29/Apr/2020:22:26:54.324673 +0200] [test1.example.com/sid#3680840][rid#f76e890][/endpoints/web][9] Input filter: Bucket type TRANSIENT contains 8192 bytes.

........Above lines are repeated 10868 time

I think its taking long time to read body and is cause of high latency. How Can we increase this 8192 limit? In our apache we are not using any mod_buffer though we have set RLimitMEM which anyway is on higher side.

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions