nxtool was not able to generate meaningful whitelist

[Onyx808]

Hello,

I run this tool against a test error log and it returned this message

Command: python nxtool.py --whitelist --flat-file=/var/www/error.log

Result: nxtool was not able to generate meaningful whitelist

When I run the same test error log against the nx_util.py included int Naxsi version 0.53.2 it does generate this white list rules.

########### Optimized Rules Suggestion ##################
# total_count:2 (20.0%), peer_count:1 (50.0%) | sql keywords
BasicRule wl:1000 "mz:$URL:/naxsi2/wp-includes/js/imgareaselect/imgareaselect.css|URL";
# total_count:1 (10.0%), peer_count:1 (50.0%) | close square bracket (]), possible js
BasicRule wl:1311 "mz:$URL:/naxsi2/|$BODY_VAR:ips[0]|NAME";
# total_count:1 (10.0%), peer_count:1 (50.0%) | open square backet ([), possible js
BasicRule wl:1310 "mz:$URL:/naxsi2/|$BODY_VAR:ips[0]|NAME";
# total_count:1 (10.0%), peer_count:1 (50.0%) | double encoding
BasicRule wl:1315 "mz:$URL:/naxsi2/|$HEADERS_VAR:cookie";

Any idea why nxtool is not creating these rules?

[dmigous]

+1
similar issue

I have only 1 rule output out of nxtool-ng on 130Mb log file

$ python nxtool.py --whitelist --flat-file ../error.log
...
[+] Generating Google analytics rules
[+] Generating Image 1002 rules
[+] The url /components/dropzone/dist/dropzone.js triggered 1 exceptions for the rule 1002, whitelisting it.
[+] Generating array-like variable name rules
[+] Generating cookies rules
[+] Generating var + zone + url rules
[+] Generating url rules
[+] Generating var + zone rules
[+] Generating zone rules
[+] Generating site rules

Generated whitelists:
	BasicRule wl:1002 "mz:$URL_X:^/components/dropzone/dist/dropzone.js|URL" "msg:Images size (0x)";

vs via nx_util.py

$ python nx_util.py -l ../error.log -o
...
########### Optimized Rules Suggestion ##################
# total_count:297288 (79.32%), peer_count:261 (72.1%) | mysql comment (--)
BasicRule wl:1007 "mz:$HEADERS_VAR:cookie";
# total_count:34891 (9.31%), peer_count:176 (48.62%) | 0x, possible hex encoding
BasicRule wl:1002 "mz:$HEADERS_VAR:cookie";
# total_count:21323 (5.69%), peer_count:30 (8.29%) | double encoding
BasicRule wl:1315 "mz:$HEADERS_VAR:cookie";
# total_count:3120 (0.83%), peer_count:186 (51.38%) | sql keywords
BasicRule wl:1000 "mz:$URL:/components/dropzone/dist/dropzone.js|URL";
# total_count:2390 (0.64%), peer_count:92 (25.41%) | sql keywords
BasicRule wl:1000 "mz:$URL:/js/mixins/dropdown.js|URL";
# total_count:2380 (0.64%), peer_count:85 (23.48%) | sql keywords
BasicRule wl:1000 "mz:$URL:/components/matches-selector/matches-selector.js|URL";
# total_count:2163 (0.58%), peer_count:10 (2.76%) | close square bracket (]), possible js
#BasicRule wl:1311 "mz:$BODY_VAR:user[username]|NAME";
# total_count:2163 (0.58%), peer_count:10 (2.76%) | open square backet ([), possible js
#BasicRule wl:1310 "mz:$BODY_VAR:user[username]|NAME";
# total_count:2163 (0.58%), peer_count:10 (2.76%) | close square bracket (]), possible js
#BasicRule wl:1311 "mz:$BODY_VAR:user[password]|NAME";
# total_count:2163 (0.58%), peer_count:10 (2.76%) | open square backet ([), possible js
#BasicRule wl:1310 "mz:$BODY_VAR:user[password]|NAME";
# total_count:902 (0.24%), peer_count:78 (21.55%) | mysql comment (#)
BasicRule wl:1016 "mz:$ARGS_VAR:_pjax";
# total_count:773 (0.21%), peer_count:145 (40.06%) | close square bracket (]), possible js
BasicRule wl:1311 "mz:$ARGS_VAR:brands[]|NAME";
# total_count:773 (0.21%), peer_count:145 (40.06%) | open square backet ([), possible js
BasicRule wl:1310 "mz:$ARGS_VAR:brands[]|NAME";
# total_count:592 (0.16%), peer_count:27 (7.46%) | ?
BasicRule wl:0 "mz:$URL:/security/csp/report|$BODY_VAR:original-policy";
# total_count:395 (0.11%), peer_count:27 (7.46%) | ?
BasicRule wl:0 "mz:$URL:/security/csp/report|$BODY_VAR:violated-directive";
# total_count:390 (0.1%), peer_count:9 (2.49%) | ?
#BasicRule wl:0 "mz:$URL:/security/hpkp/report|$BODY_VAR:known-pins";

[jvoisin]

Feel free to contribute :)
(Super-sorry, I don't have much time for now to dedicate to this project/PoC :/)