Skip to content

Vk 2052 add ja5 filter #2309

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 92 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
dbcd5e5
Implement ja5 hash calculation.
EvgeniiMekhanik Dec 3, 2024
f23040a
WIP
Kutumov Dec 10, 2024
1909270
Ja5 on the fly reconfiguration
Kutumov Dec 12, 2024
0939e3f
Merge remote-tracking branch 'origin/vk-2052-introduce-ja5t-calc' int…
Kutumov Dec 12, 2024
1e7c988
Add copyright to new files
Kutumov Dec 12, 2024
08f9637
Merge remote-tracking branch 'origin/master' into vk-2052-add-ja5t-co…
Kutumov Dec 13, 2024
8609fca
PR remarks
Kutumov Dec 13, 2024
2a78195
Implement ja5 hash calculation.
EvgeniiMekhanik Dec 3, 2024
0e890ac
Fix according review
EvgeniiMekhanik Dec 12, 2024
215da95
Move all ja5h related structures to one file
EvgeniiMekhanik Dec 18, 2024
c639d5f
Merge remote-tracking branch 'origin/MekhanikEvgenii/ja-2052' into vk…
Kutumov Dec 20, 2024
a307c0a
Implement ja5 hash calculation.
EvgeniiMekhanik Dec 3, 2024
7a15ac1
Fix according review
EvgeniiMekhanik Dec 12, 2024
9b22d0f
Move all ja5h related structures to one file
EvgeniiMekhanik Dec 20, 2024
1006133
Fix incorrect header state saving
EvgeniiMekhanik Dec 20, 2024
173a666
Merge branch 'MekhanikEvgenii/ja-2052' into vk-2052-add-ja5t-config
Kutumov Dec 21, 2024
34b5391
Ja5 config: configure storage size in bytes instead of max entries nu…
Kutumov Dec 21, 2024
08f7665
Merge branch 'vk-2052-add-ja5t-config' into vk-2052-add-ja5-filter
Kutumov Dec 21, 2024
e0ca140
Merge remote-tracking branch 'origin/MekhanikEvgenii/ja-2052' into vk…
Kutumov Dec 21, 2024
d44c104
Add ja5 filter common code.
Kutumov Dec 21, 2024
6bf77cf
Add forgotten file
Kutumov Dec 21, 2024
884fd27
Linter
Kutumov Dec 21, 2024
27def5b
Linter
Kutumov Dec 21, 2024
7038237
Fixed mods loading dependencies.
Kutumov Dec 24, 2024
e1fd719
Update tempesta_fw.conf
Kutumov Dec 24, 2024
8f183da
Merge remote-tracking branch 'origin/master' into vk-2052-add-ja5-filter
Kutumov Dec 24, 2024
dc805dc
Added callbacks that decide to limit tls connections and records in
Kutumov Dec 24, 2024
a57b13e
Move ja5_conf.h|c to fw
Kutumov Dec 24, 2024
2257f95
Fix build
Kutumov Dec 24, 2024
939475b
Fix Makefile
Kutumov Dec 24, 2024
780b49b
Fix segfault in the absense of ja5t section
Kutumov Dec 24, 2024
2b0db3f
Changes according to PR comments
Kutumov Dec 25, 2024
25c47d6
Sevetal fixes
EvgeniiMekhanik Dec 26, 2024
321b654
Zero tbd recors during initalization in ja5 filter
Kutumov Dec 26, 2024
ef1e402
Fix zeroing tbd recors during initalization in ja5 filter
Kutumov Dec 26, 2024
a445ef1
change nexus IP
RomanBelozerov Dec 27, 2024
55dfd74
update `tempesta_installer.sh` for Ubuntu 24
RomanBelozerov Dec 30, 2024
80598c9
Merge pull request #2311 from tempesta-tech/rb-432-move-nexus
krizhanovsky Dec 30, 2024
39a44cb
Update etc/tempesta_fw.conf
Kutumov Dec 30, 2024
feca065
Fix dead lock.
EvgeniiMekhanik Jan 3, 2025
d45b4af
change nexus IP
RomanBelozerov Dec 27, 2024
f3e5b25
update `tempesta_installer.sh` for Ubuntu 24
RomanBelozerov Dec 30, 2024
1dc6fdb
PR remarks
Kutumov Jan 6, 2025
bc91276
Fix incorrect syncronization in pool.c
EvgeniiMekhanik Jan 2, 2025
8e5780a
Fix removal of outdated records in ja5 filter
Kutumov Jan 8, 2025
d45bf9f
Fix incorrect syncronization in pool.c
EvgeniiMekhanik Jan 2, 2025
848c76a
Implement ja5 hash calculation.
EvgeniiMekhanik Dec 3, 2024
e4802f3
Fix according review
EvgeniiMekhanik Dec 12, 2024
c321d78
Implement ja5 hash calculation.
EvgeniiMekhanik Dec 3, 2024
566885e
Ja5 config: configure storage size in bytes instead of max entries nu…
Kutumov Dec 21, 2024
b902707
Fix according review
EvgeniiMekhanik Dec 12, 2024
6d340e0
PR remarks
Kutumov Jan 6, 2025
ab4cd36
WIP
Kutumov Dec 10, 2024
25cf979
Ja5 on the fly reconfiguration
Kutumov Dec 12, 2024
6dc10fd
Add copyright to new files
Kutumov Dec 12, 2024
dea335d
PR remarks
Kutumov Dec 13, 2024
e1cf69c
Implement ja5 hash calculation.
EvgeniiMekhanik Dec 3, 2024
0df3793
Fix according review
EvgeniiMekhanik Dec 12, 2024
11a2959
Implement ja5 hash calculation.
EvgeniiMekhanik Dec 3, 2024
3dd7caa
Ja5 config: configure storage size in bytes instead of max entries nu…
Kutumov Dec 21, 2024
df2da04
Fix according review
EvgeniiMekhanik Dec 12, 2024
8bac6f7
Add ja5 filter common code.
Kutumov Dec 21, 2024
41ed5ea
Add forgotten file
Kutumov Dec 21, 2024
0f36115
Linter
Kutumov Dec 21, 2024
47453e3
Linter
Kutumov Dec 21, 2024
8645871
Fixed mods loading dependencies.
Kutumov Dec 24, 2024
017d00b
Update tempesta_fw.conf
Kutumov Dec 24, 2024
e9cdd59
Added callbacks that decide to limit tls connections and records in
Kutumov Dec 24, 2024
76693e9
Move ja5_conf.h|c to fw
Kutumov Dec 24, 2024
5be38f3
Fix build
Kutumov Dec 24, 2024
fd45245
Fix Makefile
Kutumov Dec 24, 2024
b742d80
Fix segfault in the absense of ja5t section
Kutumov Dec 24, 2024
c1c65b6
Changes according to PR comments
Kutumov Dec 25, 2024
59e2051
Sevetal fixes
EvgeniiMekhanik Dec 26, 2024
b29dc73
Zero tbd recors during initalization in ja5 filter
Kutumov Dec 26, 2024
b57a062
Fix zeroing tbd recors during initalization in ja5 filter
Kutumov Dec 26, 2024
1e9338f
Update etc/tempesta_fw.conf
Kutumov Dec 30, 2024
26d3385
Fix dead lock.
EvgeniiMekhanik Jan 3, 2025
a04c47f
PR remarks
Kutumov Jan 6, 2025
0ed6443
Fix removal of outdated records in ja5 filter
Kutumov Jan 8, 2025
fa29133
Implement ja5 hash calculation.
EvgeniiMekhanik Dec 3, 2024
e165833
Fix according review
EvgeniiMekhanik Dec 12, 2024
9ea651a
Implement ja5 hash calculation.
EvgeniiMekhanik Dec 3, 2024
78e66f1
Ja5 config: configure storage size in bytes instead of max entries nu…
Kutumov Dec 21, 2024
a509514
Fix according review
EvgeniiMekhanik Dec 12, 2024
d64e1b6
PR remarks
Kutumov Jan 6, 2025
19e2683
Merge branch 'vk-2052-add-ja5-filter' of github.com:tempesta-tech/tem…
Kutumov Jan 8, 2025
a5751f5
Remove extra line
Kutumov Jan 8, 2025
b07710e
Fix ja5_conf.c after rebase to master
Kutumov Jan 8, 2025
4cd2688
Update http_parser.c
Kutumov Jan 8, 2025
8de1079
Update Makefile
Kutumov Jan 8, 2025
1ab3465
Update hash.h
Kutumov Jan 8, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Tempesta FW
#
# Copyright (C) 2014 NatSys Lab. (info@natsys-lab.com).
# Copyright (C) 2015-2022 Tempesta Technologies, Inc.
# Copyright (C) 2015-2024 Tempesta Technologies, Inc.
#
# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by
Expand Down Expand Up @@ -158,7 +158,7 @@ ifdef ERROR
endif
ifndef AVX2
$(warning !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!)
$(warning WARNING: YOUR PLATFORM IS TOO OLD AND IS NOT UNSUPPORTED)
$(warning WARNING: YOUR PLATFORM IS TOO OLD AND IS NOT SUPPORTED)
$(warning WARNING: THIS AFFECT PERFORMANCE AND MIGHT AFFECT SECURITY)
$(warning WARNING: PLEASE DO NOT USE THE BUILD IN PRODUCTION)
$(warning !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!)
Expand Down
74 changes: 74 additions & 0 deletions etc/tempesta_fw.conf
Original file line number Diff line number Diff line change
Expand Up @@ -1421,3 +1421,77 @@
# Example:
# access_log dmesg mmap mmap_host=localhost mmap_log=access.log;
#

# TAG: ja5t
#
# Specifies TLS filtering behaviour: which Ja5t hashes and how to filter.
#
# Syntax:
# ja5t storage_size=<STORAGE_SIZE> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please extend the comment and examples also for ja5h and ja5. For ja5h we should just use the current HTTP requests per second from http_limits.c. ja5 should also limit HTTP requests per second since the hash is fully computed only on HTTP layer.

Please don't forget to describe this behavior in wiki. BTW could you please create a branch in tempesta.wiki with the description of the new logic

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added docs for ja5h. But I dont understand what you mean under ja5

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With ja5 I meant the whole hash, including HTTP, TLS and TCP/IP in future

# hash <HASH_STRING> <CONNECTIONS_PER_SEC> <TLS_RECORDS_PER_SECOND>;
# ...
# hash <HASH_STRING> <CONNECTIONS_PER_SEC> <TLS_RECORDS_PER_SECOND>;
# }
#
#
# STORAGE_SIZE is the size of the storage holding ja5t hashes to be monitored
# by filtering code. Hashes are evicted by LRU algorithm. The value must be multiple
# of 2^21. Defalut: 25 * 2^21
#
# HASH_STRING is a string value of a ja5t hash calculated from the Client Hello.
# You can find these values in the access log or in ClickHouse-based
# analytics.
#
# CONNECTIONS_PER_SEC is a number of allowed connections per second for
# clients identified by HASH_STRING.
#
# TLS_RECORDS_PER_SECOND is a number of allowed TLS records per second for
# clients identified by HASH_STRING.
#
# Examples:
# ja5t storage_size=2097152 {
# hash deadbeef12345678 10 1000;
# ...
# hash 1234abcdeeaabbcc 0 0;
# }
#
# Default:
# No TLS filtering applied.
#

# TAG: ja5h
#
# Specifies HTTP filtering behaviour: which Ja5h hashes and how to filter.
#
# Syntax:
# ja5h storage_size=<STORAGE_SIZE> {
# hash <HASH_STRING> <CONNECTIONS_PER_SEC> <HTTP_REQUESTS_PER_SECOND>;
# ...
# hash <HASH_STRING> <CONNECTIONS_PER_SEC> <HTTP_REQUESTS_PER_SECOND>;
# }
#
#
# STORAGE_SIZE is the size of the storage holding ja5h hashes to be monitored
# by filtering code. Hashes are evicted by LRU algorithm. The value MUST be multiple
# of 2^21. Defalut: 25 * 2^21
#
# HASH_STRING is a string value of a ja5h hash calculated from the HTTP request.
# You can find these values in the access log or in ClickHouse-based
# analytics.
#
# CONNECTIONS_PER_SEC is a number of allowed connections per second for
# clients identified by HASH_STRING.
#
# HTTP_REQUESTS_PER_SECOND is a number of allowed HTTP requests per second for
# the clients identified by HASH_STRING.
#
# Examples:
# ja5h storage_size=2097152 {
# hash deadbeef12345678 10 1000;
# ...
# hash 1234abcdeeaabbcc 0 0;
# }
#
# Default:
# No HTTP filtering applied.
#
10 changes: 10 additions & 0 deletions fw/cfg.c
Original file line number Diff line number Diff line change
Expand Up @@ -1393,6 +1393,16 @@ tfw_cfg_parse_long(const char *s, long *out_long)
return kstrtol(s, base, out_long);
}

int
tfw_cfg_parse_ulonglong(const char *s, unsigned long long *out_ull)
{
int base = detect_base(&s);

if (!base)
return -EINVAL;
return kstrtoull(s, base, out_ull);
}

int
tfw_cfg_parse_uint(const char *s, unsigned int *out_uint)
{
Expand Down
94 changes: 70 additions & 24 deletions fw/cfg.h
Original file line number Diff line number Diff line change
Expand Up @@ -188,32 +188,77 @@ typedef struct {
(k) = (idx < (e)->attr_n ? (e)->attrs[(idx)].key : NULL), \
(v) = (idx < (e)->attr_n ? (e)->attrs[(idx)].val : NULL))

#define TFW_CFG_ENTRY_FOR_EACH_VAL(e, idx, v) \
for ((idx) = 0, (v) = (e)->vals[0]; \
(idx) < (e)->val_n; \
(idx)++, \
#define TFW_CFG_ENTRY_FOR_EACH_VAL(e, idx, v) \
for ((idx) = 0, (v) = (e)->vals[0]; \
(idx) < (e)->val_n; \
(idx)++, \
(v) = (idx < (e)->val_n ? (e)->vals[(idx)] : NULL))

#define TFW_CFG_CHECK_NO_ATTRS(spec, entry) \
if ((entry)->attr_n) { \
T_ERR_NL("%s: Arguments may not have the '=' sign\n", \
(spec)->name); \
return -EINVAL; \
}

#define TFW_CFG_CHECK_VAL_N(op, req, spec, entry) \
if (! ((entry)->val_n op (req)) ) { \
T_ERR_NL("%s: Invalid number of arguments: %zu, must " \
"be %s %d\n", (spec)->name, (entry)->val_n, \
#op, (req)); \
return -EINVAL; \
}

#define TFW_CFG_CHECK_VAL_DUP(name, val_was_set, code) \
if (val_was_set) { \
T_ERR_NL("Duplicate argument: '%s'\n", name); \
code; \
} \
#define TFW_CFG_CHECK_NO_ATTRS(spec, entry) \
do { \
if ((entry)->attr_n) { \
T_ERR_NL("%s: Arguments may not have " \
"the '=' sign\n", (spec)->name);\
return -EINVAL; \
} \
} while (0)

#define TFW_CFG_CHECK_ATTR_N(op, req, spec, entry) \
do { \
if (!((entry)->attr_n op (req))) { \
T_ERR_NL("%s: Invalid number of attributes: " \
"%zu, must be %s %d\n", \
(spec)->name, (entry)->attr_n, \
#op, (req)); \
return -EINVAL; \
} \
} while (0)

#define TFW_CFG_CHECK_ATTR_EQ_N(req, spec, entry) \
do { \
if (!((entry)->attr_n == (req))) { \
T_ERR_NL("%s: Invalid number of attributes: " \
"%zu, must be queal %d\n", \
(spec)->name, (entry)->attr_n, (req)); \
return -EINVAL; \
} \
} while (0)

#define TFW_CFG_CHECK_ATTR_LE_N(req, spec, entry) \
do { \
if (!((entry)->attr_n <= (req))) { \
T_ERR_NL("%s: Invalid number of attributes: " \
"%zu, must be less or equal %d\n", \
(spec)->name, (entry)->attr_n, (req)); \
return -EINVAL; \
} \
} while (0)

#define TFW_CFG_CHECK_VAL_N(op, req, spec, entry) \
do { \
if (!((entry)->val_n op (req))) { \
T_ERR_NL("%s: Invalid number of arguments: " \
"%zu, must be %s %d\n", \
(spec)->name, (entry)->val_n, #op, (req)); \
return -EINVAL; \
} \
} while (0)

#define TFW_CFG_CHECK_VAL_EQ_N(req, spec, entry) \
do { \
if (!((entry)->val_n == (req))) { \
T_ERR_NL("%s: Invalid number of arguments: " \
"%zu, must be equal %d\n", \
(spec)->name, (entry)->val_n, (req)); \
return -EINVAL; \
} \
} while (0)

#define TFW_CFG_CHECK_VAL_DUP(name, val_was_set, code) \
if (val_was_set) { \
T_ERR_NL("Duplicate argument: '%s'\n", name); \
code; \
} \
val_was_set = true;

/**
Expand Down Expand Up @@ -451,6 +496,7 @@ int tfw_cfg_check_val_n(const TfwCfgEntry *e, int val_n);
int tfw_cfg_check_single_val(const TfwCfgEntry *e);
int tfw_cfg_parse_int(const char *s, int *out_int);
int tfw_cfg_parse_long(const char *s, long *out_long);
int tfw_cfg_parse_ulonglong(const char *s, unsigned long long *out_ull);
int tfw_cfg_parse_uint(const char *s, unsigned int *out_uint);
int tfw_cfg_parse_bool(const char *in_str, bool *out_bool);
int tfw_cfg_parse_intvl(const char *s, unsigned long *i0, unsigned long *i1);
Expand Down
Loading