Resource exhaustion attack on TCP server

[kpp]

It seems like there is a limit for incoming unconfirmed connections but there are no limits for incoming confirmed connections. So it is easy to DoS servers with low -n: file descriptors ulimit.

[sudden6]

ping @iphydf @nurupo

[iphydf]

@sudden6 yes, we've been aware of this for a few years. While there are some things we can do to mitigate, it's unclear how to protect against DDoS of this type. Increasing the fd ulimit is necessary for bootstrap nodes even for normal operations. Internet servers (and bootstrap nodes) would typically set it to 32768 these days. We can document that as a recommendation. Beyond that, if someone makes many TCP connections from many unique IP addresses, there is nothing we can do. For single machine DoS, it's not that easy to sustain this attack, and it's very temporary, computationally expensive, and targeted. A sane firewall config limiting number of open connections per unique IP address would mitigate this.

[sudden6]

Ok, I was not aware that this is a common problem. I just wanted to try to notify the right people about this. If that's a common issue for servers, should we reduce the priority?

[nurupo]

Right, it's a common problem. The problem is that operating systems generally have a limit on how many TCP sockets a process can have open at the same time. For example, on my Linux system the default is 1024 file descriptors, after which any fds you try to open will error. Sockets, pipes, files, etc. are all fds. So, after a node has 1024 sockets open, it will serve only to those 1024 sockets, without being able to open up more. It won't crash or do anything bad, it just won't be able to have more than 1024 connections at the same time. All TCP applications are affected by fd limit: apache, nginx, socks servers, etc. Me and @LittleVulpix compared how many TCP connections our bootstrap daemons have, and it looks to be around 700-1000, so it might make sense to increase the number of fds tox-bootstrapd can use as it's close to hitting my system's default limit of 1024.

What you can do to get around it, is ask the OS to increase the fd limit for a particular program. Say raise it to 16k or 32k like iphydf has suggested. Toxcore being a library and tox-bootstrapd being a daemon, I think it's the most approriate that the fd limit is raised by the users of toxcore and tox-bootstrapd, so no code for raising fd limit in toxcore or tox-bootstrapd is necessary.

There are three actions we can take to address this:

1. Document that if you enable TCP Server you might want to to increase the fd limit for its normal operation as the default fd limit is generally too low. The best place for documenting this would probably be:
   • Add a comment for the tcp_port data member of Tox_Options struct in tox.h that enables the TCP Server, as well as the matching tox_options_set_tcp_port() setter function. This way client developers will see that they might want to raise the fd limit by a syscall to ulimit() or whatever is the Windows alternative if they use TCP Server.
   • Add a comment to tox-bootstrapd's README.md about increasing the fd limit, as well as set the limit to 16k or 32k in .service (systemd) and .sh (sysvinit) files we have, so that fd limit would be raised by anyone using those by default.
2. Add a new option to Tox_Options that will limit how many connections TCP Server is allowed to have. In case of tox-bootstrapd we don't care if toxcore uses up all fds that are available to the process, tox-bootstrapd doesn't create any fds after it starts toxcore's main loop, so this new option wouldn't be very useful fot it, but it will be useful for Tox clients that allow TCP Server to be enabled. Tox clients should be able to open files, sockets (e.g. to toxme.io), create pipes, etc. without worrying that libtoxcore will use up all of the available to the process fds, and you can't guarantee that this won't happen by simply bumping the fd limit too high, toxcore can still end up using all fds, so limiting the number of fds toxcore can use, i.e. adding an option for limiting the number of sockets TCP Server can have, sounds like the way to solve this. That way clients can limit TCP Server to, say, 8192 connections, and set ulimit to 16384 for the process, leaving the client with 8192 fds it can work with.
3. Make sure that we log all fd creation failures in toxcore to some logger level.