Skip to content

Commit 0189a65

Browse files
authored
[Docs] Expand security doc with firewall info (vllm-project#18081)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
1 parent 55aa7af commit 0189a65

File tree

1 file changed

+39
-0
lines changed

1 file changed

+39
-0
lines changed

docs/source/deployment/security.md

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,45 @@ Key points from the PyTorch security guide:
5353
- Implement proper authentication and authorization for management interfaces
5454
- Follow the principle of least privilege for all system components
5555

56+
## Security and Firewalls: Protecting Exposed vLLM Systems
57+
58+
While vLLM is designed to allow unsafe network services to be isolated to
59+
private networks, there are components—such as dependencies and underlying
60+
frameworks—that may open insecure services listening on all network interfaces,
61+
sometimes outside of vLLM's direct control.
62+
63+
A major concern is the use of `torch.distributed`, which vLLM leverages for
64+
distributed communication, including when using vLLM on a single host. When vLLM
65+
uses TCP initialization (see [PyTorch TCP Initialization
66+
documentation](https://docs.pytorch.org/docs/stable/distributed.html#tcp-initialization)),
67+
PyTorch creates a `TCPStore` that, by default, listens on all network
68+
interfaces. This means that unless additional protections are put in place,
69+
these services may be accessible to any host that can reach your machine via any
70+
network interface.
71+
72+
**From a PyTorch perspective, any use of `torch.distributed` should be
73+
considered insecure by default.** This is a known and intentional behavior from
74+
the PyTorch team.
75+
76+
### Firewall Configuration Guidance
77+
78+
The best way to protect your vLLM system is to carefully configure a firewall to
79+
expose only the minimum network surface area necessary. In most cases, this
80+
means:
81+
82+
- **Block all incoming connections except to the TCP port the API server is
83+
listening on.**
84+
85+
- Ensure that ports used for internal communication (such as those for
86+
`torch.distributed` and KV cache transfer) are only accessible from trusted
87+
hosts or networks.
88+
89+
- Never expose these internal ports to the public internet or untrusted
90+
networks.
91+
92+
Consult your operating system or application platform documentation for specific
93+
firewall configuration instructions.
94+
5695
## Reporting Security Vulnerabilities
5796

5897
If you believe you have found a security vulnerability in vLLM, please report it following the project's security policy. For more information on how to report security issues and the project's security policy, please see the [vLLM Security Policy](https://github.com/vllm-project/vllm/blob/main/SECURITY.md).

0 commit comments

Comments
 (0)