Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
b67dc67
feat: use `trusted_addresses` to handle `X-Forwarded-*` headers
SkyeYoung Aug 27, 2025
34b5953
chore
SkyeYoung Aug 27, 2025
5bcc9ff
fix
SkyeYoung Aug 27, 2025
a9b3e4c
feat: add to config.yaml.example
SkyeYoung Aug 27, 2025
1ab46cd
fix: handle xf headers in untrusted addresses
SkyeYoung Aug 27, 2025
b9cf720
fix: logic
SkyeYoung Aug 28, 2025
4f2ed0c
chore: format
SkyeYoung Aug 28, 2025
5411324
fix: lint
SkyeYoung Aug 28, 2025
5e12819
fix: lint
SkyeYoung Aug 28, 2025
197ce75
fix: old tests
SkyeYoung Aug 28, 2025
8a271a5
fix: old tests
SkyeYoung Aug 28, 2025
cd98ba8
test: add trusted_accesses cases
SkyeYoung Aug 29, 2025
809b1e9
try fix
SkyeYoung Aug 29, 2025
1a072b2
fix: update vars logic
SkyeYoung Aug 29, 2025
6614dc0
chore
SkyeYoung Aug 29, 2025
510b2b3
test: add cases
SkyeYoung Aug 29, 2025
16ecd73
docs: add notes to related docs
SkyeYoung Aug 29, 2025
1e3950e
chore: add more explain about the default process way
SkyeYoung Sep 1, 2025
1b6e5e6
chore: adjust config example
SkyeYoung Sep 2, 2025
f8c3aaa
chore: rm usless config
SkyeYoung Sep 2, 2025
247c208
test: add multi ips cases
SkyeYoung Sep 2, 2025
1d74451
test: add plugin cases
SkyeYoung Sep 2, 2025
31947b8
docs: adjust descs
SkyeYoung Sep 2, 2025
818cb0a
docs: adjust desc based on @moonming comments
SkyeYoung Sep 5, 2025
80d561f
fix: logic and cases
SkyeYoung Sep 5, 2025
3bad749
chore: add more cases
SkyeYoung Sep 5, 2025
0d8e1b8
chore
SkyeYoung Sep 5, 2025
395cf61
fix: cases
SkyeYoung Sep 5, 2025
9457206
fix: validate ip/cidr func
SkyeYoung Sep 5, 2025
3acc848
Merge remote-tracking branch 'upstream/master' into young/feat/truste…
SkyeYoung Sep 7, 2025
b9cfd9c
Merge branch 'master' into young/feat/trusted_addresses
SkyeYoung Sep 15, 2025
5fd40fc
Update docs/zh/latest/plugins/chaitin-waf.md
SkyeYoung Sep 15, 2025
528116c
chore: rename `update_var_x_forwarded_headers` to `set_upstream_x_for…
SkyeYoung Sep 15, 2025
c06b7c2
fix: solve comments by @moonming
SkyeYoung Sep 16, 2025
c7e372d
merge main
nic-6443 Sep 16, 2025
30ce9c2
reverse dependecy between schema_def.lua and schema.lua
nic-6443 Sep 16, 2025
bb614f2
chore: add comments
SkyeYoung Sep 16, 2025
5020eb4
Merge remote-tracking branch 'origin/young/feat/trusted_addresses' in…
SkyeYoung Sep 16, 2025
bccb5e6
fix: lint
SkyeYoung Sep 16, 2025
4a3cb12
Merge branch 'master' into young/feat/trusted_addresses
SkyeYoung Sep 17, 2025
37c639b
chore: rm code which is covered by json schema
SkyeYoung Sep 17, 2025
be9a275
Merge remote-tracking branch 'origin/young/feat/trusted_addresses' in…
SkyeYoung Sep 17, 2025
fa5a6b6
fix: lint
SkyeYoung Sep 17, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions apisix/cli/schema.lua
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ local jsonschema = require("jsonschema")
local pairs = pairs
local pcall = pcall
local require = require
local schema_def = require("apisix.schema_def")


local _M = {}
Expand Down Expand Up @@ -259,6 +260,14 @@ local config_schema = {
default = false,
description = "a global switch to disable upstream health checks",
},
trusted_addresses = {
Copy link
Member

@nic-6443 nic-6443 Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ref:

trusted_addresses = {
type = "array",
items = {anyOf = core.schema.ip_def},
minItems = 1
},

we can add more restrction to schema like this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding core.schema.ip_def doesn't seem to work.

https://github.com/apache/apisix/blob/8bb6802bd2b2ccd4dabe78e90beb8f87ec046035/apisix/schema_def.lua depends on https://github.com/apache/apisix/blob/0fd582bc299a9d90893d3843cd3ec21f217cb40e/apisix/core/schema.lua

Using it will cause an error /usr/local/openresty//luajit/bin/luajit: ./apisix/core/lrucache.lua:22: module '\''resty.lrucache'\'' not found:

image image

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using it will cause an error /usr/local/openresty//luajit/bin/luajit: ./apisix/core/lrucache.lua:22: module '''resty.lrucache''' not found:

What is the reason for this error?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason for this error is that there is no openresty environment in the apisix cli, which makes it impossible to use the resty.lru library.
I have considered the dependency relationship between schema_def.lua and schema.lua, and I think their dependency relationship is reversed. It should be that schema.lua depends on schema_def.lua, allowing schema_def.lua to retain only the definition of schema, in line with the naming of this file.
Added a new commit to this PR to fix this dependency issue: 30ce9c2

type = "array",
minItems = 1,
items = {
anyOf = schema_def.ip_def,
},
uniqueItems = true
},
}
},
nginx_config = {
Expand Down
2 changes: 1 addition & 1 deletion apisix/core.lua
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ return {
request = require("apisix.core.request"),
response = require("apisix.core.response"),
lrucache = require("apisix.core.lrucache"),
schema = require("apisix.schema_def"),
schema = require("apisix.core.schema"),
string = require("apisix.core.string"),
ctx = require("apisix.core.ctx"),
timer = require("apisix.core.timer"),
Expand Down
7 changes: 7 additions & 0 deletions apisix/core/schema.lua
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,10 @@

local jsonschema = require('jsonschema')
local lrucache = require("apisix.core.lrucache")
local schema_def = require("apisix.schema_def")
local cached_validator = lrucache.new({count = 1000, ttl = 0})
local pcall = pcall
local error = error

local _M = {
version = 0.3,
Expand Down Expand Up @@ -68,4 +70,9 @@ end

_M.valid = get_validator

setmetatable(_M, {
__index = schema_def,
__newindex = function() error("no modification allowed") end,
})

return _M
95 changes: 80 additions & 15 deletions apisix/init.lua
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ local ctxdump = require("resty.ctxdump")
local debug = require("apisix.debug")
local pubsub_kafka = require("apisix.pubsub.kafka")
local resource = require("apisix.resource")
local trusted_addresses_util = require("apisix.utils.trusted-addresses")
local ngx = ngx
local get_method = ngx.req.get_method
local ngx_exit = ngx.exit
Expand Down Expand Up @@ -162,6 +163,8 @@ function _M.http_init_worker()
-- To ensure that all workers related to Prometheus metrics are initialized,
-- we need to put the initialization of the Prometheus plugin here.
plugin.init_prometheus()

trusted_addresses_util.init_worker()
end


Expand Down Expand Up @@ -292,21 +295,6 @@ end

local function set_upstream_headers(api_ctx, picked_server)
set_upstream_host(api_ctx, picked_server)

local proto = api_ctx.var.http_x_forwarded_proto
if proto then
api_ctx.var.var_x_forwarded_proto = proto
end

local x_forwarded_host = api_ctx.var.http_x_forwarded_host
if x_forwarded_host then
api_ctx.var.var_x_forwarded_host = x_forwarded_host
end

local port = api_ctx.var.http_x_forwarded_port
if port then
api_ctx.var.var_x_forwarded_port = port
end
Comment on lines -295 to -309
Copy link
Member Author

@SkyeYoung SkyeYoung Aug 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apisix.handle_upstream = orig_handle_upstream
apisix.http_balancer_phase = orig_http_balancer_phase
else
-- replace the upstream and balancer module
apisix.handle_upstream = ai_upstream
apisix.http_balancer_phase = ai_http_balancer_phase

Because the code above will replace handle_upstream with ai_upstream below:

local function ai_upstream()
core.log.info("enable sample upstream")
end

This will cause this deleted part of the code to be skipped.

Now that we need to handle X-Forwarded-*, we need to update these var_x_forwarded_*, so we need to extract this part of the code as update_var_x_forwarded_headers.

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $var_x_forwarded_proto;
proxy_set_header X-Forwarded-Host $var_x_forwarded_host;
proxy_set_header X-Forwarded-Port $var_x_forwarded_port;

end


Expand Down Expand Up @@ -599,6 +587,79 @@ function _M.handle_upstream(api_ctx, route, enable_websocket)
end


local function handle_x_forwarded_headers(api_ctx)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add some comments to explain the purpose of each code segment

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

local addr_is_trusted = trusted_addresses_util.is_trusted(api_ctx.var.realip_remote_addr)

-- Only untrusted values need to be overwritten or cleared.
if not addr_is_trusted then
-- store the original x-forwarded-* headers
-- to allow future use by other plugins or processes
api_ctx.var.original_x_forwarded_proto = api_ctx.var.http_x_forwarded_proto
api_ctx.var.original_x_forwarded_host = api_ctx.var.http_x_forwarded_host
api_ctx.var.original_x_forwarded_port = api_ctx.var.http_x_forwarded_port
api_ctx.var.original_x_forwarded_for = api_ctx.var.http_x_forwarded_for

-- trusted ones
-- ref: ngx_tpl.lua#L831-L840
--
-- these values are observed directly by APISIX and cannot be forged,
-- making them highly credible.
local proto = api_ctx.var.scheme
local host = api_ctx.var.host
local port = api_ctx.var.server_port

-- override the x-forwarded-* headers to the trusted ones.
-- make sure that the correct values ​​are obtained
-- in the subsequent stages using `core.request.header`.
core.request.set_header(api_ctx, "X-Forwarded-Proto", proto)
core.request.set_header(api_ctx, "X-Forwarded-Host", host)
core.request.set_header(api_ctx, "X-Forwarded-Port", port)
-- later processed in ngx_tpl by `$proxy_add_x_forwarded_for`.
core.request.set_header(api_ctx, "X-Forwarded-For", nil)

-- update the cached value in http_x_forwarded_* to the trusted ones.
-- make sure that the correct values ​​are obtained
-- in the subsequent stages using `var.http_x_forwarded_*`.
api_ctx.var.http_x_forwarded_proto = proto
api_ctx.var.http_x_forwarded_host = host
api_ctx.var.http_x_forwarded_port = port
api_ctx.var.http_x_forwarded_for = nil
end
end


-- in ngx_tpl.lua#L831-L840,
-- there is such code: `proxy_set_header X-Forwarded-XXX $var_x_forwarded_xxx;`
-- that is, set the `X-Forwarded-XXX` header through `var_x_forwarded_xxx`.
--
-- therefore, it is necessary to set the trusted `http_x_forwarded_xxx` to `var_x_forwarded_xxx`.
-- So that the `X-Forwarded-XXX` header is updated to a trusted value.
--
-- currently, only following headers are updated through these variables:
-- - X-Forwarded-Proto
-- - X-Forwarded-Port
-- - X-Forwarded-Host
--
-- the `X-Forwarded-For` header is not updated through these variables.
-- because it is set by the `proxy_add_x_forwarded_for` directive.
local function set_upstream_x_forwarded_headers(api_ctx)
local proto = api_ctx.var.http_x_forwarded_proto
if proto then
api_ctx.var.var_x_forwarded_proto = proto
end
Comment on lines +646 to +649
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not set X-Forwarded-Proto as http_x_forwarded_proto?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite understand.

  1. This function is extracted from the following code without any adjustments.

https://github.com/apache/apisix/pull/12551/files/528116c496c9c5ea3ef2b3030e1da6596bac67cb#diff-7439ffe94cc4bcad12066d5d83f6c1004b8e9b3ff5097b645fa7c700eb3b3743L293-L307

  1. X-Forwarded-Proto in the header == http_x_forwarded_proto


local port = api_ctx.var.http_x_forwarded_port
if port then
api_ctx.var.var_x_forwarded_port = port
end

local host = api_ctx.var.http_x_forwarded_host
if host then
api_ctx.var.var_x_forwarded_host = host
end
end


function _M.http_access_phase()
-- from HTTP/3 to HTTP/1.1 we need to convert :authority pesudo-header
-- to Host header, so we set upstream_host variable here.
Expand Down Expand Up @@ -648,6 +709,8 @@ function _M.http_access_phase()
api_ctx.var.real_request_uri = api_ctx.var.request_uri
api_ctx.var.request_uri = api_ctx.var.uri .. api_ctx.var.is_args .. (api_ctx.var.args or "")

handle_x_forwarded_headers(api_ctx)

router.router_http.match(api_ctx)

local route = api_ctx.matched_route
Expand Down Expand Up @@ -754,6 +817,8 @@ function _M.http_access_phase()
end

_M.handle_upstream(api_ctx, route, enable_websocket)

set_upstream_x_forwarded_headers(api_ctx)
end


Expand Down
9 changes: 0 additions & 9 deletions apisix/schema_def.lua
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,8 @@
-- See the License for the specific language governing permissions and
-- limitations under the License.
--
local schema = require('apisix.core.schema')
local table_insert = table.insert
local table_concat = table.concat
local setmetatable = setmetatable
local error = error

local _M = {version = 0.5}

Expand Down Expand Up @@ -1093,10 +1090,4 @@ _M.plugin_injected_schema = {
}


setmetatable(_M, {
__index = schema,
__newindex = function() error("no modification allowed") end,
})


return _M
52 changes: 52 additions & 0 deletions apisix/utils/trusted-addresses.lua
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
--
-- Licensed to the Apache Software Foundation (ASF) under one or more
-- contributor license agreements. See the NOTICE file distributed with
-- this work for additional information regarding copyright ownership.
-- The ASF licenses this file to You under the Apache License, Version 2.0
-- (the "License"); you may not use this file except in compliance with
-- the License. You may obtain a copy of the License at
--
-- http://www.apache.org/licenses/LICENSE-2.0
--
-- Unless required by applicable law or agreed to in writing, software
-- distributed under the License is distributed on an "AS IS" BASIS,
-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-- See the License for the specific language governing permissions and
-- limitations under the License.
--
local require = require
local core = require("apisix.core")

local trusted_addresses_matcher

local _M = {}


function _M.init_worker()
local local_conf = core.config.local_conf()
local trusted_addresses = core.table.try_read_attr(local_conf, "apisix", "trusted_addresses")

if not trusted_addresses then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Member Author

@SkyeYoung SkyeYoung Sep 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This situation is already covered by TEST 1. I will add an info-level log for assertion.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

core.log.info("trusted_addresses is not configured")
return
end

local matcher, err = core.ip.create_ip_matcher(trusted_addresses)
if not matcher then
core.log.error("failed to create ip matcher for trusted_addresses: ", err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just defensive programming.

The validation of ip/cidr has already been handled in validate_trusted_addresses. Currently, no other cases have been found that require assertion here.

Therefore, only validate_trusted_addresses has been tested.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might ask why verification is needed upfront. This is because I just discovered that https://github.com/api7/lua-resty-ipmatcher has the following issue that needs to be fixed:

Image

This library considers 1.0.0.0/33 to be a valid CIDR.

return
end

trusted_addresses_matcher = matcher
end


function _M.is_trusted(address)
if not trusted_addresses_matcher then
core.log.info("trusted_addresses_matcher is not initialized")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

return false
end
return trusted_addresses_matcher:match(address)
end

return _M
4 changes: 4 additions & 0 deletions conf/config.yaml.example
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,10 @@ apisix:
# or (standalone mode) the config isn't loaded yet either via file or Admin API.
# disable_upstream_healthcheck: false # A global switch for healthcheck. Defaults to false.
# When set to true, it overrides all upstream healthcheck configurations and globally disabling healthchecks.
# trusted_addresses: # When configured, APISIX will trust the `X-Forwarded-*` Headers
# - 127.0.0.1 # passed in requests from the IP/CIDR in the list.
# - 172.18.0.0/16 # CAUTION: When not configured or the request from an untrusted address,
# APISIX will override `X-Forwarded-*` headers with trusted values.
# fine tune the parameters of LRU cache for some features like secret
lru:
secret:
Expand Down
3 changes: 3 additions & 0 deletions docs/en/latest/plugins/chaitin-waf.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,10 @@ The examples below demonstrate how you can configure chaitin-waf Plugin for diff
Before proceeding, make sure you have installed [Chaitin WAF (SafeLine)](https://docs.waf.chaitin.com/en/GetStarted/Deploy).

:::note
Only `X-Forwarded-*` headers sent from addresses in the `apisix.trusted_addresses` configuration (supports IP and CIDR) will be trusted and passed to plugins or upstream. If `apisix.trusted_addresses` is not configured or the IP is not within the configured address range, all `X-Forwarded-*` headers will be overridden with trusted values.
:::

:::note
You can fetch the `admin_key` from `config.yaml` and save to an environment variable with the following command:

```bash
Expand Down
4 changes: 4 additions & 0 deletions docs/en/latest/plugins/real-ip.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,10 @@ The Plugin is functionally similar to NGINX's [ngx_http_realip_module](https://n
| trusted_addresses | array[string] | False | | array of IPv4 or IPv6 addresses (CIDR notation acceptable) | Trusted addresses that are known to send correct replacement addresses. This configuration sets the [`set_real_ip_from`](https://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from) directive. |
| recursive | boolean | False | False | | If false, replace the original client address that matches one of the trusted addresses by the last address sent in the configured `source`.<br />If true, replace the original client address that matches one of the trusted addresses by the last non-trusted address sent in the configured `source`. |

:::note
Only `X-Forwarded-*` headers sent from addresses in the `apisix.trusted_addresses` configuration (supports IP and CIDR) will be trusted and passed to plugins or upstream. If `apisix.trusted_addresses` is not configured or the IP is not within the configured address range, all `X-Forwarded-*` headers will be overridden with trusted values.
:::

:::note
If the address specified in `source` is missing or invalid, the Plugin would not change the client address.
:::
Expand Down
4 changes: 4 additions & 0 deletions docs/zh/latest/plugins/chaitin-waf.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,10 @@ description: chaitin-waf 插件与长亭雷池 WAF 集成,以检测和阻止

继续操作之前,请确保您已安装 [长亭雷池 WAF](https://docs.waf.chaitin.com/en/GetStarted/Deploy)。

:::note
只有发送自 `apisix.trusted_addresses` 配置(支持 IP 和 CIDR)地址的 `X-Forwarded-*` 头才会被信任,并传递给插件或上游。如果未配置 `apisix.trusted_addresses` 或 ip 不在配置地址范围内的,`X-Forwarded-*` 头将全部被可信值覆盖。
:::

:::note

您可以这样从 `config.yaml` 中获取 `admin_key` 并存入环境变量:
Expand Down
4 changes: 4 additions & 0 deletions docs/zh/latest/plugins/real-ip.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,10 @@ description: real-ip 插件允许 Apache APISIX 通过 HTTP 请求头或 HTTP
| trusted_addresses | array[string] | 否 | | IPv4 或 IPv6 地址数组(接受 CIDR 表示法) | 已知会发送正确替代地址的可信地址。此配置设置 [`set_real_ip_from`](https://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from) 指令。 |
| recursive | boolean | 否 | false | | 如果为 false,则将匹配可信地址之一的原始客户端地址替换为配置的 `source` 中发送的最后一个地址。<br />如果为 true,则将匹配可信地址之一的原始客户端地址替换为配置的 `source` 中发送的最后一个非可信地址。 |

:::note
只有发送自 `apisix.trusted_addresses` 配置(支持 IP 和 CIDR)地址的 `X-Forwarded-*` 头才会被信任,并传递给插件或上游。如果未配置 `apisix.trusted_addresses` 或 ip 不在配置地址范围内的,`X-Forwarded-*` 头将全部被可信值覆盖。
:::

:::note
如果 `source` 属性中设置的地址丢失或者无效,该插件将不会更改客户端地址。
:::
Expand Down
2 changes: 2 additions & 0 deletions t/APISIX.pm
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,8 @@ apisix:
tcp:
- 9100
enable_resolv_search_opt: false
trusted_addresses:
- "127.0.0.1"
_EOC_

my $etcd_enable_auth = $ENV{"ETCD_ENABLE_AUTH"} || "false";
Expand Down
Loading
Loading