Get correct client_IP from HTTP_X_FORWARDED_FOR / REMOTE_ADDR

[davama]

Hello,

I have this snippet of a custom script:

        x_forwarded_for = self.request.META.get('HTTP_X_FORWARDED_FOR')
        if x_forwarded_for:
            remote_ip_address = x_forwarded_for
        else:
            remote_ip_address = self.request.META.get('REMOTE_ADDR')

        self.log_info(f"Running as user {remote_username} (IP: {remote_ip_address})...")

        return(remote_ip_address)

The output is 127.0.0.1 which is of course wrong

Here is my nginx config. It only proxies netbox:

server {
    listen x.x.x.252:443 ssl;
    server_name netbox.domain.com;
    ssl_certificate /etc/pki/tls/certs/server.crt;
    ssl_certificate_key /etc/pki/tls/private/server.key;
    ssl_protocols TLSv1.2;
    ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
    ssl_ecdh_curve secp384r1;
    ssl_prefer_server_ciphers on;
    ssl_stapling off;
    ssl_stapling_verify on;
    ssl_session_timeout 24h;
    ssl_session_cache shared:SSL:50m;
    ssl_session_tickets off;
    client_max_body_size 25m;
    resolver usdsb.domain.com valid=300s;
    location /static/ {
        alias /opt/netbox/netbox/static/;
    }
    location / {
	set_real_ip_from 127.0.0.1;
	set_real_ip_from x.x.x.252;
	real_ip_header X-Forwarded-For;
	real_ip_recursive on;
    	proxy_pass http://localhost:8001;
        proxy_set_header X-Forwarded-Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
	proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

I've played around with my nginx config but cannot figure out how to get the correct client_ip address and not nginx IP address. Im assuming nginx is the issue...

Any input is much appreciated!

Thank you,
Dave

[davama]

just to update:

i tried this nginx config:

server {
	listen 80;
    location /static/ {
        alias /opt/netbox/netbox/static/;
    }
	location / {
	proxy_pass http://127.0.0.1:8001;
	proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
	proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	}
}

with this python code:

        x_forwarded_for = self.request.META.get('HTTP_X_FORWARDED_FOR')
        remote_ip_address = x_forwarded_for
        self.log_info(f"Running as user {remote_username} (IP: {remote_ip_address})...")
        return(remote_ip_address)

The log output says:
Running as user davetest (IP: None)...

😞

[candlerb]

The simplest thing to do is just copy what the sample nginx.conf supplied with Netbox does.

You can use tcpdump to see what Nginx is sending to Netbox, and hence what it's putting in the X-Forwarded-For header.

tcpdump -i lo -nn -s0 -A tcp port 8001`

However I note:

	set_real_ip_from 127.0.0.1;
	set_real_ip_from x.x.x.252;

This shouldn't be necessary, unless you have another reverse proxy upstream from Netbox's Nginx; this means to trust the value provided in the incoming X-Forwarded-For header, if the request comes from one of those IPs. See realip docs.

I think all you need is

	proxy_set_header X-Forwarded-For $remote_addr;

although there may be some mangling if IPv6 addresses are used. See nginx docs

[davama]

@candlerb
thank you very much for the insightful input:

I've done as you suggested. Copied the exact config from the docs, and added x-forwaded-for :

...
location / {
        proxy_pass http://127.0.0.1:8001;
        proxy_set_header X-Forwarded-Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
	proxy_set_header X-Forwarded-For $remote_addr;
}
....

You can see my whole tcpdump here: https://pastebin.com/CZPfdyKh

From the dump i can see nginx sets the variables correctly (lines 16,98,989,1056) but django/gunicorn does not "receive/interpret" them?

Thanks again

[candlerb]

OK, so you can see X-Forwarded-For: ::ffff:x.x.x.48 is being sent. Is this not being seen in self.request.META.get('HTTP_X_FORWARDED_FOR')?

Perhaps you can log repr(self.request.META) and see what it contains.

[davama]

@candlerb

OK, so you can see X-Forwarded-For: ::ffff:x.x.x.48 is being sent. Is this not being seen in self.request.META.get('HTTP_X_FORWARDED_FOR')?

unfortunately no 😞

Perhaps you can log repr(self.request.META) and see what it contains.

Thank you for that input. So my script.py looks now like this:

        remote_ip_address = self.request.META.get('HTTP_X_FORWARDED_FOR') or self.request.META.get('REMOTE_ADDR')
        self.log_info(f"Running as user {remote_username} (IP: {remote_ip_address})...")
        self.log_info(repr(self.request.META))

Which gave me this log output

{'CONTENT_LENGTH': '503', 'CONTENT_TYPE': 'multipart/form-data; boundary=----WebKitFormBoundaryPrHAvPRqa4n0y5Bn', 'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.9', 'HTTP_ACCEPT_ENCODING': 'gzip, deflate, br', 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.9', 'HTTP_HOST': '127.0.0.1:8001', 'HTTP_REFERER': 'https://netbox.domain.com/extras/scripts/netbox.AddDevice/', 'HTTP_USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36', 'QUERY_STRING': '', 'REMOTE_ADDR': '127.0.0.1', 'REQUEST_METHOD': 'POST', 'SERVER_NAME': '127.0.0.1', 'SERVER_PORT': '8001'}

X-Forwarded-For doesnt exist... 😢

[candlerb]

In that case, I don't know what's stripping it. However I can replicate the problem with this script:

from extras.scripts import Script

class CheckIP(Script):
    class Meta:
        name = "Check IP"

    def run(self, data, commit):
        self.log_success("meta: %r" % self.request.META)

There are no HTTP_X_FORWARDED_* headers shown. I'm running behind Apache, and I can see with tcpdump that various X-Forwarded-XXX headers are added.

[candlerb]

I wrote some test middleware:

import sys
class ShowRequestMiddleware(object):

    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        print(repr(request.META), file=sys.stderr)
        return self.get_response(request)

Whether I insert this at the beginning or the end of the middleware chain, it shows the full set of headers. And I'm running this under manage.py runserver, not gunicorn, so that rules out gunicorn. Very annoying.

Ah... I think I found it here [fixed] - which leads me to this. It's not Django, it's not Gunicorn, it's Netbox that is doing this.

I will leave it to you to take this forward...

[davama]

@candlerb

Ah... I think I found it here - which leads me to this. It's not Django, it's not Gunicorn, it's Netbox that is doing this.

Your first link didnt work but i fixed here

Thank you for the input!!

I think i have an idea of what to do

[davama]

ok, after adding 'HTTP_X_FORWARDED_FOR', to netbox/utilities/constants.py rebuilt netbox ./upgrade.sh
and it works!!

note my nginx looks like so:

    location / {
        proxy_pass http://127.0.0.1:8001;
        proxy_set_header X-Forwarded-Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
	proxy_set_header X-Forwarded-For $remote_addr;
    }

will submit PR