Custom Script - stuck at generating URL to specific id (Device,IPAddress, ..)

[carsten-AEI]

Hi,

sorry for bothering with this, but I'm currently stuck writing some hopefully helpful URLs for a simple custom script which should allow me to quickly rename a device and all related dns_name entries. The script itself works (see the very bottom for reference), but as I usually also need to change the related IP addresses manually and would like to link to them directly in the output.

However, while I get the url attribute when I query via cURL I don't seem to get this information directly within the custom script. I've seen #7054 and others but I'm too much of a Python n00b to be able to use the given advice.

When looking at self.request I could try to jumble together the API URL form HTTP_REFERER but I guess this would be quite error prone. vrs(addr) only yields something like

'_state': <django.db.models.base.ModelState object at 0x7f18770d0190>,
 'address': IPNetwork('1.2.3.4/32'),
 'assigned_object_id': 10178,
 'assigned_object_type_id': 5,
 'comments': '',
 'created': datetime.datetime(2021, 2, 26, 0, 0, tzinfo=datetime.timezone.utc),
 'custom_field_data': {'IP comment': None},
 'description': '',
 'dns_name': 'changed_dns_name',
 'id': 8372,
 'last_updated': datetime.datetime(2023, 8, 23, 6, 38, 2, 904050, tzinfo=datetime.timezone.utc),
 'nat_inside_id': None,
 'role': '',
 'status': 'dhcp',
 'tenant_id': 1,
 'vrf_id': None}

and here I'm stuck. Anyone who could give me a nudge how to proceed?

Thanks a lot in advance!

class ChangeDeviceName(Script):
    class Meta:
        name = "Rename device"
        description = "Change name of device and related interfaces"
        commit_default = False

    cur_name = StringVar(label="Current device name")
    new_name = StringVar(label="Current device name")
    update_dns = BooleanVar(label="Update dns names?", default=True)

    def run(self, data, commit):
        # some checks: current name should be unique and new name should not be taken yet
        dev = Device.objects.get(name=str(data['cur_name']))
        tmp = Device.objects.filter(name=str(data['new_name']))
        if len(tmp) > 0:
            return self.log_failure(f"Device name {data['new_name']} seems to exist already!")

        output = [f"Found device '{dev.name}' with id {dev.id}"]

        # change name of dev
        dev.name = str(data['new_name'])
        dev.save()

        if data['update_dns']:
            pattern = re.compile(str(data['cur_name']))
            # iterate over all related interfaces
            ifaces = dev.interfaces.all()
            for addr in IPAddress.objects.filter(interface__in=ifaces):
                old = addr.dns_name
                if pattern.search(old):
                    addr.dns_name = re.sub(pattern, str(data['new_name']), addr.dns_name, count=1)
                    output.append(f"Changed {old} to {addr.dns_name}")
                    addr.save()

        return '\n'.join(output)

[candlerb]

However, while I get the url attribute when I query via cURL I don't seem to get this information directly within the custom script.

The Netbox REST API, and the Django ORM used within custom scripts, are two completely separate things.

Netbox provides a helper get_absolute_url to turn a model instance into its URL, although actually it's just a relative URL. For example, this is what I see in nbshell:

/opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py nbshell
### NetBox interactive shell (netbox3)
### Python 3.8.10 | Django 4.1.10 | NetBox 3.5.8
### lsmodels() will show available models. Use help(<model>) for more info.
>>> ip = IPAddress.objects.first()
>>> ip.get_absolute_url()
'/ipam/ip-addresses/467/'
>>>

If you want an absolute URL, I believe you'll need to prepend the base yourself - e.g. by wrapping it in request.build_absolute_uri(...). (See how dcim/svg/racks.py and dcim/api/views.py does this for absolute links in rack elevation SVGs)

Note that if you use the logging functions within a custom script, these support Markdown rendering which means the log messages can easily contain clickable links. For example:

self.log_success(f"Created VM [{vm.name}](/virtualization/virtual-machines/{vm.id}/)")

[carsten-AEI]

yepp, both present:

[...]
POST /netbox/extras/scripts/change_name/ChangeDeviceName/ HTTP/1.0
X-Forwarded-Host: 172.29.50.115
X-Real-IP: ::ffff:172.31.254.0
X-Forwarded-Proto: https
Host: 127.0.0.1:8001
Connection: close
Content-Length: 723
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/116.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.7,de;q=0.3
Accept-Encoding: gzip, deflate, br
Referer: https://172.29.50.115/netbox/extras/scripts/change_name/ChangeDeviceName/
Content-Type: multipart/form-data; boundary=---------------------------390705613117421445642087380259
Origin: https://172.29.50.115
Cookie: csrftoken=TOKEN sessionid=SESSION
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: same-origin
Sec-Fetch-User: ?1
[..]
Server: gunicorn
Date: Thu, 24 Aug 2023 07:11:40 GMT
Connection: close
Content-Type: text/html; charset=utf-8
Location: /netbox/extras/scripts/results/131/
X-Request-ID: 398dd334-7320-49c7-a985-4bc0893cbe58
Vary: HX-Request, Cookie, origin
X-Content-Type-Options: nosniff
Referrer-Policy: same-origin
Cross-Origin-Opener-Policy: same-origin
X-Frame-Options: SAMEORIGIN
Content-Length: 0
[..]
GET /netbox/extras/scripts/results/131/ HTTP/1.0
X-Forwarded-Host: 172.29.50.115
X-Real-IP: ::ffff:172.31.254.0
X-Forwarded-Proto: https
Host: 127.0.0.1:8001
Connection: close
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/116.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.7,de;q=0.3
Accept-Encoding: gzip, deflate, br
Referer: https://172.29.50.115/netbox/extras/scripts/change_name/ChangeDeviceName/
Cookie: csrftoken=TOKEN; sessionid=SESSION
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: same-origin
Sec-Fetch-User: ?1
[...]
Server: gunicorn
Date: Thu, 24 Aug 2023 07:11:40 GMT
Connection: close
Content-Type: text/html; charset=utf-8
X-Request-ID: 335dd1c7-ec72-45ea-806f-191b8edf1855
Vary: HX-Request, Cookie, origin
X-Content-Type-Options: nosniff
Referrer-Policy: same-origin
Cross-Origin-Opener-Policy: same-origin
X-Frame-Options: SAMEORIGIN
Content-Length: 87125
Set-Cookie: csrftoken=TOKEN; expires=Thu, 22 Aug 2024 07:11:40 GMT; Max-Age=31449600; Path=/netbox; SameSite=Lax
[..]
<p>{'CONTENT_LENGTH': '723', 'CONTENT_TYPE': 'multipart/form-data; boundary=---------------------------390705613117421445642087380259', 'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,<em>/</em>;q=0.8', 'HTTP_ACCEPT_ENCODING': 'gzip, deflate, br', 'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.7,de;q=0.3', 'HTTP_HOST': '127.0.0.1:8001', 'HTTP_REFERER': 'https://172.29.50.115/netbox/extras/scripts/change_name/ChangeDeviceName/', 'HTTP_USER_AGENT': 'Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/116.0', 'QUERY_STRING': '', 'REMOTE_ADDR': '127.0.0.1', 'REQUEST_METHOD': 'POST', 'SERVER_NAME': '127.0.0.1', 'SERVER_PORT': '8001'}</p></div></td>

Thus, no clue here either. Given that I have a work-around with the nginx.conf and seem to be the only one hitting this so far, shall we assume PEBKAC here and that I misconfigured something here?

[candlerb]

I think it's some filtering going on in Netbox, and that your particular use case hasn't been considered. Perhaps there's an allowlist of headers which end up in request.META, and some are missing.

[candlerb]

A quick grep turns up HTTP_REQUEST_META_SAFE_COPY in netbox/utilities/constants.py. You could try editing that and see what happens? If that solves your problem, you could raise a FR to increase the set of allowed headers. HTTP_X_FORWARDED_PROTO seems like an important one too, to generate http:// or https:// links correctly.

[carsten-AEI]

Sorry for the delay, I first needed to understand Python logging and how I could enabled some debug output from copy_safe_request(request) in netbox/utilities/utils.py to be useful.

With

diff --git a/netbox/utilities/constants.py b/netbox/utilities/constants.py
index 366d8f7..a0657f8 100644
--- a/netbox/utilities/constants.py
+++ b/netbox/utilities/constants.py
@@ -45,6 +45,8 @@ HTTP_REQUEST_META_SAFE_COPY = [
     'HTTP_REFERER',
     'HTTP_USER_AGENT',
     'HTTP_X_FORWARDED_FOR',
+    'HTTP_X_FORWARDED_HOST',
+    'HTTP_X_FORWARDED_PROTO',
     'QUERY_STRING',
     'REMOTE_ADDR',
     'REMOTE_HOST',

I finally get those extra bits into META:

{:CONTENT_LENGTH=>"723",
 :CONTENT_TYPE=>"multipart/form-data; boundary=---------------------------178440007540120566012032856803",
 :HTTP_ACCEPT=>"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,/;q=0.8",
 :HTTP_ACCEPT_ENCODING=>"gzip, deflate, br",
 :HTTP_ACCEPT_LANGUAGE=>"en-US,en;q=0.7,de;q=0.3",
 :HTTP_HOST=>"127.0.0.1:8001",
 :HTTP_REFERER=>"https://172.29.50.115/netbox/extras/scripts/change_name/ChangeDeviceName/",
 :HTTP_USER_AGENT=>"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/116.0",
 :HTTP_X_FORWARDED_HOST=>"172.29.50.115",
 :HTTP_X_FORWARDED_PROTO=>"https",
 :QUERY_STRING=>"",
 :REMOTE_ADDR=>"127.0.0.1",
 :REQUEST_METHOD=>"POST",
 :SERVER_NAME=>"127.0.0.1",
 :SERVER_PORT=>"8001"}

So, shall I try to formulate a FR with that?

[candlerb]

Yes, that sounds good to me. I suggest you add HTTP_X_REAL_IP to that list too, since that's another header that the default nginx config supplies.