Accessing interfaces from a device context

[aascedu]

Hi everyone.

I am using Netbox with the Validity plugin. I think my problem is relevant to Netbox rather than the plugin!

So I want to create a test (just like a script for vanilla netbox) and I need to get all the interfaces from the device I am checking and then their name and ip.

I am trying to use device.interfaces.all()
so my source code looks like this :

def verify_interfaces(device):
	if "interfaces" not in device.config:
		return ["EMPTY Serialized"]
	if device.interfaces.exists() == False:
		return ["EMPTY Modelized"]
	modelized_interfaces = device.interfaces.all()
	return modelized_interfaces

__all__ = ['verify_interfaces']

And my job executes, but it runs until it timeouts. I am running the test for only one device that have only 7 interfaces linked so the question of load shouldn't be raised according to me.

Also, I get nothing "suspicious" in the console logs...

It just stays like that until the timeout is reached (1800secondes / 30mins)...

Have you guys ever faced this problem ? Please help ❤️
Arthur

[candlerb]

so my source code looks like this

That's not complete runnable code. Is this a Custom Script? Then please write a minimal Custom Script which demonstrates your problem and can be used to reproduce it on another system, and then can be adjusted to work.

You may also find it helpful to use nbshell to try out the attributes of a device object interactively:

# /opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py nbshell
...
>>> d = Device.objects.get(name="gw1")
>>> d.config
Traceback (most recent call last):
  File "/usr/lib/python3.12/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<console>", line 1, in <module>
AttributeError: 'Device' object has no attribute 'config'
>>> d.interfaces.exists()
True
>>> d.interfaces.all()
<RestrictedQuerySet [<Interface: ether1>, <Interface: ether2>, <Interface: ether3>, <Interface: ether4>, <Interface: ether5>, <Interface: pppoe-spare2>, <Interface: sfp1>, <Interface: sit1>, <Interface: vlan3>, <Interface: vlan250>, <Interface: vlan254>]>

From this, it would appear that your line

	if "interfaces" not in device.config:

is invalid. However, if that's the only problem I'd expect the script to fail, not to keep running indefinitely. (You might get some more info from journalctl -eu netbox-rq)

Use help(Device) inside nbshell to get documentation of the available attributes.

Perhaps what you want is:

>>> d.config_template
<ConfigTemplate: Mikrotik interface vlans>
>>> d.config_template.template_code
'{%- set dif = device.interfaces.exclude(type="virtual") %}
... snip rest ...

[aascedu]

Hello friend !
As I said, my context is similar to Netbox Vanilla Script but I am using Validity plugin to run complience tests.
The context of where I try to execute the code shouldn't matter as the Validity docs for writting complience tests says :

device

This is the representation of the current device the test is running against. The instance of NetBox Device model

device.state - special dictionary which contains all device [State](https://validity.readthedocs.io/en/latest/features/state/). Keys of the dictionary are either command labels or "config". Allows getting access to its values by attribute name, e.g. device.state.show_version is equal to device.state['show_version']

device.state.config, device.config - the serialized config of the Device

device.dynamic_pair - dynamic pair for this device, it is also Device instance. May be equal to None

Device has multiple built-in fields and methods that can be used in test expressions.

It's not the purpose of Validity to list all these attributes. If you want you can discover them by yourself using django shell.

cd netbox_folder/netbox
./manage.py shell

from dcim.models import Device
device = Device.objects.first()
dir(device)

This is how you can discover the whole list of device fields and methods.

When I access what I want inside the netbox shell, here is what I find :

>>> Device.objects.last().interfaces.all()
<RestrictedQuerySet [<Interface: GigabitEthernet0/0/0>, <Interface: GigabitEthernet0/0/1>, <Interface: GigabitEthernet0/1/0>, <Interface: GigabitEthernet0/1/1>, <Interface: GigabitEthernet0/1/2>, <Interface: GigabitEthernet0/1/3>, <Interface: Vlan1>]>
>>> Device.objects.last().interfaces.all().last().ip_addresses.get()
<IPAddress: ip1/31>

Another way :

>>> Device.objects.last().vc_interfaces()
<RestrictedQuerySet [<Interface: GigabitEthernet0/0/0>, <Interface: GigabitEthernet0/0/1>, <Interface: GigabitEthernet0/1/0>, <Interface: GigabitEthernet0/1/1>, <Interface: GigabitEthernet0/1/2>, <Interface: GigabitEthernet0/1/3>, <Interface: Vlan1>]>
>>> Device.objects.last().vc_interfaces().last().ip_addresses.get()
<IPAddress:ip1/31>

as you can see, I get the infos I try to access in the little python code and even further the IP addresses of one of these interface.
My problem is not located at device.config because this part is valid and already tested. But my real problem is accessing device.interfaces.all() which runs forever until timeout in my python code...

Maybe I should ask the plugin developper but it didn't seem relevant to his plugin to me

Thanks for your help.
If you have any clues, I am open to trying them out.

ANSWER : calling device.interfaces.all() would make my script run till time out, calling device.interfaces.all().ip_addresses.get() works. No idea why but hey my solution is here, sadly it is not relevant for help for other users...

[candlerb]

But my real problem is accessing device.interfaces.all() which runs forever until timeout in my python code

I'm very sure it doesn't run forever.

However, you may not realise that device.interfaces.all() doesn't return a list of devices. It returns a proxy object which will fetch those devices on demand. Therefore, if you return this object to code which expects a list, that's maybe why it was broken.

If you want to create a concrete list of devices, then you need list(device.interfaces.all())

You can demonstrate this in nbshell:

>>> d = Device.objects.all()
>>> type(d)
<class 'extras.querysets.ConfigContextModelQuerySet'>
>>> d
<ConfigContextModelQuerySet [<Device: Macbook-Pro-5>, ...>
>>> list(d)
[<Device: Macbook-Pro-5>, ...]

You can do certain things directly on the project object, such as iterate over it:

>>> for dev in d: print(dev)
...

To find the code which "runs forever", you should look at the code which consumes the return value from your verify_instances() function.

[aascedu]

Thank you for your help.
You're right I didn't consider it not being a list.
Also I am a little bit lost when trying to debug my code since I am using netbox docker which means I have netbox, netbox worker working on different container, which I didn't grasp the whole purpose of, I suppose it is written somewhere in the documentation.
Also using Validity plugin means it adds a layer of uncertainty to where things happen as I don't have knowledge of how they are executed !
I will look for where the return value from my verify instance function is used!
However I am pretty certain in the time out error as I have seen python error in the test reports saying something along the line of "execution exceeded 1800seconds, time out"
Thanks for your explanations and ideas!

ANSWER : Treating it as a list made it work thank you. I am not a big python dev so working with type is not my strongest ability!

[candlerb]

Also I am a little bit lost when trying to debug my code since I am using netbox docker which means I have netbox, netbox worker working on different container, which I didn't grasp the whole purpose of, I suppose it is written somewhere in the documentation.

That's just standard Docker practice: one container for each process - as opposed to, say, running a process supervisor inside a container which then runs other processes. Then your orchestration framework (docker-compose, or kuberenetes or whatever) takes on the role of the process supervisor, and can schedule them on different nodes.

I'm glad it's working now.