Nested Module Bay Label's don't support {module} like Interface Label's do

[sol1-matt]

Deployment Type

Self-hosted

NetBox Version

v4.3.3

Python Version

3.11

Steps to Reproduce

1. Create a new device type with module bays with properties Name = Bay [A-B], Label = [A-B], Position = [A-B]
2. Create a new module type named 24 port with SFP with module bays with properties Name = SFP {module}-[1-2], Label = {module}-[1-2], Position = [1-2]
3. Add to the Step 2 module type more module bays with properties Name = SFP {module}-[21-24], Label = [21-24], Position = [21-24]
4. Add to the Step 2 module type 1GE interfaces with properties Name = Eth {module}-[1-20], Label = {module}-[1-20]
5. Create a new module type named 10g SFP (2 nested) with 10GE interfaces with properties Name = Eth {module}-{module}, Label = {module}-{module}
6. Create a new device from the Step 1 device type
7. Add a 24 port with SFP module to the device in Bay A
8. Add a 10g SFP (2 nested) to module bay SFP-A-21

Expected Behavior

The Label on the nested module bays should inherit the position from the parent bay: eg A-21.
This is the same behavior that interfaces have.

Observed Behavior

The labels on the bays show {module}-21 instead of A-21

Note in the second screenshot the interface labels were created correctly with the 10G interface getting the label A-21

[sol1-matt]

While this bug isn't directly related to #19796 it is part of solving the problem for nested items and the naming of them.

[sleepinggenius2]

It looks like the fix for #17436 was implemented inconsistently with the other device components. I think reverting that change and instead changing the instantiate method in ModuleBayTemplate to the following should resolve both issues in a more consistent way:

def instantiate(self, **kwargs):
    return self.component_model(
        name=self.resolve_name(kwargs.get('module')),
        label=self.resolve_label(kwargs.get('module')),
        position=self.position,
        **kwargs
    )

NOTE: The current implementation of #17436 seems to replace all {module} placeholders in the name with the same value and does not populate the placeholders from the entire tree, like it does for other components, so this would be a change in functionality and I'm not sure which is intended, but I feel like consistency with other components would be the "expected" outcome.

[fzs]

What is the argument to not resolve the position, too?

[sleepinggenius2]

Resolving the position wouldn't make any sense, as it does not support {module} placeholders. The position is a fixed value that is used to compute the list of values that are used in resolving other fields. If it supported a {module} placeholder, then you would end up with duplicated position values when resolving other fields, which I cannot think of a real world use case for. What would your use case be for that?

Examples of resolving the position:

Module: position = 1
  Sub-Module: position = {module}/2 => 1/2
    Interface: name = Eth{module}/{module} => Eth1/1/2

Module: position = 1
  Sub-Module: position = 2
    Sub-Sub-Module: position = {module}/{module}/3 => 1/2/3
      Interface: name = Eth{module}/{module}/{module} => Eth1/2/1/2/3

[fzs]

As explained in #19796, in the real world we face modules with interfaces that can be inserted into a module bay of a device, or a module bay of a module, in arbitraty levels. Example: SFPs.

Since the position does not support the {module} placeholder, it needs to be added to the interface name multiple times. Which means that the exact same module type needs to be created multiple times, just to get the interface name right. Which is superfluous, annoying and hard to understand for the users.

IMHO the position should support the {module} placeholder. The position of a module bay should not be seen isolated only in the context of a module, but in context of the complete system, i.e. device and arbitrary level of modules. Resolving the position would then form the interface name in a very natural way.

[sleepinggenius2]

Honestly, we already have to create multiple models of the same SFP because of the way that different vendors do interface naming, so adding one more to support direct linecard insertion vs MPA insertion (which are the only two levels that we have encountered in our environment) has not been a huge problem. Every solution that I've seen so far has made the assumption that all device types use the same naming for interfaces, which could not be farther from the truth. The same SFP-10G-LR optic in our network could represent an interface named at least 1, 1/1/1, Ethernet 1, Port 1, ten-gigabit-ethernet 1/1, Te0/0/1, TenGigE0/0/0/1, or FAC-1-1-1-1 depending on what device it's inserted into, so when you already have to create 8 models for it, what's one more to support TenGigE0/0/0/1 and TenGigE0/0/1/1? If you have a better solution to the interface naming problem that still supports provisioning, alarm enrichment, network reconciliation, etc., then I would be all about that.

We also have the need to support interfaces that depend on the position of their grandparent and not their direct parent, so the current system has been extremely helpful in that situation and is a use case that doesn't seem possible with your proposed solution.

[sol1-matt]

we already have to create multiple models of the same SFP because of the way that different vendors do interface naming

If anything this is a argument for supporting the {module} placeholder in position. With duplicate types we'll end up with a single module that could have multiple types that are per vendor per speed per nesting depth specific rather than your per vendor per speed duplicates at the moment.

3 vendor x 3 speeds = 6 module types
3 vendor x 3 speeds x 2 depths = 12 module types

This also screws up anything that is used to search for or track usage of a given module such as search, the Inventory plugin, etc....

A solution I'm working on at the moment for per vendor per speed interface names is to use events and a script to automatically rename interfaces on creation.

[fzs]

The same SFP-10G-LR optic in our network could represent an interface named at least 1, 1/1/1, Ethernet 1, Port 1, ten-gigabit-ethernet 1/1, Te0/0/1, TenGigE0/0/0/1, or FAC-1-1-1-1 depending on what device it's inserted into,

Interesting point, thank you. I am not sure I understand why the SFP use case would call for multiple module types if the position would support and resolve the {module} placeholder at every level. What you are saying is that the naming of the interface is very much dependent on the vendor of the device. Which sounds to me like it should be determined at the level of the device and not on the level of the module? So determining it at the level of the module where the interface is defined seems to be the least useful.

This maybe could be done with an interface name template located at the device type. Seems to be logical since the interface shows up at the device, not at the module.

My reasoning was, that if the position at each level already determines the name of the interface at this level, then e.g. for an SFP the interface name could just be {module}. So the path of positions already determine what the name of the interface is that ends up in this module bay.

This seems easier if device and module are from the same manufacturer, obviously. If they are not, and the module can be used in different devices which use different naming schemes, then we are at the same problem again that you brought up.

In my use case it would help. I have a HPE device with six module bays A-F. I can insert an HPE ethernet module the with 24 ports and end up with interfaces A1-A24. I can set the position to A at the device level and name the Eth interface {module}10 and I end up with what I need.
If the ethernet module as SFP ports at positions 21-24, then right now I have to have specific SPF modules to insert here and create the proper interface name by setting the position of the module bay on the ethernet module to e.g. 22.

What I would like to do is set the position to {module}22. This resolves to position A22 when the module type is inserted into the module bay A of the device. So now I can set the interface name on the SFP module to {module} and get the correct interface name A22.

My guess would be that this would have also covered the examples of interface names that you mentioned. But you brought up a good point that this only works when module and device come from the same vendor or the module only works in this vendors devices and thus adheres to the proper naming scheme.

If you have a better solution to the interface naming problem that still supports provisioning, alarm enrichment, network reconciliation, etc., then I would be all about that.

I don't know what the requirements would be for the additional issues you mentioned here. Again, it seems like the name of the interface should be determined closer to the device than the interface if it is on a module.

We also have the need to support interfaces that depend on the position of their grandparent and not their direct parent, so the current system has been extremely helpful in that situation and is a use case that doesn't seem possible with your proposed solution.

This is true if the parent is a module that might work in different scenarios where in one the module bays position plays into the name and in others it doesn't. Otherwise the position naming would simply be restricted to the grandparent.

But, yes, it seems like more cases would have to be considered to come up with a proper interface naming solution. I can't say that I find the current one that elegant where you have to create multiple types for the same thing just to get the interface name right.

[sleepinggenius2]

The current solution is definitely not elegant and one where I only have to create a module once would be really nice, as long as it can be done in a way that doesn't make it more complex than what we have currently. I like the idea of allowing the device type more control over the naming, but you're still going to have some challenges there. For example, I can have an SFP+ bay that will take either an SFP+ or SFP pluggable. If an SFP+ is inserted, I might get a name like TenGigE0/0/0/1, but inserting an SFP would make the name GigabitEthernet0/0/0/1. One way I can think of to support that would be to model the corresponding interfaces on the modules as SFP+ (10GE) and SFP (1GE) respectively. Some people might prefer to use 1000BASE-TX (1GE) for copper SFPs as well.

I think the introduction of module type profiles does provide an interesting new opportunity to convey even more information about a module than we were previously able to. Unfortunately, as they can be arbitrarily defined and are not required for a module, implementing something in the NetBox core to look for specific values would not be ideal. I would love to see an expanded use of callback functions within the platform to support some of these environment-specific challenges. One option would be to leverage the existing custom validator system for this, but unfortunately when a module type (or device type) is instantiated today, all of its components (except module bays) are bulk created, which bypasses the clean() step and therefore doesn't call custom validators. Since the instantiate, resolve_name, and resolve_label functions are still called though, being able to override something there would be interesting. For example, being able to register a function in the config for interfaces (I think it makes sense to allow for overriding per component type) that takes the same arguments as the instantiate function and gives you back a dictionary of values to pass to self.component_model(). This approach gives you the additional perk of setting default values on other fields too, for example a default description on an interface.

An example of what that callback function could look like:

# User-defined function
def interface_instantiation(template: InterfaceTemplate, **kwargs: Any) -> dict[str, Any]:
    if not template.module_type:
        return kwargs
    # Ex. Use module type profile attributes
    attrs = template.module_type.attributes
    if prefix := attrs.get('interface_prefix'):
        kwargs['name'] = prefix + kwargs['name']
    return kwargs
...
# Core
class InterfaceTemplate(ModularComponentTemplateModel):
    ...
    def _instantiate_values(self, **kwargs: Any) -> dict[str, Any]:
        module = kwargs.get('module')
        kwargs['name'] = self.resolve_name(module)
        kwargs['label'] = self.resolve_label(module)
        kwargs['type'] = self.type
        kwargs['enabled'] = self.enabled
        kwargs['mgmt_only'] = self.mgmt_only
        kwargs['poe_mode'] = self.poe_mode
        kwargs['poe_type'] = self.poe_type
        kwargs['rf_role'] = self.rf_role
        return kwargs

    def instantiate(self, **kwargs: Any) -> Interface:
        kwargs = self._instantiate_values(kwargs)
        if udf := get_interface_udf_from_config():
            kwargs = udf(self, kwargs)
        return self.component_model(**kwargs)

These changes could be made to ModularComponentTemplateModel to give the user better helper functions too:

class ModularComponentTemplateModel(ComponentTemplateModel):
    ...
    def get_module_positions(self, module: Module | None) -> list[str]:
        positions = list[str]()
        while module:
            positions.append(module.module_bay.position)
            module = module.module_bay.module
        positions.reverse()
        return positions

    def apply_module_positions(self, value: str, positions: Iterable[str]) -> str:
        if MODULE_TOKEN not in value:
            return value

        for position in positions:
            value = value.replace(MODULE_TOKEN, position, 1)
        return value

    def resolve_name(self, module: Module | None) -> str:
        positions = self.get_module_positions(self, module)
        return self.apply_module_positions(self.name, positions)

    def resolve_label(self, module: Module | None) -> str:
        positions = self.get_module_positions(self, module)
        return self.apply_module_positions(self.label, positions)
...
# User-defined function using new helpers
def interface_instantiation(template: InterfaceTemplate, **kwargs: Any) -> dict[str, Any]:
    if not template.module_type:
        return kwargs

    # If the number of positions is less than the number of placeholders, then add zeroes for the missing ones
    placeholder_count = template.name.count(MODULE_TOKEN)
    positions = template.get_module_positions(kwargs['module'])
    if len(positions) < placeholder_count:
        # Keep the last position value and add the zeroes before it
        positions = positions[:-1] + (['0'] * (placeholder_count - len(positions))) + positions[-1:]
        kwargs['name'] = template.apply_module_positions(template.name, positions)

    # Set a default description when the interface is disabled
    if not template.enabled:
        kwargs['description'] = 'OPEN'

    return kwargs

[sol1-matt]

What you are saying is that the naming of the interface is very much dependent on the vendor of the device. Which sounds to me like it should be determined at the level of the device and not on the level of the module?

@fzs with the use case I've been given to solve I originally though the same thing, but it is a combination of the vendor (manufacture or perhaps platform) plus the speed of the interface.

The examples I was looking at were GigabitEthernet1/1 vs TenGigabitEthernet1/1 for Cisco and ge-0/0/1 vs xe-0/0/1 for Juniper for gigabit vs 10 gigabit interfaces.