Automating Device Backups: Leveraging Netbox Dynamic Inventory for Ansible with Parsed Data to Handle Virtual Chassis and Switch Stacks

[chicocth]

Recently, I implemented Netbox at our company to manage our network devices efficiently. As part of our ongoing efforts to improve our infrastructure, I've been tasked with developing and deploying a solution to simplify the backup process for device configurations on a day-to-day basis.

I've transitioned all our device information from spreadsheets to Netbox, including hostnames and management IP addresses. To accommodate the dynamic allocation of IP addresses, I've created a custom field for Management IP in Netbox.

To facilitate the backup process, I've set up a netbox_inventory.yml file to fetch inventory data from Netbox. This file selects devices tagged with "backup" and flattens custom fields to easily access the management IP.

plugin: netbox.netbox.nb_inventory
validate_certs: false
config_context: false
flatten_custom_fields: true

device_query_filters:
  - status: "active"
  - platform: "ios"
  - tag: "backup"

And the result that gets printed out when I run a ansible-inventory -v --list -i netbox_inventory.yml

    "_meta": {
        "hostvars": {
            "PT-SW-MAINOFFICE02": {
                "asset_url": null,
                "device_roles": [
                    "switch"
                ],
                "device_types": [
                    "cisco-ws-c2960x-48fps-l"
                ],
                "firmware": null,
                "floor_id": null,
                "is_virtual": false,
                "lan_ip": null,
                "local_context_data": [
                    null
                ],
                "locations": [],
                "management_ip": "172.30.30.30",
                "manufacturers": [
                    "cisco"
                ],
                "platforms": [
                    "ios"
                ],
                "regions": [
                    "united-kingdom"
                ],
                "serial": "FCW2004A3HG",
                "services": [],
                "site_groups": [
                    "google"
                ],
                "sites": [
                    "google-peterborough"
                ],
                "status": {
                    "label": "Active",
                    "value": "active"
                },
                "tags": [
                    "backup"
                ],
                "wan_primary_ip": null,
                "wan_secondary_ip": null,
                "wireless_mac": null
            },
            "PW-SW-MAINOFFICE01-FOC2219V2ZE": {
                "asset_url": null,
                "device_roles": [
                    "switch"
                ],
                "device_types": [
                    "cisco-ws-c2960x-48fps-l"
                ],
                "firmware": null,
                "floor_id": null,
                "is_virtual": false,
                "lan_ip": null,
                "local_context_data": [
                    null
                ],
                "locations": [],
                "management_ip": "172.40.40.40",
                "manufacturers": [
                    "cisco"
                ],
                "platforms": [
                    "ios"
                ],
                "regions": [
                    "united-kingdom"
                ],
                "serial": "FOC2219V2ZE",
                "services": [],
                "site_groups": [
                    "google"
                ],
                "sites": [
                    "google-paddock-wood"
                ],
                "status": {
                    "label": "Active",
                    "value": "active"
                },
                "tags": [
                    "backup"
                ],
                "wan_primary_ip": null,
                "wan_secondary_ip": null,
                "wireless_mac": null
            },
            "PW-SW-MAINOFFICE01-FOC2219V2ZJ": {
                "asset_url": null,
                "device_roles": [
                    "switch"
                ],
                "device_types": [
                    "cisco-ws-c2960x-48fps-l"
                ],
                "firmware": null,
                "floor_id": null,
                "is_virtual": false,
                "lan_ip": null,
                "local_context_data": [
                    null
                ],
                "locations": [],
                "management_ip": "172.40.40.40",
                "manufacturers": [
                    "cisco"
                ],
                "platforms": [
                    "ios"
                ],
                "regions": [
                    "united-kingdom"
                ],
                "serial": "FOC2219V2ZJ",
                "services": [],
                "site_groups": [
                    "google"
                ],
                "sites": [
                    "google-paddock-wood"
                ],
                "status": {
                    "label": "Active",
                    "value": "active"
                },
                "tags": [
                    "backup"
                ],
                "wan_primary_ip": null,
                "wan_secondary_ip": null,
                "wireless_mac": null
            },
            "PW-SW-MAINOFFICE01-FOC2219V30A": {
                "asset_url": null,
                "device_roles": [
                    "switch"
                ],
                "device_types": [
                    "cisco-ws-c2960x-48fps-l"
                ],
                "firmware": null,
                "floor_id": null,
                "is_virtual": false,
                "lan_ip": null,
                "local_context_data": [
                    null
                ],
                "locations": [],
                "management_ip": "172.40.40.40",
                "manufacturers": [
                    "cisco"
                ],
                "platforms": [
                    "ios"
                ],
                "regions": [
                    "united-kingdom"
                ],
                "serial": "FOC2219V30A",
                "services": [],
                "site_groups": [
                    "google"
                ],
                "sites": [
                    "google-paddock-wood"
                ],
                "status": {
                    "label": "Active",
                    "value": "active"
                },
                "tags": [
                    "backup"
                ],
                "wan_primary_ip": null,
                "wan_secondary_ip": null,
                "wireless_mac": null
            }
        }
    },
    "all": {
        "children": [
            "ungrouped"
        ]
    },
    "ungrouped": {
        "hosts": [
            "PT-SW-MAINOFFICE02",
            "PW-SW-MAINOFFICE01-FOC2219V2ZE",
            "PW-SW-MAINOFFICE01-FOC2219V2ZJ",
            "PW-SW-MAINOFFICE01-FOC2219V30A"
        ]
    }
}

When running ansible-inventory -v --list -i netbox_inventory.yml, the output reveals multiple hostnames associated with the same IP address. This situation occurs when multiple physical devices share the same management IP and are connected to the same switch stack or virtual chassis, such as "PW-SW-MAINOFFICE01-STACK".

To address this redundancy and ensure efficient management of backups, I propose implementing conditions to filter and clean up the inventory file. Specifically, I suggest identifying instances where the hostname contains three hyphens, indicating a switch stack or virtual chassis configuration. By breaking the string at the third hyphen, we can derive a clean hostname that corresponds directly to the Azure Repository where the configuration is stored.

This approach allows all devices within the stack to access the same configuration file, reducing redundancy and streamlining the backup process. Moreover, it mitigates risks associated with device failure or decommissioning, as the configuration remains accessible even if individual devices are removed from the network.

{% if parts|length > 2 %}
  {% set object_name = parts[0:3]|join('-') %}
{% else %}
  {% set object_name = object.name %}
{% endif %}

https://dev.azure.com/google/google/_git/configs-backups?path=/{{ object_name }}

One of my current challenges involves refining the parsing and grouping of inventory data fetched from Netbox, particularly to eliminate duplicate IPs and break down hostnames for better organization.

The desired outcome is to create a group named netbox_inventory and ensure each device is represented once, with its hostname and corresponding IP address.

For example:

• PT-SW-MAINOFFICE02: ansible_host=172.30.30.30
• PW-SW-MAINOFFICE01: ansible_host=172.40.40.40

I have read through the official docs and followed the given examples and have not been able to achieve the end goal.
Also I don't see the need of outputting all of those fields in the json file, while I just need the hostname and IP address.

[candlerb]

One of my current challenges involves refining the parsing and grouping of inventory data fetched from Netbox, particularly to eliminate duplicate IPs and break down hostnames for better organization.

The standard way of tracking management IP addresses in Netbox is to use the "Primary IP" feature. To use this, you have to:

1. Create a device or VM
2. Create at least one interface (this may come from interface templates in the device type)
3. Create an IP address attached to that interface
4. Finally, select that IP address as the "primary" IP address of the device (in the GUI this can be done as part of step 3)

Optionally, you can create both primary IPv4 and primary IPv6. The one which is chosen as primary is controlled by the global setting PREFER_IPV4.

Then if you set ENFORCE_GLOBAL_UNIQUE=True, either in configuration.py or via "config versions" in the web UI, this prevents duplicate IPs.

[chicocth]

thanks for the tip, but that does not solve anything of what I'm asking.
The idea is to build the IPAM based on the configs really.