Merge pull request #373 from nautobot/release-4.2.5

Release 4.2.5

Author: scetron

.github/workflows/ci.yml

@@ -92,7 +92,7 @@ jobs:
       fail-fast: true
       matrix:
         python-version: ["3.11"]
-        nautobot-version: ["2.2.3"]
+        nautobot-version: ["2.3.1"]
     env:
       INVOKE_NAUTOBOT_DEVICE_ONBOARDING_PYTHON_VER: "${{ matrix.python-version }}"
       INVOKE_NAUTOBOT_DEVICE_ONBOARDING_NAUTOBOT_VER: "${{ matrix.nautobot-version }}"
@@ -145,7 +145,7 @@ jobs:
         include:
           - python-version: "3.11"
             db-backend: "postgresql"
-            nautobot-version: "2.2.3"
+            nautobot-version: "2.3.1"
           - python-version: "3.12"
             db-backend: "mysql"
             nautobot-version: "stable"

README.md

@@ -19,15 +19,17 @@ Regardless, the Onboarding App greatly simplifies the onboarding process by allo
 
 ### Support Matrix (Sync Devices From Network)
 
+
 |     Data Attribute      | Cisco IOS          | Cisco XE           | Cisco NXOS         | Cisco WLC          | Juniper Junos      | Arista EOS         | F5  | HP Comware | Palo Alto Panos | Aruba AOSCX |
 | ----------------------- | :----------------: |  :--------------:  |  :--------------:  |  :--------------:  |  :--------------:  |  :--------------:  | :-: | :-: | :-: | :-: |
-| Hostname                | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | 🧪 | 🧪 | 🧪 |
-| Platform                | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | 🧪 | 🧪 | 🧪 |
-| Manufacturer            | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | 🧪 | 🧪 | 🧪 |
-| Serial Number           | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | 🧪 | 🧪 | 🧪 |
-| Device Type             | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | 🧪 | 🧪 | 🧪 |
-| Mgmt Interface          | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | 🧪 | 🧪 | 🧪 |
-| Mgmt IP Address         | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | 🧪 | 🧪 | 🧪 |
+| Hostname                | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | 🧪 | 🧪 | 🧪 | 🧪 |
+| Platform                | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | 🧪 | 🧪 | 🧪 | 🧪 |
+| Manufacturer            | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | 🧪 | 🧪 | 🧪 | 🧪 |
+| Serial Number           | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | 🧪 | 🧪 | 🧪 | 🧪 |
+| Device Type             | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | 🧪 | 🧪 | 🧪 | 🧪 |
+| Mgmt Interface          | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | 🧪 | 🧪 | 🧪 | 🧪 |
+| Mgmt IP Address         | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | 🧪 | 🧪 | 🧪 | 🧪 |
+
 
 ### Support Matrix (Sync Data From Network)
 
@@ -43,6 +45,7 @@ Regardless, the Onboarding App greatly simplifies the onboarding process by allo
 | 802.1Q mode    | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ |
 | Lag Member     | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ |
 | Vrf Membership | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ |
+| Software Version | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ |
 
 |     VLANS          | Cisco IOS          | Cisco XE           | Cisco NXOS         | Cisco WLC          | Juniper Junos      | Arista EOS         | F5  |
 | ----------------------- | :----------------: |  :--------------:  |  :--------------:  |  :--------------:  |  :--------------:  |  :--------------:  | :-: |

docs/admin/install.md

@@ -4,7 +4,7 @@ Here you will find detailed instructions on how to **install** and **configure**
 
 ## Prerequisites
 
-- The app is compatible with Nautobot 2.0.3 and higher.
+- The app is compatible with Nautobot 2.3.1 and higher.
 - Databases supported: PostgreSQL, MySQL
 
 !!! note

docs/admin/release_notes/version_4.2.md

@@ -13,6 +13,31 @@ Versioning](https://semver.org/spec/v2.0.0.html).
 - [#340](https://github.com/nautobot/nautobot-app-device-onboarding/issues/340) - Add Aruba AOSCX support for Sync Devices from Network Job.
 - [#278](https://github.com/nautobot/nautobot-app-device-onboarding/issues/278) - Optimized VLAN loading into diffsync from Nautbot
 - [#278](https://github.com/nautobot/nautobot-app-device-onboarding/issues/278) - Improved error handling when creating VLANs
+- [#233](https://github.com/nautobot/nautobot-app-device-onboarding/issues/233) - Added support syncing in software versions from devices to nautobot core models.
+- [#334](https://github.com/nautobot/nautobot-app-device-onboarding/issues/334) - Add initial F5 Support for Network Device Sync
+- [#357](https://github.com/nautobot/nautobot-app-device-onboarding/issues/357) - Added improved error messaging for device object `get()` call in the `update()` method of `SyncDevicesDevice`
+- [#372](https://github.com/nautobot/nautobot-app-device-onboarding/issues/372) - Fixed bug when loading Nautobot Vlans with multiple locations assigned. Only Vlans with 1 location will be considered for the sync.
+
+## [v4.2.5 (2025-05-13)](https://github.com/nautobot/nautobot-app-device-onboarding/releases/tag/v4.2.5)
+
+### Added
+
+- [#233](https://github.com/nautobot/nautobot-app-device-onboarding/issues/233) - Added support syncing in software versions from devices to nautobot core models.
+- [#334](https://github.com/nautobot/nautobot-app-device-onboarding/issues/334) - Add initial F5 Support for Network Device Sync
+- [#357](https://github.com/nautobot/nautobot-app-device-onboarding/issues/357) - Added improved error messaging for device object `get()` call in the `update()` method of `SyncDevicesDevice`
+
+### Fixed
+
+- [#372](https://github.com/nautobot/nautobot-app-device-onboarding/issues/372) - Fixed bug when loading Nautobot Vlans with multiple locations assigned. Only Vlans with 1 location will be considered for the sync.
+
+### Dependencies
+
+- [#367](https://github.com/nautobot/nautobot-app-device-onboarding/issues/367) - Updated jdiff dependency pin.
+
+### Documentation
+
+- [#358](https://github.com/nautobot/nautobot-app-device-onboarding/issues/358) - Updated documentation on the README for VRF.
+
 
 ## [v4.2.4 (2025-04-08)](https://github.com/nautobot/nautobot-app-device-onboarding/releases/tag/v4.2.4)
 

docs/user/app_overview.md

@@ -47,6 +47,8 @@ Expose two new SSoT based Nautobot jobs to perform the syncing of data.
     - VRFs
         - VRF Names
         - Route Distinguishers (RD)
+    - Cabling
+    - Software Version
     - Cabling (**Note** Cables attached to Circuits will be skipped)
 
 !!! info

mkdocs.yml

@@ -79,6 +79,7 @@ markdown_extensions:
       permalink: true
   - "attr_list"
   - "md_in_html"
+  - "pymdownx.details"
   - "pymdownx.highlight":
       anchor_linenums: true
   - "pymdownx.inlinehilite"

nautobot_device_onboarding/command_mappers/arista_eos.yml

@@ -116,3 +116,8 @@ sync_network_data:
         jpath: '{admin_mode: switchports."{{ current_key }}".switchportInfo.mode, mode: switchports."{{ current_key }}".switchportInfo.mode, access_vlan: switchports."{{ current_key }}".switchportInfo.accessVlanId, trunking_vlans: switchports."{{ current_key }}".switchportInfo.trunkAllowedVlans, native_vlan: switchports."{{ current_key }}".switchportInfo.trunkingNativeVlanId}'  # yamllint disable-line rule:quoted-strings
         post_processor: "{{ obj | get_vlan_data(vlan_map, 'untagged') | tojson }}"
         iterable_type: "dict"
+  software_version:
+    commands:
+      - command: "show version"
+        parser: "textfsm"
+        jpath: "[*].image"

nautobot_device_onboarding/command_mappers/cisco_ios.yml

@@ -129,3 +129,8 @@ sync_network_data:
         parser: "textfsm"
         jpath: "[*].{local_interface:local_interface, remote_interface:neighbor_interface, remote_device:neighbor_name}"
         post_processor: "{% set result = [] %}{% for cable in obj %}{% set _=result.append({'local_interface': cable['local_interface'], 'remote_interface': cable['remote_interface'], 'remote_device': cable['remote_device'] | remove_fqdn }) %}{% endfor %}{{ result | tojson }}"
+  software_version:
+    commands:
+      - command: "show version"
+        parser: "textfsm"
+        jpath: "[*].version"

nautobot_device_onboarding/command_mappers/cisco_nxos.yml

@@ -122,3 +122,8 @@ sync_network_data:
         parser: "textfsm"
         jpath: "[*].{local_interface:local_interface, remote_interface:neighbor_interface, remote_device:neighbor_name}"
         post_processor: "{% set result = [] %}{% for cable in obj %}{% set _=result.append({'local_interface': cable['local_interface'], 'remote_interface': cable['remote_interface'], 'remote_device': cable['remote_device'] | remove_fqdn }) %}{% endfor %}{{ result | tojson }}"
+  software_version:
+    commands:
+      - command: "show version"
+        parser: "textfsm"
+        jpath: "[*].os"

nautobot_device_onboarding/command_mappers/cisco_xe.yml

@@ -126,3 +126,8 @@ sync_network_data:
         parser: "textfsm"
         jpath: "[*].{local_interface:local_interface, remote_interface:neighbor_interface, remote_device:neighbor_name}"
         post_processor: "{% set result = [] %}{% for cable in obj %}{% set _=result.append({'local_interface': cable['local_interface'], 'remote_interface': cable['remote_interface'], 'remote_device': cable['remote_device'] | remove_fqdn }) %}{% endfor %}{{ result | tojson }}"
+  software_version:
+    commands:
+      - command: "show version"
+        parser: "textfsm"
+        jpath: "[*].version"

nautobot_device_onboarding/command_mappers/f5_tmsh.yml

@@ -0,0 +1,36 @@
+---
+sync_devices:
+  hostname:
+    commands:
+      - command: "show cm device"
+        parser: "ttp"
+        jpath: "[]|[?mgmt_ip==`{{ obj }}`].mgmt_hostname"
+        post_processor: "{{ obj[0] }}"
+        interable_type: "str"
+  serial:
+    commands:
+      - command: "show sys hardware"
+        parser: "ttp"
+        jpath: "[*].serial_number"
+        post_processor: "{{ obj[0] }}"
+        interable_type: "str"
+  device_type:
+    commands:
+      - command: "show sys hardware"
+        parser: "ttp"
+        jpath: "[*].model_type"
+        post_processor: "{{ obj[0] }}"
+        interable_type: "str"
+  mgmt_interface:
+    commands:
+      - command: "show net interface | grep mgmt | grep up"
+        parser: "raw"
+        jpath: "raw"
+        post_processor: "{{ obj.split(' ')[0] }}"
+  mask_length:
+    commands:
+      - command: "show sys cluster"
+        parser: "ttp"
+        jpath: "[*].mgmt_mask"
+        post_processor: "{{ obj[0] }}"
+        iterable_type: "int"

nautobot_device_onboarding/command_mappers/juniper_junos.yml

@@ -121,3 +121,8 @@ sync_network_data:
         jpath: '"lldp-neighbors-information"[]."lldp-neighbor-information"[].{local_interface: "lldp-local-port-id"[0].data, remote_interface: "lldp-remote-port-id"[0].data, remote_device: "lldp-remote-system-name"[0].data}'  # yamllint disable-line rule:quoted-strings
         post_processor: "{% set result = [] %}{% for cable in obj %}{% set _=result.append({'local_interface': cable['local_interface'], 'remote_interface': cable['remote_interface'], 'remote_device': cable['remote_device'] | remove_fqdn }) %}{% endfor %}{{ result | tojson }}"
         iterable_type: "dict"
+  software_version:
+    commands:
+      - command: "show system information | display json"
+        parser: "none"
+        jpath: '"system-information"[]."os-version"[].data'  # yamllint disable-line rule:quoted-strings

nautobot_device_onboarding/diffsync/adapters/sync_network_data_adapters.py

@@ -7,7 +7,7 @@
 from django.conf import settings
 from django.contrib.contenttypes.models import ContentType
 from django.core.exceptions import ValidationError
-from nautobot.dcim.models import Interface
+from nautobot.dcim.models import Device, Interface, SoftwareVersion
 from nautobot.ipam.models import VLAN, VRF, IPAddress
 from nautobot_ssot.contrib import NautobotAdapter
 from netaddr import EUI, mac_unix_expanded
@@ -52,6 +52,8 @@ class SyncNetworkDataNautobotAdapter(FilteredNautobotAdapter):
     lag_to_interface = sync_network_data_models.SyncNetworkDataLagToInterface
     vrf_to_interface = sync_network_data_models.SyncNetworkDataVrfToInterface
     cable = sync_network_data_models.SyncNetworkDataCable
+    software_version = sync_network_data_models.SyncNetworkSoftwareVersion
+    software_version_to_device = sync_network_data_models.SyncNetworkSoftwareVersionToDevice
 
     primary_ips = None
 
@@ -66,6 +68,8 @@ class SyncNetworkDataNautobotAdapter(FilteredNautobotAdapter):
         "lag_to_interface",
         "vrf_to_interface",
         "cable",
+        "software_version",
+        "software_version_to_device",
     ]
 
     def _cache_primary_ips(self, device_queryset):
@@ -134,7 +138,15 @@ def load_vlans(self):
         """
         # TODO: update this to support multiple locations per VLAN after the setting for this feature has been added.
         location_ids = list(self.job.devices_to_load.values_list("location__id", flat=True))
-        for vlan in VLAN.objects.filter(location__in=location_ids):
+        for vlan in VLAN.objects.filter(locations__in=location_ids):
+            if (
+                vlan.locations.count() > 1
+            ):  # TODO: A conditional check will be needed here to support multiple locations per VLAN
+                if self.job.debug:
+                    self.job.logger.debug(
+                        f"Vlan {vlan.name} has multiple locations. Skipping Vlan load for {vlan.name}."
+                    )
+                continue
             network_vlan = self.vlan(
                 adapter=self,
                 name=vlan.name,
@@ -305,6 +317,32 @@ def load_cables(self):
                 except diffsync.exceptions.ObjectAlreadyExists:
                     continue
 
+    def load_software_versions(self):
+        """Load Software Versions into the Diffsync store."""
+        for software_version in SoftwareVersion.objects.all():
+            network_software_version = self.software_version(
+                adapter=self,
+                version=software_version.version,
+                platform__name=software_version.platform.name,
+            )
+            try:
+                network_software_version.model_flags = DiffSyncModelFlags.SKIP_UNMATCHED_DST
+                self.add(network_software_version)
+            except diffsync.exceptions.ObjectAlreadyExists:
+                continue
+
+    def load_software_version_to_device(self):
+        """Load Software Version to Device assignments into the Diffsync store."""
+        for device in self.job.devices_to_load:
+            network_software_version_to_device = self.software_version_to_device(
+                adapter=self,
+                name=device.name,
+                serial=device.serial,
+                software_version__version=device.software_version.version if device.software_version else "",
+            )
+            network_software_version_to_device.model_flags = DiffSyncModelFlags.SKIP_UNMATCHED_DST
+            self.add(network_software_version_to_device)
+
     def load(self):
         """Generic implementation of the load function."""
         if not hasattr(self, "top_level") or not self.top_level:
@@ -334,6 +372,12 @@ def load(self):
             elif model_name == "cable":
                 if self.job.sync_cables:
                     self.load_cables()
+            elif model_name == "software_version":
+                if self.job.sync_software_version:
+                    self.load_software_versions()
+            elif model_name == "software_version_to_device":
+                if self.job.sync_software_version:
+                    self.load_software_version_to_device()
             else:
                 diffsync_model = self._get_diffsync_class(model_name)
                 self._load_objects(diffsync_model)
@@ -409,6 +453,8 @@ def __init__(self, *args, job, sync=None, **kwargs):
     lag_to_interface = sync_network_data_models.SyncNetworkDataLagToInterface
     vrf_to_interface = sync_network_data_models.SyncNetworkDataVrfToInterface
     cable = sync_network_data_models.SyncNetworkDataCable
+    software_version = sync_network_data_models.SyncNetworkSoftwareVersion
+    software_version_to_device = sync_network_data_models.SyncNetworkSoftwareVersionToDevice
 
     top_level = [
         "ip_address",
@@ -421,6 +467,8 @@ def __init__(self, *args, job, sync=None, **kwargs):
         "lag_to_interface",
         "vrf_to_interface",
         "cable",
+        "software_version",
+        "software_version_to_device",
     ]
 
     def _handle_failed_devices(self, device_data):
@@ -890,6 +938,46 @@ def load_cables(self):  # pylint: disable=inconsistent-return-statements
                     model_type="cable",
                 )
 
+    def load_software_versions(self):
+        """Load software versions into the Diffsync store."""
+        for (  # pylint: disable=too-many-nested-blocks
+            hostname,
+            device_data,
+        ) in self.job.command_getter_result.items():
+            if self.job.debug:
+                self.job.logger.debug(f"Loading Software Versions from {hostname}")
+            if device_data["software_version"]:
+                device = Device.objects.get(serial=device_data["serial"])
+                try:
+                    network_software_version = self.software_version(
+                        adapter=self,
+                        platform__name=device.platform.name,
+                        version=device_data["software_version"],
+                    )
+                    self.add(network_software_version)
+                except diffsync.exceptions.ObjectAlreadyExists:
+                    continue
+
+    def load_software_version_to_device(self):
+        """Load software version to device assignments into the Diffsync store."""
+        for (  # pylint: disable=too-many-nested-blocks
+            hostname,
+            device_data,
+        ) in self.job.command_getter_result.items():
+            if self.job.debug:
+                self.job.logger.debug(f"Loading Software Version to Device assignments from {hostname}")
+            if device_data["software_version"]:
+                try:
+                    network_software_version_to_device = self.software_version_to_device(
+                        adapter=self,
+                        name=hostname,
+                        serial=device_data["serial"],
+                        software_version__version=device_data["software_version"],
+                    )
+                    self.add(network_software_version_to_device)
+                except diffsync.exceptions.ObjectAlreadyExists:
+                    continue
+
     def load(self):
         """Load network data."""
         self.execute_command_getter()
@@ -908,3 +996,7 @@ def load(self):
             self.load_vrf_to_interface()
         if self.job.sync_cables:
             self.load_cables()
+        if self.job.sync_software_version:
+            self.load_software_versions()
+        if self.job.sync_software_version:
+            self.load_software_version_to_device()

nautobot_device_onboarding/diffsync/models/sync_devices_models.py

@@ -249,7 +249,14 @@ def create(cls, adapter, ids, attrs):
 
     def update(self, attrs):
         """Update an existing nautobot device using data scraped from a device."""
-        device = Device.objects.get(name=self.name, location__name=self.location__name)
+        try:
+            device = Device.objects.get(name=self.name, location__name=self.location__name)
+        except MultipleObjectsReturned as exc:
+            raise MultipleObjectsReturned(
+                f"Multiple devices found with name {self.name} and location {self.location__name}"
+            ) from exc
+        except ObjectDoesNotExist as exc:
+            raise ObjectDoesNotExist(f"Device {self.name} does not exist at {self.location__name}") from exc
 
         if self.adapter.job.debug:
             self.adapter.job.logger.debug(f"Updating {device.name} with attrs: {attrs}")