Cable Bulk Import: Error if Decvice names are not unique

[sw8899]

Hi all
I try to import hundreds of patch panel rear-port connections. I have different sites and the rack names and patch panel names are identical in different sites. e.g. PPC-a-A01-01 for a copper patch panel in Rack A01.
The general data model of netbox supports to have identical device names in different sites.
When I try to import I get the following errors:

Enter the list of column headers followed by one line per record to be imported, using commas to separate values. Multi-line data and values containing commas may be wrapped in double quotes.
• Row 1 side_a_device: "PPC-a-A01-01" is not a unique value for this field; multiple objects were found
• Row 1 all: Termination A type has not been specified

" side_a_device: "PPC-a-A01-01" is not a unique value for this field; multiple objects were found"
Yes thats true.
Shouldn't the cable bulk import have the possibility to define the site of the device?

My trouble shooting attempts:
A) On the bottom of the cable bulk import site there is a tip:
"Related objects may be referenced by any unique attribute. For example, vrf.rd would identify a VRF by its route distinguisher."
I try to understand this and googled a lot, but I don't get it.
-Where/how can they be referenced?
-Can I reference the site?
B) I also read posts that say, that the bulk import can be customized. But I can't find any documentation about it

Basic questions: Is it possible to bulk import cables if you have devices with identical names? How?

thx

[candlerb]

Try adding extra columns "side_a_device.site" and "side_b_device.site" and put the site name in there. You may need to experiment a bit, perhaps also change the existing "side_a_device" to "side_a_device.name".

There is minimal exposition of this feature at #3147 (comment) and you will need at least Netbox v2.8.2. If you get it working and can provide some working examples, that would be helpful and could be submitted to enhance the documentation.

Regarding what you've read in posts, can you link to those posts?

[sw8899]

1st: Thank you very much for your quick answer!

I tried some things, and will try further. But here are the first results.

Netbox Version: v2.11.7

It does not seem to like adding columns:

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
side_a_device,side_a_device.site,side_a_type,side_a_name,side_b_device,side_b_device.site,side_b_type,side_b_name,type,status,label
PPC-a-A01-01,CHDC1,dcim.rearport,-5,MDC-a-A01-01,CHDC1,dcim.rearport,-5,cat6a,connected,CT001-5

Enter the list of column headers followed by one line per record to be imported, using commas to separate values. Multi-line data and values containing commas may be wrapped in double quotes.
Row 1: Expected 9 columns but found 11
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
side_a_device.name,side_a_device.site,side_a_type,side_a_name,side_b_device.name,side_b_device.site,side_b_type,side_b_name,type,status,label
PPC-a-A01-01,CHDC1,dcim.rearport,-5,MDC-a-A01-01,CHDC1,dcim.rearport,-5,cat6a,connected,CT001-5

Enter the list of column headers followed by one line per record to be imported, using commas to separate values. Multi-line data and values containing commas may be wrapped in double quotes.
Row 1: Expected 9 columns but found 11
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Trying to squeeze it in 9 columns
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
side_a_device.site,side_a_type,side_a_name,side_b_device.site,side_b_type,side_b_name,type,status,label
PPC-a-A01-01.CHDC1,dcim.rearport,-5,MDC-a-A01-01.CHDC1,dcim.rearport,-5,cat6a,connected,CT001-5

Enter the list of column headers followed by one line per record to be imported, using commas to separate values. Multi-line data and values containing commas may be wrapped in double quotes.
Row 1 side_a_device: Object not found.
Row 1 side_b_device: Object not found.
Row 1 all: Termination A type has not been specified
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

#5245
jeremystretch commented on Oct 19, 2020
Duplicate of #4429. Please note that this is expected behavior, as the CSV import fields are customizable.

#4727

[candlerb]

#5245 and #4429 are just that CSV import fields are not the same as CSV export fields. This is sad but simply how it is, and is not related to your problem to fail to import.

#4727 is more like your problem.

The message "Expected 9 columns but found 11", when the CSV header clearly has 11, implies that you can't give two columns for the same attribute: "side_a_device" and "side_a_device.site" are treated as a single column. So sadly, this doesn't solve your problem. You could give just "side_a_device.id", but you'd need to look up the device IDs yourself.

I suggest you raise your example as a feature request. However if you want a solution quickly, you can code something up yourself:

• using the REST API and pynetbox; or
• using the Django ORM and nbshell; or
• using the Django ORM in a Custom Script using FileVar - probably the cleanest solution

[Bloodiko]

you may need to identify the cables not by name - but by something other unique
like the Primary Key of those Cables

as we just figured out:

replace "side_a_device" by "side_a_device.pk" (pk = Primary Key , or id - works both the same)
then you'll only need to figure out the ID of the devices
(usually found when you selected one device - then showing as "dcim.device:IDNumber"

but since this is a lot to read you may use the RestAPI and a custom wget request or powershells invoke-webRequest
$response = Invoke-WebRequest -Method Get -Headers $header -Uri https://netboxhost/api/dcim/devices/?limit=1000

then parse the $response.result through an json parser and you get good looking objects
you may also use select, to select only "id", "display" or other columns

Otherwise - i have no idea if you can tell the Netbox import to select an device by multiple selections, like device={dcim.site,dcim.rack,dcim.position}
This does not work! - and i have no idea if something like this is possible already and not in the Documentation - Or if its not added at all.

[sw8899]

Hi Bloodiko
You are right!
This works, and its so far the only way how it works:

side_a_device.pk,side_a_type,side_a_name,side_b_device.pk,side_b_type,side_b_name,type,status,label
710,dcim.rearport,-5,657,dcim.rearport,-5,cat6a,connected,CT001-5

The cable is imported.
Thank you very much for this input!

Since I'm not able to program I see only the dirty way.
I build up a XLS-file with a lot of vlookups... ,basically a Excel-DCIM to feed Netbox-DCIM.
And I try to open a feature request.

[sw8899]

For reference the original data and error:

side_a_device,side_a_type,side_a_name,side_b_device,side_b_type,side_b_name,type,status,label
PPC-a-A01-01,dcim.rearport,-5,MDC-a-A01-01,dcim.rearport,-5,cat6a,connected,CT001-5

Enter the list of column headers followed by one line per record to be imported, using commas to separate values. Multi-line data and values containing commas may be wrapped in double quotes.
Row 1 side_a_device: "PPC-a-A01-01" is not a unique value for this field; multiple objects were found
Row 1 all: Termination A type has not been specified

[sw8899]

We decided to rename all our Patch Panels.
old: PPC-a-A01-01
new: PPC-CHDC1-a-A01-01
Then the name is unique and we get around the import problem.