Import to new 3.0.5

[skygremlin]

I have an old version of Netbox, 2.6.7, and I've never had issues so never updated. I'm now trying to update to the new 3.0.5 and running into issues... For now, I gave up running the update script on my old instance. I was either doing something wrong ( most likely ) or my environment was messed up ( Ubuntu Release: 18.04 with update python ). I couldn't get past an error saying I need to update "apt-get install python3-venv"... Which I did, I thought...

I don't have a ton of data so now I'm trying to just export -> Import to my old to new 3.0.5 instance running in a new VM Ubuntu 20.04 and python 3.8.10. I tried importing my sites using the .cvs export file from my old version, but keep getting the below error.

Row 1 status: Select a valid choice. Active is not one of the available choices

So I tried to manually enter the lines for import and get the same error. Then changed the status to Planned and got the same error.

I am able to manually enter a site with the status as Active though....

Does anyone have any thoughts on what I could be doing wrong, or did wrong during the install?

Thank you

[jeremystretch]

The table of available columns below the import form lists all acceptable values. (Click the question mark icon.) You probably need to change these values to lowercase to match what NetBox expects.

[skygremlin]

ahhh yeah that was it.... I was reading the columns backwords in the popup and had a Capital A in Active.... geez, just noticed..

Thank you!

[candlerb]

A couple of comments.

Firstly, you are not going to get good results migrating data using CSV export and import. The CSV export and import forms in Netbox aren't even compatible with each other (there are different column names and contents in some cases), and there are fundmantal problems that are hard to workaround - e.g. you can't set the "primary_ip" on a device or VM until after you've imported all its interfaces and IP addresses. As Jeremy points out, the slug values in the CSV (e.g. "active") aren't necessarily the same as the displayed values in the GUI (e.g. "Active")

Secondly, if you want to migrate your complete dataset, then I'd say you're much better off doing a SQL migration: see replicating Netbox in the documentation.

However, you cannot export from 2.6.7 and import directly into 3.0.x. You must first upgrade the database to 2.11.x, and once that's successful, move to 3.0.x. So the process will be like this:

1. Install 2.11.12 on a new machine or VM (*). Make sure it's working. You can create test objects if you want; they'll get thrown away in the next step anyway.
2. Drop the database and import the one exported from your 2.6.7 machine, as shown here.
3. Run ./upgrade.sh. This is important: it applies the "migrations" to update the database schema to be compatible with 2.11.12.
4. Now upgrade from 2.11.12 to 3.0.6. Note that part of the upgrade process is to run ./upgrade.sh again.

Step 4 is most easily done as an in-place upgrade, which is straightforward if you installed from git: it's just sudo git pull && sudo ./upgrade.sh. However if you're not comfortable with that, you can always export the database, install 3.0.6 from scratch in a new VM, and then do steps 2 and 3 in that VM.

---

(*) If you're installing from git (which I recommend): after doing the initial git clone, do git checkout v2.11.12 to switch to a temporary checkout of the older version. Once you've finished step 3, do git checkout master to get back to the latest release.

[cpmills1975]

My production NetBox is 2.10.something and running in Docker so I've built a new dev box, installed NetBox 2.11.12 in /opt/netbox/netbox-v2.11.12 and NetBox 3.0.5 in /opt/netbox/netbox-master. I've installed Postgres just once. I've scripted a pg_dump on the docker host, running over an SSH connection, to result in a netbox.sql file on the dev box. My script then imports that in to postgres, activates the 2.11.12 venv, runs the migration, activates the 3.0.5 (master) venv and runs the migration and all the other bits from the upgrade script effectively a second time. This leaves me with a 3.0.5 version of the database which, so far, I've been able to read quite successfully using the dev server.

[skygremlin]

Thank you. Yeah I was concerned with the linking tables and missing data required for some of the imports. I thought I'd start with Sites and see if I could get that to work.

Thanx for the suggestions. I'll see if I can get this going, but won't be till later tonight...

I did try to update to 2.11.11, not .12. Mainly to stay on Released versions. I couldn't get it working so just restored my snapshot.. But I can give it another go.

[candlerb]

You can go via any 2.11.x version. I only suggested 2.11.12 because that is the latest release in the 2.11 series.

If you have a problem moving to 2.11.x, then someone may be able to help if you show the actual errors that you see. At worst you can always make a fresh 2.11 installation in a new VM, then import the database into that, then upgrade the database.

One thing which often trips people up is that they have bad entries in their netbox.service - in particular the ExecStart lines should point to /opt/netbox/venv/bin/gunicorn, not to the system gunicorn.

By the time you get to Netbox 3.x then you need to be running python3.7+. If the new VM is Ubuntu 20.04 you'll be fine. If you're sticking with Ubuntu 18.04, where the default python is 3.6, you'll need to apt-get install python3.8 python3.8-venv. Furthermore, when you do the upgrade (and every time you subsequently upgrade Netbox) you'll have to remember to specify the python version to use:

sudo PYTHON=python3.8 ./upgrade.sh 

If you don't, upgrade.sh will get part-way through then crash.

[skygremlin]

Thank you for the advice.. I do have the ability to make multiple VMs and play with this which I plan to..

I was able to get the new version (v3.0.5) installed and populated by exporting and importing the tables.... and from what I can see everything is moved... I plan on running the updates per your suggestions too. To see how that goes and compare again...

It's been a good year, and I've thoroughly enjoyed NetBox, as I'm getting more free time I'm looking forward to learning more.

Thank You to all involved.