Cannot Install v2.11.9

[ejm110]

We have a very old v2.11.9 that we want to upgrade to v4.0.x.
The plan is to load psql dump to a test VM with v2.11.9 install and do the upgrades to v3.7.8 to v4.0.x then export the dump.
However, in the test VM, installing v2.11.9 during upgrade.sh leads to an error. Same error when installing v2.11.12

Ubuntu Server 24.04
Python 3.12.3

sudo git clone -b master https://github.com/netbox-community/netbox.git .
sudo git checkout v2.11.9
# configuration.py was configured but I omit here
sudo /opt/netbox/upgrade.sh

The error:

Collecting PyYAML==5.4.1 (from -r requirements.txt (line 22))
  Using cached PyYAML-5.4.1.tar.gz (175 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [54 lines of output]
      running egg_info
      writing lib3/PyYAML.egg-info/PKG-INFO
      writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt
      writing top-level names to lib3/PyYAML.egg-info/top_level.txt
      Traceback (most recent call last):
        File "/opt/netbox/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/opt/netbox/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/netbox/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-onwgcv9m/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-onwgcv9m/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-onwgcv9m/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 271, in <module>
        File "/tmp/pip-build-env-onwgcv9m/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-onwgcv9m/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 184, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-onwgcv9m/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-onwgcv9m/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-onwgcv9m/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 976, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-onwgcv9m/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-onwgcv9m/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 321, in run
          self.find_sources()
        File "/tmp/pip-build-env-onwgcv9m/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 329, in find_sources
          mm.run()
        File "/tmp/pip-build-env-onwgcv9m/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 550, in run
          self.add_defaults()
        File "/tmp/pip-build-env-onwgcv9m/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 588, in add_defaults
          sdist.add_defaults(self)
        File "/tmp/pip-build-env-onwgcv9m/overlay/lib/python3.12/site-packages/setuptools/command/sdist.py", line 102, in add_defaults
          super().add_defaults()
        File "/tmp/pip-build-env-onwgcv9m/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 250, in add_defaults
          self._add_defaults_ext()
        File "/tmp/pip-build-env-onwgcv9m/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 335, in _add_defaults_ext
          self.filelist.extend(build_ext.get_source_files())
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<string>", line 201, in get_source_files
        File "/tmp/pip-build-env-onwgcv9m/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
          raise AttributeError(attr)
      AttributeError: cython_sources
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[candlerb]

Since you are already on v2.11.x you can skip what you are trying to do entirely.

First, create a fresh VM with Netbox v3.7.8. Import your SQL dump from the v2.11.9 into this, run ./upgrade.sh, and it should be working with the imported data. You can then upgrade this VM to v4.0.x

[ejm110]

As I understand the guide, a v2 database cannot be imported to v3 netbox. Anyhow, I managed to update the DBs up to v4 by using old Ubuntu 20.

[ejm110]

I tried it, it did not work.

Server Error

A database programming error was detected while processing this request. Common causes include the following:

Database migrations missing. When upgrading to a new NetBox release, the upgrade script must be run to apply any new database migrations. You can run migrations manually by executing python3 manage.py migrate from the command line.

Unsupported PostgreSQL version. Ensure that PostgreSQL version 12 or later is in use. You can check this by connecting to the database using NetBox's credentials and issuing a query for SELECT VERSION().

The complete exception is provided below:

<class 'django.db.utils.ProgrammingError'>

relation "extras_bookmark" does not exist
LINE 1: ...rk"."object_id", "extras_bookmark"."user_id" FROM "extras_bo...
                                                             ^

Python version: 3.12.3
NetBox version: 3.7.8
Plugins: None installed

[candlerb]

As I understand the guide, a v2 database cannot be imported to v3 netbox

Not quite. You cannot upgrade from <v2.11.0 to v3, or from <v3.7.0 to v4.

But you can upgrade from v2.11.x to any v3.x, and from v3.7.x to any v4.x. This is described here.

[ejm110]

For those who are stuck in v2.11.x, here how I managed to migrate/upgrade to v4.0.6

Installed Ubuntu 20.04 in test VM1
Installed Netbox v2.11.12 on it
Imported our DB from v2.11.9
Update Netbox to v3.6.5, use sudo NETBOX_DELETE_LEGACY_DATA=1 upgrade.sh as mentioned here
Exported the DB

Installed Ubuntu 24.04 in test VM2
Installed Netbox v3.7.8 on it
Imported our DB from v3.6.5 from test VM1
Update Netbox to v4.0.6

[candlerb]

I'm glad it's working. For reference, the following simpler approach should have worked:

Install Ubuntu 20.04 in test VM1
Install Netbox v3.7.8 in it
Import DB from v2.11.9
Use sudo NETBOX_DELETE_LEGACY_DATA=1 upgrade.sh as mentioned here
Export the DB

Install Ubuntu 24.04 in test VM2
Install Netbox v4.0.6 on it
Imported DB from v3.7.8 from test VM1
Re-run ./upgrade.sh

There's no need to install Netbox v2.11 before importing the v2.11 database into v3.x; ./upgrade.sh in v3.x knows how to upgrade the database schema from v2.11.x or later. Similarly, ./upgrade.sh in v4.x knows how to upgrade the database schema from v3.7.x or later.

[ejm110]

I tried it but got the error below during sudo NETBOX_DELETE_LEGACY_DATA=1 upgrade.sh
Tho running it on Ubuntu 24.04 with v3.7.8.

Applying database migrations (python3 netbox/manage.py migrate)...
Operations to perform:
  Apply all migrations: account, admin, auth, circuits, contenttypes, core, dcim, django_rq, extras, ipam, sessions, social_django, taggit, tenancy, users, virtualization, vpn, wireless
Running migrations:
  Applying tenancy.0003_contacts...Traceback (most recent call last):
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 87, in _execute
    return self.cursor.execute(sql)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/psycopg/cursor.py", line 732, in execute
    raise ex.with_traceback(None)
psycopg.errors.InsufficientPrivilege: permission denied for schema public
LINE 1: CREATE TABLE "tenancy_contactrole" ("created" date NULL, "la...
                     ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/netbox/netbox/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/base.py", line 106, in wrapper
    res = handle_func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/core/management/commands/migrate.py", line 356, in handle
    post_migrate_state = executor.migrate(
                         ^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/migrations/executor.py", line 135, in migrate
    state = self._migrate_all_forwards(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
    state = self.apply_migration(
            ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
    state = migration.apply(state, schema_editor)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/migrations/migration.py", line 132, in apply
    operation.database_forwards(
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/migrations/operations/models.py", line 96, in database_forwards
    schema_editor.create_model(model)
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/backends/base/schema.py", line 451, in create_model
    self.execute(sql, params or None)
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/backends/postgresql/schema.py", line 45, in execute
    return super().execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/backends/base/schema.py", line 201, in execute
    cursor.execute(sql, params)
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/netbox/venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 87, in _execute
    return self.cursor.execute(sql)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.12/site-packages/psycopg/cursor.py", line 732, in execute
    raise ex.with_traceback(None)
django.db.utils.ProgrammingError: permission denied for schema public
LINE 1: CREATE TABLE "tenancy_contactrole" ("created" date NULL, "la...
                     ^

[candlerb]

psycopg.errors.InsufficientPrivilege: permission denied for schema public

Then you have a permissions problem. Maybe some tables are owned by the wrong user.

[ejm110]

I tired it again today and somehow the upgrade to V4.x went smoothly without doing any adjustment and also not encountering the permission error. ¯_(ツ)_/¯