Upgrade v2.11.12 to v3.0.2

[dman4cloud]

Using Option B as I always to update netbox.

The upgrade completes but does have the following warnings in relation to mkdocs build:

Applying database migrations (python3 netbox/manage.py migrate)...
Operations to perform:
Apply all migrations: admin, auth, circuits, contenttypes, dcim, extras, ipam, sessions, taggit, tenancy, users, virtualization
Running migrations:
Applying dcim.0132_cable_length... OK
Applying dcim.0133_port_colors... OK
Applying extras.0060_customlink_button_class... OK
Applying extras.0061_extras_change_logging... OK
Applying extras.0062_clear_secrets_changelog... OK
Applying ipam.0049_prefix_mark_utilized... OK
Applying ipam.0050_iprange... OK
Checking for missing cable paths (python3 netbox/manage.py trace_paths --no-input)...
Found no missing console port paths; skipping
Found no missing console server port paths; skipping
Found no missing interface paths; skipping
Found no missing power feed paths; skipping
Found no missing power outlet paths; skipping
Found no missing power port paths; skipping
Finished.
Building documentation (mkdocs build)...
WARNING - Config value: 'python'. Warning: Unrecognised configuration name: python
INFO - Cleaning site directory
INFO - Building documentation to directory: /opt/netbox/netbox/project-static/docs
INFO - The following pages exist in the docs directory, but are not included in the "nav" configuration:
- models/circuits/circuit.md
- models/circuits/circuittermination.md
- models/circuits/circuittype.md
- models/circuits/provider.md
- models/circuits/providernetwork.md
- models/dcim/cable.md
- models/dcim/consoleport.md
- models/dcim/consoleporttemplate.md
- models/dcim/consoleserverport.md
- models/dcim/consoleserverporttemplate.md
- models/dcim/device.md
- models/dcim/devicebay.md
- models/dcim/devicebaytemplate.md
- models/dcim/devicerole.md
- models/dcim/devicetype.md
- models/dcim/frontport.md
- models/dcim/frontporttemplate.md
- models/dcim/interface.md
- models/dcim/interfacetemplate.md
- models/dcim/inventoryitem.md
- models/dcim/location.md
- models/dcim/manufacturer.md
- models/dcim/platform.md
- models/dcim/powerfeed.md
- models/dcim/poweroutlet.md
- models/dcim/poweroutlettemplate.md
- models/dcim/powerpanel.md
- models/dcim/powerport.md
- models/dcim/powerporttemplate.md
- models/dcim/rack.md
- models/dcim/rackreservation.md
- models/dcim/rackrole.md
- models/dcim/rearport.md
- models/dcim/rearporttemplate.md
- models/dcim/region.md
- models/dcim/site.md
- models/dcim/sitegroup.md
- models/dcim/virtualchassis.md
- models/extras/imageattachment.md
- models/ipam/aggregate.md
- models/ipam/ipaddress.md
- models/ipam/iprange.md
- models/ipam/prefix.md
- models/ipam/rir.md
- models/ipam/role.md
- models/ipam/routetarget.md
- models/ipam/service.md
- models/ipam/vlan.md
- models/ipam/vlangroup.md
- models/ipam/vrf.md
- models/tenancy/tenant.md
- models/tenancy/tenantgroup.md
- models/users/objectpermission.md
- models/users/token.md
- models/virtualization/cluster.md
- models/virtualization/clustergroup.md
- models/virtualization/clustertype.md
- models/virtualization/virtualmachine.md
- models/virtualization/vminterface.md
- release-notes/index.md
WARNING - Documentation file 'core-functionality/devices.md' contains a link to '../media/models/dcim_cable_trace.png' which is not found in the documentation files.
WARNING - Documentation file 'core-functionality/power.md' contains a link to '../media/power_distribution.png' which is not found in the documentation files.
WARNING - Documentation file 'customization/custom-scripts.md' contains a link to '../media/admin_ui_run_permission.png' which is not found in the documentation files.
WARNING - Documentation file 'customization/reports.md' contains a link to '../media/admin_ui_run_permission.png' which is not found in the documentation files.
INFO - Documentation built in 19.49 seconds
Collecting static files (python3 netbox/manage.py collectstatic --no-input)...

425 static files copied to '/opt/netbox/netbox/static'.
Removing stale content types (python3 netbox/manage.py remove_stale_contenttypes --no-input)...
Removing expired user sessions (python3 netbox/manage.py clearsessions)...
Upgrade complete! Don't forget to restart the NetBox services:

sudo systemctl restart netbox netbox-rq

Service does not start after a restart:

Sep 13 14:11:16 cmdb systemd: Started NetBox WSGI Service.
Sep 13 14:11:17 cmdb gunicorn: [2021-09-13 14:11:17 +0100] [5507] [INFO] Starting gunicorn 20.0.4
Sep 13 14:11:17 cmdb gunicorn: [2021-09-13 14:11:17 +0100] [5507] [INFO] Listening at: http://127.0.0.1:8001 (5507)
Sep 13 14:11:17 cmdb gunicorn: [2021-09-13 14:11:17 +0100] [5507] [INFO] Using worker: threads
Sep 13 14:11:17 cmdb gunicorn: [2021-09-13 14:11:17 +0100] [5510] [INFO] Booting worker with pid: 5510
Sep 13 14:11:17 cmdb python3: Traceback (most recent call last):
Sep 13 14:11:17 cmdb python3: File "/opt/netbox/netbox/manage.py", line 10, in
Sep 13 14:11:17 cmdb python3: execute_from_command_line(sys.argv)
Sep 13 14:11:17 cmdb python3: File "/usr/local/lib/python3.7/site-packages/django/core/management/init.py", line 401, in execute_from_command_line
Sep 13 14:11:17 cmdb python3: utility.execute()
Sep 13 14:11:17 cmdb python3: File "/usr/local/lib/python3.7/site-packages/django/core/management/init.py", line 377, in execute
Sep 13 14:11:17 cmdb python3: django.setup()
Sep 13 14:11:17 cmdb python3: File "/usr/local/lib/python3.7/site-packages/django/init.py", line 24, in setup
Sep 13 14:11:17 cmdb python3: apps.populate(settings.INSTALLED_APPS)
Sep 13 14:11:17 cmdb python3: File "/usr/local/lib/python3.7/site-packages/django/apps/registry.py", line 91, in populate
Sep 13 14:11:17 cmdb python3: app_config = AppConfig.create(entry)
Sep 13 14:11:17 cmdb python3: File "/usr/local/lib/python3.7/site-packages/django/apps/config.py", line 90, in create
Sep 13 14:11:17 cmdb python3: module = import_module(entry)
Sep 13 14:11:17 cmdb python3: File "/usr/local/lib/python3.7/importlib/init.py", line 127, in import_module
Sep 13 14:11:17 cmdb python3: return _bootstrap._gcd_import(name[level:], package, level)
Sep 13 14:11:17 cmdb python3: File "", line 1006, in _gcd_import
Sep 13 14:11:17 cmdb python3: File "", line 983, in _find_and_load
Sep 13 14:11:17 cmdb python3: File "", line 965, in _find_and_load_unlocked
Sep 13 14:11:17 cmdb python3: ModuleNotFoundError: No module named 'graphiql_debug_toolbar'
Sep 13 14:11:17 cmdb gunicorn: [2021-09-13 14:11:17 +0100] [5511] [INFO] Booting worker with pid: 5511
Sep 13 14:11:17 cmdb gunicorn: [2021-09-13 14:11:17 +0100] [5512] [INFO] Booting worker with pid: 5512
Sep 13 14:11:17 cmdb gunicorn: [2021-09-13 14:11:17 +0100] [5513] [INFO] Booting worker with pid: 5513
Sep 13 14:11:17 cmdb systemd: netbox-rq.service: main process exited, code=exited, status=1/FAILURE
Sep 13 14:11:17 cmdb gunicorn: [2021-09-13 14:11:17 +0100] [5514] [INFO] Booting worker with pid: 5514
Sep 13 14:11:17 cmdb systemd: Unit netbox-rq.service entered failed state.
Sep 13 14:11:17 cmdb systemd: netbox-rq.service failed.
Sep 13 14:11:18 cmdb gunicorn: [2021-09-13 13:11:18 +0000] [5512] [ERROR] Exception in worker process
Sep 13 14:11:18 cmdb gunicorn: Traceback (most recent call last):
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
Sep 13 14:11:18 cmdb gunicorn: worker.init_process()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 92, in init_process
Sep 13 14:11:18 cmdb gunicorn: super().init_process()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base.py", line 119, in init_process
Sep 13 14:11:18 cmdb gunicorn: self.load_wsgi()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
Sep 13 14:11:18 cmdb gunicorn: self.wsgi = self.app.wsgi()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
Sep 13 14:11:18 cmdb gunicorn: self.callable = self.load()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
Sep 13 14:11:18 cmdb gunicorn: return self.load_wsgiapp()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
Sep 13 14:11:18 cmdb gunicorn: return util.import_app(self.app_uri)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/util.py", line 358, in import_app
Sep 13 14:11:18 cmdb gunicorn: mod = importlib.import_module(module)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/importlib/init.py", line 127, in import_module
Sep 13 14:11:18 cmdb gunicorn: return _bootstrap._gcd_import(name[level:], package, level)
Sep 13 14:11:18 cmdb gunicorn: File "", line 1006, in _gcd_import
Sep 13 14:11:18 cmdb gunicorn: File "", line 983, in _find_and_load
Sep 13 14:11:18 cmdb gunicorn: File "", line 967, in _find_and_load_unlocked
Sep 13 14:11:18 cmdb gunicorn: File "", line 677, in _load_unlocked
Sep 13 14:11:18 cmdb gunicorn: File "", line 728, in exec_module
Sep 13 14:11:18 cmdb gunicorn: File "", line 219, in _call_with_frames_removed
Sep 13 14:11:18 cmdb gunicorn: File "/opt/netbox/netbox/netbox/wsgi.py", line 7, in
Sep 13 14:11:18 cmdb gunicorn: application = get_wsgi_application()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application
Sep 13 14:11:18 cmdb gunicorn: django.setup(set_prefix=False)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/init.py", line 24, in setup
Sep 13 14:11:18 cmdb gunicorn: apps.populate(settings.INSTALLED_APPS)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/apps/registry.py", line 91, in populate
Sep 13 14:11:18 cmdb gunicorn: app_config = AppConfig.create(entry)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/apps/config.py", line 90, in create
Sep 13 14:11:18 cmdb gunicorn: module = import_module(entry)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/importlib/init.py", line 127, in import_module
Sep 13 14:11:18 cmdb gunicorn: return _bootstrap._gcd_import(name[level:], package, level)
Sep 13 14:11:18 cmdb gunicorn: File "", line 1006, in _gcd_import
Sep 13 14:11:18 cmdb gunicorn: File "", line 983, in _find_and_load
Sep 13 14:11:18 cmdb gunicorn: File "", line 965, in _find_and_load_unlocked
Sep 13 14:11:18 cmdb gunicorn: ModuleNotFoundError: No module named 'graphiql_debug_toolbar'
Sep 13 14:11:18 cmdb gunicorn: [2021-09-13 13:11:18 +0000] [5512] [INFO] Worker exiting (pid: 5512)
Sep 13 14:11:18 cmdb gunicorn: [2021-09-13 13:11:18 +0000] [5514] [ERROR] Exception in worker process
Sep 13 14:11:18 cmdb gunicorn: Traceback (most recent call last):
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
Sep 13 14:11:18 cmdb gunicorn: worker.init_process()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 92, in init_process
Sep 13 14:11:18 cmdb gunicorn: super().init_process()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base.py", line 119, in init_process
Sep 13 14:11:18 cmdb gunicorn: self.load_wsgi()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
Sep 13 14:11:18 cmdb gunicorn: self.wsgi = self.app.wsgi()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
Sep 13 14:11:18 cmdb gunicorn: self.callable = self.load()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
Sep 13 14:11:18 cmdb gunicorn: return self.load_wsgiapp()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
Sep 13 14:11:18 cmdb gunicorn: return util.import_app(self.app_uri)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/util.py", line 358, in import_app
Sep 13 14:11:18 cmdb gunicorn: mod = importlib.import_module(module)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/importlib/init.py", line 127, in import_module
Sep 13 14:11:18 cmdb gunicorn: return _bootstrap._gcd_import(name[level:], package, level)
Sep 13 14:11:18 cmdb gunicorn: File "", line 1006, in _gcd_import
Sep 13 14:11:18 cmdb gunicorn: File "", line 983, in _find_and_load
Sep 13 14:11:18 cmdb gunicorn: File "", line 967, in _find_and_load_unlocked
Sep 13 14:11:18 cmdb gunicorn: File "", line 677, in _load_unlocked
Sep 13 14:11:18 cmdb gunicorn: File "", line 728, in exec_module
Sep 13 14:11:18 cmdb gunicorn: File "", line 219, in _call_with_frames_removed
Sep 13 14:11:18 cmdb gunicorn: File "/opt/netbox/netbox/netbox/wsgi.py", line 7, in
Sep 13 14:11:18 cmdb gunicorn: application = get_wsgi_application()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application
Sep 13 14:11:18 cmdb gunicorn: django.setup(set_prefix=False)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/init.py", line 24, in setup
Sep 13 14:11:18 cmdb gunicorn: apps.populate(settings.INSTALLED_APPS)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/apps/registry.py", line 91, in populate
Sep 13 14:11:18 cmdb gunicorn: app_config = AppConfig.create(entry)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/apps/config.py", line 90, in create
Sep 13 14:11:18 cmdb gunicorn: module = import_module(entry)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/importlib/init.py", line 127, in import_module
Sep 13 14:11:18 cmdb gunicorn: return _bootstrap._gcd_import(name[level:], package, level)
Sep 13 14:11:18 cmdb gunicorn: File "", line 1006, in _gcd_import
Sep 13 14:11:18 cmdb gunicorn: File "", line 983, in _find_and_load
Sep 13 14:11:18 cmdb gunicorn: File "", line 965, in _find_and_load_unlocked
Sep 13 14:11:18 cmdb gunicorn: ModuleNotFoundError: No module named 'graphiql_debug_toolbar'
Sep 13 14:11:18 cmdb gunicorn: [2021-09-13 13:11:18 +0000] [5514] [INFO] Worker exiting (pid: 5514)
Sep 13 14:11:18 cmdb gunicorn: [2021-09-13 13:11:18 +0000] [5510] [ERROR] Exception in worker process
Sep 13 14:11:18 cmdb gunicorn: Traceback (most recent call last):
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
Sep 13 14:11:18 cmdb gunicorn: worker.init_process()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 92, in init_process
Sep 13 14:11:18 cmdb gunicorn: super().init_process()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base.py", line 119, in init_process
Sep 13 14:11:18 cmdb gunicorn: self.load_wsgi()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
Sep 13 14:11:18 cmdb gunicorn: self.wsgi = self.app.wsgi()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
Sep 13 14:11:18 cmdb gunicorn: self.callable = self.load()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
Sep 13 14:11:18 cmdb gunicorn: return self.load_wsgiapp()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
Sep 13 14:11:18 cmdb gunicorn: return util.import_app(self.app_uri)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/util.py", line 358, in import_app
Sep 13 14:11:18 cmdb gunicorn: mod = importlib.import_module(module)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/importlib/init.py", line 127, in import_module
Sep 13 14:11:18 cmdb gunicorn: return _bootstrap._gcd_import(name[level:], package, level)
Sep 13 14:11:18 cmdb gunicorn: File "", line 1006, in _gcd_import
Sep 13 14:11:18 cmdb gunicorn: File "", line 983, in _find_and_load
Sep 13 14:11:18 cmdb gunicorn: File "", line 967, in _find_and_load_unlocked
Sep 13 14:11:18 cmdb gunicorn: File "", line 677, in _load_unlocked
Sep 13 14:11:18 cmdb gunicorn: File "", line 728, in exec_module
Sep 13 14:11:18 cmdb gunicorn: File "", line 219, in _call_with_frames_removed
Sep 13 14:11:18 cmdb gunicorn: File "/opt/netbox/netbox/netbox/wsgi.py", line 7, in
Sep 13 14:11:18 cmdb gunicorn: application = get_wsgi_application()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application
Sep 13 14:11:18 cmdb gunicorn: django.setup(set_prefix=False)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/init.py", line 24, in setup
Sep 13 14:11:18 cmdb gunicorn: apps.populate(settings.INSTALLED_APPS)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/apps/registry.py", line 91, in populate
Sep 13 14:11:18 cmdb gunicorn: app_config = AppConfig.create(entry)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/apps/config.py", line 90, in create
Sep 13 14:11:18 cmdb gunicorn: module = import_module(entry)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/importlib/init.py", line 127, in import_module
Sep 13 14:11:18 cmdb gunicorn: return _bootstrap._gcd_import(name[level:], package, level)
Sep 13 14:11:18 cmdb gunicorn: File "", line 1006, in _gcd_import
Sep 13 14:11:18 cmdb gunicorn: File "", line 983, in _find_and_load
Sep 13 14:11:18 cmdb gunicorn: File "", line 965, in _find_and_load_unlocked
Sep 13 14:11:18 cmdb gunicorn: ModuleNotFoundError: No module named 'graphiql_debug_toolbar'
Sep 13 14:11:18 cmdb gunicorn: [2021-09-13 13:11:18 +0000] [5510] [INFO] Worker exiting (pid: 5510)
Sep 13 14:11:18 cmdb gunicorn: [2021-09-13 13:11:18 +0000] [5511] [ERROR] Exception in worker process
Sep 13 14:11:18 cmdb gunicorn: Traceback (most recent call last):
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
Sep 13 14:11:18 cmdb gunicorn: worker.init_process()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 92, in init_process
Sep 13 14:11:18 cmdb gunicorn: super().init_process()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base.py", line 119, in init_process
Sep 13 14:11:18 cmdb gunicorn: self.load_wsgi()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
Sep 13 14:11:18 cmdb gunicorn: self.wsgi = self.app.wsgi()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
Sep 13 14:11:18 cmdb gunicorn: self.callable = self.load()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
Sep 13 14:11:18 cmdb gunicorn: return self.load_wsgiapp()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
Sep 13 14:11:18 cmdb gunicorn: return util.import_app(self.app_uri)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/util.py", line 358, in import_app
Sep 13 14:11:18 cmdb gunicorn: mod = importlib.import_module(module)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/importlib/init.py", line 127, in import_module
Sep 13 14:11:18 cmdb gunicorn: return _bootstrap._gcd_import(name[level:], package, level)
Sep 13 14:11:18 cmdb gunicorn: File "", line 1006, in _gcd_import
Sep 13 14:11:18 cmdb gunicorn: File "", line 983, in _find_and_load
Sep 13 14:11:18 cmdb gunicorn: File "", line 967, in _find_and_load_unlocked
Sep 13 14:11:18 cmdb gunicorn: File "", line 677, in _load_unlocked
Sep 13 14:11:18 cmdb gunicorn: File "", line 728, in exec_module
Sep 13 14:11:18 cmdb gunicorn: File "", line 219, in _call_with_frames_removed
Sep 13 14:11:18 cmdb gunicorn: File "/opt/netbox/netbox/netbox/wsgi.py", line 7, in
Sep 13 14:11:18 cmdb gunicorn: application = get_wsgi_application()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application
Sep 13 14:11:18 cmdb gunicorn: django.setup(set_prefix=False)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/init.py", line 24, in setup
Sep 13 14:11:18 cmdb gunicorn: apps.populate(settings.INSTALLED_APPS)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/apps/registry.py", line 91, in populate
Sep 13 14:11:18 cmdb gunicorn: app_config = AppConfig.create(entry)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/apps/config.py", line 90, in create
Sep 13 14:11:18 cmdb gunicorn: module = import_module(entry)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/importlib/init.py", line 127, in import_module
Sep 13 14:11:18 cmdb gunicorn: return _bootstrap._gcd_import(name[level:], package, level)
Sep 13 14:11:18 cmdb gunicorn: File "", line 1006, in _gcd_import
Sep 13 14:11:18 cmdb gunicorn: File "", line 983, in _find_and_load
Sep 13 14:11:18 cmdb gunicorn: File "", line 965, in _find_and_load_unlocked
Sep 13 14:11:18 cmdb gunicorn: ModuleNotFoundError: No module named 'graphiql_debug_toolbar'
Sep 13 14:11:18 cmdb gunicorn: [2021-09-13 13:11:18 +0000] [5511] [INFO] Worker exiting (pid: 5511)
Sep 13 14:11:18 cmdb gunicorn: [2021-09-13 13:11:18 +0000] [5513] [ERROR] Exception in worker process
Sep 13 14:11:18 cmdb gunicorn: Traceback (most recent call last):
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
Sep 13 14:11:18 cmdb gunicorn: worker.init_process()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/gthread.py", line 92, in init_process
Sep 13 14:11:18 cmdb gunicorn: super().init_process()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base.py", line 119, in init_process
Sep 13 14:11:18 cmdb gunicorn: self.load_wsgi()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
Sep 13 14:11:18 cmdb gunicorn: self.wsgi = self.app.wsgi()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
Sep 13 14:11:18 cmdb gunicorn: self.callable = self.load()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
Sep 13 14:11:18 cmdb gunicorn: return self.load_wsgiapp()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
Sep 13 14:11:18 cmdb gunicorn: return util.import_app(self.app_uri)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/util.py", line 358, in import_app
Sep 13 14:11:18 cmdb gunicorn: mod = importlib.import_module(module)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/importlib/init.py", line 127, in import_module
Sep 13 14:11:18 cmdb gunicorn: return _bootstrap._gcd_import(name[level:], package, level)
Sep 13 14:11:18 cmdb gunicorn: File "", line 1006, in _gcd_import
Sep 13 14:11:18 cmdb gunicorn: File "", line 983, in _find_and_load
Sep 13 14:11:18 cmdb gunicorn: File "", line 967, in _find_and_load_unlocked
Sep 13 14:11:18 cmdb gunicorn: File "", line 677, in _load_unlocked
Sep 13 14:11:18 cmdb gunicorn: File "", line 728, in exec_module
Sep 13 14:11:18 cmdb gunicorn: File "", line 219, in _call_with_frames_removed
Sep 13 14:11:18 cmdb gunicorn: File "/opt/netbox/netbox/netbox/wsgi.py", line 7, in
Sep 13 14:11:18 cmdb gunicorn: application = get_wsgi_application()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/core/wsgi.py", line 12, in get_wsgi_application
Sep 13 14:11:18 cmdb gunicorn: django.setup(set_prefix=False)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/init.py", line 24, in setup
Sep 13 14:11:18 cmdb gunicorn: apps.populate(settings.INSTALLED_APPS)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/apps/registry.py", line 91, in populate
Sep 13 14:11:18 cmdb gunicorn: app_config = AppConfig.create(entry)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/django/apps/config.py", line 90, in create
Sep 13 14:11:18 cmdb gunicorn: module = import_module(entry)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/importlib/init.py", line 127, in import_module
Sep 13 14:11:18 cmdb gunicorn: return _bootstrap._gcd_import(name[level:], package, level)
Sep 13 14:11:18 cmdb gunicorn: File "", line 1006, in _gcd_import
Sep 13 14:11:18 cmdb gunicorn: File "", line 983, in _find_and_load
Sep 13 14:11:18 cmdb gunicorn: File "", line 965, in _find_and_load_unlocked
Sep 13 14:11:18 cmdb gunicorn: ModuleNotFoundError: No module named 'graphiql_debug_toolbar'
Sep 13 14:11:18 cmdb gunicorn: [2021-09-13 13:11:18 +0000] [5513] [INFO] Worker exiting (pid: 5513)
Sep 13 14:11:18 cmdb gunicorn: Traceback (most recent call last):
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 209, in run
Sep 13 14:11:18 cmdb gunicorn: self.sleep()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 357, in sleep
Sep 13 14:11:18 cmdb gunicorn: ready = select.select([self.PIPE[0]], [], [], 1.0)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 242, in handle_chld
Sep 13 14:11:18 cmdb gunicorn: self.reap_workers()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 525, in reap_workers
Sep 13 14:11:18 cmdb gunicorn: raise HaltServer(reason, self.WORKER_BOOT_ERROR)
Sep 13 14:11:18 cmdb gunicorn: gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
Sep 13 14:11:18 cmdb gunicorn: During handling of the above exception, another exception occurred:
Sep 13 14:11:18 cmdb gunicorn: Traceback (most recent call last):
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/bin/gunicorn", line 8, in
Sep 13 14:11:18 cmdb gunicorn: sys.exit(run())
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 58, in run
Sep 13 14:11:18 cmdb gunicorn: WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/app/base.py", line 228, in run
Sep 13 14:11:18 cmdb gunicorn: super().run()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/app/base.py", line 72, in run
Sep 13 14:11:18 cmdb gunicorn: Arbiter(self).run()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 229, in run
Sep 13 14:11:18 cmdb gunicorn: self.halt(reason=inst.reason, exit_status=inst.exit_status)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 342, in halt
Sep 13 14:11:18 cmdb gunicorn: self.stop()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 393, in stop
Sep 13 14:11:18 cmdb gunicorn: time.sleep(0.1)
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 242, in handle_chld
Sep 13 14:11:18 cmdb gunicorn: self.reap_workers()
Sep 13 14:11:18 cmdb gunicorn: File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 525, in reap_workers
Sep 13 14:11:18 cmdb gunicorn: raise HaltServer(reason, self.WORKER_BOOT_ERROR)
Sep 13 14:11:18 cmdb gunicorn: gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
Sep 13 14:11:18 cmdb systemd: netbox.service: main process exited, code=exited, status=1/FAILURE
Sep 13 14:11:18 cmdb systemd: Unit netbox.service entered failed state.
Sep 13 14:11:18 cmdb systemd: netbox.service failed.

Is the issue related to the mkdocs build as I can't recall seeing this before with other upgrades.

[candlerb]

Sep 13 14:11:17 cmdb python3: File "/usr/local/lib/python3.7/site-packages/django/core/management/init.py

This shows that you're not running Netbox inside its virtualenv. It's picking up packages from the host instead.

Almost certainly the problem is in your netbox.service and netbox-rq.service. They should look like the ones in the contrib directory, especially these lines:

# netbox.service
ExecStart=/opt/netbox/venv/bin/gunicorn --pid /var/tmp/netbox.pid --pythonpath /opt/netbox/netbox --config /opt/netbox/gunicorn.py netbox.wsgi

# netbox-rq.service
ExecStart=/opt/netbox/venv/bin/python3 /opt/netbox/netbox/manage.py rqworker high default low

Note: after changing these files (i.e. /etc/systemd/system/netbox.service etc), remember to do systemctl daemon-reload.

[dman4cloud]

Thanks @candlerb that did the trick, successfully on v3.0.2 now. Not sure how it was working before this.

[candlerb]

That's good news.

As for how it was working before: maybe at some point you'd done pip3 install -r requirements.txt to install the dependencies directly on the host, outside of the venv. However, netbox 3.0 has added some new dependencies.