Netbox 3.0.2 with LDAP - local accounts not able to login after upgrade

[ziggekatten]

Netbox version: 3.0.2
Python: 3.9.5
Redis: 4.0.14

So, we upgraded our first instance of 2.11.12 (dev and training instances) that have LDAP connectivity, and all seems to work fine, using AD-accounts logging in.

However, we have a few local accounts that we use for various reasons, and trying to login with these generates the following error:

Disabling LDAP, and the local accounts works as they should.

Are there any changes in parameters that are needed to be able to log in with both LDAP and local accounts? trace below:

`Django Version: 3.2.7
Python Version: 3.9.5
Installed Applications:
['django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.humanize',
'corsheaders',
'debug_toolbar',
'graphiql_debug_toolbar',
'django_filters',
'django_tables2',
'django_prometheus',
'graphene_django',
'mptt',
'rest_framework',
'taggit',
'timezone_field',
'circuits',
'dcim',
'ipam',
'extras',
'tenancy',
'users',
'utilities',
'virtualization',
'django_rq',
'drf_yasg']
Installed Middleware:
['graphiql_debug_toolbar.middleware.DebugToolbarMiddleware',
'django_prometheus.middleware.PrometheusBeforeMiddleware',
'corsheaders.middleware.CorsMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
'netbox.middleware.ExceptionHandlingMiddleware',
'netbox.middleware.RemoteUserMiddleware',
'netbox.middleware.LoginRequiredMiddleware',
'netbox.middleware.APIVersionMiddleware',
'netbox.middleware.ObjectChangeMiddleware',
'django_prometheus.middleware.PrometheusAfterMiddleware']

Traceback (most recent call last):

The above exception (Authentication required.) was the direct cause of the following exception:
File "/opt/netbox/venv/lib/python3.9/site-packages/django_redis/cache.py", line 31, in _decorator
return method(self, *args, **kwargs)
File "/opt/netbox/venv/lib/python3.9/site-packages/django_redis/cache.py", line 98, in _get
return self.client.get(key, default=default, version=version, client=client)
File "/opt/netbox/venv/lib/python3.9/site-packages/django_redis/client/default.py", line 260, in get
raise ConnectionInterrupted(connection=client) from e

During handling of the above exception (Redis AuthenticationError: Authentication required.), another exception occurred:
File "/opt/netbox/venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/opt/netbox/venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/opt/netbox/venv/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view
return self.dispatch(request, *args, **kwargs)
File "/opt/netbox/venv/lib/python3.9/site-packages/django/views/generic/base.py", line 98, in dispatch
return handler(request, *args, **kwargs)
File "/opt/netbox/netbox/netbox/views/init.py", line 136, in get
latest_release = cache.get('latest_release')
File "/opt/netbox/venv/lib/python3.9/site-packages/django_redis/cache.py", line 91, in get
value = self._get(key, default, version, client)
File "/opt/netbox/venv/lib/python3.9/site-packages/django_redis/cache.py", line 38, in _decorator
raise e.cause
File "/opt/netbox/venv/lib/python3.9/site-packages/django_redis/client/default.py", line 258, in get
value = client.get(key)
File "/opt/netbox/venv/lib/python3.9/site-packages/redis/client.py", line 1606, in get
return self.execute_command('GET', name)
File "/opt/netbox/venv/lib/python3.9/site-packages/redis/client.py", line 898, in execute_command
conn = self.connection or pool.get_connection(command_name, **options)
File "/opt/netbox/venv/lib/python3.9/site-packages/redis/connection.py", line 1192, in get_connection
connection.connect()
File "/opt/netbox/venv/lib/python3.9/site-packages/redis/connection.py", line 567, in connect
self.on_connect()
File "/opt/netbox/venv/lib/python3.9/site-packages/redis/connection.py", line 664, in on_connect
if nativestr(self.read_response()) != 'OK':
File "/opt/netbox/venv/lib/python3.9/site-packages/redis/connection.py", line 739, in read_response
response = self._parser.read_response()
File "/opt/netbox/venv/lib/python3.9/site-packages/redis/connection.py", line 340, in read_response
raise error

Exception Type: AuthenticationError at /ipam-dev/
Exception Value: Authentication required.`

[koratfood]

I recently upgraded my instance from 2.11.12 to 3.0.2 as well, and I am using a mixture of LDAP and local accounts. No problems with either after upgrade. If you can post your configuration.py and ldap_config.py (with your domain names and passwords obfuscated), I can try to reproduce the problem.

[candlerb]

The issue seems to be with authentication between Netbox and Redis. Is your Redis server configured to require authentication? And if so, does the REDIS block in configuration.py have the correct settings? You could test this further by exercising something else which requires redis, e.g. webhooks.

There is a smoking gun here in your backtrace:

File "/opt/netbox/netbox/netbox/views/init.py", line 136, in get
latest_release = cache.get('latest_release')

Now, this should say __init__.py (*). If so, the code looks like this:

        # Check whether a new release is available. (Only for staff/superusers.)
        new_release = None
        if request.user.is_staff or request.user.is_superuser:
            latest_release = cache.get('latest_release')

I suggest the problem isn't really the difference between LDAP users and local users; it's the difference between staff/superuser and non-staff/non-superuser. And it's this redis query which is causing the problem.

---

(*) When quoting console captures in github, put three backticks (```) on a line of their own above and below, to avoid markdown garbling the content.

[ziggekatten]

I have now tried with a brand new 3.0.2 installation on a VM with Ubuntu 20 and Python 3.9. Redis 6. Still the same issue with admin privilegies.

Maybe time to raise a bug report?

[candlerb]

Perhaps you could try tcpdump'ing the traffic between netbox and redis first, to see if the problem really is that netbox isn't sending credentials when it should.

[candlerb]

If you can replicate this on a fresh installation of netbox+redis, then certainly raise it as a bug.

As a workaround: that bit of code I highlighted (in the home page view) need not run at all if RELEASE_CHECK_URL is None.

There is another thing you could test. Set RELEASE_CHECK_URL to its github value (not None), and then run the "housekeeping.py" script:

/opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py housekeeping

This should check the release, and then set the cache entry to store it. It would be interesting to see if that fails in the same way (i.e. redis authentication error).

[macn27]

Hi - I am also having this issue after updating from 2.11.12 to 3.0.7.

I did set my RELEASE_CHECK_URL to none and that did not resolve the issue.

I also set my RELEASE_CHECK_URL to the github value like @candlerb suggested and ran the housekeeping task. That resulted in the same error.

`matthew@MBSSVIT02:/opt/netbox/netbox/netbox$ /opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py housekeeping
/opt/netbox-3.0.7/netbox/netbox/settings.py:50: UserWarning: The CACHE_TIMEOUT configuration parameter was removed in v3.0.0 and no longer has any effect.
warnings.warn(
/opt/netbox-3.0.7/netbox/netbox/settings.py:54: UserWarning: The RELEASE_CHECK_TIMEOUT configuration parameter was removed in v3.0.0 and no longer has any effect.
warnings.warn(
[] Clearing expired authentication sessions
Sessions cleared.
[] Checking for expired changelog records
No expired records found.
[*] Checking for latest release
Latest release: 3.0.7
Traceback (most recent call last):
File "/opt/netbox/venv/lib/python3.8/site-packages/django_redis/cache.py", line 31, in _decorator
return method(self, *args, **kwargs)
File "/opt/netbox/venv/lib/python3.8/site-packages/django_redis/cache.py", line 80, in set
return self.client.set(*args, **kwargs)
File "/opt/netbox/venv/lib/python3.8/site-packages/django_redis/client/default.py", line 185, in set
raise ConnectionInterrupted(connection=client) from e
django_redis.exceptions.ConnectionInterrupted: Redis AuthenticationError: Authentication required.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/netbox/netbox/manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/opt/netbox/venv/lib/python3.8/site-packages/django/core/management/init.py", line 419, in execute_from_command_line
utility.execute()
File "/opt/netbox/venv/lib/python3.8/site-packages/django/core/management/init.py", line 413, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/opt/netbox/venv/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv
self.execute(*args, **cmd_options)
File "/opt/netbox/venv/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute
output = self.handle(*args, **options)
File "/opt/netbox-3.0.7/netbox/extras/management/commands/housekeeping.py", line 94, in handle
cache.set('latest_release', latest_release, None)
File "/opt/netbox/venv/lib/python3.8/site-packages/django_redis/cache.py", line 38, in _decorator
raise e.cause
File "/opt/netbox/venv/lib/python3.8/site-packages/django_redis/client/default.py", line 175, in set
return bool(client.set(nkey, nvalue, nx=nx, px=timeout, xx=xx))
File "/opt/netbox/venv/lib/python3.8/site-packages/redis/client.py", line 1801, in set
return self.execute_command('SET', *pieces)
File "/opt/netbox/venv/lib/python3.8/site-packages/redis/client.py", line 898, in execute_command
conn = self.connection or pool.get_connection(command_name, **options)
File "/opt/netbox/venv/lib/python3.8/site-packages/redis/connection.py", line 1192, in get_connection
connection.connect()
File "/opt/netbox/venv/lib/python3.8/site-packages/redis/connection.py", line 567, in connect
self.on_connect()
File "/opt/netbox/venv/lib/python3.8/site-packages/redis/connection.py", line 664, in on_connect
if nativestr(self.read_response()) != 'OK':
File "/opt/netbox/venv/lib/python3.8/site-packages/redis/connection.py", line 739, in read_response
response = self._parser.read_response()
File "/opt/netbox/venv/lib/python3.8/site-packages/redis/connection.py", line 340, in read_response
raise error
redis.exceptions.AuthenticationError: Authentication required.
matthew@MBSSVIT02:/opt/netbox/netbox/netbox$ `

[macn27]

OK - my bad - it looks like my redis configuration in configuration.py needed to be updated - the caching section did not have a password configured. Once I added that and restarted the netbox services I was able to login.

[rfdrake]

I have a similar issue without the tracebacks. I can login to a local account in the GUI, but if I try to use a token assigned to them I get "User inactive" and it immediately deactivates the local account.

I wonder if the order of the check in TokenAuthentication authenticate_credentials might be reversed? It's doing an 'is_active' check before finding if a user exists in ldap. I think the ldap backend might be returning no for 'is_active' because the user doesn't exist in LDAP.