Doc needs refine on genericOIDC is outdated in keycloak v20.0.0+ #3726
Description
Bug description
How to reproduce
Just use the configuration provided in z2jh doc page Authentication and authorization
This is my helm values override:
--snip--
hub:
baseUrl: /jupyterhub
config:
JupyterHub:
authenticator_class: generic-oauth
GenericOAuthenticator:
client_id: jupyterhub
client_secret: [secret]
oauth_callback_url: http://[secret]/jupyterhub/hub/oauth_callback
authorize_url: http://[secret]/realms/jupyterhub/protocol/openid-connect/auth
token_url: http://[secret]/realms/jupyterhub/protocol/openid-connect/token
userdata_url: http://[secret]/realms/jupyterhub/protocol/openid-connect/userinfo
login_service: keycloak
username_claim: preferred_username
userdata_params:
state: state
# Allow all Keycloak users
allow_all: true
admin_users:
- admin
--snip--Expected behaviour
Login to jupyterhub after being authenticated in Keycloak.
Actual behaviour
In webUI, jupyterhub returns 500 internal error. And I can't login.
If I refresh the page, it shows that state cookie is invalid(expected).
Digging into log
I found something interesting in Keycloak and jupyterhub's pods's log:
Keycloak log:
2025-08-22 07:28:57,603 WARN [org.keycloak.events] (executor-thread-30) type="USER_INFO_REQUEST_ERROR", realmId="7dc965ed-7c80-4006-8492-e52503bdcf02", realmName="jupyterhub", clientId="null", userId="null", ipAddress="10.0.2.116", error="access_denied", reason="Missing openid scope", auth_method="validate_access_token"
2025-08-22 08:33:36,746 WARN [org.keycloak.events] (executor-thread-34) type="USER_INFO_REQUEST_ERROR", realmId="7dc965ed-7c80-4006-8492-e52503bdcf02", realmName="jupyterhub", clientId="null", userId="null", ipAddress="10.0.2.116", error="access_denied", reason="Missing openid scope", auth_method="validate_access_token"
--snip--(I've tried to login for many times, and each time produces one entry like this)
hub pod log:
[I 2025-08-22 09:46:41.037 JupyterHub log:192] 200 GET /jupyterhub/hub/login (@::ffff:10.0.2.181) 69.38ms
[I 2025-08-22 09:46:42.068 JupyterHub oauth2:126] OAuth redirect: http://192.168.83.23/jupyterhub/hub/oauth_callback
[I 2025-08-22 09:46:42.070 JupyterHub log:192] 302 GET /jupyterhub/hub/oauth_login?next= -> http://192.168.83.23/realms/jupyterhub/protocol/openid-connect/auth?response_type=code&redirect_uri=http%3A%2F%2F192.168.83.23%2Fjupyterhub%2Fhub%2Foauth_callback&client_id=jupyterhub&code_challenge=[secret]&code_challenge_method=[secret]&state=[secret] (@::ffff:10.0.2.181) 2.34ms
[E 2025-08-22 09:46:44.677 JupyterHub oauth2:857] Error Fetching user info... 403 GET http://192.168.83.23/realms/jupyterhub/protocol/openid-connect/userinfo:
[E 2025-08-22 09:46:44.677 JupyterHub web:1875] Uncaught exception GET /jupyterhub/hub/oauth_callback?state=eyJzdGF0ZV9pZCI6ICJkMTI4MjU3Nzg5NjU0MTBkOWMyNjliOTExMTY4MTRiNiJ9&session_state=ec1ae9bc-5134-41eb-a2e8-8fee9f8efd09&iss=http%3A%2F%2F192.168.83.23%2Frealms%2Fjupyterhub&code=52348431-04bf-4344-93a6-ad2a9f96aee3.ec1ae9bc-5134-41eb-a2e8-8fee9f8efd09.431d7f56-c74b-4429-8904-445ccce5b2cd (::ffff:10.0.2.181)
HTTPServerRequest(protocol='http', host='192.168.83.23', method='GET', uri='/jupyterhub/hub/oauth_callback?state=eyJzdGF0ZV9pZCI6ICJkMTI4MjU3Nzg5NjU0MTBkOWMyNjliOTExMTY4MTRiNiJ9&session_state=ec1ae9bc-5134-41eb-a2e8-8fee9f8efd09&iss=http%3A%2F%2F192.168.83.23%2Frealms%2Fjupyterhub&code=52348431-04bf-4344-93a6-ad2a9f96aee3.ec1ae9bc-5134-41eb-a2e8-8fee9f8efd09.431d7f56-c74b-4429-8904-445ccce5b2cd', version='HTTP/1.1', remote_ip='::ffff:10.0.2.181')
Traceback (most recent call last):
File "/usr/local/lib/python3.12/site-packages/tornado/web.py", line 1790, in _execute
result = await result
^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/oauthenticator/oauth2.py", line 245, in get
user = await self.login_user()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/jupyterhub/handlers/base.py", line 964, in login_user
authenticated = await self.authenticate(data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/jupyterhub/auth.py", line 695, in get_authenticated_user
authenticated = await maybe_future(self.authenticate(handler, data))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/oauthenticator/oauth2.py", line 1318, in authenticate
return await self._token_to_auth_model(token_info)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/oauthenticator/oauth2.py", line 1436, in _token_to_auth_model
user_info = await self.token_to_user(token_info)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/oauthenticator/oauth2.py", line 1164, in token_to_user
return await self.httpfetch(
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/oauthenticator/oauth2.py", line 892, in httpfetch
return await self.fetch(
^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/oauthenticator/oauth2.py", line 858, in fetch
raise e
File "/usr/local/lib/python3.12/site-packages/oauthenticator/oauth2.py", line 837, in fetch
resp = await self.http_client.fetch(req, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tornado.httpclient.HTTPClientError: HTTP 403: Forbidden
[E 2025-08-22 09:46:44.733 JupyterHub log:184] {
"X-Forwarded-Host": "192.168.83.23",
"X-Forwarded-Port": "80",
"X-Request-Id": "d16de3b1-1ee1-4803-af9e-b4fa2666d45e",
"X-Envoy-Internal": "true",
"X-Forwarded-Proto": "http,http",
"X-Forwarded-For": "192.168.83.44,::ffff:10.0.2.181",
"Cookie": "_xsrf=[secret]; oauthenticator-state=[secret]",
"Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6",
"Accept-Encoding": "gzip, deflate",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36 Edg/139.0.0.0",
"Upgrade-Insecure-Requests": "1",
"Cache-Control": "max-age=0",
"Host": "192.168.83.23",
"Connection": "keep-alive"
}
[E 2025-08-22 09:46:44.733 JupyterHub log:192] 500 GET /jupyterhub/hub/oauth_callback?state=[secret]&session_state=[secret]&iss=http%3A%2F%2F192.168.83.23%2Frealms%2Fjupyterhub&code=[secret] (@::ffff:10.0.2.181) 97.48ms
Looks like Keycloak has denied jupyterhub's request.
A potential fix (WORKED!)
After I added scope in values override of helm, everything was fine.
--snip--
hub:
baseUrl: /jupyterhub
config:
JupyterHub:
authenticator_class: generic-oauth
GenericOAuthenticator:
client_id: jupyterhub
client_secret: [secret]
oauth_callback_url: http://[secret]/jupyterhub/hub/oauth_callback
authorize_url: http://[secret]/realms/jupyterhub/protocol/openid-connect/auth
token_url: http://[secret]/realms/jupyterhub/protocol/openid-connect/token
userdata_url: http://[secret]/realms/jupyterhub/protocol/openid-connect/userinfo
login_service: keycloak
username_claim: preferred_username
userdata_params:
state: state
# Allow all Keycloak users
allow_all: true
admin_users:
- admin
scope:
- openid
- email
--snip--system info
$ kubectl version
Client Version: v1.33.4
Kustomize Version: v5.6.0
Server Version: v1.33.0
$ helm list -n jupyterhub
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
jupyterhub jupyterhub 5 2025-08-22 09:45:50.087621958 +0000 UTC deployed jupyterhub-4.2.0 5.3.0
Keycloak manifests: https://raw.githubusercontent.com/keycloak/keycloak-quickstarts/refs/heads/main/kubernetes/keycloak.yaml
My request
Maybe it's a good idea to add scope to sample conf in documentation of Z2JH.
But there is still a thing stopped me from opening a PR and just fix it: I didn't know why it worked with these two keys in scope and how to explain it in documentation.(help needed form someone who masters OIDC)
Reference
Here are some disscussions I have found helpful while debugging.
[[https://stackoverflow.com/questions/75890476/invalid-scopes-email-openid-public-profile]]
[[https://keycloak.discourse.group/t/issue-on-userinfo-endpoint-at-keycloak-20/18461]]
A simple disclaimer
I'm not a English native speaker, and a student who takes DevOps as hobby, I'll be appreciative if you could pointout both grammatical and technical mistakes. Thanks in advance!