`Key Error 'thumbnail_key'` for a Nuxeo collection harvested to -stage #28417

[christinklez]

Observed:
Rikolti is successfully harvesting Nuxeo collections that have docx filetypes, but Calisphere cannot support these records. Calisphere displays a "Key Error" page instead.
https://nuxeo.cdlib.org/nuxeo/nxdoc/default/31c96ae5-d617-4d20-8e66-ab7efe9592a5/view_documents
(Note that we're going to ask UCI to swap the docx to pdf's instead.)

What we expect:
Rikolti should tell us if it runs into a file format Calisphere can't support, maybe at some point during content_harvesting. We think this could go into the backlog, to revisit later.

---

Clicking to view the item record pages in this collection results in a Key Error page: https://calisphere-stage.cdlib.org/collections/28417/
Registry: https://registry.cdlib.org/admin/library_collection/collection/28417/change/
Two attempts at harvesting this collection, both successful runs in Rikolti.
Unsure if this is a Rikolti or Frontend issue.

Calisphere-stage traceback:

Environment:

Request Method: GET
Request URL: https://calisphere-stage.cdlib.org/item/ark:/81235/d88k7597d/

Django Version: 3.2.25
Python Version: 3.9.21
Installed Applications:
('exhibits.apps.ExhibitsConfig',
 'django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.sites',
 'django.contrib.humanize',
 'django.contrib.sitemaps',
 'easy_pjax',
 'calisphere',
 'static_sitemaps',
 'health_check',
 'health_check.cache',
 'health_check.storage',
 'snowpenguin.django.recaptcha2')
Installed Middleware:
('django.middleware.security.SecurityMiddleware',
 'whitenoise.middleware.WhiteNoiseMiddleware',
 'django_permissions_policy.PermissionsPolicyMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.gzip.GZipMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'calisphere.redirects.redirect_middleware.InMemoryRedirectMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware')



Traceback (most recent call last):
  File "/var/app/venv/staging-LQM1lest/lib64/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/var/app/venv/staging-LQM1lest/lib64/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/var/app/venv/staging-LQM1lest/lib64/python3.9/site-packages/sentry_sdk/integrations/django/views.py", line 84, in sentry_wrapped_callback
    return callback(request, *args, **kwargs)
  File "/var/app/current/calisphere/decorators.py", line 24, in wrapper
    return cached(request, *args, **kwargs)
  File "/var/app/venv/staging-LQM1lest/lib64/python3.9/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/var/app/current/calisphere/views.py", line 92, in item_view
    item = Record(index_result.item, request, child_index, index)
  File "/var/app/current/calisphere/record.py", line 274, in __init__
    c['format'] for c in self.get_children() if 'format' in c])
  File "/var/app/current/calisphere/record.py", line 407, in get_children
    self.children = [
  File "/var/app/current/calisphere/record.py", line 408, in <listcomp>
    make_es_carousel_display(child) for child in
  File "/var/app/current/calisphere/record.py", line 133, in make_es_carousel_display
    f"{ settings.UCLDC_NUXEO_THUMBS }thumbnails/{ child['thumbnail_key'] }"

Exception Type: KeyError at /item/ark:/81235/d88k7597d/
Exception Value: 'thumbnail_key'

[barbarahui]

@christinklez At least some of the component files for this collection are docx files, and the content harvester doesn’t support making thumbnails for these. I updated the nuxeo mapper so that it no longer maps thumbnail_source for these. I think the frontend complex object carousel will display the placeholder file icon as the thumbnail, but can you try re-harvesting and see what happens?

[barbarahui]

Hrm, actually now that I’m looking at it more closely, I don’t think this is going to fix it. Backing up: do we want to support docx files? If UCI uploaded these as PDFs, we could fully support the collection.

[christinklez]

@barbarahui - reharvested and continuing to get the error.

I don't think we want to support docx files (and it's not listed in our list of supported filetypes page: https://help.oac.cdlib.org/support/solutions/articles/9000100797-creating-objects).

I'll chat with Adrian about this and update this thread. I agree with you that ideally UCI should upload PDFs instead.

[christinklez]

@barbarahui - we've updated our documentation to indicate that docx files are not supported for Calisphere harvesting.

We were thinking that Rikolti should tell us if it runs into a file format Calisphere can't support, maybe at some point during content_harvesting. We think this could go into the backlog, to revisit later.

Thank you!!