Restore feature not working - django.core.serializers.base.DeserializationError #1865

mesenger · 2022-12-13T15:15:18Z

Self-Hosted Version

22.12.0.dev0+99926859321bd1f82d1ac4b07bc8601e3d78b9c9

CPU Architecture

x86_64

Docker Version

20.10.21

Docker Compose Version

1.29.2

Steps to Reproduce

Take backup of existing Sentry installation - OK
Remove Sentry, purge volumes, install same version ./install.sh --skip-user-prompt - OK
try to restore previous backup: docker-compose run --rm -T web import /etc/sentry/backup.json - Error:
docker-compose run --rm -T web import /etc/sentry/backup.json
Starting sentry-self-hosted_symbolicator_1 ... done
Starting sentry-self-hosted_postgres_1 ... done
Starting sentry-self-hosted_zookeeper_1 ... done
Starting sentry-self-hosted_clickhouse_1 ... done
Starting sentry-self-hosted_memcached_1 ... done
Starting sentry-self-hosted_smtp_1 ... done
Starting sentry-self-hosted_redis_1 ... done
Starting sentry-self-hosted_kafka_1 ... done
Starting sentry-self-hosted_snuba-subscription-consumer-transactions_1 ... done
Starting sentry-self-hosted_snuba-replacer_1 ... done
Starting sentry-self-hosted_snuba-sessions-consumer_1 ... done
Starting sentry-self-hosted_snuba-consumer_1 ... done
Starting sentry-self-hosted_snuba-api_1 ... done
Starting sentry-self-hosted_snuba-transactions-consumer_1 ... done
Starting sentry-self-hosted_snuba-outcomes-consumer_1 ... done
Starting sentry-self-hosted_snuba-subscription-consumer-events_1 ... done
Creating sentry-self-hosted_web_run ... done
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
15:06:27 [INFO] sentry.plugins.github: apps-not-configured
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/django/core/serializers/json.py", line 68, in Deserializer
objects = json.loads(stream_or_string)
File "/usr/local/lib/python3.8/json/init.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/bin/sentry", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.8/site-packages/sentry/runner/init.py", line 192, in main
func(**kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1128, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/sentry/runner/decorators.py", line 29, in inner
return ctx.invoke(f, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return _callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/sentry/runner/commands/backup.py", line 19, in import
for obj in serializers.deserialize("json", src, stream=True, use_natural_keys=True):
File "/usr/local/lib/python3.8/site-packages/django/core/serializers/json.py", line 73, in Deserializer
raise DeserializationError() from exc
django.core.serializers.base.DeserializationError
ERROR: 1

Unable to restore

Expected Result

Restore would finish OK and project data would be restored correctly

Actual Result

backup is not restored, producing error:

15:06:27 [INFO] sentry.plugins.github: apps-not-configured
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/django/core/serializers/json.py", line 68, in Deserializer
objects = json.loads(stream_or_string)
File "/usr/local/lib/python3.8/json/init.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/bin/sentry", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.8/site-packages/sentry/runner/init.py", line 192, in main
func(**kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1128, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/sentry/runner/decorators.py", line 29, in inner
return ctx.invoke(f, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return _callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/sentry/runner/commands/backup.py", line 19, in import
for obj in serializers.deserialize("json", src, stream=True, use_natural_keys=True):
File "/usr/local/lib/python3.8/site-packages/django/core/serializers/json.py", line 73, in Deserializer
raise DeserializationError() from exc
django.core.serializers.base.DeserializationError
ERROR: 1

Event ID

No response

emmatyping · 2022-12-13T22:03:44Z

Did you check the output of the generated JSON file? It seems that something expected is missing.

mesenger · 2022-12-14T13:26:22Z

I am not an expert on json syntax, it looks good enough to me. jq reports some errors. Here are some snippets :

root@sentry1:~/sentry# jq sentry/backup.json
jq: error: sentry/0 is not defined at <top-level>, line 1:
sentry/backup.json
jq: error: backup/0 is not defined at <top-level>, line 1:
sentry/backup.json       
jq: 2 compile errors

root@sentry1:~/sentry# head sentry/backup.json 
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
[
{
  "model": "sites.site",
  "pk": 1,
  "fields": {
    "domain": "example.com",

root@sentry1:~/sentry# tail sentry/backup.json
{
  "model": "sentry.projectoption",
  "pk": 7,
  "fields": {
    "project": 2,
    "key": "sentry:option-epoch",
    "value": 8
  }
}
]

mesenger · 2022-12-14T13:56:26Z

I removed the top part from backup.json, starting now with this:
[ { "model": "sites.site", "pk": 1, "fields": { "domain": "example.com",

now I get more specific error:

`13:44:04 [INFO] sentry.plugins.github: apps-not-configured
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/django/core/serializers/python.py", line 143, in Deserializer
data[field.name] = field.to_python(field_value)
File "/usr/local/lib/python3.8/site-packages/sentry/db/models/fields/picklefield.py", line 54, in to_python
return json.loads(value)
File "/usr/local/lib/python3.8/site-packages/sentry/utils/json.py", line 120, in loads
return _default_decoder.decode(value)
File "/usr/local/lib/python3.8/site-packages/simplejson/decoder.py", line 370, in decode
obj, end = self.raw_decode(s)
File "/usr/local/lib/python3.8/site-packages/simplejson/decoder.py", line 392, in raw_decode
raise TypeError("Input string must be text, not bytes")
TypeError: Input string must be text, not bytes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/bin/sentry", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.8/site-packages/sentry/runner/init.py", line 192, in main
func(**kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1128, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/sentry/runner/decorators.py", line 29, in inner
return ctx.invoke(f, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return _callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/sentry/runner/commands/backup.py", line 19, in import
for obj in serializers.deserialize("json", src, stream=True, use_natural_keys=True):
File "/usr/local/lib/python3.8/site-packages/django/core/serializers/json.py", line 69, in Deserializer
yield from PythonDeserializer(objects, **options)
File "/usr/local/lib/python3.8/site-packages/django/core/serializers/python.py", line 145, in Deserializer
raise base.DeserializationError.WithData(e, d['model'], d.get('pk'), field_value)
django.core.serializers.base.DeserializationError: Input string must be text, not bytes: (sentry.option:pk=1) field_value was '1670942824.3678975'
ERROR: 1
`

I can see this entry in backup.json, but not sure what could be wrong with it:
{ "model": "sentry.option", "pk": 1, "fields": { "key": "sentry:last_worker_ping", "value": 1670942824.3678975, "last_updated": "2022-12-13T14:47:04.367Z" } },

emmatyping · 2022-12-14T23:32:15Z

Thanks for digging into that! In that case, I believe this might be the same as getsentry/sentry#36868 I will close this in favor of that issue. Unfortunately right now we don't have the bandwidth to fix the backup, but we likely will tackle it Q1.

emmatyping added the Type: Support Request label Dec 13, 2022

emmatyping closed this as completed Dec 14, 2022

github-actions bot locked and limited conversation to collaborators Dec 30, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Restore feature not working - django.core.serializers.base.DeserializationError #1865

Restore feature not working - django.core.serializers.base.DeserializationError #1865

mesenger commented Dec 13, 2022

emmatyping commented Dec 13, 2022

Uh oh!

mesenger commented Dec 14, 2022

Uh oh!

mesenger commented Dec 14, 2022

Uh oh!

emmatyping commented Dec 14, 2022

Uh oh!

Uh oh!

Restore feature not working - django.core.serializers.base.DeserializationError #1865

Restore feature not working - django.core.serializers.base.DeserializationError #1865

Comments

mesenger commented Dec 13, 2022

Self-Hosted Version

CPU Architecture

Docker Version

Docker Compose Version

Steps to Reproduce

Expected Result

Actual Result

Event ID

emmatyping commented Dec 13, 2022

Uh oh!

mesenger commented Dec 14, 2022

Uh oh!

mesenger commented Dec 14, 2022

Uh oh!

emmatyping commented Dec 14, 2022

Uh oh!