Unable to perform merge operation

[cristinacivili-work]

Hello,
I'm attempting to merge an existing branch into a newly created branch (not a descendant of the first).
The situation looks like this, where target is empty:

• "source": "MAIN/2020-07-31/SNOMEDCT-UK/2021-03-17/CUSTOM-BRANCH",
• "target": "MAIN/2021-01-31/SNOMEDCT-UK/2021-05-12/CUSTOM-BRANCH"

After about 10 minutes while the merge job appears in progress, it fails returning the following:

{
    "id": "d4978c90560191f109ad740f95e410ed91db329b",
    "status": "FAILED",
    "scheduledDate": "2021-07-27T15:20:01.070+0000",
    "startDate": "2021-07-27T15:20:01.089+0000",
    "endDate": "2021-07-27T15:30:08.214+0000",
    "apiError": {
        "message": "java.lang.NegativeArraySizeException: -1390694110",
        "developerMessage": "Exception caught while executing request in remote job.",
        "code": 0,
        "status": 500,
        "additionalInfo": {
            "exception-class": "SnowowlRuntimeException"
        }
    },
    "conflicts": []
}

I tried both with squash=true and squash=false.
There are not relevant logs produced at DEBUG level.
I was wondering if you have any idea of what could be causing this?

Thanks is advance.

Cristina

[cmark]

Hi @cristinacivili-work,

Interesting, we have never seen this error before.

Could you please provide the following information for further investigation?

• Version of your Snow Owl instance?
• Elasticsearch version?
• What content sits on the MAIN/2020-07-31/SNOMEDCT-UK/2021-03-17/CUSTOM-BRANCH?
• What content sits on the MAIN/2021-01-31/SNOMEDCT-UK/2021-05-12/CUSTOM-BRANCH? You said empty but according to the path it should not be empty (apart from the CUSTOM_BRANCH which could be empty of course).
• How did you set up the content on both branches? Do they have a common path/content (either the INT or the UK part)?

Cheers,
Mark

[cristinacivili-work]

Hi Mark,
thanks for the quick reply. I'm struggling to reproduce the error myself today, as we have a data restore running overnight.
But I got the above error several times yesterday.

FYI:

• SnowOwl: 7.17.1
• ES: 7.7.0
• MAIN/2020-07-31/SNOMEDCT-UK/2021-03-17/CUSTOM-BRANCH contains is an extension of MAIN/2020-07-31/SNOMEDCT-UK/2021-03-17 (with the named International and UK releases) with custom concept created by us.
• MAIN/2021-01-31/SNOMEDCT-UK/2021-05-12 contains the named International and UK releases, while CUSTOM-BRANCH is a child branch with no additional content where I would like to copy the content of the source branch.
• The common part of the two branches should just be the content in MAIN.

Cheers,
Cristina

[cmark]

Hi @cristinacivili-work,

Thank you for sharing the requested information.

Last question, did you import the UK content on both branches via a Snapshot?
Like when you have created the MAIN/2020-07-31/SNOMEDCT-UK/ branch did you import the official 2021-03-17 UK content via Snapshot and again when you have created the MAIN/2021-01-31/SNOMEDCT-UK branch did you import the official 2021-05-12 UK content via a Snapshot RF2 release?

If yes, I'm afraid you won't be able to move content via the branch merge API. The reason is that Snow Owl uses an append-only time-series-like document model to store revisions, similar to how Git stores its revisions/blobs/etc. Importing the same content to multiple branches treats the content as NEW in both places and will report a conflict. The same thing happens when you add the same file to multiple branches in Git, they look the same, but they are treated a bit differently internally (not sure if this applies to the more recent versions of Git, but previously this was the case).
If I'm not mistaken the entire UK content up until 2021-03-17 lives at two places and when you try to merge one into the other it breaks by creating a too large array (the calculated size number got overflowed thus the NegativeArraySizeException). The exception is actually a bug and it should not happen, we will take care of that, but even with the fix the merge would not complete due to conflicts.

In these cases, the only viable and recommended option to move content between branches is RF2 export-import, which is way faster and easier.
To avoid this scenario and allow content to be "moved" via the Merge REST API without issues, Snow Owl offers a CodeSystem Upgrade API. The upgrade page of the Extension Management section describes a bit how the Upgrade functionality works and how it can be used to follow an Extensions lifecycle and maintain multiple versions of it by replicating the authoring flow and its commits (the release commits at least).
Please note the Upgrade API is in an experimental state and will be finalized in the upcoming 8.0 release.

I hope this helps and as always if you have any questions feel free to raise them here.

Cheers,
Mark