Support automated submission to ClinVar

[ahwagner]

We have completed initial testing of the GKS-compliant clinvar-this submission utility and it's looking good; there are a few final changes coming in VA-Spec in preparation for the v1 release (in May), at which point we will want to connect this service with a CIViC background (still sidekiq?) job for submitting SC-VCEP assertions.

Here is a draft illustration of what @korikuzma and I are thinking:

We should discuss what this looks like during an upcoming CIViC call.

[susannasiebert]

This will have to be updated to take into account our upcoming endorsement feature (#1213). Only assertions that have been endorsed should be submitted to ClinVar.

There are three states that an endorsement can be in that may impact what should be done with the ClinVar submission:

• Active: original endorsement status. Can be submitted to ClinVar.
• Requires Review: assertion has changes since the original/last endorsement. These could be: accepted revisions on the assertion, accepted revisions to any of the underlying evidence, flags of the assertion, flags of the underlying evidence, rejection of any of the underlying evidence. After reviewing these changes, endorsing users are allowed to re-endorse the assertion. At this point we could consider updating the ClinVar record, if possible.
• Revoked: an endorsing user revoked an endorsement for whatever reason. The ClinVar submission should be removed/deprecated.

The following status changes are possible for an existing endorsement:

• active -> requires review: automatically on any of the described changes
• requires review -> active: action done by endorsing user
• requires review -> revoked: action done by endorsing user
• active -> revoked: action done by endorsing user
• revoked -> active: action done by endorsing user

An endorsement is made on behalf of a specific organization. These do not need to be limited to SC-VCEP's. However, organizations will now also have a flag of "is_approved_vcep" that can be used to scope assertions/endorsements that we want to submit to ClinVar.

[ahwagner]

Steps needed to be addressed (ideally by May 6):

• Formalize rules for endorsed (SC-VCEP) vs. community assertions
• Add support for SC-VCEP specific API keys
• Track submission status / record in backend
• Identify strategy for handling batch submission failures

Other related steps (lower priority):

• What information for GKS model should be stored on backend vs. on-the-fly transform
• Display link to ClinVar record on frontend (when available)

[ahwagner]

Notes from 2025-04-24 meeting:

"Should Assertions that come out of Task Forces be set aside for future SC-VCEP submission, or should we submit all Assertions as 1-star, and again as 3-star once a SC-VCEP endorses a record?"

Assertions can be endorsed by more than one organization. Each such endorsement is effectively a ClinVar SCV. Endorsements from SC-VCEPs would provide 3-star support at the VCV level, multiple agreeing endorsements by different orgs would be analogous to 2-star support at the VCV level.

"Should all Assertions be endorsed before submitting to ClinVar?"

Yes, one pipeline requiring endorsements. No separate pipeline for non-endorsed (accepted-only)

"Should any user be able to endorse any assertion on behalf of any organization they are members of?"

Only an approved user for an organization may endorse on behalf of the organization.

"Can any organization endorse an assertion?"

Only approved organizations can endorse

Our focus between now and May 6th is to get endorsements live on CIViC staging, with the target of creating and submitting an assertion batch from the CIViC database to the ClinVar test API via clinvar-this