feat: Conformance Manager

[JNygaard-Skylight]

PULL REQUEST

Summary

This PR was my attempt to better utilize the conformance data available in the eCR FHIR package without having to implement a full FHIR server or download and run a bunch of scripts manually. It does this with a new project Dibbs.FhirResolver that handles interacting with the conformance data. Currently it attempts to do 2 things:

1. Get the URI for code systems from their OIDs, replacing the _SystemReference.liquid template.
2. Get the display for a given code and code system, replacing the GetLoincName, GetRxnormName filters in CustomFilters.cs.

This PR should be considered to still be in a rough first draft state as I was hoping to implement as much as possible before I had to leave it.

Current Changes

Code System URIs

Old State

In the current version of the converter code systems are turned into human-readable URIs with the _SystemReference.liquid template. This template is effectively just a large switch statement, where each case is a possible system OID. We would have to manually add every possible OID to this switch statement.

New Changes

The awful switch statement is replaced with a dictionary created on the initialization of the Dibbs.FhirResolver: _oidUriMapping. On initialization of ConformanceService every code system and naming system resource is read and their OID and URI are added to _oidUriMapping. This can then be accessed through GetCodeSystemUri.

Code Displays

Old State

The _Coding.liquid template will populate "display" in the following order:

1. First it will look for the display attribute in the XML
2. If that is null it will then look up the code in Loinc.csv and rxnorm.csv using the filters GetLoincName and GetRxnormName.

New Changes

ConformanceService will now construct a dictionary of all codes with a display value in valueset resources. While this will cover a lot of relevant codes, especially from HL7 valuesets, there will still be a lot of Loinc, Snomed, and RxNorm codes not included in the eCR package. Therefore I have written scripts to update the currently used CSVs, and moved that logic into Dibbs.FhirResolver. In addition I have added a CSV for Snomed.

What is left for this PR

Bugs/Issues

In no particular order:

• To initialize Dibbs.FhirResolver it requires an internet connection to retrieve the eCR FHIR package. To keep the converter usable in an offline state the _resolver should be created using ZipSource, or DirectorySource. this would require that the package is downloaded either manually or as part of the build process.
• The code to initialize a resolver currently in src/Microsoft.Health.Fhir.Liquid.Converter.FunctionalTests/BaseConvertDataFunctionalTests.ValidateConvertCCDAMessageIsValidFHIR should likely be moved into Dibbs.FhirResolver to reduce duplication of logic.
• I'm not very happy with the name of the new project. I first was using FhirResolver, however I then moved to ConformanceManager as I think that is a more general term. But both "resolver" and "conformance" have specific meanings in the context of the Firely SDK, and in the FHIR ecosystem so neither of them may be appropriate.
• A lot of the code in the initialization of the ConformanceService is async, but I am currently not taking advantage of that to keep things simple if development.
• ConformanceService.GetCodeSystemUri slaps an "urn:oid:" even when it is not an OID. I think it should be renamed to GetCodeSystem and the logic in GeneralFilters.PrependID moved into it. That way it becomes a general purpose function that replaces OIDs with URIs, appends urn:... when needed, and returns the given string if it is not recognized as an OID, UUID, or URI.
• Additional code displays can be found in code system resources. MakeCodeLibrary could be expanded to look in them. Downloading additional packages may also be helpful.
• I'm not happy with the porting of the old CSVs into the Dibbs.FhirResolver, and using scripts to generate the CSVs I would want to find a more integrated solution to reduce that manual overhead. But perhaps this is a good first step.
• The scripts for generating the CSVs are in this branch and need to be moved over.
• The code around handling missing information in the _Identifier.liquid template appears to be broken. I believe it's because of the change from using _SystemReference.liquid to GetSystemUri.
• The conformance manager requires initialization to run. To simplicity I have added that function to all tests that need it but it is very slow and should be ran in an initialization function before all tests.

Checklist

• ⚠️ Create an associated dibbs-ecr-viewer PR & checked that things work on the front-end.
• If necessary, update any test fixtures/bundles to reflect FHIR conversion changes (in this repo and/or dibbs-ecr-viewer)
• If this code affects the other scrum team, have they been notified? (In Slack, as reviewers, etc.)

⚠️ Do not merge this PR until the associated dibbs-ecr-viewer PR is created and validated. When both have been approved:

1. Merge the FHIR converter PR
2. Cut a new release of dibbs-fhir-converter
3. Update the fhir-converter Dockerfile in dibbs-ecr-viewer with the updated release branch number.