Middleware vs Other Approaches

[jennifer-bowser]

Overview

@jsstevenson brought up a great question about how AnyVar is currently utilizing middleware: is middleware the most appropriate approach for logic that's only needed for a small number of endpoints, given that middleware triggers for every request for every endpoint?

Currently we have 3 middleware functions that are either already in production, or are under active development:

Function	Purpose	Targeted Endpoints
add_creation_timestamp_annotation	Adds a timestamp to a variation indicating the date/time it was added to the database	/variation (though I think it should also target /vrs_variation?)
add_genomic_liftover_annotation	Performs genomic liftover on newly-registered variants between GRCh37 <-> GRCh38 and stores the converted variant as an annotation of the original	/variation /vrs_variation
store_input_payload_annotation	Stores the original input used to create a variant as an annotation of the fully-qualified output variant	/variation /vrs_variation

All of the above only target the same two endpoints used to register variants.

Potential Concerns

• Middleware functions trigger on EVERY request, whether the endpoint is one that your logic intends to target or not. You have to remember to put in checks for each middleware function to ensure the logic only runs for the desired endpoints.

• Each of these middleware functions parse and rebuild the response object when they run, meaning that this parsing/rebuilding occurs 3 times whenever one of the variant registration endpoints is hit. 3 times isn't the end of the world as far as performance goes, but it could become an issue down the line if we continue with this pattern.

Alternative Options

• a) Extract the middleware logic into external functions that can be called independently by each of the endpoints that need them
• b) Condense the logic into a single middleware function that only needs to parse/rebuild the response once
• c) Or: something else? We're open to other ideas!

Discussion Points

1. Do you agree or disagree that the current middleware pattern is a concern that should be addressed?
2. If you are in favor of refactoring our approach, what do you think our alternative strategy should be? Option A? Option B? A mix of both? Something else entirely?

Thanks for your feedback!

[jennifer-bowser]

Personally, I would vote in favor of a mix of options A and B. I think the main logic of each of these middleware functions should be abstracted out into their own utility functions, and then we can create a single middleware function that targets the registration endpoints and runs all 3 of the utility functions.

Abstracting out the core logic makes it easier to test those parts in isolation, and it will make it easier to implement that functionality for the /vcf registration endpoint once we get to tackling that. Additionally, having one central middleware function for both of the (non-VCF) registration endpoints means we don't have to call the utility functions in multiple places.

[jennifer-bowser]

Update: It seems we're all in agreement on a path forward, so I've created issue #189 to address this.