feat!: add --select-by-* options to upload-notes, remove downloading #442
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
☂️ Python Coverage
Overall Coverage
New FilesNo new covered files... Modified Files
|
mikix
force-pushed
the
mikix/upload-select
branch
from
63152dc to
a157398
Compare
This adds the --select-by-word, -regex, -athena-table, -csv, and -anon-csv options to upload-notes, deprecating the previous --docrefs and --anon-docrefs options (which now map to by-csv and by-anon-csv). This also removes the rarely used "download notes from your EHR on the fly" feature which meant you didn't need local copies of the notes. It could even do a bulk export if you didn't provide a --docrefs flag. It is too hard to square with the new set of options and is a lot of complexity for a feature we don't really use and don't really recommend to sites to use (you should be archiving your notes for easy access anyway). Downloading the wrapped clinical notes referred to by your NDJSON is still supported! This is just removing the ability to go direct to your EHR for the DocRefs and DxReports themselves.
mikix
force-pushed
the
mikix/upload-select
branch
from
a157398 to
502363d
Compare
dogversioning
approved these changes
Sep 2, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds the --select-by-word, -regex, -athena-table, -csv, and -anon-csv options to upload-notes, deprecating the previous --docrefs and --anon-docrefs options (which now map to by-csv and by-anon-csv). This new set of options matches the options used to select notes for NLP - so now the same exact option can be used at the other side of the pipeline when uploading to label studio.
This also removes the rarely used "download notes from your EHR on the fly" feature which meant you didn't need local copies of the notes. It could even do a bulk export if you didn't provide a --docrefs flag. It is too hard to square with the new set of options and is a lot of complexity for a feature we don't really use and don't really recommend to sites to use (you should be archiving your notes for easy access anyway).
(ed note: this was a feature I used early on, when we didn't have as healthy an archiving habit and not all the documents were even extracted yet from Cerner. But we recommend exporting via smart-fetch these days and then just holding on to the extracted files. So on-the-fly exporting is kind of an anti-pattern nowadays.)
Downloading the wrapped clinical notes referred to by your NDJSON is still supported! This is just removing the ability to go direct to your EHR for the DocRefs and DxReports themselves.
Checklist
docs/) needs to be updated