add sample_type to family and project table paths #842
Conversation
| ) | ||
| if hfs.exists(project_ht_path): |
There was a problem hiding this comment.
@bpblanken I think what was happening before was that HailTableTask was actually never complete() if the project table didn't exist and that's what was causing the infinite loop?
yielding inside the sample_type loop works now if the project table exists.
There was a problem hiding this comment.
🤔 definitely strange behavior! I think we do actually want this task to fail though if the project able isn't there right?
There was a problem hiding this comment.
I don't think so, all of the existing seqr projects will have project tables in only one sample_type directory.
There was a problem hiding this comment.
TODO: raise error if there is not at least 1 project table! else the task will never complete!
jklugherz
changed the title
| family_guid=family_guid, | ||
| ), | ||
| ) | ||
| for sample_type in SampleType: |
There was a problem hiding this comment.
potentially out of scope here, but I think we would actually want to be explicit about the sample type (or multiple sample types) we are deleting, so potentially we could delete one but not the other
There was a problem hiding this comment.
talked yesterday, this was intentional so that we won't have to remember to run the dag twice for both sample types
| @@ -21,21 +23,24 @@ def complete(self) -> bool: | |||
| ) | |||
|
|
|||
| def run(self): | |||
| project_table_task: luigi.Target = yield HailTableTask( | |||
| project_table_path( | |||
| for sample_type in SampleType: | |||
There was a problem hiding this comment.
same as above, I think ideally we would want to explicitly set which sample type(s) we are deleting
|
TODO: change path from v03 to 3.1, but keep reference data the same |
| @@ -17,6 +17,7 @@ class Sex(Enum): | |||
| class PipelineVersion(Enum): | |||
| V02 = 'v02' | |||
| V03 = 'v03' | |||
| V3_1 = 'v3.1' | |||
There was a problem hiding this comment.
please lmk if you have a better idea
the naming curse
This will require a one-time migration to move all existing family tables into their sampletype subfolder. This won't be too bad as sample_type is already in the table globals.