Skip to content

main <- dev #793

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .cloudbuild/vep-docker.cloudbuild.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# Run locally with:
#
# gcloud builds submit --quiet --substitutions='_VEP_VERSION=110' --config .cloudbuild/vep-docker.cloudbuild.yaml v03_pipeline/
# gcloud builds submit --quiet --substitutions='_VEP_VERSION=110' --config .cloudbuild/vep-docker.cloudbuild.yaml v03_pipeline/deploy
steps:
- name: 'gcr.io/kaniko-project/executor:v1.3.0'
args:
- --destination=gcr.io/seqr-project/vep-docker-image:${_VEP_VERSION}
- --dockerfile=deploy/Dockerfile.vep
- --dockerfile=Dockerfile.vep
- --cache=true
- --cache-ttl=168h

Expand Down
1 change: 0 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ ignore = [
"FBT", # flake-boolean-trap... disallows boolean args to functions... fixing this code will require refactors.
"ANN", # flake8-annotations is for typed code
"DJ", # django specific
"T20", # forbids print, we print quite a bit
"PYI", # pyi is typing stub files
"PT", # pytest specific
"PTH", # pathlib is preferred, but we're not using it yet
Expand Down
3 changes: 2 additions & 1 deletion requirements.in
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
elasticsearch==7.9.1
google-api-python-client>=1.8.0
hail==0.2.128
hail==0.2.130
luigi>=3.4.0
gnomad==0.6.4
google-cloud-storage>=2.14.0
google-cloud-secret-manager>=2.20.0
32 changes: 27 additions & 5 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -97,10 +97,11 @@ frozenlist==1.4.0
# hail
gnomad==0.6.4
# via -r requirements.in
google-api-core==2.14.0
google-api-core[grpc]==2.14.0
# via
# google-api-python-client
# google-cloud-core
# google-cloud-secret-manager
# google-cloud-storage
google-api-python-client==2.108.0
# via -r requirements.in
Expand All @@ -111,6 +112,7 @@ google-auth==2.23.4
# google-auth-httplib2
# google-auth-oauthlib
# google-cloud-core
# google-cloud-secret-manager
# google-cloud-storage
# hail
google-auth-httplib2==0.1.1
Expand All @@ -119,6 +121,8 @@ google-auth-oauthlib==0.8.0
# via hail
google-cloud-core==2.4.1
# via google-cloud-storage
google-cloud-secret-manager==2.20.0
# via -r requirements.in
google-cloud-storage==2.14.0
# via -r requirements.in
google-crc32c==1.5.0
Expand All @@ -127,9 +131,22 @@ google-crc32c==1.5.0
# google-resumable-media
google-resumable-media==2.7.0
# via google-cloud-storage
googleapis-common-protos==1.61.0
googleapis-common-protos[grpc]==1.61.0
# via
# google-api-core
# grpc-google-iam-v1
# grpcio-status
grpc-google-iam-v1==0.13.0
# via google-cloud-secret-manager
grpcio==1.63.0
# via
# google-api-core
# googleapis-common-protos
# grpc-google-iam-v1
# grpcio-status
grpcio-status==1.48.2
# via google-api-core
hail==0.2.128
hail==0.2.130
# via -r requirements.in
hdbscan==0.8.33
# via gnomad
Expand Down Expand Up @@ -202,7 +219,7 @@ numpy==1.26.2
# scipy
oauthlib==3.2.2
# via requests-oauthlib
orjson==3.9.11
orjson==3.9.10
# via hail
packaging==23.2
# via
Expand All @@ -226,11 +243,16 @@ portalocker==2.8.2
# via msal-extensions
prompt-toolkit==3.0.41
# via ipython
proto-plus==1.23.0
# via google-cloud-secret-manager
protobuf==3.20.2
# via
# google-api-core
# google-cloud-secret-manager
# googleapis-common-protos
# hail
# grpc-google-iam-v1
# grpcio-status
# proto-plus
ptyprocess==0.7.0
# via pexpect
pure-eval==0.2.2
Expand Down
6 changes: 0 additions & 6 deletions v03_pipeline/bin/vep-110-GRCh38.sh
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,6 @@ gcloud storage cp --billing-project $PROJECT gs://seqr-reference-data/vep/110/ve
gcloud storage cp --billing-project $PROJECT gs://seqr-reference-data/vep/110/uORF_5UTR_${ASSEMBLY}_PUBLIC.txt /vep_data/ &

# Raw data files copied from the bucket (https://console.cloud.google.com/storage/browser/dm_alphamissense;tab=objects?prefix=&forceOnObjectsSortingFiltering=false)
# Some investigation led us to want to combine the canonical and non-canonical transcript tsvs (run inside the VEP docker container):
# cat AlphaMissense_hg38.tsv.gz | gunzip | grep -v '#' | awk 'BEGIN { OFS = "\t" };{$6=""; print $0}' > AlphaMissense_combined_hg38.tsv
# cat AlphaMissense_isoforms_hg38.tsv.gz | gunzip | grep -v '#' >> AlphaMissense_combined_hg38.tsv
# cat AlphaMissense_combined_hg38.tsv | sort --parallel=12 --buffer-size=20G -k1,1 -k2,2n > AlphaMissense_combined_sorted_hg38.tsv
# cat AlphaMissense_combined_sorted_hg38.tsv | sed '1i #CHROM\tPOS\tREF\tALT\tgenome\ttranscript_id\tprotein_variant\tam_pathogenicity\tam_class' > AlphaMissense_hg38.tsv
# bgzip AlphaMissense_hg38.tsv
# tabix -s 1 -b 2 -e 2 -f -S 1 AlphaMissense_hg38.tsv.gz
gcloud storage cp --billing-project $PROJECT 'gs://seqr-reference-data/vep/110/AlphaMissense_hg38.tsv.*' /vep_data/ &

Expand Down
93 changes: 0 additions & 93 deletions v03_pipeline/bin/write_cached_reference_dataset_query_ht.py

This file was deleted.

Loading
Loading