Skip to content

Use relative paths for inputs to make the training portable #438

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from
Closed

Use relative paths for inputs to make the training portable #438

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
cc1cec9
Use relative paths for inputs to make the training portable
adamrtalbot Nov 6, 2024
2e9e2d4
Use slightly simpler map statement
adamrtalbot Nov 6, 2024
e7957de
Change reference to changing into the absolute path
adamrtalbot Nov 6, 2024
e83a38f
Use file method to convert string to path
adamrtalbot Nov 6, 2024
08428f0
Add results_genomics to gitignore
adamrtalbot Nov 7, 2024
603e7ac
Use splitCsv to parse the samplesheet
adamrtalbot Nov 7, 2024
389e560
Merge branch 'master' into use_relative_paths_for_inputs
pinin4fjords Nov 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions docs/hello_nextflow/04_hello_genomics.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -768,10 +768,11 @@ reads_ch = Channel.fromPath(params.reads_bam)

_After:_

```groovy title="hello-genomics.nf" linenums="68"
````groovy title="hello-genomics.nf" linenums="68"
// Create input channel from a text file listing input file paths
reads_ch = Channel.fromPath(params.reads_bam).splitText()
```
reads_ch = Channel.fromPath(params.reads_bam)
.splitText()
.map { it.trim() }```

!!! tip

Expand All @@ -783,7 +784,7 @@ Let's run the workflow one more time.

```bash
nextflow run hello-genomics.nf -resume
```
````

This should produce the same result as before, right?

Expand Down
6 changes: 3 additions & 3 deletions hello-nextflow/data/sample_bams.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
/workspace/gitpod/hello-nextflow/data/bam/reads_mother.bam
/workspace/gitpod/hello-nextflow/data/bam/reads_father.bam
/workspace/gitpod/hello-nextflow/data/bam/reads_son.bam
data/bam/reads_mother.bam
data/bam/reads_father.bam
data/bam/reads_son.bam
4 changes: 3 additions & 1 deletion hello-nextflow/hello-config/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,9 @@ process GATK_JOINTGENOTYPING {
workflow {

// Create input channel from a text file listing input file paths
reads_ch = Channel.fromPath(params.reads_bam).splitText()
reads_ch = Channel.fromPath(params.reads_bam)
.splitText()
.map { it.trim() }
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Annoyingly, we need a trim() in here to make it work. I'm not sure why we need this, but this is the simplest fix I can find.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adamrtalbot Can this be replaced by a .splitCsv() . From what I recall it doesn't need the trim.

Also, if we need to keep the it.trim() we are moving away from using the implicit it variable and should instead used named variables in our closures. It's much less confusing to newbies and will become mandatory in a future nextflow version according to Ben.


// Load the file paths for the accessory files (reference and intervals)
ref_file = file(params.reference)
Expand Down
4 changes: 3 additions & 1 deletion hello-nextflow/hello-modules/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,9 @@ process GATK_JOINTGENOTYPING {
workflow {

// Create input channel from a text file listing input file paths
reads_ch = Channel.fromPath(params.reads_bam).splitText()
reads_ch = Channel.fromPath(params.reads_bam)
.splitText()
.map { it.trim() }

// Load the file paths for the accessory files (reference and intervals)
ref_file = file(params.reference)
Expand Down
4 changes: 3 additions & 1 deletion hello-nextflow/hello-nf-test/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ include { GATK_JOINTGENOTYPING } from './modules/local/gatk/jointgenotyping/main
workflow {

// Create input channel from a text file listing input file paths
reads_ch = Channel.fromPath(params.reads_bam).splitText()
reads_ch = Channel.fromPath(params.reads_bam)
.splitText()
.map { it.trim() }

// Load the file paths for the accessory files (reference and intervals)
ref_file = file(params.reference)
Expand Down
4 changes: 3 additions & 1 deletion hello-nextflow/hello-operators.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,9 @@ process GATK_HAPLOTYPECALLER {
workflow {

// Create input channel from a text file listing input file paths
reads_ch = Channel.fromPath(params.reads_bam).splitText()
reads_ch = Channel.fromPath(params.reads_bam)
.splitText()
.map { it.trim() }

// Load the file paths for the accessory files (reference and intervals)
ref_file = file(params.reference)
Expand Down
6 changes: 4 additions & 2 deletions hello-nextflow/solutions/hello-config/final-main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ process SAMTOOLS_INDEX {

output:
tuple path(input_bam), path("${input_bam}.bai")

script:
"""
samtools index '$input_bam'
Expand Down Expand Up @@ -96,7 +96,9 @@ process GATK_JOINTGENOTYPING {
workflow {

// Create input channel from a text file listing input file paths
reads_ch = Channel.fromPath(params.reads_bam).splitText()
reads_ch = Channel.fromPath(params.reads_bam)
.splitText()
.map { it.trim() }

// Load the file paths for the accessory files (reference and intervals)
ref_file = file(params.reference)
Expand Down
6 changes: 4 additions & 2 deletions hello-nextflow/solutions/hello-genomics/hello-genomics-4.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ process GATK_HAPLOTYPECALLER {

output:
path "${input_bam}.vcf" , emit: vcf
path "${input_bam}.vcf.idx" , emit: idx
path "${input_bam}.vcf.idx" , emit: idx

script:
"""
Expand All @@ -67,7 +67,9 @@ process GATK_HAPLOTYPECALLER {
workflow {

// Create input channel from a text file listing input file paths
reads_ch = Channel.fromPath(params.reads_bam).splitText()
reads_ch = Channel.fromPath(params.reads_bam)
.splitText()
.map { it.trim() }

// Load the file paths for the accessory files (reference and intervals)
ref_file = file(params.reference)
Expand Down
4 changes: 3 additions & 1 deletion hello-nextflow/solutions/hello-modules/final-main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ include { GATK_JOINTGENOTYPING } from './modules/local/gatk/jointgenotyping/main
workflow {

// Create input channel from a text file listing input file paths
reads_ch = Channel.fromPath(params.reads_bam).splitText()
reads_ch = Channel.fromPath(params.reads_bam)
.splitText()
.map { it.trim() }

// Load the file paths for the accessory files (reference and intervals)
ref_file = file(params.reference)
Expand Down
6 changes: 4 additions & 2 deletions hello-nextflow/solutions/hello-operators/hello-operators-1.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ process GATK_HAPLOTYPECALLER {

output:
path "${input_bam}.g.vcf" , emit: vcf
path "${input_bam}.g.vcf.idx" , emit: idx
path "${input_bam}.g.vcf.idx" , emit: idx

script:
"""
Expand All @@ -68,7 +68,9 @@ process GATK_HAPLOTYPECALLER {
workflow {

// Create input channel from a text file listing input file paths
reads_ch = Channel.fromPath(params.reads_bam).splitText()
reads_ch = Channel.fromPath(params.reads_bam)
.splitText()
.map { it.trim() }

// Load the file paths for the accessory files (reference and intervals)
ref_file = file(params.reference)
Expand Down
6 changes: 4 additions & 2 deletions hello-nextflow/solutions/hello-operators/hello-operators-2.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ process GATK_HAPLOTYPECALLER {

output:
path "${input_bam}.g.vcf" , emit: vcf
path "${input_bam}.g.vcf.idx" , emit: idx
path "${input_bam}.g.vcf.idx" , emit: idx

script:
"""
Expand Down Expand Up @@ -99,7 +99,9 @@ process GATK_GENOMICSDB {
workflow {

// Create input channel from a text file listing input file paths
reads_ch = Channel.fromPath(params.reads_bam).splitText()
reads_ch = Channel.fromPath(params.reads_bam)
.splitText()
.map { it.trim() }

// Load the file paths for the accessory files (reference and intervals)
ref_file = file(params.reference)
Expand Down
6 changes: 4 additions & 2 deletions hello-nextflow/solutions/hello-operators/hello-operators-3.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ process GATK_HAPLOTYPECALLER {

output:
path "${input_bam}.g.vcf" , emit: vcf
path "${input_bam}.g.vcf.idx" , emit: idx
path "${input_bam}.g.vcf.idx" , emit: idx

script:
"""
Expand Down Expand Up @@ -109,7 +109,9 @@ process GATK_JOINTGENOTYPING {
workflow {

// Create input channel from a text file listing input file paths
reads_ch = Channel.fromPath(params.reads_bam).splitText()
reads_ch = Channel.fromPath(params.reads_bam)
.splitText()
.map { it.trim() }

// Load the file paths for the accessory files (reference and intervals)
ref_file = file(params.reference)
Expand Down
Loading