-
Notifications
You must be signed in to change notification settings - Fork 212
Use relative paths for inputs to make the training portable #438
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
adamrtalbot
wants to merge
7
commits into
nextflow-io:master
from
adamrtalbot:use_relative_paths_for_inputs
Closed
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
cc1cec9
Use relative paths for inputs to make the training portable
adamrtalbot 2e9e2d4
Use slightly simpler map statement
adamrtalbot e7957de
Change reference to changing into the absolute path
adamrtalbot e83a38f
Use file method to convert string to path
adamrtalbot 08428f0
Add results_genomics to gitignore
adamrtalbot 603e7ac
Use splitCsv to parse the samplesheet
adamrtalbot 389e560
Merge branch 'master' into use_relative_paths_for_inputs
pinin4fjords File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -8,3 +8,4 @@ work | |
nf-training/results | ||
transcript-index | ||
.vscode | ||
results_genomics |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
@@ -55,7 +55,7 @@ The tools we need (Samtools and GATK) are not installed in the Gitpod environmen | |||||||||||||
!!! note | ||||||||||||||
|
||||||||||||||
Make sure you're in the correct working directory: | ||||||||||||||
`cd /workspace/gitpod/hello-nextflow` | ||||||||||||||
`pwd` should return a path ending in `hello-nextflow` | ||||||||||||||
|
||||||||||||||
### 0.1. Index a BAM input file with Samtools | ||||||||||||||
|
||||||||||||||
|
@@ -562,9 +562,9 @@ This error will not reproduce consistently because it is dependent on some varia | |||||||||||||
This is what the output of the two `.view` calls we added looks like for a failed run: | ||||||||||||||
|
||||||||||||||
```console title="Output" | ||||||||||||||
/workspace/gitpod/hello-nextflow/data/bam/reads_mother.bam | ||||||||||||||
/workspace/gitpod/hello-nextflow/data/bam/reads_father.bam | ||||||||||||||
/workspace/gitpod/hello-nextflow/data/bam/reads_son.bam | ||||||||||||||
./data/bam/reads_mother.bam | ||||||||||||||
./data/bam/reads_father.bam | ||||||||||||||
./data/bam/reads_son.bam | ||||||||||||||
/workspace/gitpod/hello-nextflow/work/9c/53492e3518447b75363e1cd951be4b/reads_father.bam.bai | ||||||||||||||
/workspace/gitpod/hello-nextflow/work/cc/37894fffdf6cc84c3b0b47f9b536b7/reads_son.bam.bai | ||||||||||||||
/workspace/gitpod/hello-nextflow/work/4d/dff681a3d137ba7d9866e3d9307bd0/reads_mother.bam.bai | ||||||||||||||
|
@@ -718,9 +718,9 @@ Here we are going to show you how to do the simple case. | |||||||||||||
We already made a text file listing the input file paths, called `sample_bams.txt`, which you can find in the `data/` directory. | ||||||||||||||
|
||||||||||||||
```txt title="sample_bams.txt" | ||||||||||||||
/workspace/gitpod/hello-nextflow/data/bam/reads_mother.bam | ||||||||||||||
/workspace/gitpod/hello-nextflow/data/bam/reads_father.bam | ||||||||||||||
/workspace/gitpod/hello-nextflow/data/bam/reads_son.bam | ||||||||||||||
/data/bam/reads_mother.bam | ||||||||||||||
/data/bam/reads_father.bam | ||||||||||||||
/data/bam/reads_son.bam | ||||||||||||||
Comment on lines
+721
to
+723
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||
``` | ||||||||||||||
|
||||||||||||||
As you can see, we listed one file path per line, and they are absolute paths. | ||||||||||||||
|
@@ -758,7 +758,7 @@ This way we can continue to be lazy, but the list of files no longer lives in th | |||||||||||||
Currently, our input channel factory treats any files we give it as the data inputs we want to feed to the indexing process. | ||||||||||||||
Since we're now giving it a file that lists input file paths, we need to change its behavior to parse the file and treat the file paths it contains as the data inputs. | ||||||||||||||
|
||||||||||||||
Fortunately we can do that very simply, just by adding the [`.splitText()` operator](https://www.nextflow.io/docs/latest/reference/operator.html#operator-splittext) to the channel construction step. | ||||||||||||||
We are going to use the [`.splitCsv()`](https://www.nextflow.io/docs/latest/operator.html#operator-splitcsv) operator to parse the file into lines, and then use `.map()` to convert each line into a file path object. This introduces some advanced concepts that we'll explain in more detail later in this training series, but for now it's enough to understand that we can manipulate the contents of the samplesheet after we read it in but before we use it. | ||||||||||||||
|
||||||||||||||
_Before:_ | ||||||||||||||
|
||||||||||||||
|
@@ -769,9 +769,11 @@ reads_ch = Channel.fromPath(params.reads_bam) | |||||||||||||
|
||||||||||||||
_After:_ | ||||||||||||||
|
||||||||||||||
```groovy title="hello-genomics.nf" linenums="68" | ||||||||||||||
````groovy title="hello-genomics.nf" linenums="68" | ||||||||||||||
// Create input channel from a text file listing input file paths | ||||||||||||||
reads_ch = Channel.fromPath(params.reads_bam).splitText() | ||||||||||||||
reads_ch = Channel.fromPath(params.reads_bam) | ||||||||||||||
.splitCsv() | ||||||||||||||
.map { bamPath -> file(bamPath[0]) } | ||||||||||||||
``` | ||||||||||||||
|
||||||||||||||
!!! tip | ||||||||||||||
|
@@ -784,7 +786,7 @@ Let's run the workflow one more time. | |||||||||||||
|
||||||||||||||
```bash | ||||||||||||||
nextflow run hello-genomics.nf -resume | ||||||||||||||
``` | ||||||||||||||
```` | ||||||||||||||
|
||||||||||||||
This should produce the same result as before, right? | ||||||||||||||
|
||||||||||||||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
/workspace/gitpod/hello-nextflow/data/bam/reads_mother.bam | ||
/workspace/gitpod/hello-nextflow/data/bam/reads_father.bam | ||
/workspace/gitpod/hello-nextflow/data/bam/reads_son.bam | ||
data/bam/reads_mother.bam | ||
data/bam/reads_father.bam | ||
data/bam/reads_son.bam |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.