You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: Metagenomics/Illumina/Workflow_Documentation/NF_MGIllumina/README.md
+15-14Lines changed: 15 additions & 14 deletions
Original file line number
Diff line number
Diff line change
@@ -3,12 +3,12 @@
3
3
## General Workflow Info
4
4
5
5
### Implementation Tools
6
+
6
7
The current GeneLab Illumina metagenomics sequencing data processing pipeline (MGIllumina), [GL-DPPD-7107-A.md](../../Pipeline_GL-DPPD-7107_Versions/GL-DPPD-7107-A.md), is implemented as a [Nextflow](https://nextflow.io/) DSL2 workflow and utilizes [Singularity](https://docs.sylabs.io/guides/3.10/user-guide/introduction.html) containers or [conda](https://docs.conda.io/en/latest/) environments to install/run all tools. This workflow is run using the command line interface (CLI) of any unix-based system. While knowledge of creating workflows in nextflow is not required to run the workflow as is, [the Nextflow documentation](https://nextflow.io/docs/latest/index.html) is a useful resource for users who want to modify and/or extend this workflow.
7
8
8
9
> **Note on reference databases**
9
10
> Many reference databases are relied upon throughout this workflow. They will be installed and setup automatically the first time the workflow is run. All together, after installed and unpacked, they will take up about about 340 GB of storage, but they may also require up to 500GB during installation and initial un-packing, so be sure there is enough room on your system before running the workflow.
10
11
11
-
12
12
## Utilizing the Workflow
13
13
14
14
1.[Install Nextflow and Singularity](#1-install-nextflow-and-singularity)
@@ -19,21 +19,21 @@ The current GeneLab Illumina metagenomics sequencing data processing pipeline (M
3a. [Approach 1: Run slurm jobs in singularity containers with OSD accession as input](#3a-approach-1-run-slurm-jobs-in-singularity-containers-with-osd-accession-as-input)
24
-
3b. [Approach 2: Run slurm jobs in singularity containers with a csv file as input](#3b-approach-2-run-slurm-jobs-in-singularity-containers-with-a-csv-file-as-input)
25
-
3c. [Approach 3: Run jobs locally in conda environments and specify the path to one or more existing conda environments](#3c-approach-run-jobs-locally-in-conda-environments-and-specify-the-path-to-one-or-more-existing-conda-environments)
26
-
3d. [Modify parameters and cpu resources in the nextflow config file](#3d-modify-parameters-and-cpu-resources-in-the-nextflow-config-file)
22
+
4.[Run the workflow](#4-run-the-workflow)
23
+
3a. [Approach 1: Run slurm jobs in singularity containers with OSD accession as input](#4a-approach-1-run-slurm-jobs-in-singularity-containers-with-osd-accession-as-input)
24
+
3b. [Approach 2: Run slurm jobs in singularity containers with a csv file as input](#4b-approach-2-run-slurm-jobs-in-singularity-containers-with-a-csv-file-as-input)
25
+
3c. [Approach 3: Run jobs locally in conda environments and specify the path to one or more existing conda environments](#4c-approach-run-jobs-locally-in-conda-environments-and-specify-the-path-to-one-or-more-existing-conda-environments)
26
+
3d. [Modify parameters and cpu resources in the nextflow config file](#4d-modify-parameters-and-cpu-resources-in-the-nextflow-config-file)
27
27
28
-
4.[Workflow outputs](#4-workflow-outputs)
29
-
4a. [Main outputs](#4a-main-outputs)
30
-
4b. [Resource logs](#4b-resource-logs)
28
+
5.[Workflow outputs](#5-workflow-outputs)
29
+
4a. [Main outputs](#5a-main-outputs)
30
+
4b. [Resource logs](#5b-resource-logs)
31
31
32
32
<br>
33
33
34
34
---
35
35
36
-
### 1. Install Nextflow and Singularity
36
+
### 1. Install Nextflow and Singularity
37
37
38
38
#### 1a. Install Nextflow
39
39
@@ -70,6 +70,7 @@ All files required for utilizing the NF_XXX GeneLab workflow for processing meta
@@ -79,13 +80,13 @@ unzip NF_MGIllumina.zip && cd NF_XXX-X_X.X.X
79
80
Although Nextflow can fetch Singularity images from a url, doing so may cause issues as detailed [here](https://github.com/nextflow-io/nextflow/issues/1210).
80
81
81
82
To avoid this issue, run the following command to fetch the Singularity images prior to running the NF_MGIllumina workflow:
83
+
82
84
> Note: This command should be run in the location containing the `NF_MGIllumina` directory that was downloaded in [step 2](#2-download-the-workflow-files) above.
83
85
84
86
```bash
85
87
bash ./bin/prepull_singularity.sh nextflow.config
86
88
```
87
89
88
-
89
90
Once complete, a `singularity` folder containing the Singularity images will be created. Run the following command to export this folder as a Nextflow configuration environment variable to ensure Nextflow can locate the fetched images:
* `-run main.nf` - Instructs nextflow to run the NF_XXX workflow
139
+
138
140
* `-resume` - Resumes workflow execution using previously cached results
139
-
* `-profile` – Specifies the configuration profile(s) to load, `singularity` instructs nextflow to setup and use singularity for all software called in the workflow
140
-
141
141
142
+
* `-profile` – Specifies the configuration profile(s) to load, `singularity` instructs nextflow to setup and use singularity for all software called in the workflow
142
143
*Required only if you would like to pull and process data directly from OSDR*
143
144
144
145
* `--GLDS_accession` – A Genelab / OSD accession number e.g. OSD-574.
145
146
146
147
*Required only if --GLDS_accession is not passed as an argument*
147
148
148
-
* `--csv_file` – A 3-column (single-end) or 4-column (paired-end) input csv file (sample_id, forward, [reverse,] paired). Please see the sample [SE_file.csv](workflow_code/SE_file.csv)and [PE_file.csv]((workflow_code/PE_file.csv) in this repository for examples on how to format this file.
149
+
* `--csv_file` – A 3-column (single-end) or 4-column (paired-end) input csv file (sample_id, forward, [reverse,] paired). Please see the sample [SE_file.csv](workflow_code/SE_file.csv)and [PE_file.csv](workflow_code/PE_file.csv) in this repository for examples on how to format this file.
149
150
150
151
> See `nextflow run -h` and [Nextflow's CLI run command documentation](https://nextflow.io/docs/latest/cli.html#run) for more options and details on how to run nextflow.
0 commit comments