Skip to content

New multi-course structure for training materials #486

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 69 commits into from
Feb 18, 2025
Merged
Show file tree
Hide file tree
Changes from 59 commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
877e069
New multi-course structure for training materials
vdauwera Dec 16, 2024
c016370
Prettier
vdauwera Dec 16, 2024
7a9f351
Prune the nf-core/experimental courses from the top nav
vdauwera Dec 16, 2024
1908183
fix dash to underscore
vdauwera Dec 16, 2024
fda657c
Breaking down Hello World into smaller pieces
vdauwera Dec 23, 2024
c531f4c
Reorganized components of Hello World across Hello Nextflow
vdauwera Dec 24, 2024
5fd4852
Update the navs
vdauwera Jan 2, 2025
7a235c5
Updated the training portal index page to describe the new world order
vdauwera Jan 3, 2025
be194b5
make the buttons the same
vdauwera Jan 3, 2025
fb73251
Completed update of Hello World
vdauwera Jan 8, 2025
5db1e7a
Completed new Hello Channels (based on parts of old Hello World)
vdauwera Jan 9, 2025
4386ad4
Completed first pass of new Hello Plumbing module
vdauwera Jan 9, 2025
463eb12
minor fixes
vdauwera Jan 9, 2025
0ff8e97
Minor tweaks
vdauwera Jan 9, 2025
6e4d7de
Small improvements to Hello Plumbing
vdauwera Jan 10, 2025
0bdb899
Updated streamlined Hello Modules
vdauwera Jan 15, 2025
305e633
Minor fixes
vdauwera Jan 15, 2025
32773f9
Updated streamlined Hello Containers
vdauwera Jan 15, 2025
0ccab65
Apply suggestions from Jon's code review
vdauwera Jan 15, 2025
90422b5
Apply more suggestions from Jon's code review
vdauwera Jan 15, 2025
b97b832
Streamlined Hello Config
vdauwera Jan 18, 2025
015e57b
Moved intro to variable inputs and params back to Hello World
vdauwera Jan 19, 2025
4aa6798
Clean up parameters json bit in Hello Config and a few others
vdauwera Jan 20, 2025
1c60753
add demo-params.json back
vdauwera Jan 20, 2025
ca6794d
Minor updates based on discussion with Rob
vdauwera Jan 21, 2025
781ac39
Minor tweaks
vdauwera Jan 21, 2025
fdba39b
Move executor to after the process directives and
robsyme Jan 21, 2025
ba7f360
Merge remote-tracking branch 'refs/remotes/origin/gvda-streamline-hel…
robsyme Jan 21, 2025
89d5d8a
Test
robsyme Jan 21, 2025
a0739fb
Hello World COMPLETE
vdauwera Jan 21, 2025
745f709
Merge branch 'gvda-streamline-hello-nextflow' of https://github.com/n…
vdauwera Jan 21, 2025
b20c55c
Hello Channels COMPLETE
vdauwera Jan 21, 2025
96db041
Hello Workflow COMPLETE
vdauwera Jan 21, 2025
6507980
Hello Modules COMPLETE
vdauwera Jan 21, 2025
51f3010
Hello Containers COMPLETE
vdauwera Jan 21, 2025
1a2f327
Fix for cowsay issue -> switch to cowpy
vdauwera Jan 21, 2025
24c0234
Hello Config COMPLETE and fixed
vdauwera Jan 21, 2025
92c97ea
Minor fixes
vdauwera Jan 27, 2025
ea6f3f8
Setup and orientation updates
vdauwera Feb 3, 2025
e8b97ae
More orientation tweaks
vdauwera Feb 3, 2025
8eb6291
Update NF4 Genomics course description and main filenames
vdauwera Feb 5, 2025
9b7211d
Big wave of cleanup
vdauwera Feb 13, 2025
72664ee
Fix prettier issues
vdauwera Feb 13, 2025
534e97b
Merge branch 'master' into gvda-streamline-hello-nextflow
vdauwera Feb 13, 2025
0b0351a
Catching up to master
vdauwera Feb 14, 2025
91a043e
Prettier fixes ?
vdauwera Feb 14, 2025
3372961
Last prettier fixes I swear
vdauwera Feb 14, 2025
af528ba
Streamline the course list some more
vdauwera Feb 14, 2025
b9eaa38
Slimmed down the training portal index some more
vdauwera Feb 14, 2025
eac8587
Minor improvements to index and orientation pages
vdauwera Feb 14, 2025
783d0f9
Minor reorg of the scientific use cases
vdauwera Feb 14, 2025
9b49d72
Adding an index page for NF4 Science courses
vdauwera Feb 14, 2025
cd28242
Update Genomics materials to be standalone
vdauwera Feb 14, 2025
b2daf19
Take genomics parts 3 and 4 off the portal for now
vdauwera Feb 14, 2025
a2dcde4
fix svg image paths
vdauwera Feb 14, 2025
d32461f
Fix typos
mribeirodantas Feb 15, 2025
91a2b5e
Apply suggestions from Marcel's code review
vdauwera Feb 17, 2025
895eeaf
Minor fixes
vdauwera Feb 17, 2025
7b3afa3
No nav tabs (#502)
ewels Feb 17, 2025
17dfc00
Goodbye GitPod, hello GitHub Codespaces.
ewels Feb 17, 2025
6046bfb
Found a few more mentions
ewels Feb 17, 2025
5c43322
Two more
ewels Feb 17, 2025
5e75e01
Merge branch 'master' into gvda-streamline-hello-nextflow
vdauwera Feb 18, 2025
04f5db4
Apply suggestions from Marcel's code review
vdauwera Feb 18, 2025
06769cd
Merge branch 'gvda-streamline-hello-nextflow' into codespaces
ewels Feb 18, 2025
9ebd7a4
Tweak language around channel creation
vdauwera Feb 18, 2025
976881c
Merge branch 'gvda-streamline-hello-nextflow' of https://github.com/n…
vdauwera Feb 18, 2025
e1a7b27
Apply suggestions from Marcel's code review
vdauwera Feb 18, 2025
c99ec6f
Merge branch 'codespaces' into gvda-streamline-hello-nextflow
vdauwera Feb 18, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ Please see the [official docs](https://squidfunk.github.io/mkdocs-material/refer

- Note that we have two custom admonitions: `exercise` and `result` (alias `solution`).
- `!!!` does a regular admonition, `???` makes it collapsed (click to expand).
- Intendation is important! Make sure you check the rendered site, as it's easy to make a mistake.
- Indentation is important! Make sure you check the rendered site, as it's easy to make a mistake.

## Known limitations

Expand Down
12 changes: 8 additions & 4 deletions docs/advanced/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@

Welcome to our Nextflow workshop for intermediate and advanced users!

In this workshop, we will explore the advanced features of the Nextflow language and runtime, and learn how to use them to write efficient and scalable data-intensive workflows. We will cover topics such as parallel execution, error handling, and workflow customization.
In this workshop, we will explore the advanced features of the Nextflow language and runtime, and learn how to use them to write efficient and scalable data-intensive workflows.
We will cover topics such as parallel execution, error handling, and workflow customization.

Please note that this is not an introductory workshop, and we will assume some basic familiarity with Nextflow.
Please note that this is not an introductory workshop, and we will assume extensive familiarity with Nextflow.

By the end of this workshop, you will have the skills and knowledge to create complex and powerful Nextflow pipelines for your own data analysis projects.

Expand Down Expand Up @@ -34,9 +35,12 @@ Please note that this is **not** a beginner's workshop and familiarity with Next
- Familiarity with Nextflow and Groovy
- An understanding of common file formats

## Follow the training video
## Follow the training videos and get help

We run a free online training event for this course approximately every six months. Videos are streamed to YouTube and questions are handled in the nf-core Slack community. You can watch the recording of the most recent training ([September, 2023](https://nf-co.re/events/2023/training-sept-2023/)) below:
Video recordings are available for this course.
You can ask questions in the [Seqera community forum](https://community.seqera.io/).

You can watch the recording of the most recent training ([September, 2023](https://nf-co.re/events/2023/training-sept-2023/)) below:

<div style="text-align: center;">
<iframe width="560" height="315" src="https://www.youtube.com/embed/nPAH9owvKvI?si=Kt3WmxF7rGhRp2L1" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen="" data-ruffle-polyfilled=""></iframe>
Expand Down
2 changes: 1 addition & 1 deletion docs/basic_training/cache_and_resume.md
Original file line number Diff line number Diff line change
Expand Up @@ -349,7 +349,7 @@ So D is matched with 'a' here, which was not the intention. That order will like

??? solution

You should see that while FOO and BAR reliably re-use their cache, FOOBAR will re-run at least a subset of its tasks due to differences in the combinations of inputs it recieves.
You should see that while FOO and BAR reliably re-use their cache, FOOBAR will re-run at least a subset of its tasks due to differences in the combinations of inputs it receives.

The output will look like this:

Expand Down
15 changes: 10 additions & 5 deletions docs/basic_training/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Fundamentals Training

You are now on the path to writing reproducible and scalable scientific workflows using Nextflow. This guide complements the full [Nextflow documentation](https://www.nextflow.io/docs/latest) - if you ever have any doubts, please refer to that.
You are now on the path to writing reproducible and scalable scientific workflows using Nextflow.
This guide complements the full [Nextflow documentation](https://www.nextflow.io/docs/latest) - if you ever have any doubts, please refer to that.

Let's get started!

Expand All @@ -18,17 +19,21 @@ By the end of this course you should:

## Audience & prerequisites

Please note that this is **not** a beginner's workshop and familiarity with Nextflow, the command line, and common file formats is assumed.
Please note that this is **not** a beginner's workshop.
Familiarity with Nextflow, the command line, and common file formats is assumed.

For a beginner's introduction to Nextflow, please see the [Hello Nextflow](../hello_nextflow/) course.

**Prerequisites**

- A GitHub account
- Experience with command line
- An understanding of common file formats

## Follow the training videos
## Follow the training videos and get help

Free online training events for this course are run approximately every six months. Videos are streamed to YouTube and questions are handled in the nf-core Slack community.
Video recordings are available for this course.
You can ask questions in the [Seqera community forum](https://community.seqera.io/).

You can watch the recording of the most recent training ([March, 2024](https://nf-co.re/events/2024/training-foundational-march)) in the [YouTube playlist](https://youtu.be/dbOKB3VRpuE?si=MYBy4-gjRfEYkVRM) below:

Expand All @@ -38,7 +43,7 @@ You can watch the recording of the most recent training ([March, 2024](https://n

!!! warning

Please note that the training material is updated regularly and that the videos may be out of date.
Please note that the training material is updated regularly and that the videos linked below may be out of date.

If English is not your preferred language, you may find it useful to follow the training from the [March 2023 event](https://nf-co.re/events/2023/training-march-2023), which is available in multiple languages.

Expand Down
36 changes: 20 additions & 16 deletions docs/envsetup/01_setup.md
Original file line number Diff line number Diff line change
@@ -1,51 +1,55 @@
# Gitpod

Gitpod is a cloud development environment for teams to efficiently and securely develop software. It can improve your developer experience by coding in a cloud development environment.
Gitpod is a cloud-based development environment for teams to efficiently and securely develop software.
We use it to provide a consistent training environment for everyone.

## Creating a Gitpod account

You can create a free [Gitpod](https://gitpod.io/) account using your GitLab, GitHub, or Bitbucket account.
You can create a free [Gitpod](https://gitpod.io/) account from the [Gitpod login page](https://gitpod.io/login/).

You can create an account using the [Gitpod login page](https://gitpod.io/login/).
You will be prompted to choose between 'Gitpod Flex' and 'Gitpod Classic'.
Select 'Gitpod Classic' and click 'Continue'.

![Gitpod log in](img/login.png)

It is best to connect your LinkedIn account to receive a full 50 hours usage allocation.
![Select 'Gitpod Classic'](img/select_gitpod_classic.png)

![Gitpod log in one step](img/onestepaway.png)
Next, log in using your GitHub account.

After selecting your preferred editor, theme, and profile details, click continue and your account will be created and ready to use.
![Gitpod log in](img/login.png)

!!! note
You may need to fill out an additional form or two.
When prompted to connect a LinkedIn account, we recommend doing so if you have one, to receive the extra 50 hours usage allocation.
Don't worry too much if you don't have one; the basic allocation is more than enough to work through the introductory training course.

It is recommended to use the VS code editor.
If you are prompted to select your preferred editor, we strongly recommend choosing the VSCode editor, as that is what we use for Nextflow development in general and for trainings in particular.

## Running Gitpod

Click the following URL to run Gitpod: <https://gitpod.io/#https://github.com/nextflow-io/training>
Once you are logged in to Gitpod, open this link in your browser to open the training environment: <https://gitpod.io/#https://github.com/nextflow-io/training>

This URL is the Nextflow training repository prefixed with `https://gitpod.io/#`.
This URL is the address to the Nextflow training repository prefixed with `https://gitpod.io/#`.

Alternatively, you can click on the button below.
Alternatively, you can click on the button shown below from the many pages in the training portal where it is displayed.

[![Open Gitpod](https://img.shields.io/badge/Gitpod-%20Open%20in%20Gitpod-908a85?logo=gitpod)](https://gitpod.io/#https://github.com/nextflow-io/training)

If you are already logged in, your Gitpod environment will start to load.

### Explore your Gitpod IDE

After Gitpod has loaded, you should see something similar to the following:
After Gitpod has loaded, you should see something similar to the following (which may in light mode depending on your account preferences):

![Gitpod welcome](img/gitpod.welcome.png)

This is the interface of the VSCode IDE, a popular code development application that we recommend using for Nextflow development.

- **The sidebar** allows you to customize your Gitpod environment and perform basic tasks (copy, paste, open files, search, git, etc.). You can click the explorer button to see which files are in this repository.
- **The terminal** allows you to run all the programs in the repository. For example, both `nextflow` and `docker` are installed and can be executed.
- **The file explorer** allows you to view and edit files. Clicking on a file in the explorer will open it within the main window.
- **The Simple Browser** lets you view the nf-training material browser (<https://training.nextflow.io/>). If you close it by accident, you can load the simple browser again by typing the following in the terminal: `gp preview https://training.nextflow.io`.
- **The Simple Browser** lets you view the training instructions in a web browser (<https://training.nextflow.io/>). If you close it by accident, you can load the simple browser again by typing the following in the terminal: `gp preview https://training.nextflow.io`.

### Gitpod resources

Gitpod gives you 500 free credits per month, which is equivalent to 50 hours of free environment runtime using the standard workspace (up to 4 cores, 8 GB RAM, and 30 GB storage).
Gitpod gives you up to 500 free credits per month, which is equivalent to 50 hours of free environment runtime using the standard workspace (up to 4 cores, 8 GB RAM, and 30 GB storage).

There is also a large workspace option that gives you up to 8 cores, 16GB RAM, and 50GB storage. However, the large workspace will use your free credits quicker and you will have fewer hours of access to this space.

Expand Down
6 changes: 3 additions & 3 deletions docs/envsetup/02_local.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Local installation

If you **can not** access Gitpod you can also install everything locally.
If you **cannot** use Gitpod for any reason, you have the option of installing everything locally instead.

Some requirements may be different depending on your local machine.

Expand Down Expand Up @@ -59,7 +59,7 @@ To download the material, execute this command:
git clone https://github.com/nextflow-io/training.git
```

Then `cd` into the `nf-training` directory.
Then `cd` into the relevant directory. By default, that is `hello-nextflow`.

## Checking your installation

Expand All @@ -79,7 +79,7 @@ This should print the current version, system, and runtime.
nextflow info
```

This should come up with the Nextflow version and runtime information:
This should come up with the Nextflow version and runtime information (actual versions may differ):

```console
Version: 23.10.1 build 5891
Expand Down
Binary file modified docs/envsetup/img/gitpod.welcome.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/envsetup/img/login.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs/envsetup/img/onestepaway.png
Binary file not shown.
Binary file added docs/envsetup/img/select_gitpod_classic.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# Orientation

The Gitpod environment contains all the software, code and data necessary to work through this training course, so you don't need to install anything yourself.
The training environment contains all the software, code and data necessary to work through this training course, so you don't need to install anything yourself.
However, you do need a (free) account to log in, and you should take a few minutes to familiarize yourself with the interface.

If you have not yet done so, please follow [this link](../../envsetup/) before going any further.

## Materials provided

Throughout this training course, we'll be working in the `hello-nextflow/` directory, which loads by default when you open the Gitpod workspace.
Throughout this training course, we'll be working in the `hello-nextflow/` directory, which loads by default when you open the training workspace.
This directory contains all the code files, test data and accessory files you will need.

Feel free to explore the contents of this directory; the easiest way to do so is to use the file explorer on the left-hand side of the Gitpod workspace.
Feel free to explore the contents of this directory; the easiest way to do so is to use the file explorer on the left-hand side of the training workspace.
Alternatively, you can use the `tree` command.
Throughout the course, we use the output of `tree` to represent directory structure and contents in a readable form, sometimes with minor modifications for clarity.

Expand All @@ -24,64 +24,36 @@ If you run this inside `hello-nextflow`, you should see the following output:

```console title="Directory contents"
.
├── containers
│ ├── build
│ ├── data
│ ├── results
│ └── scripts
├── data
│ ├── bam
│ ├── greetings.csv
│ ├── ref
│ ├── sample_bams.txt
│ └── samplesheet.csv
├── hello-config
│ ├── demo-params.json
│ ├── main.nf
│ └── nextflow.config
├── greetings.csv
├── hello-channels.nf
├── hello-config.nf
├── hello-containers.nf
├── hello-genomics.nf
├── hello-modules
│ ├── demo-params.json
│ ├── main.nf
│ └── nextflow.config
├── hello-nf-core
│ ├── data
│ └── solution
├── hello-nf-test
│ ├── demo-params.json
│ ├── main.nf
│ ├── modules
│ └── nextflow.config
├── hello-operators.nf
├── hello-modules.nf
├── hello-workflow.nf
├── hello-world.nf
├── nextflow.config
└── solutions
├── hello-config
├── hello-genomics
├── hello-modules
├── hello-nf-test
├── hello-operators
└── hello-world

18 directories, 17 files
├── solutions
│ ├── 1-hello-world
│ ├── 2-hello-channels
│ ├── 3-hello-workflow
│ ├── 4-hello-modules
│ ├── 5-hello-containers
│ └── 6-hello-config
└── test-params.json

7 directories, 9 files
```

!!!note

Don't worry if this seems like a lot; we'll go through the relevant pieces at each step of the course.
This is just meant to give you an overview.

**Here's a summary of what you should know to get started:**

- **The `.nf` files** are workflow scripts that are named based on what part of the course they're used in.

- **The `hello-*` directories** are directories used in the later Parts of the course where we are working with more than just one workflow file.

- **The file `nextflow.config`** is a configuration file that sets minimal environment properties.
You can ignore it for now.

- **The `data` directory** contains the input data we'll use in most of the course. The dataset is described in detail in Part 3, when we introduce it for the first time.
- **The file `greetings.csv`** contains input data we'll use in most of the course. It is described in Part 1, when we introduce it for the first time.

- **The file `test-params.json`** is a file we'll use in Part 6. You can ignore it for now.

- **The `solutions` directory** contains the completed workflow scripts that result from each step of the course.
They are intended to be used as a reference to check your work and troubleshoot any issues.
Expand All @@ -90,7 +62,7 @@ If you run this inside `hello-nextflow`, you should see the following output:

!!!tip

If for whatever reason you move out of this directory, you can always run this command to return to it:
If for whatever reason you move out of this directory, you can always run this command to return to it (within the training environment):

```bash
cd /workspace/gitpod/hello-nextflow
Expand Down
Loading