Skip to content

Commit 26f3284

Browse files
rcannoodrcannood
committed
deploy: df3f053 
0 parents  commit 26f3284

File tree

467 files changed

+290738
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

467 files changed

+290738
-0
lines changed

‎.gitignore

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
/resources
2+
/resources_test
3+
/work
4+
/.nextflow*
5+
6+
.DS_Store
7+
/output
8+
trace-*
9+
.ipynb_checkpoints
10+
__pycache__

‎.gitmodules

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
[submodule "common"]
2+
path = common
3+
url = https://github.com/openproblems-bio/common_resources.git

‎.nojekyll

Whitespace-only changes.

‎.vscode/settings.json

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
{
2+
"yaml.schemas": {
3+
"common/schemas/api_component_spec.yaml": "**/api/comp_*.yaml",
4+
"common/schemas/api_file_format.yaml": "**/api/file_*.yaml",
5+
"common/schemas/task_config.yaml": "_viash.yaml",
6+
"common/schemas/task_method.yaml": "**/methods/**/config.vsh.yaml",
7+
"common/schemas/task_control_method.yaml": "**/control_methods/**/config.vsh.yaml",
8+
"common/schemas/task_metric.yaml": "**/metrics/**/config.vsh.yaml"
9+
}
10+
}

‎CHANGELOG.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# task_batch_integration x.y.z
2+
3+
## BREAKING CHANGES
4+
5+
<!-- * Restructured `src` directory (PR #3). -->
6+
7+
## NEW FUNCTIONALITY
8+
9+
* Added `control_methods/true_labels` component (PR #5).
10+
11+
* Added `methods/logistic_regression` component (PR #5).
12+
13+
* Added `metrics/accuracy` component (PR #5).
14+
15+
## MAJOR CHANGES
16+
17+
* Updated `api` files (PR #5).
18+
19+
* Updated configs, components and CI to the latest Viash version (PR #8).
20+
21+
## MINOR CHANGES
22+
23+
* Updated `README.md` (PR #5).
24+
25+
## BUGFIXES
26+

‎CONTRIBUTING.md

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
# Contribution guidelines
2+
3+
We encourage contributions from the community. To contribute:
4+
5+
* Star this repository: Click the star button in the top-right corner of the repository to show your support.
6+
* Fork the repository: Start by forking this repository to your account.
7+
* Develop your component: Create your Viash component, ensuring it aligns with our best practices (detailed below).
8+
* Submit a pull request: After testing your component, submit a pull request for review.
9+
10+
## Installation
11+
12+
You need to have Docker, Java, and Viash installed. Follow
13+
[these instructions](https://openproblems.bio/documentation/fundamentals/requirements)
14+
to install the required dependencies.
15+
16+
## Getting started
17+
18+
### Cloning the repository
19+
20+
To get started, fork the repository and clone it to your local machine:
21+
22+
```bash
23+
git clone <git url to fork> --recursive
24+
25+
cd <name of the repository>
26+
```
27+
28+
NOTE: If you forgot to clone the repository with the `--recursive` flag, you can run the following command to update the submodules:
29+
30+
```bash
31+
git submodule update --init --recursive
32+
```
33+
34+
### Downloading the test resources
35+
36+
Next, you should download the test resources:
37+
38+
```bash
39+
scripts/sync_resources.sh
40+
```
41+
42+
You may need to run this script again if the resources are updated.
43+
44+
## Good first steps
45+
46+
### Creating a new method
47+
48+
To create a new method, you can use the following command:
49+
50+
```bash
51+
# in Python:
52+
common/scripts/create_component \
53+
--name my_method \
54+
--language python \
55+
--type method
56+
57+
# or in R:
58+
common/scripts/create_component \
59+
--name my_method \
60+
--language r \
61+
--type method
62+
```
63+
64+
This will create a new method in `src/methods/my_method`. Next, you should:
65+
66+
* Fill in the component's metadata
67+
* Specify dependencies
68+
* Implement the method's code
69+
* Run the unit test
70+
71+
Please review our documentation on [creating a new method](https://openproblems.bio/documentation/create_component/add_a_method) for more information on how to do this.
72+
73+
74+
### Creating a new metric
75+
76+
Creating a new metric is similar to creating a new method. You can use the following command:
77+
78+
```bash
79+
# in Python:
80+
common/scripts/create_component \
81+
--name my_metric \
82+
--language python \
83+
--type metric
84+
85+
# or in R:
86+
common/scripts/create_component \
87+
--name my_metric \
88+
--language r \
89+
--type metric
90+
```
91+
92+
This will create a new metric in `src/metrics/my_metric`. Next, you should:
93+
94+
* Fill in the component's metadata
95+
* Specify dependencies
96+
* Implement the metric's code
97+
* Run the unit test
98+
99+
Please review our documentation on [creating a new metric](https://openproblems.bio/documentation/create_component/add_a_metric) for more information.
100+
101+
102+
## Frequently used commands
103+
104+
To get started, you can run the following commands:
105+
106+
### Sync the test data
107+
108+
To sync the test data, run the following command:
109+
110+
```bash
111+
scripts/sync_resources.sh
112+
```
113+
114+
### Building the components
115+
116+
To run the benchmark, you first need to build the components.
117+
118+
```bash
119+
viash ns build --parallel --setup cachedbuild
120+
```
121+
122+
For each of the components, this will:
123+
124+
* Build a Docker image
125+
* Build an executable at `target/executable/<component_name>`
126+
* Build a Nextflow module at `target/nextflow/<component_name>`
127+
128+
### Running the unit tests
129+
130+
To run the unit test of one component, you can use the following command:
131+
132+
```bash
133+
viash test src/path/to/config.vsh.yaml
134+
```
135+
136+
To run the unit tests for all components, you can use the following command:
137+
138+
```bash
139+
viash ns test --parallel
140+
```
141+
142+
### Running the benchmark
143+
144+
To run the benchmark, you can use the following command:
145+
146+
```bash
147+
scripts/run_benchmark/run.sh
148+
```
149+
150+
## Debugging nf-tower runs
151+
152+
The actual benchmark is run on the [nf-tower platform](https://cloud.seqera.io/orgs/openproblems-bio/workspaces/openproblems-bio/watch).
153+
You can find runs related to the batch integration by filtering for `label:task_batch_integration`, selecting a specific run and browsing the outputs.
154+
155+
For debugging purposes, you can download the output of a task run from nf-tower following command:
156+
157+
```bash
158+
common/scripts/fetch_task_run \
159+
--input s3://openproblems-work/work/<process ID>\
160+
--output temp_output
161+
```
162+
163+
You can find the process ID under the "Work directory" section of the Task details.

‎LICENSE

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2024 Open Problems in Single-Cell Analysis
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

0 commit comments

Comments
 (0)