Skip to content

Commit 85ba888

Browse files
LuisHeinzlmeierseohyonkimZethson
authored
Add a style guide and chapter template (#367)
* adding initial aspects for the style guide * test escaping md syntax * testing escaping ... * revise the old text * adding another sentence + testing the badge * improve wording and structure + test notebook structure style and > [!WARNING] * test > [!Environment-setup] * final version * final proofreading * update links * add guide to quizzes * added comments to the style guide and created a first template * adding code, files and rules to the template * final improvements * typos * final touches * changelog * add helpful links * Small rewordings Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net> * add a template guide * first improvements * added a few more cells/topics * implemented feedback * update changelog * filter code warnings * improvements based on the review * changes based on the review 2/2 * current state of lamindb * lamindb and filter warnings * formatting test * undo formatting test * final improvements * delete data access * Add link --------- Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net> Co-authored-by: seohyonkim <seohyon.l.kim@gmail.com> Co-authored-by: Lukas Heumos <lukas.heumos@posteo.net>
1 parent 08886ad commit 85ba888

File tree

7 files changed

+792
-70
lines changed

7 files changed

+792
-70
lines changed

‎CONTRIBUTING.md‎

Lines changed: 234 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,47 @@
11
# Contributing
22

3-
We highly welcome community contributions and encourage contributions.
3+
## Our philosophy
4+
5+
We aim for this book to be a resource for introducing people to single-cell and spatial data analysis while ensuring that newcomers and experienced analysts alike do things properly.
6+
To ensure our recommendations reflect best practices, we try to rely exclusively on external and independent benchmarks.
7+
8+
## How can I contribute?
9+
10+
We warmly welcome contributions, especially those that help us keep our best practices up to date!
11+
However, as mentioned [above](#our-philosophy), these recommendations are held to high standards.
12+
13+
If you're proposing new tutorials, corrections, or extensions, we strongly recommend opening an issue first to talk about your ideas with us.
14+
We're always eager to learn about the latest developments and are happy to discuss the current state of best practices.
15+
We also encourage contributions in the form of [quizzes and flashcards](#create-custom-quizzes-and-flashcards).
16+
Whatever proposal you have: Just open an issue and let's get in touch!
17+
18+
[![Open an Issue](https://img.shields.io/badge/Open%20Issue-blue?logo=github)](https://github.com/theislab/single-cell-best-practices/issues/new?title=Your+Issue+Title&body=Describe+your+issue+here)
419

520
## Book architecture
621

7-
The following depicts the rough structure of the book inside the `jupyter_book` folder.
8-
Beyond several configuration files, all chapters are stored inside corresponding folders, such as for example, `conditions`.
9-
Inside, the corresponding notebooks, together with the associated `reference.bib` and `environment.yml` files, are stored.
22+
The `jupyter_book` folder contains the source content and configuration for the book.
23+
In addition to several configuration files, all chapters are grouped into their respective section folders — for example, the `conditions` folder.
24+
Each section contains the relevant notebooks along with their [associated files](#essential-files-for-every-chapter).
25+
26+
Here’s an example of the folder layout:
1027

1128
```bash
1229
├── conditions
13-
│   ├── compositional_environment.yaml
30+
│   ├── compositional_keytakeaways.txt
31+
│   ├── compositional.bib
1432
│   ├── compositional.ipynb
15-
│   ├── compositional_references.bib
33+
│   ├── compositional.yml
34+
│   ├── differential_gene_expression_keytakeaways.txt
1635
│   ├── differential_gene_expression.bib
1736
│   ├── differential_gene_expression.ipynb
18-
│   ├── gsea_pathway.bib
19-
│   ├── gsea_pathway_environment.yml
20-
│   ├── gsea_pathway.ipynb
21-
│   ├── perturbation_modeling.bib
22-
│   ├── perturbation_modeling_environment.yml
23-
│   └── perturbation_modeling.ipynb
37+
│   ├── differential_gene_expression.yml
38+
│   ├── gsea_pathway_keytakeaways.txt
39+
│   └── ...
2440
├── ...
2541
├── _toc.yml
2642
├── _config.yml
2743
├── acknowledgements.md
44+
├── CHANGELOG.md
2845
├── glossary.md
2946
├── outlook.md
3047
├── preamble.md
@@ -61,7 +78,8 @@ Building the book is then as simple as:
6178
make
6279
```
6380

64-
which will build the complete book. This does not execute any notebooks and any updated chapters must be updated in a separate step.
81+
which will build the complete book.
82+
This does not execute any notebooks and any updated chapters must be updated in a separate step.
6583

6684
To clean the build directory run:
6785

@@ -73,9 +91,209 @@ make clean
7391

7492
All chapters are available as Jupyter Notebooks and end-to-end executable.
7593
The diverse requirements of tools for the chapters do not allow it for us to provide a single environment that can build all chapters.
76-
Hence, we decided to provide minimal Conda environments per chapter. These can be found in the respective folders.
94+
Hence, we decided to provide minimal Conda environments per chapter.
95+
These can be found in the respective folders.
96+
97+
> [!NOTE]
98+
> Run the following command with the environment file of choice to create the environment for the chapter that you want to build.
99+
>
100+
> ```bash
101+
> conda env create -f CHAPTER-NAME.yml
102+
> ```
103+
>
104+
> Now you can execute all cells in the notebook.
105+
106+
## Style guide for contributors
107+
108+
### Essential files for every chapter
109+
110+
Each chapter comes with a few essential files.
111+
The `.ipynb` notebook contains the main content and includes citations drawn from the accompanying `.bib` file.
112+
The `.yml` file defines a minimal Conda environment, as described [above](#building-individual-chapters).
113+
Finally, the `_keytakeaways.txt` file summarizes the chapter’s main ideas, following the specified [format](#key-takeaways-environment-and-lamin-dropdown).
114+
115+
```bash
116+
├── SECTION-NAME
117+
│   ├── CHAPTER-NAME.ipynb
118+
│   ├── CHAPTER-NAME.bib
119+
│   ├── CHAPTER-NAME.yml
120+
│   ├── CHAPTER-NAME_keytakeaways.txt
121+
│   ├── ...
122+
```
123+
124+
### Notebook Structure
125+
126+
Each `.ipynb` notebook should follow this standard structure:
127+
128+
1. 🧠 **Title**
129+
2. 🔽 **Dropdown Section**
130+
- Key Takeaways
131+
- Environment Setup
132+
- Lamin Setup
133+
3. 📖 **Main Content**
134+
4. 🔗 **See Also** _(Dropdown)_
135+
5. ❓ **Quiz / Flashcards**
136+
6. 📚 **References**
137+
7. 👥 **Contributors**
138+
139+
All dropdowns immediately following the title are automatically inserted if they meet the corresponding [requirements](#key-takeaways-environment-and-lamin-dropdown).
140+
In addition, each chapter should conclude with a few questions that reinforce the main concepts covered.
141+
[Below](#create-custom-quizzes-and-flashcards) you will find instructions on how to create quizzes and flashcards in our custom format.
142+
We’ve also prepared a chapter [template](/jupyter-book/template/template.ipynb) with which you can quickly and effectively create a chapter for our book.
143+
144+
### Helpful links
145+
146+
- [Jupyter Book documentation](https://jupyterbook.org/en/stable/intro.html)
147+
- [MyST Markdown documentation](https://mystmd.org)
148+
- [Sphinx documentation](https://www.sphinx-doc.org/en/master/)
149+
150+
### Rules
151+
152+
- Place each sentence on its own line to make reviewing easier.
153+
- Use dropdowns so that the reader is not overwhelmed by the content.
154+
- To reduce noise, we should aim to write clean code that avoids generating warnings, and also filter out any non-informative warnings at the start of the notebook.
155+
- In each chapter, link all used glossary terms with `` {term}`EXAMPLE TERM` ``.
156+
- Only link the **first occurrence** of each term within the chapter — not every time it appears.
157+
- Add a new term to the glossary only if it appears multiple times throughout the book and has not yet been listed.
158+
In this case, also add the link to this term in the other chapters.
159+
If the term is used just once and may be unclear, provide a direct explanation within the corresponding chapter.
160+
- To link a term that has the same meaning or a different spelling than its glossary entry, use this format: `` {term}`YOUR TERM <GLOSSARY TERM>` `` (e.g.: `` {term}`barcodes <Barcode>` ``).
161+
- Don't link terms in the key takeaways!
162+
- Based on hours of proofreading: Always make a space before `{cite}` (e.g., ``"This was shown by {cite}`Smith2017`."``).
163+
- References should always contain `doi` and `url`.
164+
- Write in American English.
165+
166+
### Key takeaways, environment and lamin dropdown
167+
168+
The environment and lamin dropdowns are inserted after the title of every chapter.
169+
If you don't want to have those dropdowns in your chapter, make sure to list your notebook in the blacklists of `scripts/dropdowns/keytakeaways.py` (`black_list_files_yml` or `black_list_files_lamin`).
170+
A key takeaways dropdown is only inserted if a file called `<name-notebook>_keytakeaways.txt` is in the same directory as your notebook.
171+
This file has to contain the key takeaways in the following format:
172+
173+
```
174+
1
175+
The first sentence of key takeaway 1.
176+
The second sentence of key takeaway 1.
177+
178+
2
179+
The first sentence of key takeaway 2.
77180
78-
## Adding changelog entries with `towncrier`
181+
...
182+
```
183+
184+
If you want to link a key takeaway to a certain heading in your chapter, add `<section-name>-<notebook-name>-key-takeaway-<key-takeaway-number>` as a label before the heading.
185+
Replace all `_` with `-`, and the card of the key takeaway will be linked to the heading in the text (e.g., `(preprocessing-visualization-dimensionality-reduction-key-takeaway-2)=`).
186+
187+
Our CI workflow (`.github/worksflows/build_book.yml`) will call `make dropdown` when building the book.
188+
For testing, you can insert the dropdowns locally by calling `make dropdown` before `make`.
189+
190+
> [!WARNING]
191+
> Executing `make dropdown` locally will modify nearly all notebook files.
192+
> These changes should never be committed or pushed to the repository.
193+
> We recommend discarding these changes immediately after running the command using `git restore .`.
194+
> Ensure you’ve staged your wanted changes (`git add`) beforehand.
195+
196+
### Lamindb
197+
198+
[Lamindb](https://github.com/laminlabs/lamindb) is an open-source data framework to enable learning at scale in computational biology.
199+
We use lamindb to store, share, and load datasets and notebooks using the [theislab/sc-best-practices](https://lamin.ai/theislab/sc-best-practices) instance.
200+
We acknowledge free hosting from [Lamin Labs](https://lamin.ai/).
201+
202+
When making contributions that got merged, ask us to be added to the `theislab/sc-best-practices` instance.
203+
Then, ensure that all used datasets are loaded directly from the instance.
204+
If you upload a new dataset to the instance, create a separate notebook for it and place it in the [`scripts`](/scripts/) folder.
205+
You can use the existing notebooks in the [`scripts`](/scripts/) folder as a guide.
206+
Finally, use `ln.track()` and `ln.finish()` while re-running the notebook to have the latest version in the instance as well.
207+
The [template](/jupyter-book/template/template.ipynb) shows the basic steps for tracking a notebook!
208+
209+
> [!Note]
210+
>
211+
> 1. **Create a lamin account**
212+
>
213+
> - Sign up and log in following [the instructions](https://docs.lamin.ai/setup#sign-up-log-in).
214+
> - Ask us to be added to the `theislab/sc-best-practices` instance.
215+
>
216+
> 2. **Install lamindb**
217+
>
218+
> - Install the lamindb Python package in your environment:
219+
>
220+
> ```bash
221+
> pip install lamindb[bionty,jupyter,zarr]
222+
> ```
223+
>
224+
> 3. **Connect to the [theislab/sc-best-practices instance](https://lamin.ai/theislab/sc-best-practices)**
225+
>
226+
> - Run the `lamin connect` command:
227+
>
228+
> ```bash
229+
> lamin connect theislab/sc-best-practices
230+
> ```
231+
>
232+
> - You should now see `→ connected lamindb: theislab/sc-best-practices`.
233+
> - You are ready to use lamindb in your notebook!
234+
235+
### Create custom quizzes and flashcards
236+
237+
To build quizzes or flashcards, use the helper functions in `jupyter-book/src/lib.py`.
238+
You can create multiple-choice questions or simple flip cards.
239+
240+
1. Start your notebook code cell with
241+
242+
```python
243+
%run ../src/lib.py
244+
```
245+
246+
2. Then, add as many questions as you like. For example:
247+
248+
```python
249+
flip_card("q1", "What is 2 + 2?", "4")
250+
multiple_choice_question(
251+
"q1",
252+
"What is the capital of France?",
253+
["Paris", "London", "Berlin", "Madrid"],
254+
"Paris",
255+
{
256+
"London": "London is the capital of the UK",
257+
"Berlin": "Berlin is the capital of Germany",
258+
"Madrid": "Madrid is the capital of Spain",
259+
}
260+
)
261+
```
262+
263+
3. Run the code cell and it will build the multiple-choice questions or flip cards as output.
264+
265+
> [!WARNING]
266+
> Add the cell tag `remove-input` to the code cell to remove the code, when building the book.
267+
268+
You can also adjust the font size, text color and much more.
269+
For detailed information, check out the method descriptions in `jupyter-book/src/lib.py`.
270+
271+
### Pre commit
272+
273+
Pre-commit is a tool that automatically checks your markdown and code for mistakes before you commit it.
274+
275+
1. Please install `pre-commit`:
276+
277+
```bash
278+
pip install pre-commit
279+
```
280+
281+
2. Next, activate it in the root of the repository:
282+
283+
```bash
284+
pre-commit install
285+
```
286+
287+
3. Afterwards, you can always manually run it:
288+
289+
```bash
290+
pre-commit run -a
291+
```
292+
293+
If you try to commit changes, they are automatically checked for errors and adjusted if possible.
294+
Simply add these changes to your commit with `git add`.
295+
296+
### Adding changelog entries with `towncrier`
79297
80298
We use `towncrier` to manage our changelog. Here’s how to include a changelog entry when making a PR:
81299
@@ -105,7 +323,7 @@ Valid categories for the filename of the `markdown` are:
105323
106324
5. Push your changes again.
107325
108-
### Releasing a new version (maintainers only)
326+
#### Releasing a new version (maintainers only)
109327
110328
To release a new version:
111329
@@ -125,57 +343,3 @@ This will update `CHANGELOG.md` and remove the `changelog.d/` directory.
125343
mkdir changelog.d
126344
touch changelog.d/.gitkeep
127345
```
128-
129-
## Environment setup
130-
131-
Run the following command with the environment file of choice to create the environment for the chapter that you want to build.
132-
133-
```bash
134-
conda env create -f CHAPTER.yml
135-
```
136-
137-
Now you can execute all cells in the notebook.
138-
139-
## Key takeaways, environment, and lamin dropdown
140-
141-
The environment and lamin dropdowns are inserted after the title of every chapter.
142-
If you don't want to have those dropdowns in your chapter, make sure to list your notebook in the blacklists of `scripts/dropdowns/keytakeaways.py` (`black_list_files_yml` or `black_list_files_lamin`).
143-
A key takeaways dropdown is only inserted if a file called `<name-notebook>_keytakeaways.txt` is in the same directory as your notebook.
144-
This file has to contain the key takeaways in the following format:
145-
146-
```
147-
1
148-
The first sentence of key takeaway 1.
149-
The second sentence of key takeaway 1.
150-
151-
2
152-
The first sentence of key takeaway 2.
153-
154-
...
155-
```
156-
157-
If you want to link a key takeaway to a certain heading in your chapter, add `<section-name>-<notebook-name>-key-takeaway-<key-takeaway-number>` as a label before the heading.
158-
Replace all `_` with `-`, and the card of the key takeaway will be linked to the heading in the text (e.g., `(preprocessing-visualization-dimensionality-reduction-key-takeaway-2)=`).
159-
160-
Our CI workflow (`.github/worksflows/build_book.yml`) will call `make dropdown` when building the book.
161-
For testing, you can insert the dropdowns locally by calling `make dropdown` before `make`.
162-
163-
> [!WARNING]
164-
> Executing `make dropdown` locally will modify nearly all notebook files. These changes should never be committed or pushed to the repository. We recommend discarding these changes immediately after running the command using `git restore .`. Ensure you’ve staged your wanted changes (`git add`) beforehand.
165-
166-
## Data access
167-
168-
We are currently still working on making all datasets accessible.
169-
The problem here isn't data protection or other terrible reasons, but rather the requirement to host them somewhere and ensuring that the correct versions are used per notebook.
170-
171-
## Contributing new best practices, tutorials, quizzes and solutions
172-
173-
### best practices
174-
175-
Contributing or correcting new best practices is welcome, but subject to a high standard. Our philosophy is that we base our recommendations only on external (= not by the tools' authors) and independent benchmarks. Therefore, if you propose new best practices we strongly advise you to open an issue first and discuss them with us. We will certainly have questions, but are super keen on getting to know the latest best practices.
176-
177-
### Contributing new tutorials, quizzes and solutions
178-
179-
We want this book to become a prime resource for introducing people to the field of single-cell and especially best practice data analysis. In the past we have been involved in many teaching efforts, and we noticed that it is imperative to make people reflect on their learning for the most effective outcome. Therefore, we try to add many small quizzes with solutions for self-learners to encourage such a learning style. These quizzes and solutions can always be extended, and we would be happy to get community help.
180-
181-
Entirely new tutorials on topics not yet covered or extensions are subject to "best practices" and we would encourage you to get in touch first with us by opening an issue to discuss such an addition. If best practices for a new topic do not yet exist we are generally open for new tutorials, but again, please ask us first to ensure that your work is not in vain!

‎changelog.d/367.added.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Add a style guide and chapter template for contributors ([#367](https://github.com/theislab/single-cell-best-practices/pull/367)) <sub>@LuisHeinzlmeier</sub>

‎jupyter-book/_config.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,7 @@ bibtex_bibfiles:
9191
- multimodal_integration/paired_integration.bib
9292
- multimodal_integration/advanced_integration.bib
9393
- spatial/spatial.bib
94+
- template/template.bib
9495

9596
repository:
9697
url: https://github.com/theislab/single-cell-best-practices

‎jupyter-book/template/template.bib‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
@article{Heumos2023,
2+
author = {Heumos, Lukas and Schaar, Anna C. and Lance, Christopher and Litinetskaya, Anastasia and Drost, Felix and Zappia, Luke and Lücken, Malte D. and Strobl, Daniel C. and Henao, Juan and Curion, Fabiola and Aliee, Hananeh and Ansari, Meshal and Badia-i-Mompel, Pau and Büttner, Maren and Dann, Emma and Dimitrov, Daniel and Dony, Leander and Frishberg, Amit and He, Dongze and Hediyeh-zadeh, Soroor and Hetzel, Leon and Ibarra, Ignacio L. and Jones, Matthew G. and Lotfollahi, Mohammad and Martens, Laura D. and Müller, Christian L. and Nitzan, Mor and Ostner, Johannes and Palla, Giovanni and Patro, Rob and Piran, Zoe and Ramírez-Suástegui, Ciro and Saez-Rodriguez, Julio and Sarkar, Hirak and Schubert, Benjamin and Sikkema, Lisa and Srivastava, Avi and Tanevski, Jovan and Virshup, Isaac and Weiler, Philipp and Schiller, Herbert B. and Theis, Fabian J. and {Single-cell Best Practices Consortium}},
3+
title = {Best practices for single-cell analysis across modalities},
4+
journal = {Nature Reviews Genetics},
5+
year = {2023},
6+
date = {2023/08/01},
7+
volume = {24},
8+
number = {8},
9+
pages = {550--572},
10+
issn = {1471-0064},
11+
doi = {10.1038/s41576-023-00586-w},
12+
url = {https://doi.org/10.1038/s41576-023-00586-w},
13+
}

0 commit comments

Comments
 (0)