Skip to content

Commit 9e7d1f6

Browse files
lwasserlwasser
committed
Fix: Final tutorial edits
1 parent 948729c commit 9e7d1f6

File tree

3 files changed

+160
-80
lines changed

3 files changed

+160
-80
lines changed

‎images/conda-forge-staged-recipes-ci.png

285 KB
Loading

‎images/pypi-project-landing-page-no-meta.png

328 KB
Loading

‎tutorials/publish-conda-forge.md

Lines changed: 160 additions & 80 deletions
Original file line numberDiff line numberDiff line change
@@ -57,8 +57,6 @@ Make a graphic to replace that geohackweek graphic that is also more specific.
5757
Conda channels represent various repositories that you can install packages from. Because conda-forge is community maintained, anyone can submit a recipe there. PiPY is also a community maintained repository. Anyone can submit a package to PyPI and test PyPI. Unlike conda-forge there are no manual checks of packages submitted to PyPI.
5858
:::
5959

60-
61-
6260
## Why publish to conda forge
6361

6462
There are many users, especially in the scientific Python ecosystem that use conda as their primary package manager / environment tool. Thus, having packages available to these users on the conda-forge channel is useful. In some cases packages on conda-forge can minimize dependency conflicts that can occur when mixing installations using pip and conda. This is particularly important for the spatial ecosystem.
@@ -77,9 +75,9 @@ Conda-forge will build your package from the source distribution which you [publ
7775
### Conda-forge publication steps
7876

7977
:::{figure-md} publish-pypi-conda-forge
80-
<img src="../images/pyopensci-python-package-pypi-to-conda-forge.png" alt="." width="700px">
78+
<img src="../images/pyopensci-python-package-pypi-to-conda-forge.png" alt="Image showing the steps associated with publishing to conda-forge. Check out the caption below for a detailed description." width="700px">
8179

82-
Caption
80+
The steps for publishing to conda-forge begin with publishing your Python package to PyPI. Once you have published to PyPI you can then create a yaml file recipe that can be submitted to the conda-forge staged recipes repository for review. Once that recipe is accepted, your package will get it's on repository (known as a feedstock) on conda-forge.
8381
:::
8482

8583
The steps to publish to conda-forge are:
@@ -91,56 +89,99 @@ The steps to publish to conda-forge are:
9189
4. Once someone from the conda-forge team reviews your pull request, you may need to make some changes. Eventually the pull request will be approved and merged.
9290

9391

92+
Once your recipe is accepted and merged on conda-forge, users can install your package using:
9493

94+
`conda install -c conda-forge your-package`
9595

96-
Now your package is on conda-forge.
97-
98-
You only create the recipe once. Then you just maintain the repository.
99-
96+
You only create the recipe once. Once the recipe is accepted and merged, you only need to maintain the repository.
10097

10198
## Maintaining a conda-forge package
10299

103-
Once your package is on conda-forge, the repository will track activity on your PyPI repository for that package. Any time you make a new release to PyPI with a new source distribution, conda-forge will build and update your conda-forge repository.
100+
Once your package is on conda-forge, the repository will track release activity on the package's PyPI repository. Any time you make a new PyPI release with a new source distribution, conda-forge will build and update your conda-forge repository (also known as a feedstock).
104101

105-
When that happens, conda-forge will create a new pull request with an updated distribution recipe.
102+
When the update is processed, the friendly conda-forge bot will create a new pull request with an updated distribution recipe in your feedstock.
106103

107-
You can review that pull request and then merge it once all of the tests pass.
104+
You can review that pull request and then merge it once all of the continuous integration tests pass.
108105

109-
## Publish your package to conda-forge
106+
## <i class="fa-regular fa-pen-to-square"></i> How to Publish your package on conda-forge
110107

111108
It's time to add your package to the conda-forge channel.
112109
Remember that your package needs to be on PyPI before the steps below will work. And also remember that the team managing conda-forge are all volunteers.
113110

114-
* Please only submit your package to conda-forge if you intend to maintain it over time.
115-
* Be sure that your package is on PyPI.org (not test.pypi.org) before you attempt to publish to PyPI.
111+
* Be sure that your package is on PyPI.org (not test.pypi.org) before you attempt to publish to conda-forge.
116112

117-
### Step 1 - Install grayskull
113+
:::{important}
114+
Only submit your package to conda-forge if you intend to maintain it over time.
115+
:::
118116

119-
To begin this process you need to [install grayskull](https://conda.github.io/grayskull/user_guide.html). You can install it using either pip
117+
Note - this is a tutorial aimed to help you get your package onto conda-forge. The official conda documentation for this processed [is here](https://conda-forge.org/docs/maintainer/adding_pkgs.html).
120118

121-
```
122-
pip install grayskull
119+
### Step 1: Install grayskull
120+
121+
First, [install grayskull](https://conda.github.io/grayskull/user_guide.html). You can install it using either pip:
122+
123+
```bash
124+
> pip install grayskull
123125
```
124126

125127
or conda
126128

127-
```
128-
conda install -c conda-forge grayskull
129+
```bash
130+
> conda install -c conda-forge grayskull
129131
```
130132

131-
Use the shell / terminal that you have been using to run hatch
133+
To run this command, use the same shell / terminal
134+
that you have been using to run hatch
132135
commands in the previous tutorials.
133136

134-
### Step 2: Create your recipe from PyPI
137+
:::{note}
138+
You can also install grayskull using pipx[^pipx]. pipx is a tool that allows you to install commonly used tools that you might want to have available across multiple Python environments rather than installing the package into every Python environment that ou create.
139+
:::
140+
141+
### Step 2: Fork and clone the conda-forge staged-recipes repository
142+
143+
* Next, open your shell and `cd` to a location where you want to clone the **conda-forge/staged-recipes** repository.
144+
* fork and clone the [conda-forge/staged-recipes GitHub repository](https://github.com/conda-forge/staged-recipes).
145+
* Create a new branch in your fork rather than submitting from the main branch of your fork. We suggest naming the branch your package's name.
146+
147+
`git checkout -b your-package-name `
148+
149+
* In bash, `cd` into the `staged-recipes/recipes` folder
150+
151+
```bash
152+
staged-recipes/recipes (☊ pyospackage)
153+
# Clone the repository
154+
➜ git clone git@github.com:conda-forge/staged-recipes.git
155+
```
156+
157+
Next, create a new branch in your `conda-forge/staged-recipes` cloned repository. You might want to make that branch the same name as your package.
135158

136-
Next, you can run grayskull on your package.
159+
```bash
160+
# Create a new branch called pyospackage
161+
➜ git checkout -b pyospackage
162+
➜ cd staged-recipes/recipes
163+
➜ git branch
164+
main
165+
* pyospackage
166+
```
137167

138-
Grayskull will pull information from PyPI
139-
You don't need to work about what directory you are in when you run grayskull. Grayskull will look for your package on PyPI and will generate your recipe from the PyPI distribution.
168+
### Step 3: Create your conda-forge recipe
140169

141-
Because it is pull from pypi.org, an internet connection is needed for this step.
170+
* Next, navigate to the recipes directory
142171

143-
Run the command below in your favorite shell.
172+
If you run `ls` here, you will notice there is an example directory with an example recipe for you to look at.
173+
174+
```bash
175+
staged-recipes (☊ pyospackage) via 🐍 pyenv
176+
➜ cd recipes
177+
178+
staged-recipes/recipes (☊ pyospackage)
179+
➜ ls
180+
example
181+
182+
```
183+
184+
* Next, run `grayskull pypi your-package-name` to generate a recipe.
144185

145186
```bash
146187
➜ grayskull pypi pyospackage
@@ -175,18 +216,30 @@ Maintainers:
175216
- lwasser
176217

177218
#### Recipe generated on /your/file/package/here/pyosPackage for pyospackage ####
219+
220+
# You should see a new directory created by grayskull after this run is completed.
221+
staged-recipes/recipes (☊ pyospackage)
222+
➜ ls
223+
example pyospackage
178224
```
179225

226+
227+
:::{tip}
228+
* Grayskull will pull metadata about your package from PyPI. It does not use your local installation of the package.
229+
* An internet connection is needed to run the `grayskull pypi your-package-name` step.
230+
:::
231+
232+
180233
When you run grayskull, it will grab the latest distribution of your package from PyPI and will use that to create a new recipe.
181234

182235
The recipe will be saved in a directory named after your package's name, wherever you run the command.
183236

184-
`packagename/meta.yaml`
237+
`recipes/packagename/meta.yaml`
185238

186239
At the very bottom of the grayskull output, it will also tell you
187240
where it saved the recipe file.
188241

189-
This will create a meta.yaml file that looks like the example below:
242+
Open the meta.yaml file. The finished `meta.yaml` file that grayskull creates should look like the example below:
190243

191244
```yaml
192245
{% set name = "pyospackage" %}
@@ -222,7 +275,9 @@ test:
222275
- pip
223276

224277
about:
225-
license: MIT
278+
summary: A package that adds numbers together
279+
dev_url: https://github.com/pyopensci/pyospackage
280+
license: BSD-3-Clause
226281
license_file: LICENSE
227282

228283
extra:
@@ -231,9 +286,51 @@ extra:
231286

232287
```
233288

234-
### Step 3: tests for conda-forge
289+
### Step 3b: Bug fix - add a home url to the about: section
290+
291+
There is currently a small bug in Grayskull where it doesn't populate the home: element of the recipe. if you don't include this, [you will receive an error message](https://github.com/conda-forge/staged-recipes/pull/25173#issuecomment-1917916528) from the friendly conda-forge linter bot.
292+
293+
294+
295+
```
296+
Hi! This is the friendly automated conda-forge-linting service.
297+
298+
I wanted to let you know that I linted all conda-recipes in your PR (recipes/pyospackage) and found some lint.
299+
300+
Here's what I've got...
301+
302+
For recipes/pyospackage:
303+
304+
The home item is expected in the about section.
305+
```
306+
307+
* to fix this, open your meta.yaml file in your favorite text editor.
308+
* and add a home: element to the about section
309+
310+
The about section will look like this after you create your recipe.
311+
312+
```yaml
313+
about:
314+
summary: A package that adds numbers together
315+
dev_url: https://github.com/pyopensci/pyospackage
316+
license: BSD-3-Clause
317+
license_file: LICENSE
318+
```
319+
320+
Below you add a home: element. If you have a project home page / website you can use that url. Otherwise, you can also use your PyPI landing page.
321+
322+
```yaml
323+
about:
324+
home: https://pypi.org/project/pyospackage/
325+
summary: A package that adds numbers together
326+
dev_url: https://github.com/pyopensci/pyospackage
327+
license: BSD-3-Clause
328+
license_file: LICENSE
329+
```
330+
331+
### Step 4: tests for conda-forge
235332
236-
Next, have a look at the tests section in your meta.yaml file. At a minimum you should import your package and run `pip check`.
333+
Next, have a look at the tests section in your **meta.yaml** file. At a minimum you should import your package and run `pip check`.
237334

238335
`pip check` will ensure that your package installs properly with all of the proper dependencies.
239336

@@ -247,40 +344,27 @@ test:
247344
- pip
248345
```
249346

250-
### Step 4: Submit a pull request to the staged-recipes repository
251-
252-
Finally, create a pull request in the staged-recipes GitHub repository.
347+
If you have more advanced tests that you wish to run, you can add them here. However, you can also simple leave the tests section as it is.
253348

254-
To create your pull request:
255-
256-
1. Fork and clone this repo: https://github.com/conda-forge/staged-recipes
257-
1. Create a branch in your fork rather than submitting from the main branch of your fork.
258-
1. Within your fork's branch, create a new directory with the name of your package,
259-
1. Add the meta.yaml file that you generated using grayskull to that directory.
349+
### Step 4: Submit a pull request to the staged-recipes repository
260350

261-
`staged-recipes/recipes/pyospackage/meta.yaml`
351+
Once you have completed all of the above, you are ready to open up a pull request in the `conda-forge/staged-recipes repository`.
262352

263353
1. Submit a pull request from your fork/branch of the staged-recipes repository.
264354
1. Remember that the conda-forge maintainers are volunteers. Be patient for someone to respond and supportive in your communication with them.
265355

266-
When you do this, a suite of CI actions will run that build and test the build of your package. A conda-forge maintainer will work with you to get your recipe in good shape and merged.
267356

268-
You can follow the [instructions here](https://conda-forge.org/docs/maintainer/adding_pkgs.html) to submit your package
269357

358+
:::::{dropdown} Conda-forge checklist help
359+
:icon: unlock
360+
:color: secondary
270361

271-
## Conda-forge pull request checklist
362+
### Conda-forge Staged-recipes Pull Request Checklist
272363

273364
When you submit your package to conda forge, the pull request template includes a list of checks that you want to ensure you have covered.
274365

275366
Below we break down each element of that list.
276367

277-
278-
## Conda-forge Staged-recipes Pull Request Checklist
279-
280-
When you submit a pull request for your package's recipe to conda-forge you will notice a checklist of items that you need to ensure you have covered.
281-
282-
A brief explanation of each is below.
283-
284368
::::{admonition} Pull request template checklist tips
285369
:class: note
286370

@@ -346,6 +430,26 @@ url: https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/pyospackage-{{ ver
346430

347431
This is also why we don't suggest you publish to conda-forge as a practice run.
348432
::::
433+
:::::
434+
435+
Once you create your pull request, a suite of CI actions will run that build and test the build of your package. A conda-forge maintainer will work with you to get your recipe in good shape and merged.
436+
437+
438+
439+
440+
:::{figure-md} pypi-conda-channels
441+
442+
<img src="../images/conda-forge-staged-recipes-ci.png" alt="Image showing the 5 CI tasks that will run against your package in the GitHub interface after you'ce created a pull request. " width="700px">
443+
444+
Wait until you have green checks on all of the CI steps in your pull request. At that point your pull request is ready for review.
445+
:::
446+
447+
In some cases getting things to run properly on CI might take a bit of work. If you are struggling you can ping the conda-forge maintainer team for help.
448+
449+
Please be patient and wait for them to respond.
450+
451+
452+
349453

350454
Once you have submitted your recipe, you can wait for the CI build to pass. If it's not passing, and you aren't sure why, a conda-forge maintainer can likely help you figure things out.
351455

@@ -362,37 +466,10 @@ Every time you create a new release on PyPI, the conda-forge bots will recognize
362466

363467
Once the conda-forge build it complete, all of the maintainers of your conda-forge feedstock will get a ping on GitHub that a new pull request has been opened.
364468

365-
Review the pull request. If all tests are passing, you can merge it. within the next day the conda-forge release will be available for users to install:
469+
Review the pull request. If all tests are passing, you can merge it. Shortly after merging your pull request, the conda-forge release will be available for users to install:
366470

367471
`conda install -c conda-forge yourpackage`
368472

369-
:::{todo}
370-
pr for our tutorial package is up as a draft. right now i'm the only maintainer.
371-
372-
<Create a pr on conda-forge - tag filipe (as it is a test package) and add screenshots here so people understand what maintaining a conda recipe entails >
373-
374-
375-
Questions:
376-
377-
Pandas has this for their recipe -
378-
about:
379-
home: http://pandas.pydata.org
380-
license: BSD-3-Clause
381-
license_file: LICENSE
382-
summary: >-
383-
Powerful data structures for data analysis, time series, and statistics
384-
doc_url: https://pandas.pydata.org/docs/
385-
dev_url: https://github.com/pandas-dev/pandas
386-
387-
their pyprojec.toml is like This:
388-
[project.urls]
389-
homepage = 'https://pandas.pydata.org'
390-
documentation = 'https://pandas.pydata.org/docs/'
391-
repository = 'https://github.com/pandas-dev/pandas'
392-
393-
Should we have users add a homepage or can they add a homepage at a later time??
394-
:::
395-
396473
## <i class="fa-solid fa-hands-bubbles"></i> Wrap up
397474

398475
If you have walked through this entire tutorial series you have now:
@@ -408,3 +485,6 @@ The above are the basic steps that you need to take to create and publish a Pyth
408485
## Footnotes
409486

410487
[^grayskull]: [Grayskull blogpost](https://conda-forge.org/blog/2020/03/05/grayskull/)
488+
489+
490+
[^pipx]: [Pipx documentation](https://pipx.pypa.io/stable/)

0 commit comments

Comments
 (0)