You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _sources/devdoc/release.rst.txt
+47-90Lines changed: 47 additions & 90 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,14 +5,28 @@ Releasing a new version
5
5
Introduction
6
6
------------
7
7
8
-
This guide explains how to release a new version of scikit-bio. To illustrate examples of commands you might run, let's assume that the current version is **x.y.y-dev** and we want to release version **x.y.z**.
8
+
This guide explains how to release a new version of scikit-bio. To illustrate examples of commands you might run, let's assume that the current version is **x.y.z-dev** and we want to release version **x.y.z**.
9
9
10
10
.. note:: The following commands assume you are in the top-level directory of the scikit-bio repository unless otherwise noted. They also assume that you have [Conda](https://conda.io) installed.
11
11
12
+
Large portions of the workflow are now automated through GitHub workflows. In order to check that things will work correctly before running the release workflow in ``release.yml``, please follow these steps.
13
+
14
+
The general workflow is:
15
+
16
+
1. Update the version number in the repository from **x.y.z-dev** to **x.y.z**.
17
+
18
+
2. Ensure documentation, wheels, and source distribution build correctly before publishing release.
19
+
20
+
3. Publish release.
21
+
22
+
4. Update version number in the repository from **x.y.z** to **a.b.c-dev**.
23
+
12
24
13
25
Prep the release
14
26
----------------
15
27
28
+
The purpose of these steps is to update the version within the package. At this point we are only updating the version number in the repository, and doing some basic checks.
29
+
16
30
1. Ensure the GitHub Actions CI build is passing against main.
17
31
18
32
2. Update the version strings (x.y.y-dev) to the new version (x.y.z). This will include ``__version__`` defined in ``skbio/__init__.py``. ``grep`` for the current version string to find all occurrences::
@@ -30,128 +44,71 @@ Prep the release
30
44
5. Submit a pull request and merge when CI tests are passing.
31
45
32
46
33
-
Build website docs
34
-
------------------
35
-
36
-
You will need to **fully install** the latest main branch of scikit-bio (including built extensions) and build the docs from this version. **Make sure the version of scikit-bio that is imported by ``import skbio`` is the correct one before building the docs.**
37
-
38
-
1. Build the documentation locally::
39
-
40
-
make -C doc clean html
41
-
42
-
2. Switch to the ``gh-pages`` branch of the repository.
43
-
44
-
3. Remove ``docs/latest``::
45
-
46
-
git rm -rf docs/latest
47
-
48
-
4. Copy over the built documentation to ``docs/x.y.z`` and ``docs/latest``::
49
-
50
-
cp -r doc/build/html docs/latest
51
-
cp -r doc/build/html docs/x.y.z
52
-
53
-
5. Add a new list item to ``index.html`` to link to ``docs/x.y.z/index.html``.
54
-
55
-
6. Port content from ``README.md`` to ``index.html`` if there are any changes that need to be included on the front page.
47
+
Build documentation locally
48
+
---------------------------
56
49
57
-
7. Test out your changes by opening the site locally in a browser. Be sure to check the error console for any errors.
50
+
This is a **pre-release check** to be performed by the developer to ensure that things run smoothly once we get to the actual release publication steps. It is a **critical step** to avoid failures once the automated workflows run.
58
51
59
-
8. Submit a pull request with the website updates and merge.
52
+
In an environment with **the same version of Python** specified in the ``Publish documentation`` job within the ``release.yml`` workflow, build the documentation locally using ``make doc`` and ensure that it runs without error and that everything renders correctly.
60
53
61
-
..note:: Once merged, the live website is updated, so be sure to poke through the live site to make sure things are rendered correctly with the right version strings.
54
+
We now use Sphinx and automated workflows to handle the documentation building and publishing process. Check Sphinx's most recent Python version support, and pin to this version in ``release.yml`` if it isn't the latest.
62
55
63
56
64
-
Tag the release
65
-
---------------
57
+
Test wheels and sdist fully
58
+
---------------------------
66
59
67
-
From :repo:`scikit-bio GitHub repo`, click on the releases tab and draft a new release. Use the version number for the tag name (x.y.z) and create the tag against main. Fill in a release title that is consistent with previous release titles and add a summary of the release (linking to ``CHANGELOG.md`` is a good idea). This release summary will be the primary information that we point users to when we announce the release.
68
-
69
-
Once the release is created on GitHub, it's a good idea to test out the release tarball before publishing to PyPI:
70
-
71
-
1. Create a new ``conda`` environment for testing (fill in a name for ``<environment>``)::
72
-
73
-
conda create -n <environment> python=3.10 numpy
74
-
conda activate <environment>
75
-
76
-
2. Install the release tarball from GitHub and run the tests::
Assuming the GitHub release tarball correctly installs and passes its tests, you're ready to create the source distribution (``sdist``) that will be published to PyPI. It is important to test the source distribution because it is created in an entirely different way than the release tarball on GitHub. Thus, there is the danger of having two different release tarballs: the one created on GitHub and the one uploaded to PyPI.
60
+
This is a **pre-release check** to be performed by the developer to ensure that things run smoothly once we get to the actual release publication steps. It is a **critical step** to avoid failures once the automated workflows run.
86
61
87
-
1. Download the release tarball from GitHub, extract it, and ``cd`` into the top-level directory.
62
+
The ``wheels.yml`` file may be triggered manually through GitHub Actions ``workflow_dispatch`` (https://github.com/scikit-bio/scikit-bio/actions/workflows/wheels.yml). Click the ``Run workflow`` button and run it against the ``main`` branch.
88
63
89
-
2. Build a source distribution::
64
+
When this workflow is manually triggered, it will build wheels for all supported Python versions and all supported platforms, and run the full suite of unit tests on every wheel that it builds.
65
+
This process typically takes some time, but it is well worth the time to ensure that the wheel building process will be successful when it's time to actually publish the release.
90
66
91
-
python setup.py sdist
67
+
Manually check that the correct wheels are built by downloading the artifacts from the workflow run or visually checking the output from ``cibuildwheel`` on the GitHub Actions workflow page.
68
+
If downloading, inspect the zip file containing the wheels provided by workflow for each platform. The zip file should contain all the wheels you are trying to build.
92
69
93
-
3. Create and activate a new `conda` environment, and test the `sdist`::
70
+
If you find that the desired wheels are not present in the zip files, or you are coming accross unexpected errors in the wheel building process, there are a few places to check:
94
71
95
-
pip install dist/scikit-bio-x.y.z.tar.gz
96
-
cd # cd somewhere outside the extracted scikit-bio directory
97
-
python -m skbio.test
72
+
1. Check that you have updated the ``cibuildwheel`` portion of the ``pyproject.toml`` configuration file to include all supported Python versions and any other information you want.
98
73
99
-
4. If everything goes well, it is finally time to push the release to PyPI::
74
+
2. Ensure that the version of ``cibuildwheel`` used in ``wheels.yml`` is up to date. It may be the case that the latest version of Python is not supported by whichever version is currently in use in ``wheels.yml``.
75
+
In this case, ``cibuildwheel`` will not throw an error or warning, it just won't build wheels for that version of Python.
100
76
101
-
python setup.py sdist upload
77
+
3. Check that the version of ``manylinux`` in the ``cibuildwheel`` section of ``pyproject.toml`` is up to date. This is particularly relevant if any of the errors you encounter are related to specific versions of ``glibc``.
102
78
103
-
.. warning:: You must have the proper login credentials to add a release to PyPI. Currently `@gregcaporaso <https://github.com/gregcaporaso>`_ has these, but they can be shared with other release managers.
79
+
This workflow also builds the source distribution for the package. It is **highly recommended** to download the built sdist from this workflow and test that you can build it against **every** Python version supported by scikit-bio.
80
+
Again, this is tedious, but well worth the benefit of being certain things will work before publishing the release.
104
81
105
-
5. Once the release is available on PyPI, do a final round of testing. Create a new `conda` environment and run::
106
82
107
-
pip install scikit-bio
108
-
cd # cd somewhere outside the extracted scikit-bio directory
109
-
python -m skbio.test
83
+
Tag the release and publish
84
+
---------------------------
110
85
111
-
If this succeeds, the PyPI release appears to be a success. Make sure the installed version is the correct one.
86
+
Assuming that all of the above checks and tests have been performed and are passing, it is time to publish the release.
112
87
113
-
6. Next, we'll prepare and post the release to `anaconda.org <https://www.anaconda.com/>`_.
88
+
**Once you hit the ``Publish`` button, all of the automated workflows will run, and they will actually upload the artifacts (documentation, sdist, wheels) to their respective locations (website, PyPI), so ensure that everything is in order before hitting ``Publish``!**
114
89
115
-
You'll need to have ``conda-build`` and ``anaconda-client`` installed to perform these steps. Both can be conda-installed. First, log into anaconda with your anaconda username using the following command. You should have access to push to the ``biocore`` anaconda account through your account (if you don't, get in touch with [@gregcaporaso](https://github.com/gregcaporaso) who is the owner of that account)::
116
-
117
-
anaconda login
118
-
119
-
Due to its C extensions, releasing scikit-bio packages for different platforms will require you to perform the following steps on each of those platforms. For example, an ``osx-64`` package will need to be built on OS X, and a ``linux-64`` package will need to be built on 64-bit Linux. These steps will be the same on all platforms, so you should repeat them for every platform you want to release for::
120
-
121
-
conda skeleton pypi scikit-bio
122
-
conda build scikit-bio --python 3.10
123
-
124
-
When building 64-bit Linux packages, it is recommended that you use conda-forge's ``linux-anvil``` Docker image. This ensures a consistent Linux build environment that has an old enough version of `libc` to be compatible on most Linux systems. To start up a ``linux-anvil`` Docker container::
125
-
126
-
docker run -i -t condaforge/linux-anvil
127
-
# Now you should be in the linux-anvil environment
128
-
sed -i '/conda-forge/d' ~/.condarc
129
-
# Run the build commands from above
130
-
131
-
At this stage you have built Python 3.10 packages. The absolute path to the packages will be provided as output from each ``conda build`` commands. You should now create conda environments for each, and run the tests as described above. You can install these local package as follows::
132
-
133
-
conda install --use-local scikit-bio
90
+
From :repo:`scikit-bio GitHub repo`, click on the releases tab and draft a new release. Use the version number for the tag name (x.y.z) and create the tag against main. Fill in a release title that is consistent with previous release titles and add a summary of the release (linking to ``CHANGELOG.md`` is a good idea). This release summary will be the primary information that we point users to when we announce the release.
134
91
135
-
If the tests pass, you're ready to upload::
92
+
Hit ``Publish`` when you are ready and keep an eye on the processes to see if any errors arise.
136
93
137
-
anaconda upload -u biocore <package-filepath>
138
94
139
-
``<package-filepath>`` should be replaced with the path to the package that was was created above. Repeat this for each package you created (here, the Python 3.10 package).
95
+
Add wheels to GitHub release page
96
+
---------------------------------
140
97
141
-
After uploading, you should create new environments for every package you uploaded, install scikit-bio from each package, and re-run the tests. You can install the packages you uploaded as follows::
98
+
Once the automated workflows have run, and our wheels are successfully uploaded to PyPI, you can manually download the wheels from PyPI and add them to the GitHub release page for the release we just published. Click the ``edit release`` button, and then simply add the binaries to the release page.
This isn't an elegant solution, and it may be automated in the future, but for now this is what we're doing.
144
101
145
102
146
103
Post-release cleanup
147
104
--------------------
148
105
149
-
1. Submit and merge a pull request to update the version strings from x.y.z to x.y.z-dev (``skbio.__version__`` should be the only thing needing an update). Update ``CHANGELOG.md`` to include a new section for x.y.z-dev (there won't be any changes to note here yet).
106
+
1. Submit and merge a pull request to update the version strings from x.y.z to a.b.c-dev (``skbio.__version__`` should be the only thing needing an update). Update ``CHANGELOG.md`` to include a new section for a.b.c-dev (there won't be any changes to note here yet).
150
107
151
108
2. Close the release milestone on the GitHub issue tracker if there was one.
152
109
153
110
3. Send an email to the skbio developers list and anyone else who might be interested (e.g., lab mailing lists). You might include links to the GitHub release page.
154
111
155
-
4. Tweet about the release from ``@scikit-bio``, including a link to the GitHub release page (for example, https://github.com/scikit-bio/scikit-bio/releases/tag/x.y.z). Post a similar message to `scikit-bio's Gitter <https://gitter.im/biocore/scikit-bio>`_.
112
+
4. Tweet about the release from ``@scikit-bio``, including a link to the GitHub release page (for example, https://github.com/scikit-bio/scikit-bio/releases/tag/x.y.z).
0 commit comments