What policy sets the minimum required astropy-iers-data version for Astropy tags?

[weaverba137]

When Astropy is installed a minimum version of astropy-iers-data is specified as a requirement. However, given that a typical install involving pip or conda will by default select the most recent version of astropy-iers-data, and that a typical installation will end up automatically downloading the IERS_A table after about 30 days, why is a minimum version constraint required on astropy-iers-data?

Furthermore, below is a table of important recent tags and the minimum astropy-iers-data version specified in the pyproject.toml file. Because there is some variation in the time period between the minimum astropy-iers-data version and the associated Astropy tag date, we were wondering if this was driven by a deliberate policy. If so, is that policy documented?

Tag	Tag Date	Minimum astropy-iers-data	Comment
6.0.0	2023-11-27	0.2023.10.30.0.29.53	Minimum version is about 1 month in the past.
6.1.0	2024-05-04	0.2024.4.29.0.28.48	Minimum version is about 1 week in the past.
7.0.0	2024-11-24	0.2024.10.28.0.34.7	Minimum version is about 1 month in the past.
7.1.0	2025-05-20	0.2025.4.28.0.37.27	Minimum version is less than 1 month but more than 1 week in the past.

In our use case (DESI), we would prefer to update astropy-iers-data only when we have assured the reproducibility of certain data products, rather than automatically, even if that means accepting that some amount of extrapolation past the default 30 days will be involved. Therefore, if we need to pin astropy-iers-data==0.2023.10.30.0.29.53, it would appear that it is impossible for us to install Astropy 7, or even 6.1.0. Certainly there could be workarounds, which is fine, but what is the recommended best practice for this situation?

To review our questions:

1. What is the motivation for the minimum required version?
2. How is the minimum required version for a given Astropy release actually set?
3. What is the best practice if we need to pin a much earlier version of astropy-iers-data?
4. Are there gotchas that we haven't considered?

CC: @sbailey

[pllim]

It is auto-updated by https://github.com/astropy/astropy/blob/main/.github/workflows/update_astropy_iers_data_pin.yml

[pllim]

Maybe @astrofrog or @mhvk can comment on the best practices and gotchas.

[weaverba137]

@pllim, thank you. If I understand that correctly then, the time gaps above do not reflect any deliberate policy, but simply when a release candidate was created, for example, and any work needed between a release candidate and a final tag?

[mhvk]

The point of having astropy-iers-data is to ensure that people do not have to start with downloading those data. While you are right that a pip install will generally give the latest version, so that there is no problem in that sense, if astropy-iers-data is already installed on the system, it will not automatically get updated. (Though I guess one could argue that in that case the automatic update will do it soon enough.)

I still do not quite understand, though, is why you would ever want to pin astropy-iers-data but update astropy itself. Those data include leap seconds, so it is not all small effects (of course computers update them independently of astropy too. And giving that I've seen wrong data and papers based on missing leap seconds, I'd really hesitate to do anything that risks users not having up-to-date data.

[astrofrog]

The reason we pin the minimum version and continuously update it is to ensure that when regular users update astropy, they also get an updated astropy-iers-data, otherwise it would never get updated and end up containing very old files.

[astrofrog]

we would prefer to update astropy-iers-data only when we have assured the reproducibility of certain data products

This seems reasonable but presumably the same applies to astropy too? In other words you can pin both astropy and astropy-iers-data and update both pins at the same time?

[weaverba137]

I'm going to need to get some input from the people actually running reproducibility tests. What I can say is that we need to be able to tightly control the environment in which IERS transforms are executed. Furthermore that control has to be extensible in some sense all the way back to around 2020 before astropy-iers-data even existed.

It's kind of hard to express this in terms of numerical precision because the real danger is that a target from 2020 does not get assigned to the same fiber in 2020 as it does today, and we are evaluating to what extent changes in astropy-iers-data cause or do not cause such a change.

[sbailey]

Further context: we are 5 years into an 8 year fiber-fed spectrograph redshift survey (DESI). To model systematics and efficiencies, we need to be able to reproduce the operations choices of which spectrograph fiber was assigned to which target going all the way back to the very beginning of the survey, and reproduce what would have been done given alternate universe inputs (sims/mocks).

In principle we could freeze the version of astropy and astropy-iers-data and live with that for a decade. In practice that is a pain because then all other analysis code that wraps around replaying those ops decisions also gets frozen to some ancient version of astropy (and numpy etc.) and doesn't benefit from recent work. Or if a security issue comes up that requires an upgrade, then we're toast. Even containers aren't a panacea here. Up until the astropy 6 -> 7 transition, we've been able to manage with a frozen set of IERS files and periodically upgrading astropy, while doing extensive reproducibility checks that there wasn't an additional algorithmic change impacting reproducibility.

If I understand the thread correctly, astropy versions include a constraint to a relatively recent astropy-iers-data because if a user has astropy + astropy-iers-data installed and upgrades astropy, then astropy-iers-data itself won't automatically also get upgraded unless there is a constraint. This is mitigated by the auto-download features, but it could leave users in a distasteful situation of relying entirely upon the auto-downloaded data which could disappear if the cache is cleared for other reasons. We are admittedly the corner case of purposefully wanting those old coefficients for reproducibility reasons.

At the same time, I'm not entirely sure that an old version of astropy-iers-data is so problematic — most users will rely upon the auto-download magic to keep them up-to-date, and if they are expert enough to know how to turn that off and brave enough to pass through the "if you really know what you are doing" warnings, then they are hopefully also expert enough to know if their installed version of astropy-iers-data is what they want or not. I'm completely sympathetic to the default being the most up-to-date coefficients, but we're looking for a bit more flexibility for power-users needing to control exactly which IERS coefficients are used over the timespan of many years.

Which brings us back to the core question: is there a recommended way to install a recent version of astropy while purposefully pinning to a fixed old version of astropy-iers-data even if that violates the formal package constraints? The astropy package constraints don't allow that by default, and even conda install --dry-run astropy-iers-data=0.2023.6.26.17.14.7 --no-deps --no-update-deps exited with failure complaining about constraints (more details in desihub/desiutil#224). From a strictly format-compatibility standpoint, it appears that older version of astropy-iers-data are compatible with recent versions of astropy; we just need to figure out how to manage the package installations to achieve that. Thanks for the help.

[astrofrog]

Which brings us back to the core question: is there a recommended way to install a recent version of astropy while purposefully pinning to a fixed old version of astropy-iers-data even if that violates the formal package constraints?

I'm not aware of a way to do that directly.

Note that prior to the existence of astropy-iers-data the static files included with astropy would have changed from version to version so in effect the current system is equivalent to what we were doing before.

I think the cleanest way to have a frozen set of IERS files is going to be to do:

import astropy_iers_data
astropy_iers_data.IERS_A_FILE = '<path to your file>'
astropy_iers_data.IERS_A_README = '<path to your file>'
astropy_iers_data.IERS_B_FILE = '<path to your file>'
astropy_iers_data.IERS_B_README = '<path to your file>'
astropy_iers_data.IERS_LEAP_SECOND_FILE = '<path to your file>'

This has to be done before importing astropy.

Or alternatively, if you want to support old versions of astropy pre-astropy-iers-data or be able to change things after importing astropy you can also do:

from astropy.utils import iers
iers.IERS_A_FILE = '<path to your file>'
iers.IERS_A_README = '<path to your file>'
iers.IERS_B_FILE = '<path to your file>'
iers.IERS_B_README = '<path to your file>'
iers.IERS_LEAP_SECOND_FILE = '<path to your file>'

An alternative approach would be for you to fork astropy-iers-data for local use only and include the static version of the files you want in there, and give it a high version number that will likely always satisfy the version check.

we've been able to manage with a frozen set of IERS files

How did you used to do this? Was it using a similar approach to above?

[neutrinoceros]

related to astropy/astropy#17788, where I raised other concerned with having constant upgrades in the minimum requirement on astropy-iers-data.
I'd like to bring up a couple other points that I think weakes the initial purpose of optimizing for environment upgrades:

• Python virtual envs are notoriously fragile. It's really not that common to use a single env and have it survive upgraded and additions for years.
• Modern tools (uv and pixi) make virtual envs much more disposable and easier to manage through complete destruction / re-creation cycles that are typically much faster (at least on a hot cache) than a pip freeze + pip install (or conda equivalent) cycle, so it's becoming less and less relevant to treat the latter as the default workflow.

In my opinion, a property of sane defaults is to not undermine alternative workflows. I feel like our current approach makes the easy thing trivial (good), but also makes experts struggle in unnecessary ways (very bad), which doesn't feel like a fair tradeoff.

[astrofrog]

Interestingly the reason we split out the data into its own package was in fact to cater for the advanced use case of ensuring that users could get things working properly without an internet connection. Prior to having the separate package users were forced anyway to have the static version of the files upgraded with astropy versions, so this hasn't changed.

In any case, the workaround suggested in #43 (comment) is not actually that terrible, it's using public API constants to change which file to use and it's an approach that works before and after the split.

[mhvk]

The method @astrofrog mentioned will definitely work, and one could take that even further and after installation of the iers data package, simply replace the tables in it on disk.

But actually, is this even needed? I just tried to make a virtual environment, and if after installing astropy, I install an older version of astropy-iers-data, I get an error, but things do happily proceed:

pip install astropy
pip install astropy-iers-data==0.2024.7.8.0.31.19
Collecting astropy-iers-data==0.2024.7.8.0.31.19
  Using cached astropy_iers_data-0.2024.7.8.0.31.19-py3-none-any.whl.metadata (5.1 kB)
Using cached astropy_iers_data-0.2024.7.8.0.31.19-py3-none-any.whl (1.9 MB)
Installing collected packages: astropy-iers-data
  Attempting uninstall: astropy-iers-data
    Found existing installation: astropy-iers-data 0.2025.6.30.0.39.40
    Uninstalling astropy-iers-data-0.2025.6.30.0.39.40:
      Successfully uninstalled astropy-iers-data-0.2025.6.30.0.39.40
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
astropy 7.1.0 requires astropy-iers-data>=0.2025.4.28.0.37.27, but you have astropy-iers-data 0.2024.7.8.0.31.19 which is incompatible.
Successfully installed astropy-iers-data-0.2024.7.8.0.31.19

(I checked that the older version is indeed used -- as long as conf.auto_download = False).

But, @sbailey, would it not be possible to simply use the standard method for setting a specific IERS table for a whole session, i.e., use earth_orientation_table.set(...); see https://docs.astropy.org/en/latest/api/astropy.utils.iers.earth_orientation_table.html
It was introduced more or less for your purpose, of being able to ensure a specific table was used, so one could, e.g., reproduce an older setup. If you do any DESI specific setup before running things, that might be a place where it would fit right in. (This is like the ability astropy provides to use older versions of physical constants.)

[weaverba137]

@astrofrog:

How did you used to do this? Was it using a similar approach to above?

The full details can be found in the module desiutil.iers. In summary: we created a specialized version of the IERS-A table (see the function desiutil.iers.update_iers().

We probably would have just continued to use that file, except that the format of the table has changed and Astropy 7 no longer accepts the format of the frozen file. Astropy 6 worked fine.

The other function desiutil.iers.freeze_iers() creates a subclass of IERS_B that points to the frozen table, then some of the classes in astropy.utils.iers are replaces with this "frozen" class. This also worked fine up through Astropy 6, but the class structure in Astropy 7 appears to have changed, so that no longer works either.

In addition freeze_iers() performs the recommended configuration changes to prevent downloads, which are still necessary no matter what.

The suggestion of replacing the files instead of the classes sounds promising, i.e.,

import astropy_iers_data
astropy_iers_data.IERS_A_FILE = '<path to your file>'
...

but why is it necessary to update the README files? Do they influence the computation or the metadata?

@mhvk:

We were not aware of earth_orientation_table. Although it is mentioned presently, e.g., https://docs.astropy.org/en/latest/utils/iers.html#utils-iers, it is possible that it did not exist or was not documented in 2020, at the time we had to make this work.

I also note that even now, earth_orientation_table is not mentioned in the Working offline section.

[neutrinoceros]

But actually, is this even needed? I just tried to make a virtual environment, and if after installing astropy, I install an older version of astropy-iers-data, I get an error, but things do happily proceed:

this works with an imperative workflow (pip install <stuff>), and arguably it shouldn't (I think pip does it for backward compatibility with the long gone time it didn't have a proper dependency tree solver) but doesn't with a declarative one (e.g. uv sync). In a declarative workflow, you need to actively fight your tools to make it work, and I don't see any particular reason why this approach shouldn't be supported as well.

[weaverba137]

@mhvk: @sbailey is not available today, but to answer your question about pip install with an old version of astropy-iers-data. You are probably correct, however he was using a conda environment, and I believe the install did not succeed there, although it is worth trying again.