Skip to content

Enable pytest-asyncio tests in CI #92

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
May 6, 2025
Merged
13 changes: 6 additions & 7 deletions .github/workflows/test-python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,19 +56,18 @@ jobs:
python-version: ${{ matrix.python-version }}

- name: Install a specific version of uv
uses: astral-sh/setup-uv@v3
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume enable-cache is true by default in the latest setup-uv?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, default was changed to enable-cache: true since setup-uv v5, see https://github.com/astral-sh/setup-uv/releases/tag/v5.0.0

version: "0.5.x"
version: "latest"

- name: uv sync
working-directory: python
run: uv sync --no-install-package geoindex-rs
run: uv sync --no-install-package async-tiff

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like the line above this is incorrect

Copy link
Member Author

@weiji14 weiji14 May 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean uv sync --no-install-package geoindex-rs should be just uv sync?

Edit: Just removed the entire uv sync ... line at commit 638d602, and it seems to run fine.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, it needs to be uv sync --no-install-package async-tiff. And then the lines after that still need the --no-project.

You see this line: https://github.com/developmentseed/async-tiff/actions/runs/14786441598/job/41515604193#step:7:19

When you run uv sync it builds the rust code in release mode as part of the current project. For fastest CI we want to build and test in debug mode, not release mode. That means we want to tell uv not to automatically build async-tiff, because we'll manually build it after with maturin develop

Copy link
Member Author

@weiji14 weiji14 May 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I think you're right. Checking the timings:

So let's do uv sync --no-install-package async-tiff && uv run --no-project maturin develop.

Copy link
Member Author

@weiji14 weiji14 May 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final timings at commit aa21368, job https://github.com/developmentseed/async-tiff/actions/runs/14825301405/job/41617636468:

  • uv sync --no-install-package async-tiff = 3s
  • uv run --no-project maturin develop = 46s
  • Total: 49s 🚀

- name: maturin venv Build
working-directory: python
run: uv run --no-project maturin develop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The --no-project was here because I thought whenever we ran uv sync or uv run, uv would automatically build the current project, which would compile the rust code and take a while.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I though if the working directory is set to python/ already, then it should only compile what's inside the python/ folder? Don't we need to compile the rust code anyway before compiling the pyo3 code? A little confused here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not the cargo project that's the issue. It's the uv project. We don't want uv automatically building the rust parts of async-tiff in release mode.


# - name: Run pytest
# working-directory: python
# run: uv run --no-project pytest
- name: Run pytest
working-directory: python
run: uv run --no-project pytest --verbose
7 changes: 6 additions & 1 deletion python/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,12 @@ dev-dependencies = [
"numpy>=1",
"obstore>=0.5.1",
"pip>=24.2",
"pytest-asyncio>=0.24.0",
"pytest-asyncio>=0.26.0",
"pytest>=8.3.3",
"ruff>=0.8.4",
]

[tool.pytest.ini_options]
addopts = "--color=yes"
asyncio_default_fixture_loop_scope="function"
asyncio_mode = "auto"
40 changes: 19 additions & 21 deletions python/tests/test_cog.py
Original file line number Diff line number Diff line change
@@ -1,27 +1,25 @@
import async_tiff
from time import time
from async_tiff import TIFF
from async_tiff import TIFF, enums
from async_tiff.store import S3Store

store = S3Store("sentinel-cogs", region="us-west-2", skip_signature=True)
path = "sentinel-s2-l2a-cogs/12/S/UF/2022/6/S2B_12SUF_20220609_0_L2A/B04.tif"

tiff = await TIFF.open(path, store=store, prefetch=32768)
async def test_cog_s3():
"""
Ensure that TIFF.open can open a Sentinel-2 Cloud-Optimized GeoTIFF file from an
s3 bucket, read IFDs and GeoKeyDirectory metadata.
"""
path = "sentinel-s2-l2a-cogs/12/S/UF/2022/6/S2B_12SUF_20220609_0_L2A/B04.tif"
store = S3Store("sentinel-cogs", region="us-west-2", skip_signature=True)
tiff = await TIFF.open(path=path, store=store, prefetch=32768)

start = time()
tiff = await TIFF.open(path, store=store, prefetch=32768)
end = time()
end - start
ifds = tiff.ifds
assert len(ifds) == 5

ifds = tiff.ifds
ifd = ifds[0]
ifd.compression
ifd.tile_height
ifd.tile_width
ifd.photometric_interpretation
gkd = ifd.geo_key_directory
gkd.citation
gkd.projected_type
gkd.citation
ifd = ifds[0]
assert ifd.compression == enums.CompressionMethod.Deflate
assert ifd.tile_height == 1024
assert ifd.tile_width == 1024
assert ifd.photometric_interpretation == enums.PhotometricInterpretation.BlackIsZero

dir(gkd)
gkd = ifd.geo_key_directory
assert gkd.citation == "WGS 84 / UTM zone 12N"
assert gkd.projected_type == 32612
10 changes: 6 additions & 4 deletions python/uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.