Breaking update to "rapidfuzz" dependency for Python <3.14 builds

[Stephane-Ag]

Description

RapidFuzz just had a release (v3.14.0) that breaks builds using Poetry when using Python <3.14.

This package is a dependency of cleo. It seems the cleo version removing rapidfuzz has been yanked (v2.2.0) so please pin RapidFuzz to v3.13.0 for now then? Seems like the only solution.

Workarounds

Will try to just manually install RapidFuzz v3.13.0 after installing poetry.

Poetry Installation Method

install.python-poetry.org

Operating System

macOS 15.6.1

Poetry Version

Poetry (version 2.0.1)

Poetry Configuration

cache-dir = "/Users/snouafo/Library/Caches/pypoetry"
installer.max-workers = null
installer.no-binary = null
installer.only-binary = null
installer.parallel = true
installer.re-resolve = true
keyring.enabled = true
requests.max-retries = 0
solver.lazy-wheel = true
system-git-client = false
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /Users/snouafo/Library/Caches/pypoetry/virtualenvs
virtualenvs.prompt = "{project_name}-py{python_version}"
virtualenvs.use-poetry-python = false

Python Sysconfig

sysconfig.log

Paste the output of 'python -m sysconfig', over this line.

Example pyproject.toml

Poetry Runtime Logs

poetry-runtime.log

Paste the output of 'poetry -vvv <command>', over this line.

[Secrus]

Could you provide more info about what is breaking in that release? I don't see any breaking changes in the changelog

[Stephane-Ag]

@Secrus Yes, here's a sample of the build output.

RUN poetry install --without dev --no-root     && pip install bytecode     && rm pyproject.toml poetry.lock
#33 0.160 Traceback (most recent call last):
#33 0.160   File "/usr/local/bin/poetry", line 3, in <module>
#33 0.161     from poetry.console.application import main
#33 0.161   File "/etc/poetry/venv/lib/python3.12/site-packages/poetry/console/application.py", line 13, in <module>
#33 0.162     from cleo.application import Application as BaseApplication
#33 0.163   File "/etc/poetry/venv/lib/python3.12/site-packages/cleo/application.py", line 11, in <module>
#33 0.163     from cleo.commands.completions_command import CompletionsCommand
#33 0.164   File "/etc/poetry/venv/lib/python3.12/site-packages/cleo/commands/completions_command.py", line 15, in <module>
#33 0.164     from cleo import helpers
#33 0.164   File "/etc/poetry/venv/lib/python3.12/site-packages/cleo/helpers.py", line 5, in <module>
#33 0.165     from cleo.io.inputs.argument import Argument
#33 0.167   File "/etc/poetry/venv/lib/python3.12/site-packages/cleo/io/inputs/argument.py", line 5, in <module>
#33 0.167     from cleo.exceptions import CleoLogicError
#33 0.167   File "/etc/poetry/venv/lib/python3.12/site-packages/cleo/exceptions/__init__.py", line 3, in <module>
#33 0.167     from cleo._utils import find_similar_names
#33 0.167   File "/etc/poetry/venv/lib/python3.12/site-packages/cleo/_utils.py", line 8, in <module>
#33 0.167     from rapidfuzz.distance import Levenshtein
#33 0.167   File "/etc/poetry/venv/lib/python3.12/site-packages/rapidfuzz/__init__.py", line 11, in <module>
#33 0.167     from rapidfuzz import distance, fuzz, process, utils
#33 0.167   File "/etc/poetry/venv/lib/python3.12/site-packages/rapidfuzz/distance/__init__.py", line 6, in <module>
#33 0.167     from . import (
#33 0.167   File "/etc/poetry/venv/lib/python3.12/site-packages/rapidfuzz/distance/OSA.py", line 9, in <module>
#33 0.167     from rapidfuzz._feature_detector import AVX2, SSE2, supports
#33 0.167   File "/etc/poetry/venv/lib/python3.12/site-packages/rapidfuzz/_feature_detector.py", line 8, in <module>
#33 0.167     from rapidfuzz._feature_detector_cpp import AVX2, SSE2, supports
#33 0.167   File "src/rapidfuzz/_feature_detector_cpp.pyx", line 1, in init rapidfuzz._feature_detector_cpp
#33 0.167 TypeError: Metaclasses with custom tp_new are not supported.
#33 ERROR: process "/bin/sh -c poetry install --without dev --no-root     && pip install bytecode     && rm pyproject.toml poetry.lock" did not complete successfully: exit code: 1

Adding this just before the above poetry install command resolved it for me for now.

RUN rm -rf /etc/poetry/venv/lib/python3.12/site-packages/rapidfuzz/
RUN pip install --target /etc/poetry/venv/lib/python3.12/site-packages rapidfuzz==3.13.0

[Stephane-Ag]

It seems we've come across this problem before with "msgpack" as well (dependency of "cachecontrol") and had to pin that to v1.1.0.

I believe the issue was introduced by Cython v3.1.1

[Secrus]

@maxbachmann could you take a look? Seems like unintentional breaking change from Rapidfuzz

[maxbachmann]

Is someone aware of what exactly broke/changed in cython?
The release updated from a cython 3.0.12 version to 3.1.3. So the mentioned v3.1.1 falls into that window.

The file it's complaining about is incredibly simple: https://github.com/rapidfuzz/RapidFuzz/blob/a79f3a28dc2ed32b1a6f1eb60f87f2b7b9f66cc2/src/rapidfuzz/_feature_detector_cpp.pyx

[Secrus]

Well, I found some time to do some testing of my own and can't reproduce the error. @Stephane-Ag could you provide a complete Dockerfile with reproduction?

[maxbachmann]

What is the hardware architecture? Is this apple x64 or arm?

[da-woods]

Can you check to make sure you aren't using a pre-release of Python?

I can reproduce this on Python 3.12 beta 2 but not on 3.12.0 onwards.

Specifically this was fixed in Python in 3.12 beta 3: python/cpython#105386