Build fails with Python 3.13 - compatibility issues

[rohannsahh]

Description of the Issue

I'm trying to install the Thinc package with Python 3.13 but encountering build errors. The compilation process fails with multiple C API compatibility errors, suggesting that Thinc needs to be updated to support Python 3.13's API changes.

Environment Details

• Python version: 3.13
• Operating System: Windows
• Visual Studio version: 2022 BuildTools (14.43.34808)
• Installation method: pip install (from source)

Error Details

The compilation is failing with the following errors:

• _PyList_Extend: identifier not found
• _PyInterpreterState_GetConfig: identifier not found
• _PyLong_AsByteArray: function does not take 5 arguments
• Warnings about Py_UNICODE being deprecated in 3.13

Full Error Log

    writing manifest file 'thinc.egg-info\SOURCES.txt'
    C:\Users\ROHAN\AppData\Local\Temp\pip-build-env-0nfip899\overlay\Lib\site-packages\setuptools\command\build_py.py:212: _Warning: Package 'thinc.backends' is absent from the `packages` configuration.
    !!

            ********************************************************************************
            ############################
            # Package would be ignored #
            ############################
            Python recognizes 'thinc.backends' as an importable package[^1],
            but it is absent from setuptools' `packages` configuration.

            This leads to an ambiguous overall configuration. If you want to distribute this
            package, please make sure that 'thinc.backends' is explicitly added
            to the `packages` configuration field.

            Alternatively, you can also rely on setuptools' discovery methods
            (for example by using `find_namespace_packages(...)`/`find_namespace:`
            instead of `find_packages(...)`/`find:`).

            You can read more about "package discovery" on setuptools documentation page:

            - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

            If you don't want 'thinc.backends' to be distributed and are
            already explicitly excluding 'thinc.backends' via
            `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
            you can try to use `exclude_package_data`, or `include-package-data=False` in
            combination with a more fine grained `package-data` configuration.

            You can read more about "package data files" on setuptools documentation page:

            - https://setuptools.pypa.io/en/latest/userguide/datafiles.html


            [^1]: For Python, any directory (with suitable naming) can be imported,
                  even if it does not contain any `.py` files.
                  On the other hand, currently there is no concept of package data
                  directory, all directories are treated like packages.
            ********************************************************************************

    !!
      check.warn(importable)
    C:\Users\ROHAN\AppData\Local\Temp\pip-build-env-0nfip899\overlay\Lib\site-packages\setuptools\command\build_py.py:212: _Warning: Package 'thinc.extra' is absent from the `packages` configuration.
    !!

            ********************************************************************************
            ############################
            # Package would be ignored #
            ############################
            Python recognizes 'thinc.extra' as an importable package[^1],
            but it is absent from setuptools' `packages` configuration.

            This leads to an ambiguous overall configuration. If you want to distribute this
            package, please make sure that 'thinc.extra' is explicitly added
            to the `packages` configuration field.

            Alternatively, you can also rely on setuptools' discovery methods
            (for example by using `find_namespace_packages(...)`/`find_namespace:`
            instead of `find_packages(...)`/`find:`).

            You can read more about "package discovery" on setuptools documentation page:

            - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

            If you don't want 'thinc.extra' to be distributed and are
            already explicitly excluding 'thinc.extra' via
            `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
            you can try to use `exclude_package_data`, or `include-package-data=False` in
            combination with a more fine grained `package-data` configuration.

            You can read more about "package data files" on setuptools documentation page:

            - https://setuptools.pypa.io/en/latest/userguide/datafiles.html


            [^1]: For Python, any directory (with suitable naming) can be imported,
                  even if it does not contain any `.py` files.
                  On the other hand, currently there is no concept of package data
                  directory, all directories are treated like packages.
            ********************************************************************************

    !!
      check.warn(importable)
    C:\Users\ROHAN\AppData\Local\Temp\pip-build-env-0nfip899\overlay\Lib\site-packages\setuptools\command\build_py.py:212: _Warning: Package 'thinc.layers' is absent from the `packages` configuration.
    !!

            ********************************************************************************
            ############################
            # Package would be ignored #
            ############################
            Python recognizes 'thinc.layers' as an importable package[^1],
            but it is absent from setuptools' `packages` configuration.

            This leads to an ambiguous overall configuration. If you want to distribute this
            package, please make sure that 'thinc.layers' is explicitly added
            to the `packages` configuration field.

            Alternatively, you can also rely on setuptools' discovery methods
            (for example by using `find_namespace_packages(...)`/`find_namespace:`
            instead of `find_packages(...)`/`find:`).

            You can read more about "package discovery" on setuptools documentation page:

            - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

            If you don't want 'thinc.layers' to be distributed and are
            already explicitly excluding 'thinc.layers' via
            `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
            you can try to use `exclude_package_data`, or `include-package-data=False` in
            combination with a more fine grained `package-data` configuration.

            You can read more about "package data files" on setuptools documentation page:

            - https://setuptools.pypa.io/en/latest/userguide/datafiles.html


            [^1]: For Python, any directory (with suitable naming) can be imported,
                  even if it does not contain any `.py` files.
                  On the other hand, currently there is no concept of package data
                  directory, all directories are treated like packages.
            ********************************************************************************

    !!
      check.warn(importable)
    C:\Users\ROHAN\AppData\Local\Temp\pip-build-env-0nfip899\overlay\Lib\site-packages\setuptools\command\build_py.py:212: _Warning: Package 'thinc.tests.mypy.configs' is absent from the `packages` configuration.
    !!

            ********************************************************************************
            ############################
            # Package would be ignored #
            ############################
            Python recognizes 'thinc.tests.mypy.configs' as an importable package[^1],
            but it is absent from setuptools' `packages` configuration.

            This leads to an ambiguous overall configuration. If you want to distribute this
            package, please make sure that 'thinc.tests.mypy.configs' is explicitly added
            to the `packages` configuration field.

            Alternatively, you can also rely on setuptools' discovery methods
            (for example by using `find_namespace_packages(...)`/`find_namespace:`
            instead of `find_packages(...)`/`find:`).

            You can read more about "package discovery" on setuptools documentation page:

            - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

            If you don't want 'thinc.tests.mypy.configs' to be distributed and are
            already explicitly excluding 'thinc.tests.mypy.configs' via
            `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
            you can try to use `exclude_package_data`, or `include-package-data=False` in
            combination with a more fine grained `package-data` configuration.

            You can read more about "package data files" on setuptools documentation page:

            - https://setuptools.pypa.io/en/latest/userguide/datafiles.html


            [^1]: For Python, any directory (with suitable naming) can be imported,
                  even if it does not contain any `.py` files.
                  On the other hand, currently there is no concept of package data
                  directory, all directories are treated like packages.
            ********************************************************************************

    !!
      check.warn(importable)
    C:\Users\ROHAN\AppData\Local\Temp\pip-build-env-0nfip899\overlay\Lib\site-packages\setuptools\command\build_py.py:212: _Warning: Package 'thinc.tests.mypy.outputs' is absent from the `packages` configuration.
    !!

            ********************************************************************************
            ############################
            # Package would be ignored #
            ############################
            Python recognizes 'thinc.tests.mypy.outputs' as an importable package[^1],
            but it is absent from setuptools' `packages` configuration.

            This leads to an ambiguous overall configuration. If you want to distribute this
            package, please make sure that 'thinc.tests.mypy.outputs' is explicitly added
            to the `packages` configuration field.

            Alternatively, you can also rely on setuptools' discovery methods
            (for example by using `find_namespace_packages(...)`/`find_namespace:`
            instead of `find_packages(...)`/`find:`).

            You can read more about "package discovery" on setuptools documentation page:

            - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

            If you don't want 'thinc.tests.mypy.outputs' to be distributed and are
            already explicitly excluding 'thinc.tests.mypy.outputs' via
            `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
            you can try to use `exclude_package_data`, or `include-package-data=False` in
            combination with a more fine grained `package-data` configuration.

            You can read more about "package data files" on setuptools documentation page:

            - https://setuptools.pypa.io/en/latest/userguide/datafiles.html


            [^1]: For Python, any directory (with suitable naming) can be imported,
                  even if it does not contain any `.py` files.
                  On the other hand, currently there is no concept of package data
                  directory, all directories are treated like packages.
            ********************************************************************************

    !!
      check.warn(importable)
    copying thinc\__init__.pxd -> build\lib.win-amd64-cpython-313\thinc
    copying thinc\py.typed -> build\lib.win-amd64-cpython-313\thinc
    copying thinc\backends\cblas.cpp -> build\lib.win-amd64-cpython-313\thinc\backends
    copying thinc\backends\cpu_kernels.hh -> build\lib.win-amd64-cpython-313\thinc\backends
    copying thinc\backends\linalg.cpp -> build\lib.win-amd64-cpython-313\thinc\backends
    copying thinc\backends\numpy_ops.cpp -> build\lib.win-amd64-cpython-313\thinc\backends
    copying thinc\extra\search.cpp -> build\lib.win-amd64-cpython-313\thinc\extra
    copying thinc\layers\premap_ids.cpp -> build\lib.win-amd64-cpython-313\thinc\layers
    copying thinc\layers\sparselinear.cpp -> build\lib.win-amd64-cpython-313\thinc\layers
    copying thinc\backends\__init__.pxd -> build\lib.win-amd64-cpython-313\thinc\backends
    copying thinc\backends\_custom_kernels.cu -> build\lib.win-amd64-cpython-313\thinc\backends
    copying thinc\backends\_murmur3.cu -> build\lib.win-amd64-cpython-313\thinc\backends
    copying thinc\backends\cblas.pxd -> build\lib.win-amd64-cpython-313\thinc\backends
    copying thinc\backends\cblas.pyx -> build\lib.win-amd64-cpython-313\thinc\backends
    copying thinc\backends\linalg.pxd -> build\lib.win-amd64-cpython-313\thinc\backends
    copying thinc\backends\linalg.pyx -> build\lib.win-amd64-cpython-313\thinc\backends
    copying thinc\backends\numpy_ops.pxd -> build\lib.win-amd64-cpython-313\thinc\backends
    copying thinc\backends\numpy_ops.pyx -> build\lib.win-amd64-cpython-313\thinc\backends
    copying thinc\extra\__init__.pxd -> build\lib.win-amd64-cpython-313\thinc\extra
    copying thinc\extra\search.pxd -> build\lib.win-amd64-cpython-313\thinc\extra
    copying thinc\extra\search.pyx -> build\lib.win-amd64-cpython-313\thinc\extra
    copying thinc\layers\premap_ids.pyx -> build\lib.win-amd64-cpython-313\thinc\layers
    copying thinc\layers\sparselinear.pyx -> build\lib.win-amd64-cpython-313\thinc\layers
    copying thinc\extra\tests\c_test_search.pyx -> build\lib.win-amd64-cpython-313\thinc\extra\tests
    creating build\lib.win-amd64-cpython-313\thinc\tests\mypy\configs
    copying thinc\tests\mypy\configs\mypy-default.ini -> build\lib.win-amd64-cpython-313\thinc\tests\mypy\configs
    copying thinc\tests\mypy\configs\mypy-plugin.ini -> build\lib.win-amd64-cpython-313\thinc\tests\mypy\configs
    creating build\lib.win-amd64-cpython-313\thinc\tests\mypy\outputs
    copying thinc\tests\mypy\outputs\fail-no-plugin.txt -> build\lib.win-amd64-cpython-313\thinc\tests\mypy\outputs
    copying thinc\tests\mypy\outputs\fail-plugin.txt -> build\lib.win-amd64-cpython-313\thinc\tests\mypy\outputs
    copying thinc\tests\mypy\outputs\success-no-plugin.txt -> build\lib.win-amd64-cpython-313\thinc\tests\mypy\outputs
    copying thinc\tests\mypy\outputs\success-plugin.txt -> build\lib.win-amd64-cpython-313\thinc\tests\mypy\outputs
    running build_ext
    building 'thinc.backends.cblas' extension
    creating build\temp.win-amd64-cpython-313\Release\thinc\backends
    "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.43.34808\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\ROHAN\AppData\Local\Temp\pip-build-env-0nfip899\overlay\Lib\site-packages\numpy\_core\include -IC:\Users\ROHAN\AppData\Local\Programs\Python\Python313\Include -IC:\Users\ROHAN\OneDrive\Desktop\jaggery_backend\venv\include -IC:\Users\ROHAN\AppData\Local\Programs\Python\Python313\include -IC:\Users\ROHAN\AppData\Local\Programs\Python\Python313\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.43.34808\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.43.34808\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.26100.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" /EHsc /Tpthinc/backends/cblas.cpp /Fobuild\temp.win-amd64-cpython-313\Release\thinc\backends\cblas.obj /Ox /EHsc
    cblas.cpp
    thinc/backends/cblas.cpp(871): warning C4996: 'Py_UNICODE': deprecated in 3.13
    thinc/backends/cblas.cpp(872): warning C4996: 'Py_UNICODE': deprecated in 3.13
    thinc/backends/cblas.cpp(1908): error C3861: '_PyList_Extend': identifier not found
    thinc/backends/cblas.cpp(1946): error C3861: '_PyInterpreterState_GetConfig': identifier not found
    thinc/backends/cblas.cpp(20354): error C2660: '_PyLong_AsByteArray': function does not take 5 arguments
    C:\Users\ROHAN\AppData\Local\Programs\Python\Python313\Include\cpython/longobject.h(111): note: see declaration of '_PyLong_AsByteArray'
    thinc/backends/cblas.cpp(20354): note: while trying to match the argument list '(PyLongObject *, unsigned char *, size_t, int, bool)'
    thinc/backends/cblas.cpp(20550): error C2660: '_PyLong_AsByteArray': function does not take 5 arguments
    C:\Users\ROHAN\AppData\Local\Programs\Python\Python313\Include\cpython/longobject.h(111): note: see declaration of '_PyLong_AsByteArray'
    thinc/backends/cblas.cpp(20550): note: while trying to match the argument list '(PyLongObject *, unsigned char *, size_t, int, bool)'
    thinc/backends/cblas.cpp(20822): error C2660: '_PyLong_AsByteArray': function does not take 5 arguments
    C:\Users\ROHAN\AppData\Local\Programs\Python\Python313\Include\cpython/longobject.h(111): note: see declaration of '_PyLong_AsByteArray'
    thinc/backends/cblas.cpp(20822): note: while trying to match the argument list '(PyLongObject *, unsigned char *, size_t, int, bool)'
    error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.43.34808\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
    [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
    ERROR: Failed building wheel for thinc
  Failed to build thinc
  ERROR: Failed to build installable wheels for some pyproject.toml based projects (thinc)
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

Steps to Reproduce

1. Install Python 3.13
2. Create a virtual environment
3. Run pip install thinc

Impact

Unable to install Thinc with Python 3.13, which prevents using any dependent packages (like spaCy) with the latest Python version.

Potential Solution

The code needs to be updated to be compatible with Python 3.13's C API changes. This includes:

1. Updating calls to _PyLong_AsByteArray to match the new function signature
2. Finding alternatives for deprecated or removed functions like _PyList_Extend
3. Addressing the deprecation of Py_UNICODE

Additional Information

I confirmed the issue is with Python 3.13 specifically, as the package installs correctly with Python 3.11/3.12.

[joshdavham]

I'm also experiencing this issue.

I'd really like to be able to use Spacy with Python 3.13, but this specific dependency seems to be the current blocker.

[cbornet]

It's still impossible to use Spacy on Python 3.13 due to this issue.

[honnibal]

The Thinc 3.13 Windows wheels are there. What architecture are you using? Here's the list of wheels: https://pypi.org/project/thinc/8.3.6/#files

[cbornet]

They are missing for thinc 9+ which is the version resolved if you don’t put a version limit.

[honnibal]

spaCy is using v8.3 of Thinc though

[cbornet]

Yes, for spacy it's the lower limit of 8.3.4 that should be bumped.