unearth.errors.HashMismatchError after package replaced, how to fix?

[qris]

Describe the bug

Someone rewrote a package file on an internal server, changing its hash. Now every time I try to pdm install, even after deleting pdm.lock, I get this error (eventually):

Synchronizing working set with resolved packages: 0 to add, 1 to update, 0 to remove

  ✖ Update aqt 3.1.4 -> 3.1.5 failed
  ✖ Update aqt 3.1.4 -> 3.1.5 failed

ERRORS:
update aqt failed:
Traceback (most recent call last):
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/chris/pdm/lib/python3.10/site-packages/pdm/installers/synchronizers.py", line 50, in update_candidate
    self.manager.overwrite(dist, can)
  File "/home/chris/pdm/lib/python3.10/site-packages/pdm/installers/manager.py", line 63, in overwrite
    installed = self.install(candidate)
  File "/home/chris/pdm/lib/python3.10/site-packages/pdm/installers/manager.py", line 33, in install
    prepared.build(),
  File "/home/chris/pdm/lib/python3.10/site-packages/pdm/models/candidates.py", line 412, in build
    self._obtain(allow_all=False)
  File "/home/chris/pdm/lib/python3.10/site-packages/pdm/models/candidates.py", line 466, in _obtain
    self._unpack(validate_hashes=not allow_all)
  File "/home/chris/pdm/lib/python3.10/site-packages/pdm/models/candidates.py", line 480, in _unpack
    result = finder.download_and_unpack(
  File "/home/chris/pdm/lib/python3.10/site-packages/unearth/finder.py", line 452, in download_and_unpack
    file = unpack_link(
  File "/home/chris/pdm/lib/python3.10/site-packages/unearth/preparer.py", line 364, in unpack_link
    validator.validate()
  File "/home/chris/pdm/lib/python3.10/site-packages/unearth/preparer.py", line 145, in validate
    raise HashMismatchError(self.package_link, self.allowed, gots)
unearth.errors.HashMismatchError: Hash mismatch for http://x.x.x.x/aqt-3.1.5-py3-none-any.whl:
Expected(sha256): 81d0a0faf34267a4f67e20921b5da8e01e7e5c05646da82c095181fe2ad6d9f3
  Actual(sha256): 026874eaae5e70777f592a8cf32ca04dc01953b75a3ef9ae812b863fd9f5e97d

  0:00:09 ✖ Some package operations failed. 1/1
See /home/chris/.local/state/pdm/log/pdm-install-ovc0uzfw.log for detailed debug log.
[InstallationError]: Some package operations failed.

I have no idea how to fix this. I tried pip cache purge, rm pdm.lock and recreating the venv, but PDM somehow always finds the old hash, sticks it back into pdm.lock and then barfs when the downloaded file doesn't match it.

How can I force it to forget that old hash?

To reproduce

I thought something like this would work, but it doesn't, so something subtle must be happening:

• Install a package from a URL or path
• Modify the package file (at that URL or path)
• Recreate the virtualenv using that package

Expected Behavior

There should be some way to clear out the old cached hash so that I can reinstall the package.

Environment Information

PDM version:
2.25.9
Python Interpreter:
/home/chris/XXX/.venv/bin/python (3.10)
Project Root:
/home/chris/XXX
Local Packages:
/home/chris/XXX/.venv/lib/python3.10/site-packages
INFO: PDM 2.25.9 is installed, while 2.26.1 is available.
Please run pdm self update to upgrade.
Run pdm config check_update false to disable the check.
{
"implementation_name": "cpython",
"implementation_version": "3.10.19",
"os_name": "posix",
"platform_machine": "x86_64",
"platform_release": "6.14.0-33-generic",
"platform_system": "Linux",
"platform_version": "#33~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Sep 19 17:02:30 UTC 2",
"python_full_version": "3.10.19",
"platform_python_implementation": "CPython",
"python_version": "3.10",
"sys_platform": "linux"
}
INFO: PDM 2.25.9 is installed, while 2.26.1 is available.
Please run pdm self update to upgrade.
Run pdm config check_update false to disable the check.

Verbose Command Output

STATUS: Resolving packages from lockfile...
unearth.preparer: The file . is a local directory, use it directly
Synchronizing working set with resolved packages: 1 to add, 0 to update, 0 to remove

unearth.preparer: Downloading <Link http://x.x.x.x/data/aqt_releases/aqt-3.1.5-py3-none-any.whl (from None)> (2.2 MB)
✖ Install aqt 3.1.5 failed
pdm.termui: Error occurs adding aqt:
Traceback (most recent call last):
File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/chris/pdm/lib/python3.10/site-packages/pdm/installers/synchronizers.py", line 29, in install_candidate
self.manager.install(can)
File "/home/chris/pdm/lib/python3.10/site-packages/pdm/installers/manager.py", line 33, in install
prepared.build(),
File "/home/chris/pdm/lib/python3.10/site-packages/pdm/models/candidates.py", line 412, in build
self._obtain(allow_all=False)
File "/home/chris/pdm/lib/python3.10/site-packages/pdm/models/candidates.py", line 466, in _obtain
self._unpack(validate_hashes=not allow_all)
File "/home/chris/pdm/lib/python3.10/site-packages/pdm/models/candidates.py", line 480, in _unpack
result = finder.download_and_unpack(
File "/home/chris/pdm/lib/python3.10/site-packages/unearth/finder.py", line 452, in download_and_unpack
file = unpack_link(
File "/home/chris/pdm/lib/python3.10/site-packages/unearth/preparer.py", line 364, in unpack_link
validator.validate()
File "/home/chris/pdm/lib/python3.10/site-packages/unearth/preparer.py", line 145, in validate
raise HashMismatchError(self.package_link, self.allowed, gots)
unearth.errors.HashMismatchError: Hash mismatch for http://10.40.65.11/data/aqt_releases/aqt-3.1.5-py3-none-any.whl:
Expected(sha256): 81d0a0faf34267a4f67e20921b5da8e01e7e5c05646da82c095181fe2ad6d9f3
Actual(sha256): 026874eaae5e70777f592a8cf32ca04dc01953b75a3ef9ae812b863fd9f5e97d
Retry failed jobs(2/2)
unearth.preparer: Downloading <Link http://x.x.x.x/data/aqt_releases/aqt-3.1.5-py3-none-any.whl (from None)> (2.2 MB)
✖ Install aqt 3.1.5 failed
pdm.termui: Error occurs adding aqt:
Traceback (most recent call last):
File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/chris/pdm/lib/python3.10/site-packages/pdm/installers/synchronizers.py", line 29, in install_candidate
self.manager.install(can)
File "/home/chris/pdm/lib/python3.10/site-packages/pdm/installers/manager.py", line 33, in install
prepared.build(),
File "/home/chris/pdm/lib/python3.10/site-packages/pdm/models/candidates.py", line 412, in build
self._obtain(allow_all=False)
File "/home/chris/pdm/lib/python3.10/site-packages/pdm/models/candidates.py", line 466, in _obtain
self._unpack(validate_hashes=not allow_all)
File "/home/chris/pdm/lib/python3.10/site-packages/pdm/models/candidates.py", line 480, in _unpack
result = finder.download_and_unpack(
File "/home/chris/pdm/lib/python3.10/site-packages/unearth/finder.py", line 452, in download_and_unpack
file = unpack_link(
File "/home/chris/pdm/lib/python3.10/site-packages/unearth/preparer.py", line 364, in unpack_link
validator.validate()
File "/home/chris/pdm/lib/python3.10/site-packages/unearth/preparer.py", line 145, in validate
raise HashMismatchError(self.package_link, self.allowed, gots)
unearth.errors.HashMismatchError: Hash mismatch for http://10.40.65.11/data/aqt_releases/aqt-3.1.5-py3-none-any.whl:
Expected(sha256): 81d0a0faf34267a4f67e20921b5da8e01e7e5c05646da82c095181fe2ad6d9f3
Actual(sha256): 026874eaae5e70777f592a8cf32ca04dc01953b75a3ef9ae812b863fd9f5e97d
✖ Some package operations failed.
Traceback (most recent call last):
File "/home/chris/pdm/bin/pdm", line 8, in
sys.exit(main())
File "/home/chris/pdm/lib/python3.10/site-packages/pdm/core.py", line 391, in main
return core.main(args or sys.argv[1:])
File "/home/chris/pdm/lib/python3.10/site-packages/pdm/core.py", line 270, in main
raise cast(Exception, err).with_traceback(traceback) from None
File "/home/chris/pdm/lib/python3.10/site-packages/pdm/core.py", line 265, in main
self.handle(project, options)
File "/home/chris/pdm/lib/python3.10/site-packages/pdm/core.py", line 195, in handle
command.handle(project, options)
File "/home/chris/pdm/lib/python3.10/site-packages/pdm/cli/commands/install.py", line 103, in handle
actions.do_sync(
File "/home/chris/pdm/lib/python3.10/site-packages/pdm/cli/actions.py", line 311, in do_sync
synchronizer.synchronize()
File "/home/chris/pdm/lib/python3.10/site-packages/pdm/installers/synchronizers.py", line 227, in synchronize
raise InstallationError("Some package operations failed.")
pdm.exceptions.InstallationError: Some package operations failed.

Additional Context

No response

Are you willing to submit a PR to fix this bug?

• Yes, I would like to submit a PR.

[qris]

I was able to find the bad hash in /home/chris/.cache/pdm/hashes/63/f0/d0/30/bcb45f314c300c17dcd36afac4129d175e847c01b8f07f7f and delete that file and then pdm install. So I guess that pdm cache clear hashes would have worked. Maybe the error message could have told me that?

[monchin]

I guess you could have a try to relock your project to generate a new pdm.lock, and see whether the hash value is changed to the newest. Remember to clear pdm caches before relocking.

And if the package is modified on your private pypi mirror, you must rebuild the package to get a new hash value. Modifying .py files would not change the hash value saved in the package's metadata. For example, if you're using devpi, and you want to modify your package in the same version, you must delete the package on your mirror forcely, and rebuild and publish your package. Of course a better way is to upgrade the package's version and build a new one.

[frostming]

There is a convenient command to update all hashes: pdm lock --refresh