Integrity doesn't match to checksum: not decompressing the xz stream? #128
Description
Hello, I'm a first-time user of apt-offline, and I'm having problems running apt-offline install (for just an apt update) on a Debian 10 system. I was able to do an apt-offline get just fine on an online system, but it doesn't appear to install properly on the offline system.
If I run apt-offline install ./apt-offline-download, I get the following output:
Proceeding with installation
deb.debian.org_debian_dists_buster-updates_Release.gpg synced.
deb.debian.org_debian_dists_buster-updates_Release synced.
deb.debian.org_debian_dists_buster_Release.gpg synced.
deb.debian.org_debian_dists_buster_Release synced.
security.debian.org_dists_buster_updates_Release.gpg synced.
security.debian.org_dists_buster_updates_Release synced.
At first glance, that seemed to work. However, I noticed that apt was unaware of any of those packages. After running the same command with the --verbose flag, I got some errors saying that the integrity doesn't match to the SHA-256 checksum:
VERBOSE: /tmp/tmpxz6u9k71/security.debian.org_dists_buster_updates_Release is gpg clean
VERBOSE: This error should be ignored
VERBOSE: localFile /tmp/tmpxz6u9k71/deb.debian.org_debian_dists_buster-updates_contrib_binary-all_Packages integrity doesn't match to checksum e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
VERBOSE: localFile /tmp/tmpxz6u9k71/deb.debian.org_debian_dists_buster-updates_contrib_binary-amd64_Packages integrity doesn't match to checksum e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
VERBOSE: localFile /tmp/tmpxz6u9k71/deb.debian.org_debian_dists_buster-updates_contrib_i18n_Translation-en integrity doesn't match to checksum e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
VERBOSE: localFile /tmp/tmpxz6u9k71/deb.debian.org_debian_dists_buster-updates_main_binary-all_Packages integrity doesn't match to checksum c7706e3a133853b6483914bb0e299c1ba75dd4f2639af6b19e44ba2edb3ca757
VERBOSE: localFile /tmp/tmpxz6u9k71/deb.debian.org_debian_dists_buster-updates_main_binary-amd64_Packages integrity doesn't match to checksum 702ed30a36d81d58720cf0ab522eaa985cb404f8e93390ea3b754d1c8fafde54
VERBOSE: localFile /tmp/tmpxz6u9k71/deb.debian.org_debian_dists_buster-updates_main_i18n_Translation-en integrity doesn't match to checksum a3f4d96f88291faafda9c92f54f1451ee61e614da7be7bf9917237048c9a2944
VERBOSE: localFile /tmp/tmpxz6u9k71/deb.debian.org_debian_dists_buster_contrib_binary-all_Packages integrity doesn't match to checksum e6b87699e24a5a1333348baacf3c2d47c82afe2839b43e8c6bf6df34ce245503
VERBOSE: localFile /tmp/tmpxz6u9k71/deb.debian.org_debian_dists_buster_contrib_binary-amd64_Packages integrity doesn't match to checksum 5bb3c7b795311c23d7d35032c7c9f2fb71eb6d5e3be143afdf361ad7e02b4eba
VERBOSE: localFile /tmp/tmpxz6u9k71/deb.debian.org_debian_dists_buster_contrib_dep11_Components-amd64.yml integrity doesn't match to checksum b70f521acc48449b13a9a673cf73d721448ea160b38424fbf42c8933f3e2ff74
VERBOSE: localFile /tmp/tmpxz6u9k71/deb.debian.org_debian_dists_buster_contrib_i18n_Translation-en integrity doesn't match to checksum ff58ab35d0278a628b60538cfd7e92dfc6df7fb3037221c319e16ba13b607212
VERBOSE: localFile /tmp/tmpxz6u9k71/deb.debian.org_debian_dists_buster_main_binary-all_Packages integrity doesn't match to checksum cf1130e578679181985a4cbd889535f13c322043247ff2eea935baa95da1554e
VERBOSE: localFile /tmp/tmpxz6u9k71/deb.debian.org_debian_dists_buster_main_binary-amd64_Packages integrity doesn't match to checksum c1e3ee7cd5296078ed11221612d3bdb0ccc100aa01b1319d48bf671ba30056f7
VERBOSE: localFile /tmp/tmpxz6u9k71/deb.debian.org_debian_dists_buster_main_dep11_Components-amd64.yml integrity doesn't match to checksum ae1ada6294fcf57e0d7f45401fabe4e2b92baa877c47abfc48848de02c252ccf
VERBOSE: localFile /tmp/tmpxz6u9k71/deb.debian.org_debian_dists_buster_main_i18n_Translation-en integrity doesn't match to checksum d988de623d6a7dbf6071234bb3f6c0e2a3c8230600c8bb2d935b448e6fb44c72
VERBOSE: localFile /tmp/tmpxz6u9k71/security.debian.org_dists_buster_updates_contrib_binary-all_Packages integrity doesn't match to checksum e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
VERBOSE: localFile /tmp/tmpxz6u9k71/security.debian.org_dists_buster_updates_contrib_binary-amd64_Packages integrity doesn't match to checksum e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
VERBOSE: localFile /tmp/tmpxz6u9k71/security.debian.org_dists_buster_updates_contrib_i18n_Translation-en integrity doesn't match to checksum e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
VERBOSE: localFile /tmp/tmpxz6u9k71/security.debian.org_dists_buster_updates_main_binary-all_Packages integrity doesn't match to checksum 73e3804742eb381692d17980941ff5c8053d39e0923035167b2fe4097b8ce316
VERBOSE: localFile /tmp/tmpxz6u9k71/security.debian.org_dists_buster_updates_main_binary-amd64_Packages integrity doesn't match to checksum a1922615af348d78230e26774d60c419b033cacb649c19ffb5c910d5195ace73
VERBOSE: localFile /tmp/tmpxz6u9k71/security.debian.org_dists_buster_updates_main_i18n_Translation-en integrity doesn't match to checksum b99f6bec6e7484bbf74f45f5d497a0de9e75d37d336daed0dd4cb702eb84dfbf
VERBOSE: File is deb.debian.org_debian_dists_buster-updates_Release.gpg, hence 'True'.
VERBOSE: deb.debian.org_debian_dists_buster-updates_Release.gpg file synced to APT.
deb.debian.org_debian_dists_buster-updates_Release.gpg synced.
VERBOSE: deb.debian.org_debian_dists_buster-updates_Release file synced to APT.
deb.debian.org_debian_dists_buster-updates_Release synced.
VERBOSE: File is deb.debian.org_debian_dists_buster_Release.gpg, hence 'True'.
VERBOSE: deb.debian.org_debian_dists_buster_Release.gpg file synced to APT.
deb.debian.org_debian_dists_buster_Release.gpg synced.
VERBOSE: deb.debian.org_debian_dists_buster_Release file synced to APT.
deb.debian.org_debian_dists_buster_Release synced.
VERBOSE: File is security.debian.org_dists_buster_updates_Release.gpg, hence 'True'.
VERBOSE: security.debian.org_dists_buster_updates_Release.gpg file synced to APT.
security.debian.org_dists_buster_updates_Release.gpg synced.
VERBOSE: security.debian.org_dists_buster_updates_Release file synced to APT.
security.debian.org_dists_buster_updates_Release synced.
Taking a closer look at the file hashes, see that a common one in the output above is e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855. That is the empty hash. I don't have any empty files in the apt-offline-download folder, though, and the temporary folder it is working in seems to have the same files as those in my apt-offline-download folder.
The first file (with that hash above), which in this case is /tmp/tmpxz6u9k71/deb.debian.org_debian_dists_buster-updates_contrib_binary-all_Packages, is 32 bytes in size. It turns out that all of these files are compressed with an xz stream, and I'm guessing the issue is that apt-offline is not decompressing them. Decompressing the stream by running xz -d [filename] shows that it decompresses to the empty file, which has the current hash and probably would have been accepted by apt-offline.
So I think this is where the error with not actually syncing comes from.
Here is the signature in question:
'http://security.debian.org/dists/buster/updates/InRelease' security.debian.org_dists_buster_updates_InRelease 0
'http://security.debian.org/dists/buster/updates/main/binary-amd64/Packages.xz' security.debian.org_dists_buster_updates_main_binary-amd64_Packages 0
'http://security.debian.org/dists/buster/updates/main/binary-all/Packages.xz' security.debian.org_dists_buster_updates_main_binary-all_Packages 0
'http://security.debian.org/dists/buster/updates/main/i18n/Translation-en_US.xz' security.debian.org_dists_buster_updates_main_i18n_Translation-en%5fUS 0
'http://security.debian.org/dists/buster/updates/main/i18n/Translation-en.xz' security.debian.org_dists_buster_updates_main_i18n_Translation-en 0
'http://security.debian.org/dists/buster/updates/main/dep11/Components-amd64.yml.xz' security.debian.org_dists_buster_updates_main_dep11_Components-amd64.yml 0
'http://security.debian.org/dists/buster/updates/main/dep11/Components-all.yml.xz' security.debian.org_dists_buster_updates_main_dep11_Components-all.yml 0
'http://security.debian.org/dists/buster/updates/main/dep11/icons-48x48.tar.xz' security.debian.org_dists_buster_updates_main_dep11_icons-48x48.tar 0
'http://security.debian.org/dists/buster/updates/main/dep11/icons-64x64.tar.xz' security.debian.org_dists_buster_updates_main_dep11_icons-64x64.tar 0
'http://security.debian.org/dists/buster/updates/contrib/binary-amd64/Packages.xz' security.debian.org_dists_buster_updates_contrib_binary-amd64_Packages 0
'http://security.debian.org/dists/buster/updates/contrib/binary-all/Packages.xz' security.debian.org_dists_buster_updates_contrib_binary-all_Packages 0
'http://security.debian.org/dists/buster/updates/contrib/i18n/Translation-en_US.xz' security.debian.org_dists_buster_updates_contrib_i18n_Translation-en%5fUS 0
'http://security.debian.org/dists/buster/updates/contrib/i18n/Translation-en.xz' security.debian.org_dists_buster_updates_contrib_i18n_Translation-en 0
'http://security.debian.org/dists/buster/updates/contrib/dep11/Components-amd64.yml.xz' security.debian.org_dists_buster_updates_contrib_dep11_Components-amd64.yml 0
'http://security.debian.org/dists/buster/updates/contrib/dep11/Components-all.yml.xz' security.debian.org_dists_buster_updates_contrib_dep11_Components-all.yml 0
'http://security.debian.org/dists/buster/updates/contrib/dep11/icons-48x48.tar.xz' security.debian.org_dists_buster_updates_contrib_dep11_icons-48x48.tar 0
'http://security.debian.org/dists/buster/updates/contrib/dep11/icons-64x64.tar.xz' security.debian.org_dists_buster_updates_contrib_dep11_icons-64x64.tar 0
'http://deb.debian.org/debian/dists/buster/InRelease' deb.debian.org_debian_dists_buster_InRelease 0
'http://deb.debian.org/debian/dists/buster-updates/InRelease' deb.debian.org_debian_dists_buster-updates_InRelease 0
'http://deb.debian.org/debian/dists/buster/main/binary-amd64/Packages.xz' deb.debian.org_debian_dists_buster_main_binary-amd64_Packages 0
'http://deb.debian.org/debian/dists/buster/main/binary-all/Packages.xz' deb.debian.org_debian_dists_buster_main_binary-all_Packages 0
'http://deb.debian.org/debian/dists/buster/main/i18n/Translation-en_US.xz' deb.debian.org_debian_dists_buster_main_i18n_Translation-en%5fUS 0
'http://deb.debian.org/debian/dists/buster/main/i18n/Translation-en.xz' deb.debian.org_debian_dists_buster_main_i18n_Translation-en 0
'http://deb.debian.org/debian/dists/buster/main/dep11/Components-amd64.yml.xz' deb.debian.org_debian_dists_buster_main_dep11_Components-amd64.yml 0
'http://deb.debian.org/debian/dists/buster/main/dep11/Components-all.yml.xz' deb.debian.org_debian_dists_buster_main_dep11_Components-all.yml 0
'http://deb.debian.org/debian/dists/buster/main/dep11/icons-48x48.tar.xz' deb.debian.org_debian_dists_buster_main_dep11_icons-48x48.tar 0
'http://deb.debian.org/debian/dists/buster/main/dep11/icons-64x64.tar.xz' deb.debian.org_debian_dists_buster_main_dep11_icons-64x64.tar 0
'http://deb.debian.org/debian/dists/buster/contrib/binary-amd64/Packages.xz' deb.debian.org_debian_dists_buster_contrib_binary-amd64_Packages 0
'http://deb.debian.org/debian/dists/buster/contrib/binary-all/Packages.xz' deb.debian.org_debian_dists_buster_contrib_binary-all_Packages 0
'http://deb.debian.org/debian/dists/buster/contrib/i18n/Translation-en_US.xz' deb.debian.org_debian_dists_buster_contrib_i18n_Translation-en%5fUS 0
'http://deb.debian.org/debian/dists/buster/contrib/i18n/Translation-en.xz' deb.debian.org_debian_dists_buster_contrib_i18n_Translation-en 0
'http://deb.debian.org/debian/dists/buster/contrib/dep11/Components-amd64.yml.xz' deb.debian.org_debian_dists_buster_contrib_dep11_Components-amd64.yml 0
'http://deb.debian.org/debian/dists/buster/contrib/dep11/Components-all.yml.xz' deb.debian.org_debian_dists_buster_contrib_dep11_Components-all.yml 0
'http://deb.debian.org/debian/dists/buster/contrib/dep11/icons-48x48.tar.xz' deb.debian.org_debian_dists_buster_contrib_dep11_icons-48x48.tar 0
'http://deb.debian.org/debian/dists/buster/contrib/dep11/icons-64x64.tar.xz' deb.debian.org_debian_dists_buster_contrib_dep11_icons-64x64.tar 0
'http://deb.debian.org/debian/dists/buster-updates/main/binary-amd64/Packages.xz' deb.debian.org_debian_dists_buster-updates_main_binary-amd64_Packages 0
'http://deb.debian.org/debian/dists/buster-updates/main/binary-all/Packages.xz' deb.debian.org_debian_dists_buster-updates_main_binary-all_Packages 0
'http://deb.debian.org/debian/dists/buster-updates/main/i18n/Translation-en_US.xz' deb.debian.org_debian_dists_buster-updates_main_i18n_Translation-en%5fUS 0
'http://deb.debian.org/debian/dists/buster-updates/main/i18n/Translation-en.xz' deb.debian.org_debian_dists_buster-updates_main_i18n_Translation-en 0
'http://deb.debian.org/debian/dists/buster-updates/main/dep11/Components-amd64.yml.xz' deb.debian.org_debian_dists_buster-updates_main_dep11_Components-amd64.yml 0
'http://deb.debian.org/debian/dists/buster-updates/main/dep11/Components-all.yml.xz' deb.debian.org_debian_dists_buster-updates_main_dep11_Components-all.yml 0
'http://deb.debian.org/debian/dists/buster-updates/main/dep11/icons-48x48.tar.xz' deb.debian.org_debian_dists_buster-updates_main_dep11_icons-48x48.tar 0
'http://deb.debian.org/debian/dists/buster-updates/main/dep11/icons-64x64.tar.xz' deb.debian.org_debian_dists_buster-updates_main_dep11_icons-64x64.tar 0
'http://deb.debian.org/debian/dists/buster-updates/contrib/binary-amd64/Packages.xz' deb.debian.org_debian_dists_buster-updates_contrib_binary-amd64_Packages 0
'http://deb.debian.org/debian/dists/buster-updates/contrib/binary-all/Packages.xz' deb.debian.org_debian_dists_buster-updates_contrib_binary-all_Packages 0
'http://deb.debian.org/debian/dists/buster-updates/contrib/i18n/Translation-en_US.xz' deb.debian.org_debian_dists_buster-updates_contrib_i18n_Translation-en%5fUS 0
'http://deb.debian.org/debian/dists/buster-updates/contrib/i18n/Translation-en.xz' deb.debian.org_debian_dists_buster-updates_contrib_i18n_Translation-en 0
'http://deb.debian.org/debian/dists/buster-updates/contrib/dep11/Components-amd64.yml.xz' deb.debian.org_debian_dists_buster-updates_contrib_dep11_Components-amd64.yml 0
'http://deb.debian.org/debian/dists/buster-updates/contrib/dep11/Components-all.yml.xz' deb.debian.org_debian_dists_buster-updates_contrib_dep11_Components-all.yml 0
'http://deb.debian.org/debian/dists/buster-updates/contrib/dep11/icons-48x48.tar.xz' deb.debian.org_debian_dists_buster-updates_contrib_dep11_icons-48x48.tar 0
'http://deb.debian.org/debian/dists/buster-updates/contrib/dep11/icons-64x64.tar.xz' deb.debian.org_debian_dists_buster-updates_contrib_dep11_icons-64x64.tar 0
Let me know if you need any more information. Thanks!