Feature request: a flag to also output files that matches with the package md5sum

[mbideau]

Hi, first thanks for sharing this program.

Is your feature request related to a problem? Please describe.

I would like to list all the files that haven't changed (during the check command) so I can exclude them from being backuped.
Right now the output only list changed files and permission denied.

Describe the solution you'd like

Add a CLI flag to also output files that matches their md5sum.

Describe alternatives you've considered

I currently use debsums that allow this (I am on Debian).

[VorpalBlade]

This is for paketkoll rather than konfigkoll I assume? This would be doable fairly easy on Arch Linux I believe, I'm not so sure about Debian, since the data there is incomplete. This is something that debsums also suffers from though!

I do have a slower approach used in konfigkoll to get all the data even on Debian, but that involves downloading any missing .deb files to the apt cache and then looking inside the archives (plus caching in a database file to avoid having to recompute everything every time). I never ported this slower approach back to paketkoll as konfigkoll was always my end goal.

Also: None of the tools understand post install scripts. As these are arbitrary shell script code, this is an impossible problem (without actually executing the shell script). On Arch Linux post install scripts tend to be used rather sparingly. Not so on Debian, where they are often used to generate default configuration files.

What is the use case to get the list of unmodified files though?

Contributions are welcome by the way, especially since this is not something I would use myself. Adding the basic version (debsums equivalent) shouldn't be too hard. Porting over the "get all the data" approach would be much more work.

[mbideau]

Hi,

Thanks for your feedback.

I do have a slower approach used in konfigkoll to get all the data even on Debian, but that involves downloading any missing .deb files to the apt cache and then looking inside the archives (plus caching in a database file to avoid having to recompute everything every time).

I do not understand, may be I wasn't clear. In your current code, when you check if the md5sum of a file matches, if it does, simply output a message, instead of moving to the next file silently. That's it.
I might be wrong but the implementation might be: to add another array like the one collecting issues in the function paketkoll_core/src/backend/filesystem.rs::check_file but to collect success, and then, if the option (say it is name --print-ok for example) were specified in the CLI, display all files that matches their md5sum.

What is the use case to get the list of unmodified files though?

It is part of my backup process: I exclude all the packages files that have not been modified (checksums matches) in order to save space (disk usage) and time (network transfer, compression, extracting file's list from archives, etc.) in my backup archive.

Thanks again 😉

[VorpalBlade]

In your current code, when you check if the md5sum of a file matches, if it does, simply output a message, instead of moving to the next file silently.

That isn't quite how the code works unfortunately. I run a thread pool (to use all CPU cores) scanning the files and each threads builds up a list of found "issues" that contain information about discrepancies: path, type of difference (e.g. contents, mode, symlink target, ...). This design was made to work with the configuration management part (that you don't use).

Once all the threads have finished the main thread then prints out the list.

For your use case you would need to allocate one such structure per file that is OK, and then print at the end, which might be quite a bit of memory. It might be possible to do it in a streaming way, but it would complicate the code quite a bit.