Removing /etc/conf and /var/logs when removing Ceph

[bigjust]

As part of the discussion around ceph/ceph#62790, it was brought up that the original intent on this postun script, which removes /etc/ceph and ceph logs, was for teuthology's benefit. First, do we think that this action is still required? If so, I can try to add that logic to the function below, but wasn't sure if this was still necessary since it was a really long time ago (12 years): https://tracker.ceph.com/issues/4415

teuthology/teuthology/packaging.py

Line 96 in 434ff78

def remove_package(package, remote):

[zmc]

This is something that normally won't really matter in the sepia lab, since we reimage machines for each test job. But, there are cases where people reuse test node installations to run multiple tests, so it would probably be good to handle this case. I do think it's more appropriate for teuthology to clean up after itself than to expect the rpm to go above-and-beyond what other packages do upon removal.

I think the right place to put the logic, though, is in the cleanup process of whichever task installed it. For many cases I think that would be here: https://github.com/ceph/teuthology/blob/main/teuthology/task/install/__init__.py#L222-L224