ci: Remove tox and tox-poetry-installer

This can break the poetry command when developing locally.

Author: tony