How best to keep changes in position after detaching TransformControls?

[stephent]

[Updated: cleaned up the CSB example and made the question a little more specific...]

I have a project that will have multiple meshes. Each should be independently movable by the user, using TransformControls or similar. I'm imagining a click-to-select/move/click-elsewhere-to-deselect approach. The new position of the mesh should be retained after TransformControls are removed from the scene.

This shows one possible approach: https://codesandbox.io/s/r3f-drei-transformcontrols-forked-addremove-wom61

It feels a bit over-complicated, in particular the imperative handling of the position update when the user clicks off the selected mesh.

Is there a cleaner approach to this sort of thing?