The ruby shared library is unversioned (i.e., /usr/lib/ruby/gz/math.so). Different major releases of gz-math therefore has a file with the same name and cannot be installed concurrently. Adding a major version to the shared library would fix this.
https://github.com/gazebosim/gz-math/blob/eeb17eae96b027af9497b099f706abf0637f96ec/src/ruby/CMakeLists.txt#L45C7-L45C25
I'm not a ruby user, so I can't properly test the implications of making the change.