How to prevent re-render when using useState?

[bigxixi]

Hi,
I am new to R3F, and get confused when I play with the get started example:
Every time I set a state all the components seam to re-render, if I set some properties with Math.random(), they will change, even though I never user the state.
Below is my fork of the official get started demo, I just add the random position to the mesh
position={[Math.random(), Math.random(), Math.random()]}
https://codesandbox.io/s/modest-booth-mxxxgy?file=/src/App.js:663-719
What i hope is to generate random positions when refresh, then keep the position.
But now the position changes when I hover to the boxes, because it set the hover state when hover in/leave:
hover(true)
If I remove the hover() code(and the click part), it does what I want.
So how can I keep the hover and click state change function without re-render the components?