Pointer events on HUD elements

[MeghaS94]

Hello,

From the documentation, I understand that to create Heads-Up Display(HUD) elements, I need to take over the render loop by assigning a render priority to the useFrame() of each component on the canvas.

While I am able to render the HUD element, I am having trouble getting pointer events to register when the HUD element's position or scale changes or the camera changes, I am wondering if I could get some help with understanding what might be going wrong.

Here's an example with two spheres, the blue sphere is the background element, and the other one is the HUD element- https://codesandbox.io/s/fragrant-hill-sw2tl?file=/src/index.tsx
When the HUD element is clicked, initially the event is recognized, but once the camera changes, the pointer events stop firing.

Thank you!

[drcmda]

like this: https://codesandbox.io/s/stoic-breeze-s4ovd?file=/src/App.js

1. wrong namespace, it's @react-three/fiber
2. you can get fully functioning orbitcontrols and perspcam from drei, saves you the boilerplate
3. do not render the HUD scene, or else you double render. you need to portal it into a external scene
4. the useCamera hook can make the hud aware of camera transforms on the mesh level. unfortunately there is no easy way ootb. its a technical issue, react doesn't tell me we're inside a portal and i cannot know which camera you plan to render with.
5. it is better to give the HUD full capacity to render, because now it can re-used. the moment it unmounts you're back to plain view without the HUD.

[MeghaS94]

Hello again,

I am running into another issue related to pointer events, I was wondering if I could get some some help.
I create a bufferGeometry, and I try to ask a server for some position data, and I try to update the bufferGeometry with the new position data. When I do this, the bufferGeometry stops responding to pointer events. What might be going wrong here?
Here's an example of what I am trying to do - https://codesandbox.io/s/pointer-events-flask-server-00bvk?file=/src/App.js
(I have commented out the part where there's a fetch request)

[drcmda]

you should never have these things in the render function. this will all make a mess in threejs, where you cant just exchange buffers because three holds a secret connected between the mesh and the gpu. make 100% sure your render function is clean: no variable, no "new", no function calls, no side effects, nothing but hooks and the returned view.

if you have local state in react, then that is useState:

const [someThing] = useState(() => new SomeThing())

if you have memoized calculations

const someThing = useMemo(() => new SomeThing(foo), [foo])

if you have side effects

useEffect(() => {
  ...
}, [foo])

[MeghaS94]

I did not realise that I shouldn't create a new buffergeometry in the render function, I've updated the sandbox now - https://codesandbox.io/s/pointer-events-flask-server-00bvk?file=/src/App.js
I am hoping that this is a better way to create a buffer geometry and feed it to a react component. I still can't get the pointer events to work when I set the position attribute inside the fetch call from the server. I am a little lost on how to debug this, any help would be super useful.
Thank you so much!

[drcmda]

they're still in the render function of app.js. it might not be the problem, but it will safe you lots of time later on. three handles buffergeometry almost exclusively in the constructor. when you later come and change it, it will blow.

[MeghaS94]

I was able to figure out what the issue was. Your comments led me in the right direction.
As you mention, it turns out that the bufferGeometry was not getting updated as I had assumed. The position update happens inside fetch with some delay, so for the first time, it renders an empty buffer geometry. It doesn't properly update the buffer geo positions in subsequent renders. And that's why pointer events fail to register.

In my updated code, when there is a delay in fetch, and the bufferGeo still has no attributes set. I make the component return an empty group, it only renders the bufferGeo once it is properly constructed. Now the pointer events fire as expected, although I am not sure this was the best way to fix the problem.

Here's how I am doing it now and it seems to work - https://codesandbox.io/s/pointer-events-flask-server-00bvk?file=/src/App.js

Thank you so much for the help!