Is there a way to combine react-three-fiber and regular three.js in a single app?

[camjackson]

Use cases for this:

• An existing three.js app where you want to start gradually adopting R3F without doing a big bang rewrite
• An exsting R3F app where you want to gradually go back to regular three.js without doing a big bang rewrite
• An app where R3F is great for some things, but for other parts of your app it's easier to do in 'native' three.js

Is there a way to do this? Can I have R3F controlling some parts of the canvas, but also have a regular game loop on the side that's rendering other stuff?

[drcmda]

of course, r3f just sets up renderer+cam+scene for you, and you get to write jsx, but nothing would stop you doing this:

function Foo() {
  const [scene] = useState(() => new THREE.Scene())
  const state = useThree()
  useEffect(() => {
    // ... your plain three code here
    scene.add(your result)

    function resize() {
      // state.viewport and state.size have the correct sizes
    }

    window.addEventListener("onClick", () => ...
    
    function loop() {
      requestAnimationFrame(raf)
      scene.render(state.gl, camera)
    }
    loop()
  }, [])
  return <primitive object={scene} />
}

<Canvas frameloop="never">
  <Foo />

as you see you can already start using some helpers, like the GL instance, or responsive width/height

An app where R3F is great for some things, but for other parts of your app it's easier to do in 'native' three.js

honestly i dont think that usecase exists. r3f is native three, even the jsx you write is native three. having tools like useEffect, useMemo, useFrame, suspense and so on allows you to categorize imperative code and effects, which will always make it easier than having to manage a long blob of un-integrated code. you will always have to write imperative code and shaders, but this is exactly what r3f allows you to do, but in a clean and safe manner. after two years of working on and with this lib (at work, cad system), i have yet to see a single usecase that would be easier in plain three - imo it does not exist.

[camjackson]

Thanks, that's really helpful to see what that would look like.

My team is a few months in with R3F. We're loving it for a lot of things. You're right, having React's component model and hooks is awesome. Out of the box we have easy ways to structure our app, define state, define reusable components etc.

Where we're running into trouble is the performance of per-frame state updates. For example when dragging and dropping something in the scene, we've found that trying to do setStateon every single mousemove event is not a good idea. Maybe it's just that our code is bad but it seems like the overhead a react setState+component render+reconciliation is too slow to happen 60 times per second.

We've resorted to using refs for anything that we need to change every frame. So for our drag and drop example, we get a ref to the currently-dragged mesh, then on every mousemove event we directly mutate the position of the mesh via its ref. Then on mouseup we read the mesh's position and setState in order to sync that final position back into the react component state.

It works well enough, but we feel like we're fighting with R3F a lot. With how often we're breaking out of the reactive component model and directly manipulating the scene via refs, we can't help but wonder if it would be easier if at least some parts of our app were entirely managed by an imperative game-loop-style rendering cycle.

Would love to know if this is a common problem and if there's an obvious solution that we're missing. I imagine that tons of apps (especially games) have state that changes every frame - is that usually done outside of the react render cycle like what we're doing? Or is it just fine to try to setState at 60FPS and it's just that our code is slow?

[drcmda]

you should never call setstate in a loop ever, not even in plain react-dom. and there is no need: https://docs.pmnd.rs/react-three-fiber/advanced/pitfalls this does not mean that you have to revert to old three, just don't setstate. update a ref and mutate the view inside useframe, or mutate the view directly inside the event. there are dozens of examples with complex animations or interactions in the docs, not a single one setstates. that is the # 1 reason for why useFrame exists.

you also don't need to fight it, r3f is in no way opposed to imperative code. all it does is allowing you to set it up clean:

• useMemo, memoized calculations and objecs
• useEffect, imperative side effects that happen after render
• useLayoutEffect, imperative side effects that happen before render
• useFrame, imperative mutations frame by frame

but if you directly change scene contents, add/remove children, i think that's a symptom that something's really off, this will likely collapse at some point.

I imagine that tons of apps (especially games) have state that changes every frame - is that usually done outside of the react render cycle like what we're doing? Or is it just fine to try to setState at 60FPS and it's just that our code is slow?

here's an example: https://github.com/pmndrs/racing-game most components in there function 60fps, but they avoid having to go through react for it. react is only for flags and maintaining the scene. even state managers like zustand or redux can handle 60fps without setstate. i personally divide my state into immutable, reactive state and fast, mutable state.

[camjackson]

Thanks very much, this gives us a lot to think about 🙇‍♂️