Integration with Google Maps WebGL (frameloop & interaction questions)

[neil-morrison44]

Hi,

I've been following along this: https://developers.google.com/codelabs/maps-platform/webgl#3 but converting it for react-three-fiber.

I've got it to the point where I can get arbitrary (static) react-three-fiber elements rendering where I want them on my map, which is great.

However, there's two questions I've got:

Question 1:

• Google wants to control the frame timing completely, which means I need to / should use frameloop="never"
• I can tell google to re-render the map & custom three-fiber components via a webGLOverlayView.requestRedraw() which I can fire from within the react-three-fiber tree
• Ideally I'd like to do something along the lines of frameloop="demand" where I know the state is invalid & can tell the google map it should render a frame
• But I'm not sure how to tell when there's a "dirty" canvas, doesn't look like the invalidate things function when frameloop="never"
• I can probably get around this by creating some context and allowing any component to call webGLOverlayView.requestRedraw(), but I'm not sure if that's the best way

Question 2:

• onClick etc don't seem to be working
• Google have you manipulate the camera based on map projection etc via a

const matrix = transformer.fromLatLngAltitude(latLngAltitudeLiteral);
camera.projectionMatrix = new THREE.Matrix4().fromArray(matrix);

thing (see https://developers.google.com/codelabs/maps-platform/webgl#6 )

• Do I have to tell the raycaster about this? or configure the camera in some other way? (it works visually)
• Could just be that google prevent the events getting through too, which would be annoying but entirely possible.

EDIT:

Looks like it probably is google maps catching the click & preventing it from reaching the canvas - I can probably set up a listener on the maps side if there's a way to manually send the event down to react-three-fiber though?

EDIT EDIT:

• Now doing onCreated: ({ events }) => events.connect?.(map.getDiv()), to have R3F listen on something which actually gets clicked etc
• However transformer.fromLatLngAltitude() returns a projectionMatrix which I apply to the (manual) camera via camera.projectionMatrix = new THREE.Vector4().fromArray(matrix), but the raycaster needs the camera to have its worldMatrix set correctly - it doesn't look like there's any way to back-convert from projectionMatrix to worldMatrix?

---

Incase this is useful to anyone else I got it working through roughly via these steps (which might be completely wrong):

• Using createRoot with google's canvas - configured with google's WebGLRenderingContext
• A component which gets the scene, camera & advance from useThree() and sets up a function on webGLOverlayView.onDraw in a useEffect to render alongside when the map renders

[andrewdoro]

@neil-morrison44 hey I was trying to setup react three fiber with google maps, it would really help me a lot, if you can share your implementation.

[neil-morrison44]

The full code was for my dayjob so it's not mine to give away - I think (it's been a while) everything above was the roadblocks I ran into along the way of getting something up though so it should, in theory, be enough to follow along.

I don't think I ever got the raycaster working, but it wasn't needed for anything we did.

The other thing I could never get working is showing multiple objects the way Google Maps wants them shown, with multiple renderers writing to the same canvas being loaded / unloaded based on lat / lng / zoom level - React Three Fiber expects a 1:1 between canvas and a scene rather than that 1:many.