How to get pointer events before objects in the scene?

[dangrabcad]

Sometimes in my application you go into a mode, where the pointer does things other than activate objects in the scene; for example, normally when you click on an object it selects it, using an onClick handler on the object, but when you're in "annotate mode", clicking anywhere (on an object or not) draws annotations instead of calling the object's onClick. I'd like an extra onClick method that happens before objects in the scene get called, and the same for other pointer events.

To collect pointer-on-background events, I've used an invisible skybox that's further away than all the real objects in the scene. I could use a near plane, attached to the camera, to get the preview events, but that seems riskier: it'll go wrong if the near clipping plane goes past it, or if we dolly in enough to bring scene objects past this foreground plane. For those reasons I'd prefer to avoid repeating that strategy.

Another option is capturing DOM pointer events in the capture phase, before they even reach the Canvas and R3F's event system, but I need my event handlers to get the R3F-style event with the intersection and hit object. To achieve that I'd need to replicate the way R3F creates and casts a ray and puts that information into the event, which is not very maintainable.

Do you have any other ideas?

[drcmda]

this seems to me is best solved with an abstraction of your event. centralize them and make them check for app-state in there. hijacking events in specific order, using planes, etc, that sounds a bit hacky/specific.

function useClick() {
  const annotate = useStore(state => state.annotate)
  return e => {
    if (annotate) doThis()
    else doThat()
  }
}

function Foo() {
  return <mesh onClick={useClick()} />

other than that, <Canvas raycaster={{ filter: ... }} allows you to customize/filter your events, the signature is:

filter = (items: THREE.Intersection[], state: SharedCanvasContext) => THREE.Intersection[]

but you just get the intersection, not the dom event.

[dangrabcad]

I'd imagine if it's for something like drawing over the page, you'd use a div in front of everything, and you'd put the drawing on the same div; but that wouldn't work if you need to know which DOM element behind the div was clicked. In that case I imagine you'd put an event-handler at top-level for the capture phase, using addEventListener(..., true) or onCapturePointerMove in React.

[drcmda]

bufferplanes would equal the dom solution and that would work, but everything else outside that wont work because these would just be dom event without raycasting. r3f in theory has something like addventListener as well: state.pointer.on("click", e => ...) but it will only give you the dom event.

i guess the closest to the dom would be something like this: stick a billboard in front of the camera. the state data will even tell you if further objects have been hit so you can preemptively run ops.

import { PerspectiveCamera, BillBoard } from "drei"

function AnnotateCam() {
  return (
    <PerspectiveCamera makeDefault position={[0, 0, 10]} fov={75}>
      <BillBoard position={[0,0,1]} onClick={...} />
    </PerspectiveCamera>
  )
}

<Canvas>
  <AnnotateCam />

[dangrabcad]

I got the plane in front of the camera working, with its position set to [0, 0, -camera.near], but only with a PerspectiveCamera, not an OrthographicCamera - I guess I'd need a small offset in the latter case. The thing consuming the pointer events also needs to know to use ev.intersections[1] (not 0, which is the plane itself), and also to check whether ev.intersections[1] is a real object or the skybox.

[drcmda]

i see, but this would be the same as your dom solution, you would also have to differ between the plane and the other targets. i just don't think onClickFirst={...} just for this edge case is a good api design, there must be something generic, either useland (hooks) or something else. do you think you could solve this in a clean way with the dom pointerevents api?

[dangrabcad]

With the DOM events API, you can achieve both needs with handlers on document or body. A capturing-phase handler gets events before any child elements, and a bubbling-phase handler gets events that aren't handled by any child (or didn't hit any child). I suppose the equivalent for us would be if events that didn't hit any objects got delivered to the Canvas (or to the Scene, but then what if you have multiple scenes?)