AR View (webcam image) in VR Mode #243

jorgecardoso · 2021-04-15T14:49:41Z

jorgecardoso
Apr 15, 2021

Hi,

I'm trying to create a VR mode scene (experienced through a smartphone VR enclosure) where you can see the real world through your smartphone's camera, while at the same time you see 3D objects overlaid on markers detected through AR.js.

I have tried putting the webcam image into a plane in front of the user, but the marker objects (obviously) do not appear aligned with that webcam image.

Is there anyway to achieve this with A-Frame? I don't mind small misalignments or even distortions of the webcam image seen in VR Mode...

My attempt thus far:

<!DOCTYPE html>
<html>
  <script src="https://aframe.io/releases/1.2.0/aframe.min.js"></script>

  <script src="https://raw.githack.com/AR-js-org/AR.js/3.0.2/aframe/build/aframe-ar.js"></script>

  <body style="margin : 0px; overflow: hidden;">
    <a-scene
      embedded
      arjs="sourceType:webcam; sourceUrl: markervideo.mp4;  detectionMode: mono_and_matrix; matrixCodeType: 3x3;"
    >
      <a-assets>
        <video id="webcam" playsinline></video>
      </a-assets>

      <a-sky color="gray"></a-sky>

      <a-marker id="seven" type="barcode" value="7">
        <a-box color="blue" position="0 0 0.5"></a-box>
      </a-marker>

      <a-entity camera>
        <a-plane
          id="webcamview"
          position="0 0 -100"
          scale="3 3 3"
          width="64"
          height="48"
          src="#webcam"
        ></a-plane>
        <a-cursor></a-cursor>
      </a-entity>
    </a-scene>

    <!-- Start the webcam stream and attach it to the video element -->
    <script>
      // You can also set which camera to use (front/back/etc)
      navigator.mediaDevices
        .getUserMedia({ audio: false, video: { facingMode: "environment" } })
        .then(stream => {
          let $video = document.querySelector("video");
          $video.srcObject = stream;
          $video.onloadedmetadata = () => {
            $video.play();
          };
        });
    </script>
  </body>
</html>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

AR View (webcam image) in VR Mode #243

Uh oh!

{{title}}

Uh oh!

Replies: 0 comments

Select a reply

Uh oh!

AR View (webcam image) in VR Mode #243

Uh oh!

jorgecardoso Apr 15, 2021

Replies: 0 comments

jorgecardoso
Apr 15, 2021