How to set up the pipping server to view model in AR like the editor?

[WadeMorrison]

I was wondering if anyone could point me in the right direction, or explain a little on how to set up the pipping server to view the model in AR from scanning a QR code if starting from desktop? Just like how the editor uses it.

I've looked at some of the documentation on the pipping server's github, as well as the repository for the editor, to see how it's set up. But I got pretty confused.. Any help would be greatly appreciated!

[elalish]

This file has all our piping-server logic. It's pretty simple, you just need to set up both your pages to Get and Post the data you desire and the server just forwards between them.

[WadeMorrison]

Okay that's what I saw, still a novice to it all, so wasn't sure. Is it just the utils that's important, or is it pretty much the whole ./model-view folder important to look deeper into?

I'm assuming just mainly the utils in order to determine the OS?

[elalish]

Yeah, pretty much. If you want to see the data we actually get and post you can look through the other mobile_view files, but that's pretty specific to our use case.

[WadeMorrison]

Ah okay, awesome. Thank you! I will get to messing around with it and see what I can come up with.

Do you think this could be a feature added to model-viewer at some point? To allow a QR code to be scanned if the user were to be on desktop and wanted to view it in AR? My previous work around just made a user scan the QR code which just linked the current page url, and then made the user re-configure/edit the whole model in order to then be able to "View in AR".

I've never made a PR before, but I could try if that's a possibility too/already isn't in the works.

[elalish]

No, I definitely wouldn't want a dependency on piping-server in MV - that's too fragile. I think there are a number of ways to handle the QR-AR flow and it's best that live outside our library. However, making another npm package for others to use would be useful, I'm sure. Or even just linking to examples and sharing in our show-and-tell. We could even add an FAQ for it potentially.

[WadeMorrison]

Okay, cool. Thank you for the help elalish! I will be sure to do one, or maybe both, of those suggestions if I feel like I've got a good working demo.

[WadeMorrison]

Hey, I got it kind of working finally after a bunch of trial and error. Picking apart what you linked, as well as karanganesan's example.

However, when I go to POST to the viewer page (for mobile), it won't load the model unless I either have the viewer page open first, or if I navigate to it after hitting the "View In Your Space" button, then refresh the view page, and then click on the "View In Your Space" button again. But this is also not an issue for a product that has no custom options (colors/size/etc..), it'll load no problem on the /view page.

Would you know by chance how to make it to where once the user navigates to the page after scanning the QR code, it will then fetch the model that's been uploaded to the piping server? I get a status of 200 in the response when POSTing so not really sure where I am going wrong. Any help would be greatly appreciated!

Here's the example that will load no problem (no custom attributes):
https://fathomless-brook-11892.herokuapp.com/product/64077d7d7666dc5e99194465

And the one with custom options that will only be "fetched" if the view page it opened first:
https://fathomless-brook-11892.herokuapp.com/product/64077d7d7666dc5e99194464

[elalish]

Have you dug into our mobileView code in space-opera? It's a bit complex - messages are posted to tell the other end that content has updated and cause it to pull that content up. There's a fair bit of back-and-forth to maintain state between the two ends.

[WadeMorrison]

Yeah, I dug around a little bit, definitely complex for myself haha. My code reflects more of karanganesan's example as it's a little easier to see how it all works (though I couldn't get theirs to work on their website). But I understand bits and pieces of what's in the space-opera I think. I think the part I may need is the ping? // If a ping was received, then a new page has been opened or a page was // refreshed. As maybe once it detects that the user has navigated to that page, it will then POST the content I need?

Here is what I have as the POST (using useEffect and useState of course):

  const modelViewer = document.getElementById("model-viewer");
  const blob = await modelViewer.exportScene();
  const model = new File([blob], "model.gltf", { type: blob.type });

  const endpoint = `https://piping.glitch.me/${pipingServerId}`;

  fetch(endpoint, {
    method: "POST",
    body: model
  }).then(response => {
    if (response.ok) {
      console.log("Model uploaded successfully");
      console.log(response)
      console.log(model)
    } else {
      console.error("Failed to upload model");
    }
  }).catch(error => {
    console.error(error);
  });
}

And then on the fetch /view page:

  const [src, setSrc] = useState(null);
  async function fetchModel() {
    try {
      const response =  await fetch(`https://piping.glitch.me/${params.id}`);
      const blob = await response.blob()
      const url = URL.createObjectURL(blob);
        setSrc(url);
      console.log(response)
      console.log(url)
    } catch (error) {
      console.error(error);
  }
};

[WadeMorrison]

Alright, I think I finally got it to work! But by all means please feel free to critique me/help me make this better.

Basically in the configurator I wait for a "ping" or ok/200 response from the /view page, which only happens once the endpoint is opened. Once that is received, it will then trigger the exportScene() function, and pass that blob back through to the piping server, and the /view page will then fetch it and set it as it's src.

configurator page:

 const [pipingServerId, setPipingServerId] = useState("");
  const [modal, setModal] = useState(false)

  const toggleModal = () => {
    setModal(!modal)
  }

  useEffect(() => {
    const options = {
      numberUpto: 60,
      joinBy: "-",
    };
    const id = meaningful(options);
    setPipingServerId(id.toLowerCase());
  }, []);


  async function exportGLB(){
    // Create the Blob object
  const modelViewer = document.getElementById("model-viewer");
  const glTF = await modelViewer.exportScene();
  const blob = new Blob([glTF], { type: 'application/octet-stream' });

// Make the POST request
    const endpoint = `https://piping.glitch.me/${pipingServerId}`;
    try{
    const response = await fetch(endpoint, {
      method: "POST", 
      body: blob, 
    });
    if (response.ok) {
      console.log("Model uploaded successfully");
      console.log(response)
    } else {
      console.error("Failed to upload model");
    }
    } catch (error){
      console.log(error.message)
    }
  };

  async function fetchPing(){
    try {
      const responsePing =  await fetch(`https://piping.glitch.me/${pipingServerId}`);
      const ping = await responsePing.text()
      if (responsePing.ok) {
        console.log(ping);
        await exportGLB()
      } else {
        console.error("Failed to get Ping");
      }
      } catch (error){
        console.log(error.message)
      }
  }

/view page:

const [src, setSrc] = useState(null);
const params = useParams();

async function fetchModel() {
  try {
    const response = await fetch(`https://piping.glitch.me/${params.id}`);
    const blob = await response.blob()
    const src = URL.createObjectURL(blob)
      setSrc(src);
    console.log(response)
    console.log(blob)
  } catch (error) {
    console.error(error);
}
};

async function pingUrl() {
const endpoint = `https://piping.glitch.me/${params.id}`
try{
const response = await fetch(endpoint, {
  method: "POST", 
  body: JSON.stringify(src) 
});
if (response.ok) {
  console.log("Uploading Model");
  fetchModel();
} else {
  console.error("Failed to Ping");
}
} catch (error){
  console.log(error.message)
}
};

useEffect(() => {
  pingUrl();
}, []);

Again, open to any critiques/help on this. So far it's working good for what I need it for. I'll be tweaking the endpoint URL to be a little more unique to avoid any accidental overlaps though.