How to stream USB camera video feed to frontend and save video to file?

[scandav]

I am trying to stream images acquired from an USB camera to the frontend.

I want to stream content from an OAK-D opencv camera to be consumed on the frontend and be able to save the video to file while streaming.

I found two possible ways of doing that:

• MJPEG in Rust: in this case I would need to register_uri_scheme_protocol in Tauri that reads frames from the camera and attach them to the body. Something like this:

  tauri::Builder::default()
  ...
     .register_uri_scheme_protocol("stream", move |_app, request| {
          let mut response = ResponseBuilder::new();
          response = response
                .header("Cache-Control", "no-store, must-revalidate")
                .header("Pragma", "no-cache")
                .header("Expires", "0")
                .header("Connection", "close");
          
          // append encoded images here..
          
          response.mimetype("multipart/x-mixed-replace; boundary=thisismyboundary").status(200).body(stream_of_encoded_images)
  }
  

  I am struggling with understanding how to keep the response alive on Tauri and wonder if what I am trying to do is even possible with register_uri_scheme_protocol. Streaming response body #3138 still has no solution for streaming content to a body.
• WebRTC, as suggested on Tauri discord channel. This approach works well on integrated camera (MacOS) but I cannot find the USB OAK-D camera. Also, how can I save the video to file with WebRTC?

Do you think I can reach the goal with either of the two or is there a better approach to my problem?
Thanks a ton!

[ghost]

Hi, any progress?

[Manubi]

@scandav I also would be interested in this. As I am basically trying to do the same. I would need to be able to connect to a camera either via USB or bluetooth. Ideally bluetooth from macOS to iPhone.

[abrman]

I'm currently solving a different but similar problem. I'm saving a three.js scene as video onto drive. I have a video_start endpoint, then video_pipe_image and video_complete endpoints. For video_pipe_image you generate the base64 image

const dataURL = canvas.toDataURL("image/jpeg", 0.9);
const base64Data = dataURL.replace(/^data:image\/jpeg;base64,/, "");

and send it to the backend. jpeg 1080p stream runs only 0.5x on my machine (beefy), png is even slower.
Landed here as I'm looking for more performant methods.