GoPro MAX video - upload export rectilinear 360° images spaced by GPS distance.

[Uzver123]

Camera: GoPro MAX
Format: Video
Extension: *.360

Issue: Functionality of mapillary_tools working, however there's a limitation due to software provided by GoPro
.360 video editor called GoPro Player used merge video from two lenses into one rectilinear 360° video, when run on Windows platform does not support exporting GPMF binary stream along with rectilinear 360° video (mp4) this option only works for Mac version of GoPro Player.

Adding *.360 file from GoPro Max directly to mapillary_tools only grabs first stream of the video which is front facing camera not the 360° image. If i want to add true 360° image to mapillary instead of basic from front camera.

Suggested solution 1: make mapillary_tools export both front/back images from *.360 file with suffix _front.jpg / _back.jpg so people can batch stitch them together themselves and then upload along with gps data stored in separate .GPX file.

Suggested solution 2: if i generate mapillary_image_description.json from the *.360 file with GPS distance three meters between images then add functionality to use mapillary_image_description.json for sampling the frames from rectilinear *.mp4 file (generated by GoPro Player and has no built in GPMF stream).

Suggested solution3: Allow to source images from mp4 file that does not have GPMF stream, with option of GPS distance three meters and allow to specify original file .360 video as source of GPMF data. Both .360 file and .mp4 file have same timecode its just use .360 file for source of GPMF and use .mp4 file for source of images.

[borismasis]

Hi @Uzver123 - you said "Adding *.360 file from GoPro Max directly to mapillary_tools only grabs first stream of the video which is front facing camera not the 360° image" - I don't think that's correct, unless something is broken, adding a .360 video file will result in a full 360 image on Mapillary. Are you seeing something different? If so, could you share a link to a Mapillary example and a .360 video file orginal?

[Uzver123]

adding a .360 video file will result in a full 360 image on Mapillary. Are you seeing something different?

Hello, its technically impossible because video with .360 extension straight out of camera is not yet stitched to 360 it has two separate streams for each camera, only after it went trough GoPro Player and exported as .mp4 file then its true 360 image.

https://gopro.com/en/au/news/max-tech-specs-stitching-resolution

I posted video here https://pixeldrain.com/u/DhUhQc5X click download on the left sidebar. But it won't work with mapillary_tools because travel distance is very little, however you can verify the streams manually with FFMpeg

[borismasis]

Mapillary does stitching of the .360 files on the server side, so shouldn't have to worry about it.

Can you try uploading a longer .360 directly to mapillary and confirm if you have any problems with it?

[Uzver123]

Mapillary does stitching of the .360 files on the server side, so shouldn't have to worry about it.

I do not want to upload entire videos to mapillary i want to process frames locally using mapillary_tools i.e. extract images for every 3 meters and upload them myself.

It explains possible usecase for mapillary_tools vs desktop uploader here.

Basically my request is for client side stitching of images exported directly from .360 file.

[borismasis]

Ah ok, I don't think we'll be adding support for that unfortunately. For most users the server-side stitching is easier - we'll automatically stitch and sample the video at one image every 3 meters.

[Uzver123]

one image every 3 meters.

3 meters was just example i might want to specify my own custom distance

In just two days i have recorded 93 GB of .360 files uploading them all seems like a lot of bandwidth and time will be spent compared to batch processing them with mapillary_tools and uploading just JPG's besides my .360 videos are recorded with following settings:

• Sharpness=Low
• White Balance=Native
• Color=Flat

have to first develop them into nice looking images before uploading

you do not have to add more logic for processing them as there is already tools for stitching GoPro Max frames, all i personally need is when i pass .360 file to sample_video is to export not just video for first stream, but option to export both video streams. and i can then stitch them myself.

[borismasis]

I see, unfortunately mapillary_tools probably won't be able to help with that in the short term as its not designed for this particular use case. Sorry about that.

[ptpt]

Suggested solution 1: make mapillary_tools export both front/back images from *.360 file with suffix _front.jpg / _back.jpg so people can batch stitch them together themselves using PTGui for example and then upload along with gps data stored in separate .GPX file.

Hey @Uzver123 we can support solution 1.

Note: using a GPX with GPS points of a custom distance (e.g. every 10 meters) doesn't affect the sampling distance of the video. On the server side, the video frames are still sampled every 3 meters by interpolating between GPX points. So solution 1 sounds like the only viable one.