Inquiry about type of input of task 1 for evaluation

[jackzhy96]

We would like to ask how the input looks like when evaluating the first task (find the needle). Is the input two images from left and right camera or two videos from left and right camera or maybe something else? We would like to double check the input type therefore we can design corresponding input receiver. FYI, we would assume the input would be images.

Thanks, and look forward to hearing from you!

[adnanmunawar]

Hi Jack,
The input would be stereo video as mentioned here https://collaborative-robotics.github.io/surgical-robotics-challenge/challenge-1.html.

Check this text for instance:

Section Test Conditions for challenge tasks 1:

The needle will be located between 80 mm and 200 mm from the endoscope. Lighting can vary as specified above. Stereo endoscope video will be provided at 1080p resolution at 30 fps.

[jiang-yiwei]

Also, is the final evaluation metric the sum/avg of all 60 errors? Each error is the distance between three fixed points.

[adnanmunawar]

Hi Yiwei,
Tagging @pkazanzides here.

By stereo video, we meant the real-time video streaming from the simulator, so there is no length/time limit on the video itself.
Now regarding your question:
For each configuration of the ECM and Needle pose, we expect just 1 pose, w.r.t to the current endoscope within 10 secs. Does that clarify your question?

[jiang-yiwei]

Hi Adnan and Prof. Kazanzides,

Thanks! My concern is that, after getting a pair of the images, my program may need a few second to output a needle pose. Thus, in the process, hundreds of images may has elapsed and either queued/missed by my program.

Otherwise, would you only move the ECM & the needle in every 10 seconds or after getting a results from the last configuration?

Best,
Yiwei

[pkazanzides]

We plan to treat each evaluation separately. That is, we would start your program and then wait for your program to send the "completion message" (via publishing a ROS topic or calling a Python method), where the completion message would contain the estimated needle pose. At that point, we will terminate all programs and restart everything for the next needle pose. So, there is no chance that you will miss a change of needle pose.

[jiang-yiwei]

That makes a lot of sense, thank you!