ground truth extraction for occluded objects.

[ShravanthiPatil]

Hello,
This repo has been of great support to utilize the Waymo dataset much quicker. Thank you!

I am particularly interested in your code for extracting ground truth. As I understand the code is specifically extracting the ground truth/labels for non-occluded objects.
For a specific tfrecord ( one from the training_0031 folder), I see that the length of the visibility list (as per your code) is 82, out of which 62 objects are visible and 20 are occluded ( as the count is less than 10).

Now that I print the labels/ground truth for both objects - Occluded as well as fully visible, I see the total number of labels generated is 45 ( to be precise - only 14 occluded and 31 fully visible). Unlike the number of objects for which visibility was checked, being 82. I assume this is because there are overlapping objects in the frame or is it something else? Please clarify.

Also, I see in the labels being generated, the value of truncated is by default set to 0. Any specific reason that this wasn't handled?

Meantime, For the same tfrecord ( from training_0031- segment-972142630887801133_642_740_662_740), it is noticed from the frame.context the below number of objects detected via laser and camera.

Considering just the camera_object_counts: I see, the total count of objects is 45, which matches the labels generated by the code.

**Is the object TYPE_SIGN only annotated for laser data and for camera data? Please clarify. Also the object TYPE PEDESTRIAN is identified by both Laser and camera, but one each. But the frame has two pedestrians captured. Isn't that both camera and laser identify both pedestrians and show the count as 2 each? **

stats {
laser_object_counts {
type: TYPE_VEHICLE
count: 32
}
laser_object_counts {
type: TYPE_PEDESTRIAN
count: 1
}
laser_object_counts {
type: TYPE_SIGN
count: 2
}
time_of_day: "Day"
location: "location_phx"
weather: "sunny"
camera_object_counts {
type: TYPE_VEHICLE
count: 44
}
camera_object_counts {
type: TYPE_PEDESTRIAN
count: 1
}
}

Thank you!!

[gdlg]

Hi @ShravanthiPatil ,

You are welcome. I am not currently actively using the Waymo dataset and I haven’t tried yet the updated dataset so I might be of limited help. Still happy to accept pull requests and issues though :-)

The visibility list contains all the laser labels around the vehicle, even if they are outside the front camera frustum. Objects outside the frustum are culled later in the for loop here which might explain the discrepancy which you notice.

I also found that LIDAR data is quite inaccurate to measure occlusion (at least as seen by the camera). So the hand-annotated camera labels are going to be more accurate than the laser ones.

For the truncated flag, I just didn’t need it. I am also not sure what definition of truncation was used in the KITTI dataset. I used the format as a way of quickly getting the right data format to train with my model.