2025/02/05 Weekly Meeting Notes

[himanshunaidu]

The following notes contain the progress and next steps for each sub-task:

1. Computer Vision Model Update
   Assigned to Himanshu Naidu.
   The DeepLabv3 TensorFlow 1 model has been successfully converted to CoreML, and it runs within 100 ms per iteration.
   However, the model does not seem to perform well on the more important classes (such as sidewalks). Also, the required input dimensions to this model (1024x512) does not look good for the iPhone.
   Hence, in parallel to integrating this model (as a backup) to the application, I am also developing a Machine Learning pipeline for iOSPointMapper.
   git@github.com:himanshunaidu/CoreML_Pipeline_iOSPointMapper.git (currently private)
   This will take heavy inspiration from EdgeNets, but lots of modifications will have to be made to work with newer library versions.
   https://github.com/sacmehta/EdgeNets.git

2. Segmentation Mask Post-Processing
   Assigned to Nik Wilson.
   Basic introduction to the task has been given to Nik, and the relevant repositories have also been shared. A high-level planning on the next steps has been done.
   We will be discussing weekly on the task. Currently, the meeting is set weekly on Wednesday 2 PM - 3 PM.

3. Depth-Aware Instance Segmentation
   Assigned to Himanshu Naidu.
   The current plan is to use a first version of depth-aware instance segmentation which is a combination of the watershed algorithm and depth-based clustering.
   Watershed: https://en.wikipedia.org/wiki/Watershed_(image_processing)
   For depth-based clustering, we plan to use DBSCAN, as the concept of Density-based clustering, with the setting of distance-based neighborhood, seems appropriate for our use case. The difference between the depths would be the distance metric for our use case.
   The UI changes that will have to be done for this task will be extensive, and will affect the AnnotationView significantly.
   We will have to check on how to update user feedback when dealing with multiple instances of the same class in one screen, so as to not overwhelm the user with too much validation requirement.