Object Detection vs. Color Detection

[Advay17]

We will need to decide what to focus on for our next year in regards to vision: object or color detection. My vote goes to color detection unless something changes: theoretically it should be able to do anything object detection can do, however with less effort on our own part. Plus, given that game piece detection is a first for our team, we have a lot of other logic to implement along side simple detection as well to actually make this detection useful. Namely, we have to figure out how to convert the measurements vision returns regarding the game pieces into actual Pose2ds, which is difficult on its own.
Doing this AS WELL AS setting object detection up, which won't even be possible next year until we either train our own model or photonvision releases its own(which it hasn't for coral even still, showing that the training process can be difficult) seems like a very difficult task.
While object detection should be something we are looking towards, we should first figure out color detection on its own, which would make object detection that much easier. We could also have a part of the vision people working on configuring object detection while the rest test using color detection.

[VMFortress]

What kind of processing is expected to be required for either case?

Also, is there anything each thing would or wouldn't be able to do? I think @shreshtaparthaje mentioned object detection in the case of noticing a bot in the way during auto IIRC. I'm not actually sure how you'd deal with that in color detection but even standard object detection would be tough as a robot isn't a specific object.

[shreshtaparthaje]

The reason I was in support of object detection over color detection was mostly because of the tuning required with color detection. A lot of environmental factors (like light or the background) could affect color detection, and especially with auto I dont want the robot to detect the wrong thing and move to the wrong place.

Also, depending on how fast PhotonVision releases their pre-trained models, object detection should be much harder to get set up. PhotonVision has released an algae model, so we can test using that over the offseason (they still havent made a stable coral model yet, but I am assuming that is because of the coral being white and blending in with the background). We should still experiment with color detection and making our own model though, as if PhotonVision doesnt release their own fast enough, we can still try to detect game pieces.

[VMFortress]

Looks like their object detection model for algae released January 3rd so they must be relatively quick. Though, we can't guarantee they'll always do that (but I guess we can't guarantee they'll always be around in general so maybe not worth contemplating).

That still won't apply for other objects like actual robots but that seems like a tough thing to build a model on in general.

[Advay17]

I'm not particularly worried about finding robot datasets, as there are dozens online that are fully annotated

[VMFortress]

I do still worry about the efficacy of it even in that case. Definitely something we'd have to test if it can handle robot avoidance (assuming the team decides to go that route).

[Advay17]

Also, for color detection, we don't need a better processor. The docs themselves recommend that we use the lowest camera resolution for color detection, so we probably don't need to worry about processing being too demanding on the processor.

[Advay17]

Also one other note that I have gleaned from working on basic game piece detection code: one of the known benefits of object detection is that it would potentially allow us to use a single camera to detect and differentiate multiple types of targets. However, through the code, it is very difficult to actually get the type of the target, so it is very difficult to know what the robot is even looking at in the first place. Theoretically, this should be something we can get through network tables, though I do not know how easy it would be to match this data to the correct vision targets. This means that we shouldn't necessarily anticipate that a single camera can do all types of object detection.

As a side note, color detection would mean that each camera only detects one type of game piece with no exceptions if we want it to be stable.

[shreshtaparthaje]

I found this CD post from December of 2024, and it gives some insight on how differentiate each item. Pretty much, it says that each game piece has its own ID (with the fiducial ID method). Though, the model can't give the name directly.

[Advay17]

Ah nvm then, apparently objDetectId and objDetectConf would work for this