how could I put the keypoint names on the image?

[amroybd]

I am using Detectron2 for keypoint detection for my custom dataset. I am using the below code to visualize the bounding box and the keyoints.

outputs = predictor(im[..., ::-1])
v = Visualizer(im[:, :, ::-1], MetadataCatalog.get(cfg.DATASETS.TRAIN[0]), scale=1.2)
out = v.draw_instance_predictions(outputs["instances"].to("cpu"))
plt.figure(figsize=(60,30))
plt.imshow(out.get_image()[..., ::-1][..., ::-1])

I get the output with the bounding box with the name of the class and keypoints as dots. I would like to have the keypoints name on the image as well. How could I do it?

Go to this StackOverflow post for more details information.

https://stackoverflow.com/questions/74679667/in-detectron2-how-could-i-put-the-keypoint-names-on-the-image

[tyemalshara]

Your approach to display the names of the keypoints alongside the detected keypoints is quite reasonable. Let's complete the solution by implementing it step by step:

1. Iterate through predicted keypoints:
2. Extract x and y coordinates:
3. Map keypoints to their names:
4. Draw text on the image:

Here's what I used for visualizing keypoints' labels on the image:

outputs = predictor(im[..., ::-1])
v = Visualizer(im[:, :, ::-1], MetadataCatalog.get(cfg.DATASETS.TRAIN[0]), scale=1.2)
out = v.draw_instance_predictions(outputs["instances"].to("cpu"))
# Iterate over each detected instance and its keypoints
instances = outputs["instances"].to("cpu")

for i in range(len(instances)):
    keypoints = instances.pred_keypoints[i]  # Get keypoints for the i-th instance
    for j, predicted_keypoint in enumerate(keypoints):
        x, y, prob = predicted_keypoint
        if prob > 0.5:  # Adjust the probability threshold if necessary
           position = (x.item(), y.item())
           text = MetadataCatalog.get(cfg.DATASETS.TRAIN[0]).keypoint_names[j]  # Get the corresponding keypoint name
           keypoints_positions.append((text, position))

# Draw all keypoints names on the image at once
for text, position in keypoints_positions:
    v.draw_text(text, position, font_size=12, color="g", horizontal_alignment="center")

# Visualize the result
plt.figure(figsize=(60, 30))
plt.imshow(out.get_image())
plt.show() 

In this solution:

• We iterate through each detected instance and its keypoints.
• For each keypoint, we extract its coordinates and the probability.
• We map each keypoint to its corresponding name from the keypoint_names list.
• We use v.draw_text to place the keypoint name on the image.

This should result in an image that displays both the keypoints as dots and their corresponding names. Hope this helps you achieve the desired visualization! 🛠️
I hope it'll help. Don't hesitate when you have questions about my suggested solution.