Issue with segmap missing out most objects

[Sodek93]

Hi again,
i am currently facing an issue with coco annotations.
When i call

seg_data=bproc.renderer.render_segmap(map_by=(["instance","class","name"]))

in Line 64

colors, num_splits_per_dimension, objects = _colorize_objects_for_instance_segmentation(objs_with_mats,
                                                                                        use_alpha_channel,
                                                                                        render_colorspace_size_per_dimension)

objects variable contains all Objects i need an annotation for.

But in Line 115 to ~120 max_id gets assigned 3 - even if i have 10 Objects.

segmap = Utility.map_back_from_equally_spaced_equidistant_values(segmentation,
                                                                 num_splits_per_dimension,
                                                                 render_colorspace_size_per_dimension)
print(segmap)
segmap = segmap.astype(optimal_dtype)
print(segmap)
object_ids = np.unique(segmap)
print("object_ids",object_ids)
max_id = np.max(object_ids)
print("MAXID",max_id)

so object_ids=[0,1,2,3] and len(objects)=10 i.e.
Followed by that in line ~160

for object_id in object_ids: 
        current_obj = objects[object_id]

My "high_id" numbers are not included in everything what follows.

Why does segmap ignore my other objects?
I can't give you a minimum example again :-( hope someone knows what the issue i am facing is and how i can get rid of it.

Here are my cp's from Blender

[themasterlink]

Hey,

sorry I can't completely follow your problem. Can you please try to describe your problem from another perspective? Don't look into the code. Tell me what you put in and what you expect it puts out. Then I can tell you how you get there.

My "high_id" numbers are not included in everything what follows.

Are they in the image? Only objects present in the image, will be rendered and end up in the segmap, if they are not visible the value does not appear in the final image.

Why does segmap ignore my other objects?

I don't fully get what you mean, only visible objects are rendered, as I said if the object is not visible it can not be rendered.

PS: You can also save all other values to the segmap:

bproc.renderer.render_segmap(map_by=(["instance","class","name", "cp_classname"]))

Best,
Max

[themasterlink]

Hey,

with that we can work. So, you have used this example and in it everything works: https://dlr-rm.github.io/BlenderProc/examples/advanced/coco_annotations/README.html

Correct? If that is the case, can you swap the scene.obj with one of your models saved as an .obj file. Does this work?

Best,
Max

[Sodek93]

Yep, does work.

[Sodek93]

World and Cup is in Scene, cup gets annotation with name Suzanne.

I got to point out that currently with my code not all annotations are missing, just most of them. I wrote that wrong in my answer before,sorry!

[themasterlink]

So, can you change the coco_annotations example so that it has the same problem? That would massively help with debugging the problem.

cup gets annotation with name Suzanne.

Have you checked the data or is this shown in the visualizer?

Best,
Max

[Sodek93]

This is shown in the visualizer.

I'll try to reproduce it with the example. Tried it with the Panoramic cam but still works fine

[Sodek93]

I think i found something!

i added my render settings(and some other parts of my code before,checked after every paste) and only 1 of 7 elements are annotated now.

RendUT.set_denoiser("OPTIX")
RendUT.set_light_bounces([20,20,20,150,20,20,20])
RendUT.set_max_amount_of_samples(100)
RendUT.set_noise_threshold(0.01)
bpy.context.scene.cycles.film_exposure=exposure
bpy.context.scene.cycles.device="GPU"

The picture i get looks like this:

Before all objects were annotated:

[cornerfarmer]

The problem is probably setting the bpy.context.scene.cycles.film_exposure, which is not handled by blenderproc yet. If you are directly using bpy outside of blenderproc, there might always be a possibility that it leads to complications.

For the future: Please provide a minimal example in your initial problem statement. This helps us a lot and would have prevented the whole discussion.

[Sodek93]

Seems to work now!
I thought i couldn't provide a minimal example because i didn't know where to start, but just adding my code to the working example is a great idea - i am still learning. Thank you guys a lot :-)

[themasterlink]

Great,

please, next time, try to start from an example and if you run into a problem, provide a minimal example so that we can see what is going on.

Best,
Max

[Sodek93]

Weekend-Time. Have a nice one!

I hope this is the only cause of the Issue.
Thank you for your ongoing help!