Changing object rotation between scenes efficiently

[jeehart]

Hello,
I am generating a dataset that consists of a single object in the same position, only its rotation changes. It's to train a model that learns object rotation.
Currently, I have a blenderproc code that generate and save a scene with an object with a specific rotation and then I call it with multiple rotations using a script. My code is the following :

import blenderproc as bproc
import argparse
import numpy as np
from PIL import Image 

import pathlib


parser = argparse.ArgumentParser()
parser.add_argument('--obj_model_path', nargs='?', help="Path to the model file")
parser.add_argument('--output_dir', nargs='?', help="Path to where the final files will be saved")
parser.add_argument('--obj_rot', nargs='?', type=str, help="Rotation of the object")
args = parser.parse_args()

bproc.init()
object_path = args.obj_model_path
output_dir = args.output_dir 
# load the objects into the scene

obj = bproc.loader.load_obj(object_path)[0]
# Use vertex color for texturing

for mat in obj.get_materials():
    mat.map_vertex_color()
    
# Set pose of object via local-to-world transformation matrix
rot_list_str = args.obj_rot.replace('[', '').replace(']', '')
rot = [float(e) for e in rot_list_str.split(",")]

trans = [0, 0, 0]
pose = np.vstack((np.hstack([np.array(rot).reshape(3, 3), 
                  np.array(trans).reshape(3, 1)])
                  ,[0, 0, 0, 1]))
obj.set_local2world_mat(pose.tolist())

# Scale 3D model from mm to m
obj.set_scale([0.001, 0.001, 0.001])
# Set category id which will be used in the BopWriter
obj.set_cp("category_id", 1)

# define a light and set its location and energy level
light = bproc.types.Light()
light.set_type("POINT")
light.set_location([0, 0, -0.5])
light.set_energy(1000)

# Set intrinsics via K matrix
pic_dim = 512
bproc.camera.set_intrinsics_from_K_matrix(
    [[537.4799, 0.0, pic_dim/2],
     [0.0, 536.1447, pic_dim/2],
     [0.0, 0.0, 1]], pic_dim, pic_dim
)
# Set camera pose via cam-to-world transformation matrix
cam2world = np.array([
    [1, 0, 0, 0],
    [0, 1, 0, 0],
    [0, 0, 1, -0.2],
    [0, 0, 0, 1]
])
# Change coordinate frame of transformation matrix from OpenCV to Blender coordinates
cam2world = bproc.math.change_source_coordinate_frame_of_transformation_matrix(cam2world, ["X", "-Y", "-Z"])
bproc.camera.add_camera_pose(cam2world)

# activate depth rendering
bproc.renderer.enable_depth_output(activate_antialiasing=False)

# render the whole pipeline
data = bproc.renderer.render()

# Write object poses, color and depth in bop format
# print(output_dir)
bproc.writer.write_bop(output_dir, [obj], data["depth"], data["colors"], m2mm=True, append_to_existing_output=True)

I am wondering if I am doing it the good way or there is a more efficient one ?
When sampling camera position, in the examples we generate a scene and then we sample multiple camera positions and then the rendering is done. Can I do the same with object rotation?

Thank you

[njsymanz]

Rather than using the set_local2world_mat method, you can use set_rotation_mat instead and set the frame argument just like with the camera positions. Your for loop might look something like this:

for i in range(len(rot)):
    obj.set_rotation_mat(np.array(rot[i]).reshape(3, 3), frame=i)