Problem using a wndow viewer and an offscreen viewer at the same time

[jodyxha]

I would like to have a simulation (which is shown in a window) of which images are taken at regular intervals.
So i added a offscreen viewer to the example code like this

import mujoco
import mujoco_viewer

model = mujoco.MjModel.from_xml_path('~/mujoco/mujoco-2.3.5/model/humanoid/humanoid.xml')
data = mujoco.MjData(model)
# create the viewer objects
viewer = mujoco_viewer.MujocoViewer(model, data)
viewer1 = mujoco_viewer.MujocoViewer(model, data, 'offscreen')
imgs = []
for i in range(10000):
    if viewer.is_alive:
        mujoco.mj_step(model, data)
        viewer.render()
        if (i % 100) == 0:
            print("Doing so,for %d"%i)
            img = viewer1.read_pixels(camid=0)
            imgs.append(img)
    else:
        break
viewer.close()
viewer1.close()
# do something with the images

When i do it this way, the normal window is not opened. When i exchange the order of the viewer creations, the viewer window opens but the window remains black.
Is it possible at all to have a window viewer and a offscreen viewer in the same simulation?

[p3jawors]

Hey @jodyxha ,

I ran into this problem too. However, rendering with multiple viewers is possible by ensuring that the OpenGL context is activated for the desired window before rendering. You can achieve this by using GLFW's glfw.make_context_current() function and specifying the appropriate viewer to indicate where GLFW should render.

import mujoco
import mujoco_viewer
import glfw

model = mujoco.MjModel.from_xml_path('~/mujoco/mujoco-2.3.5/model/humanoid/humanoid.xml')
data = mujoco.MjData(model)
# create the viewer objects
viewer = mujoco_viewer.MujocoViewer(model, data)
viewer1 = mujoco_viewer.MujocoViewer(model, data, 'offscreen')
imgs = []
for i in range(10000):
    if viewer.is_alive:
        mujoco.mj_step(model, data)
        glfw.make_context_current(viewer.window)
        viewer.render()
        glfw.make_context_current(None)
        if (i % 100) == 0:
            print("Doing so,for %d"%i)
            glfw.make_context_current(viewer1.window)
            img = viewer1.read_pixels(camid=0)
            glfw.make_context_current(None)
            imgs.append(img)
    else:
        break
viewer.close()
viewer1.close()
# do something with the images

By ensuring that the correct context is activated before rendering, you help OpenGL know which window's state to apply the rendering commands to. Remember that OpenGL contexts are not automatically switched when you change the active window, so you need to explicitly activate the context of the window you're rendering to. There may be a small impact by turning the context on and off all the time, so you can add some logic in there to ignore switching the viewer context off if you're waiting for your i % 100 counter.

Cheers,

Pawel