Studio native viewer cannot start on macOS: NSWindow created off the main thread
Intro
Hi!
I use MuJoCo for headless physics simulation and offscreen rendering distributed across many workers, and I have been scripting the viewer to drive job submission. That is what led me into experimental/studio.
My setup
- MuJoCo 3.13.0, installed from the PyPI wheel
- API: Python
- Architecture: arm64 (Apple M2 Pro)
- OS: macOS 26.6.2
- Python 3.11.5
What's happening? What did you expect?
launch_passive and launch_native create the viewer window on a worker thread.
macOS requires NSWindow to be created on the process main thread, so on macOS
the window is never created and the viewer does not run.
What makes this hard to handle rather than merely broken:
- No exception is raised. An error is printed to stderr by native code, and
launch_passivereturns normally. handle.is_running()returnsTrueimmediately after the failure, so a caller polling it is actively misled.- The process then terminates with exit status 1 during the first
handle.sync()loop.print("finished normally")at the end of the script never runs.
Observed output:
launch_passive returned, is_running: True
ERROR: Error creating window: NSWindow should only be instantiated on the main thread!Exit status 1.
I expected either a working viewer window, or a raised exception identifying the platform limitation so that a caller can fall back.
Cause
launch_native delegates to launch_thread, which puts the viewer on a worker
thread and leaves the sim on the main thread:
# python/mujoco/experimental/studio/launch_thread.py
def launch_thread(target_fn, *, sim_plugins=None):
viewer_endpoint, sim_endpoint = make_thread_endpoints()
thread = threading.Thread(target=target_fn, args=(viewer_endpoint,), daemon=True)
thread.start()target_fn is the viewer. That arrangement is fine on X11 and Windows, and cannot
work on macOS, where AppKit only services UI objects created on the main thread.
The classic viewer solves the same problem with the mjpython launcher, which
keeps the Cocoa run loop on the main thread; Studio has no equivalent.
Suggested fix
Two options, both inside launch_native:
- Provide a blocking main-thread launcher alongside the threaded one, for callers that do not need to keep control of the main thread. This works on every platform, so it need not be macOS-specific.
- Detect macOS in
launch_passive/launch_nativeand invert the threading. This changes the contract, since the call becomes blocking, so raising a clear error that points at a blocking launcher may be preferable to changing behaviour silently.
In either case, having the current path raise rather than print and exit would let a caller detect the condition.
A working implementation of option 1 is in the last code block below. I am happy to open a PR for whichever shape you prefer.
Steps for reproduction
pip install mujoco==3.13.0on macOS.- Save
minimal.xmlandrepro.pybelow into the same directory. python repro.py
No mjpython is involved; the same result occurs under mjpython.
Minimal model for reproduction
minimal.xml, loadable as-is, no binary assets required:
<mujoco model="studio_repro">
<option timestep="0.002"/>
<worldbody>
<light pos="0 0 2"/>
<camera name="fixed" pos="0.8 -0.8 0.6" xyaxes="1 1 0 -0.4 0.4 1"/>
<geom name="floor" type="plane" size="2 2 0.05" rgba="0.4 0.45 0.5 1"/>
<body name="box" pos="0 0 0.3">
<freejoint name="box_free"/>
<geom name="box" type="box" size="0.05 0.05 0.05" rgba="0.8 0.2 0.2 1"/>
</body>
</worldbody>
</mujoco>Code required for reproduction
repro.py:
import time
import mujoco
from mujoco.experimental.studio import launch_passive, messages, viewer_protocol
model = mujoco.MjModel.from_xml_path("minimal.xml")
data = mujoco.MjData(model)
handle = launch_passive.launch_passive(
viewer_protocol.ViewerConfig(title="studio repro", width=900, height=600)
)
handle.send_to_viewer(messages.ModelEvent(model=model, path="minimal.xml"))
print("launch_passive returned, is_running:", handle.is_running(), flush=True)
for _ in range(200):
model, data = handle.sync(model, data)
time.sleep(0.02)
print("finished normally", flush=True)Working around it, and a candidate fix
Studio already separates the two sides over channels and exposes everything needed
to wire them the other way round, so no change to MuJoCo is required to get a
working viewer today. Running the viewer on the main thread and the sim on a
worker thread works on macOS with plain python:
import threading
from mujoco.experimental.studio import (launch_native, launch_thread, messages,
step_control, viewer_app, viewer_handle,
viewer_protocol)
def launch_on_main_thread(model, data, *, scene_path="", viewer_plugins=(),
sim_plugins=None, config=None):
config = config or viewer_protocol.ViewerConfig()
sim_plugins = sim_plugins or [step_control.StepControl()]
plugins = [viewer_app.ViewerApp(), *viewer_plugins]
viewer_endpoint, sim_endpoint = launch_thread.make_thread_endpoints()
closed = threading.Event()
handle = viewer_handle.ViewerHandle(
sim_endpoint, sim_plugins=list(sim_plugins),
is_alive_fn=lambda: not closed.is_set(), shutdown_fn=lambda: None)
def sim_loop():
m, d = model, data
handle.send_to_viewer(messages.ModelEvent(model=m, path=scene_path))
while not closed.is_set():
m, d = handle.sync(m, d)
thread = threading.Thread(target=sim_loop, daemon=True)
thread.start()
try:
# Blocks. This is the call that has to be on the main thread.
launch_native.run_native_viewer(config, viewer_endpoint, plugins=plugins)
finally:
closed.set()
thread.join(timeout=5)Verified on the setup above: the window opens, Filament resolves to
[Apple], [Apple M2 Pro], [4.1 Metal - 90.5], plugins receive ModelEvent and
BuildGuiEvent, and it sustains roughly 140 fps.
Confirmations
- I searched the latest documentation thoroughly before posting.
- I searched previous Issues and Discussions, I am certain this has not been raised before.
Source: google-deepmind/mujoco