Nov 4, 2026
This 3D World Simulator predicts what a robot sees and feels, at 50+ FPS
Researchers from Columbia, Berkeley, Michigan and NVIDIA built a 3D world model that predicts both visual and tactile signals at over 50 FPS. Robots trained inside it, then ran on a real robot.
Sir Toastimir
Video · Jimmy Wang et al. (@jwang771)
A new 3D World Simulator predicts both what a robot sees and what it feels through touch, running at more than 50 frames per second.
What it predicts
The model forecasts how objects move and deform in 3D, and what a tactile sensor on the robot's gripper would read as it makes contact.
Training inside it
The researchers trained robot policies inside the simulator on tasks such as folding cloth, plugging in a cable and pushing a T-shaped block, then ran those policies on a real robot.
Who built it
The work comes from researchers at Columbia, UC Berkeley, the University of Michigan and NVIDIA (with Analog Devices also listed), led by Jimmy Wang.
Should robots learn by feel before they touch the real world?
Links
Get your daily slice of AI
One short email with the best new AI tools, robots and AI & tech stock news. Free. Unsubscribe anytime.
We only use your email to send the newsletter.