Results
Every method. Side by side.
Drawn at random with a fixed seed, before we looked at any result. Each dataset has at least one recorded instruction and one for another object.
One video per method and item, seed 0. Under each video: the VLM judge. Object: the robot moves the named object. Goal: it ends at the goal. Pass: both, and no severe visual flaw. We picked the Examples: items where baselines often fail, items where Slotlight passes and the fine-tuned Wan2.2-5B fails, and all instructions of one first frame. We did not pick the Random items. The numbers below cover all test items.
Numbers
Six datasets. Every test item.
Table 1 of the paper.
- Object
- % of items where the robot moves the named object.
- Goal
- % where the object ends at the goal.
- Pass
- % with both, and no severe visual flaw.
- FVD, JEDi
- Video quality against the recordings. Lower is better.
- —
- The method does not run on this dataset.
- Bold, underlined
- Best, second best.
Why slots
Remove the slots. The wrong object moves.
Same example, same seed.
Table 2 of the paper. Each variant removes one design. Grid tokens: a fixed grid of tokens replaces the object slots. Maps only: the renderer reads each slot's map but not its vector. The full model is the Table 1 model; its predictor trains for 16,000 steps, and those of the two variants for 8,000 steps.
Applications
Put the slots to work.
A plan a robot can follow.
LIBERO, simulator rates (%), on instructions that name another object of a trained scene. Success: the named object reaches the goal. Wrong: another object rises first or reaches the goal. Our plan: the object to move and its end place, from our slots. True goal (gray row): the simulator gives the object and the place, the upper bound of the grasp script. The grasp script reads a depth image, so this tests the choice of object and place. Fast-WAM sees its own camera view, mirrored as in its training data. Its video here is mirrored back to match ours.
Edit the plan, not the pixels.
Change a slot and render again. No training.