DuoCook: An Overcooked-Inspired Benchmark for Visual Planning and Coordination
Abstract
Coordinating two players in a cooking task requires a vision-language agent to track changing scene states, sequence recipe steps, and allocate shared resources under a time budget.
We introduce an Overcooked-inspired benchmark for evaluating these capabilities in Unreal Engine environments constructed using coding agents.
A single vision-language model controls both players from top-down visual observations \rev{through separate per-player decision calls}, selecting high-level actions \rev{and image-labeled target cells} while the engine executes navigation.
The benchmark contains 20 maps in five categories.
These categories support evaluation of \rev{two-player} coordination under equipment contention, changing terrain, and handoff dependencies.
A prop legend and pot ingredient-count indicators provide explicit visual cues, while grid-based destinations support spatial grounding\rev{; text gives no station coordinates, so target locations come mainly from the image}.
The evaluation compares \rev{five VLMs and a privileged scripted reference policy on all 20 maps}, measuring task score, order completion, delivery precision, \rev{aim accuracy}, and model API expenditure. \rev{Two paired ablations on all 20 maps toggle a shared self-authored notebook and reduce the textual prompt from about 3,600 to about 440 words while keeping the images fixed.} \rev{The strongest model completes 35\% of orders and exceeds the scripted reference, which has ground-truth station access; the notebook's effect is mixed in sign and not statistically reliable, and reducing the prompt text lowers the score of two models while leaving the other three nearly unchanged.} Together, the environment design and evaluation protocol provide a framework for investigating how visual state tracking and task scheduling contribute to coordinated cooking behavior.