학술

DuoCook: An Overcooked-Inspired Benchmark for Visual Planning and Coordination

arXiv CS.AI
CC BY
이 매체는 공공·자유 라이선스로 본문을 직접 표시합니다.

Abstract

Coordinating two players in a cooking task requires a vision-language agent to track changing scene states, sequence recipe steps, and allocate shared resources under a time budget.

We introduce an Overcooked-inspired benchmark for evaluating these capabilities in Unreal Engine environments constructed using coding agents.

A single vision-language model controls both players from top-down visual observations \rev{through separate per-player decision calls}, selecting high-level actions \rev{and image-labeled target cells} while the engine executes navigation.

The benchmark contains 20 maps in five categories.

These categories support evaluation of \rev{two-player} coordination under equipment contention, changing terrain, and handoff dependencies.

A prop legend and pot ingredient-count indicators provide explicit visual cues, while grid-based destinations support spatial grounding\rev{; text gives no station coordinates, so target locations come mainly from the image}.

The evaluation compares \rev{five VLMs and a privileged scripted reference policy on all 20 maps}, measuring task score, order completion, delivery precision, \rev{aim accuracy}, and model API expenditure. \rev{Two paired ablations on all 20 maps toggle a shared self-authored notebook and reduce the textual prompt from about 3,600 to about 440 words while keeping the images fixed.} \rev{The strongest model completes 35\% of orders and exceeds the scripted reference, which has ground-truth station access; the notebook's effect is mixed in sign and not statistically reliable, and reducing the prompt text lowers the score of two models while leaving the other three nearly unchanged.} Together, the environment design and evaluation protocol provide a framework for investigating how visual state tracking and task scheduling contribute to coordinated cooking behavior.

관련 뉴스