학술

LAGR: Language-Aware Grasp Re-Ranking for Task Oriented Grasping

arXiv CS.AI
CC BY
이 매체는 공공·자유 라이선스로 본문을 직접 표시합니다.

Abstract

Grasping is a fundamental capability for robotic manipulation, with the appropriate grasp depending on the downstream task.

For example, a hammer should be held by its handle for hammering but by its head when it is handed over.

Selecting such a grasp requires consideration of two criteria: physical feasibility on the object and suitability for the task.

Developing such an approach requires dense grasp labels with both feasibility and task scores on real observations, but existing task-oriented grasping datasets are small, synthetic, or costly to annotate.

To address this gap, we construct a dataset of 2.68 million scored 5-DoF grasp annotations over 100,911 image-task pairs from $17$ object categories.

Additionally, we introduce LAGR, a task-oriented grasp detection method that explicitly addresses both these challenges.

Given vision-language inputs, LAGR first generates task-independent grasp proposals along with a physical feasibility score, and then scores the generated grasps with a language-conditioned re-ranker that reasons over all proposals jointly.

Experiments show that LAGR outperforms the next best method by 9.3 percentage points.

We further validate LAGR on real objects by executing task-oriented grasps with a Franka Research 3 robot.

관련 뉴스