학술

Robustness of Text-Only Adapted Speech LLMs for Spoken Semantic Parsing

arXiv CS.AI
CC BY
이 매체는 공공·자유 라이선스로 본문을 직접 표시합니다.

Abstract

Large language models (LLMs) are increasingly used to generate structured outputs from natural language instructions, including semantic frames and function calls.

When the input is speech, this task becomes spoken semantic parsing, which can be performed using either Speech LLMs or ASR-LLM cascades.

Although both approaches can be adapted with text-only task supervision, their relative robustness to acoustic degradation remains unclear.

We compare these approaches under matched text-form supervision using 3B and 7B models across three datasets, evaluated on clean speech and babble noise at five SNR levels.

Results show that Speech LLMs generally achieve better performance in command understanding, intent prediction, and function calling, particularly under severe noise, while improvements in fine-grained entity prediction are less consistent.

An additional evaluation shows that Speech LLMs achieve lower average inference time per sample than ASR-LLM cascades at both model scales.

관련 뉴스