Safeguards for Speech2Speech LLM-Assistants: A Case Study in Automotive Applications

Recent advances have introduced speech-to-speech (S2S) conversational assistants capable of producing natural-sounding interactions, including non-verbal cues like tonality and mood.

In the automotive domain, this enables intuitive and humanlike in-car dialogue experiences.

However, integrating these end-to-end assistants limits architectural options for programmable domain-specific safeguards.

This paper discusses two implementation approaches for S2S guardrails: transcript-based and tool-based.

Through an empirical evaluation, we demonstrate that both strategies are insufficient for industrial deployment in most cases due to prohibitive latency (delaying each answer by 0 to 1.4 seconds even for computationally cheap checks) and technical impediments (like potentially non-deterministic tool call behavior).

Finally, we outline open challenges for S2S guardrails in the automotive context.