NEXUS: Structured Runtime Safety for Tool-Using LLM Agents

Tool-using LLM agents increasingly execute high-impact actions, making runtime safety monitoring essential.

We present NEXUS (Neural EXecution Utility and Safety), a structured-plan safety monitor that applies a formal intervention policy to select among four actions: allow, block, request confirmation, or request revision.

NEXUS combines deterministic safety rules, argument-level inspection, and a calibrated logistic-regression risk score for graded escalation.

On a 128-instance synthetic benchmark, NEXUS achieves an F1 score of 0.949 and a 4-class intervention accuracy of 0.6406, outperforming rule-only intervention selection by 27.3 percentage points.

It also improves over rule-only on R-Judge (F1 = 0.861 vs.

0.849), matches rule-only on AgentHarm due to threat-model limits, and achieves 0% ASR at 99% control allow on IPI.

On the rule-blind NEXUS-Stress benchmark, NEXUS reaches an F1 score of 0.881, highlighting the difficulty of fine-grained intervention routing.

With 0.205 ms median latency, NEXUS adds under 0.1% overhead to typical agent loops.

Code, benchmarks, and the calibrated risk scorer are publicly released.