Probing LLMs for Syntactic Structure Beyond Universal Dependencies: A Minimalist Phase Account in English

We show that LLMs encode syntactic distinctions not present in the Universal Dependencies (UD) tree distances that structural probes are trained to recover.

On English wh-movement stimuli, we measure the probe distance between an embedded subject and its verb, whose UD tree distance is invariant across conditions.

That distance is shorter than baseline when the embedded clause is finite and longer when it is infinitival -- a within-clause sign asymmetry present in all 13 models across four families we test, at a majority of layers.

No account based on UD distance, linear order, or monotone structural complexity can produce a sign reversal, while Minimalist phase theory can.

Holding the matrix verb fixed while varying only the complement type reproduces the same finite-infinitival ordering in every model, ruling out a lexical-semantic explanation.

The cross-clause pair separately reproduces the phase-count ordering of earlier work, validating the probes.

A single activation patch, interchanging the clause-selecting matrix verb, moves the two pairs in opposite directions in 8 of 13 models.

Because the causal intervention leaves the target string unchanged, it also rules out the clause-length and surface-cue explanations.

Together these results reveal syntactic structure in LLMs beyond the UD probe target, and a Minimalist phase account is consistent with it.