Rethinking Benchmarks and Models for Enzyme Specificity Prediction

Artificial Intelligence has had a profound impact on the biological sciences, and in particular has accelerated research on protein form and function.

Enzymes are no exception: a surge of predictive models have been recently developed to address a range of enzyme tasks.

Models addressing enzyme-substrate (ES) or enzyme-reaction (ER) compatibility could be especially valuable for enzyme annotation, biosynthetic pathway elucidation, and biocatalyst retrieval, the central challenge of which is the identification of a true catalyst (or truly compatible reaction) among many similar candidates.

While existing models report strong performance on alternative benchmarks, less is known about their capabilities in this regime.

Herein, we benchmark four recently released ES and ER prediction models, using tasks and datasets tailored to this setting.

We first show that two representative ES prediction models perform near random baselines across two enzyme families when considering enzymes and substrates not encountered during training.

To evaluate additional models across a consistent dataset, we next assemble the largest cytochrome P450 (CYP) reaction dataset to date, 2,922 reactions across 768 enzymes, and construct a CYP ranking benchmark requiring the correct enzyme to be prioritized among all CYPs in its native organism.

We again find that most models do not outperform sequence-based (BLAST) baselines even after fine-tuning.

We finally adapt the bimolecular structure prediction model Boltz to ES prediction by training supervised classifiers on residue-ligand pair embeddings, and show that this approach consistently surpasses the BLAST baselines on our CYP ranking benchmark.

Together, our results argue for more discovery-relevant benchmarking and suggest that interaction-aware representations from full biomolecular complexes may provide a promising basis for enzyme prioritization.