How Can AI Find My Model? A Model-Finding Experimental Study Considering Data Formats, Embeddings, and Retrieval Strategies

Discovering simulation models for reuse remains a fundamental challenge in Modeling and Simulation (M&S).

When many models coexist, identifying those that align with a given modeling intent remains difficult.

Recent advances in Artificial Intelligence (AI), particularly retrieval-based approaches, offer a promising pathway to operate at this semantic layer.

In this paper, we present an experimental study investigating the impact of data representation, transformer-based embedding models, and retrieval strategies on the discovery of simulation models using natural language queries.

We evaluated performance across multiple query types using standard information retrieval metrics, including recall@5 and nDCG@5.

Results show that data representation matters, open-source embedding models can achieve high performance, and reranking methods are important, especially as query complexity increases.

This work provides a baseline for AI-driven model discovery and discusses its role in advancing toward AI-driven composability and interoperability.