WattLayer: Get Layers Right to Estimate Inference Energy of Neural Networks

The widespread adoption of Artificial Intelligence (AI) has led to increasing concerns about energy consumption, yet there is a lack of standardized methodologies to accurately estimate AI inference energy consumption, particularly across various tasks and architectures.

In this study, we propose a task independent, layer-wise energy estimation model for AI architectures.

Our model is evaluated on a large dataset of more than 100,000 layers for 295 neural network architectures across 3 widely-used tasks and 3 distinct hardware platforms.

Our approach achieves a median error of 19.6%, outperforming state-of-the-art methods.

We further show that layer-wise decomposition generalize to new tasks without complete retraining, by leveraging shared layers across architectures.

It offer tools, insights and a precise methodology to empower stakeholders in designing energy-efficient AI systems.