Multi-Source Transfer Learning of Sparse Single-Index Models

Transfer learning leverages knowledge from related source domains to improve learning in a target domain.

Recent theoretical advances cover a broad range of regression settings within (generalized) linear models.

Despite their diversity, these methods share two common constraints: they assume a known link function or linear structure and require direct access to raw source data.

To move beyond these constraints, we propose a source-data-free transfer learning framework based on the single-index model (SIM).

Instead of requiring raw source data, our method transfers only summary statistics derived from a generalized Stein's lemma in a one-time communication.

This design preserves privacy and avoids side effects caused by dissimilarities of unknown nonlinear link functions across domains.

To capture flexible, unknown nonlinearity, we employ a multilayer perceptron guided by the pre-estimated index from the transferred statistics, which significantly mitigates overfitting.

Extensive experiments on synthetic data and a real-world application demonstrate consistent improvements over existing (generalized) linear model-based approaches.

The proposed framework thus offers a practical, privacy-preserving, and nonlinear-adaptive solution for transfer learning.