Distributed Prediction under Heterogeneity with Unidentifiable Parameter

Predicting a response based on covariates is a fundamental problem in statistics and machine learning.

However, profound difficulties arise when the underlying low-dimensional structural parameters are unidentifiable, as typified in dimension reduction contexts.

Specifically,estimating these non-identifiable parameters inherently introduces severe nonconvexity.

In distributed settings, this difficulty is further compounded by the challenges of data heterogeneity and communication cost.

To overcome these intertwined barriers, we propose a novel distributed semiparametric framework.

We formulate an adaptive homogeneity pursuit utilizing a trace-similarity penalty to effectively address data heterogeneity.

To resolve the ensuing severe nonconvexity and communication bottlenecks, we introduce an invex relaxation technique coupled with a multi-step local update algorithm, ensuring stable convergence to global optimality with significantly reduced communication overhead.

Theoretically, we establish a non-asymptotic model-free prediction error bound and prove that our estimator achieves a two-phase minimax optimal convergence rate and an sharper model-free prediction error bound.

Furthermore, we provide theoretical guarantees for algorithmic convergence and communication efficiency.

Extensive simulations and a real-world multi-center medical application validate the superiority of our method.