Cross-Platform Chinese Offensive Comment Detection via Dual-Threshold Hard Example Mining

Cross-platform deployment of offensive comment detection for Chinese social media suffers performance degradation.

The paper proposes a dual-threshold hard mining method to address this.

First, the clean-Chinese-base RoBERTa is finetuned on COLD to establish a binary baseline for fair comparison.

Second, a three-class fine-labeled test set covering Weibo, Xiaohongshu, Tieba, and Zhihu is constructed, domain distances from the source are quantified using Jaccard and Proxy-A Distance, as well as the degradation bottleneck of the baseline under domain shift is systematically revealed.

Herein, a dual threshold hard example mining strategy is proposed.

High- and low-confidence error-prone samples are filtered from unlabeled corpora by prediction confidence.

The model is secondarily finetuned under implicit contexts with merely a small set of manually labeled hard examples, realizing low-cost cross-platform domain adaptation.

Experiments reveal significant performance gains of the optimized model across four platforms.