ATRI Framework Mitigates LLM Self-Evolution Degeneration via Information Gain

The ATRI framework uses learnable information gain, calculated via KL divergence and entropy change, to prevent performance decline during iterative self-evolution. It estimates gain by fitting a small language model to previous data and scoring new samples by negative log-likelihood to reweight training data.