You can find pdf and reference to my paper and related project.
Description:
This paper introduces a principled approach to model merging in large language models by leveraging model kinship, a measure of similarity between models, to guide continual merging. Drawing inspiration from biological evolution, we propose a Top-k Greedy Merging strategy that uses model kinship to improve performance and avoid local optima. Empirical results demonstrate that model kinship is a reliable indicator for selecting merge candidates and sustaining performance gains throughout model evolution.