Interpreting gamma values/averaging gamme values

[Tannervanorden]

Hello,

I am conducting a HyDe analysis on a bunch of Salvelinus species, and the gamma values I get for different triples are very different, even when P1 is the same. I will attach an example, but I wonder if averaging the gamma values when P1 and the hybrid species are the same across all significant triples would be valid or useful?

Thanks a ton,
Tanner

subset.txt

[pblischak]

Hi Tanner -- this is definitely a bit of a weird pattern. Off the bat, I would suggest double checking that the samples that you are running through HyDe are mapped correctly to the taxa that you are testing and that the samples are in the same order in the map and the Phylip file. If that is the case, I'm still not sure if taking the mean would make sense in this case. Another thing to check might be to see if the species tree topology matches the results that you've attached here. Since the gamma values are close-ish to 0.1 or 0.9, the species tree topology should still be mostly correct if you use something like ASTRAL or SVDquartets

[Tannervanorden]

Ok, sweet. I went through the last few days and checked the map file/phylip file, and what I used was correct. I have used ASTRAL to create a species tree, and the tree did give me what I was expecting based on geography. Is it possible that the pattern in my results is driven by incomplete lineage sorting? Of the 1000 genes I used with HyDe, not a single one of the gene trees is the same. If this is caused by incomplete lineage sorting, does HyDe still give meaningful results? I went through and calculated the Abba Babba statistic based on the HyDe results, and the hybridization events do make sense based on events that I know have occurred. The only problem currently is that the gamma values do not make sense. Here is the whole output from HyDe

hyde-out-filtered.txt

[pblischak]

HyDe models both hybridization and incomplete lineage sorting, so ILS shouldn't be an issue. If the ASTRAL tree gives the result you expect, then I would think that . If the support values from ASTRAL are low (or just relatively low -- like maybe < 90?), then that suggests a deviation from just ILS affecting the phylogenetic estimate of the species tree

One thing that might be happening, depending on the topology of the species tree, is that there is an ancestral hybridization event between the clade with P1 + Hybrid and the different taxa that you are using for P2 -- then subsequent mutations and things like that are leading to different estimates of gamma for the different P2s

Sorry to suggest more analyses, but I think taking the gene trees that you've put together and running them through SNaQ to see if some pattern of ancestral hybridization/admixture could explain the patterns you're seeing is probably worth it