raise StopIteration() with tsinfer.infer

[nage0178]

Hello,

I am trying to use tsinfer and tsdate on simulated data. The data are simulated outside of tskit and then read in. Most of my simulated datasets run without error. However for a handful of my simulated datasets, I am getting the error below. The vcz file appears to be read in correctly. I have not been able to find any documentation or posts suggesting what the issue may be. Can you suggest any solutions or possible formatting or assumption errors to check?

Here is the script I am running.

import zarr
import tsinfer
import tsdate
import sys

file = sys.argv[1]


vdata = tsinfer.VariantData(file , ancestral_state="variant_AA")
 
inferred_ts = tsinfer.infer(vdata)
sim_ts = inferred_ts.simplify()
print(f"Inferred a genetic genealogy for {inferred_ts.num_samples} (haploid) genomes")
output_ts = tsdate.date(sim_ts, mutation_rate=2e-8)
print(output_ts.tables.mutations)

Here is the error message.

  File "/home/anna/simulations/mid/rep20/l2/../../../tsinfer_date.py", line 11, in <module>
    inferred_ts = tsinfer.infer(vdata)
  File "/home/anna/simulations/tsdate/lib/python3.10/site-packages/tsinfer/inference.py", line 367, in infer
    inferred_ts = match_samples(
  File "/home/anna/simulations/tsdate/lib/python3.10/site-packages/tsinfer/inference.py", line 1630, in match_samples
    ts = manager.finalise(overlay_non_inference_sites)
  File "/home/anna/simulations/tsdate/lib/python3.10/site-packages/tsinfer/inference.py", line 2770, in finalise
    ts = self.get_samples_tree_sequence(overlay_non_inference_sites)
  File "/home/anna/simulations/tsdate/lib/python3.10/site-packages/tsinfer/inference.py", line 2912, in get_samples_tree_sequence
    ts = insert_missing_sites(
  File "/home/anna/simulations/tsdate/lib/python3.10/site-packages/tsinfer/inference.py", line 1746, in insert_missing_sites
    tree = next(trees)
  File "/home/anna/simulations/tsdate/lib/python3.10/site-packages/tskit/trees.py", line 4013, in __next__
    raise StopIteration()
StopIteration

Many thanks,
Anna

[benjeffery]

Well this is very odd. Could you share the vcz file that causes this error?

[nage0178]

seq.vcz.tar.gz

Here's the file.

Thanks,
Anna

[benjeffery]

Thanks for the file!
The reason that the error is happening is that you have sites with a position that is equal to the contig_length that is set in the VCF zarr. This site is then "beyond the end" of the sequence.

We should be doing a better job of raising a more helpful error here. I've filed issue #1028 to track that work.

You can override the sequence length to allow your code to complete:
vdata = tsinfer.VariantData(file, ancestral_state="variant_AA", sequence_length=1000001)

[nage0178]

Thanks, this works for all of my runs that failed!