What is the range of seq_id?

[raulod]

can llama_seq_id random or arbitrary large value in llama_batch_add or other functions? will it affect performance?

[slaren]

You have to set the maximum sequence id in llama_context_params::n_seq_max when creating the context. It shouldn't affect performance, but it may affect memory usage of recurrent models.

[raulod]

thank you, is this because kv cache maintained per seq_id?

[slaren]

In transformers with standard attention the value n_seq_max does not make a difference for the KV cache. In recurrent models I believe that there is one context stored for each sequence.