Question about generate()

[arijitray1993]

Thanks for the great work!

What is the generate_mode in forward() function in your modified Qwen2_5_VLForConditionalGeneration? To my understanding, shouldn't that be set to True in the prepare_inputs_for_generation() under the if cache_position[0] != 0: part? Otherwise it seems to try to recompute the latent embeddings at every step during the generation process. Or, am I missing something here?

Thanks a lot!