How to a model forward and obtain logits?

[arunpatro]

Just dug into the code and looks like the way to obtain forward logits is via the llm.llm_engine.step(). I am trying to integrate this api into outlines. But I feel I would need to manage things like unfinished requests and the scheduler.

Is there a simple way to just call model.forward() or model.__call__() equivalent in hf transformers?

[LorenzoBoccaccia]

it seems it's in the scheduler update class, it generates batches of output for the various prompt under process, but the sequence is generated in blocks, not one token at a time, so there may be some waste if a sequence doesn't match the constraints