Async support #14
Description
Is it possible to stream the tokens with their log probs? It is acceptable to hold some tokens for a while within the pipe, but currently it seems we can only wait for the entire completion to complete before getting the tokens & probs.