Add token streaming support for XMLAdapter

[chenmoneygithub]

Support XMLAdapter in StreamListener

Note

This PR is generated by OpenAI codex, I am testing how deep it can understand DSPy codebase with this specific use case.

It passes on my tests:

import asyncio

import dspy

# lm = dspy.LM("gemini/gemini-2.5-flash-preview-04-17", cache=False)
lm = dspy.LM("databricks/databricks-claude-3-7-sonnet", cache=False)
# lm = dspy.LM("openai/gpt-4o-mini", cache=False)
dspy.settings.configure(lm=lm, adapter=dspy.XMLAdapter())


class MyModule(dspy.Module):
    def __init__(self):
        super().__init__()

        self.predict1 = dspy.Predict("question->answer")
        self.predict2 = dspy.Predict("question, answer->judgement, score")

    def forward(self, question: str, **kwargs):
        answer = self.predict1(question=question)
        judgement = self.predict2(question=question, answer=answer)
        return judgement


predict = MyModule()
stream_listeners = [
    dspy.streaming.StreamListener(signature_field_name="answer"),
    dspy.streaming.StreamListener(signature_field_name="judgement"),
]
stream_predict = dspy.streamify(
    predict,
    stream_listeners=stream_listeners,
    include_final_prediction_in_output_stream=False,
)


async def main():
    with dspy.settings.context(adapter=dspy.XMLAdapter()):
        output = stream_predict(question="why did a chicken cross the kitchen?")

        return_value = None
        async for value in output:
            if isinstance(value, dspy.streaming.StatusMessage):
                print(value.message)
            elif isinstance(value, dspy.Prediction):
                print(value)
            elif isinstance(value, dspy.streaming.StreamResponse):
                print(value)
            else:
                print(value.choices[0].delta.content)
    return return_value


import time

start = time.time()
asyncio.run(main())
end = time.time()


print(f"Time taken: {end - start} seconds")

dspy.inspect_history()

[TomeHirata]

Can we also make start_indicator an instance variable like we do for start_identifier and end_identifier?

[TomeHirata]

As I mentioned above, maybe it's better to move the logic into each adapter class and remove if-else branch

start_identifier = settings.adapter.get_start_identifier(self.signature_field_name)
end_identifier= settings.adapter.get_end_identifier(self.signature_field_name)
start_indicator= settings.adapter.get_start_indicator()

[TomeHirata]

nit: Can we define const for a list of adapters that support streaming?

[chenmoneygithub]

sg!

[TomeHirata]

Since we already have three adapters, shall we define a mapping for adapter-specific parameters? Alternatively, we can move the identifier into each adapter class for better encapsulation

[chenmoneygithub]

shall we define a mapping for adapter-specific parameters?
Yes, good call!

we can move the identifier into each adapter class for better encapsulation
This is a little subtle, the start_identifier/end_identifier is not a feature of the adapter, but a feature of the stream listener. The boundary is a bit vague though - adapter can fully function without start_identifier/end_identifier, and different listeners could introduce different start_identifier/end_identifier for the same adapter, so it belongs more to the stream listeners.

[TomeHirata]

q: why do we need this change for ChatAdapter?

[chenmoneygithub]

good catch, it's a codex mess

[TomeHirata]

ditto

[chenmoneygithub]

done!

[TomeHirata]

LGTM