Open
Description
What happened?
If a dspy.Image is specified as a Signature's output, it does not generate an image but will either pull a stock image or hallucinate a url.
Currently, models such as Openai 4.1/Google's Gemini support generating images natively in their responses api. It'd be great to support that directly. If that's not possible or is far away, then there should probably either be a warning if an image is specified or it should be restricted from being an output image.
Steps to reproduce
import dspy
lm = dspy.LM(model="openai/gpt-4.1")
dspy.configure(lm=lm)
class Image(dspy.Signature):
image_description: str = dspy.InputField()
image: dspy.Image = dspy.OutputField()
if __name__ == "__main__":
try:
image = dspy.Predict(Image).__call__(image_description="A picture of a cat")
print(image)
except Exception as e:
print(e)
print(lm.inspect_history())
raise e➜ codewalker git:(main) ✗ uv run imagegen.py
Prediction(
image=Image(url='https://images.unsplash.com/photo-1518717758536-85ae29035b6d?auto=format&fit=crop&w=600&q=80')
)
DSPy version
2.6.23