Why my agent suddenly speaking Chinese?! (aka: how I started actually understanding what it says to users)

[philipazachi]

Built an agent with LangGraph. Talked to it a lot. Then one day I scrolled through LangSmith traces and -BAM - everything looked like Chinese calligraphy.
Literally talkin’ Chinese. No idea what went wrong.
Turns out: debugging agent conversations is hell. But if you want to build something great, you have to understand what your agent is saying to users.
I tried dumping full convo logs into GPT. Got slow, vague answers. Expensive too. So here’s what worked:
2 key lessons:
Be specific - Know the exact question you're asking (e.g., “What integrations are users asking for?”).
Be efficient - Long chats are $$, so split them up + use a cheap LLM to filter, then a better one for insights.
I built this pipeline on my platform NexCraft (because I’m lazy and like building fast). You can use it too, or DIY.
You can plug in your DB or even connect straight to LangChain traces. Super easy.
Hope this helps. Would love to hear how you analyze your agents.
If it you found it interesting, you can watch my full article in medium:
https://medium.com/@azachi12/why-my-agent-started-to-speak-chinese-d552b3967869