DeepSeek V3.1 (thinking/no thinking)

[Oktai15]

Describe the issue

Hi!

Could you test new DeepSeek model (in two modes) and add to leaderboard?

[klieret]

Yup, already in progress :)

[klieret]

Ahaha, good that I took that screenshot, realized I have no space on the device haha

[klieret]

It seems like the thinking variant doesn't support tool calls, so I don't think most people will use it for agents and we'll just evaluate the no-thinking variant (technically mini doesn't need/use toolcalls, but most other agents use them, so I think this is fair)

[Oktai15]

@klieret I also notice this fact, but it is still interesting how DeepSeek V3.1 (thinking) is agentic-ready. So if you have resources, it would be cool to see results for thinking mode too 🙏

[Oktai15]

@klieret, do you have any result?