BUG: Ollama test suite in CI is a false positive. It succeeds even though the tests don't run to completion

[iambenkay]

The model names provided in the CI for Ollama differ from the names provided in the test suite yet the ollama integration test passes.

This behaviour is incorrect. Ideally, the test should fail if this kind of inconsistency exists and it should never pass.

ci.yaml

test-llm/lib.rs

Result in Ollama integration test suite (while passing successfully)

You can rework the tests to only pass when no errors are returned from the model so that the CI can fail and signal an issue.

[jdegoes]

/bounty $50

[algora-pbc]

💎 $50 bounty • Golem Cloud

Steps to solve:

1. Start working: Comment /attempt #58 with your implementation plan
2. Submit work: Create a pull request including /claim #58 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

❗ Important guidelines:

• To claim a bounty, you need to provide a short demo video of your changes in your pull request
• If anything is unclear, ask for clarification before starting as this will help avoid potential rework
• Low quality AI PRs will not receive review and will be closed
• Do not ask to be assigned unless you've contributed before

Thank you for contributing to golemcloud/golem-llm!

Attempt	Started (UTC)	Solution	Actions
🟢 @iambenkay	Jul 03, 2025, 07:47:18 PM	#59	Reward
🟢 @Kdmeenaa	Jul 04, 2025, 05:13:23 AM	#62	Reward

[iambenkay]

/attempt #58

[Kdmeenaa]

/attempt #58

[Kdmeenaa]

"I have submitted a fix with model log-checking to avoid silent failures in CI. PR: [link to your PR]. Would love your review 🙌"

[Nanashi-lab]

ollama-integration test in CI does not need to use golem-cli, cargo install golem-cli takes time, since we are using golem binary, we can use that directly.

[iambenkay]

Great suggestion @Nanashi-lab will use the binary

[Nanashi-lab]

Few more minor errors -

• build test component CI does not reflect the failure of the command
• cargo make build-all doesn't copy files to component/debug or component/release (For build-release), related the previous error

I will try to add a fix for these.