Thresholds evaluated incorrectly #4283
Description
Brief summary
When test script has errors thresholds is evaluated incorrectly indicating success instead of failure.
k6 version
0.49
OS
macOS 14.6
Docker version and image (if applicable)
No response
Steps to reproduce the problem
Run test with below script:
import { check } from "k6";
import http from "k6/http";
export const options = {
stages: [
{ duration: "2s", target: 1 }
],
thresholds: {
"checks": [{ threshold: "rate>=0.9", abortOnFail: true }]
}
}
export default function (setupData) {
const result = http.get("https://test-api.k6.io/public/crocodiles/");
throw new Error("This is an error");
check(result, {
"status:200": result.status === 200,
});
}
Test is not failed even though checks metric value is 0. The summary json also indicates same:
"checks": {
"type": "rate",
"contains": "default",
"values": {
"passes": 0,
"fails": 0,
"rate": 0
},
"thresholds": {
"rate>=0.9": {
"ok": true
}
}
},
Expected behaviour
Test should fail when thresholds is breached even if test script has errors.
Actual behaviour
Test is passed even though thresholds is breached.