feat: Enable web dashboard by default

[szkiba]

What

This PR changes the behavior of the built-in web dashboard to be enabled by default.

• The dashboard now starts automatically with k6 run.
• The opt-in K6_WEB_DASHBOARD environment variable and webDashboard JSON property have been removed.
• New opt-out mechanisms have been introduced to disable the dashboard:
  • Environment variable: K6_NO_WEB_DASHBOARD=true
  • JSON config property: noWebDashboard: true

NOTE:

The webDashboard property and K6_WEB_DASHBOARD environment variable are no longer supported. Users who explicitly set these to true will now have the dashboard enabled by default and should remove the old settings. To disable the dashboard, they must migrate to the new noWebDashboard or K6_NO_WEB_DASHBOARD options.

Why

The primary motivation is to improve the out-of-the-box user experience. By enabling the dashboard by default, we provide immediate, real-time visual feedback for test runs without requiring manual configuration. This makes a powerful feature more discoverable and provides instant value to both new and existing users. The opt-out mechanism ensures that users who run k6 in environments where a web server is not desired (e.g., CI/CD) retain full control.

---

Closes #4903

[ankur22]

To support environments where a web server is not desired (e.g., CI/CD),
the logic has been inverted from opt-in to opt-out. The dashboard can
now be explicitly disabled.

What will happen in CI/CD pipelines? Could this change break those?

[szkiba]

In the CI/CD environment, where it is not disabled, the web dashboard is started on IP address 127.0.0.1 on port 5665. If multiple k6s are running on the same host (e.g. non-containerized environment), if there is a port conflict on the default port, the dashboard will only log it and not consider it a fatal error. This means there will be no breaking change in the CI/CD environment.

[ankur22]

LGTM 🚀

[joanlopez]

I'd suggest to not reverse the configuration attribute name, but just its default value. Otherwise, it would be a breaking change, and I don't see much value on doing so.
Wdyt?

[szkiba]

@joanlopez

I'd suggest to not reverse the configuration attribute name, but just its default value. Otherwise, it would be a breaking change, and I don't see much value on doing so. Wdyt?

I reversed the name of the configuration attribute so that it would be consistent with other attributes with similar roles (like K6_NO_USAGE_REPORT). It doesn't cause a breaking change, because until now the K6_WEB_DASHBOARD environment variable was set in cases where they wanted to turn it on. Now it's turned on. I think it's important that the configuration attributes are coherent in the long term.

[oleiade]

Hey @szkiba 👋🏻

Thanks a lot for putting this together! I think it would make sense to find a way to provide the web-dashboard experience as a default, so it gets the visibility it deserves.

However, I have some concerns about the current approach that I think we should make sure to address before we move forward with it (keep in mind I don't have the full context in mind anymore):

Breaking change impact

I believe this can be considered a breaking change since we're modifying the options structure (removing an existing option) and flipping a previously opt-in behavior to default. While the user-experience improvement is valuable, introducing breaking changes in a minor release goes against our new versioning/maintenance policy, and could disrupt existing integrations and workflows. But I might be wrong or misinterpreting though, let me know.

Performance Considerations

I don't remember if we had thoroughly tested the performance impact of enabling the web dashboard by default across different scenarios. I think we might have though, and if such, could you point us to the results? Given that this will now run for every k6 execution, we should ensure that we have a good idea of the overhead it would introduce.

Ecosystem coordination

This change will most likely require some alignment and coordination across our internal ecosystem, as well as public one (k6-operator?), in order to explicitly disable the dashboard where appropriate. We should have a clear migration plan and timeline for these updates, before we proceed.

Solutions

I'd like to offer alternative solutions:

1. Non-breaking transition: Is there a way we could make these changes without having to drop an existing option, and/or change the default behavior of k6 for users within this major cycle: improve the visibility of this feature without having to switch to default enable in the meantime: better docs, cli hints, etc?
2. Plan for v2: Plan this as part of our next major release where breaking changes are expected.
3. Gradual rollout: Perhaps we could also look into way to "decide" whether the dashboard can be enabled by default, or at least proposed to the user based on what kind of test we run, and in what environment.

To be clear, I'm definitely not against the end goal, but I want to make sure we're thoughtful about the implementation path. What do you folks think?

[szkiba]

Hey @szkiba 👋🏻

Thanks a lot for putting this together! I think it would make sense to find a way to provide the web-dashboard experience as a default, so it gets the visibility it deserves.

However, I have some concerns about the current approach that I think we should make sure to address before we move forward with it:

Breaking change impact

This is fundamentally a breaking change since we're modifying the options structure and flipping a previously opt-in behavior to default. While the user-experience improvement is valuable, introducing breaking changes in a minor release goes against our new versioning/maintenance policy, and could disrupt existing integrations and workflows.

Performance Considerations

I don't remember if we had thoroughly tested the performance impact of enabling the web dashboard by default across different scenarios. I think we might have though, and if such, could you point us to the results? Given that this will now run for every k6 execution, we should ensure that we have a good idea of the overhead it would introduce.

Ecosystem coordination

This change will most likely require some alignment and coordination across our internal ecosystem, as well as public one (k6-operator?), in order to explicitly disable the dashboard where appropriate. We should have a clear migration plan and timeline for these updates, before we proceed.

Solutions

I'd like to offer alternative solutions:

1. **Delay for v2**: Plan this as part of our next major release where breaking changes are expected.

2. **Non-breaking transition**: Is there a way we could improve the visibility of this feature without having to switch to default enable in the meantime: better docs, cli hints, etc?

3. **Gradual rollout**: Perhaps we could also look into way to "decide" whether the dashboard can be enabled by default, or at least proposed to the user based on what kind of test we run, and in what environment.

To be clear, I'm definitely not against the end goal, but I want to make sure we're thoughtful about the implementation path. What do you folks think?

Hi @oleiade

Thank you for the thorough review and for outlining your concerns. I appreciate you taking the time to provide this detailed feedback.

Regarding the breaking change impact, I completely agree with the principle of avoiding breaking changes in minor releases. To help me better understand the full scope, could you elaborate on some of the use cases you believe would be disrupted? I'm trying to picture if this would affect CI/CD pipelines that parse logs, specific integrations, or other workflows I might not be aware of. Your perspective here would be very helpful.

On the topic of performance considerations, I agree that having solid performance data is essential before moving forward. I wanted to share some context from my previous attempt to run these tests about a year and a half ago, where I ran into some difficulties with the performance test suite itself. It seemed at the time that it wasn't in a state where I could easily get a clean measurement.

With that in mind, has the state of the performance test suite changed recently? If it's now more accessible, I would be happy to give it another shot. If the suite itself still requires maintenance, I agree that this is a blocker. In that case, I'd appreciate the core team's guidance on whether they have the capacity to investigate the performance impact, or if there are resources I could use to help get the tests running.

Thanks again for the crucial feedback. I'm keen to find the right path forward for this.

[inancgumus]

@szkiba I believe this PR takes over #4826. I'll close #4826 if that's the case. Please let me know. Thanks!

[szkiba]

@szkiba I believe this PR takes over #4826. I'll close #4826 if that's the case. Please let me know. Thanks!

@inancgumus Actually, xk6-dashboard v0.7.9 fixes the conflict mentioned in #4826 as a side effect. v0.7.9 has already been merged into master. So I think #4826 can be closed.

Honestly, I originally thought it wasn't worth fixing because the output name "web-dashboard" is unsupported and undocumented. The fact that the user found out that the built-in dashboard is an output name called "web-dashboard" doesn't justify fixing the conflict.

However, the fix is done, it might be worth merging (I thought the fix was simple and elegant).