Skip to content

Feature: Limit billable bytes per query; Add estimated cost display #245

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 6 commits into from

Conversation

abannachGrafana
Copy link

Problem

As an admin, I may want to control the cost of my query spend. With the BigQuery client this can be done by specifying the MaxBytesBilled on the QueryConfig. Also, I know what our on demand compute price is and would like to estimate the cost of running a query at validation time so that my users can conceptualize what their queries cost and consider optimizing them.

Description

This PR adds two optional configuration fields to the datasource:

  • maxBillableBytes : Applied to the queryconfig on the backend to prevent large queries from running
  • onDemandComputePrice: Strictly used for display purposes on the frontend gives the user an approximate cost of running their query to 4 decimal places. Since the price is per terabyte 4 decimal places gives better visibility to smaller queries.

image

image
image

image

@abannachGrafana abannachGrafana requested a review from a team as a code owner March 8, 2024 02:59
@abannachGrafana abannachGrafana requested review from gabor, yesoreyeram and zoltanbedi and removed request for a team March 8, 2024 02:59
@abannachGrafana abannachGrafana enabled auto-merge (squash) March 26, 2024 03:45
@duncan771
Copy link

This looks great @abannachGrafana! Would love to have this so we can continue cutting unnecessary BigQuery costs through Grafana BI.

Have y'all had a chance to look at this? @zoltanbedi, @yesoreyeram, @gabor

@gabor gabor self-assigned this Apr 30, 2024
@gabor
Copy link

gabor commented May 2, 2024

@abannachGrafana as we discussed, when you'll have time, please create a separate PR for the maxBillableBytes part, we can handle that first, and then we can discuss the onDemandComputePrice part. thanks!

@duncan771
Copy link

@abannachGrafana as we discussed, when you'll have time, please create a separate PR for the maxBillableBytes part, we can handle that first, and then we can discuss the onDemandComputePrice part. thanks!

@gabor here is one of the new PRs that splits this. It should contain just the changes required for the maxBillableBytes feature to be added.

@ivanahuckova
Copy link

@abannachGrafana Thank you for your contribution. After consideration, we have decided not to proceed with the estimated cost display feature. We believe that the potential inaccuracies in calculation could provide users with a false sense of security.

The limit billable bytes per query feature has been merged. Thank you for contribution!

auto-merge was automatically disabled July 30, 2024 08:50

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Complete
Development

Successfully merging this pull request may close these issues.

5 participants