Skip to content

Dataframe parsing bug fixes #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 4, 2025
Merged

Conversation

jealouscloud
Copy link
Contributor

This change set fixes several bugs in the Grafana dashboard plugin.

Bugs encountered

  • Missing parameters in requests
  • Columns forced as ints
  • Timestamp when not requested or relevant
  • Timestamp not translated properly / broken outside of Explore view
 SELECT distinct(agent_hostname) as agent_hostname  FROM "apache_log" 

^ Attempting such a query would return nothing or NaN

This query would also fail to render in any view outside of Explore

select 1 from "log_stream"

Fixes

getField

  • Fix bug which skips the first column regardless of if it is a timestamp
  • date(timestamp + "Z") was breaking timestamps. Use whatever Open Observe gives us, unless it's microseconds which needs converting to MS.
  • Only add Time column if it's part of the request [breaking for queries relying on this implicit field, but deduplicating for the queries that manually request it including example requests]

getGraphDataFrame

  • [bugfix] No longer skip first requested column
  • [bugfix] Remove type: number for non-time fields which breaks string responses
  • Detect timestamp column in request
  • Utilize datasource defined timestamp column automatically in data frame parser

getColumnsFromQuery

  • [bug] The dataframe parser would fail to assign fields wrapped in quotes i.e. select 1 as "server". Strip quotes.

Additionally, fixes for #14 are included so tests pass but if #15 is merged ahead of this I can revert 8561dab327348cb1dffc5b366a1a9d511b827069

* Fix bug which skips the first column regardless of if it is a timestamp
* date(timestamp + "Z") was breaking timestamps. Use whatever O2 gives us, unless it's microseconds which needs converting to MS.
* Only add Time column if it's part of the request [breaking for queries relying on this implicit field, but deduplicating for the queries that manually request it including example requests]
* [bugfix] No longer skip first requested column
* [bugfix] Remove type: number for non-time fields which breaks string responses
* Detect timestamp column in request
[big] The dataframe parser would fail to assign fields wrapped in quotes i.e. select 1 as "server"
@prabhatsharma prabhatsharma requested a review from omkarK06 January 8, 2025 22:05
@gabolera
Copy link

gabolera commented Mar 3, 2025

Thanks, this solution fixes the plugin development mode opening

@hengfeiyang hengfeiyang merged commit fd12b57 into openobserve:main Mar 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants