Skip to content

feat: enhance waveform rendering and add auto-centering #7798

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

cloudmark
Copy link
Contributor

PR Description

Reason for change

This PR addresses two significant user experience issues in the AudioUltra component, particularly impacting workflows with synchronized media like TimeSeries or GPS visualizations:

  1. Visual Tearing in Waveform: The waveform renderer previously used a partial rendering optimization that caused visual "tearing" or artifacts during scrolling and seeking. This created a jarring user experience and made it difficult to analyze audio data accurately.
  2. Lack of Centering on External Seeks: When an external component (e.g., a TimeSeries chart) triggered a seek event, the AudioUltra waveform would update its playhead position but would not center the view on it. This forced users to manually scroll to find the new position, creating confusion and disrupting the analysis workflow.

The solution implements two key enhancements:

  • Always Full Render: The partial rendering optimization has been removed from WaveformRenderer.ts. The component now always performs a full render of the visible waveform area. This completely eliminates visual tearing and ensures a smooth, consistent display.
  • Smart View Centering: A new centerToTime() method has been introduced. When a seek event originates from an external, synchronized component, the waveform and spectrogram views now automatically center on the new time position, with the playhead positioned precisely in the middle. This behavior is isolated to external sync events to avoid interfering with local user interactions.

Screenshots & Demo

Here are "before" and "after" videos demonstrating the improvements.

Before:
The video shows the lack of centering on external seeks and visual tearing during scrolling.
https://www.loom.com/share/377a287e98ef42239feea0631bdc7ce8

After:
This video showcases the new smart centering feature and the smooth, tear-free waveform rendering.
https://www.loom.com/share/a504416edb144f7b9e336a36f7fe091b

Rollout strategy

This is a direct enhancement to the existing AudioUltra component. The changes are UX improvements and bug fixes that can be rolled out directly without a feature flag or environment variable. The autoCenter parameter, which controls this behavior, is already part of the component's configuration.

Testing

The changes have been verified through the following steps:

  1. Set up a project with both an AudioUltra and a TimeSeries component synchronized on the same media.
  2. Zoomed into the AudioUltra waveform (zoom > 1).
  3. Clicked on various points in the TimeSeries chart to trigger seek events.
  4. Verification: Confirmed that the AudioUltra view correctly and reliably centers on the new time position with the playhead in the middle.
  5. Scrolled horizontally and performed various seek operations directly within the AudioUltra component.
  6. Verification: Confirmed that no visual tearing artifacts appear and the waveform renders smoothly.
  7. Confirmed that local seeks (e.g., clicking on the waveform) do not trigger auto-centering, as intended.

Risks

  • Performance: The primary risk was a potential performance degradation from removing the partial rendering optimization. However, manual testing showed no noticeable performance drop. The full render is still highly efficient, and the significant improvement in visual quality justifies the change.

Reviewer notes

  • The core logic for the tearing fix is in web/libs/editor/src/lib/AudioUltra/Visual/Renderer/WaveformRenderer.ts, where the logic for renderPartialWave has been removed in favor of always calling renderWave.
  • The centering feature is implemented across three key files:
    • web/libs/editor/src/tags/object/AudioUltra/model.js: The handleSyncSeek action now calls centerToTime(time) within a requestAnimationFrame to ensure correct timing.
    • web/libs/editor/src/lib/AudioUltra/Waveform.ts: A new centerToTime method was added, which orchestrates the centering and uses another requestAnimationFrame to prevent race conditions between scrolling and playhead updates.
    • web/libs/editor/src/lib/AudioUltra/Visual/Visualizer.ts: A corresponding centerToTime method was added to handle the scroll position calculation.
  • The nested requestAnimationFrame calls are intentional and crucial for ensuring that UI updates (scrolling, seeking) are fully processed before dependent actions (playhead positioning, drawing) are executed.

General notes

This change significantly improves the usability of Label Studio for any project involving synchronized media. It makes navigating complex, multi-modal data far more intuitive and less frustrating for the user.

…flows

- Remove partial rendering optimization to eliminate visual tearing artifacts
- Implement always-full-render approach for consistent waveform quality
- Add centerToTime() method to Visualizer and Waveform classes
- Auto-center view on external/sync seeks while preserving local interaction behavior
- Use requestAnimationFrame for proper async timing in centering operations
- Support both waveform and spectrogram view centering
- Improve UX for TimeSeries, GPS, and multi-component synchronized workflows

Fixes visual discontinuities during scrolling and eliminates user confusion
when seeking from external components by automatically centering the playhead
in the visible area.

Related to issue: HumanSignal#7797
@cloudmark cloudmark requested a review from a team as a code owner June 20, 2025 10:44
Copy link

netlify bot commented Jun 20, 2025

👷 Deploy request for heartex-docs pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit 09645a0

Copy link

netlify bot commented Jun 20, 2025

👷 Deploy request for label-studio-docs-new-theme pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit 09645a0

Copy link

netlify bot commented Jun 20, 2025

Deploy Preview for label-studio-storybook ready!

Name Link
🔨 Latest commit 09645a0
🔍 Latest deploy log https://app.netlify.com/projects/label-studio-storybook/deploys/68553b8e2b6c7d000850a822
😎 Deploy Preview https://deploy-preview-7798--label-studio-storybook.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Copy link

netlify bot commented Jun 20, 2025

Deploy Preview for label-studio-playground ready!

Name Link
🔨 Latest commit 09645a0
🔍 Latest deploy log https://app.netlify.com/projects/label-studio-playground/deploys/68553b8e4b7edb0008feeec9
😎 Deploy Preview https://deploy-preview-7798--label-studio-playground.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant