Skip to content

Conversation

@goodwillpunning
Copy link
Contributor

Changes

This PR adds an installer and uninstaller for the data warehouse profiler.

What does this PR do?

This PR adds an installer/uninstaller for the 3 major operating systems (macOS, Windows, and Linux). This PR includes relevant documentation for installing/uninstalling the profiler.

Relevant implementation details

One of the goals of the profiler is to make it easy to install on a local machine without needing a Databricks workspace. Secondly, gathering enough usage data and metrics is critical in calculating an accurate Total Cost of Ownership (TCO). This PR leverages the built-in scheduling components of the host operating system to schedule usage collection so that enough usage information is used in providing a TCO estimate.

Caveats/things to watch out for when reviewing:

  • (Docs) Are the installation/uninstallation steps clear? Do they make sense? Are they easy to understand even for someone with little experience?
  • (Impl) Since this PR leverages the underlying OS to schedule usage collection, the installer must be run under administrative (or super user) privileges. It's assumed that a data warehouse admin will be running the profiler. Do you foresee any challenges/objections to this?

Linked issues

Resolves #1470.

Functionality

  • added relevant user documentation
  • added new CLI command
  • modified existing command: databricks labs remorph ...
  • added installer/uninstaller scripts for the profiler
  • added profiler installation/uninstallation docs

Tests

  • manually tested
  • added unit tests
  • added integration tests

@goodwillpunning goodwillpunning requested a review from a team as a code owner May 6, 2025 15:36
@goodwillpunning goodwillpunning added the feat/profiler Issues related to profilers label May 6, 2025
@github-actions
Copy link

github-actions bot commented May 6, 2025

❌ 15/16 passed, 1 failed, 1 skipped, 1m24s total

❌ test_gets_maven_version: AttributeError: 'NoneType' object has no attribute 'split' (1m0.269s)
AttributeError: 'NoneType' object has no attribute 'split'
[gw3] linux -- Python 3.10.17 /home/runner/work/remorph/remorph/.venv/bin/python
[gw3] linux -- Python 3.10.17 /home/runner/work/remorph/remorph/.venv/bin/python

Running from acceptance #705

@goodwillpunning goodwillpunning changed the title Add Profiler Installer/Uninstaller Add Profiler Installer/Uninstaller for Windows May 13, 2025
Patch for integration test failure
@sundarshankar89 sundarshankar89 removed the feat/profiler Issues related to profilers label Oct 6, 2025
@sundarshankar89 sundarshankar89 added profiler-scheduler feat/profiler Issues related to profilers labels Oct 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feat/profiler Issues related to profilers profiler-scheduler

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE]: Schedule job to run extraction

3 participants