docs: Add comprehensive Switch transpiler documentation

[hiroyukinakazato-db]

Changes

This PR adds comprehensive documentation for Switch, an LLM-powered transpiler integrated into Lakebridge. Switch converts SQL and other source formats into Databricks notebooks using Large Language Models.

What does this PR do?

Adds complete documentation for the Switch transpiler, covering installation, usage, configuration, architecture, and troubleshooting. Switch enables LLM-based code conversion through the llm-transpile command, processing local files and uploading them to Unity Catalog Volumes automatically.

Relevant implementation details

New Documentation:

• docs/lakebridge/docs/transpile/pluggable_transpilers/switch.mdx - Comprehensive Switch guide including:
  • Installation with --include-llm-transpiler true flag
  • CLI usage via llm-transpile command
  • Command-line configuration parameters
  • Processing architecture and workflows
  • Custom prompt creation
  • Troubleshooting guide

Updated Existing Documentation:

• docs/lakebridge/docs/overview.mdx - Added Switch to transpiler list
• docs/lakebridge/docs/transpile/overview.mdx - Added Switch section with key characteristics
• docs/lakebridge/docs/transpile/pluggable_transpilers/index.mdx - Updated to mention Switch installation

Caveats/things to watch out for when reviewing:

• This is a draft: The llm-transpile command usage documentation in switch.mdx may need updates as the integration implementation evolves
• Current assumptions in the new Switch documentation:
  • Switch is installed as an optional direct dependency
  • The llm-transpile command takes local input paths and automatically uploads to Volumes
  • Catalog and schema are configured during installation (not at runtime)
  • All configuration is provided via command-line arguments (e.g., --endpoint-name) which are converted to job parameters (e.g., endpoint_name)

Linked issues

Resolves #2044

Functionality

• added relevant user documentation
• added new CLI command
• modified existing command: databricks labs lakebridge ...

Tests

• manually tested
• added unit tests
• added integration tests

[github-actions]

✅ 46/46 passed, 7 flaky, 3m23s total

Flaky tests:

• 🤪 test_validate_non_empty_tables (26ms)
• 🤪 test_transpiles_informatica_to_sparksql (15.459s)
• 🤪 test_transpile_teradata_sql_non_interactive[False] (16.642s)
• 🤪 test_transpile_teradata_sql (20.052s)
• 🤪 test_transpile_teradata_sql_non_interactive[True] (4.878s)
• 🤪 test_transpiles_informatica_to_sparksql_non_interactive[True] (8.619s)
• 🤪 test_transpiles_informatica_to_sparksql_non_interactive[False] (8.242s)

_{Running from acceptance #2795}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@2bf8292). Learn more about missing BASE report.
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2102   +/-   ##
=======================================
  Coverage        ?   64.87%           
=======================================
  Files           ?       96           
  Lines           ?     7895           
  Branches        ?      821           
=======================================
  Hits            ?     5122           
  Misses          ?     2593           
  Partials        ?      180           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.