Skip to content

chore: remove v2 models from schema #2107

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

chore: remove v2 models from schema #2107

wants to merge 12 commits into from

Conversation

ericallam
Copy link
Member

Removed v2 models from the schema in a safe way (migrations can be run manually on the database before actually applying the migrations during deploy). Takes the schema down from 22,268 tokens (93378 characters) to 14,317 tokens (59810 characters), a savings of 35.7%

Copy link

changeset-bot bot commented May 26, 2025

⚠️ No Changeset found

Latest commit: 5cb5865

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link
Contributor

coderabbitai bot commented May 26, 2025

Walkthrough

This set of changes implements a comprehensive removal of numerous database tables, enums, and models related to integrations, jobs, triggers, endpoints, webhooks, and associated entities from the schema and database. The migration scripts sequentially drop dozens of tables and enum types, ensuring idempotency and cascading deletions where necessary. The Prisma schema is updated to reflect these removals, eliminating many models and relations. In their place, a simplified secret management system is introduced, adding new models for secret references and storage. Several existing models are also updated to remove now-obsolete fields and relations. No public or exported entities outside the schema are affected.


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d6dd103 and 5cb5865.

📒 Files selected for processing (3)
  • apps/webapp/app/database-types.ts (0 hunks)
  • apps/webapp/app/services/platform.v3.server.ts (0 hunks)
  • apps/webapp/app/services/telemetry.server.ts (0 hunks)
💤 Files with no reviewable changes (3)
  • apps/webapp/app/services/telemetry.server.ts
  • apps/webapp/app/services/platform.v3.server.ts
  • apps/webapp/app/database-types.ts
⏰ Context from checks skipped due to timeout of 90000ms (25)
  • GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (10, 10)
  • GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (9, 10)
  • GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (8, 10)
  • GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (7, 10)
  • GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (6, 10)
  • GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (5, 10)
  • GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (4, 10)
  • GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (3, 10)
  • GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (2, 10)
  • GitHub Check: units / internal / 🧪 Unit Tests: Internal (8, 8)
  • GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 10)
  • GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
  • GitHub Check: units / internal / 🧪 Unit Tests: Internal (6, 8)
  • GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
  • GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
  • GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
  • GitHub Check: units / internal / 🧪 Unit Tests: Internal (3, 8)
  • GitHub Check: units / internal / 🧪 Unit Tests: Internal (2, 8)
  • GitHub Check: units / internal / 🧪 Unit Tests: Internal (1, 8)
  • GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
  • GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
  • GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
  • GitHub Check: typecheck / typecheck
  • GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
  • GitHub Check: Analyze (javascript-typescript)
✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (8)
internal-packages/database/prisma/migrations/20250526095826_remove_v2_round_7/migration.sql (1)

1-7: Consider wrapping destructive DDL in a transaction
Explicitly enclosing these DROP TABLE statements in BEGIN; … COMMIT; can help ensure atomicity and rollback on error. Verify whether Prisma’s migration runner already provides this guarantee or add it here.

internal-packages/database/prisma/migrations/20250526094442_remove_v2_round_3/migration.sql (1)

1-7: Consider wrapping destructive DDL in a transaction
Grouping these drops under BEGIN; … COMMIT; can improve safety by rolling back on failure. Confirm whether your migration runner already handles transactions.

internal-packages/database/prisma/migrations/20250526095444_remove_v2_round_6/migration.sql (1)

1-10: Batch DDL operations: ensure atomicity
Dropping five tables in one go increases risk of a partial state on error. Wrap these operations in a transaction or split them into smaller chunks to guarantee rollback.

internal-packages/database/prisma/migrations/20250526100059_remove_v2_round_8/migration.sql (1)

1-9: Consider wrapping destructive DDL in a transaction
A transactional wrapper (BEGIN; … COMMIT;) will help you avoid partial application if one drop fails. Validate your migration runner’s behavior.

internal-packages/database/prisma/migrations/20250526092407_remove_v2_tables_round_1/migration.sql (1)

1-25: Enhance the data‐loss warning with recovery instructions
The block of warnings is useful, but consider adding brief guidance on backing up data or pointing to a runbook/rollback procedure for safer execution.

internal-packages/database/prisma/schema.prisma (3)

330-333: Document SecretStoreProvider enum values
Consider adding comments above each provider (e.g., DATABASE, AWS_PARAM_STORE) to clarify their intended usage and any configuration requirements.


335-343: Align naming conventions or add mapping directive
The SecretStore model uses a camelCase name but defines a BTree index with raw ops. If your DB tables use snake_case, you might add an @@map("secret_store") or rename fields for consistency. This helps maintain uniform naming across Prisma and Postgres.


346-353: Extend DataMigration for richer tracking
The DataMigration model has only name and timestamps. You may want to add a status enum (e.g. PENDING, COMPLETED, FAILED) or a details Json? field for debugging and audit purposes.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e84ede9 and d6dd103.

📒 Files selected for processing (12)
  • internal-packages/database/prisma/migrations/20250526092407_remove_v2_tables_round_1/migration.sql (1 hunks)
  • internal-packages/database/prisma/migrations/20250526093738_remove_v2_round_2/migration.sql (1 hunks)
  • internal-packages/database/prisma/migrations/20250526094044_remove_v2_trigger_http_endpoint_environment/migration.sql (1 hunks)
  • internal-packages/database/prisma/migrations/20250526094442_remove_v2_round_3/migration.sql (1 hunks)
  • internal-packages/database/prisma/migrations/20250526094729_remove_v2_round_4/migration.sql (1 hunks)
  • internal-packages/database/prisma/migrations/20250526095040_remove_v2_round_5/migration.sql (1 hunks)
  • internal-packages/database/prisma/migrations/20250526095444_remove_v2_round_6/migration.sql (1 hunks)
  • internal-packages/database/prisma/migrations/20250526095826_remove_v2_round_7/migration.sql (1 hunks)
  • internal-packages/database/prisma/migrations/20250526100059_remove_v2_round_8/migration.sql (1 hunks)
  • internal-packages/database/prisma/migrations/20250526100402_remove_v2_round_9/migration.sql (1 hunks)
  • internal-packages/database/prisma/migrations/20250526100607_remove_v2_round_10/migration.sql (1 hunks)
  • internal-packages/database/prisma/schema.prisma (1 hunks)
🔇 Additional comments (21)
internal-packages/database/prisma/migrations/20250526093738_remove_v2_round_2/migration.sql (1)

1-17: Approve removal of obsolete tables
This migration safely drops JobRunExecution, ScheduleSource, and WebhookEnvironment with IF EXISTS and CASCADE, ensuring idempotency and automatic cleanup of dependent objects.

internal-packages/database/prisma/migrations/20250526095040_remove_v2_round_5/migration.sql (1)

1-9: Approve dropping of IntegrationConnection
Using DROP TABLE IF EXISTS ... CASCADE makes this migration safe to run repeatedly and automatically removes all dependencies.

internal-packages/database/prisma/migrations/20250526094044_remove_v2_trigger_http_endpoint_environment/migration.sql (1)

1-13: Approve alteration and removal of trigger HTTP environment artifacts
The script cleanly drops the httpEndpointEnvironmentId column and the TriggerHttpEndpointEnvironment table with IF EXISTS and CASCADE, preserving idempotency and cascading constraints.

internal-packages/database/prisma/migrations/20250526094729_remove_v2_round_4/migration.sql (1)

1-14: Approve removal of RunConnection and TriggerSource tables
This straightforward migration uses IF EXISTS and CASCADE for safe, idempotent deletion of the two legacy tables and their dependencies.

internal-packages/database/prisma/migrations/20250526100402_remove_v2_round_9/migration.sql (1)

1-21: Approve removal of legacy integration tables
Dropping ExternalAccount, Integration, IntegrationAuthMethod, and IntegrationDefinition with IF EXISTS ... CASCADE ensures the migration can be applied safely without leftover references.

internal-packages/database/prisma/migrations/20250526095826_remove_v2_round_7/migration.sql (2)

9-9: Verify absence of dependencies on JobVersion
Although CASCADE removes dependent objects, ensure there are no critical views, functions, or application code still referencing JobVersion to avoid unintended side effects.


12-12: Verify absence of dependencies on TriggerHttpEndpoint
Similarly, confirm that no database objects (e.g., triggers, functions) or application modules reference this table before dropping it.

internal-packages/database/prisma/migrations/20250526100607_remove_v2_round_10/migration.sql (1)

1-39: Ensure enum types aren’t still in use before dropping
If any table columns still reference these enums, the migration will fail. Consider running a check like:

SELECT n.nspname AS schema, t.typname AS type, a.attrelid::regclass AS table, a.attname AS column
FROM pg_type t
JOIN pg_attribute a ON a.atttypid = t.oid
JOIN pg_namespace n ON n.oid = t.typnamespace
WHERE t.typname IN (
  'ConnectionType','DynamicTriggerType','IntegrationAuthSource',
  'IntegrationSetupStatus','JobRunExecutionReason','JobRunExecutionStatus',
  'JobRunStatus','JobStartPosition','JobVersionStatus','PayloadType',
  'TaskChildExecutionMode','TaskStatus','TriggerChannel'
);

and migrate or drop those columns first.

internal-packages/database/prisma/migrations/20250526094442_remove_v2_round_3/migration.sql (2)

9-9: Confirm no residual dependencies on Task
Ensure no foreign keys, views, or functions reference the Task table before dropping it; otherwise, migration may cascade unexpectedly.


12-12: Confirm no residual dependencies on Webhook
Similarly, verify that the Webhook table is fully deprecated across code and schema to prevent surprises.

internal-packages/database/prisma/migrations/20250526095444_remove_v2_round_6/migration.sql (5)

12-12: Verify no dependencies on DynamicTrigger
Check for lingering references (FKs, triggers, views) to avoid unexpected cascade deletions.


15-15: Verify no dependencies on EventDispatcher
Confirm that no database objects or code paths still rely on this table before dropping.


18-18: Verify no dependencies on EventRecord
Ensure deprecation across the codebase to prevent runtime errors post-migration.


21-21: Verify no dependencies on JobRun
Confirm safe removal—look for foreign keys and application logic references.


24-24: Verify no dependencies on _DynamicTriggerToJob
Double-check that this join table is no longer in use before dropping.

internal-packages/database/prisma/migrations/20250526100059_remove_v2_round_8/migration.sql (4)

11-11: Verify no dependencies on ConcurrencyLimitGroup
Ensure the table is fully deprecated in both schema and code to avoid unintended data loss.


14-14: Verify no dependencies on Endpoint
Confirm that no views, functions, or application modules still use this table.


17-17: Verify no dependencies on Job
Double-check that all references to Job have been removed before dropping the table.


20-20: Verify no dependencies on JobQueue
Ensure this table is no longer in use in the code or other migrations.

internal-packages/database/prisma/migrations/20250526092407_remove_v2_tables_round_1/migration.sql (2)

27-85:

❓ Verification inconclusive

Verify foreign key dependencies and cascade settings
Currently only MissingConnection is dropped with CASCADE. Confirm that none of the other tables have remaining foreign‐key constraints—otherwise these DROP TABLE statements may fail. You may want to either explicitly drop dependent constraints first or apply CASCADE where required.


🏁 Script executed:

#!/bin/bash
# Verify no references to dropped tables in Prisma schema
PRISMA_SCHEMA="internal-packages/database/prisma/schema.prisma"
for tbl in JobCounter WebhookDeliveryCounter ApiIntegrationVote ConnectionAttempt DeferredScheduledEventService DynamicTriggerRegistration EndpointIndex EventExample HttpSourceRequestDelivery JobAlias JobIntegration JobRunAutoYieldExecution JobRunStatusRecord JobRunSubscription KeyValueItem TaskAttempt TriggerSourceOption WebhookRequestDelivery _JobRunToMissingConnection; do
  if rg -q "model ${tbl} " "$PRISMA_SCHEMA"; then
    echo "⚠️ Found lingering model $tbl in schema.prisma"
  fi
done

Length of output: 8805


🏁 Script executed:

#!/bin/bash
set -e

MIGRATION_FILE="internal-packages/database/prisma/migrations/20250526092407_remove_v2_tables_round_1/migration.sql"

echo "Checking for foreign key references to dropped tables"
for tbl in JobCounter WebhookDeliveryCounter ApiIntegrationVote ConnectionAttempt DeferredScheduledEventService DynamicTriggerRegistration EndpointIndex EventExample HttpSourceRequestDelivery JobAlias JobIntegration JobRunAutoYieldExecution JobRunStatusRecord JobRunSubscription KeyValueItem TaskAttempt TriggerSourceOption WebhookRequestDelivery _JobRunToMissingConnection; do
  if rg -n "REFERENCES \"$tbl\"" "$MIGRATION_FILE"; then
    echo "⚠️ Found foreign key references to $tbl"
  fi
done

echo "Checking for explicit DROP CONSTRAINT statements"
rg -n "DROP CONSTRAINT" "$MIGRATION_FILE"

Length of output: 10298


No foreign‐key references or DROP CONSTRAINTs found in migration
We searched the SQL in internal-packages/database/prisma/migrations/20250526092407_remove_v2_tables_round_1/migration.sql and found:

  • No REFERENCES "<TableName>" clauses for any of the tables being dropped
  • No explicit DROP CONSTRAINT statements

It appears these tables have no inter-table foreign keys, so each DROP TABLE IF EXISTS should succeed as is. However, please manually confirm against your current database schema (or earlier migrations) that no remaining constraints reference these tables before deploying.


87-103:

✅ Verification successful

Ensure enum types are no longer referenced anywhere
After dropping these enum types, verify that no columns in other tables still use them.


🏁 Script executed:

#!/bin/bash
# Check for leftover enum references in Prisma schema
PRISMA_SCHEMA="internal-packages/database/prisma/schema.prisma"
for enum in EndpointIndexSource EndpointIndexStatus JobRunSubscriptionEvents JobRunSubscriptionRecipientMethod JobRunSubscriptionStatus TaskAttemptStatus; do
  if rg -q "${enum}" "$PRISMA_SCHEMA"; then
    echo "⚠️ Found lingering enum $enum in schema.prisma"
  fi
done

Length of output: 1477


🏁 Script executed:

#!/bin/bash
set -eo pipefail

PRISMA_DIR="internal-packages/database/prisma"

for enum in EndpointIndexSource \
            EndpointIndexStatus \
            JobRunSubscriptionEvents \
            JobRunSubscriptionRecipientMethod \
            JobRunSubscriptionStatus \
            TaskAttemptStatus; do
  echo "Searching for references to $enum in $PRISMA_DIR..."
  if ! rg -n "$enum" "$PRISMA_DIR"; then
    echo "  ✅ No references found for $enum"
  fi
done

Length of output: 7041


✅ Prisma schema clean of removed enums
Ran a targeted search in internal-packages/database/prisma/schema.prisma for all six dropped enum names; there are no lingering references. The only matches reside in historical migration files, which is expected. You can safely drop these enum types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant