Improve migration copy in CLI (#109)

* refactor: improve migration timestamp handling and copying logic

- Added helper functions to extract and generate incremental timestamps for migrations
- Updated migration copying to assign new timestamps while preserving order
- Modified existing migration detection to prevent duplicates based on filename content
- Enhanced logging to display styled filenames and migration status more clearly
- Removed redundant code and streamlined the migration installation process

* chore: add scripts for testing pgflow CLI install and duplicate prevention

Update project.json to include new test:e2e:install:duplicates target.
Add test-install and test-install-duplicates scripts for automated testing of
installation process and migration duplication prevention, including optional
timestamp-based duplicate detection.
These scripts facilitate end-to-end testing
of the CLI's install functionality and ensure no duplicate migrations are created.

* fix: improve migration copy logic with detailed status messages

Enhance user feedback when copying migrations by displaying detailed information
about already installed migrations, including their installation method and timestamps.

* fix: improve migration copy logic in CLI for better uniqueness and compatibility

Enhance migration copying by always generating a new timestamp-based ID, prepending it to the
original filename, and improving source migration matching to ensure backward compatibility
with installed migrations.

* chore(docs): improve migration instructions and prevent duplication during pgflow setup

- Updated documentation to clarify how migration files are copied with timestamps
- Added notes on preventing duplicate migrations when rerunning the installer
- Enhanced explanation of migration timestamp prefixes and their purpose
- Minor formatting adjustments for clarity in the installation guide

* refactor: improve timestamp generation logic in copy-migrations script

- Simplify function parameter default assignment
- Enhance timestamp parsing robustness for migration file handling

* chore: update supabase init command to use --force flag in test scripts

- Added --force flag to supabase init commands in test-install and test-install-duplicates
scripts to ensure clean initialization
- Minor formatting adjustments for consistency in test scripts

* chore: update test dependency order and add test-install-duplicates script

Refactors test task dependencies to include compile step and updates test-install commands to
run both install and install-duplicates scripts.

* fix: validate timestamp format more strictly in copy-migrations script

Enhance the timestamp validation to ensure it matches a 14-digit numeric pattern, preventing
invalid timestamps from being processed.
This improves robustness when generating new timestamps based on filenames.

* fix: improve timestamp generation and migration copying logic

- Adjusted generateNewTimestamp to ensure new timestamps are always at least one greater than
the reference
- Corrected existing files reading to handle undefined or missing files safely
- Enhanced migration filename handling to accurately identify already installed migrations
- Updated progress and summary messages for clarity and consistency
- Minor formatting and code style improvements for better maintainability

* refactor: enhance timestamp parsing and generation functions for migration scripts

- Added helper functions to format Date objects into timestamp strings
- Implemented robust parsing of timestamp strings into Date objects with validation
- Updated generateNewTimestamp to produce incremented timestamps that are always forward-moving
- Improved handling of invalid or unparseable timestamps to default to current time
- Overall, these changes improve reliability and correctness of migration timestamp management

* fix: improve migration filename parsing and timestamp validation

Refactors getTimestampFromFilename to ensure correct length and format validation.
Enhances timestamp comparison logic to consider only validated timestamps when updating
latestTimestamp.

Author: jumski

.changeset/short-zebras-wish.md

@@ -0,0 +1,12 @@
+---
+'pgflow': patch
+---
+
+Improve migration copy functionality in CLI
+
+- always generate a new timestamp-id for the copied migration
+- prepend the timestamp to the original migration name
+- when comparing installed migrations to source migrations, search for source migration filename
+
+This change should be backwards compatible with already installed migrations,
+because the matching logic will just match the source filename in destination folder.

pkgs/cli/project.json

@@ -31,17 +31,22 @@
     },
     "test": {
       "executor": "nx:noop",
-      "dependsOn": ["test:e2e:install", "test:e2e:compile"]
+      "dependsOn": [
+        "test:e2e:install",
+        "test:e2e:install:duplicates",
+        "test:e2e:compile"
+      ],
+      "options": {
+        "parallel": false
+      }
     },
     "test:e2e:install": {
       "executor": "nx:run-commands",
       "dependsOn": ["build"],
       "options": {
         "commands": [
-          "rm -rf supabase/",
-          "npx -y supabase@latest init --with-vscode-settings --with-intellij-settings",
-          "node dist/index.js install --supabase-path supabase/ --yes",
-          "./scripts/assert-pgflow-installed"
+          "./scripts/test-install",
+          "./scripts/test-install-duplicates"
         ],
         "cwd": "{projectRoot}",
         "parallel": false

pkgs/cli/scripts/test-install

@@ -0,0 +1,35 @@
+#!/usr/bin/env bash
+set -e
+
+# Script to test pgflow CLI install functionality
+# This simulates the test:e2e:install target from project.json
+
+cd "$(dirname "$0")/.."
+ROOT_DIR=$(pwd)
+
+echo "🧪 Testing pgflow install functionality"
+
+# Clean up any existing supabase directory
+echo "🧹 Cleaning up old test directory"
+rm -rf supabase/
+
+# Initialize a fresh Supabase project
+echo "🏗️ Creating new Supabase project"
+npx -y supabase@latest init --force --with-vscode-settings --with-intellij-settings
+
+# Install pgflow with our CLI
+echo "📦 Installing pgflow with CLI"
+node dist/index.js install --supabase-path supabase/ --yes
+
+# Verify installation succeeded
+echo "✅ Verifying pgflow installation"
+"$ROOT_DIR/scripts/assert-pgflow-installed"
+
+# Show success message
+echo "✨ Installation test complete"
+
+# Optional: Test for duplicates by running install again
+if [ "$1" == "--test-duplicates" ]; then
+  echo "🔄 Testing duplicate installation prevention"
+  node dist/index.js install --supabase-path supabase/ --yes
+fi

pkgs/cli/scripts/test-install-duplicates

@@ -0,0 +1,66 @@
+#!/usr/bin/env bash
+set -e
+
+# Script to test pgflow CLI install duplicate prevention
+# This verifies that we don't create duplicate migrations
+
+cd "$(dirname "$0")/.."
+ROOT_DIR=$(pwd)
+
+echo "🧪 Testing pgflow migration duplicate prevention"
+
+# Clean up any existing supabase directory
+echo "🧹 Cleaning up old test directory"
+rm -rf supabase/
+
+# Initialize a fresh Supabase project
+echo "🏗️ Creating new Supabase project"
+npx -y supabase@latest init --force --with-vscode-settings --with-intellij-settings
+
+# First installation with pgflow CLI
+echo "📦 First pgflow installation"
+node dist/index.js install --supabase-path supabase/ --yes
+
+# Count number of migrations after first install
+FIRST_COUNT=$(find supabase/migrations -name "*.sql" | wc -l)
+echo "🔢 Found $FIRST_COUNT migrations after first install"
+
+# Second installation with pgflow CLI
+echo "🔄 Running second pgflow installation"
+node dist/index.js install --supabase-path supabase/ --yes
+
+# Count number of migrations after second install
+SECOND_COUNT=$(find supabase/migrations -name "*.sql" | wc -l)
+echo "🔢 Found $SECOND_COUNT migrations after second install"
+
+# Verify no duplicates were created
+if [ "$FIRST_COUNT" -eq "$SECOND_COUNT" ]; then
+  echo "✅ Success: No duplicate migrations were created"
+else
+  echo "❌ Error: Duplicate migrations detected ($SECOND_COUNT - $FIRST_COUNT = $((SECOND_COUNT - FIRST_COUNT)) new files)"
+  exit 1
+fi
+
+# Optional: Run a third time with different timestamps
+if [ "$1" == "--test-third-install" ]; then
+  # Modify a migration file timestamp to simulate a user renaming it
+  RANDOM_MIGRATION=$(find supabase/migrations -name "*.sql" | head -1)
+  NEW_NAME=$(echo "$RANDOM_MIGRATION" | sed 's/[0-9]\{14\}_/99999999999999_/')
+  
+  echo "🔄 Renaming $RANDOM_MIGRATION to $NEW_NAME"
+  mv "$RANDOM_MIGRATION" "$NEW_NAME"
+  
+  echo "🔄 Running third pgflow installation"
+  node dist/index.js install --supabase-path supabase/ --yes
+  
+  # Count number of migrations after third install
+  THIRD_COUNT=$(find supabase/migrations -name "*.sql" | wc -l)
+  echo "🔢 Found $THIRD_COUNT migrations after third install"
+  
+  if [ "$SECOND_COUNT" -eq "$THIRD_COUNT" ]; then
+    echo "✅ Success: No duplicate migrations were created (even with timestamp changes)"
+  else
+    echo "❌ Error: Duplicate migrations detected after timestamp changes"
+    exit 1
+  fi
+fi