Modernize e2e-tests

.prettierignore

@@ -4,7 +4,7 @@ dist-newstyle/
 waspc/data/packages/
 waspc/data/*/templates/
 waspc/data/Cli/starters/
-waspc/e2e-tests/test-outputs/
+waspc/e2e-tests/snapshots/
 waspc/.bin
 web/.docusaurus
 web/build/

waspc/.gitattributes

@@ -8,5 +8,5 @@
 # For .ps1 scripts on Win, it is important that their lines end with CRLF to execute correctly.
 *.ps1 text eol=crlf
 
-e2e-tests/test-outputs/*-golden/** linguist-generated=true
+e2e-tests/snapshots/*-golden/** linguist-generated=true
 waspls/tests/completionTests/*.golden linguist-generated=true

waspc/e2e-tests/FileSystem.hs

@@ -0,0 +1,39 @@
+module FileSystem
+  ( GitRepositoryRoot,
+    WaspcDir,
+    E2eTestsDir,
+    waspcDirInGitRoot,
+    getWaspcDirPath,
+    e2eTestsDirInWaspcDir,
+    getE2eTestsDir,
+  )
+where
+
+import Control.Monad (unless)
+import StrongPath (Abs, Dir, Path', Rel, reldir, (</>))
+import qualified StrongPath as SP
+import System.Directory (getCurrentDirectory)
+import System.FilePath (takeFileName)
+
+data GitRepositoryRoot
+
+data WaspcDir
+
+data E2eTestsDir
+
+waspcDirInGitRoot :: Path' (Rel GitRepositoryRoot) (Dir WaspcDir)
+waspcDirInGitRoot = [reldir|waspc|]
+
+getWaspcDirPath :: IO (Path' Abs (Dir WaspcDir))
+getWaspcDirPath = do
+  -- NOTE: Cabal launches `cabal test` from root of the project, so this should always be some absolute path to waspc.
+  absCwd <- getCurrentDirectory
+  -- Just a little extra safeguard here since we are doing destructive file ops.
+  unless (takeFileName absCwd == "waspc") (error "Expecting test process to be invoked from waspc dir")
+  SP.parseAbsDir absCwd
+
+e2eTestsDirInWaspcDir :: Path' (Rel WaspcDir) (Dir E2eTestsDir)
+e2eTestsDirInWaspcDir = [reldir|e2e-tests|]
+
+getE2eTestsDir :: IO (Path' Abs (Dir E2eTestsDir))
+getE2eTestsDir = (</> e2eTestsDirInWaspcDir) <$> getWaspcDirPath

waspc/e2e-tests/Main.hs

@@ -1,31 +1,27 @@
-import GoldenTest (runGoldenTest)
+import SnapshotTest (runSnapshotTest)
 import System.Info (os)
 import Test.Tasty (TestTree, defaultMain, testGroup)
-import Tests.WaspBuildTest (waspBuild)
-import Tests.WaspCompileTest (waspCompile)
-import Tests.WaspComplexTest (waspComplexTest)
-import Tests.WaspJobTest (waspJob)
-import Tests.WaspMigrateTest (waspMigrate)
-import Tests.WaspNewTest (waspNew)
+import Tests.KitchenSinkSnapshotTest (kitchenSinkSnapshotTest)
+import Tests.WaspBuildSnapshotTest (waspBuildSnapshotTest)
+import Tests.WaspCompileSnapshotTest (waspCompileSnapshotTest)
+import Tests.WaspMigrateSnapshotTest (waspMigrateSnapshotTest)
+import Tests.WaspNewSnapshotTest (waspNewSnapshotTest)
 
 main :: IO ()
 main = do
   if os == "mingw32"
     then putStrLn "Skipping end-to-end tests on Windows due to tests using *nix-only commands"
     else tests >>= defaultMain
 
--- TODO: Add more tests to simulate full Todo app tutorial.
---       Some of this requires waspStart and stdout parsing.
 -- TODO: Investigate automatically discovering the tests.
 tests :: IO TestTree
-tests = do
-  testGroup "All Golden Dir Tests"
+tests =
+  testGroup "Snapshot Tests"
     <$> mapM
-      runGoldenTest
-      [ waspNew,
-        waspCompile,
-        waspMigrate,
-        waspBuild,
-        waspJob,
-        waspComplexTest
+      runSnapshotTest
+      [ waspNewSnapshotTest,
+        waspCompileSnapshotTest,
+        waspBuildSnapshotTest,
+        waspMigrateSnapshotTest,
+        kitchenSinkSnapshotTest
       ]

waspc/e2e-tests/README.md

@@ -0,0 +1,41 @@
+# End-to-End tests for `waspc`
+
+The purpose of these `e2e-tests` is to verify the functionality of the **`waspc` binary**.
+We are not concerned with the internal implementation of the binary, only its interface and outputs.
+
+## More on purpose
+
+**Interface** is exposed through the Wasp CLI. We want to test the behavior of all of the Wasp CLI commands. These tests test behavior at the command level, rather than the internal implementation details.
+
+The main **outputs** of the `waspc` binary are Wasp applications. We want to validate that CLI commands correctly generate or modify applications in line with expectations.
+In addition to applications, we also cover secondary outputs, such as the installation and uninstallation of the CLI itself, `bash` completions, and more.
+
+## Snapshot tests
+
+We primarily test the `waspc` binary outputs using snapshot tests.
+Snapshot tests compare the current test outputs (`current` snapshot) against the expected test outputs (`golden` snapshot).
+
+Snapshots are compared in two ways:
+
+1. **By existence**: a file must exists in both the `current` and the `golden` snapshot.
+2. **By content**: the file contents must be identical in both the `current` and the `golden` snapshot.
+
+For more details, check out the `waspc/e2e-tests/SnapshotTest.hs` file.
+
+Snapshots are saved in the `waspc/e2e-tests/snapshots/` directory.
+While the exact files within a snapshot aren’t strictly defined, they usually have the following structure:
+
+```yaml
+# Where:
+#   `<name>`          = the snapshot test's name.
+#   `<snapshot-type>` = `current` or `golden`.
+
+e2e-tests/
+└── snapshots/
+    └── <name>-<snapshot-type>/  # snapshot dirctory, e.g. `wasp-build-current`, `wasp-build-golden`
+        ├── wasp-app/
+        └── snapshot-file-list.manifest
+```
+
+`wasp-app` contains the Wasp app for that snapshot.
+`snapshot-file-list.manifest` lists the files that should exist in the snapshot directory.