Skip to content
Open
Changes from 4 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
c83fa1b
WIP railway deploy test
infomiho Sep 15, 2025
22dd692
Update Railway cleanup cache. Naming.
infomiho Sep 15, 2025
ad40c39
Testing Railway token login
infomiho Sep 15, 2025
b8545ee
Pass RAILWAY_API_TOKEN expliclty to wasp-cli
infomiho Sep 15, 2025
8a4dd36
Login with Railway CLI before calling Wasp CLI
infomiho Sep 15, 2025
249b2fb
Add workspace ID
infomiho Sep 15, 2025
7da3c83
Use run_id instead of SHA
infomiho Sep 15, 2025
800a361
Test with Workspace name
infomiho Sep 15, 2025
db6c079
Revert installing production Wasp CLI
infomiho Sep 15, 2025
80c3596
Add Railway app cleanup
infomiho Sep 15, 2025
4b1f099
Update jq and concurency
infomiho Sep 15, 2025
5a8343c
Rewrite to use matrix
infomiho Sep 16, 2025
b21a236
Make Railway cleanup more robust
infomiho Sep 16, 2025
648848a
Add retries for smoke tests
infomiho Sep 16, 2025
d2bb57a
Format
infomiho Sep 16, 2025
389ebce
Merge branch 'main' into miho-railway-deployment-test-ci
infomiho Sep 16, 2025
9f0aba4
Show output of the smoke tests
infomiho Sep 16, 2025
b1532c8
DRY getting hostnames
infomiho Sep 16, 2025
5d2df2d
Cleanup
infomiho Sep 17, 2025
b2d5c5d
Ignore `yes` errors
infomiho Sep 17, 2025
a18dee1
Revert using bash shell
infomiho Sep 17, 2025
9fb7499
Testing job split
infomiho Sep 18, 2025
3454f9d
Dumb down env var processing
infomiho Sep 18, 2025
7cdef81
Try a different env vars approach
infomiho Sep 18, 2025
af5ac9e
Use correct Railway GraphQL query to get the project ID
infomiho Sep 18, 2025
cd71c8d
Use workspace ID instead of name
infomiho Sep 18, 2025
53c40a7
Merge branch 'main' into miho-railway-deployment-test-ci
infomiho Sep 18, 2025
d88cd0f
Bump the wait time for smoke test
infomiho Sep 18, 2025
e8e9931
Update .github/workflows/test-deploy.yaml
infomiho Oct 6, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 107 additions & 3 deletions .github/workflows/test-deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ on:
- ".github/**"
branches:
- main
# TODO: remove before merging this PR
- miho-railway-deployment-test-ci
tags:
- v*
workflow_dispatch:
Expand All @@ -15,11 +17,12 @@ env:
APP_PREFIX: ci-${{ github.sha }}
FLY_API_TOKEN: ${{ secrets.FLY_GITHUB_TESTING_TOKEN }}
FLY_REGION: mia
RAILWAY_API_TOKEN: ${{ secrets.RAILWAY_GITHUB_TESTING_TOKEN }}
APP_TO_DEPLOY: waspc/examples/todoApp

jobs:
fly_deploy_app:
name: Deploy Wasp app
name: Deploy Wasp app to Fly.io

runs-on: ubuntu-latest

Expand Down Expand Up @@ -79,7 +82,7 @@ jobs:
server_hostname: ${{ steps.save_hostnames.outputs.server_hostname }}
client_hostname: ${{ steps.save_hostnames.outputs.client_hostname }}

smoke_test_app:
fly_smoke_test_app:
name: Run smoke tests on deployed app

runs-on: ubuntu-latest
Expand All @@ -100,7 +103,7 @@ jobs:
fly_destroy_app:
runs-on: ubuntu-latest
name: Clean up deployed Fly app
needs: [fly_deploy_app, smoke_test_app]
needs: [fly_deploy_app, fly_smoke_test_app]
# NOTE: Fly deployments can sometimes "fail" but still deploy the apps. We
# want to always clean them up.
if: always()
Expand All @@ -119,3 +122,104 @@ jobs:
flyctl apps destroy -y $APP_PREFIX-server || true
flyctl apps destroy -y $APP_PREFIX-client || true
flyctl apps destroy -y $APP_PREFIX-db || true

railway_deploy_app:
name: Deploy Wasp app to Railway

runs-on: ubuntu-latest

environment:
name: railway-deploy-test

steps:
- uses: actions/checkout@v4

- uses: ./.github/actions/setup-haskell

- uses: actions/setup-node@v4
with:
cache: "npm"
node-version: "22"

- name: Install Wasp CLI
working-directory: waspc
run: ./run install

- name: Install Railway CLI
run: |
npm install -g @railway/cli

- name: Deploy App to Railway
working-directory: ${{ env.APP_TO_DEPLOY }}
run: |
set -e

# NOTE: This assumes env var values don't contain:
# - The character `#`.
# - An empty line (i.e., there are no multiline env values).
mapfile -t ENV_VAR_ARGUMENTS < <(
cat .env.server.example |
sed -E 's/#.*//' |
awk 'NF {$1=$1;print}' |
sed -E 's/^/--server-secret\n/'
)

echo "Deploying with prefix: $APP_PREFIX"
# NOTE:
# - We use a Bash array for `$ENV_VAR_ARGUMENTS` to ensure proper
# word splitting (i.e., force Bash to interpret the flags separately
# instead of passing it as a single string value).
echo $RAILWAY_API_TOKEN
RAILWAY_API_TOKEN=$RAILWAY_API_TOKEN wasp-cli deploy railway launch "$APP_PREFIX" --workspace wasp "${ENV_VAR_ARGUMENTS[@]}"

- name: Save deployed app hostnames
id: save_hostnames
working-directory: ${{ env.APP_TO_DEPLOY }}
run: |
server_service_name="${APP_PREFIX}-server"
client_service_name="${APP_PREFIX}-client"

echo "server_hostname=$(railway status --json | jq -r --arg NAME "$server_service_name" '.services.edges[] | select(.node.name == $NAME) | .node.serviceInstances.edges[0].node.domains.serviceDomains[0].domain')" >> "$GITHUB_OUTPUT"
echo "client_hostname=$(railway status --json | jq -r --arg NAME "$client_service_name" '.services.edges[] | select(.node.name == $NAME) | .node.serviceInstances.edges[0].node.domains.serviceDomains[0].domain')" >> "$GITHUB_OUTPUT"
outputs:
server_hostname: ${{ steps.save_hostnames.outputs.server_hostname }}
client_hostname: ${{ steps.save_hostnames.outputs.client_hostname }}

railway_smoke_test_app:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know it's still a draft, just FYI.

My idea here was to use the same smoke test job with a matrix for all providers (only Railway and Fly at the time). It's the same test, the only thing that changes are the inputs.

I'm not 100% sure that's possible but just a heads up to try it out.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep yep, I'll optimize once I get it to work 😄

We still have the blocker of not being able to clean up Railway apps after they are created using the CLI.

name: Run smoke tests on deployed app

runs-on: ubuntu-latest
needs: railway_deploy_app
env:
SERVER_HOSTNAME: ${{ needs.railway_deploy_app.outputs.server_hostname }}
CLIENT_HOSTNAME: ${{ needs.railway_deploy_app.outputs.client_hostname }}

steps:
- name: Smoke test the server
run: |
curl --fail --silent -X POST https://$SERVER_HOSTNAME/operations/get-date | jq '.json'

- name: Smoke test the client
run: |
curl --fail --silent https://$CLIENT_HOSTNAME | grep 'ToDo App'

railway_destroy_app:
runs-on: ubuntu-latest
name: Clean up deployed Railway app
needs: [railway_deploy_app, railway_smoke_test_app]
# NOTE: Fly deployments can sometimes "fail" but still deploy the apps. We
# want to always clean them up.
if: always()

steps:
- uses: actions/setup-node@v4
with:
node-version: "22"

- name: Install Railway CLI
run: |
npm install -g @railway/cli

- name: Clean up testing app from Railway
run: |
echo "Not implemented yet, manually delete the project in Railway dashboard"
Loading