Example Project on Test Automation using Cucumber + Playwright + TypeScript + GitHub Actions

Description

This project covers web test automation, built on Node.js core + Cucumber-JS/Playwright frameworks + TypeScript language.

Prerequisites

• Install Node.js 22.10.0
• Install Cucumber-JS 11.1.1
• Install Playwright 1.49
• With the install command:

npm install

• Local Execution Adjustments

Test location

Tests are available in the test folder, in the project root

How to Run the Tests

Local Execution

Test execution is orchestrated by Cucumber-JS, meaning you use tags tied to specific test scenarios, which follow the pattern @ + purpose (e.g., @smoke-test). For this project, run the following commands:

• Run all tests (no tag is required):

  npm test

• Run a specific group of test scenarios (specify the desired tag):

  npm test --TAGS="@smoke-test"

• Run a specific group of test scenarios and ignore another group (specify both tags):

  npm test --TAGS="@smoke-test and not @skip"

• Run test scenarios tagged A or B (specify the tags):

  npm test --TAGS="@smoke-test or @functional-test"

• Run test scenarios tagged A and B (specify the tags):

  npm test --TAGS="@smoke-test and @functional-test"

Remote Execution by Tags (GitHub Actions)

Thanks to the workflow_dispatch feature, you can run the GitHub Actions workflow remotely by tags—choosing the environment and whether to generate the QA Report in ClickUp (yes or no) using input parameters. Follow these steps:

1. Go to the Actions tab of the repository on GitHub.
2. Select the Project Example – Pipeline(CI) workflow.
3. Click the Run workflow button.
4. Choose the desired environment (dev or test).
5. In the tags input field, enter the tags you want to run, for example: @smoke-test. If left blank, all test scenarios will be executed.
6. The qa_report field defaults to “no.” If you want to generate the QA Report, select “yes.”
7. Click Run workflow to start execution.
8. After this workflow completes, the “pages build and deployment” workflow is triggered automatically, producing a test report available at: Cucumber Report HTML.

This feature allows you to execute tests remotely without modifying the code locally—just configure the tags through the GitHub Actions interface.

Note: For the GitHub Actions CI workflow to run, you must set/include the following secrets in Settings → Secrets → Actions:

Fixed secrets:

• BASEURL_DEV
• BASEURL_TEST
• PASSWORD_DEV
• PASSWORD_TEST
• USER_NAME_DEV
• USER_NAME_TEST

📊 Test Report

Test reports are generated by Cucumber-JS in two ways:

📍 Local Execution

When you run the tests (as described above), two files are created in the test-result folder:

• cucumber-report.json – Contains the base structure and metadata of the report.
• cucumber-report.html – A detailed HTML report showing:
  • Passed and failed tests.
  • Execution success rate.
  • Date and duration of the last run.
  • Operating system used.
  • Versions of Node.js and Cucumber-JS.
  • A detailed list of features and scenarios executed.

To view the HTML report, just open the cucumber-report.html file in your browser. 🚀

📍 Remote Execution

The report generation for remote runs is under maintenance and will be integrated into the CI/CD pipeline. The goal is to make them available via GitHub Actions.

The generated report can be accessed at Cucumber Report HTML.
The JSON report can be accessed directly at Cucumber Report JSON.

🛠 How to Generate the HTML Report from JSON

If you want to generate the HTML report locally, follow these steps using the Cucumber HTML Formatter:

1. Install the tool (if you don’t have it yet):

   npm install --save-dev @cucumber/html-formatter

2. Download the report JSON:

   wget -O report.json https://leonardomelgarejo.github.io/test-automation-example/report.json

3. Generate the HTML report:

   npx @cucumber/html-formatter report.json > cucumber-report.html

4. Open the report in your browser:

   • On macOS:

     open cucumber-report.html

   • On Linux:

     xdg-open cucumber-report.html

   • On Windows (CMD or PowerShell):

     start cucumber-report.html

Pipeline

Currently, the CI pipeline includes the following steps:

• test: Run the automated tests.
• deploy-report: Deploy the test report to GitHub Pages.

📦 Main Dependencies

Package	Version	Function
@cucumber/cucumber	11.1.1	BDD framework for test automation
@playwright/test	1.49.1	Web and API test automation
dotenv	16.4.7	Environment variable management
ts-node	10.9.2	Allows running TypeScript without compiling to JavaScript

Local Execution Adjustments

Adjustments for the .env.test file:
Make sure to create the .env.test file in the helper/env directory with the following environment variables:

mkdir -p helper/env
echo "USER_NAME=User example" > helper/env/.env.test
echo "PASSWORD=Password example" >> helper/env/.env.test
echo "BASEURL=http://example.com" >> helper/env/.env.test
echo "ENV=test" >> helper/env/.env.test