Skip to content

chore: Add API Breakage Checking GitHub Action #3753

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 40 commits into from
Jul 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
05794ac
Add API Checkers
rozaychen Jun 14, 2024
8e7a11a
Add Test API Dumps Files
rozaychen Jun 14, 2024
ca82873
Merge branch 'main' of https://github.com/rozaychen/amplify-swift
rozaychen Jun 14, 2024
bacf67c
Update apiBreakTest.yml
rozaychen Jun 14, 2024
0e45bdb
Add Test
rozaychen Jun 14, 2024
45bfa5b
Revert "Add Test"
rozaychen Jun 14, 2024
b842214
Update apiBreakTest.yml
rozaychen Jun 14, 2024
a916b78
Update check_api_breakage.sh
rozaychen Jun 14, 2024
bdb9b74
Update apiBreakTest.yml
rozaychen Jun 17, 2024
b10d30f
Update apiBreakTest.yml
rozaychen Jun 17, 2024
ce3b1b8
Update apiBreakTest.yml
rozaychen Jun 17, 2024
b9420f3
Update apiBreakTest.yml
rozaychen Jun 17, 2024
c89c0dd
Update apiBreakTest.yml
rozaychen Jun 17, 2024
b1df401
Update apiBreakTest.yml
rozaychen Jun 17, 2024
59db256
Merge pull request #3751 from rozaychen/main
rozaychen Jun 18, 2024
0cac0c3
Update apiBreakTest.yml
rozaychen Jun 18, 2024
213b0ab
Update API dumps for new version
github-actions[bot] Jun 18, 2024
32ea7de
Update APIDigesterCheck.yml
rozaychen Jun 19, 2024
3d17366
Update apiBreakTest.yml
rozaychen Jun 19, 2024
c13b808
Update APIDigesterCheck.yml
rozaychen Jun 19, 2024
be150ed
Update API dumps for new version
github-actions[bot] Jun 19, 2024
ee20d60
Update APIDigesterCheck.yml
rozaychen Jun 20, 2024
6faf7d6
Update API dumps for new version
github-actions[bot] Jun 20, 2024
df113d5
Auto-send Slack Message when Check Failed
rozaychen Jun 20, 2024
2bff924
Update API dumps for new version
github-actions[bot] Jun 20, 2024
26220b3
Use aws-amplify-ops for committing
rozaychen Jun 21, 2024
c0ffb41
Update API dumps for new version
aws-amplify-ops Jun 21, 2024
c7db8f2
Update apiBreakTest.yml
rozaychen Jun 25, 2024
8668033
Update CODEOWNERS
rozaychen Jun 27, 2024
3d6a9ba
Update apiBreakTest.yml to remove review adding function
rozaychen Jun 27, 2024
2895f35
Enabling Configurable Exceptions
rozaychen Jul 2, 2024
1c0d802
Update CODEOWNERS
rozaychen Jul 15, 2024
4d45785
Rename APIDigesterCheck.yml to api_digester_check.yml
rozaychen Jul 15, 2024
6d6ad03
Update api_digester_check.yml
rozaychen Jul 15, 2024
0659b85
Update api_digester_check.yml
rozaychen Jul 15, 2024
a640f08
Update and rename apiBreakTest.yml to api-breaking-changes-detection.yml
rozaychen Jul 15, 2024
e6c0aad
Rename check_api_breakage.sh to CircleciScripts/check_api_breakage.sh
rozaychen Jul 15, 2024
c3e9044
Update api-breaking-changes-detection.yml
rozaychen Jul 17, 2024
421d7a4
Update API dumps for new version
aws-amplify-ops Jul 17, 2024
2c8441f
Merge branch 'main' into feat/api-breakage-checker
rozaychen Jul 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,7 @@

# Changes to Xcode / OS runtime versions run in CI/CD requires admin approval.
/.github/composite_actions/get_platform_parameters/action.yml @aws-amplify/amplify-ios-admins

# Changes to files in the api-dump or api-dump-test folder require admin approval.
api-dump/* @aws-amplify/amplify-ios-admins
api-dump-test/* @aws-amplify/amplify-ios-admins
223 changes: 223 additions & 0 deletions .github/workflows/api-breaking-changes-detection.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
name: Public Interface Breakage Detection

on:
pull_request:

permissions:
contents: write
pull-requests: write

jobs:
build-and-check-api-breakage:
name: Build and Check API Breakage
runs-on: macos-latest

steps:
- name: Checkout repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 #v4.1.1
with:
ref: ${{ github.head_ref }} # Checkout the PR branch
fetch-depth: 1

- name: Fetch the branchs
run: |
git fetch origin ${{ github.sha }}


- name: Setup and Run Swift API Diff
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
# Define the list of exceptions to filter out
exceptions=(
'has been added as a new enum case$'
'is now with @_spi$'
)

# Define the mandatory patterns to filter out
mandatory_patterns=(
'^/\*'
'^$'
)

# Function to apply patterns with grep
apply_patterns() {
local input="$1"
local output="$input"

# Apply mandatory patterns
for pattern in "${mandatory_patterns[@]}"; do
output=$(echo "$output" | grep -v "$pattern")
done

# Apply exceptions
for exception in "${exceptions[@]}"; do
output=$(echo "$output" | grep -v "$exception")
done

echo "$output"
}

echo "Swift version: $(swift --version)"
echo "Swift package manager version: $(swift package --version)"
swift package resolve

# Ensure we are in the correct directory
cd $GITHUB_WORKSPACE

# Run swift-api-diff commands here directly
NEW_API_DIR=$(mktemp -d)
OLD_API_DIR=$(mktemp -d)
SDK_PATH=$(xcrun --show-sdk-path)

# Get all library module names
# Moduels with aws-crt-swift as dependency are not listed due to swift-api-digester's issue with analyzing C dependencies
modules=$(swift package dump-package | jq -r '.products | map(select(.name == "Amplify" or .name == "CoreMLPredictionsPlugin" or .name == "AWSDataStorePlugin" or .name == "AWSPluginsCore")) | map(.name) | .[]')
echo "Modules: $modules"

echo "Fetching old version..."
git fetch origin ${{ github.event.pull_request.base.sha }}
git checkout ${{ github.event.pull_request.base.sha }}
built=false
for module in $modules; do
# If file doesn't exits in the old directory
if [ ! -f api-dump/${module}.json ]; then
echo "Old API file does not exist in the base branch. Generating it..."
# Check if the project has been built
if ! $built; then
echo "Building project..."
swift build > /dev/null 2>&1 || { echo "Failed to build project"; exit 1; }
built=true
fi

# Generate the API file using api-digester
swift api-digester -sdk "$SDK_PATH" -dump-sdk -module "$module" -o "$OLD_API_DIR/${module}.json" -I .build/debug || { echo "Failed to dump new SDK for module $module"; exit 1; }
else
# Use the api-dump/${module}.json file from the base branch directly
cp "api-dump/${module}.json" "$OLD_API_DIR/${module}.json"
fi
done

echo "Fetching new version..."
git checkout ${{ github.sha }}
git log -1 # Print the commit info for debugging
swift build> /dev/null 2>&1 || { echo "Failed to build new version"; exit 1; }
for module in $modules; do
swift api-digester -sdk "$SDK_PATH" -dump-sdk -module "$module" -o "$NEW_API_DIR/${module}.json" -I .build/debug || { echo "Failed to dump new SDK for module $module"; exit 1; }
done

# Compare APIs for each module and capture the output
api_diff_output=""
for module in $modules; do
swift api-digester -sdk "$SDK_PATH" -diagnose-sdk --input-paths "$OLD_API_DIR/${module}.json" --input-paths "$NEW_API_DIR/${module}.json" >> "api-diff-report-${module}.txt" 2>&1
module_diff_output=$(apply_patterns "$(cat "api-diff-report-${module}.txt")")
if [ -n "$module_diff_output" ]; then
api_diff_output="${api_diff_output}\n**Module: ${module}**\n${module_diff_output}\n"

# Check if there are lines containing "has been renamed to Func"
if echo "$module_diff_output" | grep -q 'has been renamed to Func'; then
# Capture the line containing "has been renamed to Func"
renamed_line=$(echo "$module_diff_output" | grep 'has been renamed to Func')

# Append a message to the module_diff_output
api_diff_output="${api_diff_output}👉🏻 _Note: If you're just adding optional parameters to existing methods, neglect the line:_\n_${renamed_line}_\n"
fi
fi
done

echo "API_DIFF_OUTPUT<<EOF" >> $GITHUB_ENV
if [ -n "$api_diff_output" ]; then
echo "### 💔 Public API Breaking Change detected:" >> $GITHUB_ENV
echo -e "$api_diff_output" >> $GITHUB_ENV
echo "EOF" >> $GITHUB_ENV
else
echo "### ✅ No Public API Breaking Change detected" >> $GITHUB_ENV
echo "EOF" >> $GITHUB_ENV
fi

# Checkout to the branch associated with the pull request
git stash --include-untracked
git checkout ${{ github.head_ref }}

if [ ! -d "api-dump" ]; then
echo "api-dump folder does not exist. Creating it..."
mkdir -p "api-dump"
fi

# Update the api-dump folder of the new version by making a commit if there are changes
for module in $modules; do
if [ ! -f api-dump/${module}.json ]; then
echo "API file does not exist in api-dump folder. Creating it..."
echo "{}" > "api-dump/${module}.json"
fi
if ! diff "$NEW_API_DIR/${module}.json" "api-dump/${module}.json" > /dev/null; then
echo "Updating API Dumps..."
mv "$NEW_API_DIR/${module}.json" "api-dump/${module}.json"
fi
done

git config --global user.name "aws-amplify-ops"
git config --global user.email "aws-amplify@amazon.com"

git add api-dump/*.json

if ! git diff --cached --quiet --exit-code; then
# Get the file names that have changes
changed_files=$(git diff --cached --name-only)

push_changes=false
for file in $changed_files; do
if [[ $file == api-dump/* ]]; then
# Get the number of lines in the file
total_lines=$(wc -l < "$file")
# Get the line numbers of the changes
changed_lines=$(git diff --cached -U0 "$file" | grep -o '@@ [^ ]* [^ ]* @@' | awk '{print $3}' | cut -d ',' -f1 | sed 's/[^0-9]//g')
echo "Changed lines in $file: $changed_lines"
# Check if any change is not within the last 10 lines
for line in $changed_lines; do
if [ "$line" -le "$((total_lines - 10))" ]; then
push_changes=true
break
fi
done

# If any file should be pushed, break out of the loop
if [ "$push_changes" = true ]; then
break
fi
fi
done

if [ "$push_changes" = true ]; then
git commit -m "Update API dumps for new version"
git push origin HEAD:${{ github.head_ref }}
else
echo "No changes to commit in the api-dump folder."
fi
else
echo "No changes to commit in the api-dump folder."
fi

git stash pop || true

- name: Comment on PR with API Diff
uses: actions/github-script@d7906e4ad0b1822421a7e6a35d5ca353c962f410 # v6.4.1
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const apiDiffOutput = process.env.API_DIFF_OUTPUT;
const issueNumber = context.payload.pull_request.number;
const owner = context.repo.owner;
const repo = context.repo.repo;

if (apiDiffOutput && apiDiffOutput.trim().length > 0) {
github.rest.issues.createComment({
owner: owner,
repo: repo,
issue_number: issueNumber,
body: `## API Breakage Report\n${apiDiffOutput}\n`
});
} else {
console.log("No API diff output found.");
}

39 changes: 39 additions & 0 deletions .github/workflows/api_digester_check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Swift API Digester Functionality Check

on:
schedule:
- cron: '0 15 * * *' # This will run the action every day at 3:00 pm UTC (8:00 am PDT)
workflow_dispatch: # Allows manual triggering

jobs:
check-swift-api-digester:
runs-on: macos-latest

steps:
- name: Checkout repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 #v4.1.1

- name: Check API Digester
shell: bash
env:
WEBHOOK_URL: ${{ secrets.SLACK_API_CHECKER_WEBHOOK_URL }}
run: |
TEMP_DIR=$(mktemp -d)
echo "Temporary directory created at $TEMP_DIR"
SDK_PATH=$(xcrun --sdk macosx --show-sdk-path)
echo "SDK Path: $SDK_PATH"

# Run swift-api-digester
swift api-digester -sdk "$SDK_PATH" -diagnose-sdk --input-paths api-dump-test/A.json --input-paths api-dump-test/B.json >> "$TEMP_DIR/api-digester-output.txt" 2>&1

# Display the output
cat "$TEMP_DIR/api-digester-output.txt"

if diff "$TEMP_DIR/api-digester-output.txt" api-dump-test/expected-result.txt; then
echo "The output matches the expected result."
else
echo "The output does not match the expected result."
WORKFLOW_URL="https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}"
echo "$WORKFLOW_URL" | xargs -I {} curl -s POST "$WEBHOOK_URL" -H "Content-Type:application/json" --data '{"WORKFLOW_URL":"{}"}'
exit 1
fi
87 changes: 87 additions & 0 deletions CircleciScripts/check_api_breakage.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
#!/bin/bash

# Script: check_api_breakage.sh

# Ensure the script is run from the root of the repository
if [ ! -d ".git" ]; then
echo "This script must be run from the root of the repository."
exit 1
fi

# Check if a base branch is provided as an argument
if [ -z "$1" ]; then
echo "Usage: $0 <base-branch>"
exit 1
fi

BASE_BRANCH=$1

# Setup environment variables
OLD_API_DIR=$(mktemp -d)
NEW_API_DIR=$(mktemp -d)
REPORT_DIR=$(mktemp -d)
SDK_PATH=$(xcrun --show-sdk-path)

modules=$(swift package dump-package | jq -r '.products | map(select(.name == "Amplify" or .name == "CoreMLPredictionsPlugin")) | map(.name) | .[]')

# Ensure repository is up to date
git fetch origin

# Fetch and build the main branch
echo "Fetching API from base branch ($BASE_BRANCH)..."
git checkout $BASE_BRANCH
git pull origin $BASE_BRANCH
swift build > /dev/null 2>&1 || { echo "Failed to build base branch ($BASE_BRANCH)"; exit 1; }
for module in $modules; do
# If file doesn't exits in the old directory
if [ ! -f api-dump/${module}.json ]; then
echo "Old API file does not exist in the base branch. Generating it..."
# Check if the project has been built
if ! $built; then
echo "Building project..."
swift build > /dev/null 2>&1 || { echo "Failed to build project"; exit 1; }
built=true
fi

# Generate the API file using api-digester
swift api-digester -sdk "$SDK_PATH" -dump-sdk -module "$module" -o "$OLD_API_DIR/${module}.json" -I .build/debug || { echo "Failed to dump new SDK for module $module"; exit 1; }
else
# Use the api-dump/${module}.json file from the base branch directly
cp "api-dump/${module}.json" "$OLD_API_DIR/${module}.json"
fi
done

# Fetch and build the current branch
echo "Fetching API from current branch..."
git checkout -
git pull origin "$(git rev-parse --abbrev-ref HEAD)"
swift build > /dev/null 2>&1 || { echo "Failed to build current branch"; exit 1; }
for module in $modules; do
swift api-digester -sdk "$SDK_PATH" -dump-sdk -module "${module}" -o "$NEW_API_DIR/${module}.json" -I .build/debug || { echo "Failed to dump SDK for current branch"; exit 1; }
done

# Compare APIs for each module and capture the output
api_diff_output=""
for module in $modules; do
swift api-digester -sdk "$SDK_PATH" -diagnose-sdk --input-paths "$OLD_API_DIR/${module}.json" --input-paths "$NEW_API_DIR/${module}.json" > "$REPORT_DIR/api-diff-report.txt" 2>&1
module_diff_output=$(grep -v '^/\*' "$REPORT_DIR/api-diff-report.txt" | grep -v '^$' || true)
if [ -n "$module_diff_output" ]; then
api_diff_output=$(printf "%s\n Module: %s\n%s\n" "$api_diff_output" "$module" "$module_diff_output")
# Check if there are lines containing "has been renamed to Func"
if echo "$module_diff_output" | grep -q 'has been renamed to Func'; then
# Capture the line containing "has been renamed to Func"
renamed_line=$(echo "$module_diff_output" | grep 'has been renamed to Func')

# Append a message to the module_diff_output
api_diff_output="${api_diff_output}👉🏻 _Note: If you're just adding optional parameters to existing methods, neglect the line:_\n_${renamed_line}_\n"
fi
fi
done

if [ -n "$api_diff_output" ];
then
echo "💔 Public API Breaking Change detected:"
echo "$api_diff_output"
else
echo "✅ No Public API Breaking Change detected"
fi
Loading
Loading