Skip to content

add some validation, polish inputs in pipeline benchmark tests #65

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 9 additions & 11 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ on:
workflow_dispatch:
inputs:
machine_type:
description: 'The GCE machine type'
description: 'The GCE machine type. (See docs for complete list of machine types https://cloud.google.com/compute/docs/general-purpose-machines)'
default: 't2d-standard-60'
type: string
num_workers:
Expand All @@ -22,13 +22,8 @@ on:
default: '11'
required: true
type: string
memory_request:
description: 'The memory request set for the workers'
default: '32Gi'
required: false
type: string
memory_limit:
description: 'The memory limit set for the workers'
memory:
description: 'The memory set for the workers'
default: '32Gi'
required: false
type: string
Expand All @@ -47,8 +42,11 @@ on:
rpc_endpoint:
description: 'The RPC endpoint zero-bin will use'
required: false
default: 'http://35.208.84.178:8545'
type: string
default: 'IMX_RPC'
type: choice
options:
- IMX_RPC
- INTERNAL_RPC

jobs:
test:
Expand Down Expand Up @@ -85,4 +83,4 @@ jobs:
GCP_WORKLOAD_IDENTITY_ID: ${{ secrets.GCP_WORKLOAD_IDENTITY_ID }}
GCP_SERVICE_ACCOUNT: ${{ secrets.GCP_SERVICE_ACCOUNT }}
run: >
./tools/run-benchmark.sh ${{ inputs.machine_type }} ${{ inputs.num_workers }} ${{ inputs.cpu_request }} ${{ inputs.cpu_limit }} ${{ inputs.memory_request }} ${{ inputs.memory_limit }} ${{ inputs.block_start }} ${{ inputs.block_end }} ${{ inputs.other_args }} ${{ inputs.rpc_endpoint }}
./tools/run-benchmark.sh ${{ inputs.machine_type }} ${{ inputs.num_workers }} ${{ inputs.cpu_request }} ${{ inputs.cpu_limit }} ${{ inputs.memory }} ${{ inputs.memory }} ${{ inputs.block_start }} ${{ inputs.block_end }} ${{ inputs.other_args }} ${{ inputs.rpc_endpoint }}
36 changes: 32 additions & 4 deletions tools/run-benchmark.sh
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@ LEADER_ENDPOINT="http://35.238.105.189:8080"
LOG_STRING_TO_WATCH_FOR="Finalized benchmarked proofs"
CPU_THRESHOLD=100
DISK_TYPE=""
NUM_WORKERS_LIMIT=200
IMX_RPC="http://35.208.84.178:8545"
INTERNAL_RPC="http://35.208.68.173:8545"
RPC_ADDRESS=""

# parameters
machine_type=$1
Expand All @@ -37,17 +41,41 @@ block_end=$8
other_args=$9 # will be appended to the csv file name
RPC_ENDPOINT=${10}

######################
# Do some validation #
######################
if [[ "$num_workers" -gt $NUM_WORKERS_LIMIT ]]; then
echo "error: Num workers can't be greater than ${NUM_WORKERS_LIMIT}" >&2
exit 1
fi

if [[ "$machine_type" == *"n4"* ]]; then
DISK_TYPE="hyperdisk-balanced"
re='^[0-9]+$'
if ! [[ $block_start =~ $re ]] ; then
echo "error: Block start must be a number" >&2; exit 1
fi

if ! [[ $block_end =~ $re ]] ; then
echo "error: Block end must be a number" >&2; exit 1
fi

if [[ $RPC_ENDPOINT == "IMX_RPC" ]]; then
RPC_ADDRESS=$IMX_RPC
elif [[ $RPC_ENDPOINT == "INTERNAL_RPC" ]]; then
RPC_ADDRESS=$INTERNAL_RPC
else
DISK_TYPE="pd-ssd"
echo "error: Wrong RPC endpoint" >&2; exit 1
fi

########################
# Update GKE node pool #
########################

if [[ "$machine_type" == *"n4"* ]]; then
DISK_TYPE="hyperdisk-balanced"
else
DISK_TYPE="pd-ssd"
fi

gcloud container node-pools update $NODE_POOL_NAME --cluster=$CLUSTER_NAME --machine-type=$machine_type --disk-type=$DISK_TYPE --region=$REGION

##########################
Expand Down Expand Up @@ -156,7 +184,7 @@ done

# Build out the request parameters
csv_file_name=$(printf "%s.%s.%s.%s.%scpu.%sworkers.sequential.%s.csv" "$block_start" "$block_end" "$machine_type" "$CPU_PLATFORM" "$cpu_request" "$num_workers" "$other_args")
post_body=$(printf '{"block_interval":"%s..=%s","block_source":{"ZeroBinRpc":{"rpc_url":"%s"}},"benchmark_output":{"GoogleCloudStorageCsv":{"file_name":"%s","bucket":"zkevm-csv"}}}' "$block_start" "$block_end" "$RPC_ENDPOINT" "$csv_file_name")
post_body=$(printf '{"block_interval":"%s..=%s","block_source":{"ZeroBinRpc":{"rpc_url":"%s"}},"benchmark_output":{"GoogleCloudStorageCsv":{"file_name":"%s","bucket":"zkevm-csv"}}}' "$block_start" "$block_end" "$RPC_ADDRESS" "$csv_file_name")

# Run the benchmark test
echo "Triggering benchmark test..."
Expand Down
Loading