@@ -38,21 +38,22 @@ that lists all the available options:
38
38
python -m test_runner --help
39
39
usage: test_runner.py [-h] --key-name KEY_NAME --key-path KEY_PATH [-n PARALLELISM] [--sequential] [--credential CREDENTIAL] [--use-default-iam-credentials] [--retry-on-failures] [--tests-root-dir TESTS_ROOT_DIR] [--global-build-number GLOBAL_BUILD_NUMBER] [-c TESTS_CONFIG] [-i [INSTANCES ...]]
40
40
[-o [OSS ...]] [-s [SCHEDULERS ...]] [-r [REGIONS ...]] [-f FEATURES [FEATURES ...]] [--show-output] [--reports {html,junitxml,json,cw} [{html,junitxml,json,cw} ...]] [--cw-region CW_REGION] [--cw-namespace CW_NAMESPACE] [--cw-timestamp-day-start] [--output-dir OUTPUT_DIR]
41
- [--custom-node-url CUSTOM_NODE_URL] [--custom-cookbook-url CUSTOM_COOKBOOK_URL] [--createami-custom-cookbook-url CREATEAMI_CUSTOM_COOKBOOK_URL] [--createami-custom-node-url CREATEAMI_CUSTOM_NODE_URL] [--custom-awsbatchcli-url CUSTOM_AWSBATCHCLI_URL] [--pre-install PRE_INSTALL]
42
- [--post-install POST_INSTALL] [--instance-types-data INSTANCE_TYPES_DATA] [--custom-ami CUSTOM_AMI] [--pcluster-git-ref PCLUSTER_GIT_REF] [--cookbook-git-ref COOKBOOK_GIT_REF] [--node-git-ref NODE_GIT_REF] [--ami-owner AMI_OWNER] [--available-amis-oss-x86 [AVAILABLE_AMIS_OSS_X86 ...]]
43
- [--available-amis-oss-arm [AVAILABLE_AMIS_OSS_ARM ...]] [--benchmarks] [--benchmarks-target-capacity BENCHMARKS_TARGET_CAPACITY] [--benchmarks-max-time BENCHMARKS_MAX_TIME] [--scaling-test-config SCALING_TEST_CONFIG]
44
- [--cluster-custom-resource-service-token CLUSTER_CUSTOM_RESOURCE_SERVICE_TOKEN] [--resource-bucket RESOURCE_BUCKET] [--lambda-layer-source LAMBDA_LAYER_SOURCE] [--api-definition-s3-uri API_DEFINITION_S3_URI] [--api-infrastructure-s3-uri API_INFRASTRUCTURE_S3_URI] [--api-uri API_URI]
45
- [--policies-uri POLICIES_URI] [--vpc-stack VPC_STACK] [--cluster CLUSTER] [--no-delete] [--delete-logs-on-success] [--stackname-suffix STACKNAME_SUFFIX] [--dry-run] [--iam-user-role-stack-name IAM_USER_ROLE_STACK_NAME] [--directory-stack-name DIRECTORY_STACK_NAME]
46
- [--slurm-database-stack-name SLURM_DATABASE_STACK_NAME] [--slurm-dbd-stack-name SLURM_DBD_STACK_NAME] [--munge-key-secret-arn MUNGE_KEY_SECRET_ARN] [--external-shared-storage-stack-name EXTERNAL_SHARED_STORAGE_STACK_NAME] [--bucket-name BUCKET_NAME]
47
- [--custom-security-groups-stack-name CUSTOM_SECURITY_GROUPS_STACK_NAME] [--force-run-instances] [--force-elastic-ip] [--retain-ad-stack] [--proxy-stack PROXY_STACK] [--build-image-roles-stack BUILD_IMAGE_ROLES_STACK] [--api-stack API_STACK]
41
+ [--custom-node-url CUSTOM_NODE_URL] [--custom-cookbook-url CUSTOM_COOKBOOK_URL] [--createami-custom-cookbook-url CREATEAMI_CUSTOM_COOKBOOK_URL] [--createami-custom-node-url CREATEAMI_CUSTOM_NODE_URL] [--custom-awsbatchcli-url CUSTOM_AWSBATCHCLI_URL]
42
+ [--pcluster-installer-path PCLUSTER_INSTALLER_PATH] [--pre-install PRE_INSTALL] [--post-install POST_INSTALL] [--instance-types-data INSTANCE_TYPES_DATA] [--custom-ami CUSTOM_AMI] [--pcluster-git-ref PCLUSTER_GIT_REF] [--cookbook-git-ref COOKBOOK_GIT_REF]
43
+ [--node-git-ref NODE_GIT_REF] [--ami-owner AMI_OWNER] [--available-amis-oss-x86 [AVAILABLE_AMIS_OSS_X86 ...]] [--available-amis-oss-arm [AVAILABLE_AMIS_OSS_ARM ...]] [--benchmarks] [--benchmarks-target-capacity BENCHMARKS_TARGET_CAPACITY] [--benchmarks-max-time BENCHMARKS_MAX_TIME]
44
+ [--scaling-test-config SCALING_TEST_CONFIG] [--cluster-custom-resource-service-token CLUSTER_CUSTOM_RESOURCE_SERVICE_TOKEN] [--resource-bucket RESOURCE_BUCKET] [--lambda-layer-source LAMBDA_LAYER_SOURCE] [--api-definition-s3-uri API_DEFINITION_S3_URI]
45
+ [--api-infrastructure-s3-uri API_INFRASTRUCTURE_S3_URI] [--api-uri API_URI] [--policies-uri POLICIES_URI] [--vpc-stack VPC_STACK] [--cluster CLUSTER] [--no-delete] [--delete-logs-on-success] [--stackname-suffix STACKNAME_SUFFIX] [--dry-run]
46
+ [--iam-user-role-stack-name IAM_USER_ROLE_STACK_NAME] [--directory-stack-name DIRECTORY_STACK_NAME] [--slurm-database-stack-name SLURM_DATABASE_STACK_NAME] [--slurm-dbd-stack-name SLURM_DBD_STACK_NAME] [--munge-key-secret-arn MUNGE_KEY_SECRET_ARN]
47
+ [--external-shared-storage-stack-name EXTERNAL_SHARED_STORAGE_STACK_NAME] [--bucket-name BUCKET_NAME] [--custom-security-groups-stack-name CUSTOM_SECURITY_GROUPS_STACK_NAME] [--force-run-instances] [--force-elastic-ip] [--retain-ad-stack] [--proxy-stack PROXY_STACK]
48
+ [--build-image-roles-stack BUILD_IMAGE_ROLES_STACK] [--api-stack API_STACK]
48
49
49
50
Run integration tests suite.
50
51
51
52
options:
52
53
-h, --help show this help message and exit
53
54
--key-name KEY_NAME Key to use for EC2 instances (default: None)
54
55
--key-path KEY_PATH Path to the key to use for SSH connections (default: None)
55
- -n PARALLELISM , --parallelism PARALLELISM
56
+ -n, --parallelism PARALLELISM
56
57
Tests parallelism for every region. (default: None)
57
58
--sequential Run tests in a single process. When not specified tests will spawn a process for each region under test. (default: False)
58
59
--credential CREDENTIAL
@@ -67,18 +68,16 @@ options:
67
68
The build number passed from the testing pipelines (default: 0)
68
69
69
70
Test dimensions:
70
- -c TESTS_CONFIG, --tests-config TESTS_CONFIG
71
- Config file that specifies the tests to run and the dimensions to enable for each test. Note that when a config file is used the following flags are ignored: instances, regions, oss, schedulers. Refer to the docs for further details on the config format: https://github.com/aws/aws-
72
- parallelcluster/blob/develop/tests/integration-tests/README.md (default: None)
73
- -i [INSTANCES ...], --instances [INSTANCES ...]
71
+ -c, --tests-config TESTS_CONFIG
72
+ Config file that specifies the tests to run and the dimensions to enable for each test. Note that when a config file is used the following flags are ignored: instances, regions, oss, schedulers. Refer to the docs for further details on the config format: https://github.com/aws/aws-parallelcluster/blob/develop/tests/integration-tests/README.md (default: None)
73
+ -i, --instances [INSTANCES ...]
74
74
AWS instances under test. Ignored when tests-config is used. (default: [])
75
- -o [OSS ...], --oss [OSS ...]
76
- OSs under test. Ignored when tests-config is used. (default: [])
77
- -s [SCHEDULERS ...], --schedulers [SCHEDULERS ...]
75
+ -o, --oss [OSS ...] OSs under test. Ignored when tests-config is used. (default: [])
76
+ -s, --schedulers [SCHEDULERS ...]
78
77
Schedulers under test. Ignored when tests-config is used. (default: [])
79
- -r [REGIONS ...] , --regions [REGIONS ...]
78
+ -r, --regions [REGIONS ...]
80
79
AWS regions where tests are executed. Ignored when tests-config is used. (default: [])
81
- -f FEATURES [FEATURES ...] , --features FEATURES [FEATURES ...]
80
+ -f, --features FEATURES [FEATURES ...]
82
81
Run only tests for the listed features. Prepending the not keyword to the feature name causes the feature to be excluded. (default: )
83
82
84
83
Test reports:
@@ -105,6 +104,8 @@ Custom packages and templates:
105
104
URL to a custom node package for the createami command. (default: None)
106
105
--custom-awsbatchcli-url CUSTOM_AWSBATCHCLI_URL
107
106
URL to a custom awsbatch cli package. (default: None)
107
+ --pcluster-installer-path PCLUSTER_INSTALLER_PATH
108
+ Path to ParallelCluster installer. (default: None)
108
109
--pre-install PRE_INSTALL
109
110
URL to a pre install script (default: None)
110
111
--post-install POST_INSTALL
@@ -312,6 +313,54 @@ the test will be executed in
312
313
* ` eu-west-1 ` using the AZ with ZoneId ` euw1-az1 ` (ZoneId is consistent across accounts)
313
314
* ` eu-central-1 ` using a random AZ available in the region
314
315
316
+ #### OS Rotation
317
+ The framework includes automatic OS rotation to ensure that all supported operating systems are tested regularly:
318
+
319
+ - The framework calculates a rotation index daily, and assign values to the jinja variables upon test running
320
+ - Tests can be constrained to specific available AMIs using the ` --available-amis-oss-x86 ` and ` --available-amis-oss-arm ` parameters
321
+ - It can be customized to filter OS choices based on feature compatibility (e.g., DCV, Lustre, Batch scheduler support)
322
+
323
+ Use Jinja variables in test configs with the pattern:
324
+ ```
325
+ dimensions:
326
+ - oss: [ {{ OS_X86_3 }} ]
327
+ ```
328
+
329
+ #### Instance Type Rotation
330
+
331
+ Similar to OS rotation, the framework also rotates instance types for dynamic testing:
332
+
333
+ - Instance types are automatically discovered and rotated for major regions (us-east-1, us-west-2, eu-west-1)
334
+ - Excludes legacy instance types (m1, m2, t1, etc.) for better test reliability
335
+
336
+ Use Jinja variables in test configs with the pattern:
337
+ ```
338
+ dimensions:
339
+ - instances: [{{ US_EAST_1_INSTANCE_TYPE_0 }}.xlarge]
340
+ ```
341
+
342
+ #### Capacity Reservation Management
343
+
344
+ The framework includes automatic capacity reservation management for tests that require specific instance types with guaranteed capacity. This feature:
345
+
346
+ - Automatically detects capacity reservation requirements in test configuration files using Jinja variables
347
+ - Creates or modifies existing EC2 capacity reservations as needed
348
+ - Supports placement groups and time-limited reservations
349
+ - Falls back to default availability zones if reservations cannot be created
350
+
351
+ Use Jinja variables in test configs with the pattern:
352
+ ```
353
+ dimensions:
354
+ - regions: [{{ INSTANCE_TYPE_CAPACITY_RESERVATION_COUNT_INSTANCES_HOURS_HOURS_[YESPG|NOPG]_[OS] }}]
355
+ ```
356
+
357
+ Example:
358
+ ```
359
+ # Reserve 2 c5.xlarge instances for 2 hours with placement group
360
+ dimensions:
361
+ - regions: []{{ c5_xlarge_CAPACITY_RESERVATION_2_INSTANCES_2_HOURS_YESPG_alinux2023 }}]
362
+ ```
363
+
315
364
#### Using CLI options
316
365
317
366
The following options can be used to control the parametrization of test cases:
@@ -390,6 +439,13 @@ metrics. You can use the options `--cw-region` (default `us-east-1`) and `--cw-n
390
439
(default ` ParallelCluster/IntegrationTests ` ) to specify what region and what metric namespace
391
440
you want to use for the published metrics.
392
441
442
+ ### Test Metadata Management
443
+
444
+ The framework automatically manages test metadata using DynamoDB tables for tracking test execution and results across regions. This system:
445
+
446
+ - Creates metadata tables automatically in the appropriate reporting region
447
+ - Tracks test execution metadata for analysis and reporting
448
+
393
449
### Parallelize Tests Execution
394
450
The following options can be used to control tests parallelism:
395
451
* ` --sequential ` : by default the tests orchestrator executes a separate parallel process for each region under test.
0 commit comments