Skip to content

Commit 6481666

Browse files
Various updates to CI (#331)
The changes included here, at a 10,000 foot view: - An overhaul of how tests are ran on Amazon Linux 2 on aarch64 platforms. An adjacent git repo (not shown here) also includes a lot of work to make this happen. Basically, we relied on a "pool" of CI runners to sit and wait idle until a new job came through. Now, CI runners are launched ad-hoc, for as many runners that are needed with or without a matrix. This also allows for the re-run of individual failed tasks if required, whereas before this simply would not work - Re-introduces sccache back into the runs. Previously, PL/Rust and some required dependencies flat out not work with sccache. However, those issues have been addressed - Introduces a nightly run setup, which for now does two things: 1.) Tests to make sure that the Dockerfile.try setup works as per the documentation, and 2.) Tests that the Debian packages can be both built and installed. These will be ran nightly against the develop branch, and we can certainly add more to it in the future NOTE: The nightly CI runs will not happen until these changes eventually land in main on next release.
1 parent ee84102 commit 6481666

File tree

9 files changed

+401
-139
lines changed

9 files changed

+401
-139
lines changed

.github/scripts/load_cache.sh

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@
33
# Downloads and extracts cache archive from S3.
44
#
55
# Expects the following environment variables to already exist:
6-
# * ARTIFACT_USER_AWS_PROFILE: the profile to use when issuing AWS CLI commands
76
# * AWS_CACHE_BUCKET: the S3 bucket in which to obtain the archive
87
# * HOME: executing user's home directory
98
#
@@ -13,13 +12,16 @@
1312
# Example usage:
1413
# . /path/to/plrust/.github/scripts/load_cache.sh
1514
# loadcache "some-cache-key-abc123"
15+
#
16+
# Note: This assumes the host in which this script is running on has the ability
17+
# to read and write from the bucket specified by AWS_CACHE_BUCKET
1618

1719
function loadcache() {
1820
local cache_key="$1"
1921

2022
echo "Checking to see if cache archive exists: $cache_key"
2123

22-
if aws s3api head-object --profile $ARTIFACT_USER_AWS_PROFILE --bucket $AWS_CACHE_BUCKET --key $cache_key &> /dev/null; then
24+
if aws s3api head-object --bucket $AWS_CACHE_BUCKET --key $cache_key &> /dev/null; then
2325
echo "Cache archive exists for $cache_key -- downloading and extracting now."
2426

2527
mkdir -p $HOME/artifacts/
@@ -28,7 +30,6 @@ function loadcache() {
2830
echo "Downloadng archive $cache_key and storing to $archive_path"
2931

3032
aws s3api get-object \
31-
--profile $ARTIFACT_USER_AWS_PROFILE \
3233
--bucket $AWS_CACHE_BUCKET \
3334
--key $cache_key \
3435
$archive_path

.github/scripts/save_cache.sh

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@
33
# Creates cache archive and uploads to S3.
44
#
55
# Expects the following environment variables to already exist:
6-
# * ARTIFACT_USER_AWS_PROFILE: the profile to use when issuing AWS CLI commands
76
# * AWS_CACHE_BUCKET: the S3 bucket in which to obtain the archive
87
# * HOME: executing user's home directory
98
#
@@ -15,6 +14,9 @@
1514
# . /path/to/plrust/.github/scripts/save_cache.sh
1615
# my_paths=(/path/one /path/two /path/three)
1716
# savecache "some-cache-key-abc123" "${my_paths[@]}"
17+
#
18+
# Note: This assumes the host in which this script is running on has the ability
19+
# to read and write from the bucket specified by AWS_CACHE_BUCKET
1820

1921
function savecache() {
2022
local cache_key="$1"
@@ -23,7 +25,7 @@ function savecache() {
2325

2426
echo "Checking to see if cache archive exists: $cache_key"
2527

26-
if aws s3api head-object --profile $ARTIFACT_USER_AWS_PROFILE --bucket $AWS_CACHE_BUCKET --key $cache_key &> /dev/null; then
28+
if aws s3api head-object --bucket $AWS_CACHE_BUCKET --key $cache_key &> /dev/null; then
2729
echo "Cache archive exists for $cache_key -- skipping archive creation."
2830
else
2931
echo "Cache archive does not exist for $cache_key -- creating archive now."
@@ -38,7 +40,6 @@ function savecache() {
3840
echo "Created archive $archive_path -- uploading now"
3941

4042
aws s3api put-object \
41-
--profile $ARTIFACT_USER_AWS_PROFILE \
4243
--bucket $AWS_CACHE_BUCKET \
4344
--key $cache_key \
4445
--body $archive_path

0 commit comments

Comments
 (0)