2
2
Genomic data focused toolkit for working with AWS services (e.g. S3 and EC2). Includes exhaustive JUnit testing for each app.
3
3
4
4
<pre >
5
- u0028003$ java -jar -Xmx1G ~/Code/AwsApps/target/S3Copy_0.3.jar
5
+ u0028003$ java -jar ~/Code/AwsApps/target/S3Copy_0.5.jar
6
+
6
7
**************************************************************************************
7
- ** S3 Copy : Feb 2024 **
8
+ ** S3 Copy : May 2024 **
8
9
**************************************************************************************
9
10
SC copies AWS S3 objects, unarchiving them as needed, within the same or different
10
11
accounts or downloads them to your local computer. Run this as a daemon with -l or run
11
12
repeatedly until complete. To upload files to S3, use the AWS CLI.
12
13
13
14
To use the app:
14
- Create a ~/.aws/credentials file with your access, secret, and region info, chmod
15
+ Create a ~/.aws/credentials file with your access and secret info, chmod
15
16
600 the file and keep it private. Use a txt editor or the AWS CLI configure
16
17
command, see https://aws.amazon.com/cli Example ~/.aws/credentials file:
17
18
[default]
18
19
aws_access_key_id = AKIARHBDRGYUIBR33RCJK6A
19
20
aws_secret_access_key = BgDV2UHZv/T5ENs395867ueESMPGV65HZMpUQ
20
- region = us-west-2
21
21
Repeat these entries for multiple accounts replacing the word 'default' with a single
22
- unique account name.
22
+ unique account name. Temporary credentials with a session attribute also work.
23
23
24
24
Required:
25
25
-j Provide a comma delimited string of copy jobs or a txt file with one per line.
@@ -49,9 +49,10 @@ Example: java -Xmx10G -jar pathTo/S3Copy_x.x.jar -e obama@real.gov -p obama -d -
49
49
</pre >
50
50
51
51
<pre >
52
- u0028003$ java -jar -Xmx1G ~/Code/AwsApps/target/VersionManager_0.2.jar
52
+ u0028003$ java -jar ~/Code/AwsApps/target/VersionManager_0.4.jar
53
+
53
54
**************************************************************************************
54
- ** AWS S3 Version Manager : August 2023 **
55
+ ** AWS S3 Version Manager : May 2024 **
55
56
**************************************************************************************
56
57
Bucket versioning in S3 protects objects from being deleted or overwritten by hiding
57
58
the original when 'deleting' or over writing an existing object. Use this tool to
@@ -66,8 +67,9 @@ app is provided with no guarantee of proper function.
66
67
67
68
To use the app:
68
69
1) Enable S3 Object versioning on your bucket.
69
- 2) Install and configure the aws cli with your region, access and secret keys. See
70
- https://aws.amazon.com/cli
70
+ 2) Install and configure the aws cli with your region, access and secret keys.
71
+ Temporary credentials with a session attribute also work. See
72
+ https://aws.amazon.com/cli.
71
73
3) Use cli commands like 'aws s3 rm s3://myBucket/myObj.txt' or the AWS web Console to
72
74
'delete' particular objects. Then run this app to actually delete them.
73
75
@@ -86,7 +88,7 @@ Optional Parameters:
86
88
Example: java -Xmx10G -jar pathTo/VersionManager_X.X.jar -b mybucket-vm-test
87
89
-s .cram,.bam,.gz,.zip -a 7 -c MiloLab
88
90
89
- **************************************************************************************
91
+ **************************************************************************************
90
92
</pre >
91
93
92
94
See [ Misc/WorkingWithTheAWSJobRunner.pdf] ( https://github.com/HuntsmanCancerInstitute/AwsApps/blob/master/Misc/WorkingWithTheAWSJobRunner.pdf ) for details.
0 commit comments