@@ -180,4 +180,53 @@ Example: java -Xmx10G -jar pathTo/VersionManager_X.X.jar -b mybucket-vm-test
180
180
-s .cram,.bam,.gz,.zip -a 7 -c MiloLab -l us-west-2
181
181
182
182
**************************************************************************************
183
+
184
+
185
+
186
+ u0028003$ java -jar ~/Code/AwsApps/target/S3Copy_0.1.jar
187
+
188
+ **************************************************************************************
189
+ ** S3 Copy : Jan 2023 **
190
+ **************************************************************************************
191
+ SC copies AWS S3 objects, unarchiving them as needed, within the same or different
192
+ accounts or downloads them to your local computer. Run this as a daemon with -l or run
193
+ repeatedly until complete. To upload files to S3, use the AWS CLI.
194
+
195
+ To use the app:
196
+ Create a ~/.aws/credentials file with your access, secret, and region info, chmod
197
+ 600 the file and keep it private. Use a txt editor or the AWS CLI configure
198
+ command, see https://aws.amazon.com/cli Example ~/.aws/credentials file:
199
+ [default]
200
+ aws_access_key_id = AKIARHBDRGYUIBR33RCJK6A
201
+ aws_secret_access_key = BgDV2UHZv/T5ENs395867ueESMPGV65HZMpUQ
202
+ region = us-west-2
203
+ Repeat these entries for multiple accounts replacing the word 'default' with a single
204
+ unique account name.
205
+
206
+ Required:
207
+ -j Provide a comma delimited string of copy jobs or a txt file with one per line.
208
+ A copy job consists of a full S3 URI as the source and a destination separated
209
+ by '>', e.g. 's3://source/tumor.cram > s3://destination/collabTumor.cram' or
210
+ folders 's3://source/alignments/tumor > s3://destination/Collab/' or local
211
+ 's3://source/alignments/tumor > .' Note, the trailing '/' is required in the
212
+ S3 destination for a recursive copy or when the local folder doesn't exist.
213
+
214
+ Optional/ Defaults:
215
+ -d Perform a dry run to list the actions that would be taken
216
+ -r Perform a recursive copy, defaults to an exact source key match
217
+ -e Email addresse(s) to send status messages, comma delimited, no spaces. Note,
218
+ the sendmail app must be configured on your system. Test it:
219
+ echo 'Subject: Hello' | sendmail yourEmailAddress@yourProvider.com
220
+ -x Expedite archive retrieval, increased cost $0.03/GB vs $0.01/GB, 1-5min vs 3-12hr,
221
+ defaults to standard.
222
+ -l Execute every hour (standard) or minute (expedited) until complete
223
+ -t Maximum threads to utilize, defaults to 8
224
+ -p AWS credentials profile, defaults to 'default'
225
+ -n Number of days to keep restored files in S3, defaults to 1
226
+ -a Print instructions for copying files between different accounts
227
+
228
+ Example: java -Xmx20G -jar pathTo/S3Copy_x.x.jar -e obama@real.gov -p obama -d -l
229
+ -j 's3://source/Logs.zip>s3://destination/,s3://source/normal > ~/Downloads/' -r
230
+ **************************************************************************************
231
+
183
232
</pre >
0 commit comments