Skip to content

Support auto-refresh of AWS credentials when assuming IAM role for long-running batch inference jobs #22

@adivekar-utexas

Description

@adivekar-utexas

When invoking S3, Bedrock etc in a different account, a persistent issue is the need to refresh AWS credentials after they have expired. This is blocking when running long jobs, since the default timeout is 3600 seconds.

The current approach which works is to assume the IAM role before EACH call to S3/etc, i.e.

session = IAMUtil.create_session('<role_arn>')
s3 = session.client('s3')
s3.put_object(Body=obj_str, Bucket=s3_bucket, Key=object_key) ## Some API 

However, this is slow as it requires two extra network calls (one for create_session, one for session.client), each of which take ~150milliseconds as benchmarked on us-east-1.

For each of the invocations, it makes sense to return an object which triggers an auto-refresh as soon as it detects that credentials have expired.

Metadata

Metadata

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions