Skip to content

ECS Worker unable to assume IAM role #18708

@BennyG93

Description

@BennyG93

Bug summary

Hello, I am running Prefect self hosted on an EC2 instance in AWS. I have configured an AWS ECS work pool and am following this guide to create an AWS ECS based worker to pick up jobs.

The worker is correctly registered with my Prefect server and the work pool is active. The container image is prefecthq/prefect:3-latest and the start up command is:

"command": [
        "/bin/sh",
        "-c",
        "pip install prefect-aws && prefect worker start --pool aws-ecs-work-pool --type ecs"
      ],

When I execute a deployment, the job is successfully picked up by the worker, however it immediately fails when trying to submit the job to ECS for task execution. I get this error:

  File "/usr/local/lib/python3.12/site-packages/botocore/session.py", line 424, in get_scoped_config
    raise ProfileNotFound(profile=profile_name)
botocore.exceptions.ProfileNotFound: The config profile (prefect) could not be found

prefect.flow_runs.worker
Reported flow run '4ab3b0a2-5e29-4031-a621-f7159b46a880' as crashed: Flow run could not be submitted to infrastructure:
ProfileNotFound('The config profile (prefect) could not be found')

The ECS worker has an IAM role attached to it using the taskRoleArn and executionRoleArn configurations in the task definition

I have even tried hard coding the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables of the task definition.

But no matter what I do, the execution always fails immediately with this error. It's like it's ignoring all of my AWS credentials config and always trying to find a profile with the name prefect which doesn't exist and fails.

I also get the same error when running the worker on my local machine which is configured and authenticated to AWS. Running prefect worker start --pool "aws-ecs-work-pool" and I get the same error.

Version info

Version:             3.4.10
API version:         0.8.4
Python version:      3.12.5
Git commit:          fb979941
Built:               Tue, Jul 22, 2025 07:23 PM
OS/Arch:             darwin/arm64
Profile:             xxx
Server type:         server
Pydantic version:    2.11.7
Integrations:
  prefect-aws:       0.5.13

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions