China S3 bucket resource returning 401 for non-ICP accounts

[adobe-jeremy]

Terraform and AWS Provider Version

Terraform v1.7.3
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v5.98.0

Affected Resource(s) or Data Source(s)

• aws_s3_bucket

Expected Behavior

Terrraform bucket created and both plan and apply finish successfully.

Actual Behavior

Terraform plan is successful. During apply the bucket is created but the apply ultimately fails with an error.

Relevant Error/Panic Output

 Error: reading S3 Bucket (my-tf-test-bucket-10000) location: operation error S3: HeadBucket, https response error StatusCode: 401, RequestID: V2D8C443RGNNEDZ6, HostID: d6tTm73+CHIR3BhZwA6WIonZs3TB0zYI35gQaWNQgQJuPu2F72TKzXXpMv/tTr2Y4pqLyJPo8OA=, api error Unauthorized: Unauthorized
│ 
│   with aws_s3_bucket.d,
│   on main.tf line 17, in resource "aws_s3_bucket" "d":
│   17: resource "aws_s3_bucket" "d" {

Sample Terraform Configuration

terraform {
    required_version = ">= 1.0"

    required_providers {
        aws = {
            source  = "hashicorp/aws"
            version = "5.98.0"
        }
    }
}

provider "aws" {
  region = "cn-north-1"
}


resource "aws_s3_bucket" "d" {
  bucket = "my-tf-test-bucket-10000"
}

Steps to Reproduce

1. Terraform apply

Debug Logging

Click to expand log output

2025-05-23T09:57:10.855-0400 [TRACE] statemgr.Filesystem: writing snapshot at terraform.tfstate
2025-05-23T09:57:10.860-0400 [DEBUG] State storage *statemgr.Filesystem declined to persist a state snapshot
2025-05-23T09:57:10.860-0400 [ERROR] vertex "aws_s3_bucket.d" error: reading S3 Bucket (my-tf-test-bucket-10000) location: operation error S3: HeadBucket, https response error StatusCode: 401, RequestID: V2D8C443RGNNEDZ6, HostID: d6tTm73+CHIR3BhZwA6WIonZs3TB0zYI35gQaWNQgQJuPu2F72TKzXXpMv/tTr2Y4pqLyJPo8OA=, api error Unauthorized: Unauthorized
2025-05-23T09:57:10.860-0400 [TRACE] vertex "aws_s3_bucket.d": visit complete, with errors
2025-05-23T09:57:10.860-0400 [TRACE] dag/walk: upstream of "provider[\"registry.terraform.io/hashicorp/aws\"] (close)" errored, so skipping
2025-05-23T09:57:10.860-0400 [TRACE] dag/walk: upstream of "root" errored, so skipping
2025-05-23T09:57:10.860-0400 [TRACE] statemgr.Filesystem: reading latest snapshot from terraform.tfstate
2025-05-23T09:57:10.860-0400 [TRACE] statemgr.Filesystem: read snapshot with lineage "25d3357c-3547-6c8a-f959-455fa4048f8e" serial 1
2025-05-23T09:57:10.860-0400 [TRACE] statemgr.Filesystem: no original state snapshot to back up
2025-05-23T09:57:10.860-0400 [TRACE] statemgr.Filesystem: no state changes since last snapshot
2025-05-23T09:57:10.860-0400 [TRACE] statemgr.Filesystem: writing snapshot at terraform.tfstate
╷
│ Error: reading S3 Bucket (my-tf-test-bucket-10000) location: operation error S3: HeadBucket, https response error StatusCode: 401, RequestID: V2D8C443RGNNEDZ6, HostID: d6tTm73+CHIR3BhZwA6WIonZs3TB0zYI35gQaWNQgQJuPu2F72TKzXXpMv/tTr2Y4pqLyJPo8OA=, api error Unauthorized: Unauthorized
│ 
│   with aws_s3_bucket.d,
│   on main.tf line 17, in resource "aws_s3_bucket" "d":
│   17: resource "aws_s3_bucket" "d" {
│ 
╵
2025-05-23T09:57:10.867-0400 [TRACE] statemgr.Filesystem: removing lock metadata file .terraform.tfstate.lock.info
2025-05-23T09:57:10.867-0400 [TRACE] statemgr.Filesystem: unlocking terraform.tfstate using fcntl flock
2025-05-23T09:57:10.868-0400 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2025-05-23T09:57:10.871-0400 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/hashicorp/aws/5.98.0/darwin_arm64/terraform-provider-aws_v5.98.0_x5 pid=48537
2025-05-23T09:57:10.871-0400 [DEBUG] provider: plugin exited

GenAI / LLM Assisted Development

n/a

Important Facts and References

This looks to me to be a regression or at least similar to #15420. Additionally, the issue does not occur in provider version v5.97.0, so it looks to be introduced in v5.98.0. I suspect that the underlying cause is the upgrade of github.com/aws/aws-sdk-go-v2/feature/s3/manager to v1.17.75 due to https://github.com/aws/aws-sdk-go-v2/pull/3081/files

Would you like to implement a fix?

No

[github-actions]

Community Guidelines

This comment is added to every new Issue to provide quick reference to how the Terraform AWS Provider is maintained. Please review the information below, and thank you for contributing to the community that keeps the provider thriving! 🚀

Voting for Prioritization

• Please vote on this Issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize it.
• Please see our prioritization guide for additional information on how the maintainers handle prioritization.
• Please do not leave +1 or other comments that do not add relevant new information or questions; they generate extra noise for others following the Issue and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.
• For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.

[ewbankkit]

Relates #42739.

[github-actions]

Warning

This Issue has been closed, meaning that any additional comments are much easier for the maintainers to miss. Please assume that the maintainers will not see them.

Ongoing conversations amongst community members are welcome, however, the issue will be locked after 30 days. Moving conversations to another venue, such as the AWS Provider forum, is recommended. If you have additional concerns, please open a new issue, referencing this one where needed.