Support HMAC for GCS authentication

[dentiny]

Is your feature request related to a problem or challenge?

duckdb leverages HMAC keys for authentication: https://duckdb.org/docs/stable/guides/network_cloud_storage/gcs_import.html

Is this something we could consider, so I don't need to manage two types of authentication in my project?
Thank you!

Describe the solution you'd like

No response

Willingness to contribute

None

[Xuanwo]

We add this support in gcs. BTW, you can just use those HMAC key to access gcs in S3 XML API.

[dentiny]

We add this support in gcs

Thank you for the quick reply!
Sorry for the dumb question, does that mean it's already supported but I'm unaware of? 😢

The reason why I ask is I don't see option here:

iceberg-rust/crates/iceberg/src/io/storage_gcs.rs

Lines 30 to 50 in 3fa9027

	/// Google Cloud Project ID
	pub const GCS_PROJECT_ID: &str = "gcs.project-id";
	/// Google Cloud Storage endpoint
	pub const GCS_SERVICE_PATH: &str = "gcs.service.path";
	/// Google Cloud user project
	pub const GCS_USER_PROJECT: &str = "gcs.user-project";
	/// Allow unauthenticated requests
	pub const GCS_NO_AUTH: &str = "gcs.no-auth";
	/// Google Cloud Storage credentials JSON string, base64 encoded.
	///
	/// E.g. base64::prelude::BASE64_STANDARD.encode(serde_json::to_string(credential).as_bytes())
	pub const GCS_CREDENTIALS_JSON: &str = "gcs.credentials-json";
	/// Google Cloud Storage token
	pub const GCS_TOKEN: &str = "gcs.oauth2.token";
	/// Option to skip signing requests (e.g. for public buckets/folders).
	pub const GCS_ALLOW_ANONYMOUS: &str = "gcs.allow-anonymous";
	/// Option to skip loading the credential from GCE metadata server (typically used in conjunction with `GCS_ALLOW_ANONYMOUS`).
	pub const GCS_DISABLE_VM_METADATA: &str = "gcs.disable-vm-metadata";
	/// Option to skip loading configuration from config file and the env.
	pub const GCS_DISABLE_CONFIG_LOAD: &str = "gcs.disable-config-load";

[Xuanwo]

Sorry I missed a "can" here 🫣

[dentiny]

Sorry I missed a "can" here 🫣

Cool, thanks again for the quick and helpful answer! I will check s3 sdk then.

I will check how hard it is to implement.