REST catalog with S3T

[gsoundar]

I'm trying to use the REST catalog (https://github.com/apache/iceberg-rust/blob/main/crates/catalog/rest/src/catalog.rs) with S3Tables similar to what is described in https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-tables-integrating-open-source.html.

The rough snippet of code is:

    let mut catalog_props = HashMap::new();
    catalog_props.insert("warehouse".to_string(), TABLE_BUCKET_ARN.to_string());
    catalog_props.insert("rest.sigv4-enabled".to_string(), "true".to_string());
    catalog_props.insert("rest.signing-name".to_string(), "s3tables".to_string());
    catalog_props.insert("rest.signing-region".to_string(), region.to_string());
    let catalog_config = RestCatalogConfig::builder()
        .uri(TABLE_BUCKET_CATALOG.to_string())
        .props(catalog_props)
        .build();
    let catalog = RestCatalog::new(catalog_config);

and I get the error

Context:
   status: 403 Forbidden
   headers: {"date": "Wed, 23 Apr 2025 19:24:10 GMT", "content-type": "application/json", "content-length": "42", "connection": "keep-alive", "x-amzn-requestid": "XXXXX", "access-control-allow-origin": "*", "x-amzn-errortype": "MissingAuthenticationTokenException", "x-amz-apigw-id": "XXXXX"}
   json: {"message":"Missing Authentication Token"}

I think it's because we don't have Sigv4 suppport in the code. Anyone confirm this is the RCA ?
Also does anyone know how Sigv4 support can be plumbed in, maybe through the HttpClient perhaps ?

Thanks!

[liurenjie1024]

Hi, there is a S3TableCatalog, could you have a try?

[liurenjie1024]

Oh, I see. The s3table catalog support in this repo has not bee released yet, and will be included in 0.5.0 release, see https://github.com/apache/iceberg-rust/blob/45312032665f21425215fe3cf26f53ca9c552a6b/crates/catalog/s3tables/Cargo.toml

[dentiny]

I see... I'm sticking to the latest release version, and I guess for fast-developing repos I should switch to dev branch.

[dentiny]

My concern for the S3 table catalog is it doesn't implement update_table:

iceberg-rust/crates/catalog/s3tables/src/catalog.rs

Lines 476 to 485 in 5677d44

	/// Updates an existing table within the s3tables catalog.
	///
	/// This function is still in development and will always return an error.
	async fn update_table(&self, _commit: TableCommit) -> Result<Table> {
	Err(Error::new(
	ErrorKind::FeatureUnsupported,
	"Updating a table is not supported yet",
	))
	}
	}

which is required for Transaction::commit.

iceberg-rust/crates/iceberg/src/transaction/mod.rs

Lines 186 to 196 in 5677d44

	/// Commit transaction.
	pub async fn commit(self, catalog: &dyn Catalog) -> Result<Table> {
	let table_commit = TableCommit::builder()
	.ident(self.base_table.identifier().clone())
	.updates(self.updates)
	.requirements(self.requirements)
	.build();
	catalog.update_table(table_commit).await
	}
	}

If I don't get it wrong, it should take table commit and write new metadata files and update version hint file.

[liurenjie1024]

I'm surprised that S3TableClient didn't provide a table commit api, seems rest catalog is the only way to do it.

[dentiny]

Yes, the official document provides pyiceberg access example for rest catalog:
https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-tables-integrating-open-source.html