Skip to content

Documentation improvements #1151

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
May 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,6 @@
.idea
.vscode

# Tree service
internal/neofs/services/tree/

# Vendoring
vendor

Expand Down
4 changes: 0 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,6 @@ $ S3_GW_PEERS_0_ADDRESS=grpcs://192.168.130.72:8080 \
neofs-s3-gw
```

Notice that currently S3 gateway can't be used for public networks like mainnet
or testnet because of experimental tree service extension that is required for it.

## Domains

By default, s3-gw enable only `path-style access`.
Expand Down Expand Up @@ -79,7 +76,6 @@ Also, you can configure domains using `.env` variables or `yaml` file.

- [Configuration](./docs/configuration.md)
- [NeoFS S3 AuthMate](./docs/authmate.md)
- [NeoFS Tree service](./docs/tree_service.md)
- [AWS CLI basic usage](./docs/aws_cli.md)
- [AWS S3 API compatibility](./docs/aws_s3_compat.md)
- [AWS S3 Compatibility test results](./docs/s3_test_results.md)
Expand Down
4 changes: 2 additions & 2 deletions api/data/tree.go → api/data/meta.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ const (
UnversionedObjectVersionID = "null"
)

// NodeVersion represent node from tree service.
// NodeVersion represent basic object metadata.
type NodeVersion struct {
OID oid.ID
Timestamp uint64
Expand Down Expand Up @@ -93,7 +93,7 @@ func SortPartInfo(a, b *PartInfo) int {
return cmp.Compare(a.Number, b.Number)
}

// LockInfo is lock information to create appropriate tree node.
// LockInfo is lock information for a particular object.
type LockInfo struct {
legalHoldOID oid.ID
setLegalHold bool
Expand Down
8 changes: 4 additions & 4 deletions api/layer/layer.go
Original file line number Diff line number Diff line change
Expand Up @@ -270,7 +270,7 @@ type (

// Compound methods for optimizations

// GetObjectTaggingAndLock unifies GetObjectTagging and GetLock methods in single tree service invocation.
// GetObjectTaggingAndLock unifies GetObjectTagging and GetLock methods in a single search invocation.
GetObjectTaggingAndLock(ctx context.Context, p *ObjectVersion) (map[string]string, *data.LockInfo, error)
}
)
Expand All @@ -288,10 +288,10 @@ var (
)

var (
// ErrNodeNotFound is returned from Tree service in case of not found error.
// ErrNodeNotFound is returned in case of not found error.
ErrNodeNotFound = errors.New("not found")

// ErrNodeAccessDenied is returned from Tree service in case of access denied error.
// ErrNodeAccessDenied is returned in case of access denied error.
ErrNodeAccessDenied = errors.New("access denied")

// ErrPartListIsEmpty is returned if no parts available for the upload.
Expand Down Expand Up @@ -564,7 +564,7 @@ func (n *layer) GetObjectInfo(ctx context.Context, p *HeadObjectParams) (*data.O
return extendedObjectInfo.ObjectInfo, nil
}

// GetExtendedObjectInfo returns meta information and corresponding info from the tree service about the object.
// GetExtendedObjectInfo returns meta information and corresponding info about the object.
func (n *layer) GetExtendedObjectInfo(ctx context.Context, p *HeadObjectParams) (*data.ExtendedObjectInfo, error) {
var (
id oid.ID
Expand Down
4 changes: 2 additions & 2 deletions api/layer/multipart_upload.go
Original file line number Diff line number Diff line change
Expand Up @@ -459,7 +459,7 @@ func (n *layer) uploadZeroPart(ctx context.Context, multipartInfo *data.Multipar
currentVersion := version.Current()
hashlessHeaderObject.SetVersion(&currentVersion)

// encoding hash.Hash state to save it in tree service.
// encoding hash.Hash state to save it in the object metadata.
// the required interface is guaranteed according to the docs, so just cast without checks.
binaryMarshaler := multipartHash.(encoding.BinaryMarshaler)
stateBytes, err := binaryMarshaler.MarshalBinary()
Expand Down Expand Up @@ -1985,7 +1985,7 @@ func (n *layer) uploadPartAsSlot(ctx context.Context, params uploadPartAsSlotPar
mpHashBytes, homoHashBytes []byte
)

// encoding hash.Hash state to save it in tree service.
// encoding hash.Hash state to save it the object metadata.
// the required interface is guaranteed according to the docs, so just cast without checks.
binaryMarshaler := multipartHash.(encoding.BinaryMarshaler)
mpHashBytes, err := binaryMarshaler.MarshalBinary()
Expand Down
31 changes: 25 additions & 6 deletions docs/aws_s3_compat.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ Reference:

## ACL

For now there are some limitations:
There are some limitations:
* [Bucket policy](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-policies.html) supports only one `Principal` per `Statement`.
Principal must be `"AWS": "*"` or `"*"` (to refer all users) or `"CanonicalUser": "NiskPF9pfRMzg7V7PeB4d6ogLzu74a1L2Q"` (base58 encoded address of desired user).
```json
Expand Down Expand Up @@ -114,6 +114,9 @@ Principal must be `"AWS": "*"` or `"*"` (to refer all users) or `"CanonicalUser"
"Grants": []
}
```
Given that ACLs are disabled by default now and users are expected to use
alternative mechanisms to manage access improving ACL support is not
a priority.

| | Method | Comments |
|----|--------------|-----------------|
Expand All @@ -137,8 +140,6 @@ For now there are some limitations:

## Multipart

Should be supported soon.

| | Method | Comments |
|----|-------------------------|----------|
| 🟢 | AbortMultipartUpload | |
Expand All @@ -164,7 +165,7 @@ See also `GetObject` and other method parameters.
| | Method | Comments |
|----|--------------------|--------------------------|
| 🟢 | ListObjectVersions | ListBucketObjectVersions |
| 🔵 | RestoreObject | |
| 🔵 | RestoreObject | Related to lifecycles and tiering. |

## Bucket

Expand All @@ -176,7 +177,7 @@ See also `GetObject` and other method parameters.
| 🟢 | HeadBucket | |
| 🟢 | ListBuckets | |
| 🔵 | PutPublicAccessBlock | |
| 🔵 | GetPublicAccessBlock | |
| 🔵 | GetPublicAccessBlock | Related to ACL management, not a priority. |

* `CreateBucket` method allows you to select a placement policy using the `LocationConstraint` parameter in the AWS CLI. The policy name should be passed as a value.
* Policies mapping can be defined via:
Expand All @@ -203,6 +204,8 @@ See [Ownership](./aws_s3_compat.md#ownership-controls) section for details.

## Analytics

Requires a separate entity to generate/upload data daily.

| | Method | Comments |
|----|------------------------------------|----------|
| 🔵 | DeleteBucketAnalyticsConfiguration | |
Expand All @@ -220,6 +223,8 @@ See [Ownership](./aws_s3_compat.md#ownership-controls) section for details.

## Encryption

Methods below are related to AWS SSE-S3 and SSE-KMS encryption. S3 gateway supports SSE-C only for now.

| | Method | Comments |
|----|------------------------|----------|
| 🔵 | DeleteBucketEncryption | |
Expand All @@ -228,6 +233,8 @@ See [Ownership](./aws_s3_compat.md#ownership-controls) section for details.

## Inventory

Requires a separate entity to generate/upload data regularly.

| | Method | Comments |
|----|------------------------------------|----------|
| 🔵 | DeleteBucketInventoryConfiguration | |
Expand All @@ -237,6 +244,9 @@ See [Ownership](./aws_s3_compat.md#ownership-controls) section for details.

## Lifecycle

In general requires a separate entity to manage uploaded objects. Without it
can be only partially supported.

| | Method | Comments |
|----|---------------------------------|------------------------------------------------------|
| 🔵 | DeleteBucketLifecycle | |
Expand All @@ -247,13 +257,17 @@ See [Ownership](./aws_s3_compat.md#ownership-controls) section for details.

## Logging

Doesn't make much sense in non-AWS case.

| | Method | Comments |
|----|------------------|----------|
| 🔵 | GetBucketLogging | |
| 🔵 | PutBucketLogging | |

## Metrics

Doesn't make much sense in non-AWS case.

| | Method | Comments |
|----|----------------------------------|----------|
| 🔵 | DeleteBucketMetricsConfiguration | |
Expand All @@ -263,6 +277,9 @@ See [Ownership](./aws_s3_compat.md#ownership-controls) section for details.

## Notifications

Can't be exactly the same as in AWS. AMQP/NATS/etc can be supported, but it's
not exactly the gateway job. It also requires metadata synchronization.

| | Method | Comments |
|----|------------------------------------|---------------|
| 🔵 | GetBucketNotification | |
Expand Down Expand Up @@ -309,7 +326,7 @@ Pay attention to the fact that object owner in NeoFS is bucket owner in any case
| 🔵 | GetBucketReplication | |
| 🟢 | PostPolicyBucket | Upload file using POST form |
| 🟡 | PutBucketPolicy | See ACL limitations |
| 🔵 | PutBucketReplication | |
| 🔵 | PutBucketReplication | Hardly applicable to NeoFS |

You may set requiring the `bucket-owner-full-control` canned ACL for Amazon S3 PUT operations ([bucket owner preferred](https://docs.aws.amazon.com/AmazonS3/latest/userguide/ensure-object-ownership.html#ensure-object-ownership-bucket-policy)):
```shell
Expand Down Expand Up @@ -362,6 +379,8 @@ policy.json:

## Tiering

Tightly related to lifecycle support.

| | Method | Comments |
|----|---------------------------------------------|----------|
| 🔵 | DeleteBucketIntelligentTieringConfiguration | |
Expand Down
12 changes: 0 additions & 12 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,6 @@ There are some custom types used for brevity:
| `placement_policy` | [Placement policy configuration](#placement_policy-section) |
| `server` | [Server configuration](#server-section) |
| `logger` | [Logger configuration](#logger-section) |
| `tree` | [Tree configuration](#tree-section) |
| `cache` | [Cache configuration](#cache-section) |
| `nats` | [NATS configuration](#nats-section) |
| `cors` | [CORS configuration](#cors-section) |
Expand Down Expand Up @@ -327,17 +326,6 @@ logger:
| `encoding` | `string` | | `console` | Encoding type.<br/>Possible values: `console`, `json`. |
| `timestamp` | `bool` | | `false` | Flag to enable timestamps. If the parameter is not set, they will be enabled when you run with tty. |

### `tree` section

```yaml
tree:
service: s01.neofs.devenv:8080
```

| Parameter | Type | Default value | Description |
|-----------|----------|---------------|------------------------------------------------------------------------------------------------------------|
| `service` | `string` | | Endpoint of the tree service. Must be provided. Can be one of the node address (from the `peers` section). |

### `cache` section

```yaml
Expand Down
1 change: 1 addition & 0 deletions docs/labels.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ Currently only these ones are used, but the list can be extended in future:

- auth-mate
- tree-service
(outdated, was used for issues related to the tree service)
18 changes: 0 additions & 18 deletions docs/tree_service.md

This file was deleted.

Loading