Skip to content

add support for the new --estargz-gzip-helper option in stargz-snapshotter #4543

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Oct 21, 2025
+83 −50
Merged

add support for the new --estargz-gzip-helper option in stargz-snapshotter #4543

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e584314
bump github.com/containerd/stargz-snapshotter from v0.17.0 to v0.18.0
Oct 18, 2025
35cc34f
add support for the new --estargz-gzip-helper option in stargz-snapsh…
Oct 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions cmd/nerdctl/image/image_convert.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ func convertCommand() *cobra.Command {
cmd.Flags().Int("estargz-min-chunk-size", 0, "The minimal number of bytes of data must be written in one gzip stream. (requires stargz-snapshotter >= v0.13.0)")
cmd.Flags().Bool("estargz-external-toc", false, "Separate TOC JSON into another image (called \"TOC image\"). The name of TOC image is the original + \"-esgztoc\" suffix. Both eStargz and the TOC image should be pushed to the same registry. (requires stargz-snapshotter >= v0.13.0) (EXPERIMENTAL)")
cmd.Flags().Bool("estargz-keep-diff-id", false, "Convert to esgz without changing diffID (cannot be used in conjunction with '--estargz-record-in'. must be specified with '--estargz-external-toc')")
cmd.Flags().String("estargz-gzip-helper", "", "Helper command for decompressing layers compressed with gzip. Options: pigz, igzip, or gzip.")
// #endregion

// #region zstd flags
Expand Down Expand Up @@ -149,6 +150,10 @@ func convertOptions(cmd *cobra.Command) (types.ImageConvertOptions, error) {
if err != nil {
return types.ImageConvertOptions{}, err
}
estargzGzipHelper, err := cmd.Flags().GetString("estargz-gzip-helper")
if err != nil {
return types.ImageConvertOptions{}, err
}
// #endregion

// #region zstd flags
Expand Down Expand Up @@ -275,6 +280,7 @@ func convertOptions(cmd *cobra.Command) (types.ImageConvertOptions, error) {
EstargzMinChunkSize: estargzMinChunkSize,
EstargzExternalToc: estargzExternalTOC,
EstargzKeepDiffID: estargzKeepDiffID,
EstargzGzipHelper: estargzGzipHelper,
},
ZstdOptions: types.ZstdOptions{
Zstd: zstd,
Expand Down
17 changes: 15 additions & 2 deletions docs/stargz.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,20 @@ Stargz Snapshotter is not needed for building stargz images.

## Tips for image conversion

### Tips 1: Creating smaller eStargz images
### Tips 1: Using gzip helper to speed up image conversion

When converting a traditional overlayfs image encoded as tar.gz to an estargz format image, nerdctl supports specifying an additional command‑line decompression tool to speed up the conversion process. You can set `--estargz-gzip-helper` to choose different CLI gzip tools. Even using the gzip command corresponding to the Go gzip library can achieve approximately 32% speed improvement. For more details, see: [Using decompression commands to improve the layer decompression speed of gzip-formatted images](https://github.com/containerd/stargz-snapshotter/pull/2117). Currently, `--estargz-gzip-helper` supports `pigz`, `igzip`, and `gzip`. The recommended order is `pigz` > `igzip` > `gzip`.

```console
# nerdctl image convert --oci --estargz --estargz-gzip-helper pigz ghcr.io/stargz-containers/ubuntu:22.04 ghcr.io/stargz-containers/ubuntu:22.04-esgz
sha256:aa6543b9885867b8b485925b6ec69d8e018e8fce40835ea6359cbb573683a014
# nerdctl image ls
REPOSITORY TAG IMAGE ID CREATED PLATFORM SIZE BLOB SIZE
ghcr.io/stargz-containers/ubuntu 22.04-esgz aa6543b98858 About a minute ago linux/amd64 0B 32.43MB
ghcr.io/stargz-containers/ubuntu 22.04 20fa2d7bb4de 2 minutes ago linux/amd64 87.47MB 30.43MB
```

### Tips 2: Creating smaller eStargz images

`nerdctl image convert` allows the following flags for optionally creating a smaller eStargz image.
The result image requires stargz-snapshotter >= v0.13.0 for lazy pulling.
Expand Down Expand Up @@ -167,7 +180,7 @@ sha256:7f5cbd8cc787c8d628630756bcc7240e6c96b876c2882e6fc980a8b60cdfa274
sha256:7f5cbd8cc787c8d628630756bcc7240e6c96b876c2882e6fc980a8b60cdfa274
```

### Tips 2: Using zstd instead of gzip (a.k.a. zstd:chunked)
### Tips 3: Using zstd instead of gzip (a.k.a. zstd:chunked)

You can use zstd compression with lazy pulling support (a.k.a zstd:chunked) instead of gzip.

Expand Down
27 changes: 14 additions & 13 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@ require (
github.com/containerd/nerdctl/mod/tigron v0.0.0
github.com/containerd/nydus-snapshotter v0.15.5 //gomodjail:unconfined
github.com/containerd/platforms v1.0.0-rc.1 //gomodjail:unconfined
github.com/containerd/stargz-snapshotter v0.17.0 //gomodjail:unconfined
github.com/containerd/stargz-snapshotter/estargz v0.17.0 //gomodjail:unconfined
github.com/containerd/stargz-snapshotter/ipfs v0.17.0 //gomodjail:unconfined
github.com/containerd/stargz-snapshotter v0.18.0 //gomodjail:unconfined
github.com/containerd/stargz-snapshotter/estargz v0.18.0 //gomodjail:unconfined
github.com/containerd/stargz-snapshotter/ipfs v0.18.0 //gomodjail:unconfined
Comment on lines +25 to +27
Copy link
Member

@ktock ktock Oct 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you separate commits to the bumping up and the addition of the new flag?

Copy link
Author

@escapefreeg escapefreeg Oct 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

github.com/containerd/typeurl/v2 v2.2.3
github.com/containernetworking/cni v1.3.0 //gomodjail:unconfined
github.com/containernetworking/plugins v1.8.0 //gomodjail:unconfined
Expand Down Expand Up @@ -85,7 +85,7 @@ require (
github.com/djherbis/times v1.6.0 // indirect
github.com/docker/docker-credential-helpers v0.8.2 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-jose/go-jose/v4 v4.0.5 // indirect
github.com/go-jose/go-jose/v4 v4.1.2 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/godbus/dbus/v5 v5.1.0 // indirect
Expand All @@ -108,7 +108,7 @@ require (
github.com/mr-tron/base58 v1.2.0 // indirect
github.com/multiformats/go-base32 v0.1.0 // indirect
github.com/multiformats/go-base36 v0.2.0 // indirect
github.com/multiformats/go-multiaddr v0.16.0 // indirect
github.com/multiformats/go-multiaddr v0.16.1 // indirect
github.com/multiformats/go-multibase v0.2.0 // indirect
github.com/multiformats/go-multihash v0.2.3 // indirect
github.com/multiformats/go-varint v0.0.7 // indirect
Expand All @@ -127,24 +127,25 @@ require (
//gomodjail:unconfined
github.com/syndtr/gocapability v0.0.0-20200815063812-42c35b437635 // indirect
github.com/tinylib/msgp v1.3.0 // indirect
github.com/vbatts/tar-split v0.12.1 // indirect
github.com/vbatts/tar-split v0.12.2 // indirect
github.com/xhit/go-str2duration/v2 v2.1.0 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0 // indirect
go.opentelemetry.io/otel v1.35.0 // indirect
go.opentelemetry.io/otel/metric v1.35.0 // indirect
go.opentelemetry.io/otel/trace v1.35.0 // indirect
go.opentelemetry.io/otel v1.37.0 // indirect
go.opentelemetry.io/otel/metric v1.37.0 // indirect
go.opentelemetry.io/otel/trace v1.37.0 // indirect
go.yaml.in/yaml/v2 v2.4.2 // indirect
golang.org/x/exp v0.0.0-20241108190413-2d47ceb2692f // indirect
golang.org/x/mod v0.28.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250324211829-b45e905df463 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250804133106-a7a43d27e69b // indirect
//gomodjail:unconfined
google.golang.org/grpc v1.73.0 // indirect
google.golang.org/grpc v1.76.0 // indirect
//gomodjail:unconfined
google.golang.org/protobuf v1.36.7 // indirect
google.golang.org/protobuf v1.36.8 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
lukechampine.com/blake3 v1.3.0 // indirect
sigs.k8s.io/yaml v1.4.0 // indirect
sigs.k8s.io/yaml v1.6.0 // indirect
tags.cncf.io/container-device-interface/specs-go v1.0.0 // indirect
)

Expand Down
Loading
Loading