Unsupported Manifest Media in AWS Sagemaker

[SANGAMI-1410]

When I create an image using Docker Desktop, it saves in OCI v1+json format. After creating the image, I successfully push it to Amazon Elastic Container Registry (ECR) on AWS. However, when attempting to deploy the image from ECR to Amazon SageMaker, I encounter an error indicating that SageMaker requires the Docker v2+json format, which my image does not meet. Is there a way to convert the image format to Docker v2+json without rebuilding it from scratch?

error.pdf

[thompson-shaun]

Assuming you're building with a newer version of docker and buildx is being used the default media type should be OCI. The current descriptor could be exported to a file and rewritten.

As a (potentially easier) alternative you could "rebuild" the image where the FROM is your original image with the other media type. This should avoid a "from scratch" rebuild, but could involve a pull operation.

docker buildx build -t <repo>:<tag> --output=type=image,push=true,oci-mediatypes=false --provenance=false -<<< 'FROM <image>'

Example

# build image with new manifest media type
docker buildx build -t myhuborg/alpine:v2-json --output=type=image,push=true,oci-mediatypes=false --provenance=false -<<< 'FROM myhuborg/alpine:v1-oci'
# verify with docker manifest inspect
> docker manifest inspect myhuborg/alpine:v2-json
{
	"schemaVersion": 2,
	"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
	"config": {
		"mediaType": "application/vnd.docker.container.image.v1+json",
		"digest": "sha256:ea97710e8da83b616548d60d82040afdd508cf3427c6c27663ccec96388f8d53",
		"size": 822
	},
	"layers": [
		{
			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
			"digest": "sha256:bca4290a96390d7a6fc6f2f9929370d06f8dfcacba591c76e3d5c5044e7f420c",
			"size": 3347715
		},
		{
			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
			"digest": "sha256:58fa1a140fea2f039e45f3eda4c85cf6c6e992e85b1d68a34f0f549bd52445cb",
			"size": 2597813
		}
	]
}

# original
> docker manifest inspect myhuborg/alpine:v1-oci
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.oci.image.index.v1+json",
   "manifests": [
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 672,
         "digest": "sha256:58a9640fbdd75cb42dd505e7e2ea984ff995b6ec491b25e4976f93968ed8a360",
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 566,
         "digest": "sha256:c6eb79c18c4237a1daf17a84c830ad09c015a211d1ba05ad6b016939415b7ce7",
         "platform": {
            "architecture": "unknown",
            "os": "unknown"
         }
      }
   ]
}

Thanks to @dvdksn for the tip

[thaJeztah]

FWIW the 'v1' format in this case is a multi-platform image (OCI image index v1); possibly just disabling the provenance already would do the job.

cc @austinvazquez wondering if multi-platform support is on the roadmap for SageMaker

[eric-grudzien-aws]

Hi, just adding in some data points as I experienced this same issue, and final able to resolve. The issue was my build and push created a ECR docker image that was an Artifact type of "Image Index". Therefore, I received this error from SageMaker:

ClientError: An error occurred (ValidationException) when calling the CreateModel operation: Unsupported manifest media type application/vnd.oci.image.index.v1+json for image [xxxxxxx.dkr.ecr.us-west-2.amazonaws.com/custom_docker_container:v1.0.0](http://xxxxxxx.dkr.ecr.us-west-2.amazonaws.com/custom_docker_container:v1.0.0). Ensure that valid manifest media type is used for specified image.

The solution was to create an image that was of Artifact type "Image", in the manifest is would be media type: application/vnd.oci.image.manifest.v1+json

Steps I tried to build an image unsuccessfully:

`
docker build -t ${algorithm_name} .

and

docker image build --platform=linux/amd64 -t ${algorithm_name} .

and

docker image build --platform=linux/amd64 --provenance=false --output oci-mediatypes=false -t ${algorithm_name} .
`

The resolution and successful build was:
docker build --platform=linux/amd64 --provenance=false --output oci-mediatypes=false,type=image,push=true -t ${algorithm_name} .

My error occurred when building from my machine, where a colleague did not have the same problem (Windows).

My Device: Mac, M1
Docker Desktop, Docker version 27.3.1, build ce12230

Hope this helps anyone else who experiences this.

[thompson-shaun]

Looks like we've got 2 data points suggesting the --oci-mediatypes=false and --provenance=false 😄

docker build ... --output=type=image,push=true,oci-mediatypes=false --provenance=false

[thaJeztah]

Yeah, so looks like it only supports legacy, single-platform images 😬