biocontainers - linux-aarch64 and/or multi-arch

[dslarm]

We're at the point where 98 of the top 100 downloaded packages in bioconda are ported or generic - but basically, they and their dependencies install on linux-aarch64. Or, of the 50 top starred nf-core pipelines, about 30 now have full linux-aarch64 support for all dependencies.

However, biocontainers appears to be not as healthy, so I'm opening this to take stock of where we are and what the direction needs to be for this project to support it healthily.

1. Although linux/arm64/v8 Architecture images #425 says there are now images when a bioconda-recipe has linux-aarch64, this doesn't correlate with what I can find when I search on quay.io.

• take bwa-mem2 for example, available in bioconda since November;
  • the current build for x86 is https://anaconda.org/bioconda/bwa-mem2/2.2.1/download/linux-64/bwa-mem2-2.2.1-he70b90d_8.tar.bz2
  • the current build for aarch64 is https://anaconda.org/bioconda/bwa-mem2/2.2.1/download/linux-aarch64/bwa-mem2-2.2.1-h1e84f2d_8.tar.bz2
  • on quay.io, searching for bwa-mem2 ( https://quay.io/repository/biocontainers/bwa-mem2?tab=tags ) yields only the x86 one:

2. Various things like nf-core pipeline modules will explicitly ask for a specific package - but the specific package IDs are linux-x86_64, rather than multi-arch.

• those are going to all need to select on arch and then the package URL. A multi-arch container may make life easier for that, but at an effort cost for biocontainers.
• what should be done here.
• this is not unique to biocontainers - this is also an issue for galaxy project's singularity containers.

@ChristianKniep
@brainstorm

[mboudet]

Ok, so after looking at the code on the bioconda side, it seems like that while the ARM "conda packages" are currently being built, the derived docker images are not currently being upload to quay.io.

Looking at the CI on the bioconda side, there are these comments:

# For now, do not upload ARM images to biocontainers: --mulled-upload-target biocontainers

Looking at the initial PR for ARM support here, I'm guessing there might still be some issues to build a common manifest... but I'm not sure if anyone is currently working on it. (I'm not working on the bioconda side).

[brainstorm]

Thanks @dslarm for raising this issue and @mboudet for the inspection of the last PR. I'm exploring and writing this as I read its infra code.

Let's take bwa-mem2 as an example and "first principles exploration". Inspecting its manifest via docker manifest inspect quay.io/biocontainers/bwa-mem2:2.2.1--he70b90d_8, there's indeed no platform section:

{
	"schemaVersion": 2,
	"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
	"config": {
		"mediaType": "application/vnd.docker.container.image.v1+json",
		"size": 3449,
		"digest": "sha256:61c95bf409f09950958d03ab162e0aa3257bca21e72cbc0a2f9c15088f7bd532"
	},
	"layers": [
		{
			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
			"size": 4696848,
			"digest": "sha256:0cacab098358fffeef7e18bd537907ae734dcfa12ab45fbcd0e62cc9b37264a8"
		},
		{
			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
			"size": 34,
			"digest": "sha256:bd9ddc54bea929a22b334e73e026d4136e5b73f5cc29942896c72e4ece69b13d"
		},
		{
			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
			"size": 9987026,
			"digest": "sha256:9e3e4dab38b57db2d33b5601ec47aae852d27107185fc344827a313c52869050"
		}
	]
}

An official docker blogpost on multiplatform manifests points out that the "easy" path to have multi-arch manifests is via docker buildx by passing the --platform flag with a comma-separated list of the arches... so doing a few searches for --platform, docker build buildx, etc.. doesn't yield much. That's probably because that manifest building is done differently?

For now, I'm ignoring the complex CI/CD race condition considerations pointed by @daler in bioconda/bioconda-recipes#40550 which can potentially be solved by @martin-g's experiments in the same PR thread.

Timeout for today! I'll continue exploring this tomorrow and try to find the culprit unless somebody points out it earlier.

[mboudet]

Might be worth creating an issue on the bioconda-recipes side to see it the container build is still in the works.