Dockerfile and vcpkg

[aavbsouza]

Hello everyone. Is there any recommendation on using together vcpkg and container images. Usually I use multi stage builds
with a "dev image" that I use to develop using VSCode's remote container extension. On this "dev image" during the build I install the project dependencies, defined by a manifest, with vcpkg install. The artifacts of this command are kept on the home of the container user (for instance root)).

Would be possible to change the path where the artifacts are stored? Instead of always using ~/.cache/vcpkg.

Is there any pages on the documentation detailing the heuristics used by the caching algorithm ?

thanks

[Osyotr]

Would be possible to change the path where the artifacts are stored?

s there any pages on the documentation detailing the heuristics used by the caching algorithm ?

Yes. https://learn.microsoft.com/en-us/vcpkg/users/binarycaching

[aavbsouza]

Hello @Osyotr, thanks for the reply. I have already seen this page. I was looking more into best_practices (Is there a problem to use ~/.cache folder)
How to discover what change created a hash change that invalidate the cache

[Osyotr]

best_practices

Personally I store binary cache generated inside docker on a http server. I believe this is also how vcpkg CI works.
You can use mounted folder for this purpose.

How to discover what change created a hash change that invalidate the cache

It's written on the same page: https://learn.microsoft.com/en-us/vcpkg/users/binarycaching#abi-hash
Compare vcpkg_abi_info.txt between two builds of a port.

[aavbsouza]

thanks @Osyotr I will look into the vcpkg_abi_info.txt, that one I had missed

[aavbsouza]

I am currently using the container as storage medium and looking into to use a S3 bucket

[RealLitb]

I do it the exact same way. Currently I do that like this:

# in a different container based on the above, build vcpkg
FROM dev as thirdparties

RUN git clone https://github.com/microsoft/vcpkg $VCPKG_ROOT && cd $VCPKG_ROOT ...

# Build all package, so that /vcpkg-bincache is prepopulated when the repo build starts!
WORKDIR /tmp/vcpkg-installed
COPY vcpkg.json vcpkg.json
RUN $VCPKG_ROOT/vcpkg install 

FROM dev
COPY --from=thirdparties --chmod=777 ${VCPKG_DEFAULT_BINARY_CACHE}/ ${VCPKG_DEFAULT_BINARY_CACHE}/

This makes it possible to keep using manifest mode, but get fast build times because of the binary cache being in the container.

However I found that it has some drawbacks

1. you can't get hold of the error logs when the build fails. So when adding new packages that you don't know whether they build, it may be worth first executing the commands inside a container derived from "dev" or "thirdparties" (to get the cache in it too) and not inside of the Dockerfile. Once yo know they work, you can integrate them in the Dockerfile.

2. Regarding the binary cache: If you change something when you build the actual repo, like adding a PATH element (VSCode devcontainers does that!), vcpkg builds all the packages again and won't use the binary cache.

For my windows machines, I will use "export" and generate a SDK.

[aavbsouza]

Hello @RealLitb thanks for your very informative answer. Actually I am trying to discover a error. In some hosts (base os Rhel 8.8) when using with vscode or even with podman directly I get some strange errors (like no ninja build, but it is present on the image) and all the cache is invalidated. However in others hosts with base Os being Ubuntu I am able to use from vscode and the cache work as expected. Thanks

[swelborn]

I am seeing the same thing with podman, @aavbsouza