Skip to content

[OCaml] Add (cross-)compiler shards #11146

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 16 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .ci/Manifest.toml
Original file line number Diff line number Diff line change
Expand Up @@ -37,16 +37,16 @@ uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

[[deps.BinaryBuilder]]
deps = ["ArgParse", "BinaryBuilderBase", "Binutils_jll", "Dates", "Downloads", "GitHub", "HTTP", "JLD2", "JSON", "LibGit2", "Libdl", "Logging", "LoggingExtras", "ObjectFile", "OutputCollectors", "Patchelf_jll", "Pkg", "PkgLicenses", "REPL", "Random", "Registrator", "RegistryTools", "SHA", "Scratch", "Sockets", "TOML", "UUIDs", "ghr_jll"]
git-tree-sha1 = "32a89265ffd77083daf729618d4279242ee44020"
repo-rev = "master"
repo-url = "https://github.com/JuliaPackaging/BinaryBuilder.jl"
git-tree-sha1 = "c844af8893f1597a57e2d28d58082262313db77d"
repo-rev = "ct/ocaml-support"
repo-url = "https://github.com/JuliaPackaging/BinaryBuilder.jl.git"
uuid = "12aac903-9f7c-5d81-afc2-d9565ea332ae"
version = "0.6.4"

[[deps.BinaryBuilderBase]]
deps = ["Bzip2_jll", "CodecZlib", "Downloads", "Gzip_jll", "HistoricalStdlibVersions", "InteractiveUtils", "JLLWrappers", "JSON", "LibGit2", "LibGit2_jll", "Libdl", "Logging", "OrderedCollections", "OutputCollectors", "Pkg", "Printf", "ProgressMeter", "REPL", "Random", "SHA", "Scratch", "SimpleBufferStream", "TOML", "Tar", "Tar_jll", "UUIDs", "XZ_jll", "Zstd_jll", "p7zip_jll", "pigz_jll", "unzip_jll"]
git-tree-sha1 = "85259632e317f61245d8240d3ee9b08112f52903"
repo-rev = "master"
git-tree-sha1 = "91ed234470884b2dd655c0a9dd53f13d16ef3cae"
repo-rev = "ct/ocaml-runner"
repo-url = "https://github.com/JuliaPackaging/BinaryBuilderBase.jl.git"
uuid = "7f725544-6523-48cd-82d1-3fa08ff4056e"
version = "1.37.0"
Expand Down
156 changes: 156 additions & 0 deletions 0_RootFS/OCaml/build_tarballs.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
### Instructions for adding a new version of the OCaml toolchain
#
# * update the `version` variable and `sources`
# * To deploy the shard and automatically update your BinaryBuilderBase's
# `Artifacts.toml`, use the `--deploy` flag to the `build_tarballs.jl` script.
# You can build & deploy by running:
#
# julia build_tarballs.jl --debug --verbose --deploy TARGET
#

using BinaryBuilderBase, BinaryBuilder, Pkg.Artifacts

include("../common.jl")

name = "OCamlBase"
version = v"5.4.0"

sources = [
GitSource("https://github.com/ocaml/ocaml.git",
"ff1ab416a5503c8bde9fa3fae5f2bb21c7ddc81e"), # 5.4.0~alpha1
GitSource("https://github.com/ocaml/dune",
"76c0c3941798f81dcc13a305d7abb120c191f5fa"), # 3.19.1
GitSource("https://github.com/ocaml/ocamlbuild",
"131ba63a1b96d00f3986c8187677c8af61d20a08"), # 0.16.1
GitSource("https://github.com/ocaml/opam",
"e13109411952d4f723a165c2a24b8c03c4945041"), # 2.3.0
DirectorySource("./bundled"),
]

# Check if deploy flag is set
deploy = "--deploy" in ARGS

# These are the targets we support right now:
# x86_64-w64-mingw32
# x86_64-apple-darwin14
# aarch64-apple-darwin20
# x86_64-linux-gnu
# x86_64-linux-musl
# aarch64-linux-gnu
# aarch64-linux-musl
# riscv64-linux-gnu
# riscv64-linux-musl
# powerpc64le-linux-gnu
#
# Not supported:
# i686: OCaml 5.0 dropped support for 32-bit platforms
# freebsd: `POSIX threads are required but not supported on this platform`

# The first thing we're going to do is to install Rust for all targets into a single prefix
script = raw"""
cd ${WORKSPACE}/srcdir/ocaml
git submodule update --init

# Apply patches
for f in ${WORKSPACE}/srcdir/patches/*.patch; do
atomic_patch -p1 ${f}
done

# unset compiler env vars so that configure can detect them properly
unset CC CXX LD STRIP AS

if [[ "${target}" == "${MACHTYPE}" ]]; then
# Build a native compiler in $prefix
./configure --prefix=${prefix}
make -j${nproc}
make install
else
# Build a native compiler in $host_prefix (which takes PATH preference over $prefix)
./configure --prefix=${host_prefix} --build=${MACHTYPE} --host=${MACHTYPE}
make -j${nproc}
make install
git clean -fxd

# Build a cross-compiler in $prefix
./configure --prefix=${prefix} --build=${MACHTYPE} --host=${MACHTYPE} --target=${target}
make crossopt -j${nproc}
make installcross
if [[ "${target}" == *-mingw* ]]; then
# the OCaml configure script detects the executable extension by looking at the
# target compiler. for host utilities, we don't want this symlink, so remove it.
for bin in ${bindir}/*.exe; do
# (links to) target binaries should retain their extension
if file -L $bin | grep 'PE32' >/dev/null; then
continue
fi

# if this is a symlink, update both the name of the link and the target
if [[ -L $bin ]]; then
target=$(readlink $bin)
rm $bin
ln -s $(basename ${target} .exe) ${bindir}/$(basename ${bin} .exe)

# if this is a file, simply rename it
elif [[ -f $bin ]]; then
mv $bin ${bindir}/$(basename ${bin} .exe)
fi
done
fi
fi

# Fix shebang of ocamlrun scripts to not hardcode a path of the build environment
for bin in $(file ${bindir}/* | grep "a \S*/ocamlrun script" | cut -d: -f1); do
abspath=$(file ${bin} | grep -oh "a \S*/ocamlrun script" | cut -d' ' -f2)
sed -i "s?${abspath}?/usr/bin/env ocamlrun?" "${bin}"
done

# Dune
cd ${WORKSPACE}/srcdir/dune
./configure --prefix $prefix
make release
make install

# OCamlbuild
cd ${WORKSPACE}/srcdir/ocamlbuild
make configure OCAMLBUILD_PREFIX=$prefix OCAMLBUILD_BINDIR=$bindir OCAMLBUILD_LIBDIR=$prefix/lib
# XXX: can't use $libdir because on Windows it aliases with $bindir
# while ocamlbuild wants to put files in $libdir/ocamlbuild
make -j${nproc}
make install

# Opam
cd ${WORKSPACE}/srcdir/opam
./configure --prefix $prefix --host=${MACHTYPE} --with-vendored-deps
make -j${nproc}
make install
"""

platforms = Platform[ host_platform ]
products = Product[
# build with no products since all of our products are for the host, not the target

# ExecutableProduct("ocamlopt.opt", :ocamlopt),
# ExecutableProduct("ocamlc.opt", :ocamlc),
# ExecutableProduct("ocamlrun", :ocamlrun),
]
dependencies = Dependency[]

name = "OCaml"
compiler_target = try
parse(Platform, ARGS[end])
catch
error("This is not a typical build_tarballs.jl! Must provide exactly one platform as the last argument!")
end
Comment on lines +139 to +143
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opted to go the GCC route and use the compiler target as the "target" for the build
(even though the toolchain we use during the build is the host toolchain for the most part, we do need both)

deleteat!(ARGS, length(ARGS))

# Build the tarballs
ndARGS, deploy_target = find_deploy_arg(ARGS)
build_info = build_tarballs(ndARGS, name, version, sources, script, Platform[compiler_target], products, dependencies;
skip_audit=true, julia_compat="1.6", preferred_gcc_version=v"6")

build_info = Dict(host_platform => first(values(build_info)))

# Upload the artifacts (if requested)
if deploy_target !== nothing
upload_and_insert_shards(deploy_target, name, version, build_info; target=compiler_target)
end
Loading