Spec file generator

[tadiwa-aizen]

What changed and why?

Added a new RPM spec file generator (generate_spec.py) that creates distribution-specific .spec files by extracting versions from Cargo.toml/rust-toolchain.toml, scanning git submodules for bundled library declarations, and merging this data with distro-specific templates containing appropriate package names and build commands.

Also added the template file for Amazon Linux 2023 (templates/amzn2023.spec.template) containing AL2023-specific BuildRequires, runtime dependencies, and build commands including manual cargo build steps and rust-toolset requirements.

Usage :

uv run python generate_spec.py amzn2023

Generated amzn2023.spec

---

Please confirm there's no breaking change, or call out any that are made.

No breaking change

Does this change need a changelog entry? Does it require a version change?

No

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the Developer Certificate of Origin (DCO).

[muddyfish]

Does building the srpm file always produce the exact same binary?

[tadiwa-aizen]

Noted, removed, it

[muddyfish]

Why are we doing this if we're importing a pre-built license file?

[tadiwa-aizen]

So the manifest is different from the THIRD_PARTY_LICENSES, which is essentially legal stuff and not much detail on dependancies.

The manifest allows for security scanners to see everything we've bundled. It's like the provides section above but for the entire projects dependancies, and not just the CRT.

As seen here: https://docs.fedoraproject.org/en-US/packaging-guidelines/Rust/#_vendored_dependencies

[muddyfish]

Is this changelog meant to be for this rpm or for Mountpoint itself?

[tadiwa-aizen]

This is the changelog for the desert linux package mainly. We already have one here. For this will mainly track each release into AL, thus i assume its best to track each release from 1.20.0 onwards.

[yerzhan7]

Can we just provide link to our changelog on GitHub here instead? Do we really need to duplicate changelog here?

[muddyfish]

Why skip these tests?

[tadiwa-aizen]

Kj doesnt allow you to mount in the builds run.

[yerzhan7]

Could you please add instructions to always add new CRT libraries to this file?

Maybe as a comment into https://github.com/awslabs/mountpoint-s3/blob/7854f020d9f31efaf0e392367cf844f059c30b47/mountpoint-s3-crt-sys/UPDATING_CRT.md or

mountpoint-s3/mountpoint-s3-crt-sys/build.rs

Lines 14 to 25 in 7854f02

	/// The CRT libraries we use, in order of their dependencies.
	const CRT_LIBRARIES: &[&str] = &[
	"aws-c-common",
	"aws-c-cal",
	"aws-c-io",
	"aws-c-compression",
	"aws-c-http",
	"aws-c-sdkutils",
	"aws-c-auth",
	"aws-checksums",
	"aws-c-s3",
	];

so that we won't forget.

[yerzhan7]

Which version of rust is this? Can we specify exact version to match rust version in

mountpoint-s3/rust-toolchain.toml

Line 2 in 7854f02

channel = "1.88"

?

[tadiwa-aizen]

Noted, I'll leave the
BuildRequires: rust >= 1.88
BuildRequires: cargo >= 1.88

[yerzhan7]

we should match exact version as in our other builds? (if we want same/reproducable builds)

Let's do rust = 1.88 instead?

[yerzhan7]

Do we need to talk about CRT and Rust libraries in the package description? Sounds like just an implementation detail (which could change in the future).

[yerzhan7]

Can we specify that we need fuse v2? E.g. >= 2.9.0?

[yerzhan7]

Did we test/validate that fstab feature works with this new Mountpoint AL2023 RPM? As I understand there are no automatic tests for fstab for this new RPM

[tadiwa-aizen]

Yes, fstab works fine, i've tested it. We could add automated tests the the current AMI test CR if you'd like?

[yerzhan7]

In the future there might be new Amazon Linux version and we might need to release RPM versions for both AL2023 and the new Amazon Linux. So, would we re-use the same spec file for both Amazon linux versions or will we create 2 separate spec files?

I am just thinking that maybe we should rename this file to package/amazon-linux-2023-packaging.spec if we hardcode AL2023 on this line?

[yerzhan7]

I think you can do 1.%{dist} to that automatically expands to 1.amz2023 via macro?

[yerzhan7]

Should we also add BuildArchitectures: x86_64 aarch64?

[yerzhan7]

I think we should specify exact version that we are bundling so that it's easier to track vulnerabilities.

E.g. Provides: bundled(aws-lc) = 1.61.3

[tadiwa-aizen]

This is the spec file created:

%bcond_without check

Name:           mount-s3
Version:        1.21.0
Release:        amzn2023
Summary:        Mountpoint for Amazon S3

License:        Apache-2.0
URL:            https://github.com/awslabs/mountpoint-s3 
Source0:        mountpoint-s3-1.21.0.tar.gz
Source1:        LICENSE
Source2:        NOTICE
Source3:        THIRD_PARTY_LICENSES

BuildRequires:  clang
BuildRequires:  clang-devel
BuildRequires:  rust >= 1.88      
BuildRequires:  cargo >= 1.88    
BuildRequires:  cmake
BuildRequires:  gcc
BuildRequires:  gcc-c++
BuildRequires:  git
BuildRequires:  pkgconfig
BuildRequires:  fuse-devel
BuildRequires:  glibc-devel
BuildRequires:  glibc-headers
BuildRequires:  glibc-static
BuildRequires:  libstdc++-devel
BuildRequires:  nasm
BuildRequires:  make
BuildRequires:  which
BuildRequires:  rust-packaging
BuildRequires:  rust-toolset

ExclusiveArch: x86_64 aarch64

Provides: bundled(aws-c-auth) = 0.9.0
Provides: bundled(aws-c-cal) = 0.9.2
Provides: bundled(aws-c-common) = 0.12.4
Provides: bundled(aws-c-compression) = 0.3.1
Provides: bundled(aws-c-http) = 0.10.3
Provides: bundled(aws-c-io) = 0.21.1
Provides: bundled(aws-c-s3) = 0.8.5
Provides: bundled(aws-c-sdkutils) = 0.2.4
Provides: bundled(aws-checksums) = 0.2.6
Provides: bundled(aws-lc) = 1.53.1
Provides: bundled(s2n-tls) = 1.5.18

Requires:       ca-certificates
Requires:       fuse >= 2.9.0
Requires:       fuse-libs >= 2.9.0

%description
Mountpoint for Amazon S3 is a simple, high-throughput file client for
mounting an Amazon S3 bucket as a local file system. With Mountpoint for Amazon
S3, your applications can access objects stored in Amazon S3 through file
operations like open and read. Mountpoint for Amazon S3 automatically
translates these operations into S3 object API calls, giving your applications
access to the elastic storage and throughput of Amazon S3 through a file
interface.

%build
export CFLAGS="${CFLAGS:-%{optflags}} -O2 -Wno-error=cpp"
export CMAKE_C_FLAGS="$CFLAGS"

export MOUNTPOINT_S3_AWS_RELEASE="amzn2023"

cargo build --release

%cargo_vendor_manifest

%prep
%autosetup -n mountpoint-s3
%cargo_prep -v vendor

%install
mkdir -p %{buildroot}/opt/aws/mountpoint-s3/bin
mkdir -p %{buildroot}/%{_prefix}/sbin
mkdir -p %{buildroot}/%{_bindir}
install -m 755 target/release/mount-s3 %{buildroot}/opt/aws/mountpoint-s3/bin/mount-s3
install -m 644 NOTICE %{buildroot}/opt/aws/mountpoint-s3/
install -m 644 LICENSE %{buildroot}/opt/aws/mountpoint-s3/
install -m 644 THIRD_PARTY_LICENSES %{buildroot}/opt/aws/mountpoint-s3/
install -m 644 cargo-vendor.txt %{buildroot}/opt/aws/mountpoint-s3/
echo "1.21.0" > %{buildroot}/opt/aws/mountpoint-s3/VERSION
ln -sf /opt/aws/mountpoint-s3/bin/mount-s3 %{buildroot}/%{_bindir}/mount-s3
ln -sf /opt/aws/mountpoint-s3/bin/mount-s3 %{buildroot}/%{_prefix}/sbin/mount.mount-s3

%files
%defattr(-,root,root,-)
%dir %attr(755,root,root) /opt/aws/mountpoint-s3
%dir %attr(755,root,root) /opt/aws/mountpoint-s3/bin
%attr(755,root,root) /opt/aws/mountpoint-s3/bin/mount-s3
%doc %attr(644,root,root) /opt/aws/mountpoint-s3/NOTICE
%license %attr(644,root,root) /opt/aws/mountpoint-s3/LICENSE
%license %attr(644,root,root) /opt/aws/mountpoint-s3/cargo-vendor.txt
%attr(644,root,root) /opt/aws/mountpoint-s3/THIRD_PARTY_LICENSES
%attr(644,root,root) /opt/aws/mountpoint-s3/VERSION
%attr(755,root,root) %{_bindir}/mount-s3
%attr(755,root,root) %{_prefix}/sbin/mount.mount-s3

%changelog
* Thu Oct 09 2025 Mountpoint-S3 Team <s3-opensource@amazon.com> - 1.21.0+amzn2023
- Mountpoint-S3 1.21.0 amzn2023 release
- Refer to https://github.com/awslabs/mountpoint-s3/blob/main/mountpoint-s3/CHANGELOG.md

[yerzhan7]

I still see rust >= 1.88. I thought we agreed to change it to exact version, no? rust = 1.88

[yerzhan7]

Also, Source0: mountpoint-s3-%{version}.tar.gz still didn't to resolve to exact version? mountpoint-s3-1.21.0.tar.gz

[yerzhan7]

I still see rust >= 1.88. I thought we agreed to change it to exact version, no? rust = 1.88

Discussed offline.

[muddyfish]

Consider using pathlib.Path here - this makes path manipulation more readable

[muddyfish]

Use https://docs.python.org/3/library/tomllib.html

[muddyfish]

Nb: does a list comprehension look better here?

[tadiwa-aizen]

Cool, ill try something like this

def generate_bundled_provides(submodule_versions):
    return "\n".join([f"Provides: bundled({lib_name}) = {lib_version}" 
                      for lib_name, lib_version in submodule_versions.items()])

[muddyfish]

Consider using argparse

[muddyfish]

I think this should ideally be it's own file

[muddyfish]

Why are we mixing f-strings and jinja templating?

[muddyfish]

Is this accurate?

[muddyfish]

Does this if statement need to be here?

[muddyfish]

Nit: use removeprefix

[muddyfish]

Using a generator comprehension suffices

[muddyfish]

Should this be another argument?

[muddyfish]

Raise an exception

[muddyfish]

We've traditionally used this April 9, 2025 as our date format - why change it?

[tadiwa-aizen]

Its RPM changelog format, they include the day, you can double check by running dnf changelog x .

[muddyfish]

Why use kwarg splatting here when you can just pass in arguments normally?

[muddyfish]

This is always constant for a given spec file. Why do we need to pass this in as an argument?

[muddyfish]

Use FileNotFoundError

[muddyfish]

You're not actually using this function

[muddyfish]

Do we need these raw blocks for something?