Skip to content

cfg arch-specific ABIs for unsupported_{,fn_ptr}_calling_conventions #67

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

workingjubilee
Copy link

Addresses FCW lints to assure code will compile on Rust 1.89 and beyond, as these ABIs get translated to others if the cfg condition is not met.

@workingjubilee workingjubilee force-pushed the fix-retour-for-unsupported-fn-ptr-calling-conventions branch from b7fc23d to ee0b523 Compare June 10, 2025 00:32
@workingjubilee
Copy link
Author

Currently includes #68 so this passes CI.

Addresses FCW lints to assure code will compile on Rust 1.89 and beyond,
as these ABIs get translated to others if the cfg condition is not met.
@workingjubilee workingjubilee force-pushed the fix-retour-for-unsupported-fn-ptr-calling-conventions branch from ee0b523 to 19fa964 Compare June 10, 2025 00:37
Copy link
Author

@workingjubilee workingjubilee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a lot of possible decisions regarding how to mitigate the breakage from this, so I am happy to edit this appropriately. I just wrote out the boringly obvious one of "just take it on the chin" because most of the others seem like they'd want your input first.

Comment on lines 186 to 195
impl_hookable!(@impl_pair ($($nm : $ty),*) (extern "cdecl" fn($($ty),*) -> Ret));
impl_hookable!(@impl_pair ($($nm : $ty),*) (extern "stdcall" fn($($ty),*) -> Ret));
#[cfg(target_arch = "x86")]
impl_hookable!(@impl_pair ($($nm : $ty),*) (extern "fastcall" fn($($ty),*) -> Ret));
#[cfg(target_arch = "x86")]
impl_hookable!(@impl_pair ($($nm : $ty),*) (extern "stdcall" fn($($ty),*) -> Ret));
#[cfg(target_arch = "x86_64")]
impl_hookable!(@impl_pair ($($nm : $ty),*) (extern "win64" fn($($ty),*) -> Ret));
impl_hookable!(@impl_pair ($($nm : $ty),*) (extern "C" fn($($ty),*) -> Ret));
impl_hookable!(@impl_pair ($($nm : $ty),*) (extern "system" fn($($ty),*) -> Ret));

#[cfg(feature = "thiscall-abi")]
#[cfg_attr(docsrs, doc(cfg(feature = "thiscall-abi")))]
#[cfg(any(docsrs, all(target_arch = "x86", feature = "thiscall-abi")))]
impl_hookable!(@impl_pair ($($nm : $ty),*) (extern "thiscall" fn($($ty),*) -> Ret));
Copy link
Author

@workingjubilee workingjubilee Jun 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specifically, all of these ABIs are silently translated to extern "C" if the cfg is not met. The type remains the same but the pointee is a different ABI in actual fact.

Comment on lines -190 to +194
#[cfg(feature = "thiscall-abi")]
#[cfg_attr(docsrs, doc(cfg(feature = "thiscall-abi")))]
#[cfg(any(docsrs, all(target_arch = "x86", feature = "thiscall-abi")))]
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not able to build docs locally at the moment but I believe this loses the feature attribute on the docs (since doc(cfg(...)) is the attribute that adds it in the generated docs

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, is it? I'll revert that part of the change.

@Hpmason
Copy link
Owner

Hpmason commented Jun 10, 2025

Is rust 1.89 changing how extern fns are treated when compiling for targets that don't support that specific extern? Or is it just changing warnings?

@workingjubilee
Copy link
Author

workingjubilee commented Jun 10, 2025

@Hpmason Previously, using ABIs like extern "stdcall" on platforms that did not support them (for stdcall it was x86 or, oddly enough, Windows) was made a hard error. But the check was not done perfectly, so function pointers using these ABIs slipped through the linting and future compatibility warnings. So we have issued a separate lint on those for a bit now, and Rust 1.87 will warn even anyone who uses your library as a dependency about this concern.

I intend to propose that we switch this from a lint to a hard error for this case, because it makes the resulting behavior... does the code compile or not?... consistent for all usage of extern "{abi}" fn in all positions in the grammar, depending only on target support. This may land in 1.89, or 1.90, or even later.

Separately, in 1.89, we have already started linting on usage of ABIs that were allowed on Windows if they were a mismatching architecture. Those will remain functional even if my recommendation regarding switching to an error is followed. This lint will remain for a while longer as we will only start linting in 1.89, and it will be on a new axis we may get different feedback for.

So, I can fix up this PR to have either

  • the set of cfgs for extern "{abi}"s that will still compile even if all the future compatibility warnings are made hard errors (current status of this PR)
  • the set of cfgs for extern "{abi}"s that will still compile for 1.89 even if my recommendation is followed promptly, though they may trigger lints
  • later this week, if T-lang manages to agree on what cfg_version syntax they want to ship, then I could even update this PR to use that, too, but that might be a bit optimistic

@workingjubilee
Copy link
Author

Corrections to above statement in light of new information:

  • Certain linting will be enhanced in 1.89, and that will not warn on this library's users but will warn you. It is unclear when, if ever, those lints will become hard errors.
  • Certain ABIs will be hard-rejected in 1.90, and that will include use of extern "{abi}" fn(A) -> R on platforms which don't support "{abi}".

And to re-address this, specifically, and to perhaps better explain why we are making this change:

Is rust 1.89 changing how extern fns are treated when compiling for targets that don't support that specific extern? Or is it just changing warnings?

Previously to 1.90, we compute extern "{abi}" fn(A) -> R according to an arbitrary ABI chosen by the compiler. In some cases, we didn't even make such an arbitrary choice, and then it was possible the compiler would reach an internal compiler error. It was also possible for us to compute the ABI according to an impossible-to-codegen pattern and reach an error in LLVM, or to simply emit incorrect codegen that matched no ABI whatsoever.

Handling all of those edge-cases is not done in a principled way because people who initially proposed adding this ABI support to the compiler did not think about it enough to document what their intentions were. They were focused on addressing a use-case, instead of interacting with the "negative space" off the happy path.

@workingjubilee
Copy link
Author

workingjubilee commented Jun 23, 2025

I do think my changes handle it in a principled way... if somewhat depressing if you were hoping to not interact with this complexity. Namely, we cut off its head. An invalid ABI will remain syntactically valid... it parses. But it will no longer proceed past AST expansion, so we conclude we must error on it before we even reach the typechecking phase. This is important because that particular step is the only point in the compiler which is guaranteed to see every instance of extern "{abi}".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants