-
Notifications
You must be signed in to change notification settings - Fork 46
fix: c_void enum in BTF #260
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 1 of 1 files at r1.
Reviewable status: 1 of 3 files reviewed, 5 unresolved discussions
src/llvm/di.rs
line 76 at r2 (raw file):
fn visit_mdnode_item(&mut self, item: Item) { // guardrail preventing to visit non mdnode items
this kind of sucks. can item.to_mdnode
return an option instead?
src/llvm/di.rs
line 82 at r2 (raw file):
let mdnode = item.as_mdnode(); match mdnode.clone().try_into().expect("MDNode is not Metadata") {
what's with this clone here?
src/llvm/di.rs
line 90 at r2 (raw file):
let name = di_composite_type .name() .map(|name| name.to_string_lossy().to_string());
i don't think conversion to lossy is appropriate here
src/llvm/di.rs
line 104 at r2 (raw file):
new.len(), 8, // DWARF void
is this DW_ATE_void? write that, please.
src/llvm/di.rs
line 113 at r2 (raw file):
} } else { warn!("failed at replacing c_void enum, it might result in BTF parsing errors in kernels < 5.4")
this is a weird thing to do. on 32 bit size_in_bits won't be 8, but also this would all be unnecessary (right?). why log a warning then?
Definitely possible ! Will fix accordingly.
Will remove it's a leftover artifacts of test implementation.
Will fix that
Will fix this.
|
WARNING: this commit only is supposed to fail the tests
96ee9ff
to
38c41e7
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 1 of 3 files reviewed, 3 unresolved discussions (waiting on @qjerome)
src/llvm/di.rs
line 104 at r2 (raw file):
Previously, qjerome (Quentin JEROME) wrote…
Will fix this.
Actually, I no longer understand this comment. What is it anchored on?
src/llvm/di.rs
line 113 at r2 (raw file):
c_void
Rust enum is#repr[u8]
so its size should always be 8 bits.
on a 32 bit target?
anyway, I see now that this warning is logged when the item is not an operand. Please include that detail here.
src/llvm/di.rs
line 90 at r6 (raw file):
if name == c"c_void" && di_composite_type.size_in_bits() == 8 { if let Item::Operand(mut operand) = item { let new = "void";
is there a reason we need to rename it? if yes, write a comment.
also, you seem to be creating this type every time you hit an operand of this type, rather than just once.
src/llvm/di.rs
line 285 at r6 (raw file):
// if our item is also a MDNode if item.is_mdnode() {
remove this check, it is in the method (and remove is_mdnode
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@qjerome please respond to every comment before requesting additional reviews.
Reviewed 1 of 2 files at r2, all commit messages.
Reviewable status: 2 of 3 files reviewed, 7 unresolved discussions (waiting on @qjerome)
src/llvm/di.rs
line 118 at r7 (raw file):
} /// Get or cache a LLVM DIBasicType given a [DIBasicType]
This comment should say something about returning a metadata ref for the given basic type.
Code-like references should be fenced e.g.
[`DIBasicType`]
Comments should be proper sentences. Please add a period.
src/llvm/di.rs
line 123 at r7 (raw file):
.basic_types .entry(di_bt) .or_insert(di_bt.llvm_create_basic_type(self.builder))
this should be or_insert_with
. The way you've written it llvm_Create_basic_type
is always called.
src/llvm/di.rs
line 139 at r7 (raw file):
if let Some(name) = di_composite_type.name() { // we found the c_void enum if name == c"c_void" && di_composite_type.size_in_bits() == 8 {
shouldn't all this be using DIBasicType::Void instead of repeating all the details?
src/llvm/di.rs
line 144 at r7 (raw file):
// get LLVM void DIBasicType let void_bt = self.di_basic_type(DIBasicType::Void); let _ =
what is this _
?
src/llvm/di.rs
line 310 at r7 (raw file):
}; let first_visit = self.visited_nodes.insert(value_id);
is there a reason this moved?
Done
Forgot about that HashMap trickery. Will change it ...
No,
This is because we don't use the return value of the function call. If you speak about while there's a line break ... IDK, ask rust-analyzer :)
Yes, instead of making the replacement in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 2 of 3 files reviewed, 6 unresolved discussions (waiting on @qjerome)
src/llvm/di.rs
line 118 at r7 (raw file):
Previously, qjerome (Quentin JEROME) wrote…
Done
The same is true of DIBasicType.
src/llvm/di.rs
line 144 at r7 (raw file):
Previously, qjerome (Quentin JEROME) wrote…
This is because we don't use the return value of the function call. If you speak about while there's a line break ... IDK, ask rust-analyzer :)
let _
is discarding information. At the very least we should ascribe the type so I can tell if it's ok to throw this away or not.
done
This is now fixed because I had to change a bit that part of the code. |
src/llvm/di.rs line 310 at r7 (raw file): this was reverted because it was creating assertion failures in LLVM source tests This latest fix patches |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 2 of 3 files at r10, all commit messages.
Reviewable status: 3 of 4 files reviewed, 14 unresolved discussions (waiting on @qjerome)
src/llvm/di.rs
line 118 at r7 (raw file):
Previously, qjerome (Quentin JEROME) wrote…
done
Not done.
src/llvm/types/di.rs
line 454 at r11 (raw file):
impl DICompileUnit<'_> { /// Constructs a new [`DICompileUnit`] from the given `value`.
s/value/value_ref/
src/llvm/types/di.rs
line 469 at r11 (raw file):
} pub fn enum_types(&self) -> Vec<DICompositeType> {
do we need to return a vector, or can we return impl Iterator<Item=DICompositeType> + '_
?
src/llvm/types/di.rs
line 489 at r11 (raw file):
} pub fn replace_enum_types(&mut self, builder: LLVMDIBuilderRef, rep: Vec<DICompositeType>) {
this can be &[DICompositeType]
, no need to force a vector
src/llvm/types/di.rs
line 490 at r11 (raw file):
pub fn replace_enum_types(&mut self, builder: LLVMDIBuilderRef, rep: Vec<DICompositeType>) { let mut rep: Vec<LLVMMetadataRef> = rep.into_iter().map(|dct| dct.metadata_ref).collect();
nit: Vec<_>
src/llvm/types/di.rs
line 493 at r11 (raw file):
unsafe { // we create an array with our saved enums
useless comments, here and below
src/llvm/types/ir.rs
line 115 at r11 (raw file):
DIDerivedType(DIDerivedType<'ctx>), DISubprogram(DISubprogram<'ctx>), DICompileUnit(#[allow(dead_code)] DICompileUnit<'ctx>),
what's with the dead code here?
src/llvm/di.rs
line 58 at r11 (raw file):
} // DWARF encoding https://github.com/bminor/glibc/blob/2fe5e2af0995a6e6ee2c761e55e7596a3220d07c/sysdeps/generic/dwarf2.h#L375
How about https://llvm.org/docs/LangRef.html#dibasictype as a reference instead?
src/llvm/di.rs
line 126 at r11 (raw file):
} fn visit_mdnode_item(&mut self, item: Item) {
item.as_mdnode() takes &self, why does this need to be Item rather than &Item?
src/llvm/di.rs
line 504 at r11 (raw file):
} // Removes `c_void` Rust enum from all DICompileUnit.
what's going on here? did you mean to remove the changes in visit_mdnode_item? it looks like the same logic is here and there.
I hope it is OK now, otherwise maybe just write the comment you would like to see.
done
done
done
done
done
It is to have a consistent Metadata enum with the new DICompileUnit type.
done
It facilitate the
In I didn't implement the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 1 of 2 files at r12, all commit messages.
Reviewable status: 3 of 4 files reviewed, 5 unresolved discussions (waiting on @qjerome)
src/llvm/di.rs
line 113 at r2 (raw file):
Previously, tamird (Tamir Duberstein) wrote…
c_void
Rust enum is#repr[u8]
so its size should always be 8 bits.on a 32 bit target?
anyway, I see now that this warning is logged when the item is not an operand. Please include that detail here.
please address this.
src/llvm/di.rs
line 126 at r11 (raw file):
Previously, qjerome (Quentin JEROME) wrote…
It facilitate the
Operand::replace
call a bit below, which requiresmut
.
then why not pass a mut ref? if we're cloning this thing, what are we actually mutating? the copy?
src/llvm/types/di.rs
line 489 at r11 (raw file):
Previously, qjerome (Quentin JEROME) wrote…
done
not done.
src/llvm/types/ir.rs
line 115 at r11 (raw file):
Previously, qjerome (Quentin JEROME) wrote…
It is to have a consistent Metadata enum with the new DICompileUnit type.
I can remove if you don't like it.
well i guess you need something or else the lifetime has nowhere to go.
Sorry, I really don't get what you mean here ! I don't see why that enum would be different on 32-bits.
Correct ! That is fixed.
Sorry, my bad !
Yes indeed, but my proposal was more to remove all |
Hey @tamird @vadorovsky, Maybe you forgot about this PR ! Is there anything blocking ? Cheers, |
src/llvm/di.rs
Outdated
fn fix_di_compile_units(&mut self) { | ||
for mut di_cu in self.di_compile_units() { | ||
let tmp_cu = di_cu.clone(); | ||
let enum_types: Vec<_> = tmp_cu.enum_types().collect(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You don't have to collect
here. You are only using it as an iterator later, so you could skip collecting and just do
let new_enums = tmp_cu.enum_types()
.filter(|e| {
if let Some(name) = e.name() {
// we filter c_void Rust enum
if c"c_void" == name && e.size_in_bits() == 8 {
return self.replaced_enums.contains(&e.value_id());
}
}
false
});:
You don't even have to collect()
here 👆 if you make replace_enum_types
take an impl IntoIterator<Item = DICompositeType>
, so you can pass both slices and iterators to that method. I will comment there as well.
If you manage to do it, you will end up with lazy iterators, which are consumed and looped over just once, instead of 3 loops like now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a good reason to collect, it is because I used the number of enums as a trigger for enum replacement, not to replace enums when it is not needed. I will try to opt for another strategy but I don't know if we can implement that logic in a single loop ...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can do it in one loop if replace_enum_types
takes a lazy iterator over MetadataRef but I think it is a bit dirty for an API.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reduced to 2 loops instead of 3 but I don't manage to make it in 1. Maybe I am lacking to much sleep
src/llvm/types/di.rs
Outdated
}) | ||
} | ||
|
||
pub fn replace_enum_types(&mut self, builder: LLVMDIBuilderRef, rep: &[DICompositeType]) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pub fn replace_enum_types(&mut self, builder: LLVMDIBuilderRef, rep: &[DICompositeType]) { | |
pub fn replace_enum_types(&mut self, builder: LLVMDIBuilderRef, rep: impl IntoIter<Item = DICompositeType>) { |
or
pub fn replace_enum_types(&mut self, builder: LLVMDIBuilderRef, rep: &[DICompositeType]) { | |
pub fn replace_enum_types<I>(&mut self, builder: LLVMDIBuilderRef, rep: I) | |
where | |
I: IntoIterator<Item = DICompositeType>, | |
{ |
So you can pass both slices and lazy iterators here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
src/llvm/di.rs
Outdated
} | ||
} | ||
|
||
fn visit_mdnode(&mut self, mdnode: MDNode) { | ||
/// Returns a metadata ref given a [`DIBasicType`]. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this docstring is inaccurate? This method takes a basic type kind and returns DIBasicType
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed in c388fd9
src/llvm/types/di.rs
Outdated
fn dwarf_type_encoding(&self) -> LLVMDWARFTypeEncoding { | ||
match self { | ||
// DW_ATE_signed | ||
Self::I8 => 0x5, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use gimli::DWE_ATE_signed
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed in eb175e4
src/llvm/di.rs
Outdated
@@ -438,6 +521,16 @@ impl Operand { | |||
} | |||
|
|||
impl Item { | |||
/// Returns the [`Item`] as [`MDNode`] only if [Item::is_mdnode] is `true` else `None`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/// Returns the [`Item`] as [`MDNode`] only if [Item::is_mdnode] is `true` else `None`. | |
/// Returns the [`Item`] as [`MDNode`] only if [`Item::is_mdnode`] is `true` else `None`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed in 9a7e0c9
src/llvm/di.rs
Outdated
let new_enums: Vec<_> = enum_types | ||
.into_iter() | ||
.filter(|e| { | ||
let kind = unsafe { LLVMGetMetadataKind(e.metadata_ref) }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps we should add a safe method kind()
which returns LLVMMetadataKind
and hides the unsafe
call. Let's try to keep unsafe
only in wrappers.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed in 5ec1a09
src/llvm/di.rs
Outdated
|
||
true | ||
}) | ||
.collect(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since you're taking IntoIterator
in replace_enum_types
, I think this collect
can go away?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we do that we will run replace_enum_types
for every CU replacing each enums by itself which is a bit odd VS currently where we run it if and only if we know we have at least one enum to remove (which should be the case only when c_void enum is in CU).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're forcing a vector allocation here, and then you make another vector in replace_enum_types
. Surely that is not necessary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I managed to remove one collect by using a peekable iterator. Please let me know if this solution is OK with you.
@vadorovsky @tamird @alessandrod : do you think this PR could be merge ? |
This PR has 22 commits. Please squash this to an appropriate number of commits, each with a sensible commit message. |
@tamird if you validate the last fix I'll squash for the final review. |
src/llvm/di.rs
Outdated
}) | ||
.collect(); | ||
let mut enum_types = tmp_cu.enum_types().peekable(); | ||
let need_replace = enum_types.any(|ct| { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is not how peekable works. you are passing a partially consumed iterator to replace_enum_types.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes indeed ... Find the appropriate fix in the last commit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks fine.
c699014
to
3da12fc
Compare
o/ I don't know if you've seen it but I fixed the double vector allocation. Can you please validate or not this change so that I can proceed with squashing commits? |
3da12fc
to
f819423
Compare
I squashed the commits \o/, let me know if anything else is needed before merging |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR addresses BTF loading failures on kernels older than 5.4 by replacing the Rust c_void
enum BTF entry with an i8
basic type and fully removing the enum from compile‐unit metadata.
- Add a new BTF assembly test to ensure no
<ENUM> 'c_void'
entries remain. - Extend the IR metadata handling with a
DICompileUnit
type and sanitizer logic to replace enum entries and prune compile units. - Introduce
DIBasicType
andDIBasicTypeKind
for creating and managing the newi8
BTF basic type.
Reviewed Changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
File | Description |
---|---|
tests/btf/assembly/c_void_enum.rs | New test that checks the absence of the c_void enum. |
src/llvm/types/ir.rs | Add DICompileUnit to Metadata and its conversion. |
src/llvm/types/di.rs | Implement DICompileUnit iteration and replacement APIs. |
src/llvm/di.rs | Refactor DISanitizer to replace enums with i8 types. |
Comments suppressed due to low confidence (2)
tests/btf/assembly/c_void_enum.rs:19
- [nitpick] The test only validates that the
c_void
enum is removed but does not assert that the replacementi8
type appears in the BTF dump. Consider adding a positive CHECK for thei8
basic type to ensure the intended replacement.
// CHECK-NOT: <ENUM> 'c_void'
src/llvm/di.rs:69
- The impl block uses
DISanitizer<'_>
instead of specifying the concrete'ctx
lifetime. Change it toimpl<'ctx> DISanitizer<'ctx>
to avoid lifetime mismatches and make the code clearer.
impl<'ctx> DISanitizer<'_> {
Hey ! |
This PR fixes an issue with kernels < 5.4 failing to load BTF because of the
c_void
enum.When used in BPF code, the Rust
c_void
type (actually an enum) generates the following BTF:It appears that kernels < 5.4 do not allow BTF enums with a size of 1 byte.
The fix consists of replacing the
c_void
enum MDNode withi8
DI BasicType information.To achieve this, a small refactoring of
DISanitizer::visit_mdnode
was necessary. The oldvisit_mdnode
had only oneMDNode
argument, which would have required an additional argument to pass theItem::Operand
to fix the MDNode. This was not optimal because both theOperand
and theMDNode
carry the sameLLVMValueRef
, which I found confusing (two arguments with the same information). The refactoring aims to make things cleaner and less confusing.Another issue arose when using LLVM compiled with debug assertions. A casting assertion failed at this line:
The exact assertion failing downstream is
llvm::cast_if_present<llvm::DICompositeType, llvm::MDOperand> (Val=...)
. By inspectingVal
(with gdb), we observe the following:Its
MetadataKind
is0xc
-> DIDerivedTypeThis DIDerivedType is actually a pointer to the
DIBaseType
we replaced thec_void
enum with:Note:
0x5
is the LLVMDWARFTypeEncoding we use in thei8
DIBasicType to replace thec_void
enum. I have confirmed this value corresponds to the LLVMDWARFTypeEncoding value.I was unable to fully understand why and how this replacement happens. I speculate it is a side effect of replacing the
c_void
DICompositeType usingLLVMReplaceMDNodeOperandWith
. One observation is that it tends to happen when we replace thec_void
enum use in an MDTuple.However, the fact remains that the replaced
c_void
enums no longer need to exist in the DICompileUnit, as we have replaced any use of it. The proposed solution is to delete it completely using theDISanitizer::fix_di_compile_units
function.This change is