`BackTraceMarker` should be `repr(u16)`, and `Component` should not be `Pod`

[zachs18]

sigalign/sigalign-core/src/algorithm/wave_front/fill.rs

Line 291 in d600eb4

let my_struct_bytes: [u8; std::mem::size_of::<Components>()] = std::mem::transmute(components);

This test invokes UB by producing an invalid value of type u8.

BacktraceMaker is a (implicitly repr(Rust)) fieldless enum, so it's representation is unspecified. In practice, it will be represented as u8, since it has fewer than 256 variants.

sigalign/sigalign-core/src/algorithm/wave_front/mod.rs

Lines 125 to 131 in d600eb4

	#[repr(C)]
	#[derive(Debug, Clone, Copy)]
	pub struct Component {
	pub fr: i32,
	pub insertion_count: u16,
	pub bt: BackTraceMarker,
	}

The layout of Component is therefore

• bytes 0..=3: fr: i32
• bytes 4..=5: insertion_count: u16
• byte 6: bt: BackTraceMarker
• byte 7: padding

Components is effectively [Component; 3], so it therefore also has padding (at bytes 7, 15, and 23)

Since Components has padding, it is UB to transmute it to [u8; _] (and it should not implement bytemuck::Pod)

However, if you add #[repr(u16)] (or #[repr(i16)]) to BacktraceMarker, then Component will not have any padding bytes, so this particular test will no longer invoke UB.

 #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
+#[repr(u16)]
 pub enum BackTraceMarker {
     Empty = 0,
     Start = 1,
     FromM = 2,
     FromD = 3,
     FromI = 4,
 }

---

However, Component still should not implement bytemuck::Pod, which requires that "The type must allow any bit pattern", which is not true since BackTraceMarker only allows the values 0..=4. It could be NoUninit and Zeroable, however.

---

Found by https://github.com/craterbot at rust-lang/rust#140985 (comment)

[baku4]

• The byte-cast code path was an old micro-optimization and is no longer used.
• Therefore I will
  1. remove the bytemuck dependency (Pod, Zeroable); and
  2. keep BackTraceMarker as a plain fieldless enum (no explicit repr, because I don't rely on its exact size).
• If raw-byte casting ever returns, I'll add Zeroable (+ NoUninit for the enum).

Code changes

// sigalign-core/src/algorithm/wave_front/mod.rs (after the change)

#[repr(C)]
#[derive(Debug, Clone, Copy)]   // Pod and Zeroable removed
pub struct Components {
    pub m: Component,
    pub d: Component,
    pub i: Component,
}

#[repr(C)]
pub struct Component {
    pub fr: i32,
    pub insertion_count: u16,
    pub bt: BackTraceMarker,
}

// No Pod / Zeroable impls anymore.

pub enum BackTraceMarker {      // no explicit repr
    Empty  = 0,
    Start  = 1,
    FromM  = 2,
    FromD  = 3,
    FromI  = 4,
}

• bytemuck is removed from sigalign-core/Cargo.toml.
• The test that called transmute::<Components, [u8; _]>() in fill.rs (lines 288-298) is gone, so no padding is ever read and the UB disappears.

---

Thanks a lot for reporting this!
If you see a better approach, let me know.