You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Fix CBOR codegen for recursive types by applying Box to union variants (#4116)
# Fix CBOR recursive data structure handling with Box implementation
This PR addresses an issue in the CBOR codegen where recursive union
data structures aren't properly boxed during deserialization, resulting
in uncompilable Rust code.
## Problem
The current CBOR implementation correctly defines recursive data
structures with `Box` in type definitions, but fails to apply the
corresponding `Box::new` wrapper during deserialization. This causes
type mismatches when deserializing recursive Smithy models.
For example, with this Smithy model:
```smithy
structure RecursiveOperationInputOutputNested1 {
variant: FooChoice,
}
union FooChoice {
choice1: String
choice2: RecursiveOperationInputOutputNested1
}
```
The generated Rust type correctly uses `Box`:
```rust
pub enum FooChoice {
Choice1(::std::string::String),
Choice2(::std::boxed::Box<crate::model::RecursiveOperationInputOutputNested1>),
}
```
But the deserializer doesn't wrap the result with `Box::new`:
```rust
// In the deserializer function:
FooChoice::Choice2(
crate::protocol_serde::shape_recursive_operation_input_output_nested1::de_recursive_operation_input_output_nested1(decoder)
?)
```
This leads to the compiler error:
```
error[E0308]: `?` operator has incompatible types
--> shape_foo_choice.rs:14:9
|
14 | / crate::protocol_serde::shape_recursive_operation_input_output_nested1::de_recursive_operation_input_output_nested1(decoder)
15 | | ?),
| |_____^ expected `Box<RecursiveOperationInputOutputNested1>`, found `RecursiveOperationInputOutputNested1`
```
## Solution
This PR modifies the CBOR codegen to add `.map(Box::new)` when
deserializing values for union variants that are defined with `Box`. The
fixed code will generate:
```rust
FooChoice::Choice2(
crate::protocol_serde::shape_recursive_operation_input_output_nested1::de_recursive_operation_input_output_nested1(decoder)
.map(Box::new)?
)
```
This ensures type compatibility between the defined structure and the
deserialized data.
## Testing
Added a protocol test in `rpcv2Cbor-extras.smithy` file.
Closes: #4115
---------
Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
Copy file name to clipboardExpand all lines: codegen-core/src/main/kotlin/software/amazon/smithy/rust/codegen/core/smithy/protocols/parse/CborParserGenerator.kt
0 commit comments