no alloc support

[theunkn0wn1]

I note #71 Which adds support for no_std crates, but capnp still depends on an allocator.
It is desirable to have support for environments that do not have an allocator / are memory constrained such as microcontrollers.

In my application, I am presently using postcard to provide object de/encoding on both sides of the wire. Among other features, it supports de/encoding messages out of a &[u8] buffer, thus allowing it to work without alloc.

I am evaluating capnproto to see if it can be a viable replacement for postcard and gRPC (replacing the latter with capnproto's RPC) as to simplify my software situation.

It appears that capnp, even in no_std mode, does not have a mechanism for statically parsing messages.

[dwrensha]

Yes, that is accurate: the capnp crate still requires an allocator. I agree that it would be good to remove that requirement. To achieve that, we would need:

1. An implementation of ReaderSegments that does not allocate
2. To remove the description: String field from capnp::Error, maybe replacing it with an enum.

Note, however, that getting the capnp-rpc crate (as opposed to just the base crate capnp) to work with no_std and no alloc would be a lot more difficult.

[theunkn0wn1]

Thanks for the response.

For Capnproto to be viable, I don't require RPC support in no std; I only require the capability to statically decode a message of known maximum size.

[romixlab]

Tried it on STM32F051 (thumbv6m) with alloc-cortex-m crate. Unfortunately something in capnp::private::layout::wire_helpers::allocate () causes a HardFault to occur on str r1, [r5, #0] instruction, while r1 = 0.

Btw size of the binary increased by approx 10kB after adding allocator and capnp dependencies (with opt-level = "z", codegen-units = 1, lto = true)

Upd:
Here is a backtrace:

(gdb) backtrace 
#0  0x08004298 in HardFault ()
#1  <signal handler called>
#2  capnp::private::primitive::{{impl}}::set (value=0, raw=<optimized out>) at /Users/roman/.cargo/registry/src/github.com-1ecc6299db9ec823/capnp-0.14.2/src/private/primitive.rs:43
#3  capnp::private::primitive::WireValue<u32>::set<u32> (self=0x0, value=0) at /Users/roman/.cargo/registry/src/github.com-1ecc6299db9ec823/capnp-0.14.2/src/private/primitive.rs:105
#4  capnp::private::layout::WirePointer::set_kind_and_target (self=0x0, kind=<optimized out>, target=<optimized out>) at /Users/roman/.cargo/registry/src/github.com-1ecc6299db9ec823/capnp-0.14.2/src/private/layout.rs:184
#5  capnp::private::layout::wire_helpers::allocate (arena=..., reff=0x0, segment_id=0, amount=1, kind=<optimized out>) at /Users/roman/.cargo/registry/src/github.com-1ecc6299db9ec823/capnp-0.14.2/src/private/layout.rs:425
#6  0x0800101a in vhrdbutton::__cortex_m_rt_main ()
#7  0x08000cb4 in main ()

[dwrensha]

We got another report of a hard fault using the alloc-cortex-m crate here: #293

[dwrensha]

@romixlab I expect there's a good chance that if you upgrade to capnp-v0.14.10 the HardFault will go away (and probably be replaced by an allocation failure).

[romixlab]

Would be interesting to check again!

[ariel-miculas]

2. To remove the description: String field from capnp::Error, maybe replacing it with an enum.

@dwrensha I'm trying to integrate capnproto into the linux kernel, where there is an alloc crate but with limited functionality (e.g. all the allocations are fallible, so try_new is used instead of new; there is also no String). I was thinking about the same thing, using an enum instead of descriptions for errors; are you willing to accept such a PR (in order to minimize the differences between the kernel version and the upstream version). Note that it's not yet clear whether capnproto could be accepted in the kernel, for the moment I'm mostly experimenting.

[dwrensha]

Yes, I still want this to happen, and I would review a PR.

A question that I had been stuck on is: if capnp::Error is no-alloc, then what do we do about the capnp_rpc::rpc::remote_exception_to_error() function?

capnproto-rust/capnp-rpc/src/rpc.rs

Lines 368 to 384 in 3bbb520

	fn remote_exception_to_error(exception: exception::Reader) -> Error {
	let (kind, reason) = match (exception.get_type(), exception.get_reason()) {
	(Ok(exception::Type::Failed), Ok(reason)) => (::capnp::ErrorKind::Failed, reason),
	(Ok(exception::Type::Overloaded), Ok(reason)) => (::capnp::ErrorKind::Overloaded, reason),
	(Ok(exception::Type::Disconnected), Ok(reason)) => {
	(::capnp::ErrorKind::Disconnected, reason)
	}
	(Ok(exception::Type::Unimplemented), Ok(reason)) => {
	(::capnp::ErrorKind::Unimplemented, reason)
	}
	_ => (::capnp::ErrorKind::Failed, "(malformed error)"),
	};
	Error {
	description: format!("remote exception: {reason}"),
	kind,
	}
	}

The reason field is an arbitrary Text from a capnp message, and it would be sad to lose any information contained in it.

Just now I had the thought: capnp::Error could accommodate this with a variant that only exists when the alloc feature is enabled.

[ariel-miculas]

Something like this? Errors are an enum and when the alloc feature is enabled, then they also include the string in the description.

diff --git a/capnp/src/private/layout.rs b/capnp/src/private/layout.rs
index 7f5fbffa..76f3455e 100644
--- a/capnp/src/private/layout.rs
+++ b/capnp/src/private/layout.rs
@@ -1409,9 +1409,10 @@ mod wire_helpers {
                 }
                 Byte | TwoBytes | FourBytes | EightBytes => {
                     if data_size < 1 {
-                        return Err(Error::failed(
-                            "Existing list value is incompatible with expected type.".to_string(),
-                        ));
+                        let err = Error::IncompatibleWithExpectedType;
+                        #[cfg(feature = "alloc")]
+                        err.set_desc("Existing list value is incompatible with expected type.".to_string());
+                        return Err(err);
                     }
                 }
                 Pointer => {

[dwrensha]

Hm... that would work, but I'm worried that it would end up with a lot of #[cfg]-gated code. I was imagining instead that the String would be part of the enum variant. Like

pub enum Error {
  IncompatibileTypes,
  OutOfBounds,
  // ...
  #[cfg(feature="alloc")]
  RpcError(String),
}

The Debug implementation for capnp::Error can expand the variants into strings like "Existing list value is compatible...". Moreover, the variants can have nontrivial data, like integer offsets or type IDs, which can be incorporated into the output of the Debug impl.

[ariel-miculas]

Yes, but what I'm trying to do is to avoid allocating a String for the error, that's why I wanted to replace the error handling with an enum (which could still hold data, just not a String).
I think I didn't quite get your suggestion, how would the code look like? Would you throw an RpcError only from capnp-rpc?