Adding a std logging backend

[diondokter]

So I've been using defmt in a couple of libraries of mine and it's super nice!

However, when I want to add tests to the library that I run on my PC, defmt needs a global logger and there are no loggers that work for PC tests afaik.
What I want to propose is similar to #714, but maybe not quite the same.

I would like to see a feature flag that switches defmt over from using its own logger and deferred formatting to using println, panic, the log crate and Rust's built-in formatter.

I took a stab at it today, but only for the defmt::println macro to try this out: main...diondokter:defmt:std-log

In essence I think it should be possible to translate the defmt formatting to the rust formatting which can then be routed to panic, println and log::**** as required.

This is pretty doable if the API would be that every type needs to implement Debug when in this mode, but theoretically it should be possible to use the Format trait as input too.

So I want to know, is there interest in this? Has something in this direction a chance of being accepted?
If so I think I could spare some time to work on this.

[Urhengulas]

Hi @diondokter, I would like this. Many people report that they struggle with this issue, especially in CI. Since it would be behind a feature flag it is not even a breaking change as far as I understand.

Also, we mostly tried to stay close to the std::fmt grammar for the format strings, so it should be possible to translate with not too much effort. Please tell me about the cases where the defmt grammar diverges from the std::fmt grammar.

[diondokter]

Based on the defmt book I can see:

• Primitive specifications. But these can mostly be ignored, except for the bitfields and interned strings (https://defmt.ferrous-systems.com/primitives.html)
• The :a ascii hint
• The :us hint

[t-moe]

can you comment on whether this is on the v1.0.0 roadmap @jonathanpallant ?
I took another stab at this today to try how we can modify the existing derive(Format), so that it also works with the log backend. ...Its kinda tricky without introducing semver-breaking changes. So it would be nice to know, what is planned regarding this topic...

[jonathanpallant]

Working on it!

[jonathanpallant]

So, my current thinking is to merely stabilise the syntax of the defmt::info! macros (et al) for 1.0.

After that, I think there are two options for defmt work on libstd systems:

1. Add a feature flag for not interning strings, so the string is serialised with the rest of the values. You can then write a defmt-log crate which parses the format string, renders it to a String and throws it at e.g. log::info!.
   • This requires all arguments to implement defmt::Format
2. Add a feature flag for using core::fmt. That would make defmt process the format strings with a proc-macro to remove the =u32 style type specifiers, and hand the modified format string onto the log::info! macro.
   • This requires all arguments to implement defmt::Format and core::fmt::Debug

There might be other options as well, but I think we can make a decision about which approach is best after 1.0 is out.

[t-moe]

Hmm, I hadn’t considered option 1. But you’re right; since the std-logging backend will likely be used only on powerful (non-embedded) targets, using defmt encoding, followed by decoding and forwarding to log, could indeed be feasible.

The approach I tested locally is similar to your Option 2, but without requiring all arguments to implement core::fmt::Debug.

Instead I added a new hidden method to the Format trait, which basically mirrors core::fmt::Debug::fmt():

pub trait Format {
  //...

  #[doc(hidden)]
  fn _core_fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {
    Ok(())  
  }
}

Then I've extended the codegen logic for all defmt macros (e.g. defmt::error!) to wrap each argument with Format2Debug(&arg), where Format2Debug is defined as:

pub struct Format2Debug<'a, T: Format + ?Sized>(pub &'a T);
impl <'a, T: Format + ?Sized> fmt::Debug for Format2Debug<'a, T> {
    fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {
     self.0._core_fmt(fmt)
    }
}

Next I've extended the derive(defmt::Format) implementation to automatically populate the body of this method. And I've also implemented the _core_fmt method for all primitives etc.

The advantage of this solution is, that as a user of defmt I dont need to implement core::fmt::Debug and can keep using defmt::Format in my code and enable the std logging backend with a simple feature flag. Also, its not too much effort to implement this whole change. (Probably less than implementing defmt-log ?)

The problem arises, and this is where I need your attention, if the user manually implements defmt::Format:
I think it would be best if we could pack the &mut core::fmt::Formatter<'_> into our defmt::Formatter, so that the defmt::write! calls in the body of the defmt::Format::fmt implementations can forward to the core::fmt::Formatter. But this doesnt work currently, because defmt::Formatter is passed by value and implements Copy. Therefore, if we ever want to follow this approach, I propose applying the following change before 1.0.0:

pub trait Format {
-  fn format(&self, fmt: Formatter);
+  fn format(&self, fmt: &mut Formatter<'_>);

 //...
}

[Urhengulas]

There is previous discussion in #463.

[t-moe]

I’d like to renew my request to consider including the highlighted diff above into defmt 1.0.

In this comment, @Dirbaio mentioned:

(fwiw #730 can be solved at the application layer by making it use either log or defmt. embassy uses this for example. Getting actual tokenization working on non-embedded does require fixes in defmt)

While this workaround is functional, it introduces a frustrating user experience:

• Every project must either copy fmt.rs or rely on something like defmt-or-log (which ideally wouldn’t need to exist).
• Expanded file and line numbers always point to fmt.rs instead of the actual source location.
• Both defmt::Format and core::fmt::Debug traits need to be implemented in the application, often requiring #[cfg_attr] and other boilerplate.

I think it should be just a matter of enabling a feature on the defmt crate and having it work on non-embedded targets.

defmt has become the de facto standard for embedded Rust, yet we still encounter unnecessary boilerplate and log/defmt switching in every new project. I don’t have a strong preference for whether #730 or #463 is implemented first, but addressing either would significantly improve the developer experience.

[Dirbaio]

Expanded file and line numbers always point to fmt.rs instead of the actual source location.

this is no longer the case with #[collapse_debuginfo(yes)]

[t-moe]

this is no longer the case with #[collapse_debuginfo(yes)]

Interesting, TIL.

Still, there is the problem that we have no common trait. How do i write a function which accepts a struct that can be formatted with either log or defmt (depending on whether I'm compiling for CI/CD tests or embedded, log or defmt)

[pvdrz]

I think it is possible to change defmt::Format::format so it takes &mut Formatter<'_> instead of just Formatter and add a small compatibility layer for folks that already implement Format manually.

This layer would be behind a feature that I'm going to call compat (naming things is hard™). This feature would be enabled by default for defmt on 0.3.x and disabled by default on 1.0.0 and it would gate a slightly tweaked version of the Format trait:

pub trait Format<F = Formatter<'static>> where F: CompatFormatter {
   fn format(&self, fmt: F);
   // ...
}

Here CompatFormatter would basically be a sealed trait that's only implemented for Formatter<'a> and for &'a mut Formatter<'b>. This means that any impl Format for T { ... } block would not need any changes when the compat feature is enabled.

If users want to move forward, they need to do the following change:

- impl Format for T {
+ impl<F: CompatFormat> Format<F> for T {
-    fn format(&self, fmt: Formatter);
+    fn format(&self, fmt: F);  
    // ...
}

Which means their implementations could use either Formatter or &mut Formatter<'_> without issue. Or they could do the change to &mut Formatter<'_> directly and disable the compat flag all together.

Let me know what you think!

[t-moe]

Hey @pvdrz,

Thanks for considering a backward-compatible approach for releasing this change — it does seem like a viable solution.

That said, I believe this is more of a strategic decision than a purely technical one. Should we include this change in 1.0 to keep the door open for future developments, or is it unnecessary because we're likely to take a different path (like #890)?

I'm not sure who has the authority to decide this — perhaps the Embedded Rust WG or someone else?

[jonathanpallant]

For reference, Ferrous Systems hold the decision making responsibility, as the owners of the knurling-rs project.