RFC: We need some sort of configuration system

[jessebraham]

As this project continues to grow, we increasingly are feeling a need for user-configurable features. IMO we have reached a point where we have outgrown Cargo features.

I believe that we need some sort of configuration file and build system in order to continue adding hardware support and functionality. I believe that Cargo features should largely be reserved for functionality which requires additional dependencies, and that we should move towards configuring hardware via a configuration file.

I think conceptually this task is not overly difficult; I could cobble something together in a day or two I'm sure. However, I think we need to put some serious thought into what the configuration format will look like, which things should be configurable, and how users will interact with this configuration (ie. do we want to build a tool similar to idf.py menuconfig?).

I would like to open a discussion regarding potential paths forward on this work.

[bjoernQ]

This is related to #42 I guess

I still think toml-cfg could be a nice option. We are also using it in esp-wifi already. Downside might be that it (at least I think so) only supports "flat" configs but I don't think we really need nested config structures (and maybe it's supported - just never tried). After unifying the HAL it would be even easier to use. Additionally, it was created by a well-known and trusted person.

It would be a nice fit to replace things like psram-2m, psram-4m ..., 'xtal-Xmhz` etc. features which really don't need to be Cargo-features.

As you already said: there are things which need and should be kept as features but there are a lot of other things which don't need to be features.

I'm not entirely sure what "build system" means since it could mean a few different things. One thing I could think of: Have a crate which provides functionality we want in the build.rs of binary crates and the build.rs of the binary crates will just look like this

fn main() {
    awesome_esp_hal_build_support_crate::main();
}

Not sure if we need or want something like that. Maybe it's not worth it at all

I would say at least in the beginning we won't really need something like menuconfig/guiconfig. We won't have much options in the beginning - editing a config file would probably be good enough. Later we might want to have something like that

[ProfFan]

I 2nd toml-cfg but it seems that it hasn't been updated for 2 years.

[jessebraham]

I'm not sure I understand why we need toml-cfg at all? What does it do that we can't do with serde/toml already? I'd rather use maintained libraries.

[bjoernQ]

What we need is a way for the end user to configure things in our library crate. I don't see how serde/toml alone can do that.
The only way I can think of would be to require a build.rs in the user's binary crate which does "something somehow" - t.b.h. I don't really like that but we could provide the functionality via a dev-dependency crate which would make it a bit less awful.

But maybe I just don't see the obvious way - a POC might shed some light on this.

If we are concerned, we could just ask James Munns about the support status of toml-cfg - I assume there wasn't any update because it just works but maybe it's more or less abandoned - in that case we shouldn't use it and replace it in esp-wifi

[bjoernQ]

Regardless of how we configure things in future .... here are the current features of esp-hal-common and an initial guess what will be a config option - certainly needs some discussion

Current Feature	Feature/Config	Comment
esp32	feature
esp32c2	feature
esp32c3	feature
esp32c6	feature
esp32h2	feature
esp32p4	feature
esp32s2	feature
esp32s3	feature
rt-riscv	internal feature
rt-xtensa	internal feature
flip-link	feature
psram-2m	config	maybe we still need a general PSRAM feature?
psram-4m	config
psram-8m	config
opsram-2m	config
opsram-4m	config
opsram-8m	config
opsram-16m	config
psram-80mhz	config
usb-otg	needed at all?
direct-vectoring	feature
interrupt-preemption	feature
vectored	feature
log	feature
eh1	feature	probably should get removed - eh1 should be default?
embedded-io	feature	probably should get removed?
ufmt	feature
async	feature	probably should get removed and always enabled once we figured out the runtime interrupt binding
embassy	feature
embassy-executor-interrupt	feature
embassy-executor-thread	feature
embassy-integrated-timers	feature
embassy-time-systick	feature
embassy-time-timg0	feature
riscv	internal feature
xtensa	internal feature
rv-init-data	internal feature
rv-zero-rtc-bss	internal feature
rv-init-rtc-data	internal feature
debug	feature
defmt	feature

Edit by Scott: Removed the xtal features

[lu-zero]

I 2nd toml-cfg but it seems that it hasn't been updated for 2 years.

I had a look and tried to clean it up, I second that it is very nice for the purpose.

[ProfFan]

So are we zeroing on toml-cfg? 😆 I really want to configure IRAM placements, it's such a pain currently and hurts performance 10x due to cache misses

[MabezDev]

I think that toml-cfg solves part of the problem, one thing I'm missing is how to turn config options into cfg flags. For instance, in the case of placing things in ram we will need to do something like #[cfg_attr(somecfg, ram)] to tell the linker to put those sections in RAM. How do we get from toml_cfg to that?

[MabezDev]

Another thing is how do we validate the config? For instance, if someone puts their PSRAM speed value to 400mhz, which isn't supported, do we just let it die at runtime? I'd personally like to see some bounds or value sets that we can set on some fields of the configuration struct.

[MabezDev]

My final concern, in regards this time to specifically cfg-toml, is maintainability concerns. What happens if/when cfg toml starts limiting us and we need to adapt/modify it? It's currently not actively maintained, now maybe we can help out here and become a maintainer of it, or perhaps its feature complete and the author doesn't expect to add more changes.

[bjoernQ]

We can definitely go with our own solution - we probably just need to "borrow" this from cfg-toml: https://github.com/jamesmunns/toml-cfg/blob/71f45258c4f8c9a23e5ac00c5220dd8b2f0f780a/src/lib.rs#L208-L240

And then we should be able to locate the config in our esp-hal build.rs. Then we can generate Rust code and/or cfg flags.

The only thing I don't have a good idea for is validation. We could use a schema file but they are not fun to maintain