Clean up configuring log formatting

[alice-i-cecile]

I think this works for now but in general the custom_layer and fmt_layer feel a bit hacky; it would be nice to figure out some cleaner way to override/add extra layers

Originally posted by @cBournhonesque in #17722 (review)