User experience report: bevy_ui in Bevy 0.5

[alice-i-cecile]

I've spent the last few days working with bevy_ui in earnest as an end-user, and I think it would be helpful to share my experience as a whole (relevant to #254 and bevyengine/rfcs#22).

Most of these can be written up into their own issues; let me know if you think there are any you think that's particularly worthwhile for.

TL; DR: Systems to control UI behavior en-masse is super cool, the layout algorithm is incredibly frustrating, UI components are way too monolithic and not composable, and we're missing a ton of useful supporting features.

User story

I was attempting to make a simple but polished Sudoku game, seen here under an MIT license. It had a 3x3 grid, and a panel of buttons to perform various tasks to the side.

Initially, the game grid was made in world space, which was straightforward and worked well. See:

However, this had some limitations:

1. Much of my functionality (like selecting cells) was fundamentally similar to UI buttons, and it was silly to write all of the logic for that from scratch.
2. I wanted it to automatically scale and be laid out nicely with the buttons beside it.

So I tried to move it over to a UI-only solution, seen in the ui-game-grid branch, and utterly failed after several serious roadblocks.

Let's talk about what worked and what didn't as an end user.

Good

• generic systems are extremely powerful and work great for manipulating UI. See my puzzle_button system, which stores marker components on the system and then pipes them around as events.
• storing multiple handles on components is extremely natural, and is a great way to handle hover / pressed / normal reactivity. See responsive_buttons system
• when flexbox works, it creates great results

Lessons learned

• use partial transparency to debug UI nodes
• arranging UI-heavy apps by logic / graphics / input works very well
• you absolutely need to split inputs from actions to handle multiple input types in a sensible way

Bad

Usability

• bevy_ui has literally no docs. This is particularly brutal when working with Style, where I had to rely on CSS docs and hope the behavior was consistent
• Style is a terrible component name as it lumps everything together and doesn't communicate what it might do
• asset loading has a lot of boilerplate, (see here) and I wasn't even using States to check that everything is loaded
• cameras do not have their own marker components
• debugging where each layout box is is very hard, as you need to manually give everything colors. Using partially transparent colors helps a lot, since then you can see overlap
• the default color material is not Color::None, so you need to add that everywhere when using boxes for layout purposes
• many convenience methods (like multiplication of Val by floats) are missing
• there so many ways to set visibility: visible, display, material color, removing the material component, each with subtly different behavior
• size is set in an inconsistent way between UI and non-UI elements that are otherwise identical
• managing, organizing and setting large numbers of configurable constants is a nuisance. This might be solved by a data-driven (or visual) workflow eventually
• there's no way to simple way to bind a keyboard input to a button, resulting in code duplication

Architecture

• monolithic components make it very hard to reason about what's happening and why, and force you to always fetch and manipulate the giant struct and rely on nested ..Default::default() calls
• it is extremely hard to translate code between UI and not-UI. The paradigms are completely different, and there's no way to opt into just some of the desired behavior (like layout, or being drawn on top, or click handling, or...)
• parent-child-driven architecture + delayed effects of commands seriously encourages giant monolithic systems that set up your entire UI at once in a deeply nested way. Storing the Entity refs and then using .push_children helps a lot for this though

Parent-child

• the parent-child hierarchy is heavy on boilerplate (especially with_children)
• there are many confusingly similar APIs to insert children, although push_child and the like are somehow missing
• orphan handling does not work correctly right now, and I had to be mindful of this in my design
• there's no documentation on what's inherited and how
• there's no way to configure what's inherited

Layout

• the flex-box layout algorithm has so so many parameters and lacks internal structure, so it's very hard to build a mental model of what's happening and why
• changing values sometimes silently does nothing due to overriding nature of other parameters (e.g. setting Transform when layout is occurring)
• layout changes require jumping back and forth between segments of code
• it's extremely hard to correctly predict what changes to layout parameters will do, and virtually impossible to debug why the layout algorithm is doing what it is
• layout does not seem to be a pure function of window size etc, resulting in bugs like this after resizing
  

Text

• text rendering is ugly and buggy in ways that are hard to track down (check the 4s on the first image linked)
• why can't I make new lines :(

Missing features

• lack of entity inspection is still annoying (Add inspect command for debugging entities and their components #1467)
• hot reloading component values is extremely important for rapid iteration but only available in 3rd party form with bevy_inspector_egui
• no animations or state machines
• ~- no tick timers (Native Tick Timers #1633) to build simple animations or state machines from scratch~~
• 1st party audio basically doesn't exist
• no built in widget selection logic
• all the standard widgets are missing
• no edge / fill distinction
• no basic geometric shapes other than rectangles
• no styling or theming

[Byteron]

storing multiple handles on components is extremely natural, and is a great way to handle hover / pressed / normal reactivity. See responsive_buttons system

this is something that ultimately should be handled by the engine itself.
Ideally I can just drag and drop images or style resources into the editor's inspector of the button without having to write any code to handle when which image / style is displayed whatsoever.

I am sure this applies to a number of other things mentioned in this summary, but I see how it would still be beneficial to solve it for writing the editor in the first place. :P

[alice-i-cecile]

there's no way to simple way to bind a keyboard input to a button, resulting in code duplication

@plof27 notes that this pattern is actually achieved really nicely by allowing backwards propagation of actions in the action-mapping abstraction; allowing you to automatically e.g. show the jump button as depressed when space is hit.

[alice-i-cecile]

Three years later, here's what I think about bevy_ui in Bevy 0.14!
https://elk.zone/mastodon.gamedev.place/@alice_i_cecile/112837834583301070

Much better, but still not perfect!