Test: ui element selector

[justschen]

• windows
• macOS
• linux

Complexity: 4

author: @justschen

---

Summary

This iteration, I worked on a UI element selector in the Simple Browser effort to provide developers with more means of vibe coding and simplifying web development. By selecting an item and attaching it to chat, users can easily pinpoint what element they are interested in.

Test Steps:

1. Spin up a website's localhost. This can be by creating a new website with agent mode or using an existing website you can deploy locally.
2. Navigate to the Simple Browser (command palette -> simple browser: show).
3. Enter in the localhost URL.
4. You can also start the simple browser by asking chat something along the lines of "start the browser at http://10.0.0.66:8080" or "i want to see a preview of http://10.0.0.66:8080"
5. Note that there is now a "select element" tool bar on the bottom left of the Simple Browser.
6. Attach some items to chat.
7. Note that the attached items are correct and the screenshot is correct. You can click on either attachments to view them in editor.
8. Feel free to try this out and try different scenarios.
9. Please test all the buttons! (collapsing, cancelling, closing out, etc)

Scenarios:

1. You don't want to see this cool new toolbar in your simple browser?

• fine. turn off chat.sendElementsToChat.enabled. (make sure this actually hides the toolbar). You can also collapse the toolbar with the chevron.

2. I'm unsure how useful attaching the image to chat actually might be.

• Between (1) attaching the image of the element, (2) attaching an image of the entire webpage and the element circled, or (3) no image at all, i'm not really sure what is best.
• To compare the (1) and (3) cases you can disable chat.sendElementsToChat.attachImages to attach only html and css.
• when attaching the image, you may have to prompt chat with something along the lines of "this is what i currently have in my {element/hero/div/etc.}"

Feel free to add notes on whatever you think led to the best results if you have any!

Things to Note:

• We are using chrome dev tools behind the scenes, so unfortunately the overlay UI is not as pretty :(
• This should also work with multiple simple browsers open (in different windows, etc).
• Next iteration, will try to make this work in external browsers as well.
• There aren't a lot of things in core that need this kind of toolbar, but if you have thoughts and feedback on the toolbar, the button options we provide, or anything like that, please don't hesitate the be harsh. i kinda just went with whatever i thought of first, since this was intended to be a proof of concept anyways.

Please provide any feedback at all regarding the UI/UX! Thanks for testing 🚀