[Developers] Network page map

[Nerivec]

A lot of early feedback is about the Network page!
The old frontend "map" has lots of problems, and is basically unusable on larger networks.
The "data" display in WindFront attempted to remedy this by providing a more data-oriented display, less "fancy", but far more complete.
A "map" was however re-added due to popular demand. It uses more of the underlaying data to provide a "fuller" view of the network using connections and nodes, but it needs improvements/optimizations.

We'd like to gather specific ideas here, on how to best handle the Network page map to address previous concerns (as much as possible), and to maintain a good network overview in a "map-ish" form.

CC: @sjorge @sarthurdev @mundschenk-at @MickLesk @Koenkk

Please refrain from posting specific bugs/issues in this thread.

[MickLesk]

What about to refactor this like the build-in HomeAssistant "Bluetooth"-Map ?
https://rc.home-assistant.io/blog/2025/05/26/release-20256/#understanding-your-bluetooth-network

Or like the Zigbee2Mqtt-Networkcard (HACS) from @azuwis, although I'm not sure if he pulls the card from Z2M or if he just pulls the data from there and renders the card with vue-d3 live. I can't say what it looks like in live operation with 100+ devices, unfortunately I only have 20 because the rest are running on Matter & Somfy.
https://github.com/azuwis/zigbee2mqtt-networkmap

[Nerivec]

The bluetooth map is very limited in terms of what data it displays (bluetooth connections are "basic").
ZigBee needs to have the concepts described in the Help section of the current map:

I think the addition of target/source in the design (top/bottom attach) is pretty helpful, but it has created some confusion so far since it's very different from the old map.

From what I've seen, anything with d3 forces is unusable when the device count is higher. It's also not stable in several scenario.

react-flow supports more complex layouts (than current Dagre), but they need to be investigated in more details (might need customization):
https://reactflow.dev/learn/layouting/layouting

[bartkummel]

First, get the basics correct:

• The map should load in a reasonable amount of time. The old map already took long to load. It shouldn't get any longer than that, but preferably faster.
• Currently, it's not possible to zoom out to see the entire map, if your network is beyond a couple of devices. This is essential, the map is useless if you can only see a part of it.
• The map should display the friendly names instead of the IEEE address by default.

Then, a couple of thoughts for further improvement:

• You seem to head in the direction of showing more rather than less information. I think it should be the other way around: the old map already shows too much information. The number of links grows exponential with the size of the network, it gets crowded and overwhelming quickly. It should at least be possible to dynamically show/hide certain data so a user can try to find a balance between information and clutter.
• In the example above, you show multiple lines between two devices, e.g. one for the "parent" link, another for the "sibling" link. While that might be more correct, it only makes the graph more cluttered. Instead, show a single line. If you want to go fancy, you might give it both colors (for "parent" and "sibling" in the example). But for most use cases, it is enough to know that there is a link. (You could consider showing the link type(s) in a popup on hover.)
• You might consider hiding certain details depending on the zoom level. E.g. the LQI number: only show it when zoomed in enough, like in the small example you show above. In larger networks, it only adds clutter if zoomed out. Again, you could consider showing it in a popup when hovering the link.
• In the old map, I frequently used the checkboxes to show/hide certain links. E.g., showing both parent/child and sibling links quickly becomes very cluttered. I then disabled the siblings, although that wasn't ideal either. It would be great to have more advanced filtering options, e.g. to only show the link with the highest LQI for any device.
• Put the coordinator in the center of the map. Most lines will go there, so this will almost always be the most compact way to display the map.
• I'm not sure whether I like the device images. I think sticking with simple shapes (square, circle, etc.) will make it easier on the eyes.

[Nerivec]

The map should load in a reasonable amount of time.

This is dependent on the ZigBee network behind, nothing the frontend can do about it (except for the actual display time, which is minimal compared to it in any case). The number of devices, the stability of devices, the nesting of devices all affect the process... I've seen cases where it take over 20 minutes to complete the requests...

zoom & information

The larger the network, the more unusable the current "maps" are in all the attempts I've made. I am currently not seeing a way to display this properly, even with removing information / tweaking zoom. 30 devices or more has already too much stuff to display to get a descent overview (else it's just so basic, it's pointless). That's why I initially looked into a "data-oriented" display (far better at displaying details, but of course, less "fancy"...).

There is also the problem of users having a different "use" for the map. You can use it to identify crowding, isolation, nesting, good/bad LQI zones, overall network quality… each would practically need a different display.

Also, a lot of this is constrained by the available libraries that draw these maps, and what they are capable of; writing a custom library is a whole other beast... That would blow up my Open Source time 😅

Friendly name display was corrected already.

[bartkummel]

This is dependent on the ZigBee network behind, nothing the frontend can do about it (except for the actual display time, which is minimal compared to it in any case). The number of devices, the stability of devices, the nesting of devices all affect the process...

My impression is that the new map takes longer to load than the old one. But I'll try to time it to be sure.

That said: one thing that annoys me with the old map, is that the data is lost as soon as you navigate away and then return. Would be great if the new UI gives users control of how the network data is cached.

The larger the network, the more unusable the current "maps" are in all the attempts I've made. I am currently not seeing a way to display this properly, even with removing information / tweaking zoom.

That's not true, the old map is way better than the current attempt. You can at least try to come closer to the old map.

One additional idea to make the map more manageable, would be to choose a device to start from. If you start from the coordinator, you'll get a map of the whole network. But if you start from any other device, you would only get the devices between that device and the coordinator.

That's why I initially looked into a "data-oriented" display (far better at displaying details, but of course, less "fancy"...).

It's not about "fancyness"! A good visual presentation can be interpreted much easier than a data table. The old network map helped me diagnose issues in my network a couple of times, and I don't know if I would have been able to diagnose those with only a table.

There is also the problem of users having a different "use" for the map. You can use it to identify crowding, isolation, nesting, good/bad LQI zones, overall network quality… each would practically need a different display.

True, that's why it will pay off to make it more flexible. I already mentioned how the old map allows you to show/hide links based on type. I can think of more ways to make the display more flexible.

Also, a lot of this is constrained by the available libraries that draw these maps, and what they are capable of; writing a custom library is a whole other beast...
The solution used in the old map seems to do a decent job.

I'd like to end this message by saying that I really appreciate all the hard work on this. My remarks and ideas are only meant to help you make it better. I hope it's helpful and I fully understand that there are limitations here and there.

[Nerivec]

My impression is that the new map takes longer

If you check "routes" it will create more requests (more time). The original frontend is always false on that option.

data is lost as soon as you navigate away and then return

It's in the page store, so, yes, if you navigate away from the frontend, it's lost. It's a rather large piece of data to cache, it would likely make more sense to cache it on the Z2M side (broker retained option maybe?). Though that does introduce issues like stale management (on paper, the map is pretty much stale the moment it's completed, as the network could already have changed...).

The original frontend map has a more basic approach to links, so, it's easier to read, but that lack in relationship details is a big miss in my opinion (it could be very helpful in debugging routing issues and prevent confusion - once users are used to it). With d3 forces in the mix and browser quirks, it's very quickly unusable though, adding more details with the original map design would further that problem.
Here's an extreme example of what we need to "better"...

For sure, the current WindFront prototype is only viable for small networks and will require a lot of work, might even need to look into a different library (or at least a different placement algorithm)...

I've thought about the "parent-derived" approach, either by using it directly in the request (instead of starting on Coordinator), or by splitting the map into smaller ones. But each introduce new limitations. First one needs to be implemented on Z2M side, and will miss relationships due to how the requests work. Second one will omit some relationships too due to the split.

[thk-socal]

As an end user with very large networks the current network map is more novelty than useful. It would be AMAZING, if I could start at any device and spiral out a set amount of levels. BUT, what about a completely different approach? How about a living network map that builds over 48 hours? Perhaps creates a network.yaml file that is the network that is constantly changing based upon a slow query update time. With 48 hours of time, there would be little additional traffic and the network map could be loaded instantly from the generated file. The interface could present a refresh now option, that would still use a similar approach as current to update the stored file and be able to display as desired.

As for the staleness of 48 hours, we could highlight the links in different colors based upon the last time the network diagram was refreshed for that link.