Visual exploration of event streams

[jakubpnf]

Together with my friend @marusek we developed a POC of integrating Materialize+dbt+Redpanda with our data exploration tool we've been experimenting for some time - nPond.

To visualize data with nPond:

1. We translated a DAG that can be consumed by nPond into a Materialize query (view)
2. The query enriches events with metadata needed for visualisation
3. Enriched data is written to a Redpada sink.
4. nPond reads from the topic and draws events
   

The query and sink: jakubpnf@ff94823
The DAG for nPond:

{
  id: 0,
  childNodes: [
    {
      id: 1,
      label: 'Air',
      childNodes: [
        {
          id: 3,
          label: 'US',
          childNodes: null,
        },
        {
          id: 5,
          label: 'China',
          childNodes: null,
        },
        {
          id: 4,
          label: 'Rest of the world',
          childNodes: null,
        },
      ],
    },
    {
      id: 2,
      label: 'Ground',
      childNodes: null,
    },
  ],
}

[christineberger]

Wow, this is really cool @jakubpnf! 🪐