Add analytics to control panel

[depatchedmode]

Description: We want to track success and failure at various points along the path of onboarding and activating as an Everywhere Computer user.

Proposed Onboarding funnel:

1. Visit to the landing page
2. Click the "beta intake" button
3. Email validation sent
4. Account created
5. Homestar installation started
6. Homestar installation completed
7. First workflow run

We'll also want to look into

Job Stories

2. Once a developer has created an EC account:
   • As Fission
     • Monitoring Growth: We want to be alerted that a new account has been created, so that (1) we can get a sense of the pace at which new accounts are being created and (2) we can go review their intake form answers.
   • As a developer
     - Consent: I want to be informed about the purpose of the analytics collection and be given the option to opt-in or out, so that I feel my digital rights are being respected by the Everywhere Computer team.
3. When a developer has finished installing and running their Homestar node:
   • As Fission
     • Monitor Progress: We want to know they successfully achieved that milestone, so that we can measure dropoff in our onboarding process
     • Track Basic Usage: We want some basic telemetry collected through the control panel about EC usage, so that we can track changes in usage over time

Additional Details & Quetsions

• Can we fire events to plausible from integrations with other systems? Eg. Tracking when folks actually join the Discord. Integrating into a CLI at some point.
• How does Plausible handle local-first/offline? We can wrap it in the Analytics framework to support offline aggregation of events: https://github.com/ian/analytics/tree/main/packages/plausible
• How do we correlate user activity across devices? The point of Plausible is to not do that. We'll just want to consider how we construct events and the funnel to ensure that we don't double count folks who setup a second device.

[avivash]

@depatchedmode is First workflow run what we want here or just Workflow run? Moving to the new auth flow, we won't have a great way to detect if the user is running a workflow for the first time. I can track it using localforage, but that won't prevent against the user logging in on a different device and triggering the event again 🤔

[avivash]

@zeeshanlakhani i've got the other events firing in JS, but for Homestar installation started and Homestar installation completed is it possible to use https://github.com/goddtriffin/plausible-rs to push those events to plausible?

[depatchedmode]

The event we can track is Workflow Run. And then the analysis I'll end up doing is see which accounts have done that at least once.

For 'Homestar Installation Started' and 'Homestar Installation Complete' we can just get people to self-report in the control panel, rather than try to fire things natively from the homestar module.

I think it's fine for us to assume that all analytics originate from the plausible instance in the Control Panel. Only question I have is "what happens when the user isn't online?"

[avivash]

@depatchedmode

The event we can track is Workflow Run. And then the analysis I'll end up doing is see which accounts have done that at least once.

sounds good, so I should basically send the user email along with all these events, except for Visit to the landing page because that happens before the user logs in, right?

I think it's fine for us to assume that all analytics originate from the plausible instance in the Control Panel. Only question I have is "what happens when the user isn't online?"

I guess we create a plausible homestar function to handle those cases for us and send the report when it comes back online? 😅 (future problem probably)

[depatchedmode]

@avivash Let's discuss how Posthog changes this in the product sync this afternoon: https://posthog.com

[avivash]

@depatchedmode i've setup posthog analytics for Visit to the landing page. Do you want me to block it after the first time using localstorage or something until we have accounts setup or should we just send every visit? It shows videos of how the user interacted with the site, which is kind of cool https://us.posthog.com/insights/dNLF5753