Grafana HTTPS Breaks the Powerflow Animation

[cwagz]

I figured out how to get Grafana working with HTTPS and my certificates. The problem is this appears to break the powerflow animation. Is there any way to get this working? I can go back to HTTP for Grafana, I was just tired of my password manager giving me "unsecure site" warning every time I went to login to Grafana.

[jasonacox]

Yes, this is tricky. I haven't added automated support for this because it is hit-or-miss with the various browsers and I ran out of motivation to track it all down. The issue is that the animation is an iFrame in Grafana with some Javascript that loads the Tesla animation via proxy through pypowerwall. The error you are getting is because it is non-HTTPS.

If you are interested in making it work, here is what would be needed to move it to HTTPS. Keep in mind that this is beta and may still not work (I had some odd cases where it refused to ender based on setup, browser, or something else). Please feel free to poke at it and help us all figure it out. 😁

To activate HTTPS (TLS), you will need to edit your pypowerwall.env file and add this line:

PW_HTTPS=yes

You will need to edit telegraf.conf to change http to https on these lines:

                "http://pypowerwall:8675/aggregates",
                "http://pypowerwall:8675/soe",
                "http://pypowerwall:8675/strings",
                "http://pypowerwall:8675/temps/pw",
                "http://pypowerwall:8675/freq",
                "http://pypowerwall:8675/pod"

And then restart everything:

docker-compose -f powerwall.yml up -d

A test should be available if you load https://10.2.3.4:8675/ - Key: replace 10.1.2.3 with the actual IP address of the host running the Dashboard containers. To test iFrame outside of Grafana, you can use this built-in test page: http://10.2.3.4:8675/example.html

[cwagz]

This is good information. I will need to give it a shot at some point. For now, I just moved back to http.

[cwagz]

I think I see what you mean. With everything using https I get a different error. Likely due to the browser not accepting the connection to the iFrame. I would probably need to get pypowerwall to use a certificate that is trusted by my browser.

[cwagz]

I have been using pfSense for a long time. It includes an easy way to setup a CA and then manage server certificates. Getting the certs trusted involves ensuring you issue compliant server certs and also that you import the CA into the trusted store on the device you are using (PC / Phone.)

You must give the server certificate a FQDN and IP entry for it to be trusted. Some things will no longer accept server certificates that have an expiration date longer than 398 days. This is mostly occurring with iOS now although it still allowed Grafana to work after the initial warning.

I am sure this can all be done with OpenSSL, but I have never had to do it. pfSense make the process really easy.

https://docs.netgate.com/pfsense/en/latest/certificates/ca.html
https://docs.netgate.com/pfsense/en/latest/certificates/certificate.html

[cwagz]

I was messing around with this some more.

I setup a reverse proxy using HAProxy and a legit certificate from Let's Encrypt. This was all done in pfSense using the HAProxy and ACME packages. By using SSL offloading I can have the proxy connect securely to the client browser and then pass the traffic to Grafana via HTTP.

grafana.env additions required:

GF_SERVER_DOMAIN=powerwall.wags6.com
GF_SERVER_ROOT_URL=https://powerwall.wags6.com
GF_SERVER_ENFORCE_DOMAIN=true

This allowed me to connect to the Powerwall dashboard remotely using SSL. I had forgotten that the powerflow animation is just an iframe pulling the data straight from the Powerwall. Since I was now remote, the iframe was looking for powerwall.wags6.com:8675 and could not get there.

What I ended up doing just to prove that it could work, was creating another reverse proxy in HAProxy for the pypowerwall port 8675 only. This was also enabled with SSL Offloading. Once setup I have a fully functional, reverse proxied dashboard available from anywhere via the internet.

I am looking for some opinions on the security / risk of this. The only way to make the dashboard work was to expose the powerwall animation to the world. Anyone browsing to powerwall.wags6.com:8675 will see my powerflow animation. What do you guys think? It's too bad there is no way to have Grafana get the animation and display it natively as part of the dashboard. It's like we need a plugin to grab the animation locally and then just display it.

I am also assuming that it is safe to publish Grafana dashboards on the internet. From my research it seemed like this was safe as long as it was protected behind a proxy. I am also using a strong password.

[mcbirse]

Interesting experiment @cwagz !

This would be a really nice addition if it can be done securely. I wanted to access mine from the internet as well, but as I wasn't sure or confident that it was entirely secure and since I was accessing from known IP addresses only, I have setup remote access just from those IPs.

I am looking for some opinions on the security / risk of this. The only way to make the dashboard work was to expose the powerwall animation to the world. Anyone browsing to powerwall.wags6.com:8675 will see my powerflow animation. What do you guys think?

One thing to keep in mind is I believe the pypowerwall proxy will actually proxy any API request to your gateway that is not in the standard API allow list (so not just / for the powerflow).

So couldn't opening up port 8675 to the world mean potentially someone could control or change settings on your gateway?

If the API request is in the allow list I think it is safe, as it would only poll/query data.

But what if there is an API request we don't know about.... not in the allow list - it will be proxied directly to your gateway. Maybe it would need to be a POST request to change something, and I don't think(?) that would be proxied... but it seems like a risk that I personally wouldn't like to take.

[jasonacox]

I have also thought about the best way to make this more remotely viewable. Nice work @cwagz . @mcbirse is also right about pypowerwall's proxy. The good news is that it will respond with ERROR for any API call not on the allow list and it only handles GET requests (POST will not proxy). However, the allowed APIs would allow anyone to see your system metrics, vitals, and serial numbers. As you figured out, the powerflow animation makes javascript calls back to the pypowerwall proxied metric APIs to render the flows.

Some possible ways that could allow a secure way to see the data outside our home networks:

• Expose a revised pypowerwall proxy (server.py) that has an even more limited set of APIs needed for the powerflow. Removing calls that expose serial number, vitals or other sensitive data. Tesla has attempted to make it impossible to make changes to the Powerwall without write (installer) token access, but as @mcbirse points out, if there is an unknown flaw in the Powerwall read-only APIs we use, that could still be a potential risk. Mitigation is that pypowerwall proxy is very explicit on API calls (no passthrough URI data, just exact matches only or you get an ERROR response).

• Create a Grafana plugin (as @cwagz mentions) that essentially does the above with Grafana being the proxy.

• VPN/SSH tunnel - One way is the tunnel approach - Create a public ssh endpoint on your router mapped to your linux host with required ssh key (eg. port 8822). And when remote, use an ssh tunnel. I'm not sure how well this would work with HTTPS but it does work with the base HTTP setup. Of course, exposing ssh has its own risks and mitigations you should take.

  ssh -N -p 8822 username@my.home.endpoint.com -L 9000:10.1.2.3:9000 -L 8675:10.1.2.3:8675
  # Access grafana via opening http://localhost:9000 on your remote machine

[cwagz]

@mcbirse and @jasonacox - Thank you for the heads up on the potential security risk. I am no longer allowing traffic from outside my network to hit the pypowerwall instance.

I have IPsec and Tailscale VPNs running on pfSense and tried to get a configuration where my phone would stay connected 100% of the time. This works well if you are on WiFi outside of the home but absolutely kills the battery if connected via cellular. The real drive for me to expose the dashboard is to eliminate the hassle of manually connecting to the VPN when I want to check the dashboard via cellular.

I removed the rule allowing incoming WAN connections on port 8765 but left the reverse proxy in-place. Now Grafana is able to still use the reverse proxy to reach the animation as long as the client device is on the local network. This is kind of working out well as now the dashboard is fully functional if I am on VPN or local. If I pull the dashboard up from outside, the powerflow animation just does not load but the rest of the dashboard works.

I am going to search around and see if some sort of Grafana plugin already exists that can proxy the iframe data (maybe this is futile).

Would something like this work?

https://grafana.com/grafana/plugins/yesoreyeram-infinity-datasource/