Update install document for existing grafana instance

[lightbeem]

Would help to update install documents to cover scenario where there may already be a grafana container running. I've tried several tweaks to the docker compose file but my experience (or lack of) is holding me back.

[edasque]

I don't know if you're still looking for help with this. It's not trivial but it's not all that complicated though it does require some docker & Grafana knowledge and does involve a few manual steps and might make updating a little more complex

1. Duplicate the docker-compose.yml file to docker-compose.existing-grafana.yml, remove the teslamate Grafana definition. Modify the docker-compose with your existing Grafana to add a network to the Grafana container that's shared with the Postgres container from teslamate. I personally have Grafana on traefik & TeslaMate-backend, teslamate on traefik and teslatemate-front-end and teslamate-backend and the rest on teslamate-back-end. That way your existing Grafana container can talk to the teslamate pg database
2. Create a datasource in Grafana called TeslaMate, the host is teslamate_database-teslamate_1 for me (check your full PG container name. Set the user and password, disable pls/ssl mode, version 13 - leverage https://github.com/adriankumpf/teslamate/blob/master/grafana/datasource.yml to see what information is filled - Save and test. Note that while the test worked for me initially, it actually wasn't able to connect to my host because I didn't have my docker containers networking set up correct - for some reason it still tested ok.
3. Create a TeslaMate folder via the Grafana UI
4. Add the plugins in Grafana that are listed here by running docker exec -i -t grafana /bin/bash and using https://github.com/adriankumpf/teslamate/blob/master/grafana/Dockerfile as a reference of which plugins to install (for example grafana-cli plugins install natel-discrete-panel 0.1.0 )
5. Import the 20 or so dashboards from https://github.com/adriankumpf/teslamate/tree/master/grafana/dashboards json files, choose TeslaMate as the folder
6. Configure the Grafana URL from teslamate settings

There are downsides to that approach, mostly it's pretty manual today. I think the main one is that when Teslamate is updated, you'll need to watch what is updated and re-import the Grafana dashboards by hand every time (there probably is a better way to do this via command-line though.

@adriankumpf I'd love some feedback if you see major gaps in my approach

[ghost]

New Answer (now with a script)

You still need to edit the docker-compose.yml of your Grafana container (once) like this before running the script

environment:
   - GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS=natel-discrete-panel,pr0ps-trackmap-panel,panodata-map-panel,natel-plotly-panel
command: >
  sh -c "grafana-cli --pluginsDir /var/lib/grafana/plugins plugins install pr0ps-trackmap-panel 2.1.2 &&
         grafana-cli --pluginsDir /var/lib/grafana/plugins plugins install natel-plotly-panel 0.0.7 &&
         grafana-cli --pluginsDir /var/lib/grafana/plugins --pluginUrl https://github.com/panodata/panodata-map-panel/releases/download/0.16.0/panodata-map-panel-0.16.0.zip plugins install grafana-worldmap-panel-ng"
volumes:
  - ./dashboards_config:/etc/grafana/provisioning/dashboards
  - ./dashboards:/dashboards
  - ./dashboards_internal:/dashboards_internal
  - ./dashboards_reports:/dashboards_reports

Now here's the little bash script that does everything (minus the edits to your docker-compose.yml). Simply put it anywhere and chmod +x or chmod 755 it (whichever floats your boat). It will install subversion if it's not installed, create the folders, download the needed files from GitHub and restart your Grafana container.
Important: Edit the GRAFANA_PATH variable in line 5 to reflect the correct path in your system.

#!/bin/bash

### Variables to edit (maybe)
# Path to the docker-compose.yml of your Grafana instance
GRAFANA_PATH=~/docker_container/grafana
# Github URL of the folder containing TeslaMate's dashboards (charge-level.json, charges.json, charging-stats.json, ...)
TESLAMATE_DASHBOARDS_URL=https://github.com/adriankumpf/teslamate/tree/master/grafana/dashboards
# Github URL of the folder containing TeslaMate's dashboards.yml
TESLAMATE_DASHBOARDS_CONFIG=https://github.com/adriankumpf/teslamate/tree/master/grafana
### No more editing after this line

# Checking if subversion is installed and installing it if not
SUBVERSION_INSTALLED=$(dpkg-query -W --showformat='${Status}\n' subversion|grep "install ok installed")
echo Checking for subversion: $SUBVERSION_INSTALLED
if [ "" = "$SUBVERSION_INSTALLED" ]; then
  echo "No subversion installed. Setting up subversion."
  sudo apt --yes install subversion
fi

# Removing the folders if they exist
rm -rf $GRAFANA_PATH/dashboards/ $GRAFANA_PATH/dashboards_config/ $GRAFANA_PATH/dashboards_internal/ $GRAFANA_PATH/dashboards_reports/

# Creating the needed folders
mkdir -p $GRAFANA_PATH/dashboards $GRAFANA_PATH/dashboards_config $GRAFANA_PATH/dashboards_internal $GRAFANA_PATH/dashboards_reports

# Replacing tree/master with trunk in the URLs to be able to use svn export
TESLAMATE_DASHBOARDS_URL=$(sed 's!tree/master!trunk!g' <<<"$TESLAMATE_DASHBOARDS_URL")
TESLAMATE_DASHBOARDS_CONFIG=$(sed 's!tree/master!trunk!g' <<<"$TESLAMATE_DASHBOARDS_CONFIG")

# Exporting the TeslaMate dashboards to the local dashboards folder
svn export --force $TESLAMATE_DASHBOARDS_URL $GRAFANA_PATH/dashboards/

# Exporting the TeslaMate dashboards.yml to the local dashboards_config folder
svn export --force $TESLAMATE_DASHBOARDS_CONFIG/dashboards.yml $GRAFANA_PATH/dashboards_config/

# Moving the internal dashboards to dashboards_internal
mv $GRAFANA_PATH/dashboards/internal/* $GRAFANA_PATH/dashboards_internal/

# Moving the reports dashboards to dashboards_reports
mv $GRAFANA_PATH/dashboards/reports/* $GRAFANA_PATH/dashboards_reports/

# Removing the left over, empty folders
rm -rf $GRAFANA_PATH/dashboards/internal/ $GRAFANA_PATH/dashboards/reports/

# Restarting the Grafana container
docker-compose -f $GRAFANA_PATH/docker-compose.yml up -d --force-recreate

If you did everything right, you should find the 3 folders (Internal, Reports, TeslaMate) with their respective dashboards in your Grafana instance.

In case of a new TeslaMate version, simply run the script again.

Old Answer (the manual way)

So I kinda automated the process (not really) or at least made it as simple to update to a new version as possible:

For the purpose of this guide, we assume that the docker-compose.yml of your Grafana stack sits in a folder called /your-grafana-docker-folder/ (so the full path would be ~/your-grafana-docker-folder/docker-compose.yml) and we assume that you've already created a datasource called "TeslaMate" in Grafana

1. Create the following folders next to your docker-compose.yml

   1. dashboards
   2. dashboards_config
   3. dashboards_internal
   4. dashboards_reports

   It should look like this now:

   choque@host:~/your-grafana-docker-folder$ ls -Gg
   total 20
   drwxrwxr-x 2 4096 Aug  3 11:06 dashboards
   drwxrwxr-x 2 4096 Aug  3 11:06 dashboards_config
   drwxrwxr-x 2 4096 Aug  3 11:00 dashboards_internal
   drwxrwxr-x 2 4096 Aug  3 11:00 dashboards_reports
   -rw-r--r-- 1 2179 Aug  3 11:06 docker-compose.yml

2. Copy https://github.com/adriankumpf/teslamate/blob/master/grafana/dashboards.yml to /your-grafana-docker-folder/dashboards_config/

   It should look like this now:

   choque@host:~/your-grafana-docker-folder/dashboards_config$ ls -Gg
   total 4
   -rw-r--r-- 1 635 Aug  2 11:50 dashboards.yml

3. Copy all json-files from https://github.com/adriankumpf/teslamate/tree/master/grafana/dashboards/internal to /your-grafana-docker-folder/dashboards_internal/

   It should look like this now:

   choque@host:~/your-grafana-docker-folder/dashboards_internal$ ls -Gg
   total 80
   -rw-r--r-- 1 28161 Aug  2 11:50 charge-details.json
   -rw-r--r-- 1 50433 Aug  2 11:50 drive-details.json

4. Copy all json-files from https://github.com/adriankumpf/teslamate/tree/master/grafana/dashboards/reports to /your-grafana-docker-folder/dashboards_reports/

   It should look like this now:

   choque@host:~/your-grafana-docker-folder/dashboards_reports$ ls -Gg
   total 16
   -rw-r--r-- 1 12823 Aug  2 11:50 dutch-tax.json

5. Copy all json-files from https://github.com/adriankumpf/teslamate/tree/master/grafana/dashboards to /your-grafana-docker-folder/dashboards/

   It should look like this now:

   choque@host:~/your-grafana-docker-folder/dashboards$ ls -Gg
   total 508
   -rw-r--r-- 1  6138 Aug  2 11:50 charge-level.json
   -rw-r--r-- 1 34606 Aug  2 11:50 charges.json
   -rw-r--r-- 1 50246 Aug  2 11:50 charging-stats.json
   -rw-r--r-- 1 28386 Aug  2 11:50 drives.json
   -rw-r--r-- 1 30866 Aug  2 11:50 drive-stats.json
   -rw-r--r-- 1 32483 Aug  2 11:50 efficiency.json
   -rw-r--r-- 1 26856 Aug  2 11:50 locations.json
   -rw-r--r-- 1  7520 Aug  2 11:50 mileage.json
   -rw-r--r-- 1 46571 Aug  2 11:50 overview.json
   -rw-r--r-- 1 22059 Aug  2 11:50 projected-range.json
   -rw-r--r-- 1 13829 Aug  2 11:50 states.json
   -rw-r--r-- 1 27114 Aug  2 11:50 statistics.json
   -rw-r--r-- 1 32783 Aug  2 11:50 timeline.json
   -rw-r--r-- 1 78283 Aug  2 11:50 trip.json
   -rw-r--r-- 1 19610 Aug  2 11:50 updates.json
   -rw-r--r-- 1 21609 Aug  2 11:50 vampire-drain.json
   -rw-r--r-- 1  5531 Aug  2 11:50 visited.json

6. Add the following to your docker-compose.yml (from https://github.com/adriankumpf/teslamate/blob/master/grafana/Dockerfile)

   environment:
      - GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS=natel-discrete-panel,pr0ps-trackmap-panel,panodata-map-panel,natel-plotly-panel
   command: >
     sh -c "grafana-cli --pluginsDir /var/lib/grafana/plugins plugins install pr0ps-trackmap-panel 2.1.2 &&
            grafana-cli --pluginsDir /var/lib/grafana/plugins plugins install natel-plotly-panel 0.0.7 &&
            grafana-cli --pluginsDir /var/lib/grafana/plugins --pluginUrl https://github.com/panodata/panodata-map-panel/releases/download/0.16.0/panodata-map-panel-0.16.0.zip plugins install grafana-worldmap-panel-ng"
   volumes:
     - ./dashboards_config:/etc/grafana/provisioning/dashboards
     - ./dashboards:/dashboards
     - ./dashboards_internal:/dashboards_internal
     - ./dashboards_reports:/dashboards_reports

7. Restart your Grafana container via docker-compose up -d --force-recreate

If you did everything right, you should find the 3 folders (Internal, Reports, TeslaMate) with their respective dashboards in your Grafana instance.

In case of a new TeslaMate version, simply replace the files in the folders (dashboards, dashboards_config, dashboards_internal, dashboards_reports) with the ones from the new version, edit the environment and command part of your Grafana's docker-compose.yml (if something changed in https://github.com/adriankumpf/teslamate/blob/master/grafana/Dockerfile) and restart the container.

[maxiride]

I've stumbled upon this discussion while searching setups with an existing grafana instance.
While the scripts and suggestions provided do work, I was wondering why Teslamate dashboards are not published to the grafana registry? It would allow importing them by URL and subsequently update them way more easily.

Except from the needed plugins, are there other technical limitations or reasons to not streamline the setup for existing grafana instances?

[JakobLichterfeld]

We provide a Script dashboard.sh for manual installations.

NixOS installs support existing dashboards and datasources since #4870

I was wondering why Teslamate dashboards are not published to the grafana registry?

TeslaMate is self-hosted without (minimal) external dependencies. The needed plugins are a great point against Grafana registry.

It would allow importing them by URL and subsequently update them way more easily.

External registry is an external dependency, no control who has which dashboard, no DevOps pipeline, Dashboards need to match TeslaMate version, etc.

The beauty of self-hosting, is: you can use the provided docker image, the provided script, single dashboards etc. Or use NixOS module which has no limitations.

If there is something which can be improved, PRs are always welcome