• Prometheus and Grafana
  • Installing Grafana
  • Installing Prometheus
  • Adding Data Source
  • Additional Panels
    • vLLM Accumulated Generation Tokens
  • References

Please refer to Grafana Download Section for the latest version.

First, install the required dependencies:

sudo apt-get install -y adduser libfontconfig1 musl
wget https://dl.grafana.com/oss/release/grafana_11.5.1_amd64.deb
sudo dpkg -i grafana_11.5.1_amd64.deb

Then, start the Grafana service if you are using systemd:

### NOT starting on installation, please execute the following statements to configure grafana to start automatically using systemd
 sudo /bin/systemctl daemon-reload
 sudo /bin/systemctl enable grafana-server
### You can start grafana-server by executing
 sudo /bin/systemctl start grafana-server

Otherwise, you can start it manually:

sudo service grafana-server start

You can change the default port from 3000 to a custom one by editing the grafana.ini configuration file.

This is an example script to download Prometheus on Linux. Please refer to Prometheus Donwload Section and Getting Started for the latest version.

wget https://github.com/prometheus/prometheus/releases/download/v3.2.0-rc.1/prometheus-3.2.0-rc.1.linux-amd64.tar.gz
tar xvfz prometheus-3.2.0-rc.1.linux-amd64.tar.gz
cd prometheus-3.2.0-rc.1.linux-amd64

Launch Prometheus with the following command:

# default port is 9090
./prometheus --config.file=../prometheus.yaml --web.listen-address=:<port> --storage.tsdb.retention.time=<time>

Futher Reading: How to Change Prometheus Port - A Step-by-Step Guide

Also, please note this

--storage.tsdb.retention.time: How long to retain samples in storage. If neither this flag nor storage.tsdb.retention.size is set, the retention time defaults to 15d. Supported units: y, w, d, h, m, s, ms. (ref: https://prometheus.io/docs/prometheus/latest/storage/#operational-aspects)

You can access Prometheus Web UI at <prometheus_ip:port>/targets.

Before upload grafana.json provided in vLLM Docs, you need to modify the data source uid in the json file. Just replace all edx8memhpd9tsa in the json file with your own data source uid.

You can find the data source id in the URL when you are editing the data source in Grafana. (Connections -> Data Sources)

Grafana Web UI → Dashboard → Add Panel → Query

increase(vllm:generation_tokens_total[$__range])

# or with model_name
increase(vllm:generation_tokens_total{model_name="$model_name"}[$__range])

You can get the panel like this:

• vLLM Docs - Prometheus and Grafana
• Day29 - vLLM 實作篇 PART5：監控 vLLM