Skip to content

Commit 0e0dbd6

Browse files
JaimeSalasJaimeSalas
committed
added configuring Prometheus demo
1 parent 7be6442 commit 0e0dbd6

File tree

5 files changed

+255
-1
lines changed

5 files changed

+255
-1
lines changed

‎06-monitoring/00-prometheus/.demos/03-running-prometheus/readme.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -121,7 +121,7 @@ Cambiamos al directorio de descarga y ejecutamos el binario:
121121
122122
```bash
123123
cd prometheus-2.32.1.linux-amd64
124-
./prometheus.yml
124+
./prometheus
125125
```
126126

127127
Si nos fijamos en el `output` deberáimos ver algo como esto:

‎06-monitoring/00-prometheus/.demos/04-configuring-prometheus/Vagrantfile

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
$prometheus_script = <<-SCRIPT
2+
echo Downloading Prometheus
3+
wget https://github.com/prometheus/prometheus/releases/download/v2.32.1/prometheus-2.32.1.linux-amd64.tar.gz
4+
echo Getting files
5+
tar xvfz prometheus-2.32.1.linux-amd64.tar.gz
6+
SCRIPT
7+
8+
$exporter_script = <<-SCRIPT
9+
echo Installing prometheus node exporter linux
10+
apt-get -y install prometheus-node-exporter
11+
12+
SCRIPT
13+
14+
Vagrant.configure("2") do |config|
15+
config.vm.provision "shell", inline: <<-SHELL
16+
apt-get update -y
17+
echo "10.0.0.10 prometheus" >> /etc/hosts
18+
echo "10.0.0.11 linux-batch" >> /etc/hosts
19+
echo "10.0.0.12 linux-web" >> /etc/hosts
20+
SHELL
21+
22+
config.vm.define "prometheus" do |prometheus|
23+
prometheus.vm.box = "bento/ubuntu-20.04"
24+
prometheus.vm.hostname = "prometheus"
25+
prometheus.vm.network "private_network", ip: "10.0.0.10"
26+
prometheus.vm.provision "shell", inline: $prometheus_script
27+
prometheus.vm.provider "virtualbox" do |vb|
28+
vb.memory = 4048
29+
vb.cpus = 2
30+
end
31+
end
32+
33+
config.vm.define "linux-batch" do |linuxbatch|
34+
linuxbatch.vm.box = "bento/ubuntu-20.04"
35+
linuxbatch.vm.hostname = "linux-batch"
36+
linuxbatch.vm.network "private_network", ip: "10.0.0.11"
37+
linuxbatch.vm.provision "docker" do |d|
38+
d.run "jaimesalas/prom-batch",
39+
args: "-e PORT=8080 -p 8080:8080"
40+
end
41+
linuxbatch.vm.provision "shell", inline: $exporter_script
42+
linuxbatch.vm.provider "virtualbox" do |vb|
43+
vb.memory = 1012
44+
vb.cpus = 1
45+
end
46+
end
47+
48+
config.vm.define "linux-web" do |linuxweb|
49+
linuxweb.vm.box = "bento/ubuntu-20.04"
50+
linuxweb.vm.hostname = "linux-web"
51+
linuxweb.vm.network "private_network", ip: "10.0.0.12"
52+
linuxweb.vm.provision "docker" do |d|
53+
d.run "jaimesalas/prom-web",
54+
args: "-e PORT=8080 -p 8080:8080"
55+
end
56+
linuxweb.vm.provision "shell", inline: $exporter_script
57+
linuxweb.vm.provider "virtualbox" do |vb|
58+
vb.memory = 1012
59+
vb.cpus = 1
60+
end
61+
end
62+
end

‎06-monitoring/00-prometheus/.demos/04-configuring-prometheus/prometheus-config.yml

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
global:
2+
scrape_interval: 20s # default 1m
3+
4+
scrape_configs:
5+
- job_name: "prometheus"
6+
static_configs:
7+
- targets: ["localhost:9090"]
8+
9+
- job_name: "linuxbatch"
10+
static_configs:
11+
- targets: ["linux-batch:9100"]
12+
13+
- job_name: "batch"
14+
static_configs:
15+
- targets: ["linux-batch:8080"]
16+
17+
- job_name: "linuxweb"
18+
scrape_timeout: 15s # default 10 s
19+
static_configs:
20+
- targets: ["linux-web:9100"]
21+
22+
- job_name: "web"
23+
metrics_path: /metrics
24+
scheme: http # can include TLS and auth for HTTPS
25+
static_configs:
26+
- targets: ["linux-web:8080"]

‎06-monitoring/00-prometheus/.demos/04-configuring-prometheus/readme.md

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
# Configuring Prometheus
2+
3+
## Setup
4+
5+
```bash
6+
vagrant up
7+
```
8+
9+
# 1. Static configuration for scrape targets
10+
11+
Vamos a crear un fichero de configuración para Prometheus en lugar de usar el que viene por defecto:
12+
13+
```yaml
14+
global:
15+
scrape_interval: 20s # default 1m
16+
17+
scrape_configs:
18+
- job_name: 'prometheus'
19+
static_configs:
20+
- targets: ['localhost:9090']
21+
22+
- job_name: 'linuxbatch'
23+
static_configs:
24+
- targets: ['linux-batch:9100']
25+
26+
- job_name: 'batch'
27+
static_configs:
28+
- targets: ['linux-batch:8080']
29+
30+
- job_name: 'linuxweb'
31+
scrape_timeout: 15s # default 10 s
32+
static_configs:
33+
- targets: ['linux-web:9100']
34+
35+
- job_name: 'web'
36+
metrics_path: /metrics
37+
scheme: http # can include TLS and auth for HTTPS
38+
static_configs:
39+
- targets: ['linux-web:8080']
40+
41+
```
42+
43+
Abrimos sesión SSH contra la vm de Prometheus
44+
45+
```bash
46+
vagrant ssh prometheus
47+
```
48+
49+
Podemos comprobar que tenemos conectividad con las otras máquinas:
50+
51+
```bash
52+
ping linux-batch
53+
```
54+
55+
```bash
56+
ping linux-web
57+
```
58+
59+
Ahora vamos a arrancar `prometheus` de forma interactiva para alimentarle nuestra configuración por defecto, primero vamos a crear el fichero:
60+
61+
```bash
62+
nano prometheus-config.yml
63+
```
64+
65+
`Prometheus` viene con una utilidad `promtool` para verificar que el `yaml` es correcto
66+
67+
```bash
68+
prometheus-2.32.1.linux-amd64/promtool \
69+
check config prometheus-config.yml
70+
```
71+
72+
Debemos obtener una salida como la siguiente:
73+
74+
```
75+
Checking prometheus-config.yml
76+
SUCCESS: 0 rule files found
77+
```
78+
79+
Y ahora podemos arrancar `Prometheus` de la siguiente forma:
80+
81+
```bash
82+
prometheus-2.32.1.linux-amd64/prometheus \
83+
--config.file="prometheus-config.yml" \
84+
--web.enable-lifecycle
85+
```
86+
87+
`--web.enable-lifecycle` abilitará un endpoint para recargar la configuración de `Prometheus` sin reiniciar el servidor.
88+
89+
## 2. Exploramos los nuevos target Explore the new targets
90+
91+
Comprobamos http://10.0.0.10:9090/config y http://10.0.0.10:9090/targets
92+
93+
94+
Vamos a comprobar que las métricas son recogidas en la página _Graph_:
95+
96+
- `up`
97+
- `scrape_duration_seconds`
98+
- `scrape_samples_scraped`
99+
- `worker_jobs_active` - labels & graph
100+
101+
> Navegar & refrescar la web app en http://10.0.0.12:8080/quote
102+
103+
Check metrics:
104+
105+
- `http_request_duration_seconds_bucket`
106+
107+
108+
## 3. Hacer una actualización en vivo del config
109+
110+
Podemos comprobar el último load timestamp de la configuración:
111+
112+
- `prometheus_config_last_reload_success_timestamp_seconds`
113+
114+
> Hacemos un cambio en este fichero [config file](./prometheus-config.yml)
115+
116+
```diff
117+
global:
118+
scrape_interval: 20s # default 1m
119+
120+
scrape_configs:
121+
- job_name: "prometheus"
122+
static_configs:
123+
- targets: ["localhost:9090"]
124+
125+
- job_name: "linuxbatch"
126+
static_configs:
127+
- targets: ["linux-batch:9100"]
128+
129+
- job_name: "batch"
130+
static_configs:
131+
- targets: ["linux-batch:8080"]
132+
133+
- job_name: "linuxweb"
134+
scrape_timeout: 15s # default 10 s
135+
static_configs:
136+
- targets: ["linux-web:9100"]
137+
138+
- job_name: "web"
139+
metrics_path: /metrics
140+
scheme: http # can include TLS and auth for HTTPS
141+
static_configs:
142+
- - targets: ["linux-web:8080"]
143+
+ - targets: ["linux-web:8081"]
144+
```
145+
146+
Hemos cambiado el puerto a uno incorrecto.
147+
148+
```bash
149+
curl -X POST http://10.0.0.10:9090/-/reload
150+
```
151+
152+
Si ejecutamos de nuevo `prometheus_config_last_reload_success_timestamp_seconds`
153+
154+
Y echamos un ojo en `targets` veremos que uno de ellos ha desaparecido.

‎06-monitoring/00-prometheus/02-running-configuring-prometheus/readme.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,3 +6,15 @@
66

77
[Demo: Ejecutando Prometheus](../.demos/03-running-prometheus/readme.md)
88

9+
10+
## Configurando Targets y Service Discovery
11+
12+
La configuración más básica que podemos utilizar es la estática (la que acabamos de ver), la cual es una lista fija con nombres de dominios o IP's.
13+
14+
Una mejor opción incluso para proyectos pequeños es utilizar `file service discovery`
15+
16+
> Prometheus también soporta `dinamic service discovery` pero depende de la plataforma.
17+
18+
## Demo: Configurando Prometheus
19+
20+
[Demo: Configurando Prometheus](../.demos/04-configuring-prometheus/readme.md)

0 commit comments

Comments
 (0)