Skip to content

Commit b29dc37

Browse files
scaleoutseanscaleoutsean
committed
New README.md
1 parent cdf298c commit b29dc37

File tree

1 file changed

+341
-0
lines changed

1 file changed

+341
-0
lines changed

‎README.md

Lines changed: 341 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,341 @@
1+
# NetApp E-Series Performance Analyzer ("EPA")
2+
3+
- [NetApp E-Series Performance Analyzer ("EPA")](#netapp-e-series-performance-analyzer-epa)
4+
- [What is this thing](#what-is-this-thing)
5+
- [Requirements](#requirements)
6+
- [Quick start](#quick-start)
7+
- [Slow start](#slow-start)
8+
- [Environment variables and configuration files](#environment-variables-and-configuration-files)
9+
- [Adjust firewall settings for InfluxDB and Grafana ports](#adjust-firewall-settings-for-influxdb-and-grafana-ports)
10+
- [Add or remove a monitored array](#add-or-remove-a-monitored-array)
11+
- [Update password of a monitor account](#update-password-of-a-monitor-account)
12+
- [Walk-through](#walk-through)
13+
- [Using public Docker images](#using-public-docker-images)
14+
- [Sample Grafana screenshots](#sample-grafana-screenshots)
15+
- [FAQs](#faqs)
16+
- [Changelog](#changelog)
17+
18+
19+
## What is this thing
20+
21+
This is a friendly fork of [E-Series Performance Analyzer aka EPA](https://github.com/NetApp/eseries-perf-analyzer) v3.0.0 (see its README.md for additional information) created with the following objectives:
22+
23+
- Disentangle E-Series Collector from the rest of EPA and make it easy to run it anywhere (shell, Docker/Docker Compose, Kubernetes, Nomad)
24+
- Remove SANtricity Web Services Proxy (WSP) dependency so that one collector container or script captures data for one and only one E-Series array
25+
26+
In terms of services, collectors collects metrics from E-Series and sends them to InfluxDB. dbmanager doesn't do much at this time - it periodically sends array names as folder tags to InfluxdDB.
27+
28+
![E-Series Performance Analyzer](/images/epa-eseries-perf-analyzer.png)
29+
30+
Each of the light-blue rectangles can be in a different location (host, network, Kubernetes namespace, etc.). But if you want to consolidate, that's still possible.
31+
32+
Changelog and additional details are at the bottom of this page and in the Releases tab.
33+
34+
## Requirements
35+
36+
- NetApp SANtricity OS: >= 11.70 (11.74 is recommended; 11.52 and 11.74 have been tested and work, 11.6[0-9] not yet)
37+
- Containers:
38+
- Docker: Docker CE 20.10.22 (recent Docker CE or Podman should work) and Docker Compose v1 or v2 (both v1 and v2 should work)
39+
- Kubernetes: dbmanager and collector should work on any
40+
- Nomad: dbmanager and collector should work on any
41+
- CLI:
42+
- dbmanager and collector should work on any Linux with recent Python 3, possibly other Operating Systems
43+
- Architecture: dbmanager and collector work on (at least) AMD64 and ARM64 systems that support Python 3
44+
45+
These requirements are soft but this is a community fork without a variety of hardware and software to use in testing and debugging.
46+
47+
## Quick start
48+
49+
Docker Compose users:
50+
51+
- Clone and enter the `epa` subdirectory:
52+
```sh
53+
git clone https://github.com/scaleoutsean/eseries-perf-analyzer/
54+
cd eseries-perf-analyzer/epa
55+
```
56+
- in the `epa` subdirectory, run `make run` to build and run InfluxDB and Grafana
57+
- Unless these containers need a change or update, going back to this folder is generally not necessary
58+
- go to the `collector` sub-directory edit `docker-compose.yml` and `config.json`: `SYSNAME` in docker-compose.yml must be present and identical to `name` value(s) in `config.json`. Then run `docker-compose build && docker-compose up` to start dbmanager and collector(s)
59+
- When E-Series arrays are added or removed, edit the same files and run `docker-compose build && docker-compose down && docker-compose up` to update
60+
61+
Kubernetes users should skim through this page to get the idea how EPA works, and then follow [Kubernetes README](kubernetes/README.md).
62+
63+
## Slow start
64+
65+
It is suggested to get EPA working in Docker Compose, unless you're good at Kubernetes. There's also a [Kubernetes](kubernetes)-specific folder.
66+
67+
- Older existing EPA (v3.0.0, v3.1.0), images, volumes and services may cause container name, volume and port conflicts. Either use a new VM or find the existing (old) deployment and run `make stop; docker-compose down; make rm` to stop and remove old EPA pre-v3.2.0 containers before building new ones. Data (InfluxDB and Grafana) can be left in place.
68+
- Clone this repository to a new location
69+
- Descend to the `epa` directory, run `make run` to download, build and start InfluxDB v1 and Grafana v8. You may move the pre-existing InfluxDB folder to the EPA directory if you want to keep the data. Both services will listen on all public VM interfaces, so configure your firewall accordingly.
70+
- Go to the `collector` directory, edit two files (`config.json` and `docker-compose.yml`) and run `docker-compose build` to create collector and dbmanager containers and then `docker-compose up` to start them.
71+
72+
```sh
73+
git clone github.com/scaleoutsean/eseries-perf-analyzer
74+
cd eseries-perf-analyzer
75+
# make and run Grafana and InfluxDB
76+
cd epa; make run
77+
# go to the collector subdirectory
78+
cd ..; cd collector
79+
# Enter names of E-Series array (or arrays) to show in Grafana drop-down list.
80+
# "docker-comose build" will copy this file to dbmanager.
81+
vim config.json
82+
# Edit docker-compose file leave dbmanager unchanged. Collector containers should reflect config.json:
83+
# container_name, specifically , must be the same as storage array name in config.json.
84+
vim docker-compose.yml
85+
# We are still in the ./collector subdirectory. InfluxDB and Grafana are already running.
86+
# Build and start collector(s) and dbmanager:
87+
docker-compose build
88+
docker-compose up
89+
# Check Grafana and if OK, hit CTRL+C, restart with:
90+
docker-compose up -d
91+
# If not OK, CTRL+C and "docker-compose down".
92+
# Then review config.json and docker-compose.yml.
93+
# collector.py and db_manager.py can be started from the CLI for easier troubleshooting without containers.
94+
```
95+
96+
### Environment variables and configuration files
97+
98+
- `./epa/.env` has some env data used by its Makefile for InfluxDB and Grafana. Use `make` to start, stop, clean, remove, and restart these two
99+
- `./collector` is simpler: use `docker-compose` to build/start/stop/remove collector and dbmanager containers and don't forget `config.json`
100+
- When editing `./collector/docker-compose.yml`, provide the following for each E-Series array:
101+
- `USERNAME` - SANtricity account for monitoring such as `monitor` (read-only access to SANtricity)
102+
- `PASSWORD` - SANtricity password for the account used to monitor
103+
- `SYSNAME` - SANtricity array name, such as `R26U25-EF600` - get this from the SANtricity Web UI, but you can use your own - just keep it consistent with the name in `./collector/config.json`. If you want to make the name identical to actual E-Series array name, [this image](/images/sysname-in-santricity-manager.png) shows where to look it up
104+
- `SYSID` - SANtricity WWN for the array, such as 600A098000F63714000000005E79C888 - see [this image](/images/sysid-in-santricity-manager.png) on where to find it in the SANtricity Web UI.
105+
- `API` - SANtricity controller's IP address such as 6.6.6.6. Port number (`:8443`) is automatically set in scripts
106+
- `RETENTION_PERIOD` - data retention in InfluxDB, such as 52w (52 weeks)
107+
- `DB_ADDRESS` - external IPv4 of the InfluxDB host. If the host IP where InfluxDB is running is remote that could be something like 7.7.7.7. If dbmanager, collector and InfluxDB are on the same host then it can be 127.0.0.1; if they're in the same Kubernetes namespace then `influxdb`, etc.)
108+
109+
Where to find the `API` value(s)? `API` address (or addresses) are IPv4 addresses (or FQDNs) used to connect to the E-Series Web management UI. You can see them in the browser when you manage an E-Series array.
110+
111+
For consistency's sake it is recommended that `SYSNAME` in EPA is the same as the actual E-Series system name, but it doesn't have to be - it can consist of arbitrary alphanumeric characters (and `_` and `-`; if interested please check the Docker Compose documentation). Just make sure the array names in `./collector/docker-compose.yml` and `./collector/config.json` are identical; otherwise array metrics and events may get collected, but drop-down lists with array names in Grafana dashboards won't match so the dashboards will be empty even though the InfluxDB is not.
112+
113+
Example of `docker-compose.yml` with collector for one array:
114+
115+
```yaml
116+
services:
117+
118+
collector-R26U25-EF600:
119+
image: ntap-grafana-plugin/eseries_monitoring/collector:latest
120+
container_name: R26U25-EF600
121+
mem_limit: 64m
122+
restart: unless-stopped
123+
logging:
124+
driver: "json-file"
125+
options:
126+
max-file: "5"
127+
max-size: 10m
128+
environment:
129+
- USERNAME=monitor
130+
- PASSWORD=monitor123
131+
- SYSNAME=R26U25-EF600
132+
- SYSID=600A098000F63714000000005E79C888
133+
- API=6.6.6.6
134+
- RETENTION_PERIOD=26w
135+
- DB_ADDRESS=7.7.7.7
136+
- DB_PORT=8086
137+
```
138+
139+
`SYSNAME` from `./collector/docker-comopose.yml` should be the same as `name` in `config.json` used by dbmanager. Here the `name` matches `environment:SYSNAME` value in `docker-compose.yml` above.
140+
141+
```json
142+
{
143+
"storage_systems": [
144+
{
145+
"name": "R26U25-EF600"
146+
}
147+
]
148+
}
149+
```
150+
151+
`dbmanager` doesn't do much and doesn't yet make use of `RETENTION_PERIOD` (just leave that value alone for now). Only `DB_ADDRESS` parameter need to be correct, and the name(s) in `config.json` need to match `SYSNAME` in `docker-compose.yml`.
152+
153+
```yaml
154+
version: '3.6'
155+
services:
156+
collector-dbmanager:
157+
image: ntap-grafana-plugin/eseries_monitoring/dbmanager:latest
158+
container_name: dbmanager
159+
mem_limit: 32m
160+
restart: unless-stopped
161+
logging:
162+
driver: "json-file"
163+
options:
164+
max-file: "5"
165+
max-size: 10m
166+
environment:
167+
- RETENTION_PERIOD=52w
168+
- DB_ADDRESS=6.6.6.6
169+
- DB_PORT=8086
170+
```
171+
172+
### Adjust firewall settings for InfluxDB and Grafana ports
173+
174+
The original EPA v3.0.0 exposes the SANtricity WSP (8080/tcp) and Grafana (3000/tcp) to the outside world.
175+
176+
This fork does not use WSP. Grafana is the same (3000/tcp), but InfluxDB is now exposed externally at 8086/tcp. The idea is to be able to run several collectors in various locations (closer to E-Series, for example) and send data to a centrally managed InfluxDB.
177+
178+
To protect InfluxDB service open 8086/tcp to IP's or FQDNs where collector, dbmanager and Grafana run.
179+
180+
### Add or remove a monitored array
181+
182+
To add a new SANtricity array, we don't need to do anything in the `epa` subdirectory.
183+
184+
- Go to `./collector`
185+
- Edit `docker-compose.yml` - if you copy-paste, make sure you get the variables and `container_name` right!
186+
- Edit `config.json` to add a matching record for the new array
187+
- `docker-compose down`
188+
- `docker-compose build`
189+
- `docker-compose up -d`
190+
191+
To remove an array, remove it from `config.json` and `docker-compose.yml` and do the last three `docker-compose` steps the same way.
192+
193+
### Update password of a monitor account
194+
195+
To change the monitor account password for one particular collector, say the one used for array `R11U01-EF300`, change it on the array first, find this array in `docker-compose.yml`, change the password value in the `PASSWORD=` row for the array, run `docker-compose down R11U01-EF300` followed by `docker-compose up R11U01-EF300`.
196+
197+
The array name has not changed, so it wasn't necessary to edit `./collector/config.json` and rebuild `./collector/dbmanager`, so running `docker-compose build` wasn't necessary.
198+
199+
## Walk-through
200+
201+
- Build and run InfluxDB and Grafana:
202+
203+
```sh
204+
$ cd epa
205+
206+
$ make build
207+
208+
$ docker images
209+
REPOSITORY TAG IMAGE ID CREATED SIZE
210+
ntap-grafana-plugin/eseries_monitoring/python-base latest 9d5f8085ab4a 51 seconds ago 50.1MB
211+
<none> <none> 510d1a737cad 52 seconds ago 12.9MB
212+
ntap-grafana-plugin/eseries_monitoring/alpine-base latest 85a1ebbfbc5e 54 seconds ago 7.05MB
213+
ntap-grafana/influxdb 3.2 4c650d02806a 55 seconds ago 173MB
214+
ntap-grafana/ansible 3.2 94ee4e4a0405 About a minute ago 398MB
215+
<none> <none> bd3051fd74a4 About a minute ago 621MB
216+
ntap-grafana/python-base 3.2 5216517bec73 2 minutes ago 50.1MB
217+
<none> <none> e9b76094f71d 2 minutes ago 12MB
218+
219+
$ make run # runs: docker-compose up -d in the epa directory
220+
221+
$ # expect to see two containers listening on external ports - InfluxDB and Grafana
222+
223+
$ docker ps -a | grep '0.0.0.0'
224+
95dd8ec86b82 ntap-grafana/grafana:3.0 0.0.0.0:3000->3000/tcp, :::3000->3000/tcp grafana
225+
f00b858c0728 ntap-grafana/influxdb:3.0 0.0.0.0:8086->8086/tcp, :::8086->8086/tcp influxdb
226+
```
227+
228+
- Login to Grafana with admin/admin, change admin password and optionally disable anonymous read-only access. At this point you're not supposed to see anything in the EPA dashboards
229+
230+
- Go to top-level `collector` directory to build Collector-related containers
231+
232+
```sh
233+
$ pwd
234+
/home/sean/eseries-perf-analyzer/collector
235+
236+
$ # edit docker-compose.yml and config.json
237+
238+
$ cat docker-compose.yml | grep name
239+
container_name: dbmanager
240+
container_name: R26U25-EF600
241+
container_name: R24U04-E2824
242+
243+
$ # ensure container names in docker-compose.yml and system names in config.json are consistent
244+
245+
$ cat config.json
246+
{
247+
"storage_systems": [
248+
{
249+
"name": "R26U25-EF600"
250+
},
251+
{
252+
"name": "R24U04-E2824"
253+
}
254+
]
255+
}
256+
257+
$ docker-compose build
258+
```
259+
260+
- This `build` operation builds two containers, collector & dbmanager
261+
- There should be two new container images (collector & dbmanager) used by two or more containers (here three, because there's one dbmanager and two arrays)
262+
263+
```sh
264+
$ docker ps -a | grep monitoring
265+
CONTAINER ID IMAGE NAMES
266+
9d725fa1a756 ntap-grafana-plugin/eseries_monitoring/collector R24U04-E2824
267+
1048f321d631 ntap-grafana-plugin/eseries_monitoring/collector R26U25-EF600
268+
61d3cb5e83bc ntap-grafana-plugin/eseries_monitoring/dbmanager dbmanager
269+
```
270+
271+
- Stop and remove any existing collectors and dbmanager. Start new (or updated) containers:
272+
273+
```sh
274+
$ pwd
275+
/home/sean/eseries-perf-analyzer/collector
276+
277+
$ # MIND the location! Don't do this in /home/sean/eseries-perf-analyzer/epa and wipe your Grafana and InfluxDB.
278+
279+
$ docker-compose down && docker-compose up
280+
```
281+
282+
### Using public Docker images
283+
284+
Remember to edit Docker image location if you want to use local images or images from local registry. You may also use public images such as:
285+
286+
- docker.io/scaleoutsean/epa-dbmanager:v3.2.0
287+
- docker.io/scaleoutsean/epa-collector:v3.2.0
288+
289+
## Sample Grafana screenshots
290+
291+
This fork's dashboards are identical to upstream v3.0.0, but upstream repository has no screenshots - in fact they're hard to find on the Internet - so a sample of each dashboard is provided below.
292+
293+
- System view
294+
295+
![E-Series System](/images/sample-screenshot-epa-collector-system.png)
296+
297+
- Array interfaces
298+
299+
![E-Series Array Interfaces.png](/images/sample-screenshot-epa-collector-interfaces.png)
300+
301+
This screenshot shows *aggregate* values for all arrays (useful in HPC environments where workloads span across multiple arrays). Further below there are other charts with individual metrics.
302+
303+
- Physical disks
304+
305+
![E-Series Physical Disks.png](/images/sample-screenshot-epa-collector-disks.png)
306+
307+
- Logical volumes
308+
309+
![E-Series Volumes.png](/images/sample-screenshot-epa-collector-volumes.png)
310+
311+
- Physical disks - SSD wear level (%)
312+
313+
![E-Series SSD Wear Level](/images/sample-screenshot-epa-collector-disks-ssd-wear-level.png)
314+
315+
This is the second example with physical disks and it's highlighted because this data is collected by collector, but not shown in dashboards. In order to collect this data, an E-Series array with a recent SANtricity OS (11.74, for example) and at least one SSD is required. Visualization can then be done by duplicating one of the existing disk charts and modifying it to show "percentEnduranceUsed" values. This screenshot shows that SSD wear level metrics are collected from just one of two arrays.
316+
317+
## FAQs
318+
319+
Find them [here](FAQ.md) or check [Discussions](https://github.com/scaleoutsean/eseries-perf-analyzer/discussions) for questions that aren't in the FAQ document.
320+
321+
## Changelog
322+
323+
- 3.2.0 (Jan 30, 2023):
324+
- No new features vs. v3.1.0
325+
- No changes to Grafana container, Grafana charts, and InfluxDB container
326+
- collector and dbmanager are now completely independent of containers built by InfluxDB and Grafana Makefile
327+
- New kubernetes folder with Kubernetes-related instructions and sample YAML files
328+
- collector and dbmanager can work on both AMD64 and ARM64 systems
329+
330+
- 3.1.0 (Jan 12, 2023):
331+
- No changes to Grafana dashboards
332+
- Updated Grafana v8 (8.5.15), Python Alpine image (3.10-alpine3.17) and certifi (2022.12.7)
333+
- Remove SANtricity Web Services Proxy (WSP) and remove WSP-related code from collector
334+
- Make InfluxDB listen on public (external) IP address, so that collectors from remote locations can send data in
335+
- Add the ability to alternate between two E-Series controllers to collector (in upstream v3.0.0 the now-removed WSP would do that)
336+
- Add collection of SSD wear level for flash media (panel(s) haven't been added, it's up to the user to add them if they need 'em)
337+
- Expand the number of required arguments in `collector.py` to avoid unintentional mistakes
338+
- Collector can run in Kubernetes and Nomad
339+
- Add dbmanager container for the purpose of uploading array configuration to InfluxDB (and potentially other DB-related tasks down the road)
340+
- Add simple Makefile for collector containers (collector itself, and dbmanager)
341+
- Old unit tests are no longer maintained

0 commit comments

Comments
 (0)