Skip to content

WMStats

Seangchan Ryu edited this page Dec 30, 2018 · 16 revisions

WMStats Overview

  1. Monitor current status of request and collective job. (snap shot of the current status)
  2. Works as a debugging tool by providing a hierarchical view on request (starts from request to job level to track down the problem)
  3. Provide search tool for finding specific requests. (REST API, Web GUI)
  4. Provide error, warning information about the request and infrastructure (WMAgent components)

DB backend (CouchDB)

  1. wmstats (t0_wmstats): contains job summary for given workflow (including jobs per site, task and wmagent) - data is replicated from wmagents. It is transitional data which updated every given cycle.
  2. wmstats_logdb: contains errors for each request and wmagent (components/thread) not in terms of individual jobs. (i.e. a request is not advanced to next state due to the missing configuation, etc)

Design

WMStats uses MVVM (Model view view-model) architectural pattern.

Implementation

WMStats collects the data from different sources (ReqMgr2, GlobalWorkQueue - pull, WMAgent -push) and provide the snapshot of job summary by requests) All the active data (not archived status requests) is pulled down to the cache periodically (every 10 min) in server side. And API provide the access to cached data not directly to backend DB. WMStats GUI interfaces is created using JavaScript, JQuery. It is client heavy operation (pull down all the data at once). Then data is refreshed every 5 min in the background. (Currently there is memory leak on client side)

Deployment

WMStats contains duplicate code in minified version. Only Minified js files (which contains all the js files) are deployed but not the original versions. This minified version is create by running the script combineMinifyWMStats.py. (Don't manually change those files

Clone this wiki locally