QA checklist for bundled webapps

== general ==

These are interactive web applications, so not well suited to
automated batch testing.  Among the complications:

- different browsers have different rendering engines / bugs
- different user preferences (fonts, window sizes) 
- performance (interactivity)
- memory usage (absence of leaks)
- interactive reconfigurations
- server-side dependencies (available data sets, running services) 

This adds up to many degrees of freedom, more than we can hope to
test.  But we can do a few things:

- try several mainstream web browsers (firefox, webkit
  (epiphany/konqueror/chrome/safari), msie) for a smoke-test of the
  out-of-the-box webapp experience, namely hitting the top level
  index.html file, then each webapp "quick-launch link", and
  confirming that the views successfully load & look acceptable

- increase pmwebd logging (-v in /etc/pcp/pmwebd/pmwebd.options)


== blinkenlights ==

- tweak the refresh-interval slider to observe timely refreshes
- add other predicates such as
  "kernel.all.nprocs < 5" and "kernel.all.nprocs >= 5"
  to confirm calculation polarities
- test "local" connection; kernel.* metrics should stay resolveable
- test a remote pmcd hostspec, if available; metrics should change value but stay alive
- check pmwebd.log to see new contexts coming, old ones expiring


== graphite ==

- ensure some local pmlogger and/or pmmgr data is already available, ideally from
  more than one host
- initial view should show "no data in range" as no metrics were selected
- exercise the metric tree browser
  - under tree tab, a voluminous but fast-responding navigable tree should appear
    - pick metrics from a timewise-recent archive to populate the composer
      - (archives for any given host tend to be sorted in increasing-time order)
    - unpick some; the curves should come & go
  - use "Search" function; type in regular expression (not glob), observe matches
    below (filtered from complete list of all archive X metric combinations)
  - use "Auto-Completer" function; widen the split pane, since metric names get very
    long; incrementally type in parts of the archive file name / metrics, with or without
    wildcards; note dropdowns of partial matches

- exercise the composer window
  - click "graph data" in composer widget to see actual metrics
    - edit one of the metrics selected to replace the first component (archive-path)
      with a wildcard (*).  This requires clicking outside the wildcard-expanded
      dropdown, and manually hitting the "OK" button instead; the graph should show
      corresponding metric curves from other hosts
  - hit "update graph" in composer (top left) a couple of times over some minutes
    to see time go; activate "auto-refresh" to let it happen automatically
  - change the selected time range with "select date range" and/or "select recent data"
    selectors.  Note intervals in units larger than "week" are not supported by pmwebd
    (since a month has no fixed number of seconds)
    
- hit the "open in graphlot" button for this other miniwebapp
  - timewise zoom in by dragging over grid
  - unzoom by "clear zoom" at far bottom right
  - add extra metrics/wildcards


== grafana - png & flot.js modes ==

- ensure some local pmlogger and/or pmmgr data is already available
- check all graphs come around in reasonable time (1-30 seconds)
  - on png rendering mode, check legends/labels are readable & apropos
  - on flot.js rendering mode, check popups when cursor on curves
- resize browser window; graphs should resize
- check time zoom-in (random interesting time slice)
  - on png rendering mode, time selection bar is the top thin graph
- check time zoom-out
  - menu bar zoom-out or "last 30d" and beyond for maximum archive
    scaning burden
- check pmwebd.log for plausible metric-digestion times

bonus:

- try adding and/or removing metrics from a graph via the interactive
  editor
- try changing visual preferences (color schemes, legend activation)


== vector ==

- check that default widgets (for localhost pmwebd & pmcd) are alive,
  producing updates every few seconds
- check popups as cursor hovers over charts
- change the interval to slow down or speed up data collection
- change "window" (data-retention duration) to extend or shorten history
- delete some widgets; add some back
- drag some widgets around
- test hostname/hostspec control by connecting webapp to another pmcd
  ("hostspec") and/or another pmwebd ("hostname"); actual pmcd.hostname
  should show up
- resize window; graphs should resize
- test out embed mode ..../vector/index.html#/embed?host=....&hostspec=localhost&widgets=kernel.all.cpu
- check pmwebd logs (-v -v) that traffic is sensible (good rate, needed metrics only)
- start/stop some containers (docker run --rm -t -i busybox /bin/sh)
  add container metric widgets, set a container name filter, observe graphs following