feat(#64) added multiple aggregate levels

.gitignore

@@ -1,3 +1,4 @@
 /node_modules
 npm-debug.log
 coverage
+.vscode

CHANGELOG.md

@@ -1,5 +1,8 @@
 # Change log
 
+## [Unreleased] - 2017-07-24
+- **Added:** Longer history for graphs [\#64]
+
 ## [v0.4.1] - 2017-03-21
 - **Added:** Historical memory usage graph [\#63]
 - **Added:** Support for log filtering - see README [\#62]
@@ -50,6 +53,7 @@
 [v0.1.2]: https://github.com/FormidableLabs/nodejs-dashboard/compare/v0.1.1...v0.1.2
 [v0.1.1]: https://github.com/FormidableLabs/nodejs-dashboard/compare/v0.1.0...v0.1.1
 
+[\#64]: https://github.com/FormidableLabs/nodejs-dashboard/pull/66
 [\#63]: https://github.com/FormidableLabs/nodejs-dashboard/pull/63
 [\#62]: https://github.com/FormidableLabs/nodejs-dashboard/pull/62
 [\#59]: https://github.com/FormidableLabs/nodejs-dashboard/pull/59

METRIC-AGGREGATION.md

@@ -0,0 +1,81 @@
+# Metric Aggregation
+
+The MetricsProvider class provides metric data for various data points.
+
+The data provided is streamed in two modes:
+* Real Time
+  * A slice of the data that has been collected over time.
+* Aggregate
+  * Data that has been rolled-up to various time increments.
+
+## Time Indexes
+
+The strategy for aggregation centers around time indexes.  Imagine an array
+of values and each element of that array being a data point.  If the data
+is real-time, then each index is just one logical second.  When the data
+is aggregated, each data point is the average of the data, for one logical
+grouping of time.
+
+To determine which data elements from the real-time array are to be grouped,
+the difference in time from the start of the process and the moment a data
+point was captured is passed through this formula:
+
+<img style="margin: 0 auto; display: block;" src="./images/time-index-equation.png"></img>
+
+Where
+
+* `i`
+  * Time Index
+* `t[k]`
+  * Time of element `k`
+* `t[s]`
+  * Start time of process
+* `u`
+  * Time unit of measure (e.g. 5000ms=5s)
+
+All Times are expressed in milliseconds (ms) and are obtained from Date.now().
+
+**Example**
+
+<img style="margin: 0 auto; display: block;" src="./images/time-index-equation-example.png"></img>
+
+The above example demonstrates that for a 5000ms (5s) aggregate, the time index
+for the data point is one.  This formula is applied to all data points, so
+when many data points share the same logical time index, they can be averaged.
+When two or more array elements have a common time index, they form a time band.
+
+<img style="margin: 0 auto; display: block;" src="./images/aggregation-visual.png"></img>
+
+In the above visual, the start time `t[s]` is `7581298`.  The unit of time is `5000ms`.  By applying
+the time index equation against each data points' moment in time, the time index values are
+derived.  For those that match, a time band is formed.  These are colorized similarly.
+
+## Averaging
+
+To efficiently perform the average, care is taken to reduce the number of
+CPU cycles required to analyze the data.  To accomplish this, the program
+calculates the average inline with the originating transaction.
+
+Also, to prevent re-aggregating the data needlessly, each aggregate level
+maintains its position in the base array, thereby keeping track of where it
+left off.  This is done using two simple numbers: the last array element
+examined and the start of the current time band.
+
+So that data is still streamed on an event-basis, an aggregate data point is
+only captured and emitted when it is complete.  To detect when an aggregate
+is complete, the algorithm traverses the base real-time array of data, beginning
+where it left off for any given aggregate.  Walking each element from there
+forward, a simplistic level-break algorithm is implemented.  As and when a
+logical time index changes, this indicates that the previous set of data is
+complete.
+
+Once this occurs, the program then averages the data from the starting index
+through the ending index that spans a logical time unit, regardless of aggregate
+level (using the time index formula above).
+
+To ensure no overflows (and therefore `NaN`) are produced, the average is done
+using the formula on the left-side of this equivalence:
+
+<img style="margin: 0 auto; display: block;" src="./images/average-equivalence-equation.png"></img>
+
+The sum of memory usage even over a 10s aggregate is enough to produce `NaN`.

lib/constants.js

@@ -0,0 +1,52 @@
+"use strict";
+
+// these define the time levels that data will be aggregated into
+// each number is in milliseconds
+//
+// since these are used as object keys, they are strings
+//
+// For example, 5000 = 5s and means that one of the aggregate levels
+// will be at 5s increments of time
+//
+// 300000 = 30s and the corresponding zoom will show 30s aggregates
+var AGGREGATE_TIME_LEVELS = [
+  "5000",
+  "10000",
+  "15000",
+  "30000",
+  "60000",
+  "300000",
+  "600000",
+  "900000",
+  "1800000",
+  "3600000"
+];
+
+// this array object is used to reduce ms to its highest human-readable form
+// see lib/providers/metrics-provider.js::getTimeIndexLabel
+var TIME_SCALES = [
+  {
+    units: "ms",
+    divisor: 1
+  }, {
+    units: "s",
+    divisor: 1000
+  }, {
+    units: "m",
+    divisor: 60
+  }, {
+    units: "h",
+    divisor: 60
+  }, {
+    units: "d",
+    divisor: 24
+  }, {
+    units: "y",
+    divisor: 365.24
+  }
+];
+
+module.exports = {
+  AGGREGATE_TIME_LEVELS: AGGREGATE_TIME_LEVELS,
+  TIME_SCALES: TIME_SCALES
+};

lib/dashboard.js

@@ -75,6 +75,12 @@ Dashboard.prototype._configureKeys = function () {
   this.screen.key(["q", "C-c"], function () {
     process.exit(0); // eslint-disable-line no-process-exit
   });
+
+  this.screen.key(["up", "down"], _.throttle(function (ch, key) {
+    var zoom = key.name === "down" ? -1 : 1;
+    this.screen.emit("zoomGraphs", zoom);
+    this.screen.render();
+  }.bind(this), THROTTLE_TIMEOUT));
 };
 
 Dashboard.prototype.onEvent = function (event) {