raise the aggregation-function value max value?

[andy-esch]

Sorry @fdansv , tl;dr.

Right now, it seems that the value returned by the data aggregation function has a large upper bound, but within the CartoCSS, it seems to max out at 255.

Using a number that grows well beyond 255 with a date, I found a behavior in the colors that suggests that value = value % 256 somewhere along the line.

Map here. Every time the large black dot flashes, value = 255 even though value should be growing monotonically in time. Checking the json object in the console, you can see that vals_uint8 are above 255. Since it's uint8, does that mean that it will be input into a Uint8Array() so that the modulus is taken? This seems pretty small and forces users to write custom aggregation functions that keep the values below 256.

I think we either need to document this in the torque cartocss, or raise the value to something that's more intuitive from what people would expect from their dataset. For instance, I was dealing with data where the max values were 1e6. I was forced to do log(sum(num_column)) to put value within 0 - 255.

The dataset was generated from this SQL:

SELECT 
    ST_Transform(CDB_LatLng(random()*90,random()*180),3857) the_geom_webmercator, 
    a,
    now()::date + a dateof 
FROM 
    generate_series(1,5000) a

and this CartoCSS:

/** torque visualization */

Map {
-torque-frame-count:512;
-torque-animation-duration:30;
-torque-time-attribute:"dateof";
-torque-aggregation-function:"sum(a)";
-torque-resolution:2;
-torque-data-aggregation:linear;
}

#torque_large_number_cartocss_test{
  comp-op: lighter;
  marker-fill-opacity: 0.9;
  marker-line-color: #FFF;
  marker-line-width: 1.5;
  marker-line-opacity: 1;
  marker-type: ellipse;
  marker-width: 6;
  marker-fill: #FF9900;
}

#torque_large_number_cartocss_test [value <= 5000] {
  marker-fill: black;
}

#torque_large_number_cartocss_test [value <= 500] {
  marker-fill: red;
}
#torque_large_number_cartocss_test [value <= 400] {
  marker-fill: orange;
}
#torque_large_number_cartocss_test [value <= 300] {
  marker-fill: yellow;
}
#torque_large_number_cartocss_test [value <= 200] {
  marker-fill: green;
}
#torque_large_number_cartocss_test [value <= 100] {
  marker-fill: blue;
}
#torque_large_number_cartocss_test [value <= 50] {
  marker-fill: violet;
}

#torque_large_number_cartocss_test [value = 255] {
  marker-width: 50;
  marker-fill: black;
 #torque_large_number_cartocss_test[frame-offset=1] {
 marker-width:50;
}
#torque_large_number_cartocss_test[frame-offset=2] {
 marker-width:50;
}
}

[fdansv]

Hell of an issue report, @ohasselblad , you're the man. Let me look into it although me might have to move this to Windshaft.

[fdansv]

OK, so here's the thing: it's a known issue, for which we have no immediate solution, that the maximum value in non-cumulative torque has to be 255. However, as discussed with @javisantana, there's a better way that we can do this.

As you mentioned, because we currently user UInt8Arrays, values higher than 255 are modulo'd by 256. This makes point values go all crazy when values go above that. So, instead of raising the limit (which is unfeasible at this moment), or keeping this modulo nonsense, I suggest changing the array type to Uint8ClampedArray. This puts an effective cap of 255 over any values set:

new Uint8ClampedArray([1, 255, 20000])
// [1, 255, 255]

[fdansv]

Done in #179

[andy-esch]

👍 thanks!