Identify when Miovision volume are unusually high

[gabrielwol]

Last week I discovered a case where bike volumes had gone up more than 4x in recent weeks and I wanted to investigate ways of identifying this automagically.

[gabrielwol]

I worked on a data check to identify when volumes had doubled week over week for a specific intersection/mode: https://github.com/CityofToronto/bdit_data-sources/tree/951-identify-high-miovision-volumes

Running the function for a month:

• we would have received info about 148 failures 😵

SELECT dates.dt, UNNEST(d.summary)
FROM generate_series('2024-04-01'::date, '2024-05-01'::date, interval '1 day') AS dates(dt),
LATERAL (
    SELECT summary FROM gwolofs.miovision_doubling(dates.dt::date)
) AS d

Identifying all the runs:

• 48 runs -> could reduce alerts to this number if we figure out how to notify only on first day
• 1 for cars, 8 for peds, 21 bike TMC, 18 bike approach

WITH doublings AS (
    SELECT dates.dt,
        SPLIT_PART(UNNEST(d.summary), ',', 1) AS intersection,
        SPLIT_PART(UNNEST(d.summary), ',', 2) AS mode,
        SPLIT_PART(UNNEST(d.summary), 'volume: ', 2) AS volumes
    FROM generate_series('2024-04-01'::date, '2024-05-01'::date, interval '1 day') AS dates(dt),
    LATERAL (
        SELECT summary FROM gwolofs.miovision_doubling(dates.dt::date)
    ) AS d
    ORDER BY 2, 3, 1
)

SELECT * FROM (
    SELECT *, dt - lag(dt) OVER (PARTITION BY intersection, mode ORDER BY dt) AS lag_diff
    FROM doublings
) diffs
WHERE lag_diff IS NULL OR lag_diff > interval '1 day'

[gabrielwol]

Reluctant to enable a torrent of notifications like what I identified above. Some ideas:

• increase threshold / add a absolute threshold as well as percentage
• use a threshold based on historical data rather than recent data. Difficult to come up with, would need to be able to override.
• only look at light autos? (~1 notification per month)
• only notify for 1st alert in a run
• Note a lot of these are presumably related to reconfigurations, but we have no way to identify that with the api.