Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collect match failure statistics from Reporter instances #35

Open
drewda opened this issue Jun 9, 2017 · 0 comments
Open

Collect match failure statistics from Reporter instances #35

drewda opened this issue Jun 9, 2017 · 0 comments

Comments

@drewda
Copy link
Contributor

drewda commented Jun 9, 2017

Each Reporter instance now logs statistics on match failures (see opentraffic/reporter#34). We should aggregate this at the Datastore for debugging and evaluation purposes.

An option to consider:

  • send from Reporter to an S3 bucket (either directly or through Kinesis Firehose)
  • store in structured files in S3 (likely CSV or JSON)
  • whenever it's necessary to run queries on the stats, load the files from S3 into AWS Athena

We should also consider any privacy-related concerns around the statistics being aggregated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant