Log categories

[katrin-freihofner]

Summary of the problem (If there are multiple problems or use cases, prioritize them)
As a user I want to investigate

• logs per category
• rare log messages(we are going to focus on categories for now)

Filter/Search option
Search for free text
Filter timestamp and dataset

Next steps

Prep

• analyze available example data in kibana dashboard
• get use cases and requirements docu

Design

• create a basic user flow diagram(exploratory use case, therefore this is not needed)
• create wireframes (low-fi) for each of the main usecases
• think about licensing
• think about the experience for new users
• grok patterns vs. more specific log lines -> will be covered in latest mockups (comment from 11-11-2019)
• do we need to show example logs? Where and how? -> The first version of this feature will be without any examples.
• how could we show a trend?

Implementation
elastic/kibana#42776

[katrin-freihofner]

WIP

Layout

Categories

a)

b)

Rare logs

[weltenwort]

This is a great start, thank you.

Looking at the mock-ups lots of questions come to mind about the semantics of the chart, the column headers, the search bar and a few more. There's a lot of room for interpretation, so I feel I should wait until you're ready to give a walk-through and give a few details on those aspects.

[katrin-freihofner]

Notes from yesterdays zoom meeting:

• We will focus on categories for now (in both, design and development)
• We will show a table/list of top categories (top 25) by message count
• Additionally, we are going to show the max anomaly score or the no. of (severe) anomalies in the table
• the table can be sorted (+pagination)
• indicate dataset (it's rare, but one cat. could have multiple datasets)
• We are going to work on a visualization of the grok pattern/regex that is easy to grasp (messages can be very long)
• Categories will have context actions
• show sparklines/trends of categories

Next

• Show examples for categories
• Examples can have context actions

[katrin-freihofner]

A quick sketch of how the things we agreed on (see comment above) could look like

[weltenwort]

That looks great. What does the "+201" besides the sparkline mean?

[katrin-freihofner]

"+ 201 messages compared to the previous timeframe"

I think a number added to the sparkline could make sense, especially to indicate tiny changes? What is your opinion on that?

[weltenwort]

So you mean it's the difference between the displayed count (calculated by summing over the values in (startTime, startTime + timeframeDuration)) and the sum over the values within (startTime - timeframeDuration, startTime)?

[katrin-freihofner]

I'm thinking about adding a visual indicator for the dataset.

[jasonrhodes]

I like where this is headed -- have we considered using the non regex version of the pattern? I think they are both pretty similar in information but losing all of the .+? characters would make it much easier to parse at a glance, I think...