[Obs AI Assistant] Add timestamp to title when the conversation is generated by the rule connector

[viduni94]

Closes #185022

Summary

Problem

When the Obs AI Assistant is set as the action via the Rule Connector, the conversations generated for the alert has the same title. Therefore, it's hard to distinguish between the different conversations generated by the rule connector.

Solution

Append a timestamp to the title generated for conversations generated by the rule connector.

Few notes about the implementation:

• the isPublic variable that already existed as a chat complete property was used to determine whether the conversation was generated by the rule connector (if a new variable such as isGeneratedByRuleConnector is preferred instead of re-using isPublic, let me know, I can update the code).
• The timestamp format is Nov 4, 2024, 08:28:56.817 (UTC-5)

Screenshots

Checklist

• Unit or functional tests were updated or added to match the most common scenarios

[elasticmachine]

Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant)

[github-actions]

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

• /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[dgieselaar]

@viduni94 is this from repeated runs on the same alert? The title is still generated by the LLM, I assume. Is it an option to perhaps increase the temperature instead?

[viduni94]

@viduni94 is this from repeated runs on the same alert? The title is still generated by the LLM, I assume. Is it an option to perhaps increase the temperature instead?

@dgieselaar
Yeah this is for repeated runs of the same alert. Adding the timestamp was suggested in the issue description.

I can try adjusting the temperature for more dynamic titles.. however, I didn't find a way to update the temperature only for a certain function call.

If I update the temperature in here, it will update the temperature for all messages..

Let me know if you have any suggestions.

[dgieselaar]

@viduni94 Yeah we should be able to add a top-level temperature setting, which is useful anyway. but maybe you can first try tweaking the temperature a bit to see if we get more variable results?

[viduni94]

@viduni94 Yeah we should be able to add a top-level temperature setting, which is useful anyway. but maybe you can first try tweaking the temperature a bit to see if we get more variable results?

Hey @dgieselaar
I tried title generation with a higher temperature (0.8), and it does provide slightly varied titles.. however, I feel like even with slightly different titles, it's hard to distinguish between each conversation.

Do you recommend having a higher temperature over adding the timestamp?

[dgieselaar]

@viduni94 I see, that doesn't help a lot. One thing I'm unsure about is whether we'll be able to show those long titles, IIRC in the unification working group I saw designs where the title is cut off. Another option we have is showing the timestamp or some relative diff (like "30 minutes ago") above the title. Also curious what others feel is a good option

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 4897fab
• Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-198791-4897fab407a9

Metrics [docs]

✅ unchanged

History

• 💚 Build #248083 succeeded b0aeed3

cc @viduni94

[viduni94]

@viduni94 I see, that doesn't help a lot. One thing I'm unsure about is whether we'll be able to show those long titles, IIRC in the unification working group I saw designs where the title is cut off. Another option we have is showing the timestamp or some relative diff (like "30 minutes ago") above the title. Also curious what others feel is a good option

@dgieselaar
I see the cut-off title in the unified UX. The timestamp would appear in the tooltip in that case.

If we are adding the timestamp on top of the conversation, this would also be removed as part of the unification is it? As it's not there in the mocks. Or should we request the design team for a mock with timestamps?

I'll post on our team channel and see whether anyone has any ideas.

[sorenlouv]

Also curious what others feel is a good option

Not a big fan of increasing temperature. Slightly different but very similar titles are not going to help imo. I don't know what the best option is but I think a reasonable bandaid is adding a timestamp. I'm also not sure how big this problem is so I wonder how much we need to invest in solving it.

[sorenlouv]

... we also have this issue that aims to improve the list of conversations by categorising by time. I think that may also help in this case.

[viduni94]

... we also have this issue that aims to improve the list of conversations by categorising by time. I think that may also help in this case.

Thank you @sorenlouv

[emma-raffenne]

@viduni94 @sorenlouv @dgieselaar

Conversations generated from alerting may end up with the same title for the same alert type which makes very complicated to find them afterwards. As @sorenlouv mentioned, adding the timestamp is a reasonable bandaid for the current UI while we agree on and implement a UX for the unified assistant.

FYI @teknogeek0

[viduni94]

@viduni94 @sorenlouv @dgieselaar

Conversations generated from alerting may end up with the same title for the same alert type which makes very complicated to find them afterwards. As @sorenlouv mentioned, adding the timestamp is a reasonable bandaid for the current UI while we agree on and implement a UX for the unified assistant.

FYI @teknogeek0

Thank you @emma-raffenne
I'll go ahead and merge this PR today.

[dgieselaar]

Conversations generated from alerting may end up with the same title for the same alert type which makes very complicated to find them afterwards. As @sorenlouv mentioned, adding the timestamp is a reasonable bandaid for the current UI while we agree on and implement a UX for the unified assistant.

TBH, I think it's a little weird, I'm sure there's a better solution (and also curious if this is actually an issue or mostly something that just comes out of the way we have evaluated this connector, which is to repeatedly trigger it over a short time frame). It's a small change, so we can just get it in, but as a matter of principle it would also be good to reflect on whether this is a sunk cost fallacy thing, or if we're really adding value. We are adding 150 lines of code.

[viduni94]

After a discussion with the team, it was decided to not proceed with this solution.

Instead, the timestamp should be placed on the UI in a better way.
This issue is related to the above - #176295

cc: @dgieselaar @emma-raffenne @teknogeek0 @sorenlouv