-
Notifications
You must be signed in to change notification settings - Fork 8.6k
Added markdown links to ai insgihts and obs agent #250030
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 13 commits
7da0a64
e6e40fa
723e75e
87254ab
793cc6d
c541789
deb9333
c5f9cba
fb47965
b6e6c13
454aa19
7729f92
fcee958
d7b4432
26d6bf9
e7b600b
fe7ba06
01fe739
a59e57b
f7df33b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -17,34 +17,39 @@ import type { | |
| ObservabilityAgentBuilderPluginSetupDependencies, | ||
| } from '../../../types'; | ||
| import { fetchApmErrorContext } from './fetch_apm_error_context'; | ||
| import { getEntityLinkingInstructions } from '../../../agent/register_observability_agent'; | ||
| import type { AiInsightResult, ContextEvent } from '../types'; | ||
|
|
||
| const ERROR_AI_INSIGHT_SYSTEM_PROMPT = dedent(` | ||
| You are an expert SRE Assistant within Elastic Observability. Your job is to analyze an APM error using ONLY the provided context (APM trace items, related errors, downstream dependencies, and log categories). | ||
| function getErrorAiInsightSystemPrompt(spaceId?: string) { | ||
| return dedent(` | ||
| You are an expert SRE Assistant within Elastic Observability. Your job is to analyze an APM error using ONLY the provided context (APM trace items, related errors, downstream dependencies, and log categories). | ||
|
|
||
| Output structure (concise, Markdown): | ||
| - Error summary (1-2 sentences): What is observed and why it matters. | ||
| - Failure pinpoint: Whether failure is in application code vs dependency. Name the likely failing component/endpoint. Reference specific fields or key frames if available. | ||
| - Impact: Scope and severity (services/endpoints and the extent of the error if evident). | ||
| - Immediate actions (2-4): Ordered, concrete steps (config/network checks, retries/backoff, circuit breakers, targeted tracing/logging). | ||
| - Open questions: Short list of unknowns and the quickest queries to resolve them, if any (this is strictly optional and should not be present if there are no open questions). | ||
|
|
||
| Output structure (concise, Markdown): | ||
| - Error summary (1-2 sentences): What is observed and why it matters. | ||
| - Failure pinpoint: Whether failure is in application code vs dependency. Name the likely failing component/endpoint. Reference specific fields or key frames if available. | ||
| - Impact: Scope and severity (services/endpoints and the extent of the error if evident). | ||
| - Immediate actions (2-4): Ordered, concrete steps (config/network checks, retries/backoff, circuit breakers, targeted tracing/logging). | ||
| - Open questions: Short list of unknowns and the quickest queries to resolve them, if any (this is strictly optional and should not be present if there are no open questions). | ||
| Guardrails: | ||
| - Strict Factuality: Only mention signals present in the JSON. If a signal is missing, do not mention it. | ||
| - Only assert a cause if multiple signals support it. Otherwise mark Assessment "Inconclusive". | ||
| - Prefer corroborated explanations. If only one source supports it, state that support is limited. | ||
| - Do NOT repeat raw stacks verbatim (reference only key frames/fields). | ||
| - Conciseness: Use bullet points. Avoid flowery language. Be direct and technical. | ||
|
|
||
| Guardrails: | ||
| - Strict Factuality: Only mention signals present in the JSON. If a signal is missing, do not mention it. | ||
| - Only assert a cause if multiple signals support it. Otherwise mark Assessment "Inconclusive". | ||
| - Prefer corroborated explanations. If only one source supports it, state that support is limited. | ||
| - Do NOT repeat raw stacks verbatim (reference only key frames/fields). | ||
| - Conciseness: Use bullet points. Avoid flowery language. Be direct and technical. | ||
| Available context tags: | ||
| - <ErrorDetails>: Full error document (exception, message, stacktrace, labels) | ||
| - <TransactionDetails>: Transaction linked to the error (if present) | ||
| - <DownstreamDependencies>: Downstream dependencies for the erroring service | ||
| - <TraceItems>: Span/transaction samples with service, name, type, eventOutcome, statusCode, duration, httpUrl, downstreamServiceResource | ||
| - <TraceErrors>: Related errors within the trace (type, message, culprit, spanId, timestampUs) | ||
| - <TraceServices>: Service aggregates for the trace (serviceName, count, errorCount) | ||
| - <TraceLogCategories>: Categorized log patterns tied to the trace (errorCategory, docCount, sampleMessage) | ||
|
|
||
| Available context tags: | ||
| - <ErrorDetails>: Full error document (exception, message, stacktrace, labels) | ||
| - <TransactionDetails>: Transaction linked to the error (if present) | ||
| - <DownstreamDependencies>: Downstream dependencies for the erroring service | ||
| - <TraceItems>: Span/transaction samples with service, name, type, eventOutcome, statusCode, duration, httpUrl, downstreamServiceResource | ||
| - <TraceErrors>: Related errors within the trace (type, message, culprit, spanId, timestampUs) | ||
| - <TraceServices>: Service aggregates for the trace (serviceName, count, errorCount) | ||
| - <TraceLogCategories>: Categorized log patterns tied to the trace (errorCategory, docCount, sampleMessage) | ||
| `); | ||
| ${getEntityLinkingInstructions(spaceId)} | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nice to see you are importing the instruction here. This got me thinking: what about the other instructions: Should they be included as well? How are you handling this for alerts and log AI insights? @viduni94 @neptunian?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Actually, nevermind. The other instructions are only relevant when doing tool calling, which the AI insight is not doing atm. So probably only the instructions related to formatting (like markdown links) are needed. |
||
| `); | ||
| } | ||
|
|
||
| const buildUserPrompt = (errorContext: string) => { | ||
| return dedent(` | ||
|
|
@@ -62,6 +67,7 @@ const buildUserPrompt = (errorContext: string) => { | |
| export interface GenerateErrorAiInsightParams { | ||
| core: ObservabilityAgentBuilderCoreSetup; | ||
| plugins: ObservabilityAgentBuilderPluginSetupDependencies; | ||
| spaceId: string; | ||
| errorId: string; | ||
| serviceName: string; | ||
| environment?: string; | ||
|
|
@@ -77,6 +83,7 @@ export interface GenerateErrorAiInsightParams { | |
| export async function generateErrorAiInsight({ | ||
| core, | ||
| plugins, | ||
| spaceId, | ||
| errorId, | ||
| serviceName, | ||
| environment, | ||
|
|
@@ -103,7 +110,7 @@ export async function generateErrorAiInsight({ | |
| const userPrompt = buildUserPrompt(errorContext); | ||
|
|
||
| const events$ = inferenceClient.chatComplete({ | ||
| system: ERROR_AI_INSIGHT_SYSTEM_PROMPT, | ||
| system: getErrorAiInsightSystemPrompt(spaceId), | ||
| messages: [ | ||
| { | ||
| role: MessageRole.User, | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for adding the space path changes.
How about the base path?
The URLs will result in 404 if there is a base path configured.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If possible, you could try to use this shared util by agent builder which handles the space and base path:
kibana/x-pack/platform/plugins/shared/agent_builder/server/utils/get_kibana_url.ts
Lines 13 to 34 in f79649a