Skip to content

[NL2ESQL] Fix tool going into endless loop for ES|QL LOOKUP JOIN #231217

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
qn895 merged 4 commits into from
Aug 12, 2025
Merged

[NL2ESQL] Fix tool going into endless loop for ES|QL LOOKUP JOIN #231217

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ff05bff
Fix lookup joint
qn895 Aug 8, 2025
d4b621b
Merge remote-tracking branch 'upstream/main' into ai-fix-lookup-join-…
qn895 Aug 11, 2025
ad187de
Merge branch 'main' into ai-fix-lookup-join-loop
elasticmachine Aug 12, 2025
d908399
Readd tests
qn895 Aug 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion x-pack/platform/plugins/shared/inference/server/tasks/nl_to_esql/doc_base/aliases.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
const aliases: Record<string, string[]> = {
STATS: ['STATS_BY', 'BY', 'STATS...BY', 'STATS ... BY'],
OPERATORS: ['LIKE', 'RLIKE', 'IN'],
JOIN: ['LOOKUP JOIN'],
LOOKUP_JOIN: ['LOOKUPJOIN'],
};

const getAliasMap = () => {
Expand Down
3 changes: 1 addition & 2 deletions x-pack/platform/plugins/shared/inference/server/tasks/nl_to_esql/doc_base/esql_doc_base.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,7 @@ export class EsqlDocumentBase {
}: GetDocsOptions = {}
) {
const keywords = rawKeywords.map((raw) => {
// LOOKUP JOIN has space so we want to retain as is
let keyword = raw.toLowerCase().includes('join') ? raw : format(raw);
let keyword = format(raw);
if (resolveAliases) {
keyword = tryResolveAlias(keyword);
}
Expand Down
34 changes: 16 additions & 18 deletions ...solutions/observability/packages/kbn-evals-suite-obs-ai-assistant/evals/esql/esql.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -740,23 +740,21 @@ evaluate.describe('ES|QL query generation', { tag: '@svlOblt' }, () => {
},
metadata: {},
},
// This results in a timeout because the NL-to-ESQL tasks goes into a loop
// See trace: https://35-187-109-62.sslip.io/projects/UHJvamVjdDo5/traces/630f92adcb3620295794180f71ccb37a?selectedSpanNodeId=U3BhbjozODYxNTQ%3D
// {
// input: {
// question:
// "Assume user login data is logs-auth_service-*. The event action for user login is `login`. Generate an example query to fetch successful logins today and for each successful login, show the user's full name and department. The user meta data is in the users_metadata index.",
// },
// output: {
// expected: `FROM logs-auth_service-*
// | WHERE @timestamp >= NOW() - 1 day AND event.action == "login"
// | LOOKUP JOIN users_metadata ON user.id
// | KEEP @timestamp, user.id, full_name, department
// | LIMIT 20`,
// execute: false,
// },
// metadata: {},
// },
{
input: {
question:
"Assume user login data is logs-auth_service-*. The event action for user login is `login`. Generate an example query to fetch successful logins today and for each successful login, show the user's full name and department. The user meta data is in the users_metadata index.",
},
output: {
expected: `FROM logs-auth_service-*
| WHERE @timestamp >= NOW() - 1 day AND event.action == "login"
| LOOKUP JOIN users_metadata ON user.id
| KEEP @timestamp, user.id, full_name, department
| LIMIT 20`,
execute: false,
},
metadata: {},
},
],
},
});
Expand Down Expand Up @@ -979,7 +977,7 @@ evaluate.describe('ES|QL query generation', { tag: '@svlOblt' }, () => {
expected: [
`FROM main
| WHERE user_id IN (FROM suspicious_users | KEEP user_id)`,
`FROM main
`FROM main
| LOOKUP JOIN suspicious_users ON user_id`,
],
execute: false,
Expand Down
6 changes: 2 additions & 4 deletions ...ty/plugins/observability_ai_assistant_app/scripts/evaluation/scenarios/esql/index.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -667,9 +667,7 @@ describe('ES|QL query generation', () => {
});
});

// This works on the Kibana UI but times out via the /chat/complete API
// See trace: https://35-187-109-62.sslip.io/projects/UHJvamVjdDo5/traces/630f92adcb3620295794180f71ccb37a?selectedSpanNodeId=U3BhbjozODYxNTQ%3D
it.skip('enriches login logs with user metadata using a lookup join', async () => {
it('enriches login logs with user metadata using a lookup join', async () => {
await evaluateEsqlQuery({
question:
"Assume user login data is logs-auth_service-*. The event action for user login is `login`. Generate an example query to fetch successful logins today and for each successful login, show the user's full name and department. The user meta data is in the users_metadata index.",
Expand Down Expand Up @@ -868,7 +866,7 @@ describe('ES|QL query generation', () => {
expected: [
`FROM main
| WHERE user_id IN (FROM suspicious_users | KEEP user_id)`,
`FROM main
`FROM main
| LOOKUP JOIN suspicious_users ON user_id`,
],
execute: false,
Expand Down