add evaluation

[VladimirKadlec]

Description

Add Evaluation slides

Type of change

• Refactor
• New feature
• Bug fix
• CVE fix
• Optimization
• Documentation Update
• Configuration Update
• Bump-up service version
• Bump-up dependent library
• Bump-up library or tool used for development (does not change the final image)
• CI configuration change
• Konflux configuration change
• Unit tests improvement
• Integration tests improvement
• End to end tests improvement

Related Tickets & Documents

• Related Issue #LCORE-872

Summary by CodeRabbit

• Documentation
  • Expanded evaluation guidance with clearer goals (performance, UX, bias/harm, ethics/legal compliance)
  • Added framework overview with evaluation capabilities including multi-framework support and metrics
  • Included comprehensive configuration examples demonstrating schema details and setup/cleanup workflows

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[coderabbitai]

Walkthrough

The change restructures and expands the Evaluation subsection in the lcore documentation. It replaces outdated evaluation-related bullets with new sections covering evaluation goals, benefits, and the Lightspeed Evaluation Framework. A YAML example is expanded with detailed schema.

Changes

Cohort / File(s)

Summary

Documentation content restructuring and expansion docs/demos/lcore/lcore.md

Replaced outdated Evaluation subsection bullets with new sections: "Why Evaluate an LLM System?" covering performance/UX/bias/ethics goals; "Benefits of Evaluation" describing improvements and benchmarking; and "Lightspeed Evaluation Framework" with framework description and capabilities. Expanded YAML example with conversation_group schema including setup/cleanup scripts, metrics, metadata, turns, per-turn metric overrides, and expected responses/intents.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

• Single file modification with homogeneous, repetitive nature of changes (documentation content only)
• No executable logic or runtime behavior modifications
• Main effort involves verifying documentation accuracy, YAML schema correctness, and consistency of new framework descriptions

Pre-merge checks and finishing touches

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Title check	❓ Inconclusive	The title 'add evaluation' is vague and generic, failing to convey the specific nature of the changes beyond the general addition of evaluation-related content.	Use a more descriptive title that specifies the scope, such as 'Expand Evaluation section with framework overview and YAML examples' or 'Add Lightspeed Evaluation Framework documentation'.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 18ca0fb and 166b168.

📒 Files selected for processing (1)

• docs/demos/lcore/lcore.md (1 hunks)

🧰 Additional context used

🪛 markdownlint-cli2 (0.18.1)

docs/demos/lcore/lcore.md

401-401: Heading style
Expected: atx; Actual: setext

(MD003, heading-style)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)

• GitHub Check: Konflux kflux-prd-rh02 / lightspeed-stack-on-pull-request
• GitHub Check: build-pr
• GitHub Check: e2e_tests (ci)
• GitHub Check: e2e_tests (azure)

🔇 Additional comments (2)

docs/demos/lcore/lcore.md (2)

379-415: Well-structured evaluation narrative and framework documentation.

The new sections provide clear motivation, tangible benefits, and a comprehensive overview of the framework capabilities. The progression from "Why Evaluate" → "Benefits" → "Framework Features" is logical and pedagogically sound for a presentation slide deck. The inline links and bullet structure enhance readability.

---

417-448: Excellent YAML example with comprehensive schema documentation.

The expanded YAML example provides clear documentation of the evaluation configuration structure, including setup/cleanup scripts, conversation-level and per-turn metrics, and expected response/intent fields. Inline comments effectively clarify the purpose of each section. This is valuable reference documentation for users.

[coderabbitai]

⚠️ Potential issue | 🟡 Minor

Resolve markdown linting violation and avoid deprecated HTML.

Line 401 uses the deprecated <font> tag and the --- on line 402 creates a setext-style heading, which violates the markdownlint rule MD003 (heading-style). Modernize this by using an atx-style heading (##).

Apply this diff:

-<font size="10">[https://github.com/lightspeed-core/lightspeed-evaluation/](https://github.com/lightspeed-core/lightspeed-evaluation)</font>
----
+## [Lightspeed Evaluation Framework](https://github.com/lightspeed-core/lightspeed-evaluation/)

This replaces the deprecated <font> tag with a proper markdown heading, aligns with markdown linting standards, and keeps the GitHub link accessible.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	<font size="10">[https://github.com/lightspeed-core/lightspeed-evaluation/](https://github.com/lightspeed-core/lightspeed-evaluation)</font>
	---
	## [Lightspeed Evaluation Framework](https://github.com/lightspeed-core/lightspeed-evaluation/)

🧰 Tools

🪛 markdownlint-cli2 (0.18.1)

401-401: Heading style
Expected: atx; Actual: setext

(MD003, heading-style)

🤖 Prompt for AI Agents

In docs/demos/lcore/lcore.md around lines 401 to 402, the content uses a
deprecated HTML <font> tag and a setext-style heading (the '---') which triggers
markdownlint MD003; replace the HTML and setext underline with a proper
ATX-style markdown heading that contains the GitHub link (e.g., start the line
with "##" followed by the link text or the link itself) so the heading is valid
markdown and the link remains accessible.

[tisnik]

LGTM