-
Notifications
You must be signed in to change notification settings - Fork 21
feat(studio): add guardrail Test and Validate tab (checks, runs, detail) #831
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
34 changes: 34 additions & 0 deletions
34
web/packages/studio/src/components/dataViews/GuardrailChecksDataView/checkMessages.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| // SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| import type { GuardrailCheckMessage } from '@studio/api/guardrail-checks/types'; | ||
|
|
||
| /** Coerce a message's `content` (string or content-part array) to plain display text. */ | ||
| const textFromContent = (content: unknown): string => { | ||
| if (typeof content === 'string') { | ||
| return content; | ||
| } | ||
| if (Array.isArray(content)) { | ||
| return content | ||
| .map((part) => | ||
| part && typeof part === 'object' && 'text' in part | ||
| ? String((part as { text?: unknown }).text ?? '') | ||
| : '' | ||
| ) | ||
| .join(' ') | ||
| .trim(); | ||
| } | ||
| return ''; | ||
| }; | ||
|
|
||
| /** First user message text — the check's "Input" (single-turn). */ | ||
| export const getCheckInputText = (messages: GuardrailCheckMessage[]): string => { | ||
| const userMsg = messages.find((m) => m.role === 'user'); | ||
| return userMsg ? textFromContent(userMsg.content) : ''; | ||
| }; | ||
|
|
||
| /** First assistant message text — the check's "Output" (single-turn). */ | ||
| export const getCheckOutputText = (messages: GuardrailCheckMessage[]): string => { | ||
| const assistantMsg = messages.find((m) => m.role === 'assistant'); | ||
| return assistantMsg ? textFromContent(assistantMsg.content) : ''; | ||
| }; |
13 changes: 13 additions & 0 deletions
13
web/packages/studio/src/components/dataViews/GuardrailChecksDataView/checkStatus.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| // SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| import type { GuardrailCheckEntity, Verdict } from '@studio/api/guardrail-checks/types'; | ||
|
|
||
| /** | ||
| * Overall status of a check's most recent run (the `status` returned by the | ||
| * /checks endpoint). `undefined` means the check has never been run. | ||
| */ | ||
| export const getLatestRunStatus = (check: GuardrailCheckEntity): Verdict | undefined => { | ||
| const { runs } = check.data; | ||
| return runs.length ? runs[runs.length - 1].status : undefined; | ||
| }; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
166 changes: 166 additions & 0 deletions
166
web/packages/studio/src/routes/guardrails/GuardrailChecksTab/GuardrailMessageRow.tsx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,166 @@ | ||
| // SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| import { ControlledSelect } from '@nemo/common/src/components/form/ControlledSelect'; | ||
| import { Button, Flex, Stack, Text, Tooltip } from '@nvidia/foundations-react-core'; | ||
| import type { GuardrailCheckMessage } from '@studio/api/guardrail-checks/types'; | ||
| import cn from 'classnames'; | ||
| import { ArrowDown, ArrowUp, Copy, type LucideIcon, Trash2 } from 'lucide-react'; | ||
| import { type FC, useRef } from 'react'; | ||
| import { type Control, type Path, useController } from 'react-hook-form'; | ||
|
|
||
| /** RHF row shape for a single guardrail-check message. No collapse state — the body is always shown. */ | ||
| export interface GuardrailMessageFormRow { | ||
| role: GuardrailCheckMessage['role']; | ||
| content: string; | ||
| } | ||
|
|
||
| export interface GuardrailCheckFormValues { | ||
| messages: GuardrailMessageFormRow[]; | ||
| } | ||
|
|
||
| /** Roles offered in the row's role dropdown (subset of the chat-completion roles). */ | ||
| const ROLE_ITEMS: { value: string; children: string }[] = [ | ||
| { value: 'user', children: 'user' }, | ||
| { value: 'assistant', children: 'assistant' }, | ||
| { value: 'system', children: 'system' }, | ||
| ]; | ||
|
|
||
| /** Plain-text role label (mock shows e.g. "user ⌄", not a colored badge). */ | ||
| const renderRoleValue = (value: string) => <Text kind="label/bold/sm">{value}</Text>; | ||
|
|
||
| /** A single icon action in the row header. */ | ||
| const RowIconButton: FC<{ | ||
| label: string; | ||
| icon: LucideIcon; | ||
| onClick: () => void; | ||
| disabled?: boolean; | ||
| }> = ({ label, icon: Icon, onClick, disabled }) => ( | ||
| <Tooltip slotContent={label} side="top"> | ||
| <Button | ||
| type="button" | ||
| size="tiny" | ||
| kind="tertiary" | ||
| aria-label={label} | ||
| disabled={disabled} | ||
| onClick={onClick} | ||
| > | ||
| <Icon className="size-3.5" aria-hidden /> | ||
| </Button> | ||
| </Tooltip> | ||
| ); | ||
|
|
||
| export interface GuardrailMessageRowProps { | ||
| control: Control<GuardrailCheckFormValues>; | ||
| /** Path to this message row in the form, e.g. `messages.0`. */ | ||
| name: string; | ||
| onMoveUp?: () => void; | ||
| onMoveDown?: () => void; | ||
| onDuplicate: () => void; | ||
| onRemove: () => void; | ||
| /** When false, the delete action is disabled (never remove the last message). */ | ||
| allowRemove: boolean; | ||
| dataTestId?: string; | ||
| } | ||
|
|
||
| /** | ||
| * One guardrail-check message: a role dropdown + an always-visible, manually resizable | ||
| * text body, with reorder / duplicate / delete actions. Purpose-built for the guardrails | ||
| * Tests tab (replaces the shared ChatCompletionInput). | ||
| * | ||
| * The row is a single bordered card with a native textarea — no nested input shell — so the | ||
| * body has exactly one border (which highlights on focus) rather than a doubled one. | ||
| */ | ||
| export const GuardrailMessageRow: FC<GuardrailMessageRowProps> = ({ | ||
| control, | ||
| name, | ||
| onMoveUp, | ||
| onMoveDown, | ||
| onDuplicate, | ||
| onRemove, | ||
| allowRemove, | ||
| dataTestId, | ||
| }) => { | ||
| const rolePath = `${name}.role` as Path<GuardrailCheckFormValues>; | ||
| const contentPath = `${name}.content` as Path<GuardrailCheckFormValues>; | ||
| const { field: content } = useController({ control, name: contentPath }); | ||
| const bodyRef = useRef<HTMLTextAreaElement>(null); | ||
|
|
||
| // After a role is chosen the Select closes and restores focus to its trigger at an unknown tick. | ||
| // Re-focus the message body across a short window so it wins whenever that restoration fires, | ||
| // keeping the user in the typing flow. | ||
| const focusBody = () => { | ||
| let ticks = 0; | ||
| const id = setInterval(() => { | ||
| bodyRef.current?.focus(); | ||
| if (++ticks >= 8) clearInterval(id); | ||
| }, 25); | ||
| }; | ||
|
nakolean marked this conversation as resolved.
|
||
|
|
||
| return ( | ||
| <Stack | ||
| gap="density-sm" | ||
| className={cn( | ||
| 'rounded-md border border-interaction-base bg-surface-base px-density-md py-density-sm', | ||
| 'transition-[border-color] focus-within:border-[var(--border-color-brand)]' | ||
| )} | ||
| {...(dataTestId ? { 'data-testid': dataTestId } : {})} | ||
| > | ||
| <Flex align="center" justify="between" gap="density-sm"> | ||
| {/* Role dropdown: plain text + chevron, sized to the selected role (not the widest). */} | ||
| <div className="w-max shrink-0"> | ||
| <ControlledSelect | ||
| useControllerProps={{ control, name: rolePath }} | ||
| items={ROLE_ITEMS} | ||
| renderValue={renderRoleValue} | ||
| onChange={focusBody} | ||
| hideError | ||
| attributes={{ | ||
| SelectTrigger: { | ||
| className: cn( | ||
| 'w-max border-none bg-transparent shadow-none items-center gap-density-xs', | ||
| 'h-6 min-h-0 max-h-6 p-0 [&_button]:!px-0 [&_*]:!min-w-max', | ||
| 'data-[state=open]:shadow-none' | ||
| ), | ||
| }, | ||
| // Menu width is decoupled from the (content-hugging) trigger so item labels aren't clipped. | ||
| SelectContent: { className: 'min-w-[8rem]' }, | ||
| }} | ||
| /> | ||
| </div> | ||
|
|
||
| <Flex align="center" gap="density-xs" className="shrink-0 text-text-secondary"> | ||
| {onMoveUp ? <RowIconButton label="Move up" icon={ArrowUp} onClick={onMoveUp} /> : null} | ||
| {onMoveDown ? ( | ||
| <RowIconButton label="Move down" icon={ArrowDown} onClick={onMoveDown} /> | ||
| ) : null} | ||
| <RowIconButton label="Duplicate message" icon={Copy} onClick={onDuplicate} /> | ||
| <RowIconButton | ||
| label="Delete message" | ||
| icon={Trash2} | ||
| onClick={onRemove} | ||
| disabled={!allowRemove} | ||
| /> | ||
| </Flex> | ||
| </Flex> | ||
|
|
||
| <textarea | ||
| name={content.name} | ||
| ref={(el) => { | ||
| content.ref(el); | ||
| bodyRef.current = el; | ||
| }} | ||
| value={typeof content.value === 'string' ? content.value : ''} | ||
| onChange={content.onChange} | ||
| onBlur={content.onBlur} | ||
| placeholder="Type your message..." | ||
| aria-label="Message content" | ||
| data-testid="guardrail-check-message-content" | ||
| className={cn( | ||
| 'min-h-[3.5rem] w-full resize-y border-none bg-transparent p-0 outline-none', | ||
| 'text-sm leading-normal text-text-primary placeholder:text-muted focus:outline-none' | ||
| )} | ||
| /> | ||
| </Stack> | ||
| ); | ||
| }; | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.