Skip to content

feat(evaluator): add metric model resolvers - #38

Merged
SandyChapman merged 1 commit into
mainfrom
4050-evaluator-metric-resolvers/schapman
May 29, 2026
Merged

feat(evaluator): add metric model resolvers#38
SandyChapman merged 1 commit into
mainfrom
4050-evaluator-metric-resolvers/schapman

Add evaluator metric model resolvers

9dcbc66
Select commit
Loading
Failed to load commit list.
GitHub Advanced Security / CodeQL succeeded May 29, 2026 in 2s

9 new alerts

New alerts in code changed by this pull request

  • 9 notes

See annotations below for details.

View all branch alerts.

Annotations

Check notice on line 100 in packages/nemo_evaluator_sdk/src/nemo_evaluator_sdk/metrics/protocol.py

See this annotation in the file changed.

Code scanning / CodeQL

Statement has no effect Note

This statement has no effect.

Check notice on line 203 in packages/nemo_evaluator_sdk/src/nemo_evaluator_sdk/metrics/protocol.py

See this annotation in the file changed.

Code scanning / CodeQL

Statement has no effect Note

This statement has no effect.

Check notice on line 20 in packages/nemo_evaluator_sdk/src/nemo_evaluator_sdk/resolver_protocols.py

See this annotation in the file changed.

Code scanning / CodeQL

Statement has no effect Note

This statement has no effect.

Check notice on line 29 in packages/nemo_evaluator_sdk/src/nemo_evaluator_sdk/resolver_protocols.py

See this annotation in the file changed.

Code scanning / CodeQL

Statement has no effect Note

This statement has no effect.

Check notice on line 31 in packages/nemo_evaluator_sdk/src/nemo_evaluator_sdk/values/metrics.py

See this annotation in the file changed.

Code scanning / CodeQL

Unused import Note

Import of 'default_judge_prompt_template_chat' is not used.

Check notice on line 34 in packages/nemo_evaluator_sdk/src/nemo_evaluator_sdk/values/metrics.py

See this annotation in the file changed.

Code scanning / CodeQL

Unused import Note

Import of 'default_judge_prompt_template_completions' is not used.

Check notice on line 37 in packages/nemo_evaluator_sdk/src/nemo_evaluator_sdk/values/metrics.py

See this annotation in the file changed.

Code scanning / CodeQL

Unused import Note

Import of 'is_chat_inference' is not used.

Check notice on line 40 in plugins/nemo-evaluator/src/nemo_evaluator/resolvers.py

See this annotation in the file changed.

Code scanning / CodeQL

Statement has no effect Note

This statement has no effect.

Check notice on line 48 in plugins/nemo-evaluator/src/nemo_evaluator/resolvers.py

See this annotation in the file changed.

Code scanning / CodeQL

Statement has no effect Note

This statement has no effect.