Skip to content

Commit 7e80aaa

Browse files
authored
Math normalization: do not crash on invalid format (#331)
Wrap the asserts in a try except so that malformatted generations do not crash lighteval. See the original implem: https://github.com/hendrycks/math/blob/main/modeling/eval_math_gpt.py
1 parent ba20248 commit 7e80aaa

File tree

1 file changed

+11
-8
lines changed

1 file changed

+11
-8
lines changed

src/lighteval/metrics/normalizations.py

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -98,17 +98,20 @@ def _remove_boxed(text: str | None) -> str:
9898
"""
9999
if text is None:
100100
return ""
101-
if "\\boxed " in text:
102-
left = "\\boxed "
103-
assert text[: len(left)] == left
104-
return text[len(left) :]
101+
try:
102+
if "\\boxed " in text:
103+
left = "\\boxed "
104+
assert text[: len(left)] == left
105+
return text[len(left) :]
105106

106-
left = "\\boxed{"
107+
left = "\\boxed{"
107108

108-
assert text[: len(left)] == left
109-
assert text[-1] == "}"
109+
assert text[: len(left)] == left
110+
assert text[-1] == "}"
110111

111-
return text[len(left) : -1]
112+
return text[len(left) : -1]
113+
except Exception:
114+
return ""
112115

113116
def _last_boxed_only_string(text: str) -> str | None:
114117
"""Extract the last \\boxed{...} or \\fbox{...} element from a string."""

0 commit comments

Comments
 (0)