Skip to content

Issues: UKGovernmentBEIS/inspect_ai

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

33 Open 236 Closed
33 Open 236 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

hf_dataset: None or 0-length values to be mapped to input field cause dataset loading failure
#1160 opened Jan 20, 2025 by stefanTrawicki
1
Enable sandboxes to connect to existing Docker or k8s instances
#1153 opened Jan 19, 2025 by jjallaire
@craigwalton-dsit
Enforcing cost limits
#1152 opened Jan 19, 2025 by jjallaire
@craigwalton-dsit
Ability to use agents with no Inspect dependencies
#1151 opened Jan 19, 2025 by jjallaire
@jjallaire
Broader notions of "model" for evals
#1149 opened Jan 19, 2025 by jjallaire
@jjallaire
Bridge to convert from compose.yaml to HELM values
#1148 opened Jan 19, 2025 by jjallaire
@craigwalton-dsit
1
Improve task scheduler to not require model as an axis
#1146 opened Jan 19, 2025 by jjallaire
@jjallaire
Direct linking to samples and transcript entries in Inspect View
#1145 opened Jan 19, 2025 by jjallaire
@dragonstyle
Checkpointing and restarting from checkpoints
#1144 opened Jan 19, 2025 by jjallaire
@jjallaire
Log annotation UI
#1143 opened Jan 19, 2025 by jjallaire
@dragonstyle
Live updating log viewer
#1142 opened Jan 19, 2025 by jjallaire
@dragonstyle
Recording pauses for rate-limiting and transient errors
#1139 opened Jan 19, 2025 by jjallaire
@jjallaire
Documentation Request
#1136 opened Jan 18, 2025 by eoghanf
1
Feature request: global token cap
#1135 opened Jan 17, 2025 by edkins
GAIA web browser tool returns <empty> when affected by packet loss
#988 opened Dec 12, 2024 by alex-treebeard
viewer: display cost estimates in usage tab
#980 opened Dec 10, 2024 by jjallaire-aisi
support fractional unicode code points (e.g. ½, ¾) in numeric normalisation.
#897 opened Nov 25, 2024 by jjallaire
Add more documentation on CLI
#868 opened Nov 20, 2024 by mrahtz
1
[Question] Custom metric type
#775 opened Oct 30, 2024 by us2547
2
@dragonstyle
6
Adding a scorer without rerunning existing scorers.
#770 opened Oct 27, 2024 by sohaibimran7
2
Failed epoch doesn't stop rest of task's epochs
#746 opened Oct 23, 2024 by craigwalton-dsit
Web browser struggles with large pages web browser
#740 opened Oct 21, 2024 by art-dsit
1
plot_results(): Are there any frameworks that allow summarising and visualising inspect logs?
#704 opened Oct 16, 2024 by sohaibimran7
2
support for web browser navigations that open new windows web browser
#681 opened Oct 10, 2024 by jjallaire-aisi
@MariaIzobava
Error running inspect score with a scoring function that uses a task-level sandbox
#679 opened Oct 10, 2024 by bienehito
ProTip! Exclude everything labeled bug with -label:bug.