Caching samples PR #909
Caching samples PR #909
Conversation
…1)test with DP 2) add a system where we load cached samples in mem *after* processing the other items
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
…n theory - added docs on caching
… idea why the imports did not fail
| GenerativeResponse, | ||
| LoglikelihoodResponse, | ||
| LoglikelihoodSingleTokenResponse, | ||
| ModelResponse, |
There was a problem hiding this comment.
Clean up imports, unrelated to the PR
| GenerativeResponse, | ||
| LoglikelihoodResponse, | ||
| LoglikelihoodSingleTokenResponse, | ||
| ModelResponse, |
There was a problem hiding this comment.
Clean up imports, unrelated to the PR
| config = yaml.safe_load(f)["model_parameters"] | ||
| else: | ||
| # We extract the model args | ||
| config: dict = ModelConfig._parse_args(model_args) |
There was a problem hiding this comment.
Clean up unused params to simplify tests
| def loglikelihood_rolling(self, docs: list[Doc], override_bs=None) -> list[ModelResponse]: | ||
| return self._loglikelihood(docs, rolling=True) | ||
|
|
||
| def _loglikelihood(self, docs: list[Doc], rolling: bool = False) -> list[ModelResponse]: |
There was a problem hiding this comment.
Grouped logic of both functions, and separated the API function from the logic to avoid breaking changes in people's pipeline if we change the core logic
| ) | ||
|
|
||
| def loglikelihood(self, requests: List[LoglikelihoodRequest]) -> List[LoglikelihoodResponse]: | ||
| @cached("predictions") |
There was a problem hiding this comment.
Removed the single token loglikelihood since it should have been removed with the refacto, plus some cleaning up of imports
| Attributes: | ||
| base_model (str): | ||
| HuggingFace Hub model ID or path to the base model. This is the original |
There was a problem hiding this comment.
removed unused params
| during loading to reconstruct the full fine-tuned model. | ||
| Attributes: | ||
| base_model (str): |
There was a problem hiding this comment.
removed unused useless param
clefourrier
force-pushed
the
test_caching
branch
from
8da1750 to
35b7e94
Compare
Adds a new caching system for generative evals, plus test suite, plus doc - the system loads indices first, then runs samples as needed, then lastly loads the cached items as needed. (We don't keep the cache in mem when running models). Contains a test suite and doc page
Todos
Predictions
We'll need to tokenize inputs later.