Fixing Exception Cause By Type Error When Scanning LLMs Via Replicate #401
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi professor,
Recently I launched a garak scan using debug mode of vscode with the following configuration:
But I got exception raised by
....../python3.11/site-packages/backoff/_sync.py:106
:input.seed: Invalid type. Expected: integer, given: null
Finally I found the reason of the exception:
The value assignment operation of
self.seed = 9
ingarak/generators/replicate.py:38
has been proved invalid according to the collision described below:garak/resources/garak.core.yaml:9
: the attributeseed
has a default value "empty", which is regarded asNone
in Python syntax;None
value could assign toself.seed
of classReplicateGenerator
byself.seed = _config.run.seed
ingarak/generators/replicate.py:40
andgenerator.seed = _config.run.seed
ingarak/cli.py:445
;garak/generators/replicate.py:63
needsself.seed
to beint
type but it isNone
type.To address this problem, I made some modifications to avoid value assignment operation when
self.seed
appears toNone
type. I believe those modifications could enhance the robustness of the framework.Please review the code comparison for detail. Any problems, feel free to point it out. 😊