diff --git a/FAQ.md b/FAQ.md index 27e76c5f7..feec2da05 100644 --- a/FAQ.md +++ b/FAQ.md @@ -87,9 +87,25 @@ Recommended practice: it's really context dependent. The builtin "fast" config w We update garak by improving existing probes or adding new ones quite frequently, and so scores will go down over time - garak isn't a benchmark, and the more we learn about failures in LLMs, the harder garak gets. But if you're looking at a short period of just a month or two, then the scores will probably stay pretty much the same. We do not recommend relying on scores over six months old. +## How can I create my own generator? + +Adding a custom generator is fairly straight forward. One can either add a new class in the existing module, or a new module in the `generators/` directory with a class that extends garak.generators.base.Generator that will be loaded at runtime. The reference documentation has a [full guide to creating garak generators](https://reference.garak.ai/en/latest/contributing.generator.html). + +## How can I redirect `garak_runs/` and `garak.log` to another place instead of `~/.local/share/garak/`? + +* `garak_runs` is configured via top-level config param `reporting.report_dir` and also CLI argument `--report_prefix` (which currently can include directory separator characters, so an absolute path can be given) +* An example of the location of the config param can be seen in https://github.com/leondz/garak/blob/main/garak/resources/garak.core.yaml +* If `reporting.report_dir` is set to an absolute path, you can move it anywhere +* If it's a relative path, it will be within the garak directory under the "data" directory following the cross-platform [XDG base directory specification](https://specifications.freedesktop.org/basedir-spec/latest/) for local storage +* There's no CLI or config option for moving `garak.log`, which is also stored in the XDG data directory +* We would welcome a PR implementing configurability of logfile path +* The Python implementation of XDG that garak uses allows overriding the data directory using the `XDG_DATA_HOME` environment variable +* An alternative is to symlink the paths to where you want them to be