using xdg_data_home for report_path

[lanonse]

Summary

I tried using garak within a github workflow and i wanted to upload the report as an artifact. I use a runner as a service with ubuntu, and xdg_data_home returns the path /root/.local/share. With the report being placed in /root/.local/share/garak/garak_runs the github action actions/upload-artifact@v4 returns Error: EACCES: permission denied, lstat /root/.local/share/garak/garak_runs.
Would it be a possibility to have the folder garak_runs as a path relative to the project? That is, are there clear downsides to using something like :

project_path = os.path.abspath(os.path.join(os.path.dirname(__file__)))
report_path = os.path.join(project_path, _config.reporting.report_dir) 

instead of this line
https://github.com/leondz/garak/blob/1e58295e2f8789ac95f72a9b5df8d9f9276322a0/garak/command.py#L68

or at least have both options ?

Thank you,
Lacramioara

Relevant information

I suppose this issue actions/upload-artifact#192 is related to the error i get.

[hadisbabakhani]

It looks like the issue you're facing is due to permission errors when trying to access the /root/.local/share/garak/garak_runs directory in your GitHub Actions workflow. To fix this, you could adjust Garak to store the report in a directory relative to your project instead of a system directory that requires elevated permissions.

You can modify the path in the code to make it relative to your project folder. Here's an approach you could take:

Determine the Project Path: You can set up a relative project path by using Python’s os module to find the current project directory:

import os
project_path = os.path.abspath(os.path.join(os.path.dirname(file)))
report_path = os.path.join(project_path, _config.reporting.report_dir)
This will ensure the garak_runs folder is placed inside your project directory, avoiding permission issues with /root/.local/share.

Current Line Causing Issues: In garak/garak/command.py, line 68:

report_path = _config.transient.data_dir / _config.reporting.report_dir
This line is using a system directory (like /root/.local/share) for the reports, leading to the permission issue. Replacing it with the relative path method shown above should fix the problem.

[leondz]

I use a runner as a service with ubuntu, and xdg_data_home returns the path /root/.local/share. With the report being placed in /root/.local/share/garak/garak_runs the github action actions/upload-artifact@v4 returns Error: EACCES: permission denied, lstat /root/.local/share/garak/garak_runs.

Strongly recommend against running garak as root, which seems like it's what's happening here.

With the report being placed in /root/.local/share/garak/garak_runs the github action actions/upload-artifact@v4 returns Error: EACCES: permission denied, lstat /root/.local/share/garak/garak_runs.

Good to know, thanks

Would it be a possibility to have the folder garak_runs as a path relative to the project?

We disprefer relative paths because they present a security risk. The XDG pattern is also well-supported across platforms.

I think in this case I'd recommend configuring report_dir in a yaml config, to override the default. An absolute value should be possible here. e.g:

report.yaml

---
reporting:
  report_dir: /home/user/garak_runs/

and then invoke garak with something like garak --config report.yaml.

[lanonse]

Thank you for your response. I do not run garak as a root. In my github workflow, when i echo $XDG_DATA_HOME i see "root/.local/share/". The runner is on a server to which i do not have access.

I did try the --config option with an absolute path but something strange happens in that garak says it has written the jsonl inside "garak_runs" but when i list the content "garak_runs" i don't see it:

When i modified the garak code at line 68 as mentioned in my initial message (and install garak from my fork), i have no issue uploading the files in "garak_runs" so the issue i face happens precisely when i give the path to garak_runs as an absolute path. Puzzling.
In any case, as i understand you have security concerns regarding relative paths, i close this issue and try to find a workaround on my own.

Thank you again,
Lacramioara