enable HFDetector model configuration with hf_args

[jmartin-tech]

Fix #803

Allows hf_args configuration for simple HFDetectors:

• default to cpu device when no configuration is provided for the detector
• use AutoConfig to pass device to from_pretrained
• ensure MustContradictNLI.detect() places prompt for evaluation in the configured device space
• enable configuration of detector_model_path and detector_target_class

Example config of detector using cuda:
detector_cuda.yaml

plugins:
  detectors:
    misleading:
      MustContradictNLI:
        hf_args:
          device: cuda

The format above is based on HFDetector being a class specifically designed to use a TextClassificationPipeline. Is the is the approach desired or is further abstraction of ModelAsJudge be a goal of this revision?

Test example:

python -m garak -m nim -n meta/llama3-8b-instruct -p misleading.FalseAssertion50 --config detector_cuda.yaml

Logs and nvidia-smi show detector model loading in GPU.

More validation of compatible models may be desired if this pattern is determined to be a way forward.

[leondz]

The format above is based on HFDetector being a class specifically designed to use a TextClassificationPipeline. Is the is the approach desired or is further abstraction of ModelAsJudge be a goal of this revision?

I anticipate ModelAsJudge primarily using external APIs, and so not using the HFDetector abstraction. It may later make sense to rename HFDetector to HFClassificationDetector to make its coupling to TextClassificationPipeline clear.

[leondz]

Thanks for this, working great.

Q - both misleading.MustContradictNLI and misleading.MustRefuteClaimModel, that are associated with the misleading probes, can use cuda. Is it possible to configure these (or even all HFDetectors) in one go?

This ofc works:

plugins:
  detectors:
    misleading:
      MustContradictNLI:
        hf_args:
          device: cuda
      MustRefuteClaimModel:
        hf_args:
          device: cuda

Configuring the base class as below didn't work, I guess predicated on how config matching is applied, probably class name matching is more conservative/tidy than isinstance() testing:

plugins:
  detectors:
    base:
      HFDetector:
        hf_args:
          device: cuda

Is there a way to configure groups of plugins?

[leondz]

Side issue - moving HFDetector to GPU exposes a further possible optimisation through using an HF dataset. This message comes up, presumably from transformers:

You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset

The optimisation is out of scope here; tracking in #812

[jmartin-tech]

Is there a way to configure groups of plugins?

Yes although it might be too broad in some detectors. We can configure at the module level. Currently there is no support for configuration of all implemented classes of a base plugin.

This would provide hf_args for all misleading detectors. Configurable intentionally

plugins:
  detectors:
    misleading:
      hf_args:
        device: cuda

While I like the idea of having less configuration required, I think in practical usage explicit configuration for a base class would be overkill and requires more internal knowledge of the implementation structure than would be typical for most users. I am skeptical of the trade off in value for powerusers vs complexity, maintenance and support for configuration by instance type.

[leondz]

Yeah, if doing this to base classes requires extra code I'm having a hard time justifying that too. Thanks for the yaml, this works fine.