added support for hugging face datasets

[Tyrest]

We can now load datasets from Huggingface as show below

dataset = load_dataset("lex_glue", "ledgar")

test_dataset = dataset["test"]
test_dataset = map_label_to_string(test_dataset, "label")

test_dataset = test_dataset.rename_column("text", "example")

ledgar_path = Path("../autolabel/examples/ledgar")
with open(ledgar_path / "config_ledgar.json", "r") as f:
    config = json.load(f)
if "few_shot_examples" in config["prompt"] and isinstance(config["prompt"]["few_shot_examples"], str):
    config["prompt"]["few_shot_examples"] = str(ledgar_path / config["prompt"]["few_shot_examples"])
agent = LabelingAgent(config)

agent.plan(test_dataset, max_items=8)
agent.run(test_dataset, max_items=8)

[nihit]

do we need to do any sanity checks on the dataset object before attempting to convert to dataframe?

[Tyrest]

This seems to be a more robust solution so I'll implement it here as well

[nihit]

can we access the dataframe with a helper method in dataset loader (vs directly accessing it here) ? for readability

[Tyrest]

Can you give an example of what you want it to look like instead?

[nihit]

i was thinking just having a def as_datafame() method in the data loader that returns the internal pandas dataframe object, instead of the labeling agent accessing it directly

[nihit]

let's change this to contains a task name prefix_labeled.csv so that the output files from different tasks don't overwrite each other