kwargs hashing fails in benchmark.evaluate

[robsdavis]

Description

Benchmarks.Evaluate accepts a pathlib.Path object for workspace which breaks json.dumps() call in kwargs hashing due to TypeError: Object of type PosixPath is not JSON serializable

How to Reproduce

import json
from pathlib import Path

# benchmark.__init__: 52
workspace = Path('workspace')
# benchmark.__init__: 124
kwargs = {'workspace':workspace}
# benchmark.__init__: 114
kwargs_hash_raw = json.dumps(kwargs, sort_keys=True).encode()

Expected Behavior

Produces kwargs hash without error, even if pathlib.Path object passed to workspace.