feat: more flexible JSON parsing

[klkvr]

Motivation

closes #8326

Solution

Adds parseJsonType set of cheatcodes which accepts stringified Solidity type as guidance for the parser, allowing to express complex types without the need to resort to parsing of them in multiple parseJson* calls.

Example usage of such cheatcode:

// for simple types
uint256 value = abi.decode(vm.parseJsonType("100", "uint256"), (uint256)));
// for more complex stuff
address[][3] memory values = abi.decode(vm.parseJsonType('[[],["0xd8da6bf26964af9d7eed9e03e53415d37aa96045"], []]', "address[][3]"), (address[][3]));

struct Mail {
    address from;
    address to;
    string contents;
}

// struct types are encoded as EIP-712 `encodeType`
// fields order doesn't matter
Mail memory mail = abi.decode(
    vm.parseJsonType(
        '{"from": "0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045", "contents": "something", "to": "0xEe5F5c53CE2159fC6DD4b0571E86a4A390D04846"}',
        "Mail(address from, address to, string contents)"
    ),
    (Mail)
);

For simpler creation of type strings for structs added forge eip712 <path> command which creates EIP-712 encodeType string for all structs in the given file.

Example output:

forge eip712 lib/forge-std/src/Vm.sol   
[⠊] Compiling...
[⠢] Compiling 1 files with Solc 0.8.26
[⠆] Solc 0.8.26 finished in 156.26ms
No files changed, compilation skipped
Log(bytes32[] topics,bytes data,address emitter)

Rpc(string key,string url)

EthGetLogs(address emitter,bytes32[] topics,bytes data,bytes32 blockHash,uint64 blockNumber,bytes32 transactionHash,uint64 transactionIndex,uint256 logIndex,bool removed)

DirEntry(string errorMessage,string path,uint64 depth,bool isDir,bool isSymlink)

FsMetadata(bool isDir,bool isSymlink,uint256 length,bool readOnly,uint256 modified,uint256 accessed,uint256 created)

Wallet(address addr,uint256 publicKeyX,uint256 publicKeyY,uint256 privateKey)

FfiResult(int32 exitCode,bytes stdout,bytes stderr)

ChainInfo(uint256 forkId,uint256 chainId)

AccountAccess(ChainInfo chainInfo,uint8 kind,address account,address accessor,bool initialized,uint256 oldBalance,uint256 newBalance,bytes deployedCode,uint256 value,bytes data,bool reverted,StorageAccess[] storageAccesses,uint64 depth)ChainInfo(uint256 forkId,uint256 chainId)StorageAccess(address account,bytes32 slot,bool isWrite,bytes32 previousValue,bytes32 newValue,bool reverted)

StorageAccess(address account,bytes32 slot,bool isWrite,bytes32 previousValue,bytes32 newValue,bool reverted)

Gas(uint64 gasLimit,uint64 gasTotalUsed,uint64 gasMemoryUsed,int64 gasRefunded,uint64 gasRemaining)

This might also be useful for #4818 later

Will be working on serialization next

[klkvr]

Advantage of using EIP-712 is that we can reuse some of existing alloy structures for it and it's also pretty much human readable and compact.

However, it has some limitations

1. We can't represent enums with it. Currently enums are treated as uint8 but we could allow (de)serialization of enums by stringified member names.
2. It's pretty limited and can't be extended much. It wouldn't be too hard to support additional properties like this if we had more flexible format:

/// @param number forge-json: hex
/// @param inner forge-json: flatten
struct SomeStruct {
    uint256 number;
    AnotherStruct inner; 
}

However, all of this can be added in the future by just allowing more formats for typeDescription param and adding helpers for generating new description formats.

[klkvr]

Added cheatcode for serialization and forge bind-json command. It can be used to generate JsonBindings library containing helpers for serialization/deserialization of all structs in the project.

Example output for project with 1 struct. In case of multiple structs and duplicating names, those are getting resolved correctly

pragma solidity >=0.6.2 <0.9.0;
pragma experimental ABIEncoderV2;

import {Vm} from "forge-std/Vm.sol";
import {SomeStruct} from "src/SomeStruct.sol";

library JsonBindings {
    Vm constant vm = Vm(address(uint160(uint256(keccak256("hevm cheat code")))));

    string constant schema_SomeStruct = "SomeStruct(uint256 a,uint256 b)";

    function serialize(SomeStruct memory value) internal pure returns (string memory) {
        return vm.serializeJsonType(schema_SomeStruct, abi.encode(value));
    }

    function deserializeSomeStruct(string memory json) public pure returns (SomeStruct memory) {
        return abi.decode(vm.parseJsonType(json, schema_SomeStruct), (SomeStruct));
    }

    function deserializeSomeStruct(string memory json, string memory path) public pure returns (SomeStruct memory) {
        return abi.decode(vm.parseJsonType(json, path, schema_SomeStruct), (SomeStruct));
    }

    function deserializeSomeStructArray(string memory json, string memory path) public pure returns (SomeStruct[] memory) {
        return abi.decode(vm.parseJsonTypeArray(json, path, schema_SomeStruct), (SomeStruct[]));
    }
}

bindings are written to utils/JsonBindings.sol by default and can be later used like this:

import "../utils/JsonBindings.sol";
import "forge-std/Test.sol";

struct SomeStruct {
    uint256 a;
    uint256 b;
}

contract SomeStructJsonTest is Test {
    using JsonBindings for *;

    function test() public {
        SomeStruct memory s = SomeStruct(1, 2);
        string memory json = s.serialize();
        SomeStruct memory s2 = json.deserializeSomeStruct();
        assertEq(keccak256(abi.encode(s)), keccak256(abi.encode(s2)));
    }
}

The command itself is pretty straightforward - just walk the AST and write the methods, the only non-trivial part is preprocessing which we need to obtain valid AST from solc even when bindings are outdated and would cause compiler to fail normally. We rely on solang parser for this.

I think it might make sense to add more configuration for this, maybe even a config section. Currently for large codebases bindings file is ending up pretty large and there are also some issues with multi-version projects which I've addressed by only generating bindings for a single compiler input

[mattsse]

nice, this is pretty cool!

still need to take a closer look but I believe this should work, and the helper commands should make this convenient to use.

if possible, we def want some forgetest! for this as well

[mattsse]

nice work!

only have a few questions/suggestions

[klkvr]

Added config section:

[bind_json]
# path for generated bindings file
out = "utils/JsonBindings.sol"
# array of globs to include, if empty defaults to all project sources, excluding library files
include = []
# array of globs to exclude 
exclude = []

[DaniPopes]

lgtm pending @mattsse

[mattsse]

gg!