feat(evm): compile etherscan sources and use them on the debugger #1413

joshieDo · 2022-04-25T15:10:38Z

Motivation

WIP

Debugging with sources coming from etherscan is nice.

Solution

When identifying addresses for traces, collect addresses & sources.
Compile each contract address [and downloads the solc version required)
Provide them to the TUI debugger. Here I changed the TUI initialization. @brockelmore wdyt ?

Missing:

etherscan multi-file support
cache
cleanup

It's working now with some limitations:

flattened sources only
no vyper
no v0.4.*
--via-ir isn't provided by etherscan (i think...).

Working example:

cargo run --bin cast -- run 0xe6288cf5bb88281801302d558e434b8fced0dbaa8edc007ef2f5b0ffec694f88 --rpc-url https://api.avax.network/ext/bc/C/rpc --debug

If there are many contracts, be ready to wait a while...

onbjerg · 2022-04-26T13:45:09Z

I think there was talk of source maps being added to the Etherscan API (it's already present in the UI) - is this something we can get confirmed somewhere, and if so, are we willing to wait for that considering that this is sort of a workaround for that?

mattsse · 2022-04-26T13:49:33Z

it's not yet included in the getsourcecode api https://api.etherscan.io/api?module=contract&action=getsourcecode

onbjerg · 2022-04-26T13:52:24Z

it's not yet included in the getsourcecode api https://api.etherscan.io/api?module=contract&action=getsourcecode

Yeah, I meant that there was talks that it would be included soon-ish. I just don't know when, but if we can get a timeline and its not too far out we should maybe hold off and wait for that?

joshieDo · 2022-04-26T13:54:08Z

It will still take some time for etherscan to add it from my understanding. And when it does arrive, some of these changes will need to happen anyway.(eg. passing sourcemaps per external contracts to the debugger UI).

But even then, I feel like it could be added because:

it's not clear all contracts will have a sourcemap. For example. USDT does not have a sourcemap in the UI.
other chains' "etherscans" might not add this to their API or be slow to add it.

onbjerg · 2022-04-26T13:55:52Z

Alright, I was just saying that if it would be added soon there probably wouldn't be much of a point adding something we are going to delete, but if you feel strongly that it will take a long time/be incomplete then you're free to go ahead :)

onbjerg · 2022-04-26T13:57:04Z

cli/src/cmd/cast/run.rs

+    decoder: CallTraceDecoder,
+    known_contracts: BTreeMap<ArtifactId, ContractBytecodeSome>,


We should probably merge this at some earlier point instead of passing both of these to the function since they serve the same purpose

Wanted to clarify this real quick, did you mean merge this contract map into one object or merge it with the sources map?

onbjerg · 2022-04-26T13:57:16Z

cli/src/cmd/cast/run.rs

+    result: RunResult,
+    decoder: CallTraceDecoder,
+    known_contracts: BTreeMap<ArtifactId, ContractBytecodeSome>,
+    sources: BTreeMap<ArtifactId, String>,


onbjerg · 2022-04-26T13:59:55Z

evm/src/trace/identifier/etherscan.rs

+        &self,
+    ) -> eyre::Result<(BTreeMap<ArtifactId, String>, BTreeMap<ArtifactId, ContractBytecodeSome>)>
+    {
+        if self.compiles {


Any reason you opted for a bool here instead of just ommitting the call to get_compiled_contracts_with_sources when we don't want to compile?

older code, part of the planned clean-up

onbjerg · 2022-04-26T14:00:59Z

evm/src/trace/identifier/etherscan.rs

            for (addr, _) in addresses {
                fetcher.push(*addr);
            }


We can probably filter out addresses here if it is already present in self.contracts

onbjerg · 2022-04-26T14:02:10Z

utils/src/lib.rs

@@ -1011,6 +1018,67 @@ pub fn init_tracing_subscriber() {
        .ok();
 }

+pub async fn compile_contract_source(


Maybe this should be in the CLI crate instead since we already have a compile module there? 🤔

@onbjerg thinking about this a bit. Problem with having this in the cli crate is that we introduce a cyclical dependency between the evm crate and the cli crate (evm references itself)

One level higher, maybe the cli crate isn't the right place for the compile module at all, it could be in common instead?

wyt?

I think OK to pull it all in common

onbjerg · 2022-04-26T14:49:09Z

evm/src/trace/identifier/etherscan.rs

+                    self.contracts
+                        .insert(label.clone(), (source_code, optimization, runs, version));


I think we should do this keyed by address instead - if we have multiple addresses with the same label (which IIRC is just the contract name for this identifier) then we might overwrite info for two contracts with the same name, but with different implementations. E.g. if you interact with two tokens and both of them show up as "Token", then only one of them would be decoded... I think

gakonst · 2022-07-30T22:47:53Z

@joshieDo are we interested in picking this up again?

mds1 · 2022-08-03T05:20:00Z

IMO yes, this would be a really great feature. Having a module for etherscan compilation would also be useful for things like #2494 (comment)

joshieDo · 2022-08-10T16:51:20Z

i'd love to have this feature, but with all the refactors/conflicts (also the incoming TUI one), it's probably better to start from scratch. Maybe use this one as a potential reference.

gakonst · 2022-08-11T00:10:57Z

Agree that this should be started from scratch. Will see if I can find someone to take it over the finish line using this as a reference.

omkarb · 2022-08-23T19:45:48Z

Sorry for the delay on this team, I'll have a new draft PR building on the work that JoshieDo has done in the next couple days.

gakonst · 2022-08-23T23:11:02Z

Thanks boss

gakonst · 2022-08-30T17:44:01Z

superseded by #3006

joshieDo added 2 commits April 25, 2022 14:51

wip

c8d87ae

Merge remote-tracking branch 'ga/master' into etherscan_compilation

438b476

joshieDo changed the title ~~evm(debugger): compile etherscan sources and use them~~ feat(evm): compile etherscan sources and use themon the debugger Apr 25, 2022

joshieDo changed the title ~~feat(evm): compile etherscan sources and use themon the debugger~~ feat(evm): compile etherscan sources and use them on the debugger Apr 25, 2022

onbjerg reviewed Apr 26, 2022

View reviewed changes

onbjerg added the T-feature Type: feature label Apr 27, 2022

gakonst force-pushed the master branch 4 times, most recently from e4cdc0a to 8112635 Compare May 24, 2022 07:21

rkrasiuk added the A-debugger Area: debugger label Aug 23, 2022

gakonst assigned omkarb Aug 23, 2022

omkarb mentioned this pull request Aug 29, 2022

feat(evm): Compile Etherscan Sources for use in Debugger #3006

Merged

gakonst closed this Aug 30, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(evm): compile etherscan sources and use them on the debugger #1413

feat(evm): compile etherscan sources and use them on the debugger #1413

joshieDo commented Apr 25, 2022 •

edited

Loading

onbjerg commented Apr 26, 2022

mattsse commented Apr 26, 2022

onbjerg commented Apr 26, 2022

joshieDo commented Apr 26, 2022 •

edited

Loading

onbjerg commented Apr 26, 2022

onbjerg Apr 26, 2022

omkarb Aug 29, 2022 •

edited

Loading

onbjerg Apr 26, 2022

onbjerg Apr 26, 2022

joshieDo Apr 26, 2022 •

edited

Loading

onbjerg Apr 26, 2022

onbjerg Apr 26, 2022

omkarb Aug 26, 2022

gakonst Aug 26, 2022 •

edited

Loading

onbjerg Apr 26, 2022

gakonst commented Jul 30, 2022

mds1 commented Aug 3, 2022

joshieDo commented Aug 10, 2022 •

edited

Loading

gakonst commented Aug 11, 2022

omkarb commented Aug 23, 2022

gakonst commented Aug 23, 2022

gakonst commented Aug 30, 2022

		decoder: CallTraceDecoder,
		known_contracts: BTreeMap<ArtifactId, ContractBytecodeSome>,

		self.contracts
		.insert(label.clone(), (source_code, optimization, runs, version));

feat(evm): compile etherscan sources and use them on the debugger #1413

feat(evm): compile etherscan sources and use them on the debugger #1413

Conversation

joshieDo commented Apr 25, 2022 • edited Loading

Motivation

Solution

onbjerg commented Apr 26, 2022

mattsse commented Apr 26, 2022

onbjerg commented Apr 26, 2022

joshieDo commented Apr 26, 2022 • edited Loading

onbjerg commented Apr 26, 2022

onbjerg Apr 26, 2022

Choose a reason for hiding this comment

omkarb Aug 29, 2022 • edited Loading

Choose a reason for hiding this comment

onbjerg Apr 26, 2022

Choose a reason for hiding this comment

onbjerg Apr 26, 2022

Choose a reason for hiding this comment

joshieDo Apr 26, 2022 • edited Loading

Choose a reason for hiding this comment

onbjerg Apr 26, 2022

Choose a reason for hiding this comment

onbjerg Apr 26, 2022

Choose a reason for hiding this comment

omkarb Aug 26, 2022

Choose a reason for hiding this comment

gakonst Aug 26, 2022 • edited Loading

Choose a reason for hiding this comment

onbjerg Apr 26, 2022

Choose a reason for hiding this comment

gakonst commented Jul 30, 2022

mds1 commented Aug 3, 2022

joshieDo commented Aug 10, 2022 • edited Loading

gakonst commented Aug 11, 2022

omkarb commented Aug 23, 2022

gakonst commented Aug 23, 2022

gakonst commented Aug 30, 2022

joshieDo commented Apr 25, 2022 •

edited

Loading

joshieDo commented Apr 26, 2022 •

edited

Loading

omkarb Aug 29, 2022 •

edited

Loading

joshieDo Apr 26, 2022 •

edited

Loading

gakonst Aug 26, 2022 •

edited

Loading

joshieDo commented Aug 10, 2022 •

edited

Loading