This repository was archived by the owner on Nov 6, 2020. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Resumable EVM and heap-allocated callstack #9360
Merged
Merged
Changes from all commits
Commits
Show all changes
49 commits
Select commit
Hold shift + click to select a range
31c1130
Add new Vm trappable interface
sorpaas 7e8c0b5
Exec/Resume interface
sorpaas dda51e8
Basic implementation of CallCreateExecutive
sorpaas f98195e
Implement resume_call and resume_create for executive
sorpaas 10b7f30
Move convertion to call/create result to separate function
sorpaas 458a31c
Implement consume that converts resumable to non-resumable
sorpaas 851f5b4
Use consume for Executive::call/create
sorpaas 4625943
Resumable EVM
sorpaas 354aaa7
Implement tracing mode without needing subtracers
sorpaas e27ff3e
Implement vmtracer so it doesn't require extra structs for subtracing
sorpaas 9be2d3d
Use the new tracing mode in executive
sorpaas e8683d3
Fix most of the linting errors for cargo build
sorpaas c8c8377
Add the concept of stack_depth
sorpaas cdff7c9
Add back crossbeam
sorpaas 2042397
Fix some test compile
sorpaas 4fe09ad
Fix prefix address test
sorpaas 635c11f
Fix evm crate tests
sorpaas 8d391bb
Fix wasm crate test compile
sorpaas 9b0dc13
Fix wasm runner compile
sorpaas 98b92c1
Fix jsontests compile
sorpaas fcbeff1
Fix evmbin compile
sorpaas f6e4d77
Fix an issue with create nonce and better vm tracing interface
sorpaas 5088854
Fix linting
sorpaas 4b7c6de
Fix evmbin compile
sorpaas 0bceec2
Fix unconfirmed_substate and static_flag
sorpaas eab180f
Fix an issue in create address logic
sorpaas 8aef879
Fix top-level tracing
sorpaas d766e37
Handle builtin tracing
sorpaas b159734
Fix suicide and reward tracing index stack
sorpaas 9581ad4
Fix an issue where trap conflicts with tracing
sorpaas 41ad7ee
Fix an issue in parent step vm tracing
sorpaas d033fe8
Fix revert tracing
sorpaas 434f1d3
Fix evmbin tests
sorpaas 37aa837
Remove params clone
sorpaas bb8fae5
Fix TODO proofs
sorpaas 0ebefdb
Fix jsontests compile
sorpaas 03e6119
Merge branch 'master' into sp-resume-executive2
sorpaas 9f20224
Fix evmbin merge issue
sorpaas 42cdd7c
Fix wasm merge issue
sorpaas 28676e2
Fix wasm test
sorpaas 2c5f285
Merge branch 'master' of https://github.com/paritytech/parity into sp…
sorpaas ec30751
Fix ethcore merge warnings
sorpaas 4a6b559
Fix evmbin compile
sorpaas cf3f12f
Merge branch 'master' of https://github.com/paritytech/parity into sp…
sorpaas f485ce2
Merge branch 'master' into sp-resume-executive2
sorpaas 9544f1d
Merge branch 'master' into sp-resume-executive2
debris 60cf639
Merge branch 'master' of https://github.com/paritytech/parity into sp…
sorpaas e9916f6
Merge branch 'master' of https://github.com/paritytech/parity into sp…
sorpaas 9c603c6
Better expect messages and add some trace::skip_one asserts
sorpaas File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -32,7 +32,8 @@ use ethereum_types::{U256, U512, H256, Address}; | |
|
|
||
| use vm::{ | ||
| self, ActionParams, ParamsType, ActionValue, CallType, MessageCallResult, | ||
| ContractCreateResult, CreateContractAddress, ReturnData, GasLeft, Schedule | ||
| ContractCreateResult, CreateContractAddress, ReturnData, GasLeft, Schedule, | ||
| TrapKind, TrapError | ||
| }; | ||
|
|
||
| use evm::CostType; | ||
|
|
@@ -103,6 +104,7 @@ enum InstructionResult<Gas> { | |
| apply: bool, | ||
| }, | ||
| StopExecution, | ||
| Trap(TrapKind), | ||
| } | ||
|
|
||
| enum Never {} | ||
|
|
@@ -161,6 +163,7 @@ pub enum InterpreterResult { | |
| Done(vm::Result<GasLeft>), | ||
| /// The VM can continue to run. | ||
| Continue, | ||
| Trap(TrapKind), | ||
| } | ||
|
|
||
| impl From<vm::Error> for InterpreterResult { | ||
|
|
@@ -182,22 +185,89 @@ pub struct Interpreter<Cost: CostType> { | |
| valid_jump_destinations: Option<Arc<BitSet>>, | ||
| gasometer: Option<Gasometer<Cost>>, | ||
| stack: VecStack<U256>, | ||
| resume_output_range: Option<(U256, U256)>, | ||
| resume_result: Option<InstructionResult<Cost>>, | ||
| last_stack_ret_len: usize, | ||
| _type: PhantomData<Cost>, | ||
| } | ||
|
|
||
| impl<Cost: CostType> vm::Vm for Interpreter<Cost> { | ||
| fn exec(&mut self, ext: &mut vm::Ext) -> vm::Result<GasLeft> { | ||
| impl<Cost: 'static + CostType> vm::Exec for Interpreter<Cost> { | ||
| fn exec(mut self: Box<Self>, ext: &mut vm::Ext) -> vm::ExecTrapResult<GasLeft> { | ||
| loop { | ||
| let result = self.step(ext); | ||
| match result { | ||
| InterpreterResult::Continue => {}, | ||
| InterpreterResult::Done(value) => return value, | ||
| InterpreterResult::Done(value) => return Ok(value), | ||
| InterpreterResult::Trap(trap) => match trap { | ||
| TrapKind::Call(params) => { | ||
| return Err(TrapError::Call(params, self)); | ||
| }, | ||
| TrapKind::Create(params, address) => { | ||
| return Err(TrapError::Create(params, address, self)); | ||
| }, | ||
| }, | ||
| InterpreterResult::Stopped => panic!("Attempted to execute an already stopped VM.") | ||
| } | ||
| } | ||
| } | ||
| } | ||
|
|
||
| impl<Cost: 'static + CostType> vm::ResumeCall for Interpreter<Cost> { | ||
| fn resume_call(mut self: Box<Self>, result: MessageCallResult) -> Box<vm::Exec> { | ||
| { | ||
| let this = &mut *self; | ||
| let (out_off, out_size) = this.resume_output_range.take().expect("Box<ResumeCall> is obtained from a call opcode; resume_output_range is always set after those opcodes are executed; qed"); | ||
|
|
||
| match result { | ||
| MessageCallResult::Success(gas_left, data) => { | ||
| let output = this.mem.writeable_slice(out_off, out_size); | ||
| let len = cmp::min(output.len(), data.len()); | ||
| (&mut output[..len]).copy_from_slice(&data[..len]); | ||
|
|
||
| this.return_data = data; | ||
| this.stack.push(U256::one()); | ||
| this.resume_result = Some(InstructionResult::UnusedGas(Cost::from_u256(gas_left).expect("Gas left cannot be greater than current one"))); | ||
| }, | ||
| MessageCallResult::Reverted(gas_left, data) => { | ||
| let output = this.mem.writeable_slice(out_off, out_size); | ||
| let len = cmp::min(output.len(), data.len()); | ||
| (&mut output[..len]).copy_from_slice(&data[..len]); | ||
|
|
||
| this.return_data = data; | ||
| this.stack.push(U256::zero()); | ||
| this.resume_result = Some(InstructionResult::UnusedGas(Cost::from_u256(gas_left).expect("Gas left cannot be greater than current one"))); | ||
| }, | ||
| MessageCallResult::Failed => { | ||
| this.stack.push(U256::zero()); | ||
| this.resume_result = Some(InstructionResult::Ok); | ||
| }, | ||
| } | ||
| } | ||
| self | ||
| } | ||
| } | ||
|
|
||
| impl<Cost: 'static + CostType> vm::ResumeCreate for Interpreter<Cost> { | ||
| fn resume_create(mut self: Box<Self>, result: ContractCreateResult) -> Box<vm::Exec> { | ||
| match result { | ||
| ContractCreateResult::Created(address, gas_left) => { | ||
| self.stack.push(address_to_u256(address)); | ||
| self.resume_result = Some(InstructionResult::UnusedGas(Cost::from_u256(gas_left).expect("Gas left cannot be greater."))); | ||
| }, | ||
| ContractCreateResult::Reverted(gas_left, return_data) => { | ||
| self.stack.push(U256::zero()); | ||
| self.return_data = return_data; | ||
| self.resume_result = Some(InstructionResult::UnusedGas(Cost::from_u256(gas_left).expect("Gas left cannot be greater."))); | ||
| }, | ||
| ContractCreateResult::Failed => { | ||
| self.stack.push(U256::zero()); | ||
| self.resume_result = Some(InstructionResult::Ok); | ||
| }, | ||
| } | ||
| self | ||
| } | ||
| } | ||
|
|
||
| impl<Cost: CostType> Interpreter<Cost> { | ||
| /// Create a new `Interpreter` instance with shared cache. | ||
| pub fn new(mut params: ActionParams, cache: Arc<SharedCache>, schedule: &Schedule, depth: usize) -> Interpreter<Cost> { | ||
|
|
@@ -215,6 +285,9 @@ impl<Cost: CostType> Interpreter<Cost> { | |
| do_trace: true, | ||
| mem: Vec::new(), | ||
| return_data: ReturnData::empty(), | ||
| last_stack_ret_len: 0, | ||
| resume_output_range: None, | ||
| resume_result: None, | ||
| _type: PhantomData, | ||
| } | ||
| } | ||
|
|
@@ -244,50 +317,57 @@ impl<Cost: CostType> Interpreter<Cost> { | |
| /// Inner helper function for step. | ||
| #[inline(always)] | ||
| fn step_inner(&mut self, ext: &mut vm::Ext) -> Result<Never, InterpreterResult> { | ||
| let opcode = self.reader.code[self.reader.position]; | ||
| let instruction = Instruction::from_u8(opcode); | ||
| self.reader.position += 1; | ||
|
|
||
| // TODO: make compile-time removable if too much of a performance hit. | ||
| self.do_trace = self.do_trace && ext.trace_next_instruction( | ||
| self.reader.position - 1, opcode, self.gasometer.as_mut().expect(GASOMETER_PROOF).current_gas.as_u256(), | ||
| ); | ||
|
|
||
| let instruction = match instruction { | ||
| Some(i) => i, | ||
| None => return Err(InterpreterResult::Done(Err(vm::Error::BadInstruction { | ||
| instruction: opcode | ||
| }))), | ||
| }; | ||
| let result = match self.resume_result.take() { | ||
| Some(result) => result, | ||
| None => { | ||
| let opcode = self.reader.code[self.reader.position]; | ||
| let instruction = Instruction::from_u8(opcode); | ||
| self.reader.position += 1; | ||
|
|
||
| // TODO: make compile-time removable if too much of a performance hit. | ||
| self.do_trace = self.do_trace && ext.trace_next_instruction( | ||
| self.reader.position - 1, opcode, self.gasometer.as_mut().expect(GASOMETER_PROOF).current_gas.as_u256(), | ||
| ); | ||
|
|
||
| let info = instruction.info(); | ||
| self.verify_instruction(ext, instruction, info)?; | ||
| let instruction = match instruction { | ||
| Some(i) => i, | ||
| None => return Err(InterpreterResult::Done(Err(vm::Error::BadInstruction { | ||
| instruction: opcode | ||
| }))), | ||
| }; | ||
|
|
||
| // Calculate gas cost | ||
| let requirements = self.gasometer.as_mut().expect(GASOMETER_PROOF).requirements(ext, instruction, info, &self.stack, self.mem.size())?; | ||
| if self.do_trace { | ||
| ext.trace_prepare_execute(self.reader.position - 1, opcode, requirements.gas_cost.as_u256()); | ||
| } | ||
| let info = instruction.info(); | ||
| self.last_stack_ret_len = info.ret; | ||
| self.verify_instruction(ext, instruction, info)?; | ||
|
|
||
| // Calculate gas cost | ||
| let requirements = self.gasometer.as_mut().expect(GASOMETER_PROOF).requirements(ext, instruction, info, &self.stack, self.mem.size())?; | ||
| if self.do_trace { | ||
| ext.trace_prepare_execute(self.reader.position - 1, opcode, requirements.gas_cost.as_u256(), Self::mem_written(instruction, &self.stack), Self::store_written(instruction, &self.stack)); | ||
| } | ||
|
|
||
| self.gasometer.as_mut().expect(GASOMETER_PROOF).verify_gas(&requirements.gas_cost)?; | ||
| self.mem.expand(requirements.memory_required_size); | ||
| self.gasometer.as_mut().expect(GASOMETER_PROOF).current_mem_gas = requirements.memory_total_gas; | ||
| self.gasometer.as_mut().expect(GASOMETER_PROOF).current_gas = self.gasometer.as_mut().expect(GASOMETER_PROOF).current_gas - requirements.gas_cost; | ||
| self.gasometer.as_mut().expect(GASOMETER_PROOF).verify_gas(&requirements.gas_cost)?; | ||
| self.mem.expand(requirements.memory_required_size); | ||
| self.gasometer.as_mut().expect(GASOMETER_PROOF).current_mem_gas = requirements.memory_total_gas; | ||
| self.gasometer.as_mut().expect(GASOMETER_PROOF).current_gas = self.gasometer.as_mut().expect(GASOMETER_PROOF).current_gas - requirements.gas_cost; | ||
|
|
||
| evm_debug!({ self.informant.before_instruction(self.reader.position, instruction, info, &self.gasometer.as_mut().expect(GASOMETER_PROOF).current_gas, &self.stack) }); | ||
| evm_debug!({ self.informant.before_instruction(self.reader.position, instruction, info, &self.gasometer.as_mut().expect(GASOMETER_PROOF).current_gas, &self.stack) }); | ||
|
|
||
| let (mem_written, store_written) = match self.do_trace { | ||
| true => (Self::mem_written(instruction, &self.stack), Self::store_written(instruction, &self.stack)), | ||
| false => (None, None), | ||
| }; | ||
| // Execute instruction | ||
| let current_gas = self.gasometer.as_mut().expect(GASOMETER_PROOF).current_gas; | ||
| let result = self.exec_instruction( | ||
| current_gas, ext, instruction, requirements.provide_gas | ||
| )?; | ||
|
|
||
| // Execute instruction | ||
| let current_gas = self.gasometer.as_mut().expect(GASOMETER_PROOF).current_gas; | ||
| let result = self.exec_instruction( | ||
| current_gas, ext, instruction, requirements.provide_gas | ||
| )?; | ||
| evm_debug!({ self.informant.after_instruction(instruction) }); | ||
|
|
||
| result | ||
| }, | ||
| }; | ||
|
|
||
| evm_debug!({ self.informant.after_instruction(instruction) }); | ||
| if let InstructionResult::Trap(trap) = result { | ||
| return Err(InterpreterResult::Trap(trap)); | ||
| } | ||
|
|
||
| if let InstructionResult::UnusedGas(ref gas) = result { | ||
| self.gasometer.as_mut().expect(GASOMETER_PROOF).current_gas = self.gasometer.as_mut().expect(GASOMETER_PROOF).current_gas + *gas; | ||
|
|
@@ -296,9 +376,8 @@ impl<Cost: CostType> Interpreter<Cost> { | |
| if self.do_trace { | ||
| ext.trace_executed( | ||
| self.gasometer.as_mut().expect(GASOMETER_PROOF).current_gas.as_u256(), | ||
| self.stack.peek_top(info.ret), | ||
| mem_written.map(|(o, s)| (o, &(self.mem[o..o+s]))), | ||
| store_written, | ||
| self.stack.peek_top(self.last_stack_ret_len), | ||
| &self.mem, | ||
| ); | ||
| } | ||
|
|
||
|
|
@@ -451,21 +530,24 @@ impl<Cost: CostType> Interpreter<Cost> { | |
|
|
||
| let contract_code = self.mem.read_slice(init_off, init_size); | ||
|
|
||
| let create_result = ext.create(&create_gas.as_u256(), &endowment, contract_code, address_scheme); | ||
| let create_result = ext.create(&create_gas.as_u256(), &endowment, contract_code, address_scheme, true); | ||
| return match create_result { | ||
| ContractCreateResult::Created(address, gas_left) => { | ||
| Ok(ContractCreateResult::Created(address, gas_left)) => { | ||
| self.stack.push(address_to_u256(address)); | ||
| Ok(InstructionResult::UnusedGas(Cost::from_u256(gas_left).expect("Gas left cannot be greater."))) | ||
| }, | ||
| ContractCreateResult::Reverted(gas_left, return_data) => { | ||
| Ok(ContractCreateResult::Reverted(gas_left, return_data)) => { | ||
| self.stack.push(U256::zero()); | ||
| self.return_data = return_data; | ||
| Ok(InstructionResult::UnusedGas(Cost::from_u256(gas_left).expect("Gas left cannot be greater."))) | ||
| }, | ||
| ContractCreateResult::Failed => { | ||
| Ok(ContractCreateResult::Failed) => { | ||
| self.stack.push(U256::zero()); | ||
| Ok(InstructionResult::Ok) | ||
| }, | ||
| Err(trap) => { | ||
| Ok(InstructionResult::Trap(trap)) | ||
| }, | ||
| }; | ||
| }, | ||
| instructions::CALL | instructions::CALLCODE | instructions::DELEGATECALL | instructions::STATICCALL => { | ||
|
|
@@ -524,32 +606,37 @@ impl<Cost: CostType> Interpreter<Cost> { | |
|
|
||
| let call_result = { | ||
| let input = self.mem.read_slice(in_off, in_size); | ||
| ext.call(&call_gas.as_u256(), sender_address, receive_address, value, input, &code_address, call_type) | ||
| ext.call(&call_gas.as_u256(), sender_address, receive_address, value, input, &code_address, call_type, true) | ||
| }; | ||
|
|
||
| let output = self.mem.writeable_slice(out_off, out_size); | ||
| self.resume_output_range = Some((out_off, out_size)); | ||
|
|
||
| return match call_result { | ||
| MessageCallResult::Success(gas_left, data) => { | ||
| Ok(MessageCallResult::Success(gas_left, data)) => { | ||
| let output = self.mem.writeable_slice(out_off, out_size); | ||
| let len = cmp::min(output.len(), data.len()); | ||
| (&mut output[..len]).copy_from_slice(&data[..len]); | ||
|
|
||
| self.stack.push(U256::one()); | ||
| self.return_data = data; | ||
| Ok(InstructionResult::UnusedGas(Cost::from_u256(gas_left).expect("Gas left cannot be greater than current one"))) | ||
| }, | ||
| MessageCallResult::Reverted(gas_left, data) => { | ||
| Ok(MessageCallResult::Reverted(gas_left, data)) => { | ||
| let output = self.mem.writeable_slice(out_off, out_size); | ||
| let len = cmp::min(output.len(), data.len()); | ||
| (&mut output[..len]).copy_from_slice(&data[..len]); | ||
|
|
||
| self.stack.push(U256::zero()); | ||
| self.return_data = data; | ||
| Ok(InstructionResult::UnusedGas(Cost::from_u256(gas_left).expect("Gas left cannot be greater than current one"))) | ||
| }, | ||
| MessageCallResult::Failed => { | ||
| Ok(MessageCallResult::Failed) => { | ||
| self.stack.push(U256::zero()); | ||
| Ok(InstructionResult::Ok) | ||
| }, | ||
| Err(trap) => { | ||
| Ok(InstructionResult::Trap(trap)) | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why is
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Looks like the same case as #9360 (comment) |
||
| }, | ||
| }; | ||
| }, | ||
| instructions::RETURN => { | ||
|
|
@@ -1095,10 +1182,10 @@ mod tests { | |
| use rustc_hex::FromHex; | ||
| use vmtype::VMType; | ||
| use factory::Factory; | ||
| use vm::{self, Vm, ActionParams, ActionValue}; | ||
| use vm::{self, Exec, ActionParams, ActionValue}; | ||
| use vm::tests::{FakeExt, test_finalize}; | ||
|
|
||
| fn interpreter(params: ActionParams, ext: &vm::Ext) -> Box<Vm> { | ||
| fn interpreter(params: ActionParams, ext: &vm::Ext) -> Box<Exec> { | ||
| Factory::new(VMType::Interpreter, 1).create(params, ext.schedule(), ext.depth()) | ||
| } | ||
|
|
||
|
|
@@ -1118,7 +1205,7 @@ mod tests { | |
|
|
||
| let gas_left = { | ||
| let mut vm = interpreter(params, &ext); | ||
| test_finalize(vm.exec(&mut ext)).unwrap() | ||
| test_finalize(vm.exec(&mut ext).ok().unwrap()).unwrap() | ||
| }; | ||
|
|
||
| assert_eq!(ext.calls.len(), 1); | ||
|
|
@@ -1140,7 +1227,7 @@ mod tests { | |
|
|
||
| let err = { | ||
| let mut vm = interpreter(params, &ext); | ||
| test_finalize(vm.exec(&mut ext)).err().unwrap() | ||
| test_finalize(vm.exec(&mut ext).ok().unwrap()).err().unwrap() | ||
| }; | ||
|
|
||
| assert_eq!(err, ::vm::Error::OutOfBounds); | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why is
Errturned intoOkhere?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If
ext.createreturnsErr, it can only be atrap. (That also means that iftrapparameter is passedfalse,Errcase is impossible.) So in here if we want to trap the interpreter, we can directly match this.We really have three cases for the
exec_instructionreturn type:vm::Result::Errwhich indicates some errors happened.vm::Result::Ok(Instruction::Result::Trap(..))which indicates that a trap in the EVM happened.vm::Result::Ok(..)which indicates some other instruction results.Putting
Trapto error and wrapvm::Result::Errcan also work, but it means that we need to replicate a lot moreFrom/Intoimpls to support?syntax in this function, and I think it may be a little bit overkill.