Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes Map OOM bug #429

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions clar2wasm/src/wasm_generator.rs
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ pub struct WasmGenerator {
/// The locals for the current function.
pub(crate) bindings: HashMap<String, Vec<LocalId>>,
/// Size of the current function's stack frame.
frame_size: i32,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some methods return u32 like ListTypeData::get_max_len(). Although, if something can't be negative shouldn't it be unsigned? I can't see that frame_size is going to be -10 or something.

pub(crate) frame_size: u32,
}

#[derive(Hash, Eq, PartialEq)]
Expand Down Expand Up @@ -263,7 +263,7 @@ impl WasmGenerator {
.next()
.ok_or_else(|| GeneratorError::InternalError("No Memory found".to_owned()))?;

let total_memory_bytes = self.literal_memory_end + (self.frame_size as u32);
let total_memory_bytes = self.literal_memory_end + self.frame_size;
let pages_required = total_memory_bytes / (64 * 1024);
let remainder = total_memory_bytes % (64 * 1024);

Expand Down Expand Up @@ -775,7 +775,7 @@ impl WasmGenerator {
// [ new_stack_ptr ]
.global_set(self.stack_pointer);
// [ ]
self.frame_size += size;
self.frame_size += size as u32;

(offset, size)
}
Expand Down
45 changes: 38 additions & 7 deletions clar2wasm/src/words/sequences.rs
Original file line number Diff line number Diff line change
Expand Up @@ -552,12 +552,13 @@ impl ComplexWord for Map {
let mut input_offsets = vec![];
let mut input_element_types = vec![];
let mut input_element_sizes = vec![];
let mut smallest_sequence_length = u32::MAX;
let mut input_num_elements = vec![];

for arg in args.iter().skip(1) {
// get the type of the seq, and the sizes.

let (element_ty, element_size) = match generator
let (element_ty, element_size, element_length) = match generator
.get_expr_type(arg)
.ok_or_else(|| {
GeneratorError::TypeError("sequence expression must be typed".to_owned())
Expand All @@ -566,16 +567,21 @@ impl ComplexWord for Map {
{
TypeSignature::SequenceType(SequenceSubtype::ListType(lt)) => {
let element_ty = lt.get_list_item_type().clone();
let max_length = lt.get_max_len();
let element_size = get_type_size(&element_ty);
(SequenceElementType::Other(element_ty), element_size)
(
SequenceElementType::Other(element_ty),
element_size,
max_length,
)
}
TypeSignature::SequenceType(SequenceSubtype::BufferType(_))
TypeSignature::SequenceType(SequenceSubtype::BufferType(length))
| TypeSignature::SequenceType(SequenceSubtype::StringType(StringSubtype::ASCII(
_,
))) => (SequenceElementType::Byte, 1),
length,
))) => (SequenceElementType::Byte, 1, length.into()),
TypeSignature::SequenceType(SequenceSubtype::StringType(StringSubtype::UTF8(
_,
))) => (SequenceElementType::UnicodeScalar, 4),
length,
))) => (SequenceElementType::UnicodeScalar, 4, length.into()),
_ => {
return Err(GeneratorError::TypeError(
"expected sequence type".to_string(),
Expand All @@ -585,6 +591,10 @@ impl ComplexWord for Map {
input_element_types.push(element_ty);
input_element_sizes.push(element_size);

if element_length < smallest_sequence_length {
smallest_sequence_length = element_length;
}

generator.traverse_expr(builder, arg)?;
// [ offset, length ]
builder.i32_const(element_size);
Expand Down Expand Up @@ -728,6 +738,14 @@ impl ComplexWord for Map {
generator.visit_call_user_defined(&mut loop_, return_element_type, fname)?;
}

smallest_sequence_length = if smallest_sequence_length == u32::MAX {
0
} else {
smallest_sequence_length
};

generator.frame_size += return_element_size as u32 * smallest_sequence_length;
Comment on lines +741 to +747
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lines 634 -> 652 look like create_call_stack_local but with extra steps.
Not sure how I feel about this:

  • on the one hand, it seems this is "optimized" so we don't allocate more space than needed
  • on the other hand, any change/optimization to create_call_stack_local won't be reflected here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've a fix in mind, but another reason to avoid create_call_stack_local was we don't need to increase stack-pointer here.


// Write the result to the output sequence.
generator.write_to_memory(&mut loop_, output_offset, 0, return_element_type)?;

Expand Down Expand Up @@ -1532,6 +1550,7 @@ impl ComplexWord for Slice {

#[cfg(test)]
mod tests {
use clarity::vm::types::{ListData, ListTypeData, SequenceData, TypeSignature};
use clarity::vm::Value;

use crate::tools::{crosscheck, evaluate};
Expand Down Expand Up @@ -1980,6 +1999,18 @@ mod tests {
crosscheck("(map - (list 10 20 30))", evaluate("(list -10 -20 -30)"));
}

#[test]
fn multiple_maps() {
let snippet = "(map + (list 1 2 3) (list 1 2 3 4) (list 1 2 3 4 5))".repeat(700);

let expected = Value::Sequence(SequenceData::List(ListData {
data: vec![Value::Int(3), Value::Int(6), Value::Int(9)],
type_signature: ListTypeData::new_list(TypeSignature::IntType, 3).unwrap(),
}));

crosscheck(&snippet, Ok(Some(expected)));
}

#[test]
fn slice_right_lt_left() {
crosscheck("(slice? \"abc\" u1 u0)", evaluate("none"));
Expand Down
Loading