Skip to content

Commit

Permalink
MatmachJS using Emscripten's malloc, all tests are passing :)
Browse files Browse the repository at this point in the history
  • Loading branch information
dherre3 committed Nov 23, 2018
1 parent 2c9dce3 commit 29d0353
Show file tree
Hide file tree
Showing 31 changed files with 23,932 additions and 23,147 deletions.
Binary file modified bin/get_mem.wasm
Binary file not shown.
390 changes: 327 additions & 63 deletions bin/lib.js

Large diffs are not rendered by default.

10 changes: 1 addition & 9 deletions run_wasm.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,5 @@ echo $filebase
echo -e -n '\033[0m'
if [ -f "bin/$filename.wasm" ]; then
rm bin/$filename.wasm
rm bin/$filename-standalone.wasm
fi
sed -e 's/\(^[ ]*(;dummy;)(.* $.*\)/\ \ /g' -e 's/\(^[ ]*;; (import\)/\ \(import/g' -e 's/\(^[ ]*;; (import\)/\ \(import/g' -e 's/\(^[ ]*(mem\)/\ \;; (mem/g' $file > $filepath-temp.wat
wat2wasm $filepath-temp.wat -o bin/$filename.wasm
wat2wasm $file -o bin/$filename-standalone.wasm
rm $filepath-temp.wat

if [ -f "bin/$filename.wasm" ]; then
wasm-interp bin/$filename-standalone.wasm --trace
fi
wat2wasm $file -o bin/$filename.wasm
298 changes: 298 additions & 0 deletions src/emscripten-malloc/emscripten_malloc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,298 @@
# Emscripten Malloc

- For calls where the size in memory is known, the stack memory is used,
otherwise if its not known ahead of time, the heap is used.
- The size of the Emscripten stack is of 80 bytes

## `_malloc` - Function 22
**Lines**: (Lines: 194 - 6234)

**External Dependencies**:

- Global 12 -> Stack Top
- Global 13 -> Stack Max
- Call 3 -> `abortStackOverflow` - Imported
- Call 71 -> `_sbrk` - Impletemented
- Call 28 ->`__errno_location` - Implemented

In terms of handling the stack, this function asks for 80 bytes from the stack.
Before this it checks the global 13 for stack max to see whether there is space.
If there is, it sets the STACKTOP to STACKTOP+80, and when the function returns it returns the stack to its original value. Moreover, it handles the stack address from then
on via a local, which it sets at the beginning from the global.
### `_sbrk` - Function 71
**Lines**: (Lines: 18198 - 18249)
Grows memory when necessary, checks if there is enough memory, and if the
parameter passed is appropriate if not, it throws an error. Returns the value of
the dynamic\_ptr in memory
**External Dependencies**:
- Global 9 - DYNAMICTOP\_PTR
- Call 2 - `abortOnCannotGrowMemory` - Imported
- Call 7 - `__setErrNo` - Imported (Sets error in external emscripten state,
CODE:12)
- Call 1 - `getTotalMemory` - Imported
- Call 0 - `enlargeMemory` - Imported
**Return**: Returns value where dynamic memory is stored.
## `__errono_location`
- Does not do anything, simply returns the address where the errors are saved. i.e. 132, it seems like all the errors are thrown by the outside environment.
```javascript
(func (;29;) (type 2) (result i32)
(local i32 i32)
get_global 12
set_local 1
i32.const 1648
return)
```
**dependencies**
- Global 12 -> Stack top

## Free
Frees chunks
- Global 12 -> Stack Top

# Entry.js
All starts with a constant GLOBAL\_BASE ptr at line 378.
The memory is organized as follows:
- Above 0 is static memory, starting with globals. (Example location of stack
top, stack base, and start of dynamic memory).
- Then the stack.
- Then 'dynamic' memory for sbrk
```javascript
GLOBALBASE = 1024;
```
## Static memory
Meant for globals, and positions of ptrs in the program
```javascript
// Line: 1730
STATICBASE = GLOBALBASE;
STATICTOP = STATICBASE + 5456;// Setting the top
STATICBUMP = 5456; //size of static memory
```
They weirdly then statically allocate for some other poiters. This functions are
never called for this program. We may get rid of them for our purposes.
```
var tempDoublePtr = STATICTOP; STATICTOP += 16;
```
DYNAMICTOP\_PTR = staticAlloc(4) // Sets the ptr where the location of dynamic
memory starts.

After it sets the location where the dynamic ptr constant will be stored, it
sets `STACK_BASE`, `STACKTOP`,`DYNAMIC_BASE`, finally it sets the dynamic ptr.


```javascript
TOTAL_STACK=5242880 // Could be set via MODULE['TOTAL_STACK'];
STACK_BASE = STACKTOP = alignMemory(STATICTOP);
STACK_MAX = STACK_BASE + TOTAL_STACK;
DYNAMIC_BASE = alignMemory(STACK_MAX);
HEAP32[DYNAMICTOP_PTR>>2] = DYNAMIC_BASE;
```

### `staticAlloc()`
- Uses STATICTOP allocate statically a given size, makes sure the result is
16 byte aligned.
```javascript
function staticAlloc(size) {

assert(!staticSealed);
var ret = STATICTOP;
STATICTOP = (STATICTOP + size + 15) & -16;
return ret;
}
```
### `alignMemory()`
- Aligns 16 byte align of memory, by taking ceiling, so in both cases above it
increases the pointer by 16
```javascript
const STACK_ALIGN = 16;
function alignMemory(size, factor) {
if (!factor) factor = STACK_ALIGN; // stack alignment (16-byte) by default
var ret = size = Math.ceil(size / factor) * factor;
return ret;
}
```
In terms of Emscripten, there are three functions to allocate memory, they all
come into play depending on the Runtime initialization. This all materializes
in a function called `getMemory()`
```javascript
function getMemory(size) {
if (!staticSealed) return staticAlloc(size);
if (!runtimeInitialized) return dynamicAlloc(size);
return _malloc(size);
}
```
If the pointers for the stack and dynamic ptr is not ready, then staticAlloc,
if the `runtimeInitialized` is false, which means the `_malloc` function from
emscripten is not ready, then use `dynamicAlloc(size)`
## `abortStackOverflow(allocSize)`
Function simple throws error when there is no more space in the stack. Thrown
by every function if its the case. Note that Emscripten is smart enough to
eliminate this call when it can statically determined this is not true.
```javascript
function abortStackOverflow(allocSize) {
abort('Stack overflow! Attempted to allocate ' + allocSize + ' bytes on the stack, but stack has only ' + (STACK_MAX - stackSave() + allocSize) + ' bytes available!');
}
```
## `__setErrNo(num)`
```javascript
function ___setErrNo(value) {
if (Module['___errno_location']) HEAP32[((Module['___errno_location']())>>2)]=value;
else Module.printErr('failed to set errno from JS');
return value;
}
```
This function simply sets the place for the error location inside the Emscripten
WebAssembly memory. i.e. Each C function has a set of string errors it throws
under particular circumstances, this functions just sets the location of those
functions in memory using the `__errno_location` base address.
## `getTotalMemory()`
- Simply returns the total\_memory for the program, the emscripten program gives
the user the option to either set this total memory, allow for the growth of
it, etc.
**dependencies**
- `TOTAL_MEMORY`

## `abortOnCannotGrowMemory`
**dependencies**
- abort()
- `TOTAL_MEMORY`
```javascript
function abortOnCannotGrowMemory() {
abort('Cannot enlarge memory arrays. Either (1) compile with -s TOTAL_MEMORY=X with X higher than the current value ' + TOTAL_MEMORY + ', (2) compile with -s ALLOW_MEMORY_GROWTH=1 which allows increasing the size at runtime, or (3) if you want malloc to return NULL (0) instead of this abort, compile with -s ABORTING_MALLOC=0 ');
}
```
# `abortOnStackOverflow()`
- Quits application when a stackoverflow happens
**dependencies**:
- `abort()`
```javascript
function abortStackOverflow(allocSize) {
abort('Stack overflow! Attempted to allocate ' + allocSize + ' bytes on the stack, but stack has only ' + (STACK_MAX - stackSave() + allocSize) + ' bytes available!');
}
```



# `enlargeMemory()`
- Grows the Emscripten memory, now this is the most complicated functions, this
functions has many dependencies and roles.
**depedencies**
- `HEAP32[DYNAMIC\_PTR>>2]`
- `TOTAL_MEMORY`
- `Module["usingWasm"]`
- `WASM_PAGE_SIZE` and `ASMJS_PAGE_SIZE`
- `Module.printErr()`
- `alignUp()`
- `Module.reallocBuffer(TOTAL_MEMORY)` Reallocates all memory, returns old if
not possible.
- `updateGlobalBuffer()`
- `updateGlobalBufferViews()`
- `Module.usingWasm`

1. Checks the `DYNAMIC_PTR`, it should have been increased by now, sets the limit
to one page short of 2gb
2. Checks to see if the `HEAP[DYNAMIC_PTR>>2] > LIMIT`, if it is, throws error.
3. Then proceeds to increase `TOTAL_MEMORY` towards 2GBs, if possible.
4. Calls `Module.reallocBuffer(TOTAL_MEMORY)` to reallocate the buffer, and copy
the data from the old to the new buffer.
5. If error, prints error using `Module.printErr`
6. Otherwise it calls `updateGlobalBuffer()` and `updateGlobalBufferViews()`
7. Prints new memory, warning if is asmjs and finally returns.

### `updateGlobalBuffer` & `updateGlobalBufferViews()`
```javascript
function updateGlobalBuffer(buf) {
Module['buffer'] = buffer = buf;
}

function updateGlobalBufferViews() {
Module['HEAP8'] = HEAP8 = new Int8Array(buffer);
Module['HEAP16'] = HEAP16 = new Int16Array(buffer);
Module['HEAP32'] = HEAP32 = new Int32Array(buffer);
Module['HEAPU8'] = HEAPU8 = new Uint8Array(buffer);
Module['HEAPU16'] = HEAPU16 = new Uint16Array(buffer);
Module['HEAPU32'] = HEAPU32 = new Uint32Array(buffer);
Module['HEAPF32'] = HEAPF32 = new Float32Array(buffer);
Module['HEAPF64'] = HEAPF64 = new Float64Array(buffer);
}

```

### `reallocBuffer`
```javascript
var wasmReallocBuffer = function(size) {
var PAGE_MULTIPLE = Module["usingWasm"] ? WASM_PAGE_SIZE : ASMJS_PAGE_SIZE; // In wasm, heap size must be a multiple of 64KB. In asm.js, they need to be multiples of 16MB.
size = alignUp(size, PAGE_MULTIPLE); // round up to wasm page size
var old = Module['buffer'];
var oldSize = old.byteLength;
if (Module["usingWasm"]) {
// native wasm support
try {
var result = Module['wasmMemory'].grow((size - oldSize) / wasmPageSize); // .grow() takes a delta compared to the previous size
if (result !== (-1 | 0)) {
// success in native wasm memory growth, get the buffer from the memory
return Module['buffer'] = Module['wasmMemory'].buffer;
} else {
return null;
}
} catch(e) {
console.error('Module.reallocBuffer: Attempted to grow from ' + oldSize + ' bytes to ' + size + ' bytes, but got error: ' + e);
return null;
}
}
};
```

# `MEMORY SET UP`

```javascript
var byteLength;
try {
byteLength = Function.prototype.call.bind(Object.getOwnPropertyDescriptor(ArrayBuffer.prototype, 'byteLength').get);
byteLength(new ArrayBuffer(4)); // can fail on older ie
} catch(e) { // can fail on older node/v8
byteLength = function(buffer) { return buffer.byteLength; };
}

var TOTAL_STACK = Module['TOTAL_STACK'] || 5242880;
var TOTAL_MEMORY = Module['TOTAL_MEMORY'] || 16777216;
if (TOTAL_MEMORY < TOTAL_STACK) Module.printErr('TOTAL_MEMORY should be larger than TOTAL_STACK, was ' + TOTAL_MEMORY + '! (TOTAL_STACK=' + TOTAL_STACK + ')');

// Initialize the runtime's memory
// check for full engine support (use string 'subarray' to avoid closure compiler confusion)
assert(typeof Int32Array !== 'undefined' && typeof Float64Array !== 'undefined' && Int32Array.prototype.subarray !== undefined && Int32Array.prototype.set !== undefined,
'JS engine does not provide full typed array support');



// Use a provided buffer, if there is one, or else allocate a new one
if (Module['buffer']) {
buffer = Module['buffer'];
assert(buffer.byteLength === TOTAL_MEMORY, 'provided buffer should be ' + TOTAL_MEMORY + ' bytes, but it is ' + buffer.byteLength);
} else {
// Use a WebAssembly memory where available
if (typeof WebAssembly === 'object' && typeof WebAssembly.Memory === 'function') {
assert(TOTAL_MEMORY % WASM_PAGE_SIZE === 0);
Module['wasmMemory'] = new WebAssembly.Memory({ 'initial': TOTAL_MEMORY / WASM_PAGE_SIZE });
buffer = Module['wasmMemory'].buffer;
} else
{
buffer = new ArrayBuffer(TOTAL_MEMORY);
}
assert(buffer.byteLength === TOTAL_MEMORY);
Module['buffer'] = buffer;
}
updateGlobalBufferViews();
```


## Error Messages in Static Memory

Another consideration when dealing with WebAssembly is the
error strings that are written down at start-up in static memory via the
`data` WebAssembly construct. When dealing with your own
library use the static offset to set the limit of your constant
messages. Also, to the top of your static offset, make sure to set the `__errno_location()`
function, (which could be set statically in JavaScript, if you only have one version of `wasm`
This way it does not have to be imported from the WebAssembly instantiated module.).


Binary file added src/emscripten-malloc/emscripten_malloc.pdf
Binary file not shown.
20 changes: 20 additions & 0 deletions src/emscripten-malloc/entry.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#include <stdio.h>
#include <stdlib.h>
int factorial(int n){
if(n == 0) return 1;
return n*factorial(n-1);
}

int main(int argc, char **argv){
//int *ptr = (int *)malloc(sizeof(int)*10)i;
int *ptr;
for(int i = 0;i< 10;i++){
ptr = (int *)malloc(sizeof(int)*1000000);
ptr[i] = i+10;
// printf("VALUE IN MEMORY: %d\n", ptr[i]);
}
int b = ptr[9];
int c = factorial(4);
free(ptr);
return b + c;
}
Loading

0 comments on commit 29d0353

Please sign in to comment.