Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix Incredibly Slow write Implementation of TmpFs In-Memory Files #3903

Closed
theduke opened this issue May 24, 2023 · 1 comment · Fixed by #4165
Closed

Fix Incredibly Slow write Implementation of TmpFs In-Memory Files #3903

theduke opened this issue May 24, 2023 · 1 comment · Fixed by #4165
Assignees
@theduke @Arshia001
Labels
bug Something isn't working priority-medium Medium priority issue
Milestone
backlog

Comments

@theduke
Copy link
Contributor

theduke commented May 24, 2023 •
edited by Michael-F-Bryan
Loading

Currently writing to somewhere else than the end of a file always allocates a new buffer in the mem fs.
That will have ... horrible performance.

Need to fix this.

This is bad enough to call it a bug.

wasmer/lib/virtual-fs/src/mem_fs/file.rs

Lines 1323 to 1327 in 458330e

self.buffer.reserve_exact(buf.len());
let mut remainder = self.buffer.split_off(position as usize);
self.buffer.extend_from_slice(buf);
self.buffer.append(&mut remainder);

@theduke theduke added the bug Something isn't working label May 24, 2023
@theduke theduke self-assigned this May 24, 2023
@theduke theduke mentioned this issue May 24, 2023
Closed
theduke added a commit that referenced this issue May 24, 2023
@theduke
fix(virtual-fs): Fix horrible write performance for in-memory files
310ae50 
The write() implementation for in-memory files used by the TmpFs /
mem_fs was terrible: it used to re-allocate and copy the whole data if
the write position was not at the end of the file.

This re-writes the implementation to grow the buffer in place if
required.

Also adds some tests.

Closes #3903
theduke added a commit that referenced this issue May 24, 2023
@theduke
fix(virtual-fs): Fix horrible write performance for in-memory files
4def192 
The write() implementation for in-memory files used by the TmpFs /
mem_fs was terrible: it used to re-allocate and copy the whole data if
the write position was not at the end of the file.

This re-writes the implementation to grow the buffer in place if
required.

Also adds some tests.

Closes #3903
theduke added a commit that referenced this issue May 24, 2023
@theduke
fix(virtual-fs): Fix horrible write performance for in-memory files
047823e 
The write() implementation for in-memory files used by the TmpFs /
mem_fs was terrible: it used to re-allocate and copy the whole data if
the write position was not at the end of the file.

This re-writes the implementation to grow the buffer in place if
required.

Also adds some tests.

Closes #3903
@ptitSeb ptitSeb added this to the v4.0 milestone May 30, 2023
@ptitSeb ptitSeb added the priority-medium Medium priority issue label May 30, 2023
@ptitSeb
Copy link
Contributor

ptitSeb commented May 30, 2023

Stretch goal

@ptitSeb ptitSeb modified the milestones: v4.0, v4.1 Jun 6, 2023
@Arshia001 Arshia001 mentioned this issue Aug 18, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@theduke theduke

@Arshia001 Arshia001

Labels
bug Something isn't working priority-medium Medium priority issue
Projects
None yet
Milestone
backlog
3 participants
@theduke @ptitSeb @Arshia001