fix(virtual-fs): Fix horrible write performance for in-memory files #3905
Conversation
theduke
force-pushed
the
issue-3903-fix-slow-memfs-write
branch
from
310ae50 to
4def192
Compare
The write() implementation for in-memory files used by the TmpFs / mem_fs was terrible: it used to re-allocate and copy the whole data if the write position was not at the end of the file. This re-writes the implementation to grow the buffer in place if required. Also adds some tests. Closes #3903
theduke
force-pushed
the
issue-3903-fix-slow-memfs-write
branch
from
4def192 to
047823e
Compare
|
|
||
| self.buffer = new_buffer; | ||
| } | ||
| if buf.is_empty() { |
There was a problem hiding this comment.
shouldn't the file by truncated in that case, depending on the write mode?
There was a problem hiding this comment.
The buf is only the newly written data.
| self.buffer.append(&mut remainder)?; | ||
| } | ||
| if end_cursor > self.buffer.len() { | ||
| self.buffer.resize(end_cursor, 0)?; |
There was a problem hiding this comment.
doesn't the file (well the vec) be srinked if it's smaller?
There was a problem hiding this comment.
That's why it's inside an if check.
This only runs when the new data makes the file larger than it previously was.
|
|
||
| file.write(b" mkay?", &mut cursor).unwrap(); | ||
| assert_eq!(file.buffer.deref(), b"alter the universe! mkay?"); | ||
| } |
There was a problem hiding this comment.
maybe a test with a seek and truncate could be interesting?
There was a problem hiding this comment.
Definitely.
We should have a proper test suite for trait FileSystem implementations.
I only added a tests for write() because that's the only thing I changed.
|
Superseded by #4165 |
The write() implementation for in-memory files used by the TmpFs /
mem_fs was terrible: it used to re-allocate and copy the whole data if
the write position was not at the end of the file.
This re-writes the implementation to grow the buffer in place if
required.
Also adds some tests.
Closes #3903