-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WasmFS JS API: Implement truncate #19543
Changes from 7 commits
e0d56a1
f305495
58bd56a
cfc0213
b354716
408fcc3
5eaaeba
08773f5
044e8e4
a0a3aa6
ad54b31
23c4326
e857699
c1b3f46
d3a8326
279f9a5
c924202
750a978
a0e4ca5
b1703ff
0cabd03
b262202
058dfa6
f057bc4
1d18411
bce2613
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -137,6 +137,22 @@ int _wasmfs_unlink(char* path) { | |
return __syscall_unlinkat(AT_FDCWD, (intptr_t)path, 0); | ||
} | ||
|
||
int _wasmfs_truncate(char *path, off_t length) { | ||
int err = __syscall_truncate64((intptr_t)path, length); | ||
if (err == -1) { | ||
return errno; | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Syscalls don't set errno, so this doesn't look right. Syscalls return negative error codes and libc handles those by setting errno appropriately. I would expect this code to be checking whether There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it possible that There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ah, no, all those places should be fixed. For example, in There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I see what went wrong now. The whole time I thought the syscalls were returning |
||
return err; | ||
} | ||
|
||
int _wasmfs_ftruncate(int fd, off_t size) { | ||
int err = __syscall_ftruncate64(fd, size); | ||
if (err == -1) { | ||
return errno; | ||
} | ||
return err; | ||
} | ||
|
||
int _wasmfs_chdir(char* path) { return __syscall_chdir((intptr_t)path); } | ||
|
||
int _wasmfs_symlink(char* old_path, char* new_path) { | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,7 @@ | ||
#include <emscripten/emscripten.h> | ||
#include <stdio.h> | ||
#include <sys/stat.h> | ||
#include <assert.h> | ||
|
||
int main() { | ||
/********** test FS.open() **********/ | ||
|
@@ -28,6 +30,84 @@ int main() { | |
assert(createFileNotHere && createFileNotHere.fd >= 0); | ||
); | ||
|
||
/********** test FS.truncate() **********/ | ||
EM_ASM( | ||
FS.writeFile('truncatetest', 'a=1\nb=2\n'); | ||
); | ||
|
||
struct stat s; | ||
stat("truncatetest", &s); | ||
assert(s.st_size == 8); | ||
|
||
EM_ASM( | ||
FS.truncate('truncatetest', 2); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As with all these changes it would be good to look for other tests that already use this API, if any? If we were previously disabling, or not running, those tests under WASMFS it would be good to enable them now. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I didn't see anything explicitly calling There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No, it would only be in the test dir. I guess we had not tests for that API. Thanks for looking into it. Hopefully after we are done with this we will have much better test coverage for the JS API! |
||
); | ||
stat("truncatetest", &s); | ||
assert(s.st_size == 2); | ||
|
||
EM_ASM( | ||
FS.truncate('truncatetest', 10); | ||
); | ||
stat("truncatetest", &s); | ||
assert(s.st_size == 10); | ||
|
||
EM_ASM( | ||
var truncateStream = FS.open('truncatetest', 'w'); | ||
#if WASMFS | ||
FS.ftruncate(truncateStream, 4); | ||
#else | ||
FS.ftruncate(truncateStream.fd, 4); | ||
#endif | ||
); | ||
stat("truncatetest", &s); | ||
assert(s.st_size == 4); | ||
|
||
EM_ASM( | ||
var ex; | ||
try { | ||
FS.truncate('truncatetest', -10); | ||
} catch(err) { | ||
ex = err; | ||
} | ||
|
||
assert(ex.name === "ErrnoError" && ex.errno === 28 /* EINVAL */); | ||
); | ||
|
||
EM_ASM( | ||
var ex; | ||
try { | ||
var truncateStream = FS.open('truncatetest', 'w'); | ||
#if WASMFS | ||
FS.ftruncate(truncateStream, -10); | ||
#else | ||
FS.ftruncate(truncateStream.fd, -10); | ||
#endif | ||
} catch(err) { | ||
ex = err; | ||
} | ||
|
||
assert(ex.name === "ErrnoError" && ex.errno === 28 /* EINVAL */); | ||
); | ||
|
||
EM_ASM( | ||
var ex; | ||
try { | ||
FS.truncate('nonexistent', 10); | ||
} catch(err) { | ||
ex = err; | ||
} | ||
assert(ex.name === "ErrnoError" && ex.errno === 44 /* ENOENT */); | ||
|
||
var ex; | ||
try { | ||
FS.ftruncate(99, 10); | ||
} catch(err) { | ||
ex = err; | ||
} | ||
|
||
assert(ex.name === "ErrnoError" && ex.errno === 8 /* EBADF */); | ||
); | ||
|
||
/********** test FS.rename() **********/ | ||
EM_ASM( | ||
FS.mkdir('renamedir'); | ||
|
@@ -130,6 +210,7 @@ int main() { | |
assert(ex.name === "ErrnoError" && ex.errno === 8 /* EBADF */) | ||
); | ||
|
||
remove("truncatetest"); | ||
remove("testfile"); | ||
remove("renametestfile"); | ||
remove("readtestfile"); | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If one calls
truncate("foo", -1)
I would probably expect it to try to resize of MAX_INT or something like that.. so maybe we don't need this check and we can just let the underlying call fail?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's what I was thinking, but I had one test that tried calling
truncate
with -10. I tried printing out the value the syscall receives, which was4294867286
. But instead of setting the return value to -1 and settingerrno
accordingly, I think the syscall aborts.This is the error. It doesn't seem to match the pattern of how we've been returning JS API errors before (Normally the syscall returns
-1
and I can returnerrno
to JS, then throw an error from JS with an error name and code). Is this close to what you were saying by letting the call fail?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree we don't want to syscall aborting like that. If you build with
-g
or--profiling-funcs
you should get functions names for those wasm functions in the backtrace BTW.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is strange because
doTruncate
does return -EINVAL when its input is negative. It would be good to get to the bottom of this!There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I noticed that
doTruncate
takes inoff_t size
, buttruncate64
andftruncate64
take in size as auint64_t
, so is it possible that the size is always positive indoTruncate
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is needed for the Node backend, as it silently accepts negative offsets rather than failing with
EINVAL
, see: nodejs/node#35632.Though, it looks like
setSize()
is currently not implemented for the Node backend:emscripten/system/lib/wasmfs/backends/node_backend.cpp
Lines 120 to 122 in 48b0645
(i.e. this PR won't work for
-sNODERAWFS
+-sWASMFS
)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's really interesting. I don't think I've been testing with
NODEFS
orNODERAWFS
. Should I start adding those flags totest_fs_js_api
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If
setSize()
(andinsertMove()
due toFS.rename()
) is supported in WasmFS' Node backend (which is probably better as a follow-up), you could do:After that, you can verify with:
$ ./test/runner wasmfs.test_fs_js_api* core2.test_fs_js_api*
Whether the test passes on both memory and Node backend, either with or without WasmFS.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After going back to the problem this morning, I realized I never checked that the right value was getting passed to
js_api.cpp
. I originally had_wamsfs_truncate
injs_api.cpp
take in size as typeoff_t
, and for some reason negative values would not be passed in correctly. After changingoff_t
tolong
, the tests pass without making any changes tosyscalls.cpp
. I also tried passing in anint64_t
tojs_api.cpp
, but that didn't seem to work. I initially wanted to useoff_t
to match the POSIX API, but could there be a potential problem with using that type injs_api.cpp
?Out of curiosity, I tried printing the values in
syscalls.cpp
, and I could printuint64_t
as a signed integer using%lld
. When theuint64_t
then gets passed todoTruncate
(Which takes in size as typeoff_t
), does C automatically force theuint64_t
to become signed? What if the unsigned value is larger than the signed max?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
off_t
is defined here:emscripten/system/lib/libc/musl/arch/emscripten/bits/alltypes.h
Lines 240 to 243 in 48b0645
So, it corresponds to
int64_t
. I just opened PR #19559 for this.