You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently the CDB reader uses io.ReaderAt, that for each read operation will issue a read system call (or equivalent). An alternative would be to use mmap-ed memory to access the backing file.
In my experiments (see the patch below) by using mmap yields a 35 times performance improvement:
The following are a few patches that provide such support. If accepted I can submit a pull request:
cipriancraciun@6c948e5 -- adds support of using a []byte buffer that backs the CDB reader; such a []byte buffer can be either the result of a mmap or just reading the entire file in memory;
cipriancraciun@0b1bb73 -- this adds support for a custom io.Close to be called on cdb.Close; (it will be used to munmap the memory if needed;)
the []byte value returned as result of Get should not be written to by the application, as it's actually a slice from the mmap-ed file;
the []byte value can't be used after the cdb.Close was called;
if this is not acceptable, and one still wants to keep the current (undocumented) semantic, one could clone that slice (thus incurring an extra alloc, which in my patch doesn't happen);
The text was updated successfully, but these errors were encountered:
Currently the CDB reader uses
io.ReaderAt
, that for each read operation will issue aread
system call (or equivalent). An alternative would be to usemmap
-ed memory to access the backing file.In my experiments (see the patch below) by using
mmap
yields a 35 times performance improvement:The following are a few patches that provide such support. If accepted I can submit a pull request:
[]byte
buffer that backs the CDB reader; such a[]byte
buffer can be either the result of ammap
or just reading the entire file in memory;io.Close
to be called oncdb.Close
; (it will be used tomunmap
the memory if needed;)mmap
support;The only semantic change is the following:
[]byte
value returned as result ofGet
should not be written to by the application, as it's actually a slice from themmap
-ed file;[]byte
value can't be used after thecdb.Close
was called;The text was updated successfully, but these errors were encountered: