Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RangeError: Invalid string length #407

Open
susnick opened this issue Oct 12, 2023 · 2 comments
Open

RangeError: Invalid string length #407

susnick opened this issue Oct 12, 2023 · 2 comments
Assignees

Comments

@susnick
Copy link

susnick commented Oct 12, 2023

Using version 0.7.31 trying to export document index to file system. Issue #68 seems to think the issue is corrected. If so, im not sure if I need to add anything to export. I am indexing a few million eml files for searching at later time.

function SaveIndex(account) {
let dir = path.join(INDEX_PATH, account);
if (!fs.existsSync(dir)){
fs.mkdirSync(dir);
}
document.export((key, data) => {
fs.writeFileSync(path.join(INDEX_PATH, account,${key}.json), data !== undefined ? data : '');
})
}

@susnick
Copy link
Author

susnick commented Oct 13, 2023

For what its worth, I just changed line 10 in serialize.js to remove the stringify. Then i used big-json to save the stream.

`function SaveFile(filename, object) {

const stringifyStream = json.createStringifyStream({
    body: object
});

stringifyStream.on('data', function(strChunk) {
    fs.appendFile(filename, strChunk, function (err) {
        if (err) throw err;
    })
});

}`

@susnick
Copy link
Author

susnick commented Oct 16, 2023

Loading the data was a pain still, so i went back and stored the id and some additional data in mysql so the index would be small as possible.

@ts-thomas ts-thomas self-assigned this Jan 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants