Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Create in-memory index for sheet command #254

Merged
merged 22 commits into from
Nov 3, 2024

Conversation

richiejp
Copy link
Contributor

@richiejp richiejp commented Nov 1, 2024

This adds an in-memory index to the sheet command. The index is built inside a background thread
and once it is complete navigating a 13GB file happens without latency on my development laptop.

TODO:

  • filtering
  • check find works correctly
  • move the index into the main library and improve the naming and documentation
  • test on older hardware and other platforms
  • fix filtering

commits:

  • Add Clang tools generated files to gitignore
  • Prevent JQ from being rebuilt every time
  • Minor fixes to silence warnings
  • Hack in-memory index in sheet command <-- main commit

Related blog post on adding the index

Somehow the source directory's mtime gets updated after libjq.a is
built, possibly because some object file for the jq cli is created
after that. So this copies the directory so that the original is left untouched.
Such as an unused function and the wrong enum type being returned.
@liquidaty liquidaty marked this pull request as ready for review November 3, 2024 02:34
@liquidaty liquidaty merged commit bc61cf2 into liquidaty:main Nov 3, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants