Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: add support for indexing documents using pagefind, and pushing to s3 #2934

Merged
merged 3 commits into from
Dec 19, 2024

Conversation

cdxker
Copy link
Member

@cdxker cdxker commented Dec 11, 2024

Please indicate what issue this PR is related to and @ any maintainers who are relevant

@cdxker
Copy link
Member Author

cdxker commented Dec 13, 2024

Instructions for running locally

Creating pagefind index

Set env's for pagefind

S3_ENDPOINT_PAGEFIND=https://pagefind-index-west.s3.us-west-1.amazonaws.com
S3_ACCESS_KEY_PAGEFIND=********************
S3_SECRET_KEY_PAGEFIND=**********************************************
S3_BUCKET_PAGEFIND=pagfind-index-west
AWS_REGION_PAGEFIND=us-west-1
cargo run --bin pagefind-worker
cargo run
let body = {
}
httppost $"($url)/api/dataset/pagefind" $body

After the index is created, it should be accessible under https://pagefind-testing-index.trieve.ai/pagfind-index-west/pagefind/

Using pagefind index for search componetn

Set env's under clients/search-component/example/.env

VITE_USE_GROUP_SEARCH="false"
VITE_USE_PAGEFIND="true"
VITE_PAGEFIND_URL="https://pagefind-testing-index.trieve.ai/pagfind-index-west/pagefind"

Run component locally under docsearch mode and search

pagefind

@cdxker cdxker force-pushed the cd/add-pagefind-support branch from a097bbd to c282d23 Compare December 14, 2024 00:34
@cdxker cdxker force-pushed the cd/add-pagefind-support branch from a34d7e0 to 3e4cef0 Compare December 16, 2024 21:02
@cdxker cdxker force-pushed the cd/add-pagefind-support branch 2 times, most recently from 8b5f373 to 85b4ef4 Compare December 19, 2024 00:19
@cdxker cdxker force-pushed the cd/add-pagefind-support branch 9 times, most recently from 329b93d to 5ae3c4b Compare December 19, 2024 02:41
@cdxker cdxker marked this pull request as ready for review December 19, 2024 02:41
skeptrunedev
skeptrunedev previously approved these changes Dec 19, 2024
@cdxker cdxker merged commit 52b5c00 into main Dec 19, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants