Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NYUP 739 schema changes #2

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

NYUP 739 schema changes #2

wants to merge 4 commits into from

Conversation

teikmeout
Copy link
Collaborator

Jira Ticket NYUP739
Jira Ticket NYUP546

Do not merge this PR!

Intended for testing #2
Once #2 is merged I will make changes to this PR since right now in this state the submodule is pointing at a specific Branch to use newer schema until it's merged into a staging branch.

This PR will add the following:

  • ignore dlts-epub-metadata until we decide if it's a submodule to this repository or not
  • add/remove deno ingestion script to accomodate for the new schema changes (note if this schema change is supposed to happen before data in put into dlts-epub-metadata then this might cause conflicts once the new schema shape is commited to that repo
  • update the branch the submodule points at for the solr configuration to ensure testability (note that this might be best to remove since once we merge the changes of the schema in we will have our changes reflected on the already configured branch

To test this PR:

  • at the root of the project: git submodule update --init to align submodule branches
    • dlts-epub-metadata should be on main
    • solr_conf_open-square-metadata should be on NYUP-739-schema-changes
  • docker compose up (should trigger rebuild since there are changes on the configuration layer with the new branch change), solr dashboard on http://localhost:8983/
  • cd src/scripts/ingest-document
  • deno run --allow-net --allow-read index.ts
  • query data to proof ingestion

Before merging I recommend:

  • we merge in the schema #2
  • we change the submodule branch back to devdiscovery.dlib.nyu.edu on this PR

Thank you for your review.

@teikmeout teikmeout requested a review from dismorfo May 14, 2024 19:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant