Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implementation Plan for Ingestion Server removal #3981

Closed
Tracked by #3925
stacimc opened this issue Mar 27, 2024 · 0 comments · Fixed by #4026
Closed
Tracked by #3925

Implementation Plan for Ingestion Server removal #3981

stacimc opened this issue Mar 27, 2024 · 0 comments · Fixed by #4026
Assignees
@stacimc
Labels
📄 aspect: text Concerns the textual material in the repository 🌟 goal: addition Addition of new feature 🟧 priority: high Stalls work on the project or its dependents 🧱 stack: catalog Related to the catalog and Airflow DAGs 🧱 stack: ingestion server Related to the ingestion/data refresh server
Milestone
Removal of the da...

Comments

@stacimc
Copy link
Collaborator

stacimc commented Mar 27, 2024

Problem

Write the Implementation Plan for the ingestion server removal project.

Description

As decided in the project proposal, the single implementation plan should cover:

  • mechanism for spinning up indexer workers from Airflow, along with required infrastructure changes
  • process for moving the rest of the data refresh into Airflow (data copy and cleaning, table/index promotion, etc)
@stacimc stacimc added 🟧 priority: high Stalls work on the project or its dependents 🌟 goal: addition Addition of new feature ✨ goal: improvement Improvement to an existing user-facing feature 📄 aspect: text Concerns the textual material in the repository 🧱 stack: ingestion server Related to the ingestion/data refresh server 🧱 stack: catalog Related to the catalog and Airflow DAGs labels Mar 27, 2024
@stacimc stacimc self-assigned this Mar 27, 2024
@stacimc stacimc removed the ✨ goal: improvement Improvement to an existing user-facing feature label Mar 27, 2024
@stacimc stacimc moved this to 🏗 In Progress in Openverse Backlog Mar 27, 2024
@openverse-bot openverse-bot moved this from 🏗 In Progress to 📋 Backlog in Openverse Backlog Mar 27, 2024
@openverse-bot openverse-bot moved this from 📋 Backlog to 📅 To Do in Openverse Backlog Mar 27, 2024
@stacimc stacimc mentioned this issue Apr 3, 2024
Merged
8 tasks
@openverse-bot openverse-bot moved this from 📅 To Do to 🏗 In Progress in Openverse Backlog Apr 3, 2024
@openverse-bot openverse-bot moved this from 🏗 In Progress to ✅ Done in Openverse Backlog Apr 17, 2024
@stacimc stacimc mentioned this issue Apr 17, 2024
Open
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@stacimc stacimc

Labels
📄 aspect: text Concerns the textual material in the repository 🌟 goal: addition Addition of new feature 🟧 priority: high Stalls work on the project or its dependents 🧱 stack: catalog Related to the catalog and Airflow DAGs 🧱 stack: ingestion server Related to the ingestion/data refresh server
Projects
Openverse Backlog
Archived in project
Milestone
Removal of the data refresh server
Development

Successfully merging a pull request may close this issue.

Implementation plan: Ingestion server removal WordPress/openverse
1 participant
@stacimc