Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OCT-169: fix migration of ES index #18374

Merged
merged 2 commits into from
Nov 9, 2022
Merged

OCT-169: fix migration of ES index #18374

merged 2 commits into from
Nov 9, 2022

Conversation

tseho
Copy link
Contributor

@tseho tseho commented Nov 8, 2022

Context

The migration is executed asynchronously.
Before the doctrine migration was executed, the webhook consumer started indexing new documents in the index akeneo_connectivity_connection_events_api_debug, with a new property id.
Because you can index documents without configuring a mapping beforehand, the new property has been automatically created by ES with the type text instead of keyword.

In ES, you cannot directly change the type of a property. (see https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html#updating-field-mappings)

The migration is failing for customers having sent webhooks between the deployment and the doctrine migration execution.

The dirty solution

Since we cannot rename a property, cannot change the type of a property, cannot remove a property of an existing index without re-indexing everything, and we cannot re-index this particular index.

Here what's happening in the new migration:

  • if the id property already exists
  • then, creates a temporary indice
  • copy documents from the corrupted indice to the temporary one (using re-index with source + dest)
  • redirects the alias from the corrupted indice to the temporary one (so what we are still writing/reading somewhere)
  • delete the corrupted indice
  • create a new indice with the correct mapping and the original name
  • copy documents from the temporary indice to the new one
  • redirects the alias from the temporary indice to the new one
  • removes the temporary indice

Definition Of Done (for Core Developer only)

  • Tests
  • Migration & Installer
  • PM Validation (Story)
  • Changelog (maintenance bug fixes)
  • Tech Doc

@tseho tseho merged commit 1422682 into master Nov 9, 2022
@tseho tseho deleted the OCT-169 branch November 9, 2022 09:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants