Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

When editing an item of a document with lots of holdings and items, ES takes to much time #1943

Closed
pronguen opened this issue May 25, 2021 · 1 comment · Fixed by #1912
Closed

When editing an item of a document with lots of holdings and items, ES takes to much time #1943

pronguen opened this issue May 25, 2021 · 1 comment · Fixed by #1912
Assignees
@jma @rerowep
Labels
bug (critical) Blocking issue, should be fixed ASAP bug Breaks something but is not blocking f: editor Concerns editor based on JSON schema AND custom editor f: search p-High High priority (to be solved in the 2-3 next months)
Milestone
v1.3.0

Comments

@pronguen
Copy link
Contributor

pronguen commented May 25, 2021 •
edited by iGormilhit
Loading

Describe the bug

When I edit this item (https://ilspilotdev.test.rero.ch/professional/records/items/edit/1612144), I receive an error "The form contains error". But the changes are saved.

This does not happen with other issues or standard items.

image

The issue is that ES takes too much time (because of too much records to reindex, see below) and the client closes the connexion.

To Reproduce

  1. Log in as an admin in organisation "Bulle"
  2. Edit https://ilspilotdev.test.rero.ch/professional/records/items/edit/1612144
  3. Try to save

Expected behavior

It is possible to save the changes without error message.

Context

  • server: ilspilotdev.test.rero.ch (will be soon ilspilot.test.rero.ch)
  • version: v1.2.0

Additional information

This document contains 23 holdings and fast 1000 items. Updating on item (even with no modification) result in reindexing the whole document stack (document/contributions/holdings/items), which takes too much time. As this is a regular use case, another method has to be implemented.
@pronguen pronguen added f: editor Concerns editor based on JSON schema AND custom editor bug Breaks something but is not blocking p-High High priority (to be solved in the 2-3 next months) labels May 25, 2021
@iGormilhit iGormilhit added this to the v1.3.0 milestone May 26, 2021
@iGormilhit
Copy link

@rerowep says it's related to Elasticsearch. He's on it.

@iGormilhit iGormilhit changed the title When editing an issue, I receive an error "The form contains error". When editing an item of a document with lots of holdings and items, ES crashes because of the reindexing May 26, 2021
@iGormilhit iGormilhit added f: search bug (critical) Blocking issue, should be fixed ASAP labels May 26, 2021
@iGormilhit iGormilhit changed the title When editing an item of a document with lots of holdings and items, ES crashes because of the reindexing When editing an item of a document with lots of holdings and items, ES takes to much time May 26, 2021
@iGormilhit iGormilhit modified the milestones: v1.3.0, v1.4.0 Jun 2, 2021
zannkukai added a commit to zannkukai/rero-ils that referenced this issue Jun 7, 2021
@zannkukai
resources: improve indexing perf & ES mapping
82b741e 
* Marks an electronic holdings as always available.
* Fixes ES mapping for some resources.
* Cleans some tests from fixture `es_clear`.
* Improves indexing process :
  - removes item related collection metadata during document indexing
    process.
  - removes `local fields` indexation on Item and Holding.
  - remove 'availability' keys during item indexing process for doc,
    holding and item resources. This can only be consistent for an
    organisation view and can't be stored into ES.
* Closes rero#1872.
* Closes rero#1943.

Co-Authored-by: Renaud Michotte <[email protected]>
@zannkukai zannkukai mentioned this issue Jun 8, 2021
Merged
7 tasks
zannkukai added a commit to zannkukai/rero-ils that referenced this issue Jun 8, 2021
@zannkukai
resources: improve indexing perf & ES mapping
4a254a0 
* Marks an electronic holdings as always available.
* Fixes ES mapping for some resources.
* Cleans some tests from fixture `es_clear`.
* Improves indexing process :
  - removes item related collection metadata during document indexing
    process.
  - removes `local fields` indexation on Item and Holding.
  - remove 'availability' keys during item indexing process for doc,
    holding and item resources. This can only be consistent for an
    organisation view and can't be stored into ES.
* Closes rero#1872.
* Closes rero#1943.

Co-Authored-by: Renaud Michotte <[email protected]>
zannkukai added a commit that referenced this issue Jun 8, 2021
@zannkukai
resources: improve indexing perf & ES mapping
51386d1 
* Marks an electronic holdings as always available.
* Fixes ES mapping for some resources.
* Cleans some tests from fixture `es_clear`.
* Improves indexing process :
  - removes item related collection metadata during document indexing
    process.
  - removes `local fields` indexation on Item and Holding.
  - remove 'availability' keys during item indexing process for doc,
    holding and item resources. This can only be consistent for an
    organisation view and can't be stored into ES.
* Closes #1872.
* Closes #1943.

Co-Authored-by: Renaud Michotte <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jma jma

@rerowep rerowep

Labels
bug (critical) Blocking issue, should be fixed ASAP bug Breaks something but is not blocking f: editor Concerns editor based on JSON schema AND custom editor f: search p-High High priority (to be solved in the 2-3 next months)
Projects
None yet
Milestone
v1.3.0
Development

Successfully merging a pull request may close this issue.

resources: improve indexing performance & ES mapping zannkukai/rero-ils
4 participants
@jma @iGormilhit @pronguen @rerowep