Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

items: improve response time for csv export #1842

Merged
merged 1 commit into from
May 31, 2021

Conversation

lauren-d
Copy link
Contributor

@lauren-d lauren-d commented Apr 20, 2021

The CSV export of the inventory list has an important waiting time.
With this commit, we try to improve the response time.

Co-Authored-by: Laurent Dubois [email protected]

Why are you opening this PR?

Dependencies

My PR depends on rero-ils#<xx>'s PR(s):

Code review check list

  • Commit message template compliance.
  • Commit message without typos.
  • File names.
  • Functions names.
  • Functions docstrings.
  • Unnecessary commited files?
  • Cypress tests successful?

@lauren-d lauren-d self-assigned this Apr 20, 2021
@lauren-d lauren-d marked this pull request as draft April 20, 2021 14:51
@lauren-d lauren-d force-pushed the lau-#2085-improve-csv-export branch 3 times, most recently from 9c69b84 to 3732771 Compare April 21, 2021 06:16
@lauren-d lauren-d added the enhancement Improvement of an existing feature label Apr 22, 2021
@lauren-d lauren-d added this to the v1.3.0 milestone Apr 22, 2021
@lauren-d lauren-d force-pushed the lau-#2085-improve-csv-export branch from f02285c to 51255b1 Compare April 30, 2021 08:29
@lauren-d lauren-d force-pushed the lau-#2085-improve-csv-export branch 3 times, most recently from f9401a2 to 48f6d49 Compare April 30, 2021 08:59
@lauren-d lauren-d changed the title items: improve time for csv export items: improve response time for csv export Apr 30, 2021
@lauren-d lauren-d force-pushed the lau-#2085-improve-csv-export branch 11 times, most recently from 00db7de to 8a51793 Compare April 30, 2021 14:22
@lauren-d lauren-d marked this pull request as ready for review April 30, 2021 14:26
@lauren-d lauren-d force-pushed the lau-#2085-improve-csv-export branch 5 times, most recently from e22d8aa to 812794d Compare April 30, 2021 19:54
@lauren-d lauren-d force-pushed the lau-#2085-improve-csv-export branch 3 times, most recently from 4891da7 to 975c04b Compare April 30, 2021 22:42
Copy link

@iGormilhit iGormilhit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commit message approved.

@lauren-d lauren-d force-pushed the lau-#2085-improve-csv-export branch from 975c04b to 43deba6 Compare May 3, 2021 06:40
@lauren-d lauren-d force-pushed the lau-#2085-improve-csv-export branch from 43deba6 to 1670b3a Compare May 3, 2021 07:22
@lauren-d lauren-d requested a review from zannkukai May 3, 2021 07:22
@lauren-d lauren-d force-pushed the lau-#2085-improve-csv-export branch from 1670b3a to b0296ae Compare May 3, 2021 10:13
@JoelleDosimont
Copy link
Contributor

JoelleDosimont commented May 3, 2021

  • The Temporary item type seems not to be taken into account if there is no "end-date".
  • The Temporary location should appear in the export.
  • Serial issue should have the call number of the holdings if left blank.
  • [Not important] For an inventory, Call number should be the first column.
  • The button "Add" should not exist on the page https://ilsdev.test.rero.ch/professional/records/items
  • How are we supposed to test the performance if only 742 items are loaded in the Aoste organisation? Is it possible to loan more than 10'000 items in one organisation so that we can test?

@benerken
Copy link
Contributor

benerken commented May 3, 2021

image

@lauren-d
Copy link
Contributor Author

lauren-d commented May 3, 2021

  • The Temporary location should appear in the export.

According to Joëlle and Benoit, it's not necessary to have that in the export

  • [Not important] For an inventory, Call number should be the first column.

Maybe that could be done in a spreadsheet ?

@lauren-d lauren-d force-pushed the lau-#2085-improve-csv-export branch from b0296ae to 079ac0b Compare May 5, 2021 05:55
@lauren-d lauren-d force-pushed the lau-#2085-improve-csv-export branch from 079ac0b to 01beedd Compare May 11, 2021 08:17
The CSV export of the inventory list has an important waiting time.
With this commit, we try to improve the response time.

* Uses ciso8601 to parse date fields instead `format_date_filter`.
* Adds new api endpoint for inventory list.
* Uses streaming to process download csv file.
* Uses bunch request to reduce elasticsearch calls.
* Adds some fields in csv file.
* Closes rero#1456.
* Closes rero#1329.

Co-Authored-by: Laurent Dubois <[email protected]>
@lauren-d lauren-d force-pushed the lau-#2085-improve-csv-export branch from 01beedd to bb0fffd Compare May 31, 2021 09:19
@github-actions github-actions bot added f: circulation Concerns the circulation interface or backend f: search and removed data-export labels May 31, 2021
@lauren-d lauren-d merged commit 45d8fd8 into rero:dev May 31, 2021
@lauren-d lauren-d deleted the lau-#2085-improve-csv-export branch June 15, 2021 11:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Improvement of an existing feature f: circulation Concerns the circulation interface or backend f: search
Projects
None yet
7 participants