-
Notifications
You must be signed in to change notification settings - Fork 46
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a retry mechanism for scraping failure #42
Comments
I will check more in-depth later, but for now, I see two options:
I think option 2 is more user-friendly, since the scraping, by definition, is not stable so you need to retry one of the importers in many cases. But this option is more complicated since then you have more than one button to trigger the process, so you need to create a locking architecture to avoid a parallel process. |
The third option we discussed is to identify the events that require us to retry- the timeout events, for example, and automatically retry the importer. |
# [1.16.0](v1.15.0...v1.16.0) (2023-12-04) ### Upgrade * Bump word-wrap from 1.2.3 to 1.2.5 (#516) ([9c9251b](9c9251b)), closes [#516](#516) [jonschlinkert/word-wrap#24](jonschlinkert/word-wrap#24) [jonschlinkert/word-wrap#41](jonschlinkert/word-wrap#41) [jonschlinkert/word-wrap#33](jonschlinkert/word-wrap#33) [jonschlinkert/word-wrap#42](jonschlinkert/word-wrap#42) [jonschlinkert/word-wrap#24](jonschlinkert/word-wrap#24) [jonschlinkert/word-wrap#41](jonschlinkert/word-wrap#41) [jonschlinkert/word-wrap#33](jonschlinkert/word-wrap#33) [#42](#42) [#41](#41)
Issue has been marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
Since scraping is a very integrative process it can fail from many causes. Instead of making the user retry himself, it would be good if there was a retry mechanism in the code that would retry x times if the scraping failed.
The text was updated successfully, but these errors were encountered: