MS api throttling mitigation #203
Conversation
fixed side effects in main function of this change
|
Oh @snobu, please update the subversion number and maybe a little bit of description? Also about the title template since I don't remember if we added that last update |
|
Looks like we have some conflicts now, can you resolve them before we go to review? |
| let i = 0; | ||
| finalTitle = title; | ||
| while (match) { | ||
| let value = video[match[1] as keyof Video] as string; |
There was a problem hiding this comment.
(____/)
( ͡ ͡° ͜ ʖ ͡ ͡°)
\╭☞ \╭☞. Love the keyof T here.
Sure can, between this evening and tomorrow I'll push |
|
Looks like after this change we won't be able to verify things upfront, meaning if one of the URLs returns 404 (invalid GUID or video has been deleted by owner) we won't be able to catch it early. What if we introduce a small delay before calling the API for each GUID yet we still do it all upfront? Did you notice aggressive throttling from the MSStream API, meaning can we do more than one API call a second? Edit: |
|
Bizarre, i'm unable to go into throttling. Tried both HTTP/1.1 and HTTP/2, even 10 req/second work out fine. However throttling may not be even across all API endpoints, i'm only able to call Not sure how to approach this right now to be honest. I'd say let's park it for now unless there are more throttling reports coming in and then maybe we have more data to draw a more meaningful conclusion. |
|
I occur into it especially with groups link, where there are like 30+ videos I tried a delay of 1 2 and 3 secs and all of them get throttled after the 10th But I agree with you, let's park this and let's see how it goes |
As told you in the last PR here is the change that will help with #170, so now it should not request thing in bulk so that it doesn't get soft-banned for 10 sec