Skip to content

Update regex expressions to handle unexpected '};'#1953

Merged
TheFrenchGhosty merged 1 commit intoiv-org:masterfrom
syeopite:fix1673
May 13, 2021
Merged

Update regex expressions to handle unexpected '};'#1953
TheFrenchGhosty merged 1 commit intoiv-org:masterfrom
syeopite:fix1673

Conversation

@syeopite
Copy link
Member

@syeopite syeopite commented Mar 30, 2021

Fixes: #1673
I want to be awarded the bounty associated to the issue this PR is fixing.

Tested with the following ids:

Details

O4r9D2jI0_w

VQOdmckqNro

3AF3sYnvj5o

BFlF9gGnyro

BUU7WFd8XLU

channel/UCRZY6MGMG1Fmzt50rYalTwQ

I wasn't able to reproduce the error in any of the videos linked in duplicate issues. However, this does fix the error in the original video and any other content with "};".

def extract_polymer_config(body)
params = {} of String => JSON::Any
player_response = body.match(/(window\["ytInitialPlayerResponse"\]|var\sytInitialPlayerResponse)\s*=\s*(?<info>{.*?});/m)
player_response = body.match(/(window\["ytInitialPlayerResponse"\]|var\sytInitialPlayerResponse)\s*=\s*(?<info>{.*?});\s*var\s*meta/m)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have noticed that this extra javascript (var meta = ...) is not always present in the script.
I can't tell what triggers it (I've tried multiple combinations of URL arguments, with no luck).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh that's strange. Its always been present on my end.

@TheFrenchGhosty TheFrenchGhosty added need-code-review A crystal developper need to check if the code is correct. need-testing This feature needs to be deployed and tested to see if it's working, and doesn't break something labels Apr 1, 2021
Copy link
Member

@saltycrys saltycrys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The most robust way to handle this would be to put a full JSON parser in there so only the starting point needs to be found. That would be terrible for performance probably.

But as long as this works that's not necessary.

@TheFrenchGhosty TheFrenchGhosty removed the need-code-review A crystal developper need to check if the code is correct. label Apr 6, 2021
@SamantazFox
Copy link
Member

The most robust way to handle this would be to put a full JSON parser in there so only the starting point needs to be found. That would be terrible for performance probably.

Or, as proposed in #1981, find a direct API providing the required ytInitialData (and ytInitialPlayerResponse for videos)...

@saltycrys
Copy link
Member

Or, as proposed in #1981, find a direct API providing the required ytInitialData (and ytInitialPlayerResponse for videos)...

That is fantastic! @TheFrenchGhosty put some bounties on that please.

@Perflyst
Copy link
Contributor

Perflyst commented Apr 8, 2021

watch?v=VQOdmckqNro works on my instance without this patch so I can't really test it

@TheFrenchGhosty TheFrenchGhosty added bounty in-testing This feature has been deployed and is being tested and removed need-testing This feature needs to be deployed and tested to see if it's working, and doesn't break something in-testing This feature has been deployed and is being tested labels May 13, 2021
@TheFrenchGhosty TheFrenchGhosty merged commit 73da846 into iv-org:master May 13, 2021
@TheFrenchGhosty
Copy link
Member

Merged.

@syeopite to receive your reward either send your BTC address in a comment (preferred), or contact us directly, with a proof that this is indeed you.

@syeopite
Copy link
Member Author

bc1qpgyc5hkcwg4l4a28j274g4tufym07e0as7pwwk

@TheFrenchGhosty
Copy link
Member

A 20$ bounty has been rewarded to @syeopite for this PR.

Transaction (caa3a36040c086642a0262048f95b0c5d84009e87ddc34acf65b6928895f3fdb):

Amount sent: 0.000405221 BTC (~16.46€ or ~20$)

Mining fees (10 sat/byte): 0.0000481 BTC (~1.95€)

@syeopite syeopite deleted the fix1673 branch May 15, 2021 16:34
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 15, 2021
@syeopite syeopite added the bounty:paid The bounty has been paid label Jul 17, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

bounty:paid The bounty has been paid bounty

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Unexpected char '<' at 1:1 (JSON::ParseException) - On videos

5 participants