[BUG] Can't fetch the API key

[Eniterusx]

I have been using the API recently, but looks like it just broke?

File "\Lib\site-packages\howlongtobeatpy\HTMLRequests.py", line 102, in send_web_request
search_url_with_key = HTMLRequests.SEARCH_URL + "/" + api_key_result
                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~
TypeError: can only concatenate str (not "NoneType") to str

It has problems with fetching the api_key. Did they change the API yet again?
The request to howlongtobeat works perfectly, but it looks like the api key is no longer present in the html content.

[thebobbyboy420]

00:55:03.267 [SYSTEM] Fetching HLTB data for game: Black Myth: Wukong
00:55:21.198 WARNING [SYSTEM] HLTB search attempt 1 failed: can only concatenate str (not "NoneType") to str

yeah its broken sadly

[Eniterusx]

After messing around, this request is able to fetch the games correctly.

fetch("https://howlongtobeat.com/api/search", {
    method: "POST",
    headers: {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:132.0) Gecko/20100101 Firefox/132.0",
        "Accept": "*/*",
        "Accept-Language": "pl,en-US;q=0.7,en;q=0.3",
        "Accept-Encoding": "gzip, deflate, br, zstd",
        "Referer": "",
        "Content-Type": "application/json",
        "Origin": "https://howlongtobeat.com",
        "Sec-Fetch-Dest": "empty",
        "Sec-Fetch-Mode": "cors",
        "Sec-Fetch-Site": "same-origin",
        "Priority": "u=4",
        "TE": "trailers"
    },
    body: JSON.stringify({"searchType":"games","searchTerms":["celeste"],"searchPage":1,"size":20,"searchOptions":{"games":{"userId":0,"platform":"","sortCategory":"popular","rangeCategory":"main","rangeTime":{"min":null,"max":null},"gameplay":{"perspective":"","flow":"","genre":""},"rangeYear":{"min":"","max":""},"modifier":""},"users":{"id":"90f8120e015db09f","sortCategory":"postcount"},"lists":{"sortCategory":"follows"},"filter":"","sort":0,"randomizer":0},"useCache":true}),
    credentials: "same-origin"
})
.then(response => response.json())
.then(data => console.log(data))
.catch(error => console.error("Error:", error));

The only thing that looks similar to an api_key of sorts is the "id".
To fetch the id, first you make a request to the main site and extract the location of the _app.js file. Here's the line we're looking for
<script defer="" src="/_next/static/chunks/pages/_app-0ceb28b36df43067.js">
Then we simply request the contents of the file and extract the id.

Here's a quick python code that does exactly this:

import requests
from bs4 import BeautifulSoup
import re

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3"
}

main_page = "https://howlongtobeat.com/"

response = requests.get(main_page, headers=headers)
soup = BeautifulSoup(response.content, "html.parser")


for line in soup.prettify().split("\n"):
    if re.search("chunks/pages/_app", line):
        src = re.search(r'src="([^"]*)"', line).group(1)
        break

app_page = "https://howlongtobeat.com" + src

response = requests.get(app_page, headers=headers)

for line in response.content.decode().split("\n"):
    pattern = r'users:\{id:"([^"]+)"'
    match = re.search(pattern, line)
    if match:
        user_id = match.group(1)
        break

print(user_id)

[ScrappyCocco]

Thank you for the support
Fixed in https://github.com/ScrappyCocco/HowLongToBeat-PythonAPI/releases/tag/1.0.10

[nullseade]

Hi

it looks they've changed the code again, current javascript to scrape for me was
"/_next/static/chunks/pages/_app-858f64c972a7c8b1.js"
and users: tag there does not have id: field any more, just {sortCategory:X}

or is it working fine for you?

[godver3]

Also not working for me unfortunately.

[nullseade]

okay, it looks to me they went back to concat, current fetch in that .js I gave above is
fetch("/api/search/".concat("7b0f03b2").concat("54cc3099")
so combining these :alnum: regex matches gives the API key,

this following is not general advice, this is how it works for me
search needs to modified back to PATH+APIKEY search_url_with_key = HTMLRequests.SEARCH_URL + "/" + api_key_result,

[ScrappyCocco]

@nullseade @godver3 see #31 for updates