Skip to content

Commit

Permalink
Fix response handling
Browse files Browse the repository at this point in the history
## Following
- [this issue](qbittorrent/qBittorrent#22074) on the main qBittorrent repository
- and the discussion on [this subsequent pull request](qbittorrent/qBittorrent#22075) 

Here is the fix for the piratebay search engine. A gist of the code is available [here](https://gist.github.com/biskweet/f06ff7b260ef1ce3a31d27ac1a9edcbf) for testing.

## Recalling the problem:
API apibay.org returns weird JSON that causes the piratebay search engine to crash when handling its response. If some search results contain `"` (quotation marks) characters, the server escapes them by replacing `"` with `"` HTML entities in order to still provide a syntactically valid JSON response. While this is not incorrect, it would be best if apibay.org returned properly escaped quotes, i.e. using backslashes.

When handling the response data, functions [`retrieve_url`](https://github.com/LightDestory/qBittorrent-Search-Plugins/blob/master/src/helpers.py#L75-L117) and [`htmlentitydecode`](https://github.com/LightDestory/qBittorrent-Search-Plugins/blob/master/src/helpers.py#L75-L117) blindly unescape all entities thereby corrupting previously valid JSON. As a consequence, `json.loads` crashes. For example:
```json
{
  "title": "Ubuntu 22.04.5 LTS ("Jammy Jellyfish")"
}
```
becomes
```json
{
  "title": "Ubuntu 22.04.5 LTS ("Jammy Jellyfish")"
}
```

## Solution proposed
We no longer use the `retrieve_url` function -- instead, I created a dedicated `retrieve_url` function (which is almost a copy-paste of the original) that fixes the problem by manually escaping quotes *before* escaping the rest of the data.

PR #331.
  • Loading branch information
biskweet authored Jan 12, 2025
1 parent 1e4b595 commit 37e6741
Show file tree
Hide file tree
Showing 2 changed files with 40 additions and 5 deletions.
43 changes: 39 additions & 4 deletions nova3/engines/piratebay.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# VERSION: 3.3
# VERSION: 3.4
# AUTHORS: Fabien Devaux ([email protected])
# CONTRIBUTORS: Christophe Dumez ([email protected])
# Arthur ([email protected])
Expand Down Expand Up @@ -28,11 +28,16 @@
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.

import gzip
import html
import io
import json
import urllib.error
import urllib.request
from urllib.parse import urlencode, unquote

from helpers import getBrowserUserAgent
from novaprinter import prettyPrinter
from helpers import retrieve_url


class piratebay(object):
Expand Down Expand Up @@ -70,8 +75,10 @@ def search(self, what, cat='all'):
params = {'q': what}
if category != '0':
params['cat'] = category
response = retrieve_url(base_url % urlencode(params))
response_json = json.loads(response)

# Calling custom `retrieve_url` function with adequate escaping
data = self.retrieve_url(base_url % urlencode(params))
response_json = json.loads(data)

# check empty response
if len(response_json) == 0:
Expand All @@ -96,3 +103,31 @@ def search(self, what, cat='all'):
def download_link(self, result):
return "magnet:?xt=urn:btih:{}&{}&{}".format(
result['info_hash'], urlencode({'dn': result['name']}), self.trackers)

def retrieve_url(self, url):
# Request data from API
request = urllib.request.Request(url, None, {'User-Agent': getBrowserUserAgent()})

try:
response = urllib.request.urlopen(request)
except urllib.error.HTTPError:
return ""

data = response.read()

if data[:2] == b'\x1f\x8b':
# Data is gzip encoded, decode it
with io.BytesIO(data) as stream, gzip.GzipFile(fileobj=stream) as gzipper:
data = gzipper.read()

charset = 'utf-8'
try:
charset = response.getheader('Content-Type', '').split('charset=', 1)[1]
except IndexError:
pass

dataStr = data.decode(charset, 'replace')
dataStr = dataStr.replace('"', '\\"') # Manually escape " before
dataStr = html.unescape(dataStr)

return dataStr
2 changes: 1 addition & 1 deletion nova3/engines/versions.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
eztv: 1.16
jackett: 4.0
limetorrents: 4.10
piratebay: 3.3
piratebay: 3.4
solidtorrents: 2.4
torlock: 2.24
torrentproject: 1.5
Expand Down

0 comments on commit 37e6741

Please sign in to comment.