Skip to content

Providers

Jump to bottom
davidreedmichigan edited this page Jul 21, 2019 · 2 revisions

What is it?

Providers are really simple JSON objects that tell to helios where and how to find sources.

Please note that for legal reasons, I won't discuss, develop nor distribute any providers connecting to illegal sources. So there is no need to ask me. While I can partake in general discussions regarding provider development, I won't do so for illegal sources specific problems.

How it works

If you want to create a new provider, here is the Provider interface (you need some HTML/JavaScript background): Provider Interface

Let's see an example:

{
  "my_first_provider": {
    "name": "MyProvider1",
    "enabled": true,
    "enabled_in_list": true,
    "languages": ["en"],
    "base_url": "https://www.my-provider.com/search/",
    "response_type": "text",
    "movie": {
      "query": "{query}/Movies/1/",
      "keywords": "{title} {year}"
    },
    "episode": {
      "query": "{query}/TV/1/",
      "keywords": "{title} {episodeCode}"
    },
    "anime": {
      "query": "{query}/Anime/1/",
      "keywords": "{title} {episode}"
    },
    "html_parser": {
      "row": "doc.querySelectorAll('tbody > tr')",
      "title": "row.querySelector('a:nth-child(2)').innerHTML",
      "peers": "row.querySelector('.leeches').innerHTML",
      "seeds": "row.querySelector('.seeds').innerHTML",
      "size": "row.querySelector('tbody > tr .size').textContent.split('B')[0] + 'B'",
      "url": "'https://www.my-provider.com/'+ row.querySelector('a:nth-child(2)').getAttribute('href')"
    },
    "source_is_in_sub_page": true,
     "title_replacement": {
      "'s": "s",
      "\"": ""
    }
  },
  "my_second_provider": {
    "name": "MyProvider2",
    "enabled": true,
    "enabled_in_list": true,
    "languages": [
      "en"
    ],
    "base_url": "https://my-provider2.org/api",
    "response_type": "json",
    "time_to_wait_between_each_request_ms": 300,
    "time_to_wait_on_too_many_request_ms": 5000,
    "token": {
      "query": "?get_token=get_token",
      "token_validity_time_ms": 840000,
      "token_format": {
        "token": "token"
      }
    },
    "movie": {
      "query": "?mode=search&search_imdb={query}&category=movies&token={token}",
      "keywords": "{imdbId}"
    },
    "episode": {
      "query": "?mode=search&search_string={query}&category=tv&token={token}",
      "keywords": "{title} {episodeCode}"
    },
    "json_format": {
      "results": "torrent_results",
      "url": "download",
      "title": "title",
      "seeds": "seeders",
      "peers": "leechers",
      "size": "size"
    },
     "title_replacement": {
      "'s": "s",
      "\"": ""
    }
  }
}

Here we have 2 providers my_first_provider and my_second_provider. The first one will parse html pages while the second will use a JSON API

my_first_provider

Let's say we want to look for the movie "Helios Movie" aired on 2010. Here are the steps helios is gonna do to search for this query:

  1. Replace {title} {year} by Helios Movie 2010 (bellow all available variables)
  2. Replace {query}/Movies/1/ by Helios%20Movie%202010/Movies/1/
  3. Call this url https://www.my-provider.com/search/Helios%20Movie%202010/Movies/1/

If the url returns a 200 response, helios will parse the HTML received using html_parser property. It will:

  1. Execute doc.querySelectorAll('tbody > tr') where doc is the HTML document received
  2. For each returned row it will try to find all the other properties:
    2.1. Get the title from row.querySelector('a:nth-child(2)').innerHTML 2.2. Get the peers from row.querySelector('.leeches').innerHTML
    2.3. Get the seeds from row.querySelector('.seeds').innerHTML
    2.4. Get the size from row.querySelector('tbody > tr .size').textContent.split('B')[0] + 'B' (If the size is a number it will be converted automatically)
    2.5. This provider has source_is_in_sub_page = true which means the url we're gonna get is not the torrent url but a page where the magnet url will be find inside the source code. We get this sub page url like this from row.querySelector('a:nth-child(2)').getAttribute('href').

Ok we saw how to search for a movie, it's basically the same for a TV show. {episodeCode} will look like sXXeYY where XX is the season number and YY is the episode number.

my_second_provider

Dealing with JSON API is better than parsing HTML. Let's search for the same movie "Helios Movie 2010" with the IMDb id: "tt99999"

This API needs a token for every request, first helios will retrieve this token because it is dynamic and it doesn't last forever.

helios will call "https://my-provider2.org/api?get_token=get_token". It expects a JSON response like that:

{
  "token":"MyToKeN"
}

Once the token is retrieved, it will be stored for "840000" ms, after that another token will be asked before new requests.

Let's search. Here the keywords are not title+year but imdbId. Searching by imdbId will provide accurate results.

helios will proceed the same for all the replacements and finally it will call the URL: "https://my-provider2.org/api?mode=search&search_imdb=tt99999&category=movies&token=MyToKeN"

The expected result for this request will look like this:

{
 "torrent_results": [
    {
      "url":"magnet:XXXXX",
      "title":"Helios Movie 2010 By YYY",
      "seeds":100,
      "peers": 90,
      "size":10000
    },

    {
      "url":"magnet:XXXXX",
      "title":"Helios Movie 2010 By ZZZ",
      "seeds":55,
      "peers": 20,
      "size":90000
    },
      ....
 ]
}

Real example

Here is the provider for Legit Torrent (a torrent tracker for only legal torrents)

{
    "legittorrents": {
    "name": "Legit Torrents",
    "enabled": true,
    "enabled_in_list": false,
    "languages": [
      "en"
    ],
    "base_url": "http://www.legittorrents.info/index.php?page=torrents&active=1",
    "response_type": "text",
    "movie": {
      "query": "&search={query}&category=1",
      "keywords": "{title} "
    },
    "html_parser": {
      "row": "doc.querySelectorAll('#bodyarea tr table:nth-child(3) table.lista tr')",
      "title": "row.querySelector('td:nth-child(2) a').textContent",
      "peers": "row.querySelector('td:nth-child(6)').textContent",
      "seeds": "row.querySelector('td:nth-child(5)').textContent",
      "size": null,
      "url": "'http://www.legittorrents.info/'+ row.querySelector('td:nth-child(3) a').getAttribute('href')"
    },
    "source_is_in_sub_page": false
  }
}

Available variables

This is the list of all variable that can be added.

  • title: Helios Movie
  • titleFirstLetter: W
  • imdbId
  • year
  • episodeCode (i.e. s02e06)
  • season (i.e. 02)
  • episode (i.e. 06)
  • query
  • token For the title you can choose the title in the language you want. By default "title" will be replaced by the original title. If you want the french title, use "title.fr"
Clone this wiki locally