Ability to identify in the classifier movies and TV shows from lists in sonarr, radarr, jellyfin and TMDB discover

[rraymondgh]

• I have checked the existing issues to avoid duplicates
• I have redacted any info hashes and content metadata from any logs or screenshots attached to this issue

Is your feature request related to a problem?

In order to use the flexibility of the classifier, the flexibility to define custom rules and workflows is invaluable. This could be further enhanced by having a plugin system that sources external lists of TMDB ids for use in classifier rules.

For example

    - if_else:
        condition:
          and:
            - "result.contentType == contentType.tv_show"
            - "result.contentSource == 'tmdb' "
            - "result.contentId in extensions.sonarr "
        if_action:
          add_tag: sonarr

would be far more flexible if extensions.sonarr list was sourced from sonarr

Describe the solution you'd like

plugin defined in classifier.yml

for example:

plugins:
  - url: http://192.168.1.202:8989/api/v3/movie?apikey=abc123
    extension: radarr
  - url: http://192.168.1.202:7878/api/v3/series?apikey=def789
    extension: sonarr

source values in classifier go code

internal/classifier/source.go

@@ -7,6 +7,7 @@ type Source struct {
        Flags           Flags           `json:"flags"`
        Keywords        keywordGroups   `json:"keywords"`
        Extensions      extensionGroups `json:"extensions"`
+       Plugins         pluginGroups    `json:"plugins,omitempty"`
 }
 
 func (s Source) merge(other Source) (Source, error) {
@@ -20,6 +21,7 @@ func (s Source) merge(other Source) (Source, error) {
                Keywords:        s.Keywords.merge(other.Keywords),
                Extensions:      s.Extensions.merge(other.Extensions),
                Workflows:       s.Workflows.merge(other.Workflows),
+               Plugins:         s.Plugins.merge(other.Plugins),
        }, nil
 }
 
@@ -81,3 +83,19 @@ func (s workflowSources) merge(other workflowSources) workflowSources {
        }
        return result
 }
+
+type pluginGroups []struct {
+       Url       string `json:"url"`
+       Extension string `json:"extension"`
+}
+
+func (s pluginGroups) merge(other pluginGroups) pluginGroups {
+       var result pluginGroups
+       for _, v := range s {
+               result = append(result, v)
+       }
+       for _, v := range other {
+               result = append(result, v)
+       }
+       return result
+}

internal/classifier/source_provider.go

 func newSourceProvider(config Config, tmdbConfig tmdb.Config) sourceProvider {
@@ -42,6 +43,14 @@ func (m mergeSourceProvider) source() (Source, error) {
                        source = merged
                }
        }
+       for _, plugin := range source.Plugins {
+               err, data := extensionPlugin{url: plugin.Url}.source()
+               if err != nil {
+                       return source, err
+               }
+               source.Extensions[plugin.Extension] = data
+       }
+
        return source, nil
 }

internal/classifier/extension_plugins.go

package classifier

import (
	"strconv"

	"github.com/go-resty/resty/v2"
)

type servarrContent []struct {
	TmdbId int `json:"tmdbId"`
}

func (l servarrContent) TmdbIds() []string {
	var list []string
	for _, item := range l {
		if item.TmdbId > 0 {
			list = append(list, strconv.Itoa(item.TmdbId))
		}
	}

	return list
}

type extensionPlugin struct {
	url string
}

func (p extensionPlugin) source() (error, []string) {
	client := resty.New()
	var r servarrContent
	_, err := client.R().
		SetResult(&r).
		Get(p.url)
	if err != nil {
		return err, nil
	}

	return nil, r.TmdbIds()
}

Describe alternatives you've considered

No change to bitmagnet, have completely separate utility that generates classifier.yml. This works but is not integrated

Additional context

Feed back wanted - happy to change and submit a PR. Already have a working solution within my build. Concept could potentially work for any plugin list. However concept code in this issue at moment is limited to TMDB Ids sourced from radarr or sonarr.

[mgdigital]

Hi, for clarity, I think a better title for this might be something like "Ability to identify in the classifier movies and TV shows that I've added in Radarr and Sonarr" - please correct me if I'm wrong as I initially struggled to understand what you meant by "extension" in this feature request.

An "extension" in the context of the classifier is intended to be a file extension, for example .mp3 or .mkv. I understand some users have repurposed extensions to be generic string lists . The string_list flag type already exists for this purpose, for example:

flag_definitions:
  my_radarr_movies: string_list
  my_sonarr_series: string_list

I think your feature request can be achieved quite simply with some standalone script that keeps a list of your IDs up-to-date and is provided as a config file to bitmagnet using the EXTRA_CONFIG_FILES environment variable, such a config file could look something like:

classifier:
  flags:
    my_radarr_movies:
      - abc123
      - xyz123
    my_sonarr_series:
      - abc123
      - xyz123

Having done the above, you can then refer to these in your classifier, like result.contentId in flags.my_radarr_movies etc.

A slight inconvenience with this is having to restart the queue worker when the list is updated. Dynamically loading configs is something I also have in mind, e.g. so preferences can be set in the UI without restarting the process, or so this config file can be reloaded.

I feel your requested feature probably doesn't belong in the core. A plugin system could ultimitely avoid the need for an external script. This is definitely on the roadmap but is a substantial piece of work that I can't give an expected timeline for right now.

[rraymondgh]

Hi, for clarity, I think a better title for this might be something like "Ability to identify in the classifier movies and TV shows that I've added in Radarr and Sonarr" - please correct me if I'm wrong as I initially struggled to understand what you meant by "extension" in this feature request.

Agreed - I'm also looking at this from perspective of sources of TMDB IDs that could be used by classifier to categorise torrents. Have code working for radarr, sonarr, TMDB (discover API) and jellyfin

An "extension" in the context of the classifier is intended to be a file extension, for example .mp3 or .mkv. I understand some users have repurposed extensions to be generic string lists .

I fall into that category :-) Plus have used flag definitions. Will update to consistently use single purpose string lists.

I feel your requested feature probably doesn't belong in the core. A plugin system could ultimitely avoid the need for an external script. This is definitely on the roadmap but is a substantial piece of work that I can't give an expected timeline for right now.

That's my view too - would also be good to provide these capabilities to wider user community. I know the core of any software product needs to be kept within functional boundaries so that future change does not get compromised or slowed down. For now I'll use classifier config loading as an integration point and in a fork and keep PR in mergeable state.