v0.9.0

• The main improvement for v0.9.0 is ordering of results in the web UI and GraphQL API
• An --apisDisabled flag was added to the reprocess command to allow significantly faster reprocessing/classifying without any API calls (i.e. TMDB), without disabling this globally.
• Optimisations were made to the web UI that significantly improve responsiveness; the default pagination limit has therefore been increased to 20
• Reprocessing is now done fully within the queue (so you'll no longer see a progress bar when running bitmagnet reprocess, this is handled by the new process_torrent_batch task)

IMPORTANT: Some refactoring of the database was needed to allow for performant ordering. On first run, a database migration will be applied that should take no longer than a few minutes, and tasks will be added to the queue to reindex all torrents. The reindex tasks must complete before ordering will work correctly; if you have millions of torrents in your index this may take a day or more, but is significantly faster in this release than prior versions. Please ensure you have no pending items in your queue (at the /metrics endpoint) before opening any issues about incorrect ordering.

What's Changed

• Use fontsource packages for fonts to ensure deterministic build by @mgdigital in #257
• More Filters against cp. by @Ewoodss in #254
• Ordering by @mgdigital in #260
• Bump rexml from 3.2.6 to 3.2.8 in /bitmagnet.io by @dependabot in #259
• Split gen task to individual tasks by @mgdigital in #261
• Parallel search strategies by @mgdigital in #264
• Classifier runtime flags by @mgdigital in #265
• Bump nokogiri from 1.16.4 to 1.16.5 in /bitmagnet.io by @dependabot in #256
• ENH: Classification result protobuf includes content id and source by @rraymondgh in #266
• Add batch reprocessing queue job and 0.9.0 upgrade hook by @mgdigital in #267
• Increase slow threshold by @mgdigital in #268
• Don't track imported info hashes by @mgdigital in #269
• Add sponsor links to web UI by @mgdigital in #270
• Website updates by @mgdigital in #271
• Tweak build flags by @mgdigital in #272

New Contributors

• @Ewoodss made their first contribution in #254
• @rraymondgh made their first contribution in #266

Full Changelog: v0.8.0...v0.9.0

v0.8.0

Classifier reimplementation

The classifier has been re-implemented and now uses a DSL allowing for full customisation. Several bugs have also been fixed, and a preparatory database migration for search ordering has been added.

The classifier documentation can be found at https://bitmagnet.io/tutorials/classifier.html.

Build reimplementation

The build has been re-implemented using https://github.com/goreleaser/goreleaser, thanks to @KyleSanderson. A number of new platform-specific build artifacts are now available.

Seeders/Leechers bug

There has somehow been a bug present, since day 1, whereby seeders and leechers were being swapped by the DHT crawler! This is fixed in 0.8.x. The first beta included a migration that would fix existing data, but the query just takes too long (for larger indexes), and so it has been removed from the migration.

Until now, no-one noticed - but If you care and would like to fix this in your index, you can run the following query, replacing the time with the time at which you did the 0.8.x upgrade. Be warned, this query may take a while for larger indexes! In time, much of the data will naturally fix itself anyway as the S/L numbers get re-scraped by the crawler when the torrent is re-discovered.

update torrents_torrent_sources set seeders = leechers, leechers = seeders where source = 'dht' and  updated_at < '2024-04-30 00:00:00+00';

What's Changed

• Create FUNDING.yml by @mgdigital in #219
• Fix angular cli version by @mgdigital in #220
• Torrent search: Fix for Safari on iOS. by @josteink in #235
• Classifier rewrite by @mgdigital in #213
• Fix slow migration of published_at field by @mgdigital in #240
• Fix slow migration by @mgdigital in #241
• Fix slow migration by @mgdigital in #242
• Keep a default placeholder value for published_at to avoid a breaking change by @mgdigital in #243
• Add an order by updated_at column, as there may be many records with the same published_at by @mgdigital in #244
• Add orphans option to reprocess command by @mgdigital in #245
• Use budgeted count for reprocess by @mgdigital in #246
• Bump golang.org/x/net from 0.20.0 to 0.23.0 by @dependabot in #239
• Bump express from 4.18.2 to 4.19.2 in /webui by @dependabot in #236
• Bump webpack-dev-middleware from 5.3.3 to 5.3.4 in /webui by @dependabot in #231
• Bump follow-redirects from 1.15.5 to 1.15.6 in /webui by @dependabot in #221
• Bump github.com/jackc/pgx/v5 from 5.5.2 to 5.5.4 by @dependabot in #218
• Add Nix shell and refactor actions by @mgdigital in #249
• Use goreleaser for build process (#212) by @mgdigital in #216
• Upgrade dependencies by @mgdigital in #250
• Don't run GHCR action on main by @mgdigital in #251
• Fix docsite plugin by @mgdigital in #252

New Contributors

• @josteink made their first contribution in #235
• @KyleSanderson made their first contribution in #212

Full Changelog: v0.7.14...v0.8.0

v0.8.0-beta.7

What's Changed

• Create FUNDING.yml by @mgdigital in #219
• Fix angular cli version by @mgdigital in #220
• Torrent search: Fix for Safari on iOS. by @josteink in #235
• Classifier rewrite by @mgdigital in #213
• Fix slow migration of published_at field by @mgdigital in #240
• Fix slow migration by @mgdigital in #241
• Fix slow migration by @mgdigital in #242
• Keep a default placeholder value for published_at to avoid a breaking change by @mgdigital in #243
• Add an order by updated_at column, as there may be many records with the same published_at by @mgdigital in #244
• Add orphans option to reprocess command by @mgdigital in #245
• Use budgeted count for reprocess by @mgdigital in #246
• Bump golang.org/x/net from 0.20.0 to 0.23.0 by @dependabot in #239
• Bump express from 4.18.2 to 4.19.2 in /webui by @dependabot in #236
• Bump webpack-dev-middleware from 5.3.3 to 5.3.4 in /webui by @dependabot in #231
• Bump follow-redirects from 1.15.5 to 1.15.6 in /webui by @dependabot in #221
• Bump github.com/jackc/pgx/v5 from 5.5.2 to 5.5.4 by @dependabot in #218
• Add Nix shell and refactor actions by @mgdigital in #249
• Use goreleaser for build process (#212) by @mgdigital in #216
• Upgrade dependencies by @mgdigital in #250

New Contributors

• @josteink made their first contribution in #235

Full Changelog: v0.7.14...v0.8.0-beta.7

v0.8.0-beta.6

What's Changed

• Revamp build process by @KyleSanderson in #212

New Contributors

• @KyleSanderson made their first contribution in #212

Full Changelog: v0.7.14...v0.8.0-beta.6

v0.8.0-beta.5

What's Changed

• Revamp build process by @KyleSanderson in #212

New Contributors

• @KyleSanderson made their first contribution in #212

Full Changelog: v0.7.14...v0.8.0-beta.5

v0.8.0-beta.4

The classifier has been re-implemented and now uses a DSL allowing for full customisation. Several bugs have also been fixed.

During the beta phase, the classifier documentation can be found at https://github.com/bitmagnet-io/bitmagnet/blob/main/bitmagnet.io/tutorials/classifier.md.

Beta 4 aims to reduce the migration time vs beta 3. Migration now takes just a few minutes for larger indexes. If you've already started an earlier migration, either kill it or wait for it. As it runs in a transaction there's no risk of data loss

Confession time

There has somehow been a bug present, since day 1, whereby seeders and leechers were being swapped by the DHT crawler! This is fixed in 0.8.x. The first beta included a migration that would fix existing data, but the query just takes too long (for larger indexes), and so it has been removed from the migration.

Until now, no-one noticed - but If you care and would like to fix this in your index, you can run the following query, replacing the time with the time at which you did the 0.8.x upgrade. Be warned, this query may take a while for larger indexes! In time, much of the data will naturally fix itself anyway as the S/L numbers get re-scraped by the crawler when the torrent is re-discovered.

update torrents_torrent_sources set seeders = leechers, leechers = seeders where source = 'dht' and  updated_at < '2024-04-22 00:00:00+00';

What's Changed

• Create FUNDING.yml by @mgdigital in #219
• Fix angular cli version by @mgdigital in #220
• Torrent search: Fix for Safari on iOS. by @josteink in #235
• Classifier rewrite by @mgdigital in #213
• Fix slow migration of published_at field by @mgdigital in #240
• Fix slow migration by @mgdigital in #241
• Fix slow migration by @mgdigital in #242

New Contributors

• @josteink made their first contribution in #235

Full Changelog: v0.7.14...v0.8.0-beta.4

v0.8.0-beta.3

The classifier has been re-implemented and now uses a DSL allowing for full customisation. Several bugs have also been fixed.

Beta 3 aims to reduce the migration time vs beta 2. If you've already started the beta 2 migration, either kill it or wait for it. As it runs in a transaction there's no risk of data loss

During the beta phase, the classifier documentation can be found at https://github.com/bitmagnet-io/bitmagnet/blob/main/bitmagnet.io/tutorials/classifier.md.

What's Changed

• Create FUNDING.yml by @mgdigital in #219
• Fix angular cli version by @mgdigital in #220
• Torrent search: Fix for Safari on iOS. by @josteink in #235
• Classifier rewrite by @mgdigital in #213
• Fix slow migration of published_at field by @mgdigital in #240
• Fix slow migration by @mgdigital in #241

New Contributors

• @josteink made their first contribution in #235

Full Changelog: v0.7.14...v0.8.0-beta.3

v0.8.0-beta.2

The classifier has been re-implemented and now uses a DSL allowing for full customisation. Several bugs have also been fixed.

Beta 2 aims to reduce the migration time vs beta 1. If you've already started the beta 1 migration, either kill it or wait for it. As it runs in a transaction there's no risk of data loss

During the beta phase, the classifier documentation can be found at https://github.com/bitmagnet-io/bitmagnet/blob/main/bitmagnet.io/tutorials/classifier.md.

What's Changed

• Create FUNDING.yml by @mgdigital in #219
• Fix angular cli version by @mgdigital in #220
• Torrent search: Fix for Safari on iOS. by @josteink in #235
• Classifier rewrite by @mgdigital in #213
• Fix slow migration of published_at field by @mgdigital in #240

New Contributors

• @josteink made their first contribution in #235

Full Changelog: v0.7.14...v0.8.0-beta.2

v0.8.0-beta.1

The classifier has been re-implemented and now uses a DSL allowing for full customisation. Several bugs have also been fixed.

IMPORTANT: This release includes a database migration that may take around 30 minutes to run for larger indexes.

During the beta phase, the classifier documentation can be found at https://github.com/bitmagnet-io/bitmagnet/blob/main/bitmagnet.io/tutorials/classifier.md.

What's Changed

• Create FUNDING.yml by @mgdigital in #219
• Fix angular cli version by @mgdigital in #220
• Torrent search: Fix for Safari on iOS. by @josteink in #235
• Classifier rewrite by @mgdigital in #213

New Contributors

• @josteink made their first contribution in #235

Full Changelog: v0.7.14...v0.8.0-beta.1

v0.7.14

What's Changed

• Fix missing table for condition by @mgdigital in #215
• Windows fixes: Implement Windows Socket interface and recover from socket read error by @mgdigital in #203

Full Changelog: v0.7.12...v0.7.14