Merge pull request #572 from arabcoders/dev

Dev

Author: arabcoders

.github/workflows/build.yml

@@ -37,7 +37,7 @@ jobs:
     strategy:
       fail-fast: true
       matrix:
-        php: [ 8.3 ]
+        php: [ 8.4 ]
     steps:
       - name: Checkout
         uses: actions/checkout@v4

.gitignore

@@ -3,3 +3,4 @@
 /.env
 /.phpunit.result.cache
 /.vscode
+/frontend/exported/

Dockerfile

@@ -11,7 +11,7 @@ COPY --from=composer:2 /usr/bin/composer /opt/bin/composer
 LABEL maintainer="[email protected]"
 
 ARG TZ=UTC
-ARG PHP_V=php83
+ARG PHP_V=php84
 ARG PHP_PACKAGES="common ctype curl dom fileinfo fpm intl mbstring opcache pcntl pdo_sqlite phar posix session shmop simplexml snmp sockets sodium sysvmsg sysvsem sysvshm tokenizer xml openssl xmlreader xmlwriter zip pecl-igbinary pecl-xhprof pecl-redis"
 ARG TOOL_PATH=/opt/app
 ARG USER_ID=1000

FAQ.md

@@ -516,7 +516,7 @@ Click `Save Changes`
 >
 > If you use multiple plex servers and use the same PlexPass account for all of them, You have to add each backend
 > using the same method above, while enabling `limit webhook events to` `selected user` and `backend unique id`.
-> Essentially, this method replaced the old unified webhook.token for backends.
+> Essentially, this method replaced the old unified webhook token for backends.
 
 -----
 
@@ -625,45 +625,37 @@ https://watchstate.example.org {
 
 ### WS_API_AUTO
 
-The purpose of this environment variable is to automate the configuration process. It's mainly used for people who uses
-many browsers
-to access the `WebUI` and want to automate the configuration process. as it's requires the API settings to be configured
-before it
-can be used. This environment variable can be enabled by setting `WS_API_AUTO=true` in `${WS_DATA_PATH}/config/.env`.
+The purpose of this environment variable is to automate the configuration process. It's mainly used for people who use
+many browsers to access the `WebUI` and want to automate the configuration process. as it's requires the API settings to
+be configured before it can be used. This environment variable can be enabled by setting `WS_API_AUTO=true` in
+`${WS_DATA_PATH}/config/.env`.
 
 #### Why you should use it?
 
 You normally should not use it, as it's a **GREAT SECURITY RISK**. However, if you are using the tool in a secure
-environment
-and not worried about exposing your API key, you can use it to automate the configuration process.
+environment and not worried about exposing your API key, you can use it to automate the configuration process.
 
 #### Why you should not use it?
 
 Because, by exposing your API key, you are also exposing every data you have in the tool. This is a **GREAT SECURITY
-RISK**,
-any person or bot that are able to access the `WebUI` will also be able to visit `/v1/api/system/auto` and get your API
-key. And with this key
-they can do anything they want with your data. including viewing your media servers api keys.
-
-So, please while we have this option available, we strongly recommend not to use it if `WatchState` is exposed to the
-internet.
+RISK**, any person or bot that are able to access the `WebUI` will also be able to visit `/v1/api/system/auto` and get
+your API key. And with this key they can do anything they want with your data. including viewing your media servers API
+keys. So, please while we have this option available, we strongly recommend not to use it if `WatchState` is exposed to
+the internet.
 
 > [!IMPORTANT]
 > This environment variable is **GREAT SECURITY RISK**, and we strongly recommend not to use it if `WatchState` is
-> exposed to the internet.
-> I cannot stress this enough, please do not use it unless you are in a secure environment.
+> exposed to the internet. I cannot stress this enough, please do not use it unless you are in a secure environment.
 
 ---
 
 ### How to disable the included cache server and use external cache server?
 
 Set this environment variable in your `compose.yaml` file `DISABLE_CACHE` with value of `1`. to use external redis
-server
-you need to alter the value of `WS_CACHE_URL` environment variable. the format for this variable
-is `redis://host:port?password=auth&db=db_num`,
-for example to use redis from another container you could use something
-like `redis://172.23.1.10:6379?password=my_secert_password&db=8`.
-We only support `redis` and API compatible alternative.
+server you need to alter the value of `WS_CACHE_URL` environment variable. the format for this variable is
+`redis://host:port?password=auth&db=db_num`, for example to use redis from another container you could use something
+like `redis://172.23.1.10:6379?password=my_secert_password&db=8`. We only support `redis` and API compatible
+alternative.
 
 Once that done, restart the container.
 
@@ -784,14 +776,14 @@ their server, You can follow these steps.
 
 #### Requirements
 
-* [PHP 8.3](http://https://www.php.net/downloads.php) with both the `CLI` and `fpm` mode.
+* [PHP 8.4](http://https://www.php.net/downloads.php) with both the `CLI` and `fpm` mode.
 * PHP Extensions `pdo`, `pdo-sqlite`, `mbstring`, `json`, `ctype`, `curl`, `redis`, `sodium` and `simplexml`.
 * [Composer](https://getcomposer.org/download/) for dependency management.
 * [Redis-server](https://redis.io/) for caching or a compatible implementation that works
   with [php-redis](https://github.com/phpredis/phpredis).
 * [Caddy](https://caddyserver.com/) for frontend handling. However, you can use whatever you like. As long as it has
   support for fastcgi.
-* [nodejs v20+](https://nodejs.org/en/download/) for `WebUI` compilation.
+* [Node.js v20+](https://nodejs.org/en/download/) for `WebUI` compilation.
 
 #### Installation
 
@@ -903,7 +895,7 @@ and treat them as not found.
 This helps with slow stat calls in network shares, or cloud storage.
 
 Everytime we do a stat call we cache it for 1 hour, so if we have multiple records reporting the same path, we only do
-the stat check once.
+the stat check once. Assuming all your media backends are using same path for the media files.
 
 ---
 
@@ -957,7 +949,8 @@ Note: the tip about adding the group_add came from the user `binarypancakes` in
 ### Advanced: How to extend the GUID parser to support more GUIDs or custom ones?
 
 By going to `More > Custom GUIDs` in the WebUI, you can add custom GUIDs to the parser. We know not all people,
-like using GUI, as such You can extend the parser by creating new file at `/config/config/guid.yaml` with the following content.
+like using GUI, as such You can extend the parser by creating new file at `/config/config/guid.yaml` with the following
+content.
 
 ```yaml
 # (Optional) The version of the guid file. If omitted, it will default to the latest version.
@@ -1018,8 +1011,32 @@ As you can see from the config, it's roughly how we expected it to be. The `guid
 custom GUIDs. the `links` array is where you map from client/backends GUIDs to the custom GUID in `WatchState`.
 
 Everything in this file should be in lower case. If error occurs, the tool will log a warning and ignore the guid,
-By default, we only show `ERROR` levels in log file, You can lower it by setting `WS_LOGGER_FILE_LEVEL` environment variable
+By default, we only show `ERROR` levels in log file, You can lower it by setting `WS_LOGGER_FILE_LEVEL` environment
+variable
 to `WARNING`.
 
-If you added or removed a guid from the `guid.yaml` file, you should run `system:index --force-reindex` command to update the
+If you added or removed a guid from the `guid.yaml` file, you should run `system:index --force-reindex` command to
+update the
 database indexes with the new guids.
+
+---
+
+### Sync watch progress.
+
+In order to sync the watch progress between media backends, you need to enable the following environment variable
+`WS_SYNC_PROGRESS` in `(WebUI) > Env` page or via the cli using the following command:
+
+```bash
+$ docker exec -ti watchstate console system:env -k WS_SYNC_PROGRESS -e true
+```
+
+For best experience, you should enable the `Webhooks` feature for the media backends you want to sync the watch
+progress,
+however, if you are unable to do so, the `Tasks > import` task will also generate progress watch events. However, it's
+not as reliable as the `Webhooks` or as fast. using `Webhooks` is the recommended way and offers the best experience.
+
+To check if there is any watch progress events being registered, You can go to `(WebUI) > More > Events` and check
+`on_progress` events, if you are seeing those, this means the progress is being synced. Check the `Tasks logs` to see
+the event log.
+
+

NEWS.md

@@ -1,5 +1,16 @@
 # Old Updates
 
+### 2024-09-14
+
+We have recently added support for extending WatchState with more GUIDs, as of now, the support for it is done via
+editing a`/config/guid.yaml` file in the config directory. We plan to hopefully add management via WebUI in near the
+future. For more information please check out the associated
+FAQ entry about it at [this link](FAQ.md#advanced-how-to-extend-the-guid-parser-to-support-more-guids-or-custom-ones).
+
+The mapping should work for all officially supported clients. If you have a client that is not supported, you have to
+manually add support for that client,
+or request the maintainer to add support for it.
+
 ### 2024-08-19
 
 We have migrated the `state:push` task into the new events system, as such the old task `state:push` is now gone.

README.md

@@ -9,22 +9,22 @@ out of the box, this tool support `Jellyfin`, `Plex` and `Emby` media servers.
 
 ## Updates
 
-### 2024-10-07
+### 2024-12-30
 
-We have added a WebUI page for Custom GUIDs and stabilized on `v1.0` for the `guid.yaml` file spec. We strongly recommend
-to use the `WebUI` to manage the GUIDs, as it's much easier to use than editing the `guid.yaml` file directly. and both the
-`WebUI` and `API` have safeguards to prevent you from breaking the parser. For more information please check out the associated
-FAQ entry about it at [this link](FAQ.md#advanced-how-to-extend-the-guid-parser-to-support-more-guids-or-custom-ones).
+We have removed the old environment variables `WS_CRON_PROGRESS` and `WS_CRON_PUSH` in favor of the new ones
+`WS_SYNC_PROGRESS` and `WS_PUSH_ENABLED`. please update your environment variables accordingly. We have also added
+new FAQ entry about watch progress syncing via [this link](FAQ.md#sync-watch-progress).
 
-### 2024-09-14
+### 2024-10-07
 
-We have recently added support for extending WatchState with more GUIDs, as of now, the support for it is done via
-editing a`/config/guid.yaml` file in the config directory. We plan to hopefully add management via WebUI in near the future. For more information please check out the associated
+We have added a WebUI page for Custom GUIDs and stabilized on `v1.0` for the `guid.yaml` file spec. We strongly
+recommend
+to use the `WebUI` to manage the GUIDs, as it's much easier to use than editing the `guid.yaml` file directly. and both
+the
+`WebUI` and `API` have safeguards to prevent you from breaking the parser. For more information please check out the
+associated
 FAQ entry about it at [this link](FAQ.md#advanced-how-to-extend-the-guid-parser-to-support-more-guids-or-custom-ones).
 
-The mapping should work for all officially supported clients. If you have a client that is not supported, you have to manually add support for that client,
-or request the maintainer to add support for it.
-
 --- 
 Refer to [NEWS](NEWS.md) for old updates.
 
@@ -38,7 +38,8 @@ Refer to [NEWS](NEWS.md) for old updates.
 * Search your backend for `title` or `item id`.
 * Display and filter your play state. Can be exported as `yaml` or `json`.
 * Check if your media servers reporting same data via the parity command.
-* Track your watch progress via webhooks.
+* Sync your watch progress via webhooks or scheduled tasks.
+* Check if your media backends have stale references to old files.
 
 ----
 
@@ -115,7 +116,7 @@ You can check the message in the notification itself to know what went wrong. Or
 error has been logged to a file named `app.YYYYMMDD.log`.
 
 If everything went ok, you should see the backend shows up in the same page. You can then go to the Tasks page and click
-on `Qeueu Task`, for first time import we recommand letting
+on `Queue Task`, for first time import we recommend letting
 the task run in the background, as it might take a while to import all the data.
 
 Once you have done all for your backends, You should go back again to `Tasks` page and enable the `Import` and `Export`
@@ -134,7 +135,7 @@ After starting the container you should start adding your backends and to do so
 > [!NOTE]
 > to get your plex token, please
 > visit [this plex page](https://support.plex.tv/articles/204059436-finding-an-authentication-token-x-plex-token/) to
-> know how to extract your plex token. For jellyfin & emby. Go to Dashboard > Advanced > API keys > then create new api
+> know how to extract your plex token. For jellyfin & emby. Go to Dashboard > Advanced > API keys > then create new API
 > keys.
 
 ```bash

bin/console

@@ -51,7 +51,7 @@ if (!file_exists(__DIR__ . '/../vendor/autoload.php')) {
 require __DIR__ . '/../vendor/autoload.php';
 
 try {
-    $app = (new App\Libs\Initializer())->boot();
+    $app = new App\Libs\Initializer()->boot();
 } catch (Throwable $e) {
     $message = strtr(
             'CLI: Exception [{kind}] was thrown unhandled during CLI boot context. Error [{message} @ {file}:{line}].',

composer.json

@@ -14,7 +14,7 @@
         "test": "vendor/bin/phpunit"
     },
     "require": {
-        "php": "^8.3",
+        "php": "^8.4",
         "ext-pdo": "*",
         "ext-pdo_sqlite": "*",
         "ext-mbstring": "*",
@@ -26,34 +26,36 @@
         "ext-redis": "*",
         "ext-posix": "*",
         "ext-openssl": "*",
-        "monolog/monolog": "^3.4",
-        "symfony/console": "^6.1.4",
-        "symfony/cache": "^6.1.3",
-        "symfony/yaml": "^6.1.4",
-        "symfony/process": "^6.1.3",
-        "symfony/http-client": "^6.1.4",
-        "symfony/lock": "^6.1.3",
-        "league/container": "^4.2",
-        "psr/http-client": "^1.0.1",
-        "psr/simple-cache": "3.0 as 1.0",
-        "psr/http-server-middleware": "^1.0.1",
-        "nyholm/psr7": "^1.5.1",
-        "nyholm/psr7-server": "^1.0.2",
-        "dragonmantank/cron-expression": "^3.3.2",
-        "halaxa/json-machine": "^1.1.1",
-        "league/route": "^5.1.2",
-        "psy/psysh": "^0.11.22",
-        "symfony/event-dispatcher": "^6.1.4",
-        "ramsey/uuid": "^4.5.1"
+        "psr/http-client": "^1.0.3",
+        "psr/simple-cache": "^3.0",
+        "psr/http-server-middleware": "^1.0.2",
+        "monolog/monolog": "^3.8.1",
+        "symfony/console": "^7.2.1",
+        "symfony/cache": "^7.2.1",
+        "symfony/yaml": "^7.2",
+        "symfony/process": "^7.2",
+        "symfony/http-client": "^7.2.1",
+        "symfony/lock": "^7.2",
+        "symfony/event-dispatcher": "^7.2",
+        "league/container": "^4.2.4",
+        "league/route": "^6.2.0",
+        "nyholm/psr7": "^1.8.2",
+        "nyholm/psr7-server": "^1.1.0",
+        "dragonmantank/cron-expression": "^3.4.0",
+        "halaxa/json-machine": "^1.2.0",
+        "psy/psysh": "^0.12.7",
+        "ramsey/uuid": "^4.7.6"
     },
     "suggest": {
-        "ext-sockets": "For UDP commincations."
+        "ext-sockets": "For UDP communications."
     },
     "require-dev": {
         "roave/security-advisories": "dev-latest",
-        "symfony/var-dumper": "^6.1.3",
-        "perftools/php-profiler": "^1.1.1",
-        "phpunit/phpunit": "^9.5.24"
+        "symfony/var-dumper": "^7.2",
+        "perftools/php-profiler": "^1.1.2",
+        "phpunit/phpunit": "^11.5.2",
+        "phpstan/phpstan": "^2.0",
+        "psalm/phar": "^5.26"
     },
     "autoload-dev": {
         "psr-4": {
@@ -80,6 +82,8 @@
         "symfony/polyfill-php82": "*",
         "symfony/polyfill-php83": "*",
         "symfony/polyfill-php84": "*",
+        "symfony/polyfill-php85": "*",
+        "symfony/polyfill-php86": "*",
         "symfony/polyfill-ctype": "*",
         "symfony/polyfill-mbstring": "*",
         "symfony/polyfill-intl-normalizer": "*",