Version [4.0.0]

CHANGELOG.md

@@ -7,6 +7,37 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 
 
+## Version [4.0.0] - 2024-08-28 🚀
+
+### Breaking Changes ⚠️
+
+- Riven: Riven backend no longer supports sqlite as a database option; PostgreSQL is now the only supported database option 📉
+- PostgreSQL: To ensure the database files are persisted, a volume must be mounted to /postgres_data 📂
+
+### Added ✨
+
+- PUID & PGID: Environment variables to set the user and group IDs for the DMB container; Default is 1001 🆔
+- POSTGRES_DATA: Environment variable to set the path for the PostgreSQL database files; Default is /postgres_data 📁
+- POSTGRES_PASSWORD: Environment variable to set the password for the PostgreSQL database; Default is postgres 🔐
+- POSTGRES_USER: Environment variable to set the user for the PostgreSQL database; Default is postgres 👤
+- POSTGRES_DB: Environment variable to set the database name for the PostgreSQL database; Default is riven 🗄️
+
+### Changed 🔄
+
+- Riven: Riven backend now uses PostgreSQL as the database option; Default is postgresql+psycopg2://postgres:postgres@127.0.0.1/riven 🔧
+- Riven: Riven frontend now uses PostgreSQL as the database option; Default is postgres://postgres:postgres@127.0.0.1/riven 🔄
+
+### Fixed 🛠️
+
+- [Issue #8](https://github.com/I-am-PUID-0/DMB/issues/8) Add support for PUID/GUID ✨
+- [Issue #34](https://github.com/I-am-PUID-0/DMB/issues/34) Add PostgreSQL option for Riven backend ✨
+- [Issue #35](https://github.com/I-am-PUID-0/DMB/issues/35) Riven frontend not properly connecting to the database 🐛
+
+### Notes 📝📌
+
+- If the Riven backend shows errors related to the database or alembic, then the Riven data directory may need to be deleted before starting the new version of Riven w/ PostgreSQL 🗑️ - backup your settings.json before deleting the data directory 📂
+
+
 ## Version [3.5.0] - 2024-08-08 🚀
 
 ### Added ✨

Dockerfile

@@ -18,7 +18,7 @@ ENV \
   TERM=xterm
 
 RUN \
-  apk add --update --no-cache gcompat libstdc++ libxml2-utils curl tzdata nano ca-certificates wget fuse3 build-base linux-headers py3-cffi libffi-dev rust cargo openssl openssl-dev pkgconfig git npm ffmpeg
+  apk add --update --no-cache gcompat libstdc++ libxml2-utils curl tzdata nano ca-certificates wget fuse3 build-base linux-headers py3-cffi libffi-dev rust cargo openssl openssl-dev pkgconfig git npm ffmpeg postgresql-dev postgresql-client postgresql
 
 RUN \
   mkdir -p /log /riven /riven/frontend && \

README.md

@@ -58,8 +58,12 @@ services:
       - /home/username/docker/DMB/Riven/data:/riven/backend/data
       ## Location for Riven symlinks
       - /home/username/docker/DMB/Riven/mnt:/mnt
+      ## Location for PostgreSQL database if using Riven
+      - /home/username/docker/DMB/PostgreSQL/data:/postgres_data
     environment:
       - TZ=
+      - PUID=
+      - PGID=
       ## Zurg Required Settings
       - ZURG_ENABLED=true      
       - RD_API_KEY=
@@ -85,6 +89,11 @@ services:
      # - RCLONE_VFS_CACHE_MAX_SIZE=100G
      # - RCLONE_BUFFER_SIZE=32M
      # - RCLONE_VFS_CACHE_MAX_AGE=4h
+      ## PostgreSQL Optional Settings
+     # - POSTGRES_DATA=/postgres_data
+     # - POSTGRES_USER=postgres
+     # - POSTGRES_PASSWORD=postgres
+     # - POSTGRES_DB=riven
       ## Riven Backend Required Settings
       - RIVEN_BACKEND_ENABLED=true
       ## Riven Frontend Required Settings
@@ -99,10 +108,10 @@ services:
      # - RIVEN_BACKEND_UPDATE=true
      # - RIVEN_FRONTEND_UPDATE=true
      # - RIVEN_LOG_LEVEL=DEBUG
-     # - RIVEN_BACKEND_URL=http://127.0.0.1:8080 # Default is http://127.0.0.1:8080 when not enabled
-     # - RIVEN_DATABASE_HOST=sqlite:////riven/backend/data/media.db # Default is sqlite:////riven/backend/data/media.db when not enabled
-     # - RIVEN_DATABASE_URL=/riven/backend/data/media.db  # Default is /riven/backend/data/media.db when not enabled
-     # - RIVEN_FRONTEND_DIALECT=sqlite
+     # - RIVEN_BACKEND_URL=
+     # - RIVEN_DATABASE_HOST=
+     # - RIVEN_DATABASE_URL= 
+     # - RIVEN_FRONTEND_DIALECT=
      # - PLEX_TOKEN=
      # - PLEX_ADDRESS=
      # - SEERR_API_KEY=
@@ -174,6 +183,8 @@ of this parameter has the format `<VARIABLE_NAME>=<VALUE>`.
 | Variable       | Description                                  | Default | Used w/ rclone| Used w/ Riven| Used w/ zurg|
 |----------------|----------------------------------------------|---------|:-:|:-:|:-:|
 |`TZ`| [TimeZone](http://en.wikipedia.org/wiki/List_of_tz_database_time_zones) used by the container |  |
+|`PUID`| The user ID of the user running the container | `1001` | :heavy_check_mark:| :heavy_check_mark:| :heavy_check_mark:|
+|`PGID`| The group ID of the user running the container | `1001` | :heavy_check_mark:| :heavy_check_mark:| :heavy_check_mark:|
 |`RD_API_KEY`| [RealDebrid API key](https://real-debrid.com/apitoken) | `none` | | :heavy_check_mark:| :heavy_check_mark:|
 |`AD_API_KEY`| [AllDebrid API key](https://alldebrid.com/apikeys/) | `none` | | :heavy_check_mark:| :heavy_check_mark:|
 |`RCLONE_MOUNT_NAME`| A name for the rclone mount | `none` | :heavy_check_mark:|
@@ -186,6 +197,10 @@ of this parameter has the format `<VARIABLE_NAME>=<VALUE>`.
 |`RCLONE_VFS_CACHE_MAX_AGE`| [Max age of the VFS cache](https://rclone.org/commands/rclone_mount/#vfs-file-caching) |`none`  | :heavy_check_mark:|
 |`PLEX_TOKEN`| The [Plex Token](https://support.plex.tv/articles/204059436-finding-an-authentication-token-x-plex-token/) associated with your Plex user |`none`  || :heavy_check_mark:|
 |`PLEX_ADDRESS`| The URL of your Plex server. Example: http://192.168.0.100:32400 or http://plex:32400 - format must include ```http://``` or ```https://``` and have no trailing characters after the port number (32400). E.g., ```/``` |`none`|| :heavy_check_mark:|
+|`POSTGRES_DATA`| The location of the PostgreSQL database |`/postgres_data`|| :heavy_check_mark:|
+|`POSTGRES_USER`| The username for the PostgreSQL database |`postgres`|| :heavy_check_mark:|
+|`POSTGRES_PASSWORD`| The password for the PostgreSQL database |`postgres`|| :heavy_check_mark:|
+|`POSTGRES_DB`| The name of the PostgreSQL database |`riven`|| :heavy_check_mark:|
 |`RIVEN_ENABLED`| Set the value "true" to enable the Riven backend and frontend processes | `false ` | | :heavy_check_mark: | |
 |`RIVEN_BACKEND_ENABLED`| Set the value "true" to enable the Riven backend process | `false ` | | :heavy_check_mark: | |
 |`RIVEN_FRONTEND_ENABLED`| Set the value "true" to enable the Riven frontend process | `false ` | | :heavy_check_mark: | |
@@ -198,8 +213,8 @@ of this parameter has the format `<VARIABLE_NAME>=<VALUE>`.
 |`RIVEN_FRONTEND_UPDATE`| Enable automatic updates of the Riven frontend. Adding this variable will enable automatic updates to the latest version of Riven locally within the container.| `false` || :heavy_check_mark:|
 |`ORIGIN`| The origin URL for the Riven frontend | http://0.0.0.0:3000 | | :heavy_check_mark: | |
 |`RIVEN_BACKEND_URL`| The URL for the Riven backend | http://127.0.0.1:8080 | | :heavy_check_mark: | |
-|`RIVEN_DATABASE_HOST`| The database host for Riven backend | `sqlite:////riven/backend/data/media.db` | | :heavy_check_mark: | |
-|`RIVEN_DATABASE_URL`| The database URL for Riven frontend | `/riven/backend/data/media.db` | | :heavy_check_mark: | |
+|`RIVEN_DATABASE_HOST`| The database host for Riven backend | `postgresql+psycopg2://postgres:[email protected]/riven` | | :heavy_check_mark: | |
+|`RIVEN_DATABASE_URL`| The database URL for Riven frontend | `postgres://postgres:[email protected]/riven` | | :heavy_check_mark: | |
 |`RIVEN_FRONTEND_DIALECT`| The dialect for the Riven frontend | `sqlite` | | :heavy_check_mark: | |
 |`AUTO_UPDATE_INTERVAL`| Interval between automatic update checks in hours. Vaules can be any positive [whole](https://www.oxfordlearnersdictionaries.com/us/definition/english/whole-number) or [decimal](https://www.oxfordreference.com/display/10.1093/oi/authority.20110803095705740;jsessionid=3FDC96CC0D79CCE69702661D025B9E9B#:~:text=The%20separator%20used%20between%20the,number%20expressed%20in%20decimal%20representation.) point based number. Ex. a value of .5 would yield thirty minutes, and 1.5 would yield one and a half hours | `24` || :heavy_check_mark:| :heavy_check_mark:|
 |`DUPLICATE_CLEANUP`| Automated cleanup of duplicate content in Plex.  | `false` |
@@ -233,11 +248,12 @@ format: `<HOST_DIR>:<CONTAINER_DIR>[:PERMISSIONS]`.
 |-----------------|-------------|-------------|
 |`/config`| rw | This is where the application stores the rclone.conf, and any files needing persistence. CAUTION: rclone.conf is overwritten upon start/restart of the container. Do NOT use an existing rclone.conf file if you have other rclone services |
 |`/log`| rw | This is where the application stores its log files |
-|`/data`| shared  | This is where rclone will be mounted. Not required when only utilizing Riven   |
+|`/data`| shared  | This is where rclone will be mounted.|
 |`/zurg/RD`| rw| This is where Zurg will store the active configuration and data for RealDebrid. Not required when only utilizing Riven   |
 |`/zurg/AD`| rw | This is where Zurg will store the active configuration and data for AllDebrid. Not required when only utilizing Riven   |
 |`/riven/data`| rw | This is where Riven will store its data. Not required when only utilizing Zurg   |
 |`/riven/mnt`| rw | This is where Riven will set its symlinks. Not required when only utilizing Zurg   |
+|`/postgres_data`| rw | This is where PostgreSQL will store its data. Not required when only utilizing Zurg   |
 
 
 

base/__init__.py

@@ -7,6 +7,8 @@
 from packaging.version import Version, parse as parse_version
 import time
 import os
+import pwd
+import grp
 import ast
 import requests
 import zipfile
@@ -76,4 +78,10 @@ def load_secret_or_env(secret_name, default=None):
 RFUPDATE = os.getenv('RIVEN_FRONTEND_UPDATE')
 RFDIALECT = os.getenv('RIVEN_FRONTEND_DIALECT')
 SYMLINKLIBRARYPATH = os.getenv('SYMLINK_LIBRARY_PATH')
-SYMLINKRCLONEPATH = os.getenv('SYMLINK_RCLONE_PATH')
+SYMLINKRCLONEPATH = os.getenv('SYMLINK_RCLONE_PATH')
+postgres_system_user = "DMB"
+postgres_data = os.getenv('POSTGRES_DATA', '/postgres_data')
+postgres_user = os.getenv('POSTGRES_USER', 'postgres')
+postgres_password = os.getenv('POSTGRES_PASSWORD', 'postgres')
+postgres_db = os.getenv('POSTGRES_DB', 'riven')
+db_host = '127.0.0.1'  

docker-compose.yml

@@ -3,9 +3,9 @@ version: "3.8"
 services:
   DMB:
     container_name: DMB
-    image: iampuid0/DMB:latest
+    image: iampuid0/dmb:latest
     ## Optionally, specify a specific version of DMB
-    # image: iampuid0/DMB:2.0.0 #etc...
+    # image: iampuid0/dmb:2.0.0 #etc...
     stdin_open: true # docker run -i
     tty: true        # docker run -t    
     volumes:
@@ -23,8 +23,12 @@ services:
       - /home/username/docker/DMB/Riven/data:/riven/backend/data
       ## Location for Riven symlinks
       - /home/username/docker/DMB/Riven/mnt:/mnt
+      ## Location for PostgreSQL database if using Riven
+      - /home/username/docker/DMB/PostgreSQL/data:/postgres_data
     environment:
       - TZ=
+      - PUID=
+      - PGID=
       ## Zurg Required Settings
       - ZURG_ENABLED=true      
       - RD_API_KEY=
@@ -50,6 +54,11 @@ services:
      # - RCLONE_VFS_CACHE_MAX_SIZE=100G
      # - RCLONE_BUFFER_SIZE=32M
      # - RCLONE_VFS_CACHE_MAX_AGE=4h
+      ## PostgreSQL Optional Settings
+     # - POSTGRES_DATA=/postgres_data
+     # - POSTGRES_USER=postgres
+     # - POSTGRES_PASSWORD=postgres
+     # - POSTGRES_DB=riven
       ## Riven Backend Required Settings
       - RIVEN_BACKEND_ENABLED=true
       ## Riven Frontend Required Settings
@@ -64,10 +73,10 @@ services:
      # - RIVEN_BACKEND_UPDATE=true
      # - RIVEN_FRONTEND_UPDATE=true
      # - RIVEN_LOG_LEVEL=DEBUG
-     # - RIVEN_BACKEND_URL=http://127.0.0.1:8080 # Default is http://127.0.0.1:8080 when not enabled
-     # - RIVEN_DATABASE_HOST=sqlite:////riven/backend/data/media.db # Default is sqlite:////riven/backend/data/media.db when not enabled
-     # - RIVEN_DATABASE_URL=/riven/backend/data/media.db  # Default is /riven/backend/data/media.db when not enabled
-     # - RIVEN_FRONTEND_DIALECT=sqlite
+     # - RIVEN_BACKEND_URL=
+     # - RIVEN_DATABASE_HOST=
+     # - RIVEN_DATABASE_URL= 
+     # - RIVEN_FRONTEND_DIALECT=
      # - PLEX_TOKEN=
      # - PLEX_ADDRESS=
      # - SEERR_API_KEY=

main.py

@@ -3,8 +3,7 @@
 import riven_ as r
 import zurg as z 
 from rclone import rclone
-from utils import duplicate_cleanup
-
+from utils import duplicate_cleanup, user_management, postgres
 
 def shutdown(signum, frame):
     logger = get_logger()
@@ -25,7 +24,7 @@ def shutdown(signum, frame):
 def main():
     logger = get_logger()
 
-    version = '3.5.0'
+    version = '4.0.0'
 
     ascii_art = f'''
                                                                        
@@ -66,7 +65,9 @@ def healthcheck():
     thread = threading.Thread(target=healthcheck)
     thread.daemon = True
     thread.start()
-
+
+    user_management.create_system_user()
+
     try:
         if ZURG is None or str(ZURG).lower() == 'false':
             pass
@@ -104,6 +105,7 @@ def healthcheck():
     try:
         if (RIVENBACKEND is not None and str(RIVENBACKEND).lower() == 'true') or (RIVEN is not None and str(RIVEN).lower() == 'true'):
             try:
+                postgres.postgres_setup()
                 r.setup.riven_setup('riven_backend')
                 r_updater = r.update.RivenUpdate()
                 if (RBUPDATE or RUPDATE) and not RBVERSION:

rclone/rclone.py

@@ -112,7 +112,8 @@ def setup():
             logger.info(f"Configuring rclone for {mn}")
             subprocess.run(["umount", f"/data/{mn}"], check=False, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
             os.makedirs(f"/data/{mn}", exist_ok=True)
-
+            user_id = int(os.getenv('PUID', 1001))
+            group_id = int(os.getenv('PGID', 1001))
             if NFSMOUNT is not None and NFSMOUNT.lower() == "true":
                 if NFSPORT:
                     port = NFSPORT
@@ -123,7 +124,7 @@ def setup():
                     logger.info(f"Setting up rclone NFS mount server for {mn} at 0.0.0.0:{port}")
                     rclone_command = ["rclone", "serve", "nfs", f"{mn}:", "--config", "/config/rclone.config", "--addr", f"0.0.0.0:{port}", "--vfs-cache-mode=full", "--dir-cache-time=10"]
             else:
-                rclone_command = ["rclone", "mount", f"{mn}:", f"/data/{mn}", "--config", "/config/rclone.config", "--allow-other", "--poll-interval=0", "--dir-cache-time=10"]
+                rclone_command = ["rclone", "mount", f"{mn}:", f"/data/{mn}", "--config", "/config/rclone.config", f"--uid={user_id}" , f"--gid={group_id}" , "--allow-other", "--poll-interval=0", "--dir-cache-time=10"]
             if not RIVENBACKEND or idx != len(mount_names) - 1:
                 rclone_command.append("--daemon")
 

requirements.txt

@@ -7,4 +7,5 @@ requests
 packaging
 ruamel.yaml
 poetry
-colorlog
+colorlog
+psycopg2-binary

riven_/settings.py

@@ -1,5 +1,8 @@
 from base import *
 from utils.logger import *
+import psycopg2
+from psycopg2 import sql
+from utils.processes import ProcessHandler
 
 
 logger = get_logger()
@@ -47,19 +50,18 @@ def set_env_variable(key, value, default=None):
         'SYMLINK_RCLONE_PATH': SYMLINKRCLONEPATH,
         'SYMLINK_LIBRARY_PATH': SYMLINKLIBRARYPATH,
         'BACKEND_URL': RIVENBACKENDURL,
-        'DIALECT' : RFDIALECT,
+        'DIALECT': RFDIALECT,
         'DATABASE_URL': RIVENDATABASEURL,
         'DATABASE_HOST': RIVENDATABASEHOST
-
     }
 
     default_env_vars = {
         'SYMLINK_RCLONE_PATH': f'/data/{RCLONEMN}/__all__',
         'SYMLINK_LIBRARY_PATH': '/mnt',
-        'DATABASE_HOST': 'sqlite:////riven/backend/data/media.db',
-        'DATABASE_URL': '/riven/backend/data/media.db',
+        'DATABASE_HOST': f'postgresql+psycopg2://{postgres_user}:{postgres_password}@{db_host}/{postgres_db}',
+        'DATABASE_URL': f'postgres://{postgres_user}:{postgres_password}@{db_host}/{postgres_db}',
         'BACKEND_URL': 'http://127.0.0.1:8080',
-        'DIALECT' : 'sqlite'
+        'DIALECT': 'postgres'
     }
 
     for key, value in env_vars.items():