Add timeout and ssl verify setup options & simplify requests command handler

Author: kennymc-c

CHANGELOG.md

@@ -7,15 +7,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Unreleased
 
+## [0.2-beta] - 2024-06-26
+
+### Breaking changes
+- This integration now uses a configuration file to store the setup state and advanced settings (see below). Existing users therefore need to run the integration setup again to create this file. Just open the integration settings and click on "Start integration setup". You don't need to enter the advanced settings.
+
 ### Added
 
 - Added a more granular http status code response handling
 - Added optional parameter to send form data in the request body as key/value pairs (see README)
+- Added optional custom global entity-independent timeout and ssl verify option in the integration setup. For self signed ssl certificates to work the ssl verify option needs to be deactivated.
 
 ### Changed
 - Only return an error response to the remote if the http response code is in the 400 or 500 range. Otherwise display the status code in the integration log if it's not 200/Ok
 
-## [0.1-beta] - 2024-03-27
+## [0.1-beta] - 2024-04-27
 
 ### Added
 

README.md

@@ -1,43 +1,46 @@
-# HTTP Get/Post/Put/Patch & Wake on LAN Integration for Unfolded Circle Remote Two
+# HTTP Get/Post/Put/Patch & Wake on LAN Integration for Unfolded Circle Remote Two and Remote 3
 
 ## ⚠️ WARNING ⚠️
 
 ### Disclaimer: This software is at an early stage of development and may contain serious bugs that could affect system stability. Please use it at your own risk!
 
 ##
 
-Integration for [Unfolded Circle Remote Two](https://unfoldedcircle.com) to send network requests to a specified url or mac address.
+Integration for Unfolded Circle [Remote Two](https://www.unfoldedcircle.com/remote-two) and [Remote 3](https://www.unfoldedcircle.com) running [Unfolded OS](https://www.unfoldedcircle.com/unfolded-os) to send network requests to a specified url or mac address.
 
 Using [uc-integration-api](https://github.com/aitatoi/integration-python-library), [requests](https://github.com/psf/requests) and [pywakeonlan](https://github.com/remcohaszing/pywakeonlan).
 
 
 ### Supported features
 
 - Send http(s) get, post, patch & put requests to a specified url
+  - Set a custom timeout and deactivate ssl certificate verification
 - Send Wake on LAN magic packets to a specified mac address
 
 
 ### Planned features
 
 - Send Wake on LAN magic packets by entering the ip address
 - Support for sending json and xml data in request body
-- Configurable timeout in integration setup
-- Support for self signed SSL certificates / deactivate SSL verification in integration setup
 
 *Planned improvements are labeled with #TODO in the code*
 
 
+## Configuration
+
+During the integration setup you can change the default http request timeout of 2 seconds to a custom value. You also can deactivate the ssl certificate verification. This is needed for self signed certificates. You can run the setup process again in the integration settings after adding entities to the remote.
+
 ## Usage
 
-The integration exposes a media player entity for each supported request command. These entities only support the source feature. Just enter the desired url (including http(s)://) or mac address in the source field when you configure your activity/macro sequences or activity ui. The default timeout is 2 seconds.
+The integration exposes a media player entity for each supported request command. These entities only support the source feature. Just enter the desired url (including http(s)://) or mac address in the source field when you configure your activity/macro sequences or activity ui.
 <br>
-For http requests your server needs to respond with a 200 OK or any other informational or redirection http status codes (100s, 200s or 300s). In case of a client or server error (400s or 500s) the command will fail on the remote and the error and status code will be shown in the log of the integration.
+For http requests your server needs to respond with a *200 OK* status or any other informational or redirection http status codes (100s, 200s or 300s). In case of a client or server error (400s or 500s) the command will fail on the remote and the error message and status code will be shown in the integration log.
 <br>
 <br>
-Optional form data in the request body as key/value pairs can be added with a paragraph as a separator like this:
+Optional form data in the request body as key/value pairs can be added with a paragraph character (§) as a separator like this:
 - https://httpbin.org/post§key1=value1,key2=value2
 
-Note that if your url contains a paragraph you need to url-encode it first (%C2%A7, see https://www.urlencoder.io)
+Note that if your url contains a paragraph character you need to url-encode it first (%C2%A7, see https://www.urlencoder.io)
 
 
 ### Setup

intg-requests/config.json

@@ -0,0 +1 @@
+{"rq_timeout": 5, "rq_ssl_verify": true, "setup_complete": true}

intg-requests/config.py

@@ -1,15 +1,19 @@
+import json
+import os
 import logging
 
 _LOG = logging.getLogger(__name__)
 
+CFG_FILENAME = "config.json"
 
 
 class setup:
-    cmds = ["get", "post", "patch", "put", "wol"]
-
     __conf = {
     "standby": False,
-    "rq-timeout": 2,
+    "setup_complete": False,
+    "setup_reconfigure": False,
+    "rq_timeout": 2,
+    "rq_ssl_verify": True,
     "id-get": "http-get",
     "name-get": "HTTP Get",
     "id-post": "http-post",
@@ -21,19 +25,95 @@ class setup:
     "id-wol": "wol",
     "name-wol": "Wake on LAN"
     }
-    __setters = ["standby"]
-
+    __setters = ["standby", "setup_complete", "setup_reconfigure", "rq_timeout", "rq_ssl_verify"]
+    __storers = ["setup_complete", "rq_timeout", "rq_ssl_verify"] #Skip runtime only related values in config file
+    all_cmds = ["get", "post", "patch", "put", "wol"]
+    rq_ids = [__conf["id-get"], __conf["id-post"], __conf["id-patch"], __conf["id-put"]]
+    rq_names = [__conf["name-get"], __conf["name-post"], __conf["name-patch"], __conf["name-put"]]
+    
     @staticmethod
-    def get(value):
-        if setup.__conf[value] != "":
-            return setup.__conf[value]
+    def get(key):
+        if setup.__conf[key] != "":
+            return setup.__conf[key]
         else:
-            _LOG.error("Got empty value from config storage")
+            _LOG.error("Got empty value for " + key + " from config storage")
 
     @staticmethod
     def set(key, value):
+
         if key in setup.__setters:
-            setup.__conf[key] = value
-            _LOG.debug("Stored " + key + ": " + str(value) + " into runtime storage")
+            if setup.__conf["setup_reconfigure"] == True and key == "setup_complete":
+                _LOG.debug("Ignore setting and storing setup_complete flag during reconfiguration")
+            else:
+                setup.__conf[key] = value
+                _LOG.debug("Stored " + key + ": " + str(value) + " into runtime storage")
+
+                #Store key/value pair in config file
+                if key in setup.__storers:
+                        
+                    jsondata = {key: value}
+                    if os.path.isfile(CFG_FILENAME):
+                            try:
+                                with open(CFG_FILENAME, "r+") as f:
+                                    l = json.load(f)
+                                    l.update(jsondata)
+                                    f.seek(0)
+                                    f.truncate() #Needed when the new value has less characters than the old value (e.g. false to true or 11 to 5 seconds timeout)
+                                    json.dump(l, f)
+                                    f.close
+                                    _LOG.debug("Stored " + key + ": " + str(value) + " into " + CFG_FILENAME)
+                            except OSError as o:
+                                raise Exception(o)
+                            except:
+                                raise Exception("Error while storing " + key + ": " + str(value) + " into " + CFG_FILENAME)
+
+                    #Create config file first if it doesn't exists yet
+                    else:
+                        try:
+                            with open(CFG_FILENAME, "w") as f:
+                                json.dump(jsondata, f)
+                                f.close
+                            _LOG.debug("Created " + CFG_FILENAME + " and stored " + key + ": " + str(value) + " in it")
+                        except OSError as o:
+                            raise Exception(o)
+                        except:
+                            raise Exception("Error while storing " + key + ": " + str(value) + " into " + CFG_FILENAME)
+                        
+                else:
+                    _LOG.debug(key + " not found in __storers because it should not be stored in the config file")
+
+        else:
+            raise NameError(key + " not found in __setters because it should not be changed")
+        
+    @staticmethod
+    def load():
+        if os.path.isfile(CFG_FILENAME):
+
+            try:
+                with open(CFG_FILENAME, "r") as f:
+                    configfile = json.load(f)
+            except:
+                raise OSError("Error while reading " + CFG_FILENAME)
+            if configfile == "":
+                raise OSError("Error in " + CFG_FILENAME + ". No data")
+
+            setup.__conf["setup_complete"] = configfile["setup_complete"]
+            _LOG.debug("Loaded setup_complete: " + str(configfile["setup_complete"]) + " into runtime storage from " + CFG_FILENAME)
+
+            if not setup.__conf["setup_complete"]:
+                _LOG.warning("The setup was not completed the last time. Please restart the setup process")
+            else:
+                if "rq_timeout" in configfile:
+                    setup.__conf["rq_timeout"] = configfile["rq_timeout"]
+                    _LOG.info("Loaded requests timeout of " + str(configfile["rq_timeout"]) + " seconds into runtime storage from " + CFG_FILENAME)
+                else:
+                    _LOG.info("Skip loading custom requests timeout as it has not been changed during setup. Default value of " + str(setup.get("rq_timeout")) + " seconds will be used")
+
+                if "rq_ssl_verify" in configfile:
+                    setup.__conf["rq_ssl_verify"] = configfile["rq_ssl_verify"]
+                    _LOG.info("Loaded http ssl verification: " + str(configfile["rq_ssl_verify"]) + " into runtime storage from " + CFG_FILENAME)
+                else:
+                    _LOG.info("Skip loading http ssl verification flag as it has not been changed during setup. Default value " + str(setup.get("rq_ssl_verify")) + " will be used")
+
         else:
-            raise NameError("Name not accepted in set() method")
+            _LOG.info(CFG_FILENAME + " does not exist (yet). Please start the setup process")

intg-requests/driver.py

@@ -9,6 +9,7 @@
 
 import config
 import media_player
+import setup
 
 _LOG = logging.getLogger("driver")  # avoid having __main__ in log messages
 
@@ -19,23 +20,22 @@
 
 async def startcheck():
     """
-    Called at the start of the integration driver to add a media player entity for all configured cmds list entries in config.py
+    Called at the start of the integration driver to load the config file into the runtime storage and add a media player entity for all configured cmds
     """
+    try:
+        config.setup.load()
+    except OSError as o:
+        _LOG.critical(o)
+        _LOG.critical("Stopping integration driver")
+        raise SystemExit(0)
 
-    for cmd in config.setup.cmds:
-        id = config.setup.get("id-"+cmd)
-        name = config.setup.get("name-"+cmd)
-
-        if api.available_entities.contains(id):
-            _LOG.debug("Entity with id " + id + " is already in storage as available entity")
-        else:
-            _LOG.info("Add entity with id " + id + " and name " + name + " as available entity")
-            await add_mp(id, name)
+    if config.setup.get("setup_complete"):
+        await setup.add_mp_all()
 
 
 
 async def add_mp(id: str, name: str):
-    #TODO Only works when in driver.py. When in media_player.py the response to get_available entities is an empty list
+    # Only works when in driver.py. When in media_player.py the response to get_available entities is an empty list
     """
     Creates the media player entity definition and adds the entity to the remote via the api
 
@@ -72,7 +72,7 @@ async def mp_cmd_handler(entity: ucapi.MediaPlayer, cmd_id: str, _params: dict[s
     if _params == None:
         _LOG.info(f"Received {cmd_id} command for {entity.id}")
     else:
-        _LOG.info(f"Received {cmd_id} command with parameter {_params} for {entity.id}")
+        _LOG.info(f"Received {cmd_id} command with parameter {_params} for entity id {entity.id}")
 
     return media_player.mp_cmd_assigner(entity.id, cmd_id, _params)
 
@@ -91,7 +91,6 @@ async def on_r2_connect() -> None:
 
 
 @api.listens_to(ucapi.Events.DISCONNECT)
-#TODO Find out how to prevent the remote from constantly reconnecting when the integration is not running without deleting the integration configuration on the remote every time
 async def on_r2_disconnect() -> None:
     """
     Disconnect notification from the remote Two.
@@ -174,7 +173,7 @@ async def main():
 
     _LOG.debug("Starting driver")
 
-    await api.init("setup.json")
+    await setup.init()
     await startcheck()