Code rework, cleanup

Zigbee HEX file validation speedup x4.5 (Zigbee flashing still in work)
Zigbee file validation progress indication with BSL config in console (Zigbee flashing still in work)
Add ability to flash ESP32 OTA from custom URL http://IP_ADDRESS/api?action=8&cmd=9&url=LINK_TO_BIN_FILE.bin
Add timezone selection on system control tab (for NTP need for WG and OTA)
**Announcement** Cause typofix mqttPublishAvail() in 0.2.5 topic **/avty** will be changed to **/avail** . It you use AutoDiscovery don't keep mind.

Author: xyzroe

.gitignore

@@ -3,7 +3,7 @@
 .vscode
 .gcc-flags.json
 .clang_complete
-commit
+commit*
 test.html
 gzipped
 webh

README.md

@@ -20,8 +20,10 @@ This repository contains the latest firmware for the ESP32 peripheral module of
 - ESP32 (peripheral) and CC2652P (Zigbee) OTA updates, ESP32  and CC2652P restart;
 - Switch CC2652P (Zigbee) to flash mode;
 - Pairing mode for the adapter in router mode;
-- Firewall for incoming connections to secure your network;
+- Filter IP addresses for incoming connections to secure your network;
 - Fully responsive web interface based on the latest Bootstrap 5.3.
+- WireGuard VPN
+- MQTT client for gateway monitoring and control
 
 ## Installation and Configuration
 Please refer to the installation and configuration articles in our [documentation](https://uzg.zig-star.com).  
@@ -31,20 +33,11 @@ Please follow this link for web-flasher - just plug-and-flash: [ZigStar WebFlash
 
 If you made changes to the code and want to compile you own firmware, please do the following:
 
-### You did not change web-interface appearance
 - Download repository;
 - Install Microsoft Visual Code (MVC);
 - Install PlatformIO extension to MVC;
 - Press "PlatformIO: Build" and wait until firmware.bin is generated;
 
-### You made changes to web-interface
-
-In such case - you have to rebuild web-interface fisrt before building the firmware.
-- Go to the folder tools/webfilesbuilder;
-- Run: npm install
-- Run: npx gulp
-
-
 ## Contribute
 
 You can contribute to UZG-01 Firmware by
@@ -55,25 +48,24 @@ You can contribute to UZG-01 Firmware by
 ## Credits
 
 People helping to keep the show on the road - **developers and contributors**:
-- [@smlight-dev](https://github.com/smlight-dev/) - for improvement and refactoring of [ZigStarGW-FW](https://github.com/xyzroe/ZigStarGW-FW)
-- [@Tarik2142](https://github.com/Tarik2142) for refactoring, code optimizations done under [smlight-dev](https://github.com/mercenaruss/uzg-firmware)
-- [@xyzroe](https://github.com/xyzroe/) for initial firmware released for ZigStar devices
+
+- [@xyzroe](https://github.com/xyzroe/) for initial firmware released for ZigStar devices and all work being doing till now  
+- [@Tarik2142](https://github.com/Tarik2142) for refactoring, code optimizations done under [smlight-dev](https://github.com/smlight-dev/)
+
 
 Special thanks goes also to all authors of 3rd party libraries which are used in this project:
 
 - [bblanchon / arduinojson](https://github.com/bblanchon/ArduinoJson), 
-- [esp8266 / arduino](https://github.com/esp8266/Arduino), 
 - [espressif / arduino-esp32](https://github.com/espressif/arduino-esp32), 
 - [esprfid / esp-rfid](https://github.com/esprfid/esp-rfid), 
 - [fairecasoimeme / zigate-ethernet](https://github.com/fairecasoimeme/ZiGate-Ethernet), 
 - [rlogiacco / circularbuffer](https://github.com/rlogiacco/CircularBuffer), 
 - [sstaub / ticker](https://github.com/sstaub/Ticker), 
-- [vurtun / lib](https://github.com/vurtun/lib) 
+- [vurtun / lib](https://github.com/vurtun/lib),
+- [ciniml / WireGuard-ESP32-Arduino] (https://github.com/ciniml/WireGuard-ESP32-Arduino)  
 
 ## License and attribution
 
-UZG-01-Firmware is a fork of [smlight-dev](https://github.com/mercenaruss/uzg-firmware),what was refactored from our original firmware[ZigStarGW-FW](https://github.com/xyzroe/ZigStarGW-FW)
-
 UZG-01-Firmware is licensed under GNU General Public License v3.
 3rd party libraries that are used by this project are licensed under different license schemes, please check them out as well.  
 Copyright (c) 2023 ZigStar.  

bin/UZG-01.bin

@@ -0,0 +1,8 @@
+name=CC26XX-ESP32
+version=0.0.1
+author=xyzroe
+maintainer=xyzroe
+sentence=TI CC26XX dev tools on ESP32
+paragraph=
+includes=CC26XX.h
+architectures=esp32

bin/UZG-01_v0.2.3.full.bin

@@ -146,6 +146,17 @@ void CommandInterface::_sendNAck()
     _stream.write(cmd2);
 }
 
+void CommandInterface::_eraseFlash() {
+    const u_int32_t cmd1 = 0x2C; 
+    _stream.write(cmd1);
+}
+
+bool CommandInterface::_ping() {
+    const u_int32_t cmd1 = 0x20; 
+    _stream.write(cmd1);
+    return _wait_for_ack(1);
+}
+
 byte *CommandInterface::_cmdMemRead(uint32_t address)
 {
     const u_int32_t cmd = 0x2A;
@@ -355,3 +366,11 @@ String CC26XX_detect::detectChipInfo()
     return chip_str;
 }
 
+
+bool CC26XX_detect::eraseFlash()
+{
+    _ping();
+    _eraseFlash();
+    
+    return true;
+}

bin/UZG-01_v0.2.4.full.bin

@@ -94,6 +94,8 @@ class CommandInterface
     bool _checkLastCmd();
     void _sendAck();
     void _sendNAck();
+    void _eraseFlash();
+    bool _ping();
     void _encodeAddr(unsigned long addr, byte encodedAddr[4]);
     unsigned long _decodeAddr(byte byte0, byte byte1, byte byte2, byte byte3);
     byte _calcChecks(byte cmd, unsigned long addr, unsigned long size);
@@ -110,6 +112,8 @@ class CC26XX_detect : public CommandInterface
 public:
     CC26XX_detect(Stream &serial);
     bool begin(int CC_RST_PIN, int CC_BSL_PIN, int BSL_MODE = 0);
+    bool eraseFlash();
+    bool ping();
     String detectChipInfo();
 };
 

lib/CC26XX/library.properties

@@ -0,0 +1,8 @@
+name=IntelHEX-ESP32
+version=0.0.2
+author="OpenAI's ChatGPT with contributions from xyzroe"
+maintainer=xyzroe
+sentence=IntelHEX file actions on ESP32
+paragraph=
+includes=intelhex.h
+architectures=esp32

lib/CC26XX/src/CC26XX.cpp

@@ -1,4 +1,6 @@
 #include "intelhex.h"
+#include <esp_task_wdt.h>
+
 /*
  * IntelHex Arduino Library
  *
@@ -15,15 +17,8 @@
 
 If you want library to control LittleFS you need to uncomment some lines.
 
-
-TO-DO List
-
-1. Line length check
-2. Error if not IntelHEX line after start till end
-
 */
 
-
 #ifndef DEBUG_PRINT
 #ifdef DEBUG
 #define DEBUG_PRINT(x) Serial.print(String(x))
@@ -34,9 +29,6 @@ TO-DO List
 #endif
 #endif
 
-
-
-
 IntelHex::IntelHex(const char *filename) : _filename(filename) {}
 
 IntelHex::~IntelHex()
@@ -63,13 +55,16 @@ void IntelHex::close()
     // LittleFS.end();
 }
 
-bool IntelHex::parse(void (*preCallback)(), void (*parseCallback)(uint32_t address, uint8_t len, uint8_t *data), void (*postCallback)())
+bool IntelHex::parse(void (*preCallback)(), void (*parseCallback)(uint32_t address, uint8_t len, uint8_t *data, size_t currentPosition, size_t totalSize), void (*postCallback)())
 {
     if (!open())
     {
         return false;
     }
 
+    _totalSize = _file.size();
+    DEBUG_PRINTLN(String("_file.size() = ") + String(_totalSize));
+
     preCallback();
 
     bool status = true;
@@ -96,8 +91,115 @@ bool IntelHex::parse(void (*preCallback)(), void (*parseCallback)(uint32_t addre
     return status;
 }
 
-bool IntelHex::_munchLine(void (*parseCallback)(uint32_t address, uint8_t len, uint8_t *data))
+/*
+bool IntelHex::validateChecksum()
+{
+    DEBUG_PRINTLN("Starting checksum validation");
+
+    if (!open())
+    {
+        DEBUG_PRINTLN("Failed to open file");
+        return false;
+    }
+
+    String line;
+    while (_file.available())
+    {
+        line = _file.readStringUntil('\n');
+        line.trim();
+
+        if (line.length() < 11 || line[0] != ':')
+        {
+            DEBUG_PRINTLN("Skipping non-data line or line too short");
+            continue;
+        }
+
+        // Проверяем на строку окончания файла Intel HEX
+        if (line == ":00000001FF")
+        {
+            DEBUG_PRINTLN("Intel HEX file end line found, checksum validation passed");
+            close();
+            return true;
+        }
+
+        uint8_t sum = 0;
+        // Считаем сумму всех байтов строки, за исключением начального ':'
+        for (int i = 1; i < line.length(); i += 2)
+        {
+            uint8_t byteValue = strtol(line.substring(i, i + 2).c_str(), nullptr, 16);
+            sum += byteValue;
+        }
+
+        // Проверка контрольной суммы
+        if (sum != 0)
+        {
+            DEBUG_PRINTLN("Checksum validation failed");
+            close();
+            return false;
+        }
+    }
+
+    DEBUG_PRINTLN("Checksum validation passed, but no end line found - file might be incomplete");
+    close();
+    return false; // Если мы достигли этой точки, значит, строка окончания файла не была найдена.
+}
+*/
+
+/*
+bool IntelHex::checkBSLConfiguration() {
+    DEBUG_PRINTLN("checkBSLConfiguration: Starting BSL configuration check");
+
+    if (!open()) {
+        DEBUG_PRINTLN("checkBSLConfiguration: Failed to open file");
+        return false;
+    }
+
+    const uint32_t addresses[] = {ALL_CHIP_ADDRESS, P7_CHIP_ADDRESS};
+    bool foundBSL = false;
+
+    for (auto address : addresses) {
+        DEBUG_PRINT("checkBSLConfiguration: Checking address: ");
+        DEBUG_PRINTLN(String(address, HEX));
+
+        // Предполагаем, что BSL конфигурация начинается непосредственно с этого адреса.
+        // Важно отметить, что в файле Intel Hex адреса не обязательно идут последовательно и могут быть разбросаны.
+        // Это может потребовать специфической логики для перемещения по файлу.
+
+        // Чтение данных из файла, предполагая, что они могут находиться по этому адресу.
+        _file.seek(address, SeekSet);
+        uint8_t data[4]; // Предполагаем, что для проверки BSL конфигурации достаточно 4 байтов.
+        if (_file.read(data, 4) != 4) {
+            DEBUG_PRINTLN("checkBSLConfiguration: Failed to read data");
+            continue; // Если не удается прочитать 4 байта, переходим к следующему адресу.
+        }
+
+        // Добавляем логирование прочитанных данных, чтобы увидеть, что было считано.
+        DEBUG_PRINT("checkBSLConfiguration: Read data: ");
+        for (int i = 0; i < 4; ++i) {
+            DEBUG_PRINT(String(data[i], HEX));
+            DEBUG_PRINT(" ");
+        }
+        DEBUG_PRINTLN();
+
+        foundBSL = _checkBSLconfig(address, 4, data);
+        if (foundBSL) {
+            DEBUG_PRINTLN("checkBSLConfiguration: BSL configuration found and valid");
+            break; // Если нашли и проверили конфигурацию BSL, прерываем цикл.
+        }
+    }
+
+    if (!foundBSL) {
+        DEBUG_PRINTLN("checkBSLConfiguration: BSL configuration not found or invalid");
+    }
+
+    close();
+    return foundBSL;
+}
+*/
+
+bool IntelHex::_munchLine(void (*parseCallback)(uint32_t address, uint8_t len, uint8_t *data, size_t currentPosition, size_t totalSize))
 {
+
     String line = _file.readStringUntil('\n');
     line.trim();
 
@@ -135,11 +237,6 @@ bool IntelHex::_munchLine(void (*parseCallback)(uint32_t address, uint8_t len, u
     // DEBUG_PRINTLN("Record Type: " + String(recordType, HEX) + ", Offset Low: " + String(offset_low, HEX) + ", Length: " + String(len, HEX)); // Print record info
 
     uint8_t data[255];
-    for (uint8_t i = 0; i < len; i++)
-    {
-        data[i] = strtol(line.substring(9 + i * 2, 11 + i * 2).c_str(), nullptr, 16);
-    }
-
     uint32_t address = offset_low;
     // https://jimmywongiot.com/2021/04/20/format-of-intelhex/
     if (recordType == 4)
@@ -153,28 +250,51 @@ bool IntelHex::_munchLine(void (*parseCallback)(uint32_t address, uint8_t len, u
 
     address += _offset_high;
 
-    if (!_bsl_valid)
+    esp_task_wdt_reset();
+//#ifdef DEBUG
+//    delay(5); // to avoid reboot
+//#endif
+
+    for (int i = 0; i < ELEMENTCOUNT(CCFG_ADDRESS); i++)
     {
-        _bsl_valid = _checkBSLconfig(address, len, data);
+        // DEBUG_PRINT(CCFG_ADDRESS[i]);
+        // DEBUG_PRINT(" ");
+        if (address <= CCFG_ADDRESS[i] && address + len > CCFG_ADDRESS[i] + 4)
+        {
+            //DEBUG_PRINTLN(" ");
+            //DEBUG_PRINTLN("CCFG_ADDRESS[" + String(i) + "] in range");
+
+            
+            for (uint8_t i = 0; i < len; i++)
+            {
+                data[i] = strtol(line.substring(9 + i * 2, 11 + i * 2).c_str(), nullptr, 16);
+            }
+
+            if (!_bsl_valid)
+            {
+                _bsl_valid = _checkBSLconfig(address, len, data);
+            }
+        }
     }
-    parseCallback(address, len, data);
+
+    size_t currentPosition = _file.position();
+
+    parseCallback(address, len, data, currentPosition, _totalSize);
 
     return true;
 }
 
 bool IntelHex::_checkBSLconfig(uint32_t address, uint8_t len, uint8_t *data)
 {
     // Check if CCFG is within the buffer
-     //DEBUG_PRINTLN(" ");
-     //DEBUG_PRINT(ELEMENTCOUNT(CCFG_ADDRESS));
-     //DEBUG_PRINT(" ");
+    // DEBUG_PRINTLN(" ");
+    // DEBUG_PRINT(ELEMENTCOUNT(CCFG_ADDRESS));
+    // DEBUG_PRINT(" ");
 
-    // DEBUG_PRINTLN(ESP.getFreeHeap() / 1024);
-    delay(5);
     for (int i = 0; i < ELEMENTCOUNT(CCFG_ADDRESS); i++)
     {
-         //DEBUG_PRINT(CCFG_ADDRESS[i]);
-         //DEBUG_PRINT(" ");
+        // DEBUG_PRINT(CCFG_ADDRESS[i]);
+        // DEBUG_PRINT(" ");
         if (address <= CCFG_ADDRESS[i] && address + len > CCFG_ADDRESS[i] + 4)
         {
             DEBUG_PRINTLN(" ");