OTA download with SPI flash as storage and modified SDU library

[JEG8]

Hi,

I am running into an issue on my Arduino IOT 33 where the OTA downloads the file and I receive a 200 status code, but the file is not replaced on the device. The old file is loaded back on. Do you know why this is happening? Could it be check sum related? The performance is very inconsistent.

Joe

[JAndrassy]

there is no checksum.
how do you test if the new or old version is running?

[JEG8]

I have a firmware version number that gets updated when I add a new version to my server. Also, I can see that when I make edits to print statements, they do not show up on the serial monitor, indicating that the new version was never loaded on. However, I can see that the download occurs correctly because the size of the file matches the BIN output of the IDE.

[JAndrassy]

you use InternalStorage directly?

[JEG8]

No, I don't know what you mean by "InternalStorage". I am saving the downloaded file to a SPI Flash. I am then checking the size of the file, and comparing it to the "length" returned by the server. If they match, I proceed with the update. That all works fine. However, it appears that the bootloader portion isnt working properly. The file is not replaced on the hardware.

[JAndrassy]

so you don't use the ArduinoOTA library at all.
you download the file to external SPI flash and it should be bootloaded by what? I don't know about a library similar to SDU library, which would load the update binary from SPI flash as the SDU library does from SD card

[JEG8]

Is the SDU library strictly for SD card? I am using Adafruit library which can be used with either an SD card or a SPI Flash. I figured because they both use SDFat library, they could be used interchangeably and the update would still work with the SDU library.

[JAndrassy]

SDU library is just an array set to be prepended by the linker before your sketch. And that array is a compiled extra sketch which loads the binary from SD card, writes it to flash and hands over the execution to regular sketch. It knows nothing about your SPI flash or the Adaruit_SPIFlash libray.

[JEG8]

Hi,

I should clarify, when I say SPIFlash - I am using a Winbond W25Q128JVSIM flash memory in place of an SD Card (to save space).

I think you pointed me to the right location. It looks like the SDU Library uses the following on (Line 49 SDUBoot.INO) to look for the update file on the SD Card.

if (SD.begin(SDCARD_SS_PIN) && SD.exists(UPDATE_FILE)) {
File updateFile = SD.open(UPDATE_FILE);
uint32_t updateSize = updateFile.size();
bool updateFlashed = false;

That file is actually on my SPI Flash, stored the same way. I should just need to point the SDU to look at my SPI Flash for the file, right?

[JAndrassy]

That file is actually on my SPI Flash, stored the same way. I should just need to point the SDU to look at my SPI Flash for the file, right?

right. you have to write and test the modified 'extras' sketch which will use the SPFlash library instead of the SD library. Then you have to convert the compiled binary of that sketch to an array of bytes in a .h file (there is some tool I found long time ago on Internet which can help with it). Then you create a library with that .h file and use it with your sketch.

[JAndrassy]

but if you sketch is smaller the half of the internal flash memory, you can use it to store the update binary. see the OTASketchDownloadWifi example

[JEG8]

Unfortunately, my sketch is > half. I would like to use a SPI Flash module with the Adafruit SPIFlash Library. Below is the code I am using to make the SDU Boot BIN file.

I am under the assumption that this SDU bootloader code is supposed to read the file from the SD card (in this case the SPI Flash) and load the new BIN file into the FlashStorage. Do you see any issues with this implementation?

My logic on my main sketch works correctly. It downloads the file, and stores it into the SPI Flash. The sizes are all correct. It then restarts the device.

It just doesnt seem like the SDU logic is working correctly to load the file as the bootloader (meaning I am doing something wrong :) )

#include <SPI.h>
#include "SdFat.h"
#include "Adafruit_SPIFlash.h"
#include <FlashStorage.h>

#ifdef ARDUINO_SAM_ZERO
#define SDU_START 0x4000
#else
#define SDU_START 0x2000
#endif
#define SDU_SIZE 0x6000

#define SKETCH_START (uint32_t*)(SDU_START + SDU_SIZE)

#define SPI_SPEED SD_SCK_MHZ(3) // Set the SD Clock speed, was (4)

// SET UP SPI FLASH
// SPI Flash File System Declarations
#define CUSTOM_CS 8
#define CUSTOM_SPI SPI

#if defined(CUSTOM_CS) && defined(CUSTOM_SPI)
Adafruit_FlashTransport_SPI flashTransport(CUSTOM_CS, CUSTOM_SPI);
#endif

Adafruit_SPIFlash flashWIN(&flashTransport);

static const SPIFlash_Device_t my_flash_devices[] = {
W25Q128JV_IM,
};
const int flashDevices = 1;

FatFileSystem fatfs;

const char* BIN_FILENAME = "UPDATE.BIN"; // expected by the SDU library

FlashClass flash;

File myFile;

// Initialize C library
extern "C" void __libc_init_array(void);

int main() {
init();

__libc_init_array();

delay(1);

// Initialize flash library and check its chip ID.
if ((flashWIN.begin(my_flash_devices, flashDevices)) && (fatfs.begin(&flashWIN)) && (fatfs.exists(BIN_FILENAME))){

    myFile = fatfs.open(BIN_FILENAME, FILE_READ);
    uint32_t updateSize = myFile.size();
    bool updateFlashed = false;

    if (updateSize > SDU_SIZE) {
      // skip the SDU section
      myFile.seek(SDU_SIZE);
      updateSize -= SDU_SIZE;

      uint32_t flashAddress = (uint32_t)SKETCH_START;

      // erase the pages from the flash
      flash.erase((void*)flashAddress, updateSize);

      uint8_t buffer[512];

      // write the pages
      for (uint32_t i = 0; i < updateSize; i += sizeof(buffer)) {
        myFile.read(buffer, sizeof(buffer));

        flash.write((void*)flashAddress, buffer, sizeof(buffer));

        flashAddress += sizeof(buffer);
      }

      updateFlashed = true;
    }

    myFile.close();

    if (updateFlashed) {
      fatfs.remove(BIN_FILENAME);
    }

}

// jump to the sketch
__set_MSP(*SKETCH_START);

//Reset vector table address
SCB->VTOR = ((uint32_t)(SKETCH_START) & SCB_VTOR_TBLOFF_Msk);

// address of Reset_Handler is written by the linker at the beginning of the .text section (see linker script)
uint32_t resetHandlerAddress = (uint32_t) * (SKETCH_START + 1);
// jump to reset handler
asm("bx %0"::"r"(resetHandlerAddress));
}

[JAndrassy]

how do you know it is not working? what happens if you run this sketch?
what is the size of the loader sketch?

[JEG8]

Hi,

I was able to get it to work. I had two different issues.

The first issue was that I am using an SD card and a SPI Flash in my circuit. With use of the fatfs library (for the Adafruit spi flash filesystem) The SD Card needs to be initialized in the SDUBoot code in this case, even if I wanted to use just the SPI flash for my updates. I think the FATFS was defaulting to the SD Card after discovering it on the SPI interface.

The other issue was that the Adafruit SPI flash library was printing ascii into the hex file, no writing bytes. Once I changed this to a write function (from the SD library), it worked properly.

Thanks for your help
Joe