Issue with mcrun Command Freezing on M5Stack Core2

[lamfan]

I am encountering an issue while trying to run mcrun to install a mod on the M5Stack Core2. The mcconfig command works fine, but mcrun freezes during the installation process.

Steps to Reproduce

Successfully run the following command:
mcconfig -d -m -p esp32/m5stack_core -t build ./stackchan/manifest_local.json

Attempt to run the following command:
mcrun -d -m -p esp32/m5stack_core2 ./mods/look_around/manifest.json

Environment

ESP-IDF version: esp-idf-v5.2.2
Operating System: macOS

[phoddie]

Thank you for the report.

In your "steps" to reproduce, the command line for the host is incorrect. It is only building (-t build) but not installing the host. Also, the platform is set to esp32/m5stack_core instead of esp32/m5stack_core2. Please try:

mcconfig -d -m -p esp32/m5stack_core2 ./stackchan/manifest_local.json

I don't have an M5Stack Core2. It is powered by an ESP32, the same as Moddable Two. I did a test of a host and mod on Moddable Two and it was successful.

[lamfan]

@phoddie I forget I did run deploy after build

mcconfig -d -m -p esp32/m5stack_core2 -t build ./stackchan/manifest_local.json
mcconfig -d -m -p esp32/m5stack_core2 -t deploy ./stackchan/manifest_local.json

so I try your code

mcconfig -d -m -p esp32/m5stack_core2 ./stackchan/manifest_local.json

after build it and I run

mcrun -d -m -p esp32/m5stack_core2 ./mods/look_around/manifest.json

but it still freeze at Installing mod.
really don't understand what's wrong with it?

[stc1988]

Hi @lamfan , Sharing capture of your terminal when you run mcrun will help us to understand the situation.

[lamfan]

After running mcrun, it freeze at Installing mod.
The M5Stack core2 did reload the screen, but nothing change....

[phoddie]

The screen image shows one .. after "Installing mod" That means that the first fragment of the mod was sent and the Core2 acknowledged it was successful written to flash. After than, mcrun sends the next fragment or, if the mod fits into one fragment, mcrun sends the final command to update the mod header. Either way, there should be at least one more . when the Core2 acknowledges the command.

If the flash write fails (it uses esp_partition_write) the ESP32 reports an error code, which would be displayed in Terminal. Since that is not shown, it seems that the ESP32 is either hung in a loop or rebooted.

Maybe the partition map is corrupt? Unlikely, but.... just in case:

• erase the full flash (esptool.py erase_flash)
• rebuild the host with mcconfig
• reinstall the mod with mcrun

[lamfan]

Hi @phoddie ,
Here are the steps I used to setup my Moddable environment on macOS

1. xs-dev setup
2. xs-dev setup --list-devices
3. cd ~/.local/share/esp32
4. rm -rf esp-idf
5. git clone -b v5.2.2 --recursive https://github.com/espressif/esp-idf.git
6. cd esp-idf
7. ./install.sh

xs-dev-export.sh
export MODDABLE=/Users/lamfan/.local/share/moddable
export PATH="/Users/lamfan/.local/share/moddable/build/bin/mac/release:$PATH"
export IDF_PATH=/Users/lamfan/.local/share/esp32/esp-idf
source $IDF_PATH/export.sh 1> /dev/null

is this setup ok? I manually deleted the esp-idf folder and cloned the updated 5.2.2 esp-idf repository.

I had reinstall everything and run the mcrun again

Please help to check is there any missing? Thank you!

[phoddie]

I had reinstall everything and run the mcrun again

You need to build and install the Stack-chan host using mcconfig before trying to install the mod. (I am sure the Stack-chan documentation explains that.)

[lamfan]

Yes I did run

npm run build --target=esp32/m5stack_core2
npm run deploy --target=esp32/m5stack_core2
npm run mod --target=esp32/m5stack_core2 ./mods/look_around/manifest.json

But it still freeze at Installing mod.

[phoddie]

I see. And did you try erasing flash on the M5Stack Core2 as suggested above?

[lamfan]

Here is my step

1. esptool.py --chip esp32 --port /dev/cu.usbserial-58640313411 --baud 115200 erase_flash
2. mcconfig -d -m -p esp32/m5stack_core2 ./stackchan/manifest_local.json
3. mcrun -d -m -p esp32/m5stack_core2 ./mods/look_around/manifest.json

after reinstall, it still freeze at Installing mod.

I try to edit the mod.js inside look_around folder
this can complete installed

this freeze again at installing mod.

It look like the problem on javascript, how do you think?

[stc1988]

To add some context, we were discussing this issue on Discord, but this issue does not reproduce in my environment.

Original souce code is here.

[stc1988]

@lamfan It might not be related, but which one do you have? I don't have the new one.

• https://shop.m5stack.com/products/m5stack-core2-esp32-iot-development-kit-v1-1
• https://shop.m5stack.com/products/m5stack-core2-esp32-iot-development-kit

[phoddie]

By removing lines from the mod source code, you are making the mod's binary archive size smaller. With the full source code referenced by @stc1988, the mod is only about 1020 bytes. Removing lines, makes it smaller. But 1020 bytes is tiny already. It fits within a a single flash sector (a sector is 4 KB).

I don't think the JavaScript matters. Installing a mod is basically a file copy. The contents of the mod do not run and are not even parsed until the install is successful and the ESP32 reboots.

[lamfan]

@lamfan It might not be related, but which one do you have? I don't have the new one.

• https://shop.m5stack.com/products/m5stack-core2-esp32-iot-development-kit-v1-1
• https://shop.m5stack.com/products/m5stack-core2-esp32-iot-development-kit

I'm using v1.1

[stc1988]

@meganetaaan Have you ever seen mod of stackchan works before in M5stack core2 v1.1?

@lamfan In Discord discussion, I think the installation failed even with a shorter source code before. Are the conditions the same now as they were then?

[phoddie]

This looks like it could be a communication failure. The baud rate is relatively high on this device (1500000). We have seen situations where a damaged USB cable can cause data transfer failures. This would cause mod installation to fail. It might not always cause failures flashing the ESP32 firmware, because that protocol has some retry and checksum logic builti-in.

[lamfan]

Hi @stc1988 @phoddie
I'm using the sample code from "IoT Development for ESP32 and ESP8266 with JavaScript"

1. ch3-network/host -> mcconfig -d -m -p esp32/m5stack_core2
2. ch3-network/http-get -> mcrun -d -m -p esp32/m5stack_core2 ssid="xxxx" password="xxxx"
   It's freeze at "Installing mod." again, I think this is a problem with my ESP-IDF settings. What do you guys think?

[stc1988]

• Did you try using a different USB cable?
• I don't think it checks for Python when running mcrun, but are you able to run $IDF_PATH/export.sh at the start of the terminal session?

[lamfan]

Yes, I did try to different USB-C cable
source $IDF_PATH/export.sh did run here is the screenshot and after that I run mcrun.

[stc1988]

@lamfan @phoddie
By coincidence, the same issue occurred while installing a different mod in my M5stack core2. However, it was successful on another device(M5stack coreS3).

I will check the details a bit more.

[stc1988]

Indeed, the success or failure changes depending on the number of lines. It seems to fail when the size of the XS archive file exceeds around 2.6 KB.

[lamfan]

@stc1988 Thanks for your suggestion. Given the ongoing issue, I’ve decided to purchase a new M5Stack CoreS3 to continue my development on Stack Chan.

[stc1988]

@lamfan
I changed DEBUGGER_SPEED to 921600 following files and re-run mcconfig and mcrun, then it works!!

• examples/manifest_mod.json
• build/devices/esp32/targets/m5stack_core2/manifest.json

[lamfan]

@stc1988
Oh! That’s good. I didn’t know there was a DEBUGGER_SPEED property to set either. Thank you so much for the testing!

[phoddie]

@lamfan – does reducing the baud rate solve the problem you are having?

My recollection is that this device required 1500000 baud to work. It did not support 921600. Maybe my memory is wrong. Or... perhaps the serial bridge chip changed? Sometimes M5 changes components in their products.

We should try to understand that. If 921600 works for all M5Stack Core2 hardware, we should make that change to the Moddable SDK.

[stc1988]

It was working fine before, but the mod isn't working this time. Could some changes have affected it?

[phoddie]

It is possible some change had an impact here. But, the relevant code hasn't changed significantly in some time. It is difficult to guess where the problem might be:

• ESP-IDF version could matter
• Moddable SDK runtime for ESP32 (debugger communication, mod install, etc)
• Tooling to install mod (serial2xsbug)

Using the archive of Moddable SDK release it could be possible to go back to find where the behavior changed. That's a little tedious, unfortunately.

[lamfan]

@stc1988 @phoddie
I tried export DEBUGGER_SPEED=921600 and export DEBUGGER_SPEED=1500000
but neither of them is working... (freeze at installing mod. again)
Should I buy a new M5Stack CoreS3 SE for better testing? Thank you!

[stc1988]

@lamfan Did you modify files? Or set environment variables?

[lamfan]

.zshrc

1. mcconfig -d -m -p esp32/m5stack_core2 ./stackchan/manifest_local.json
2. mcrun -d -m -p esp32/m5stack_core2 ./mods/look_around/manifest.json

[stc1988]

@lamfan
Could you fix below files?

examples/manifest_mod.json
build/devices/esp32/targets/m5stack_core2/manifest.json

[lamfan]

Hi @stc1988 , what should I need to modify the files?

manifest.json

{
	"include": [
		"$(MODDABLE)/examples/manifest_mod.json"
	],
	"modules": {
		"*": "./mod"
	}
}

mod.js

import Timer from 'timer'
import { randomBetween } from 'stackchan-util'

export function onRobotCreated(robot) {
  let isFollowing = false
  robot.button.a.onChanged = function () {
    if (this.read()) {
      trace('pressed A\n')
      isFollowing = !isFollowing
    }
  }
  robot.button.b.onChanged = function () {
    if (this.read()) {
      trace('pressed B\n')
    }
  }
  robot.button.c.onChanged = function () {
    if (this.read()) {
      trace('pressed C\n')
    }
  }
  const targetLoop = () => {
    if (!isFollowing) {
      robot.lookAway()
      return
    }
    const x = randomBetween(0.4, 1.0)
    const y = randomBetween(-0.4, 0.4)
    const z = randomBetween(-0.02, 0.2)
    trace(`looking at: [${x}, ${y}, ${z}]\n`)
    robot.lookAt([x, y, z])
  }
  Timer.repeat(targetLoop, 5000)
}

[stc1988]

The file to be modified is not for Stackchan, but for Moddable SDK.

The installation directory for Moddable is $MODDABLE.

[lamfan]

Hi @stc1988 , after I modify the manifest_mod.json and manifest.json, here is the screen

the Installing mod. disappear and freeze at Python requirements are satisfied.

examples/manifest_mod.json

"esp32/m5stack_core2": {
            "build": {
                "UPLOAD_SPEED": "1500000",
                "DEBUGGER_SPEED": "921600"
            }
        }

build/devices/esp32/targets/m5stack_core2/manifest.json

 "build": {
        "UPLOAD_SPEED": "1500000",
        "DEBUGGER_SPEED": "921600",
        "SDKCONFIGPATH": "./sdkconfig",
        "PARTITIONS_FILE": "./sdkconfig/partitions.csv"
    }

[stc1988]

Did you re-run mcconfig?

[phoddie]

@lamfan – Is the M5Stack Core2 connected to your Mac directly or is it connected through a USB Hub? If you are using a hub, please retry with a direct connection. Thank you.

[lamfan]

@stc1988 I did re-run the following command after changing DEBUGGER_SPEED to 921600

mcconfig -d -m -p esp32/m5stack_core2 ./stackchan/manifest_local.json
mcrun -d -m -p esp32/m5stack_core2 ./mods/look_around/manifest.json

@phoddie yes, it is directly connect to my iMac with apple type-c cable.

[lamfan]

@stc1988 @phoddie
Finally I got it work by modify the DEBUGGER_SPEED to 460800.
also need to delete all files in %MODDABLE%\build\bin\esp32 and %MODDABLE%\build\tmp\esp32
then run
mcconfig -d -m -p esp32/m5stack_core2 ./stackchan/manifest_local.json
mcrun -d -m -p esp32/m5stack_core2 ./mods/look_around/manifest.json

@stc1988 why your m5stack_core2 v1.1 can work on 921600?

@phoddie when I run
mcconfig -d -m -p mac/m5stack ./stackchan/manifest_local.json on my Mac simulator, it show /Users/lamfan/Projects/moddable/modules/io/serial/mac/serial.c (147) # Break: Serial: can't open serial! I check the source code is created by you, do you know what's the problem?

fd = open(device, O_RDWR | O_NOCTTY | O_NONBLOCK);
if (-1 == fd)
   xsUnknownError("can't open serial");

[stc1988]

@lamfan
My M5stack core2 is not V1.1, I have old version.

Does error in your mac simulator look like a different issue, right?

[phoddie]

Finally I got it work by modify the DEBUGGER_SPEED to 460800.

Glad to hear you have it working. Some ESP32 development boards require a slower debugger speed (460800). But, those are usually older boards. It is unexpected that would happen with an update to the M5Stack Core2 hardware.

Regarding serial, @stc1988 is correct: It is a separate topic, and so should go into a separate report.

[stc1988]

"DEBUGGER_SPEED": "460800" also works my side.

[lamfan]

@phoddie when I run
mcconfig -d -m -p mac/m5stack ./stackchan/manifest_local.json on my Mac simulator, it show /Users/lamfan/Projects/moddable/modules/io/serial/mac/serial.c (147) # Break: Serial: can't open serial! I check the source code is created by you, do you know what's the problem?

fd = open(device, O_RDWR | O_NOCTTY | O_NONBLOCK);
if (-1 == fd)
   xsUnknownError("can't open serial");