Unable to flash M5 board -> {ModuleNotFoundError: No module named 'lark'} #23477

phonnakasturi-apple · 2022-11-03T18:51:59Z

Reproduction steps

With SHA 706e9bb14f9931b70c0d4f3478a3460f56ebd099

I am unable to flash M5 board, as I am running into -> ModuleNotFoundError: No module named 'lark'

Steps to reproduce:

1. cd connectedhomeip
2. git reset --hard 706e9bb14f9931b70c0d4f3478a3460f56ebd099
3. git clean -Xdf
4. git submodule update --init
5. export PKG_CONFIG_PATH="/opt/homebrew/opt/openssl@3/lib/pkgconfig"
6. source scripts/bootstrap.sh
7. source scripts/activate.sh
8. cd ${HOME}/tools/esp-idf
9. git checkout v4.4
10. git submodule update --init --recursive
11. ./install.sh
12. . ./export.sh
13. cd ~/connectedhomeip/examples/all-clusters-app/esp32
14. idf.py build

Bug prevalence

Always

GitHub hash of the SDK that was being used

706e9bb

Platform

darwin

Platform Version(s)

No response

Anything else?

Attaching the log;

M5_error_log.txt

The text was updated successfully, but these errors were encountered:

andy31415 · 2022-11-03T19:09:49Z

I think IDF (which is from ./.export.sh) will set their own compier defs.

Generally we do NOT build with idf.py build. Please do this (on a clean shell without idf or activate)

source scripts/activate.sh  # or bootstrap.sh if first run
./scripts/build/build_examples.py --target esp32-m5stack-all-clusters build

if you are wondering what it does, there is a sequencing difference: you MUST first export IDF and only then activate. using --dry-run in build_examples shows the steps:

# Generating esp32-m5stack-all-clusters
mkdir -p /workspace/out/esp32-m5stack-all-clusters

cp examples/all-clusters-app/esp32/sdkconfig_m5stack.defaults /workspace/out/esp32-m5stack-all-clusters/sdkconfig.defaults

rm -f examples/all-clusters-app/esp32/sdkconfig

bash -c 'source $IDF_PATH/export.sh; source scripts/activate.sh;
export SDKCONFIG_DEFAULTS=/workspace/out/esp32-m5stack-all-clusters/sdkconfig.defaults
idf.py -C examples/all-clusters-app/esp32 -B /workspace/out/esp32-m5stack-all-clusters reconfigure'

rm -f examples/all-clusters-app/esp32/sdkconfig

# Building esp32-m5stack-all-clusters
bash -c 'source $IDF_PATH/export.sh; source scripts/activate.sh;
export SDKCONFIG_DEFAULTS=/workspace/out/esp32-m5stack-all-clusters/sdkconfig.defaults
idf.py -C examples/all-clusters-app/esp32 -B /workspace/out/esp32-m5stack-all-clusters build'

bzbarsky-apple · 2022-11-03T19:15:55Z

@phonnakasturi-apple Hmm, the ESP docs do say to run install/export first, before running activate, afaict. If changing the order works, please close?

andy31415 · 2022-11-03T19:17:29Z

I would also be very interested if build_examples.py works. Never tested on mac, but it is intended to be the build invocation to use so we will make some effort in making it work properly.

phonnakasturi-apple · 2022-11-03T19:24:50Z

Executing -> ./scripts/build/build_examples.py --target esp32-m5stack-all-clusters build

throws the following error:

priyankahonnakasturi@Priyankas-MBP esp32 % ./scripts/build/build_examples.py --target esp32-m5stack-all-clusters build
zsh: no such file or directory: ./scripts/build/build_examples.py
priyankahonnakasturi@Priyankas-MBP esp32 % cd ../..
priyankahonnakasturi@Priyankas-MBP examples % cd ..
priyankahonnakasturi@Priyankas-MBP connectedhomeip % ./scripts/build/build_examples.py --target esp32-m5stack-all-clusters build
Traceback (most recent call last):
  File "./scripts/build/build_examples.py", line 22, in <module>
    import coloredlogs
ModuleNotFoundError: No module named 'coloredlogs'

Order of execution:

cd connectedhomeip
export PKG_CONFIG_PATH="/opt/homebrew/opt/openssl@3/lib/pkgconfig"
source scripts/bootstrap.sh
source scripts/activate.sh
./scripts/build/build_examples.py --target esp32-m5stack-all-clusters build

shubhamdp · 2022-11-04T04:40:30Z

@phonnakasturi-apple below sequence should work

cd connectedhomeip
source path/to/esp-idf/export.sh
source scripts/activate.sh
scripts/build/build_examples.py --target esp32-m5stack-all-clusters build

phonnakasturi-apple · 2022-11-04T19:12:07Z

@shubhamdp Following the above sequence, gives the following error:

2022-11-04 12:10:40 WARNING 
2022-11-04 12:10:40 WARNING ESP-IDF v4.4
Traceback (most recent call last):
  File "scripts/build/build_examples.py", line 195, in <module>
    main()
  File "/Users/priyankahonnakasturi/connectedhomeip/.environment/pigweed-venv/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/priyankahonnakasturi/connectedhomeip/.environment/pigweed-venv/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/priyankahonnakasturi/connectedhomeip/.environment/pigweed-venv/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
    rv.append(sub_ctx.command.invoke(sub_ctx))
  File "/Users/priyankahonnakasturi/connectedhomeip/.environment/pigweed-venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/priyankahonnakasturi/connectedhomeip/.environment/pigweed-venv/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/priyankahonnakasturi/connectedhomeip/.environment/pigweed-venv/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "scripts/build/build_examples.py", line 185, in cmd_build
    context.obj.Build()
  File "/Users/priyankahonnakasturi/connectedhomeip/scripts/build/build/__init__.py", line 66, in Build
    self.Generate()
  File "/Users/priyankahonnakasturi/connectedhomeip/scripts/build/build/__init__.py", line 61, in Generate
    builder.generate()
  File "/Users/priyankahonnakasturi/connectedhomeip/scripts/build/builders/esp32.py", line 186, in generate
    self._IdfEnvExecute(cmd)
  File "/Users/priyankahonnakasturi/connectedhomeip/scripts/build/builders/esp32.py", line 149, in _IdfEnvExecute
    self._Execute(
  File "/Users/priyankahonnakasturi/connectedhomeip/scripts/build/builders/builder.py", line 94, in _Execute
    self._runner.Run(cmdarray, title=title)
  File "/Users/priyankahonnakasturi/connectedhomeip/scripts/build/runner/shell.py", line 73, in Run
    raise Exception('Command %r failed: %d' % (cmd, code))
Exception: Command ['bash', '-c', 'source $IDF_PATH/export.sh; source scripts/activate.sh; \nexport SDKCONFIG_DEFAULTS=/Users/priyankahonnakasturi/connectedhomeip/out/esp32-m5stack-all-clusters/sdkconfig.defaults\nidf.py -C examples/all-clusters-app/esp32 -B /Users/priyankahonnakasturi/connectedhomeip/out/esp32-m5stack-all-clusters reconfigure'] failed: 1
priyankahonnakasturi@Priyankas-MacBook-Pro connectedhomeip %

phonnakasturi-apple · 2022-11-04T19:16:33Z

Note: With SHA 0c00923

if I execute in the following sequence ;

cd connectedhomeip
git reset --hard 0c00923
git clean -Xdf
git submodule update --init
export PKG_CONFIG_PATH="/opt/homebrew/opt/openssl@3/lib/pkgconfig"
source scripts/bootstrap.sh
source scripts/activate.sh
cd ${HOME}/tools/esp-idf
git checkout v4.4
git submodule update --init --recursive
./install.sh
. ./export.sh
cd ~/connectedhomeip/examples/all-clusters-app/esp32
idf.py build

I get "No module named 'stringcase'" error

complete log:

[11/1258] Generating ../../gen/app-cod...p-app/app/PluginApplicationCallbacks.h
FAILED: gen/app-codegen/cpp-app/app/PluginApplicationCallbacks.h /Users/priyankahonnakasturi/connectedhomeip/examples/all-clusters-app/esp32/build/gen/app-codegen/cpp-app/app/PluginApplicationCallbacks.h 
cd /Users/priyankahonnakasturi/connectedhomeip/examples/all-clusters-app/esp32/build/esp-idf/main && /Users/priyankahonnakasturi/connectedhomeip/scripts/codegen.py --generator cpp-app --output-dir /Users/priyankahonnakasturi/connectedhomeip/examples/all-clusters-app/esp32/build/gen/app-codegen/cpp-app --expected-outputs /Users/priyankahonnakasturi/connectedhomeip/examples/all-clusters-app/esp32/build/gen/app-codegen/cpp-app/expected.outputs /Users/priyankahonnakasturi/connectedhomeip/examples/all-clusters-app/all-clusters-common/all-clusters-app.matter
Traceback (most recent call last):
  File "/Users/priyankahonnakasturi/connectedhomeip/scripts/codegen.py", line 34, in <module>
    from idl.generators import FileSystemGeneratorStorage, GeneratorStorage
  File "/Users/priyankahonnakasturi/connectedhomeip/scripts/idl/generators/__init__.py", line 23, in <module>
    from .filters import RegisterCommonFilters
  File "/Users/priyankahonnakasturi/connectedhomeip/scripts/idl/generators/filters.py", line 16, in <module>
    import stringcase
ModuleNotFoundError: No module named 'stringcase'
[22/1258] Building C object esp-idf/mb...CMakeFiles/mbedcrypto.dir/bignum.c.obj
ninja: build stopped: subcommand failed.
ninja failed with exit code 1

phonnakasturi-apple · 2022-11-07T20:03:56Z

Note: It stopped working from SHA 559c7d5

phonnakasturi-apple · 2022-11-07T22:36:49Z

By manually installing the following modules, I could get the M5 board to flash;

pip install lark
pip install coloredlogs
pip install stringcase

andy31415 · 2022-11-08T09:47:47Z

@phonnakasturi-apple I believe the order of python invocation must be first idf then activate/bootstrap.

if you switch the order (i.e. run export.sh last) then a python with different modules will be used and I expect lark/stringcase errors.

phonnakasturi-apple · 2022-11-08T21:20:08Z

@andy31415 Following is the order of execution:

cd ~/tools/esp-idf
./install.sh
. ./export.sh
cd ~/connectedhomeip/examples/all-clusters-app/esp32
idf.py build

And this gives the modules missing error;

FAILED: gen/app-codegen/cpp-app/app/PluginApplicationCallbacks.h /Users/priyankahonnakasturi/connectedhomeip/examples/all-clusters-app/esp32/build/gen/app-codegen/cpp-app/app/PluginApplicationCallbacks.h 
cd /Users/priyankahonnakasturi/connectedhomeip/examples/all-clusters-app/esp32/build/esp-idf/main && /Users/priyankahonnakasturi/connectedhomeip/scripts/codegen.py --generator cpp-app --output-dir /Users/priyankahonnakasturi/connectedhomeip/examples/all-clusters-app/esp32/build/gen/app-codegen/cpp-app --expected-outputs /Users/priyankahonnakasturi/connectedhomeip/examples/all-clusters-app/esp32/build/gen/app-codegen/cpp-app/expected.outputs /Users/priyankahonnakasturi/connectedhomeip/examples/all-clusters-app/all-clusters-common/all-clusters-app.matter
Traceback (most recent call last):
  File "/Users/priyankahonnakasturi/connectedhomeip/scripts/codegen.py", line 28, in <module>
    from idl.matter_idl_parser import CreateParser
  File "/Users/priyankahonnakasturi/connectedhomeip/scripts/idl/matter_idl_parser.py", line 7, in <module>
    from lark import Lark
ModuleNotFoundError: No module named 'lark'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/priyankahonnakasturi/connectedhomeip/scripts/codegen.py", line 32, in <module>
    from idl.matter_idl_parser import CreateParser
  File "/Users/priyankahonnakasturi/connectedhomeip/scripts/idl/matter_idl_parser.py", line 7, in <module>
    from lark import Lark
ModuleNotFoundError: No module named 'lark'
[591/1243] Performing configure step for 'chip_gn'
AUTO-DETECTING input matter IDL file.
Done. Made 221 targets from 142 files in 169ms
[593/1243] Building C object esp-idf/n....dir/nghttp2/lib/nghttp2_session.c.obj
ninja: build stopped: subcommand failed.
ninja failed with exit code 1

andy31415 · 2022-11-09T10:11:08Z

@andy31415 Following is the order of execution:

cd ~/tools/esp-idf
./install.sh
. ./export.sh
cd ~/connectedhomeip/examples/all-clusters-app/esp32
idf.py build

And this gives the modules missing error;

FAILED: gen/app-codegen/cpp-app/app/PluginApplicationCallbacks.h /Users/priyankahonnakasturi/connectedhomeip/examples/all-clusters-app/esp32/build/gen/app-codegen/cpp-app/app/PluginApplicationCallbacks.h 
cd /Users/priyankahonnakasturi/connectedhomeip/examples/all-clusters-app/esp32/build/esp-idf/main && /Users/priyankahonnakasturi/connectedhomeip/scripts/codegen.py --generator cpp-app --output-dir /Users/priyankahonnakasturi/connectedhomeip/examples/all-clusters-app/esp32/build/gen/app-codegen/cpp-app --expected-outputs /Users/priyankahonnakasturi/connectedhomeip/examples/all-clusters-app/esp32/build/gen/app-codegen/cpp-app/expected.outputs /Users/priyankahonnakasturi/connectedhomeip/examples/all-clusters-app/all-clusters-common/all-clusters-app.matter
Traceback (most recent call last):
  File "/Users/priyankahonnakasturi/connectedhomeip/scripts/codegen.py", line 28, in <module>
    from idl.matter_idl_parser import CreateParser
  File "/Users/priyankahonnakasturi/connectedhomeip/scripts/idl/matter_idl_parser.py", line 7, in <module>
    from lark import Lark
ModuleNotFoundError: No module named 'lark'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/priyankahonnakasturi/connectedhomeip/scripts/codegen.py", line 32, in <module>
    from idl.matter_idl_parser import CreateParser
  File "/Users/priyankahonnakasturi/connectedhomeip/scripts/idl/matter_idl_parser.py", line 7, in <module>
    from lark import Lark
ModuleNotFoundError: No module named 'lark'
[591/1243] Performing configure step for 'chip_gn'
AUTO-DETECTING input matter IDL file.
Done. Made 221 targets from 142 files in 169ms
[593/1243] Building C object esp-idf/n....dir/nghttp2/lib/nghttp2_session.c.obj
ninja: build stopped: subcommand failed.
ninja failed with exit code 1

I am saying that I specifically do NOT expect this to work: you are executing export.sh and that will invalidate all the python settings set up by activate/bootstrap.sh. That sequence of steps will not work and I don't think we can make it work in a clean matter as we have less control over what export.sh will set up. You found the workaround by manually installing packages, but it is a workaround.

please switch your steps by making sure export.sh is run before activate.sh and use activate.sh to setup the final build environment.

phonnakasturi-apple · 2022-11-09T15:41:45Z

@andy31415 I switched the steps to the following;

cd ~/tools/esp-idf
./install.sh
. ./export.sh
cd ~/connectedhomeip
export PKG_CONFIG_PATH="/opt/homebrew/opt/openssl@3/lib/pkgconfig"
source scripts/activate.sh
cd examples/all-clusters-app/esp32
idf.py build

And I git the following error;

esp32 % idf.py build
The following Python requirements are not satisfied:
future>=0.15.2
idf-component-manager>=0.2.99-beta
gdbgui==0.13.2.0
pygdbmi<=0.9.0.2
python-socketio<5
kconfiglib==13.7.1
reedsolo>=1.5.3,<=1.5.4
bitstring>=3.1.6
ecdsa>=0.16.0
construct==2.10.54
To install the missing packages, please run "/Users/priyankahonnakasturi/tools/esp-idf/install.sh"
Diagnostic information:
    IDF_PYTHON_ENV_PATH: /Users/priyankahonnakasturi/.espressif/python_env/idf4.4_py3.8_env
    Python interpreter used: /Users/priyankahonnakasturi/connectedhomeip/.environment/pigweed-venv/bin/python
    Warning: python interpreter not running from IDF_PYTHON_ENV_PATH
    PATH: /Users/priyankahonnakasturi/connectedhomeip/third_party/pigweed/repo/out/host/host_tools:/Users/priyankahonnakasturi/connectedhomeip/.environment/pigweed-venv/bin:/Users/priyankahonnakasturi/connectedhomeip/.environment/cipd/packages/pigweed/bin:/Users/priyankahonnakasturi/connectedhomeip/.environment/cipd/packages/pigweed:/Users/priyankahonnakasturi/connectedhomeip/.environment/cipd:/Users/priyankahonnakasturi/tools/esp-idf/components/esptool_py/esptool:/Users/priyankahonnakasturi/tools/esp-idf/components/espcoredump:/Users/priyankahonnakasturi/tools/esp-idf/components/partition_table:/Users/priyankahonnakasturi/tools/esp-idf/components/app_update:/Users/priyankahonnakasturi/.espressif/tools/xtensa-esp32-elf/esp-2021r2-8.4.0/xtensa-esp32-elf/bin:/Users/priyankahonnakasturi/.espressif/tools/xtensa-esp32s2-elf/esp-2021r2-8.4.0/xtensa-esp32s2-elf/bin:/Users/priyankahonnakasturi/.espressif/tools/xtensa-esp32s3-elf/esp-2021r2-8.4.0/xtensa-esp32s3-elf/bin:/Users/priyankahonnakasturi/.espressif/tools/riscv32-esp-elf/esp-2021r2-8.4.0/riscv32-esp-elf/bin:/Users/priyankahonnakasturi/.espressif/tools/esp32ulp-elf/2.28.51-esp-20191205/esp32ulp-elf-binutils/bin:/Users/priyankahonnakasturi/.espressif/tools/esp32s2ulp-elf/2.28.51-esp-20191205/esp32s2ulp-elf-binutils/bin:/Users/priyankahonnakasturi/.espressif/tools/openocd-esp32/v0.11.0-esp32-20211220/openocd-esp32/bin:/Users/priyankahonnakasturi/.espressif/python_env/idf4.4_py3.8_env/bin:/Users/priyankahonnakasturi/tools/esp-idf/tools:/Users/priyankahonnakasturi/chip-nordic-tools/nrfconnect/sdk-nrf/zephyr/scripts:/opt/homebrew/opt/openssl@3/bin:/Library/Frameworks/Python.framework/Versions/3.8/bin:/Library/Frameworks/Python.framework/Versions/3.10/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/appleinternal/bin:/opt/apple/bin:/usr/appleinternal/bin:/private/var/tanuki:/opt/xbs/bin:/Library/Apple/usr/bin

ESP-IDF v4.4

I tried executing again -> /Users/priyankahonnakasturi/tools/esp-idf/install.sh and . /Users/priyankahonnakasturi/tools/esp-idf/export.sh

still got the same error

dhrishi · 2022-11-09T15:48:46Z

@phonnakasturi-apple Can you try the same set of steps on a new terminal instance/window with ESP IDF v4.4.2 (that's the one we support with Matter) and see if it helps. Steps here

phonnakasturi-apple · 2022-11-09T16:34:58Z

@dhrishi With ESP IDF v4.4.2 and same steps;

I get:

idf.py build
The following Python requirements are not satisfied:
future>=0.15.2
idf-component-manager~=1.0
gdbgui==0.13.2.0
pygdbmi<=0.9.0.2
python-socketio<5
itsdangerous<2.1
kconfiglib==13.7.1
reedsolo>=1.5.3,<=1.5.4
bitstring>=3.1.6
ecdsa>=0.16.0
construct==2.10.54
To install the missing packages, please run "/Users/priyankahonnakasturi/tools/esp-idf/install.sh"
Diagnostic information:
    IDF_PYTHON_ENV_PATH: /Users/priyankahonnakasturi/.espressif/python_env/idf4.4_py3.8_env
    Python interpreter used: /Users/priyankahonnakasturi/connectedhomeip/.environment/pigweed-venv/bin/python
    Warning: python interpreter not running from IDF_PYTHON_ENV_PATH
    PATH: /Users/priyankahonnakasturi/connectedhomeip/third_party/pigweed/repo/out/host/host_tools:/Users/priyankahonnakasturi/connectedhomeip/.environment/pigweed-venv/bin:/Users/priyankahonnakasturi/connectedhomeip/.environment/cipd/packages/pigweed/bin:/Users/priyankahonnakasturi/connectedhomeip/.environment/cipd/packages/pigweed:/Users/priyankahonnakasturi/connectedhomeip/.environment/cipd:/Users/priyankahonnakasturi/tools/esp-idf/components/esptool_py/esptool:/Users/priyankahonnakasturi/tools/esp-idf/components/espcoredump:/Users/priyankahonnakasturi/tools/esp-idf/components/partition_table:/Users/priyankahonnakasturi/tools/esp-idf/components/app_update:/Users/priyankahonnakasturi/.espressif/tools/xtensa-esp32-elf/esp-2021r2-patch3-8.4.0/xtensa-esp32-elf/bin:/Users/priyankahonnakasturi/.espressif/tools/xtensa-esp32s2-elf/esp-2021r2-patch3-8.4.0/xtensa-esp32s2-elf/bin:/Users/priyankahonnakasturi/.espressif/tools/xtensa-esp32s3-elf/esp-2021r2-patch3-8.4.0/xtensa-esp32s3-elf/bin:/Users/priyankahonnakasturi/.espressif/tools/riscv32-esp-elf/esp-2021r2-patch3-8.4.0/riscv32-esp-elf/bin:/Users/priyankahonnakasturi/.espressif/tools/esp32ulp-elf/2.28.51-esp-20191205/esp32ulp-elf-binutils/bin:/Users/priyankahonnakasturi/.espressif/tools/esp32s2ulp-elf/2.28.51-esp-20191205/esp32s2ulp-elf-binutils/bin:/Users/priyankahonnakasturi/.espressif/tools/openocd-esp32/v0.11.0-esp32-20220411/openocd-esp32/bin:/Users/priyankahonnakasturi/.espressif/python_env/idf4.4_py3.8_env/bin:/Users/priyankahonnakasturi/tools/esp-idf/tools:/Users/priyankahonnakasturi/chip-nordic-tools/nrfconnect/sdk-nrf/zephyr/scripts:/opt/homebrew/opt/openssl@3/bin:/Library/Frameworks/Python.framework/Versions/3.8/bin:/Library/Frameworks/Python.framework/Versions/3.10/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/appleinternal/bin:/opt/apple/bin:/usr/appleinternal/bin:/private/var/tanuki:/opt/xbs/bin:/Library/Apple/usr/bin

ESP-IDF v4.4.2

dhrishi · 2022-11-10T09:31:42Z

@phonnakasturi-apple Is it possible for you to clean the connectedhomeip/.environment folder and re-run the bootstrap script: source scripts/bootstrap.sh? And then follow the above sequence again

phonnakasturi-apple · 2022-11-10T21:31:19Z

@dhrishi With SHA 599ad3b -> After clearing the ~/connectedhomeip/.environment and executing the following sequence; it worked and I no longer see missing modules error

cd ~/connectedhomeip
sudo rm -rf ~/connectedhomeip/.environment
git reset --hard 599ad3b
git clean -Xdf
git submodule update --init
export PKG_CONFIG_PATH="/opt/homebrew/opt/openssl@3/lib/pkgconfig"
source scripts/bootstrap.sh
source scripts/activate.sh
cd ~/tools/esp-idf
./install.sh
. ./export.sh
cd ~/connectedhomeip/examples/all-clusters-app/esp32
idf.py build

andy31415 · 2022-11-14T15:58:00Z

I think the issue is that our requirements for esp32 in bootstrap is linked to linux:

https://github.com/project-chip/connectedhomeip/blob/master/scripts/requirements.esp32.txt

sets everything around ; platform_machine != 'aarch64' and sys_platform == 'linux'

andy31415 · 2022-11-14T15:59:17Z

This seems to have changed in #22071 which claims that mac compile fails due to gdbui bringing in gevent and failing.

Maybe the solution would be to not restrict all of them but only restrict gdbgui.

andy31415 · 2022-11-14T16:00:56Z

@phonnakasturi-apple would you be able to test this out: remove all restrictions from requirements.esp32.txt and see if that makes it work in the sequence of "export.sh BEFORE activate.sh" ?

We may need to adjust some constraints as well.

phonnakasturi-apple · 2022-11-14T18:20:12Z

@andy31415 I tried with SHA b87a86b, after removing all the restrictions from requirements.esp32.txt and I still see the same issue.

Sequence of commands;

cd ~/tools/cd esp-idf
./install.sh
. ./export.sh
cd ~/connectedhomeip/
git reset --hard b87a86b
git clean -Xdf
git submodule update --init
sudo rm -rf ~/connectedhomeip/.environment
source scripts/activate.sh
cd ~/connectedhomeip/examples/all-clusters-app/esp32
idf.py build

Output:

esp32 % idf.py build
The following Python requirements are not satisfied:
future>=0.15.2
idf-component-manager~=1.0
gdbgui==0.13.2.0
pygdbmi<=0.9.0.2
python-socketio<5
itsdangerous<2.1
kconfiglib==13.7.1
reedsolo>=1.5.3,<=1.5.4
bitstring>=3.1.6
ecdsa>=0.16.0
construct==2.10.54
To install the missing packages, please run "/Users/priyankahonnakasturi/tools/esp-idf/install.sh"
Diagnostic information:
    IDF_PYTHON_ENV_PATH: /Users/priyankahonnakasturi/.espressif/python_env/idf4.4_py3.8_env
    Python interpreter used: /Users/priyankahonnakasturi/connectedhomeip/.environment/pigweed-venv/bin/python
    Warning: python interpreter not running from IDF_PYTHON_ENV_PATH
    PATH: /Users/priyankahonnakasturi/connectedhomeip/third_party/pigweed/repo/out/host/host_tools:/Users/priyankahonnakasturi/connectedhomeip/.environment/pigweed-venv/bin:/Users/priyankahonnakasturi/connectedhomeip/.environment/cipd/packages/pigweed/bin:/Users/priyankahonnakasturi/connectedhomeip/.environment/cipd/packages/pigweed:/Users/priyankahonnakasturi/connectedhomeip/.environment/cipd:/Users/priyankahonnakasturi/tools/esp-idf/components/esptool_py/esptool:/Users/priyankahonnakasturi/tools/esp-idf/components/espcoredump:/Users/priyankahonnakasturi/tools/esp-idf/components/partition_table:/Users/priyankahonnakasturi/tools/esp-idf/components/app_update:/Users/priyankahonnakasturi/.espressif/tools/xtensa-esp32-elf/esp-2021r2-patch3-8.4.0/xtensa-esp32-elf/bin:/Users/priyankahonnakasturi/.espressif/tools/xtensa-esp32s2-elf/esp-2021r2-patch3-8.4.0/xtensa-esp32s2-elf/bin:/Users/priyankahonnakasturi/.espressif/tools/xtensa-esp32s3-elf/esp-2021r2-patch3-8.4.0/xtensa-esp32s3-elf/bin:/Users/priyankahonnakasturi/.espressif/tools/riscv32-esp-elf/esp-2021r2-patch3-8.4.0/riscv32-esp-elf/bin:/Users/priyankahonnakasturi/.espressif/tools/esp32ulp-elf/2.28.51-esp-20191205/esp32ulp-elf-binutils/bin:/Users/priyankahonnakasturi/.espressif/tools/esp32s2ulp-elf/2.28.51-esp-20191205/esp32s2ulp-elf-binutils/bin:/Users/priyankahonnakasturi/.espressif/tools/openocd-esp32/v0.11.0-esp32-20220411/openocd-esp32/bin:/Users/priyankahonnakasturi/.espressif/python_env/idf4.4_py3.8_env/bin:/Users/priyankahonnakasturi/tools/esp-idf/tools:/Users/priyankahonnakasturi/chip-nordic-tools/nrfconnect/sdk-nrf/zephyr/scripts:/opt/homebrew/opt/openssl@3/bin:/Library/Frameworks/Python.framework/Versions/3.8/bin:/Library/Frameworks/Python.framework/Versions/3.10/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/appleinternal/bin:/opt/apple/bin:/usr/appleinternal/bin:/private/var/tanuki:/opt/xbs/bin:/Library/Apple/usr/bin

ESP-IDF v4.4.2

andy31415 · 2022-12-06T19:49:19Z

@phonnakasturi-apple I got this as well now and I think my scenario reduces to:

scripts/activate.sh will clear PW_ENVIRONMENT_ROOT
if you use some form of docker/podman and you have PW_ENVIRONMENT_ROOT set, idf will effectively ignore it and expect '.environment' to exist and if it does, it will use is as is (and if built for host, it will be ### wrong)

Is this what is happening maybe for you as well?
Could you:

echo $PW_ENVIRONMENT_ROOT on a fresh shell
do you have .environment ? what happens if you delete it?

phonnakasturi-apple · 2022-12-10T00:56:42Z

@andy31415
When I executed echo $PW_ENVIRONMENT_ROOT I did not get any output

echo $PW_ENVIRONMENT_ROOT

And after deleting the .environment, and executing the above mentioned steps I get the same error

cd examples/all-clusters-app/esp32 
priyankahonnakasturi@Priyankas-MacBook-Pro esp32 % idf.py build
The following Python requirements are not satisfied:
future>=0.15.2
idf-component-manager>=0.2.99-beta
gdbgui==0.13.2.0
pygdbmi<=0.9.0.2
python-socketio<5
kconfiglib==13.7.1
reedsolo>=1.5.3,<=1.5.4
bitstring>=3.1.6
construct==2.10.54
To install the missing packages, please run "/Users/priyankahonnakasturi/test/tools/esp-idf/install.sh"
Diagnostic information:
    IDF_PYTHON_ENV_PATH: /Users/priyankahonnakasturi/.espressif/python_env/idf4.4_py3.8_env
    Python interpreter used: /Users/priyankahonnakasturi/test/connectedhomeip/.environment/pigweed-venv/bin/python
    Warning: python interpreter not running from IDF_PYTHON_ENV_PATH
    PATH: /Users/priyankahonnakasturi/test/connectedhomeip/third_party/pigweed/repo/out/host/host_tools:/Users/priyankahonnakasturi/test/connectedhomeip/.environment/pigweed-venv/bin:/Users/priyankahonnakasturi/test/connectedhomeip/.environment/cipd/packages/pigweed/bin:/Users/priyankahonnakasturi/test/connectedhomeip/.environment/cipd/packages/pigweed:/Users/priyankahonnakasturi/test/connectedhomeip/.environment/cipd:/Users/priyankahonnakasturi/test/tools/esp-idf/components/esptool_py/esptool:/Users/priyankahonnakasturi/test/tools/esp-idf/components/espcoredump:/Users/priyankahonnakasturi/test/tools/esp-idf/components/partition_table:/Users/priyankahonnakasturi/test/tools/esp-idf/components/app_update:/Users/priyankahonnakasturi/.espressif/tools/xtensa-esp32-elf/esp-2021r2-8.4.0/xtensa-esp32-elf/bin:/Users/priyankahonnakasturi/.espressif/tools/xtensa-esp32s2-elf/esp-2021r2-8.4.0/xtensa-esp32s2-elf/bin:/Users/priyankahonnakasturi/.espressif/tools/xtensa-esp32s3-elf/esp-2021r2-8.4.0/xtensa-esp32s3-elf/bin:/Users/priyankahonnakasturi/.espressif/tools/riscv32-esp-elf/esp-2021r2-8.4.0/riscv32-esp-elf/bin:/Users/priyankahonnakasturi/.espressif/tools/esp32ulp-elf/2.28.51-esp-20191205/esp32ulp-elf-binutils/bin:/Users/priyankahonnakasturi/.espressif/tools/esp32s2ulp-elf/2.28.51-esp-20191205/esp32s2ulp-elf-binutils/bin:/Users/priyankahonnakasturi/.espressif/tools/openocd-esp32/v0.11.0-esp32-20211220/openocd-esp32/bin:/Users/priyankahonnakasturi/.espressif/python_env/idf4.4_py3.8_env/bin:/Users/priyankahonnakasturi/test/tools/esp-idf/tools:/Users/priyankahonnakasturi/chip-nordic-tools/nrfconnect/sdk-nrf/zephyr/scripts:/opt/homebrew/opt/openssl@3/bin:/Library/Frameworks/Python.framework/Versions/3.8/bin:/Library/Frameworks/Python.framework/Versions/3.10/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/appleinternal/bin:/opt/apple/bin:/usr/appleinternal/bin:/private/var/tanuki:/opt/xbs/bin:/Library/Apple/usr/bin

ESP-IDF v4.4.2

woody-apple · 2023-01-25T19:42:37Z

Closing per @kean-apple, given this looks like things have been resolved

bzbarsky-apple assigned andy31415 Nov 3, 2022

andy31415 removed their assignment Nov 3, 2022

andy31415 added darwin esp32 build issue labels Nov 3, 2022

Damian-Nordic mentioned this issue Nov 14, 2022

Fix intermittent BLE connection failures #23598

Merged

woody-apple closed this as completed Jan 25, 2023

shubhamdp mentioned this issue Feb 2, 2023

Unable to compile all-clusters-app on M1 Mac #24734

Closed

shubhamdp mentioned this issue Feb 10, 2023

[ESP32] Fix host setup failure on mac and some build docs #24973

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to flash M5 board -> {ModuleNotFoundError: No module named 'lark'} #23477

Unable to flash M5 board -> {ModuleNotFoundError: No module named 'lark'} #23477

phonnakasturi-apple commented Nov 3, 2022

andy31415 commented Nov 3, 2022 •

edited

Loading

bzbarsky-apple commented Nov 3, 2022

andy31415 commented Nov 3, 2022

phonnakasturi-apple commented Nov 3, 2022 •

edited

Loading

shubhamdp commented Nov 4, 2022

phonnakasturi-apple commented Nov 4, 2022

phonnakasturi-apple commented Nov 4, 2022

phonnakasturi-apple commented Nov 7, 2022

phonnakasturi-apple commented Nov 7, 2022

andy31415 commented Nov 8, 2022

phonnakasturi-apple commented Nov 8, 2022

andy31415 commented Nov 9, 2022

phonnakasturi-apple commented Nov 9, 2022

dhrishi commented Nov 9, 2022

phonnakasturi-apple commented Nov 9, 2022

dhrishi commented Nov 10, 2022

phonnakasturi-apple commented Nov 10, 2022 •

edited

Loading

andy31415 commented Nov 14, 2022

andy31415 commented Nov 14, 2022

andy31415 commented Nov 14, 2022

phonnakasturi-apple commented Nov 14, 2022

andy31415 commented Dec 6, 2022

phonnakasturi-apple commented Dec 10, 2022

woody-apple commented Jan 25, 2023

Unable to flash M5 board -> {ModuleNotFoundError: No module named 'lark'} #23477

Unable to flash M5 board -> {ModuleNotFoundError: No module named 'lark'} #23477

Comments

phonnakasturi-apple commented Nov 3, 2022

Reproduction steps

Bug prevalence

GitHub hash of the SDK that was being used

Platform

Platform Version(s)

Anything else?

andy31415 commented Nov 3, 2022 • edited Loading

bzbarsky-apple commented Nov 3, 2022

andy31415 commented Nov 3, 2022

phonnakasturi-apple commented Nov 3, 2022 • edited Loading

shubhamdp commented Nov 4, 2022

phonnakasturi-apple commented Nov 4, 2022

phonnakasturi-apple commented Nov 4, 2022

phonnakasturi-apple commented Nov 7, 2022

phonnakasturi-apple commented Nov 7, 2022

andy31415 commented Nov 8, 2022

phonnakasturi-apple commented Nov 8, 2022

andy31415 commented Nov 9, 2022

phonnakasturi-apple commented Nov 9, 2022

dhrishi commented Nov 9, 2022

phonnakasturi-apple commented Nov 9, 2022

dhrishi commented Nov 10, 2022

phonnakasturi-apple commented Nov 10, 2022 • edited Loading

andy31415 commented Nov 14, 2022

andy31415 commented Nov 14, 2022

andy31415 commented Nov 14, 2022

phonnakasturi-apple commented Nov 14, 2022

andy31415 commented Dec 6, 2022

phonnakasturi-apple commented Dec 10, 2022

woody-apple commented Jan 25, 2023

andy31415 commented Nov 3, 2022 •

edited

Loading

phonnakasturi-apple commented Nov 3, 2022 •

edited

Loading

phonnakasturi-apple commented Nov 10, 2022 •

edited

Loading