Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error while building: ValueError: I/O operation on closed pipe! (IDFGH-12390) #13418

Closed
3 tasks done
masterli19 opened this issue Mar 19, 2024 · 2 comments
Closed
3 tasks done

Error while building: ValueError: I/O operation on closed pipe! (IDFGH-12390) #13418

masterli19 opened this issue Mar 19, 2024 · 2 comments
Assignees
@fhrbata
Labels
Resolution: Done Issue is done internally Status: Done Issue is done internally Type: Bug bugs in IDF

Comments

@masterli19
Copy link

Answers checklist.

  • I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
  • I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
  • I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

V5.2.1

Operating System used.

Windows

How did you build your project?

Command line with idf.py

If you are using Windows, please specify command line type.

PowerShell

What is the expected behavior?

Continue to build whole project after generating partion table

What is the actual behavior?

After the partion table was generated, the build crashed.
When I typed Ctrl+C to end the build, it showed something like:

Aborted!
Exception ignored in: <function BaseSubprocessTransport.del at 0x000002B2E5A1C4A0>
Traceback (most recent call last):
File "asyncio\base_subprocess.py", line 125, in del
File "asyncio\base_subprocess.py", line 78, in repr
File "asyncio\proactor_events.py", line 80, in repr
File "asyncio\windows_utils.py", line 102, in fileno
ValueError: I/O operation on closed pipe
Exception ignored in: <function _ProactorBasePipeTransport.del at 0x000002B2E5A1DB20>
Traceback (most recent call last):
File "asyncio\proactor_events.py", line 116, in del
File "asyncio\proactor_events.py", line 80, in repr
File "asyncio\windows_utils.py", line 102, in fileno
ValueError: I/O operation on closed pipe
Exception ignored in: <function _ProactorBasePipeTransport.del at 0x000002B2E5A1DB20>
Traceback (most recent call last):
File "asyncio\proactor_events.py", line 116, in del
File "asyncio\proactor_events.py", line 80, in repr
File "asyncio\windows_utils.py", line 102, in fileno
ValueError: I/O operation on closed pipe

I had reinstalled the esp-idf, and tried v5.1.2, 5.2.0, and 5.2.1, never succeed.
I had removed esp-adf components from my computer, it might be the course.

Steps to reproduce.

  1. Step
  2. Step
  3. Step
    ...
  4. Install esp-idf v5.1.2
  5. install esp-adf from github or gitee
  6. remove esp-adf components
  7. reinstall esp-idf v5.1.2 or v5.2.1, or other v5 branches
  8. enter any project directory and run idf.py build

Build or installation Logs.

PS D:\ESP32_Workspace\hello_world> idf.py build
Executing action: all (aliases: build)
Running cmake in directory D:\ESP32_Workspace\hello_world\build
Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DPYTHON=D:\Espressif\python_env\idf5.2_py3.11_env\Scripts\python.exe -DESP_PLATFORM=1 -DCCACHE_ENABLE=1 D:\ESP32_Workspace\hello_world"...
-- IDF_TARGET is not set, guessed 'esp32' from sdkconfig 'D:/ESP32_Workspace/hello_world/sdkconfig'
-- Found Git: D:/Espressif/tools/idf-git/2.43.0/cmd/git.exe (found version "2.43.0.windows.1")
-- ccache will be used for faster recompilation
-- The C compiler identification is GNU 13.2.0
-- The CXX compiler identification is GNU 13.2.0
-- The ASM compiler identification is GNU
-- Found assembler: D:/Espressif/tools/xtensa-esp-elf/esp-13.2.0_20230928/xtensa-esp-elf/bin/xtensa-esp32-elf-gcc.exe
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: D:/Espressif/tools/xtensa-esp-elf/esp-13.2.0_20230928/xtensa-esp-elf/bin/xtensa-esp32-elf-gcc.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: D:/Espressif/tools/xtensa-esp-elf/esp-13.2.0_20230928/xtensa-esp-elf/bin/xtensa-esp32-elf-g++.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- git rev-parse returned 'fatal: not a git repository (or any of the parent directories): .git'
-- Could not use 'git describe' to determine PROJECT_VER.
-- Building ESP-IDF components for target esp32
-- Project sdkconfig file D:/ESP32_Workspace/hello_world/sdkconfig
-- Compiler supported targets: xtensa-esp-elf
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of time_t
-- Check size of time_t - done
-- Found Python3: D:/Espressif/python_env/idf5.2_py3.11_env/Scripts/python.exe (found version "3.11.2") found components: Interpreter
-- Performing Test C_COMPILER_SUPPORTS_WFORMAT_SIGNEDNESS
-- Performing Test C_COMPILER_SUPPORTS_WFORMAT_SIGNEDNESS - Success
-- App "hello_world" version: 1
-- Adding linker script D:/ESP32_Workspace/hello_world/build/esp-idf/esp_system/ld/memory.ld
-- Adding linker script D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_system/ld/esp32/sections.ld.in
-- Adding linker script D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_rom/esp32/ld/esp32.rom.ld
-- Adding linker script D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_rom/esp32/ld/esp32.rom.api.ld
-- Adding linker script D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_rom/esp32/ld/esp32.rom.libgcc.ld
-- Adding linker script D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_rom/esp32/ld/esp32.rom.newlib-data.ld
-- Adding linker script D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_rom/esp32/ld/esp32.rom.syscalls.ld
-- Adding linker script D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld
-- Adding linker script D:/Espressif/frameworks/esp-idf-v5.2.1/components/soc/esp32/ld/esp32.peripherals.ld
-- Components: app_trace app_update bootloader bootloader_support bt cmock console cxx driver efuse esp-tls esp_adc esp_app_format esp_bootloader_format esp_coex esp_common esp_eth esp_event esp_gdbstub esp_hid esp_http_client esp_http_server esp_https_ota esp_https_server esp_hw_support esp_lcd esp_local_ctrl esp_mm esp_netif esp_netif_stack esp_partition esp_phy esp_pm esp_psram esp_ringbuf esp_rom esp_system esp_timer esp_wifi espcoredump esptool_py fatfs freertos hal heap http_parser idf_test ieee802154 json log lwip main mbedtls mqtt newlib nvs_flash nvs_sec_provider openthread partition_table perfmon protobuf-c protocomm pthread sdmmc soc spi_flash spiffs tcp_transport ulp unity usb vfs wear_levelling wifi_provisioning wpa_supplicant xtensa
-- Component paths: D:/Espressif/frameworks/esp-idf-v5.2.1/components/app_trace D:/Espressif/frameworks/esp-idf-v5.2.1/components/app_update D:/Espressif/frameworks/esp-idf-v5.2.1/components/bootloader D:/Espressif/frameworks/esp-idf-v5.2.1/components/bootloader_support D:/Espressif/frameworks/esp-idf-v5.2.1/components/bt D:/Espressif/frameworks/esp-idf-v5.2.1/components/cmock D:/Espressif/frameworks/esp-idf-v5.2.1/components/console D:/Espressif/frameworks/esp-idf-v5.2.1/components/cxx D:/Espressif/frameworks/esp-idf-v5.2.1/components/driver D:/Espressif/frameworks/esp-idf-v5.2.1/components/efuse D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp-tls D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_adc D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_app_format D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_bootloader_format D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_coex D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_common D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_eth D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_event D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_gdbstub D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_hid D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_http_client D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_http_server D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_https_ota D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_https_server D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_hw_support D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_lcd D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_local_ctrl D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_mm D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_netif D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_netif_stack D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_partition D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_phy D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_pm D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_psram D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_ringbuf D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_rom D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_system D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_timer D:/Espressif/frameworks/esp-idf-v5.2.1/components/esp_wifi D:/Espressif/frameworks/esp-idf-v5.2.1/components/espcoredump D:/Espressif/frameworks/esp-idf-v5.2.1/components/esptool_py D:/Espressif/frameworks/esp-idf-v5.2.1/components/fatfs D:/Espressif/frameworks/esp-idf-v5.2.1/components/freertos D:/Espressif/frameworks/esp-idf-v5.2.1/components/hal D:/Espressif/frameworks/esp-idf-v5.2.1/components/heap D:/Espressif/frameworks/esp-idf-v5.2.1/components/http_parser D:/Espressif/frameworks/esp-idf-v5.2.1/components/idf_test D:/Espressif/frameworks/esp-idf-v5.2.1/components/ieee802154 D:/Espressif/frameworks/esp-idf-v5.2.1/components/json D:/Espressif/frameworks/esp-idf-v5.2.1/components/log D:/Espressif/frameworks/esp-idf-v5.2.1/components/lwip D:/ESP32_Workspace/hello_world/main D:/Espressif/frameworks/esp-idf-v5.2.1/components/mbedtls D:/Espressif/frameworks/esp-idf-v5.2.1/components/mqtt D:/Espressif/frameworks/esp-idf-v5.2.1/components/newlib D:/Espressif/frameworks/esp-idf-v5.2.1/components/nvs_flash D:/Espressif/frameworks/esp-idf-v5.2.1/components/nvs_sec_provider D:/Espressif/frameworks/esp-idf-v5.2.1/components/openthread D:/Espressif/frameworks/esp-idf-v5.2.1/components/partition_table D:/Espressif/frameworks/esp-idf-v5.2.1/components/perfmon D:/Espressif/frameworks/esp-idf-v5.2.1/components/protobuf-c D:/Espressif/frameworks/esp-idf-v5.2.1/components/protocomm D:/Espressif/frameworks/esp-idf-v5.2.1/components/pthread D:/Espressif/frameworks/esp-idf-v5.2.1/components/sdmmc D:/Espressif/frameworks/esp-idf-v5.2.1/components/soc D:/Espressif/frameworks/esp-idf-v5.2.1/components/spi_flash D:/Espressif/frameworks/esp-idf-v5.2.1/components/spiffs D:/Espressif/frameworks/esp-idf-v5.2.1/components/tcp_transport D:/Espressif/frameworks/esp-idf-v5.2.1/components/ulp D:/Espressif/frameworks/esp-idf-v5.2.1/components/unity D:/Espressif/frameworks/esp-idf-v5.2.1/components/usb D:/Espressif/frameworks/esp-idf-v5.2.1/components/vfs D:/Espressif/frameworks/esp-idf-v5.2.1/components/wear_levelling D:/Espressif/frameworks/esp-idf-v5.2.1/components/wifi_provisioning D:/Espressif/frameworks/esp-idf-v5.2.1/components/wpa_supplicant D:/Espressif/frameworks/esp-idf-v5.2.1/components/xtensa
-- Configuring done
-- Generating done
-- Build files have been written to: D:/ESP32_Workspace/hello_world/build
Running ninja in directory D:\ESP32_Workspace\hello_world\build
Executing "ninja all"...
[3/918] Generating ../../partition_table/partition-table.bin
Partition table binary generated. Contents:
*******************************************************************************
# ESP-IDF Partition Table
# Name, Type, SubType, Offset, Size, Flags
nvs,data,nvs,0x9000,24K,
phy_init,data,phy,0xf000,4K,
factory,app,factory,0x10000,1M,

Aborted!
Exception ignored in: <function BaseSubprocessTransport.__del__ at 0x000002B2E5A1C4A0>
Traceback (most recent call last):
  File "asyncio\base_subprocess.py", line 125, in __del__
  File "asyncio\base_subprocess.py", line 78, in __repr__
  File "asyncio\proactor_events.py", line 80, in __repr__
  File "asyncio\windows_utils.py", line 102, in fileno
ValueError: I/O operation on closed pipe
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000002B2E5A1DB20>
Traceback (most recent call last):
  File "asyncio\proactor_events.py", line 116, in __del__
  File "asyncio\proactor_events.py", line 80, in __repr__
  File "asyncio\windows_utils.py", line 102, in fileno
ValueError: I/O operation on closed pipe
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000002B2E5A1DB20>
Traceback (most recent call last):
  File "asyncio\proactor_events.py", line 116, in __del__
  File "asyncio\proactor_events.py", line 80, in __repr__
  File "asyncio\windows_utils.py", line 102, in fileno
ValueError: I/O operation on closed pipe
PS D:\ESP32_Workspace\hello_world>

More Information.

before the esp-adf installing, everything went well on esp-idf v5.1.2
@masterli19 masterli19 added the Type: Bug bugs in IDF label Mar 19, 2024
@espressif-bot espressif-bot added the Status: Opened Issue is new label Mar 19, 2024
@github-actions github-actions bot changed the title Error while building: ValueError: I/O operation on closed pipe! Error while building: ValueError: I/O operation on closed pipe! (IDFGH-12390) Mar 19, 2024
@dobairoland
Copy link
Collaborator

We will take a look at solving/improving the error message. But that won't change the fact of the build being broken.

Have you tried to ccache -C and idf.py fullclean or your project?

@espressif-bot espressif-bot added Status: Reviewing Issue is being reviewed and removed Status: Opened Issue is new labels Apr 4, 2024
@masterli19
Copy link
Author

masterli19 commented Apr 7, 2024 via email

@espressif-bot espressif-bot added Status: Done Issue is done internally Resolution: NA Issue resolution is unavailable Resolution: Done Issue is done internally and removed Status: Reviewing Issue is being reviewed Resolution: NA Issue resolution is unavailable labels Apr 12, 2024
espressif-bot pushed a commit that referenced this issue Apr 23, 2024
@fhrbata
fix: exit gracefully when process started via asyncio is terminated
4bc2985 
Currently when process is started through asyncio Runner and it is termited
e.g. with SIGINT(ctrl+c) a traceback is printed instead of gracefully
exit.

Exception ignored in: <function BaseSubprocessTransport.__del__ at 0x7fe980970900>
Traceback (most recent call last):
  File "/usr/lib64/python3.12/asyncio/base_subprocess.py", line 129, in __del__
    self.close()
  File "/usr/lib64/python3.12/asyncio/base_subprocess.py", line 107, in close
    proto.pipe.close()
  File "/usr/lib64/python3.12/asyncio/unix_events.py", line 568, in close
    self._close(None)
  File "/usr/lib64/python3.12/asyncio/unix_events.py", line 592, in _close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/usr/lib64/python3.12/asyncio/base_events.py", line 793, in call_soon
    self._check_closed()
  File "/usr/lib64/python3.12/asyncio/base_events.py", line 540, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

This is caused because asyncio Runner context in asyncio.run is closing the event
loop and if exception is unhandled in coroutine(run_command) the transport is not
closed before the even loop is closed and we get RuntimeError: Event loop is closed
in the transport __del__ function because it's trying to use the closed
even loop.

Let's catch asyncio.CancelledError in case the process we are trying to
read from is terminated, print message, let the asyncio finish and exit
gracefully.

Closes #13418

Signed-off-by: Frantisek Hrbata <[email protected]>
espressif-bot pushed a commit that referenced this issue May 11, 2024
@fhrbata
fix: exit gracefully when process started via asyncio is terminated
0de3c3b 
Currently when process is started through asyncio Runner and it is termited
e.g. with SIGINT(ctrl+c) a traceback is printed instead of gracefully
exit.

Exception ignored in: <function BaseSubprocessTransport.__del__ at 0x7fe980970900>
Traceback (most recent call last):
  File "/usr/lib64/python3.12/asyncio/base_subprocess.py", line 129, in __del__
    self.close()
  File "/usr/lib64/python3.12/asyncio/base_subprocess.py", line 107, in close
    proto.pipe.close()
  File "/usr/lib64/python3.12/asyncio/unix_events.py", line 568, in close
    self._close(None)
  File "/usr/lib64/python3.12/asyncio/unix_events.py", line 592, in _close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/usr/lib64/python3.12/asyncio/base_events.py", line 793, in call_soon
    self._check_closed()
  File "/usr/lib64/python3.12/asyncio/base_events.py", line 540, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

This is caused because asyncio Runner context in asyncio.run is closing the event
loop and if exception is unhandled in coroutine(run_command) the transport is not
closed before the even loop is closed and we get RuntimeError: Event loop is closed
in the transport __del__ function because it's trying to use the closed
even loop.

Let's catch asyncio.CancelledError in case the process we are trying to
read from is terminated, print message, let the asyncio finish and exit
gracefully.

Closes #13418

Signed-off-by: Frantisek Hrbata <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@fhrbata fhrbata

Labels
Resolution: Done Issue is done internally Status: Done Issue is done internally Type: Bug bugs in IDF
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@fhrbata @dobairoland @espressif-bot @masterli19