mbedtls: cmake function link_to_source should not hard-link on Windows (IDFGH-8033)

[N3xed]

Environment

• Development Kit: none
• Kit version (for WroverKit/PicoKit/DevKitC): N/A
• Module or chip used: N/A
• IDF version (run git describe --tags to find it): v5.1-dev-63-gd622bcfd46
• Build System: CMake
• Compiler version (run xtensa-esp32-elf-gcc --version to find it): 11.2.0
• Operating System: Windows
• (Windows only) environment type: Plain Command Prompt
• Using an IDE?: vscode
• Power Supply: N/A

Problem Description

Compilation of esp-idf fails if the project is on a different drive than the esp-idf itself. This is because the function link_to_source tries to create a hard-link on Windows (symlinks require Administrator privileges) and this doesn't work across filesystem boundaries (see Debug Logs).

This function already existed before esp-idf-v5.0 but was not used in the relevant code path.

Expected Behavior

Compilation should succeed.

Actual Behavior

Compilation fails.

Steps to reproduce

1. Create an esp-idf project on a different drive than the esp-idf is installed on.
2. Compilation fails.

Solution

The function should copy the files on Windows:

Diff of `esp-idf/components/mbedtls/mbedtls/CMakeLists.txt`

diff --git a/CMakeLists.txt b/CMakeLists.txt
index eae3b6024affc68b7fce635cd6472326a56cb61b..928196e2e40535fdaf41db905630d388d23ceb75 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -118,34 +118,23 @@ endif()
 
 # Create a symbolic link from ${base_name} in the binary directory
 # to the corresponding path in the source directory.
+#
+# Note: Copies the file(s) on Windows.
 function(link_to_source base_name)
-    # Get OS dependent path to use in `execute_process`
-    if (CMAKE_HOST_WIN32)
-        #mklink is an internal command of cmd.exe it can only work with \
-        string(REPLACE "/" "\\" link "${CMAKE_CURRENT_BINARY_DIR}/${base_name}")
-        string(REPLACE "/" "\\" target "${CMAKE_CURRENT_SOURCE_DIR}/${base_name}")
-    else()
-        set(link "${CMAKE_CURRENT_BINARY_DIR}/${base_name}")
-        set(target "${CMAKE_CURRENT_SOURCE_DIR}/${base_name}")
-    endif()
+    set(link "${CMAKE_CURRENT_BINARY_DIR}/${base_name}")
+    set(target "${CMAKE_CURRENT_SOURCE_DIR}/${base_name}")
 
     if (NOT EXISTS ${link})
         if (CMAKE_HOST_UNIX)
-            set(command ln -s ${target} ${link})
-        else()
-            if (IS_DIRECTORY ${target})
-                set(command cmd.exe /c mklink /j ${link} ${target})
-            else()
-                set(command cmd.exe /c mklink /h ${link} ${target})
-            endif()
-        endif()
-
-        execute_process(COMMAND ${command}
-            RESULT_VARIABLE result
-            ERROR_VARIABLE output)
+            execute_process(COMMAND ln -s ${target} ${link}
+                RESULT_VARIABLE result
+                ERROR_VARIABLE output)
 
-        if (NOT ${result} EQUAL 0)
-            message(FATAL_ERROR "Could not create symbolic link for: ${target} --> ${output}")
+            if (NOT ${result} EQUAL 0)
+                message(FATAL_ERROR "Could not create symbolic link for: ${target} --> ${output}")
+            endif()
+        else()
+            file(COPY ${target} DESTINATION ${link})
         endif()
     endif()
 endfunction(link_to_source)

Debug Logs

Build output

-- Found Git: C:/Users/n3xed/scoop/apps/git/2.33.1.windows.1/mingw64/bin/git.exe (found version "2.33.1.windows.1") 
-- The C compiler identification is GNU 11.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Users/n3xed/.espressif/tools/riscv32-esp-elf/esp-2022r1-RC1-11.2.0/riscv32-esp-elf/bin/riscv32-esp-elf-gcc.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Checking Python dependencies...
Python requirements are satisfied.
Skipping the download of C:\Users\n3xed\.espressif\espidf.constraints.v5.1.txt because it was downloaded recently.
Constraint file: C:\Users\n3xed\.espressif\espidf.constraints.v5.1.txt
Requirement files:
 - C:\Users\n3xed\.espressif\esp-idf\master\tools\requirements\requirements.core.txt
Python being checked: C:\Users\n3xed\.espressif\python_env\idf5.1_py3.10_env\Scripts\python.exe
-- Project sdkconfig file D:/dev/esp-idf-test/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-5ed947d2b5b0c5f5/out/sdkconfig
Loading defaults file D:/dev/esp-idf-test/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-5ed947d2b5b0c5f5/out/gen-sdkconfig.defaults...
Loading defaults file D:/dev/esp-idf-test/sdkconfig.defaults...
-- The CXX compiler identification is GNU 11.2.0
-- The ASM compiler identification is GNU
-- Found assembler: C:/Users/n3xed/.espressif/tools/riscv32-esp-elf/esp-2022r1-RC1-11.2.0/riscv32-esp-elf/bin/riscv32-esp-elf-gcc.exe
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Users/n3xed/.espressif/tools/riscv32-esp-elf/esp-2022r1-RC1-11.2.0/riscv32-esp-elf/bin/riscv32-esp-elf-g++.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of time_t
-- Check size of time_t - done
-- Found Python3: C:/Users/n3xed/.espressif/python_env/idf5.1_py3.10_env/Scripts/python.EXE (found version "3.10.0") found components: Interpreter 
-- Performing Test C_COMPILER_SUPPORTS_WFORMAT_SIGNEDNESS
-- Performing Test C_COMPILER_SUPPORTS_WFORMAT_SIGNEDNESS - Success
CMake Error at C:/Users/n3xed/.espressif/esp-idf/master/components/mbedtls/mbedtls/CMakeLists.txt:148 (message):
  Could not create symbolic link for:
  C:\Users\n3xed\.espressif\esp-idf\master\components\mbedtls\mbedtls\library\error.c
  --> Das System kann die Datei nicht auf ein anderes Laufwerk verschieben.

Call Stack (most recent call first):
  C:/Users/n3xed/.espressif/esp-idf/master/components/mbedtls/mbedtls/library/CMakeLists.txt:162 (link_to_source)

Hopefully, this is the right location for this issue.

[igrr]

Linking the upstream issue: Mbed-TLS/mbedtls#5751 and the earlier issue report: #8787

[mahavirj]

@N3xed I recommend that we track this through existing #8787. Closing this issue.