Skip to content

Fix: include file contents in custom_sdkconfig hash computation #441

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
antoinecellerier wants to merge 1 commit into from
Closed

Fix: include file contents in custom_sdkconfig hash computation #441

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions builder/frameworks/arduino.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ def framework_lib_dir(self):
@property
def sdk_dir(self):
if self._sdk_dir is None:
if self.framework_lib_dir is None:
return None
self._sdk_dir = fs.to_unix_path(
str(Path(self.framework_lib_dir) / self.chip_variant / "include")
)
Expand Down Expand Up @@ -277,6 +279,23 @@ def safe_remove_sdkconfig_files():
# Custom SDKConfig check
if config.has_option(current_env_section, "custom_sdkconfig"):
entry_custom_sdkconfig = env.GetProjectOption("custom_sdkconfig")
# When custom_sdkconfig references a file, include its contents in the
# value used for hash computation. Otherwise, editing the file doesn't
# change the hash and stale libs are silently reused.
# Combine file content with inline options (both are applied by the build).
for line in entry_custom_sdkconfig.splitlines():
line = line.strip()
if line.startswith("file://"):
file_ref = line[7:]
file_path = file_ref if isabs(file_ref) else join(project_dir, file_ref)
if exists(file_path):
try:
with open(file_path, 'r') as f:
file_content = f.read()
entry_custom_sdkconfig = file_content + "\n" + entry_custom_sdkconfig
except IOError:
pass
break
flag_custom_sdkconfig = True

if board_sdkconfig:
Expand Down
13 changes: 10 additions & 3 deletions builder/frameworks/espidf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -779,11 +779,18 @@ def write_sdkconfig_file(idf_config_flags, checksum_source):
# Convert to list for processing
idf_config_list = [line for line in idf_config_flags.splitlines() if line.strip()]

# Write final configuration file with checksum
# Write final configuration file with checksum.
# Include resolved file content (not just the raw "file://..." reference)
# so that editing the referenced file changes the hash and triggers
# recompilation. Combine file content + inline options to match what
# build_idf_config_flags() produces.
custom_sdk_config_flags = ""
if config.has_option("env:" + env["PIOENV"], "custom_sdkconfig"):
custom_sdk_config_flags = env.GetProjectOption("custom_sdkconfig").rstrip("\n") + "\n"

raw = env.GetProjectOption("custom_sdkconfig")
resolved = load_custom_sdkconfig_file()
# Combine resolved file content with raw inline options (both are applied)
custom_sdk_config_flags = ((resolved + "\n") if resolved else "") + raw.rstrip("\n") + "\n"
Comment on lines +782 to +792

@coderabbitai coderabbitai Bot Mar 24, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Keep this new hash expansion scoped to file://, or align arduino.py with it.

load_custom_sdkconfig_file() also resolves http:// / https:// entries, but builder/frameworks/arduino.py::matching_custom_sdkconfig() still hashes the literal URL string. That means URL-backed custom_sdkconfig values will now miss the cached hash every run and keep forcing reinstall/recompile. If this PR is only fixing file://, narrow the checksum input here to local-file entries; otherwise both files should share the same resolver.

💡 Minimal fix 
     if config.has_option("env:" + env["PIOENV"], "custom_sdkconfig"):
         raw = env.GetProjectOption("custom_sdkconfig")
-        resolved = load_custom_sdkconfig_file()
+        resolved = ""
+        for line in raw.splitlines():
+            line = line.strip()
+            if not line.startswith("file://"):
+                continue
+            file_ref = line[7:]
+            file_path = file_ref if os.path.isabs(file_ref) else str(Path(PROJECT_DIR) / file_ref)
+            if os.path.exists(file_path):
+                try:
+                    with open(file_path, "r") as f:
+                        resolved = f.read()
+                except IOError:
+                    pass
+            break
         # Combine resolved file content with raw inline options (both are applied)
         custom_sdk_config_flags = ((resolved + "\n") if resolved else "") + raw.rstrip("\n") + "\n"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# Write final configuration file with checksum.
# Include resolved file content (not just the raw "file://..." reference)
# so that editing the referenced file changes the hash and triggers
# recompilation. Combine file content + inline options to match what
# build_idf_config_flags() produces.
custom_sdk_config_flags = ""
if config.has_option("env:" + env["PIOENV"], "custom_sdkconfig"):
custom_sdk_config_flags = env.GetProjectOption("custom_sdkconfig").rstrip("\n") + "\n"
raw = env.GetProjectOption("custom_sdkconfig")
resolved = load_custom_sdkconfig_file()
# Combine resolved file content with raw inline options (both are applied)
custom_sdk_config_flags = ((resolved + "\n") if resolved else "") + raw.rstrip("\n") + "\n"
# Write final configuration file with checksum.
# Include resolved file content (not just the raw "file://..." reference)
# so that editing the referenced file changes the hash and triggers
# recompilation. Combine file content + inline options to match what
# build_idf_config_flags() produces.
custom_sdk_config_flags = ""
if config.has_option("env:" + env["PIOENV"], "custom_sdkconfig"):
raw = env.GetProjectOption("custom_sdkconfig")
resolved = ""
for line in raw.splitlines():
line = line.strip()
if not line.startswith("file://"):
continue
file_ref = line[7:]
file_path = file_ref if os.path.isabs(file_ref) else str(Path(PROJECT_DIR) / file_ref)
if os.path.exists(file_path):
try:
with open(file_path, "r") as f:
resolved = f.read()
except IOError:
pass
break
# Combine resolved file content with raw inline options (both are applied)
custom_sdk_config_flags = ((resolved + "\n") if resolved else "") + raw.rstrip("\n") + "\n"
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@builder/frameworks/espidf.py` around lines 782 - 792, The combined-config
hashing currently uses resolved content from load_custom_sdkconfig_file() for
all schemes (file/http/https), but arduino.py's matching_custom_sdkconfig()
still hashes the literal URL, causing cache misses for URL-backed configs; to
fix, change the logic that builds custom_sdk_config_flags (the
custom_sdk_config_flags variable in builder/frameworks/espidf.py where raw =
env.GetProjectOption("custom_sdkconfig") and resolved =
load_custom_sdkconfig_file()) so it only expands/resolves and concatenates
file:// entries (local file content) into the checksum input, leaving http://
and https:// entries as their original literal strings, or alternatively update
matching_custom_sdkconfig() in builder/frameworks/arduino.py to resolve URLs the
same way—pick one approach and implement it so both sides use the same
representation for URL-backed custom_sdkconfig entries.


write_sdkconfig_file(idf_config_list, custom_sdk_config_flags)


Expand Down