Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support empty extra_flags_per_feature #576

Merged
merged 3 commits into from
Sep 4, 2024

Conversation

hofbi
Copy link
Contributor

@hofbi hofbi commented Aug 26, 2024

Used the wrong serialization in #575. Since this is not really used yet, we set always set an empty dict for now.

Closes #578

@hofbi hofbi requested a review from benradf as a code owner August 26, 2024 15:56
@jtszalay
Copy link

jtszalay commented Aug 26, 2024

I tried to use this (as I was still getting an error after PR #575:

ERROR: ../bazel-user-root/output/external/nixpkgs_config_cc/BUILD.bazel:154:31: syntax error at ',': expected expression
ERROR: ../bazel-user-root/output/external/nixpkgs_config_cc/BUILD.bazel: no such target '@@nixpkgs_config_cc//:cc-compiler-k8': target 'cc-compiler-k8' not declared in package '' defined by ../bazel-user-root/output/external/nixpkgs_config_cc/BUILD.bazel
ERROR: ../bazel-user-root/output/external/rules_go~/BUILD.bazel:86:17: Target '@@rules_go~//:cgo_context_data' depends on toolchain '@@nixpkgs_config_cc//:cc-compiler-k8', which cannot be found: no such target '@@nixpkgs_config_cc//:cc-compiler-k8': target 'cc-compiler-k8' not declared in package '' defined by ../bazel-user-root/output/external/nixpkgs_config_cc/BUILD.bazel'

It's correct that my extra flags get gen'd as:

    extra_flags_per_feature = ,
)

Previously they were:

extra_flags_per_feature = [],
ERROR: ../bazel-user-root//output/external/nixpkgs_config_cc/BUILD.bazel:92:20: @@nixpkgs_config_cc//:local: expected value of type 'dict(string, list(string))' for attribute 'extra_flags_per_feature' in 'cc_toolchain_config' rule, but got [] (list)
ERROR: ../bazel-user-root/output/external/rules_go~/BUILD.bazel:86:17: Target '@@rules_go~//:cgo_context_data' depends on toolchain '@@nixpkgs_config_cc//:cc-compiler-k8', which cannot be found: Target '@@nixpkgs_config_cc//:cc-compiler-k8' contains an error and its package is in error'

It seems to be expecting something of shape {} not []

@aherrmann
Copy link
Member

Interesting, the template built into Bazel includes this

    extra_flags_per_feature = %{extra_flags_per_feature},

other fields that should be set via get_starlark_list look like this instead

    coverage_link_flags = [%{coverage_link_flags}],

FWIW, Bazel itself also uses repr. @hofbi In which way does it fail without this PR?

@hofbi
Copy link
Contributor Author

hofbi commented Aug 27, 2024

Without this PR, I get the same error as reported by @jtszalay

ERROR: ../bazel-user-root/output/external/nixpkgs_config_cc/BUILD.bazel:154:31: syntax error at ',': expected expression
ERROR: ../bazel-user-root/output/external/nixpkgs_config_cc/BUILD.bazel: no such target '@@nixpkgs_config_cc//:cc-compiler-k8': target 'cc-compiler-k8' not declared in package '' defined by ../bazel-user-root/output/external/nixpkgs_config_cc/BUILD.bazel
ERROR: ../bazel-user-root/output/external/rules_go~/BUILD.bazel:86:17: Target '@@rules_go~//:cgo_context_data' depends on toolchain '@@nixpkgs_config_cc//:cc-compiler-k8', which cannot be found: no such target '@@nixpkgs_config_cc//:cc-compiler-k8': target 'cc-compiler-k8' not declared in package '' defined by ../bazel-user-root/output/external/nixpkgs_config_cc/BUILD.bazel'

I will try to fully reproduce and test with a local_override but might only get to to next week.

@avdv
Copy link
Member

avdv commented Aug 27, 2024

Interesting, the template built into Bazel includes this

    extra_flags_per_feature = %{extra_flags_per_feature},

Yes, but In Bazel, the extra_flags_per_feature variable is a dict (see https://github.com/bazelbuild/bazel/blob/e88a7d982411e3d6bbfb658b88e76944ea5ed720/tools/cpp/unix_cc_configure.bzl#L559C5-L559C33) and using repr is just writing that dict to the BUILD file.

So, the EXTRA_FLAGS_PER_FEATURE variable in cc.nix should be an associative array and be written to the CC_TOOLCHAIN_INFO file in a form that makes it possible to decode it as a dict again. Arguably, since the variable never has a meaningful value currently, always writing {} would be the easiest thing to fix this.

@aherrmann
Copy link
Member

Thank you @avdv, good catch!

Arguably, since the variable never has a meaningful value currently, always writing {} would be the easiest thing to fix this.

Yes, that's an easy quick fix.

So, the EXTRA_FLAGS_PER_FEATURE variable in cc.nix should be an associative array and be written to the CC_TOOLCHAIN_INFO file in a form that makes it possible to decode it as a dict again.

This may be a good motivation to finally replace the CC_TOOLCHAIN_INFO format by json. This format was introduced before Bazel gained json.decode.

@hofbi hofbi force-pushed the fix-toolchain-serialization branch from 52f9694 to e35d136 Compare September 3, 2024 12:42
@@ -221,7 +221,7 @@ def _nixpkgs_cc_toolchain_config_impl(repository_ctx):
"%{coverage_compile_flags}": get_starlark_list(info.coverage_compile_flags),
"%{coverage_link_flags}": get_starlark_list(info.coverage_link_flags),
"%{supports_start_end_lib}": repr(info.supports_start_end_lib),
"%{extra_flags_per_feature}": repr(info.extra_flags_per_feature),
"%{extra_flags_per_feature}": repr(dict(info.extra_flags_per_feature)),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dict() expects an iterable of pairs as its first argument. info.extra_flags_per_feature is a list of strings. This will only work out if this list is empty. So, this is a bit complicated way of always writing out {}.

Is there a way forward for the EXTRA_FLAGS_PER_FEATURE in cc.nix to be useful? Otherwise I'd rather remove it until there is a use case for it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you suggest we simply do

Suggested change
"%{extra_flags_per_feature}": repr(dict(info.extra_flags_per_feature)),
"%{extra_flags_per_feature}": repr({}),

and remove the rest until there is a concrete usecase?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, even just using "{}" would do.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@hofbi hofbi changed the title Fix serialization in toolchain file Support empty extra_flags_per_feature Sep 4, 2024
Copy link
Member

@avdv avdv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @hofbi , LGTM!

@avdv avdv added the merge-queue merge on green CI label Sep 4, 2024
@hofbi
Copy link
Contributor Author

hofbi commented Sep 4, 2024

@Mergifyio update

Copy link
Contributor

mergify bot commented Sep 4, 2024

update

❌ Mergify doesn't have permission to update

For security reasons, Mergify can't update this pull request. Try updating locally.
GitHub response: refusing to allow a GitHub App to create or update workflow .github/workflows/workflow.yaml without workflows permission

@mergify mergify bot merged commit 8c9f6be into tweag:master Sep 4, 2024
13 checks passed
@mergify mergify bot removed the merge-queue merge on green CI label Sep 4, 2024
@hofbi hofbi deleted the fix-toolchain-serialization branch September 4, 2024 11:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Default value for extra_flags_per_feature CC configuration is invalid
4 participants