Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: use clang-13 to build bpf programs #171

Merged
merged 2 commits into from
Aug 9, 2022

Conversation

willfindlay
Copy link
Contributor

This needs more testing on additional kernels before we merge. For example, while testing locally I noticed some verifier errors on 5.4 that we should fix first.

@willfindlay willfindlay temporarily deployed to release-clang June 15, 2022 18:26 Inactive
context: .
file: ./Dockerfile.clang
push: true
platforms: linux/amd64
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add linux/arm64 here?

@willfindlay
Copy link
Contributor Author

This PR is currently blocked by a verifier failure on 5.4 for the large exec prog. I'll post the logs here tomorrow.

@willfindlay willfindlay force-pushed the pr/willfindlay/use-upstream-clang-new branch from 4a118d1 to da194d9 Compare July 13, 2022 14:07
willfindlay added a commit that referenced this pull request Jul 13, 2022
Addressing Chance's feedback [0].

[0]: #171 (comment)

Signed-off-by: William Findlay <[email protected]>
@willfindlay willfindlay force-pushed the pr/willfindlay/use-upstream-clang-new branch from 0d27e30 to da194d9 Compare July 13, 2022 14:24
@willfindlay willfindlay temporarily deployed to release-clang July 13, 2022 14:24 Inactive
@willfindlay
Copy link
Contributor Author

NB: this is blocked on #221

@olsajiri
Copy link
Contributor

tested on 5.4 and 4.19 seems ok

note I see following fails on 4.19:

--- FAIL: TestKprobeObjectWriteReadHostNs (42.65s)
--- FAIL: TestKprobeObjectWriteRead (42.57s)
--- FAIL: TestKprobeObjectWriteReadNsOnly (44.32s)

but I see them with the old clang as well, so I'll check on those separately

@olsajiri olsajiri force-pushed the pr/willfindlay/use-upstream-clang-new branch from da194d9 to 2a092c8 Compare July 28, 2022 13:17
@olsajiri olsajiri temporarily deployed to release-clang July 28, 2022 13:17 Inactive
@olsajiri olsajiri force-pushed the pr/willfindlay/use-upstream-clang-new branch from 2a092c8 to 3da9fa9 Compare August 1, 2022 17:31
@olsajiri olsajiri temporarily deployed to release-clang August 1, 2022 17:31 Inactive
@olsajiri olsajiri force-pushed the pr/willfindlay/use-upstream-clang-new branch from 3da9fa9 to 2fbae30 Compare August 1, 2022 19:44
@olsajiri olsajiri temporarily deployed to release-clang August 1, 2022 19:44 Inactive
@olsajiri olsajiri force-pushed the pr/willfindlay/use-upstream-clang-new branch from 2fbae30 to 137967d Compare August 2, 2022 09:15
@olsajiri olsajiri force-pushed the pr/willfindlay/use-upstream-clang-new branch from 137967d to 4fb8b70 Compare August 3, 2022 07:13
@olsajiri olsajiri temporarily deployed to release-clang August 3, 2022 07:13 Inactive
@olsajiri olsajiri marked this pull request as ready for review August 3, 2022 08:52
@olsajiri olsajiri requested a review from a team as a code owner August 3, 2022 08:52
@olsajiri olsajiri requested a review from sharlns August 3, 2022 08:52
@willfindlay
Copy link
Contributor Author

@olsajiri this can probably be done as a follow-up but we should look into adding a linux/amd64 target as @chancez requested. I think it's needed to finally realize our ARM dreams.

@olsajiri
Copy link
Contributor

olsajiri commented Aug 3, 2022

ok, would be great if we could do that as follow up ;-) I'll put that on my todo list

Copy link
Contributor

@kkourt kkourt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just some minor comments.

Dockerfile.clang Outdated Show resolved Hide resolved
Dockerfile.clang Outdated Show resolved Hide resolved
.github/workflows/build-clang-image.yaml Show resolved Hide resolved
.github/workflows/gotests.yml Show resolved Hide resolved
@olsajiri olsajiri force-pushed the pr/willfindlay/use-upstream-clang-new branch from 4fb8b70 to 96905cd Compare August 6, 2022 21:08
@willfindlay
Copy link
Contributor Author

Can't review my own PR but looks gucci @olsajiri once all of @kkourt's comments are addressed (I think we can probably leave the clang-version tags for later but whatever you all think is best).

@willfindlay
Copy link
Contributor Author

One other thought: Maybe the "right" thing to do is to have the Dockerfile and deployment job live in a separate repo. I'm not in love with the way it works now where we have to push to build the new image then push again to update the CI jobs to use the new builder (my fault since I wrote the first draft, but my intention was always to move it over to something like this and maybe it makes sense to just do it now). What do you think @kkourt @olsajiri?

@olsajiri olsajiri force-pushed the pr/willfindlay/use-upstream-clang-new branch from 96905cd to 275f994 Compare August 8, 2022 16:14
Use apt-get instead of apt to silence warnings.
Remove tools-install target because it no longer exists.

This re-introduces a previously reverted commit.

Signed-off-by: Jiri Olsa <[email protected]>
Signed-off-by: William Findlay <[email protected]>
Introduce a new Docker image that provides a suitable clang version for compiling our BPF
programs.

This re-introduces a previously reverted commit.

Signed-off-by: William Findlay <[email protected]>
@olsajiri olsajiri force-pushed the pr/willfindlay/use-upstream-clang-new branch from 275f994 to b315284 Compare August 8, 2022 21:01
@kkourt
Copy link
Contributor

kkourt commented Aug 9, 2022

One other thought: Maybe the "right" thing to do is to have the Dockerfile and deployment job live in a separate repo. I'm not in love with the way it works now where we have to push to build the new image then push again to update the CI jobs to use the new builder (my fault since I wrote the first draft, but my intention was always to move it over to something like this and maybe it makes sense to just do it now). What do you think @kkourt @olsajiri?

If I understand correctly, this is about having to do a first version of PR to figure out the image tag, and then update the PR with the produced image tag?

No strong opinions either way. Having a different repo is sensible, but having everything in the same repo is simpler. Fwiw, this is also how cilium is doing it: https://docs.cilium.io/en/latest/contributing/development/images/ (cilium is different in that it requires clang in its runtime image as well.)

$(CONTAINER_ENGINE) stop ${id}
.PHONY: image-clang
image-clang:
$(CONTAINER_ENGINE) build -f Dockerfile.clang --build-arg VERSION=1:13.0.1-2ubuntu2 -t "cilium/clang:${DOCKER_IMAGE_TAG}" .
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reading https://docs.cilium.io/en/latest/contributing/development/images/, how about we use cilium/tetragon-clang for the image name? I think it would be less confusing for folk

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@willfindlay please address this comment if needed with follow up.

@jrfastab jrfastab merged commit d38baa4 into main Aug 9, 2022
@jrfastab jrfastab deleted the pr/willfindlay/use-upstream-clang-new branch August 9, 2022 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants