Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interrupting kernel build with Ctrl+C doesn't kill all CC processes #2

Closed
xairy opened this issue Aug 10, 2020 · 3 comments
Closed

Comments

@xairy
Copy link
Contributor

xairy commented Aug 10, 2020

The kernel still keeps building in the background. Restarting the build at this point might lead to flaky errors, I assume due to the conflict of two separately running build processes.

@a13xp0p0v
Copy link
Owner

Huh, thank you very much for this report.
I can reproduce it.
Python process is killed by CTRL+C, but the docker container is not:

[a13x@hackbase kernel-build-containers]$ sudo docker ps
CONTAINER ID        IMAGE                          COMMAND                  CREATED             STATUS              PORTS               NAMES
ccbfcd4f49d3        kernel-build-container:gcc-8   "make O=~/out/ -j5 2…"   7 minutes ago       Up 7 minutes                            zealous_swanson

I'll fix it.

a13xp0p0v added a commit that referenced this issue Nov 2, 2020
This commit fixes the issue #2.

Stopping of the docker container using SIGTERM is not fast and stable.
So I decided to use `docker run` with `--cidfile` for storing
the container id.

Now I catch the KeyboardInterrupt in python and execute
finish_container.sh, which kills the container and removes the cidfile.

In case of normal build without KeyboardInterrupt, finish_container.sh
checks that the container is happily finished on its own and removes
the cidfile as well.
@a13xp0p0v
Copy link
Owner

Hi @xairy,

Look how it works now:

$ ./make_linux.py -a aarch64 -k ~/develop_local/linux/experiment.config -s ~/develop_local/linux/linux -o ~/develop_local/linux/build_out -c gcc-8 -- -j5
[+] Going to build the Linux kernel for aarch64
[+] Using "/home/a13x/develop_local/linux/experiment.config" as kernel config
[+] Using "/home/a13x/develop_local/linux/linux" as Linux kernel sources directory
[+] Using "/home/a13x/develop_local/linux/build_out" as build output directory
[+] Going to build with: gcc-8
[+] Have additional arguments for 'make': -j5

=== Building with gcc-8 ===
Output subdirectory for this build: /home/a13x/develop_local/linux/build_out/experiment__aarch64__gcc-8
Output subdirectory already exists, use it (no cleaning!)
Copy kconfig to output subdirectory as ".config"
Going to save build log to "build_log.txt" in output subdirectory
Create additional arguments for cross-compilation: ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-
Run the container: bash ./start_container.sh gcc-8 /home/a13x/develop_local/linux/linux /home/a13x/develop_local/linux/build_out/experiment__aarch64__gcc-8 -n make O=~/out/ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j5 2>&1
    Hey, we gonna use sudo for running docker
    Starting "kernel-build-container:gcc-8"
    Source code directory "/home/a13x/develop_local/linux/linux" is mounted at "~/src"
    Build output directory "/home/a13x/develop_local/linux/build_out/experiment__aarch64__gcc-8" is mounted at "~/out"
    Run docker in NON-interactive mode
    Gonna run "make O=~/out/ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j5 2>&1"
    
[sudo] пароль для a13x: 
    make[1]: Entering directory '/home/a13x/out'
      GEN     Makefile
    scripts/kconfig/conf  --syncconfig Kconfig
      GEN     Makefile
      CALL    /home/a13x/src/scripts/atomic/check-atomics.sh
      CALL    /home/a13x/src/scripts/checksyscalls.sh
      CC      arch/arm64/mm/dma-mapping.o
      CHK     include/generated/compile.h
      UPD     include/generated/compile.h
      CC      init/version.o
      CC      kernel/time/alarmtimer.o
      CC      mm/shmem.o
      OBJCOPY arch/arm64/kernel/efi-entry.stub.o
      AR      init/built-in.a
      CC      arch/arm64/net/bpf_jit_comp.o
      CC      arch/arm64/kernel/mte.o
      CC      arch/arm64/mm/extable.o
      CC      kernel/time/posix-timers.o
      CC      arch/arm64/mm/fault.o
^C[!] Got keyboard interrupt, stopping build process...
Kill the container and remove the container id file:
    Hey, we gonna use sudo for running docker
    Search "container.id" file in build output directory "/home/a13x/develop_local/linux/build_out/experiment__aarch64__gcc-8"
    OK, "container.id" file exists, removing it
    Killing the docker container 0463bef60bc7b295dbd9c84a5639693b5dd7768113037d4a8c0bd10e3062ad4c
    0463bef60bc7b295dbd9c84a5639693b5dd7768113037d4a8c0bd10e3062ad4c
    Container 0463bef60bc7b295dbd9c84a5639693b5dd7768113037d4a8c0bd10e3062ad4c is killed
Finished with the container
[!] Early exit
$ sudo docker ps 
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
$

Do you like it?
Can you try it please?

@xairy
Copy link
Contributor Author

xairy commented Nov 2, 2020

Yay, it works! =)

I've noticed a few times that killing make manually with -9 corrupts some kernel build files and the build doesn't finish correctly when restarted. I don't know whether this will be an issue with this approach, I'll report it separately if so. For now I think this is resolved.

Thank you!

@xairy xairy closed this as completed Nov 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants