migrate all commmunity supported boards to coreboot 4.13 (Except KGPE-D16 which stays to coreboot 4.11) #1015
Conversation
tlaurion
force-pushed
the
maximized_boards-coreboot-4_13
branch
from
f901a7b
to
b2b4707
Compare
tlaurion
changed the title
tlaurion
force-pushed
the
maximized_boards-coreboot-4_13
branch
from
b3542d6
to
9b59af0
Compare
|
TODO: Review results of with/without blobs set for xx20 and xx30 boards Results are weird and coreboot configs are still inconsistent (some with some without) across boards' coreboot used configs. With: Without: With:
Without:
EDIT: It doesn't change a thing for x230. Will remove for all boards since microcode is in as all other required files, with or without statement and size diff is locally accountable for "dirty" being added in config alongside of the config having more caracters and being uncompressed. |
tlaurion
force-pushed
the
maximized_boards-coreboot-4_13
branch
3 times, most recently
from
dca9991
to
7870545
Compare
|
All boards now building with cryptsetup2 and coreboot config CONFIG_USE_BLOBS is not set for xx20 and xx30. |
tlaurion
force-pushed
the
maximized_boards-coreboot-4_13
branch
from
7870545
to
b8696ac
Compare
|
rebased on master |
tlaurion
force-pushed
the
maximized_boards-coreboot-4_13
branch
from
b8696ac
to
6d559de
Compare
|
rebased on master with build optimization in to compare speeds of fresh build and rebuilds |
tlaurion
force-pushed
the
maximized_boards-coreboot-4_13
branch
from
37d20af
to
b9468f5
Compare
|
b9468f5 reduced compile time of https://app.circleci.com/pipelines/github/tlaurion/heads/758/workflows/f42445f7-258b-472f-8a60-1a87b491cbed by 45 minutes (2h43 vs 3h30+) Fresh builds doing the cache will result in even greater gains since based on all reused binaries for x86 to be based on cache passed to workspaces. And not building coreboot 4.8.1 is also a big win. |
|
From #692: I would love to have at least one board owner for each platform to report results of externally flashing artifacts of https://app.circleci.com/pipelines/github/tlaurion/heads/758/workflows/f42445f7-258b-472f-8a60-1a87b491cbed if possible? |
|
Edit: direct links now invalid. See artifacts for latest commit. Direct link to roms from past successfully builds for external flashing (note below for people having IFD unlocked so internal flashing can be done. But please only do if you have an external reprogrammer available, and backup first so you can restore!) Jargonmaximized (without HOTP): Requires you to have a GPG smartcard (standard for Heads) to detach sign /boot digest (sha256sum) content. If flashed internally with ME unlocked and IFD unlocked, a maximized-hotp: As maximized build above, where that rom additionally requires users to have a Librem Key/Nitrokey Pro/Nitrokey Storage USB Security dongle to do remote attestation. Only flash those images if you have such devices and remember your GPG Admin PIN, required to seal measurements into your dongle's smartcard. Heads permits you to factory reset those devices at boot if no user's public key is found injected in the ROM. You can also inject previously generated public key, or flash the rom keeping previous settings if flashed internally from previous builds. Legacy (only for xx30 builds: t430 and x230): Doesn't require initial unlocking of IFD regions. Doesn't require either neutering ME. Consequently less space is available in BIOS's coreboot IFD region (CBFS region of coreboot config) compared to freed ME space under maximized boards. Legacy boards consequently have some features deactivated and some UX regressions. For example, dropbear (ssh) is not in. Neither is FBWhiptail, but console based Whiptail is enforced (using gui-init, not as flashy as FBWhiptail is with a framebuffer (which toolstack is more space expensive.) Please read the new board configurations in this PR. ( @flammit and others: your input is important here. This is the choice I made to continue having those boards with minimal regression testing under CircleCI for each commit.) xx20 (needs more testing, this is where #693 #1004 seem to have occurred and its not clear that its completely fixed as of now. I DO NOT OWN A xx20! (t420 nor x220))t420-hotp-maximized: t420-maximized: x220-hotp-maximized: x220-maximized: xx30x230-hotp-maximized (already tested multiple times. T430 counterpart needs to be tested though @daringer @jans23 others, see below). Legacy x230 (not maximized builds. Not requiring to neuter ME per instructions:https://osresearch.net/Clean-the-ME-firmware/ nor unlocking IFD per same instructions.): Legacy x230-hotp (not maximized builds. Not requiring to neuter ME per instructions:https://osresearch.net/Clean-the-ME-firmware/ nor unlocking IFD per same instructions.): t430-hotp-maximized: t430-maximized: Legacy t430 (not maximized. Not requiring to neuter ME per instructions:https://osresearch.net/Clean-the-ME-firmware/ nor unlocking IFD per same instructions.): Please report back! |
|
x220 (xx20): @techge @eganonoa @Thrilleratplay @BlackMaria Please tag me here if you test the roms. Particularly: |
|
Clean rebuild requested on CircleCI (changing CACHE_VERSION on CI) to test effectiveness of b9468f5 Of course, that will change the hashes of some binaries, per #1008 and other reproducibility opened ticket. |
|
I am willing to test my x220, but I am currently short of time and do not know yet, when I will get to it... |
|
For USE_OPTION_TABLE, STATIC_OPTION_TABLE and on t430 : #944 (comment) The reasoning behind adding this and hardcoding higher gfx_uma_size configs should be discussed and added if needed, inside of this PR or later on to cover such use case,
But:
@jans23 @alex-nitrokey and Nitrokey T430 hardware support team? Needed for T430? Your call! |
|
Note that builds from cache now builds under 3 hours for all boards dfined under CircleCI builder config. This means, once again, that unless a module definition changes, the biggest cache matching created modules digest is reused to build. If all modules are consistent (no modification on those modules in past commit for current build, meaning only scripts have changed or a rebuild was asked) like in the previous build, a CircleCI build takes 2h30 instead of 3h40 because most complete cache including board reused built binaries and libraries is being downloaded and used prior of attempting to build anything. This is due by not having to build 3 versions of coreboot (4.8.1 is deprecated and moved to 4.13 here for all boards but KGPE-D16 still being based on coreboot 4.11) and having the most complex board (x230-hotp-maximized includes most andof what is reused by others) being built first where libraries and binaries being in the cache are reused instead of being recompiled from scratch since available, for each subsequent boards being built with same dependencies and just not included if not requested from board config. |
- xx30 legacy boards (x230, x230-flash, t430, t430-flash) now rely also on coreboot 4.13
- DOWNSIDE: x230 and t430 legacy boards now rely on WHIPTAIL (NOT FBWhiptail) to have enough space to fit under 7mb)
- xx20 boards moved to 4.13 (no need of xx20-flash boards here since single SPI boards with 7.5mb useable since blobs scripts are required)
- DOWNSIDE: all xx20 boards now have dropbear deactivated, while still having ethernet driver in.
- qemu-coreboot and qemu-coreboot-fbwhiptail switched to coreboot 4.13 WITHOUT TPM SUPPORT (with cryptsetup 2.x support)
- DOWNSIDE:
- coreboot-qemu board CBFS_SIZE=0x700000 -> 0x750000
- coreboot-qemu-fbwhiptail CBFS_SIZE=0x750000 -> 0x780000
- CircleCi build recipe removes 4.8.1 boards altogether
- KGPE-D16 workstation is used as new base build to save workspace layer (we removed one workspace layer)
- Removing one workspace layer will save approx 2 hours of build time on fresh builds
- Removing one coreboot version will save us approx 2 hours of build time on fresh builds
- KGPE-D16 will stay to coreboot 4.11 until forward notice.
- All other board configs SHOULD be built on latest coreboot versions
…ree more building time.
…lobs/t420/* presence.
…O: change when 4.13 boards bumped to 4.14)
… x230-hotp-verification board
- me_cleaner downloaded from https://github.com/corna/me_cleaner/blob/43612a630c79f3bc6f2653bfe90dfe0b7b137e08/me_cleaner.py - placed under xx30 blobs dir - CircleCI uses it locally without downloading it everytime (me_cleaner hasn<t changed since 2018)
…tps://support.circleci.com/hc/en-us/articles/4410707277083-Context-deadline-exceeded-after-1-hour-Build-timed-out-Free-tier-only- Readd linuxboot#984 without cache Add kgpe-d16 musl-cross target prior of having kgpe-d16 depend on musl-cross target (To try to have musl-cross step successfull under 1h CircleCI new limit) CircleCI: add a subcommand that can follow a target (to build musl-cross-make now and coreboot version specific musl-cross later) Output of hashes is now optional 29/11/2021 CircleCI public information available states parallelization of up to 30 jobs at a time. Let's play - We first build heads musl-cross-make and persist (passing musl-cross-make into next job) - We then build per coreboot version board with coreboot make statement only and persist (passing musl-cross-make + coreboot's musl-cross buildstack) - We then build per coreboot version board (reusing past build musl-cross-make and coreboot's version musl-cross buildstack) Remove 4.11 boards for the moment to test only build time and parallelization
CircleCI: We currently drop coreboot 4.11 builds. - There is a file missing in the builds. Not sure why/how this is happening src/soc/intel/fsp_broadwell_de/romstage/romstage.c:41:10: fatal error: build.h: No such file or directory Example:https://app.circleci.com/pipelines/github/tlaurion/heads/877/workflows/7d0248d2-459c-42ad-b741-8fd56a75d527/jobs/2487 - kgpe-d16_workstation building for all GPUs is unfortunately taking too much time to build (40 minutes). - Not sure why, but it seems that the kernel build paralellization is not working for 4.11 while it works for 4.13 Makefile: Uncomment MAKE_JOBS which passes the number of jobs to numbers cores by default and --max-load of 16 CircleCI: Remove CPUS statement to use Makefile default modules/newt: force build with one make job, otherwise there is a race condition in module which fails randomly expecting build modules. (TODO: FIX) Interestingly, building all coreboot 4.13 boards is happening on a clean commit just above 1h limit. More details: - CircleCI changed job build time to a maximum of 1h each. - CircleCI now permits parallelization of 30 jobs - 6000 build minutes a month. - Still waiting for osresearch/heads CircleCI project to be unlocked (currently not recognized as open source project?!)
tlaurion
force-pushed
the
maximized_boards-coreboot-4_13
branch
from
5e4309c
to
8f9ccae
Compare
|
@0xdd7fq the builds' roms are boards' artifacts, built for for latest successfully built commit 8f9ccae, which is accessible by clicking on the green mark of each commit id being built by CircleCI. From there you see links for boards that were built, by name of boards which can be accessed from there. I would appreciate t420 and t430 board owners to retest roms: Example, all being done without being logged into Github nor CircleCI as you can see in the pictures (extracting links is not so fun where people can download for themselves!) |
|
@icequbes1 report oem-factory-reset issue on the t430, which I cannot replicate, here: #1063 Someone else has that issue? I cannot replicate on x230 for commit 8f9ccae Notes:
|
@shamen123 your T430 has a IGPU only or also a dGPU? That fix was also applied to t420 coreboot configs. |
@tlaurion just to clarify: my board is a dGPU t430, not iGPU-- the nvidia dGPU can be found under a second heatsink pad attached to the CPU heatsink/fan assembly. I believe @icequbes1 has iGPU only, but I cannot speak for them. I did not have any issue with the OEM reset either using the new build based on the hotp-maximized coreboot config you produced in #1057 . It is now running perfectly for me. |
|
Now only missing a t420 board owner report from either @alexmaloteaux @natterangell @akfhasodh! |
Thanks @walliams for your feedback. I updated #692 accordingly and can now confirm that t430 boards, with dGPU or iGPU only should now all function under #1015 which should be finally merged soon. If there is no report from t420 owners, life goes on and the problems reported will have to be corrected after bug reports. |
|
Flashed t420-hotp-maximized from |
|
For clarity: iGPU |
|
@natterangell updated #692 accordingly. Waiting for a t420 dGPU report until tomorrow and/or merging anyway |
|
Fixes #1057 |
ROMS to be tested by board owners
#1015 (comment)
History
Supersedes
Still unanswered and needs testing and confirmation of need (@daringer @jans23 @alex-nitrokey ?)
Edit: Tested by @nitrosimon here
Includes (will rebase once merged)
Bug fixes needed to install QubesOS 4.1 from verified ISO under Heads (a dd'ed ISO over USB will boot and install from boot from USB menu)
Todos:
Before merging
ROM testing (needs feedback)
Per commited users
EDIT : tested by @BlackMaria
t430s (@Siproqu)Edit: timeout reached. Will be seperate PR.
Rom testing needing users to stand up:
Reduce/synthetise