feat(vmm): MPS support when ACPI is not supported/available

[rinor]

Well probably on Firecracker it works (kind of) because afaik Firecracker does not implement/support ACPI, which is used by nanos for power handling and CPU(s) discovery.

Hence nanos, from what I understand, being unable to process MP detection without ACPI support, (couldn't find anything related to MP Floating Pointer Structure/MP Configuration Table parsing), falls back to single CPU by default (no matter the VM config you provide to it)

Firecracker nanos

we pass 4 vcpus, but still nanos starts just 1

  "machine-config": {
    "vcpu_count": 4,
    "cpu_template": "None",
    "smt": false,
    "mem_size_mib": 2000,
    "track_dirty_pages": true
  }

nanos output:

INIT: init_service
INIT: physical memory:
INIT:  [0000000000400000, 000000007ceff000)
INIT: parsing cmdline
INIT: in init_service_new_stack
INIT: init_hwrand
INIT: init cpu features
INIT: calling kernel_runtime_init
ACPI: find_rsdp: could not find valid RSDP
ACPI: AcpiInitializeTables returned 5
APIC: walking MADT table...
APIC: MADT not found, detecting apic interface...
x2APIC: x2APIC detected
APIC: using x2APIC interface
x2APIC: per cpu init, writing 0xc00
x2APIC: read from reg 0x803
x2APIC:  -> read 0x50014
x2APIC: read from reg 0x802
x2APIC:  -> read 0x0
x2APIC: apic id 0, apic ver 50014
x2APIC: write to reg 0x80f, val 0x120
x2APIC: write to reg 0x835, val 0x10000
x2APIC: write to reg 0x836, val 0x10000
x2APIC: write to reg 0x837, val 0x21
x2APIC: write to reg 0x832, val 0x40024
INIT: KVM detected
warning: ACPI MADT not found, default to 1 processor
INIT: init_mxcsr
INIT: starting APs
INIT: started 1 total processors
INIT: hypervisor undetected or HVM platform; registering all PCI drivers...
x2APIC: read from reg 0x802
x2APIC:  -> read 0x0
x2APIC: read from reg 0x802
x2APIC:  -> read 0x0
ACPI: AcpiEnableSubsystem returned 2
x2APIC: write to reg 0x80b, val 0x0
x2APIC: write to reg 0x80b, val 0x0
x2APIC: write to reg 0x80b, val 0x0
x2APIC: write to reg 0x80b, val 0x0
x2APIC: write to reg 0x80b, val 0x0
x2APIC: write to reg 0x80b, val 0x0

Originally posted by @rinor in #1838 (comment)

[rinor]

Just to be clear, atm I don't care much about Firecracker support, at least not for production and/or long running services. Have yet to test, but I suspect that also qemu microvm requires this. While I understand that this may not be the target for nanos, I still think it has a valid use case. I.e: we could use this in dev environments onprem or in clouds that support nested virtualization (do,gcp,...) where booting the whole dev infrastructure in milliseconds is a nice dev experience.

[francescolavra]

MP tables are a legacy, x86-only mechanism for reporting CPU information, and you are right, Nanos does not implement MP table detection. We could add MP table support, but even Firecracker is likely going to drop MP table support and transition to ACPI: there is a feature request for this, and a PR has already been opened.

[rinor]

Just for information, some work was done/completed at firecracker-microvm/firecracker#3155 although not yet available on main/release branches. It needs firecracker-microvm/firecracker#4073 or similar before moving to main branch.

edit: this is the correct pr firecracker-microvm/firecracker#4428 and nanos works as expected

  "machine-config": {
    "vcpu_count": 4,
  }

2024-04-24T12:23:48.509652483 [anonymous-instance:main] Running Firecracker v1.8.0-dev
2024-04-24T12:23:48.540906510 [anonymous-instance:main] Artificially kick devices.
2024-04-24T12:23:48.541003839 [anonymous-instance:fc_vcpu 0] Received a VcpuEvent::Resume message with immediate_exit enabled. immediate_exit was disabled before proceeding
2024-04-24T12:23:48.541028852 [anonymous-instance:fc_vcpu 1] Received a VcpuEvent::Resume message with immediate_exit enabled. immediate_exit was disabled before proceeding
2024-04-24T12:23:48.541054060 [anonymous-instance:fc_vcpu 3] Received a VcpuEvent::Resume message with immediate_exit enabled. immediate_exit was disabled before proceeding
2024-04-24T12:23:48.541126165 [anonymous-instance:fc_vcpu 2] Received a VcpuEvent::Resume message with immediate_exit enabled. immediate_exit was disabled before proceeding
2024-04-24T12:23:48.541162264 [anonymous-instance:main] Successfully started microvm that was configured from one single json
en1: assigned 10.0.2.15
en1: assigned FE80::A8FC:FF:FE00:1

[francescolavra]

It seems to me that the PRs you mentioned are unrelated to ACPI. Anyway, I just saw another PR (firecracker-microvm/firecracker#4428) which finally adds ACPI support to Firecracker, and has been merged into the main branch last month.

[rinor]

thanks, somehow I mixed the links (was investigating some other unrelated stuff around pvh), but yeah firecracker-microvm/firecracker#4428 is the one

[rinor]

for reference firecracker-microvm/firecracker#4428 was merged and released at https://github.com/firecracker-microvm/firecracker/releases/tag/v1.8.0