AArch64: let ubuntu 20.04+ boot from rust hypervisor firmware #262

jongwu · 2023-07-06T05:09:26Z

There are several issues block ubuntu 20.04+ boot from rust hypervisor firmware:

Grub or UEFI Shim may large enough and overlaps the firmware code region;
UEFI Shim or Grub file name length longer than 11 can't be expressed correctly: for example: "BOOTAA64.CSV" will be carried as "BOOTAA64CSV" in rust hypervisor firmware;
EFI memory allocations can be ran out;
Pass full fdt to OS blocks ACPI initialization;

This PR try to fix them one by one:

Move code start address upward to enlarge the memory region used for loading Grub/Shim;
Fixup fat file name if needed;
Bump allocations max number;
Don't pass fdt in system configuration table;

After above fix, ubuntu 20.04+ can boot from rust hypervisor firmware. Test pass for ubuntu 18.04, 20.04 and 22.04.

Fixes: #261

src/main.rs

rbradford · 2023-07-06T09:04:21Z

aarch64-unknown-none.ld

@@ -6,7 +6,7 @@ ENTRY(ram64_start)
    ACPI:   [0x4020_0000-0x403f_ffff)
    kernel: [0x4048_0000-]
   The stack start is at the end of the DRAM region. */
-ram_min = 0x40480000;
+ram_min = 0x40600000;

 SECTIONS
 {


I don't understand how this gives more room for GRUB. This is file determines the layout of the ELF executable that is compiled from the project. Your commit message seems to imply that the EFI payload is loaded into the memory just below the RHF executable code. I think this is a bit different to how we do it elsewhere. If you want to keep it the same it would be better to.

Make ram_min = 0x4000000 the true start of RAM and then add a payload_start/payload_end region to the linker file to save the memory and push back the code region.

@retrage Perhaps you can clarify on this?

Hi @rbradford -, In current design, 0x40400000 ~ 0x40480000 is reserved for RHF payload. 0x40000000 ~0x40400000 is reserved for DTB and ACPI.

So what does that have to do with the EFI payload?

I think it would be better to load the kernel/grub so that it doesn't overwrite the memory that RHF is loaded at e.g. on x86 RHF is loaded at 1MiB and the EFI binary/kernel is loaded at 2MiB. The correct place to change is probably the value provided to StartInfo::new for kernel_load_addr as that is the address that is passed to the EFI PE loader.

I think this is a big change that impact CH, RHF and maybe also future change for EDK2 (I'd like to unify the boot process of EDK2 and RHF), so let me think it through carefully.

The points are:

Assume that 2MB of space is enough to load GRUB. To do this:

Move ram_min to 0x40600000.

Add a run-time payload size check.
I think the change in 6735c61 is little bit ad hoc, but it's acceptable because it works. It's future work to implement better a loading process.

So, my suggestion is to just add comments to the memory layout assumption. See #262 (comment).

src/arch/aarch64/ram64.s

retrage

@jongwu Thanks for fixing it to boot Ubuntu. This is a big milestone in #198.

src/efi/mod.rs

aarch64-unknown-none.ld

src/fat.rs

retrage

It looks good to me. Thanks! Once this PR is merged, I'll add integration tests for aarch64.

rbradford

Thanks @retrage for the review - just a few bits from me!

rbradford · 2023-07-18T14:37:16Z

src/fat.rs

@@ -232,6 +232,17 @@ fn name_to_str(input: &str, output: &mut [u8]) {
            break;
        }
    }
+
+    // Fixup for Director or file name, when the length of its name and extend name equals 11. There maybe no


nit: typo here - I think you mean directory. Could you also wrap you comment to 80 characters

rbradford · 2023-07-18T14:38:27Z

src/efi/mod.rs

@@ -1115,6 +1114,8 @@ pub fn efi_exec(
    let mut ct_index = 0;

    // Populate with FDT table if present
+    // To boot with ACPI, we can't offer full fdt. Simply, neglect it. See https://github.com/torvalds/linux/blob/d528014517f2b0531862c02865b9d4c908019dc4/arch/arm64/kernel/acpi.c#L203


Could you wrap this comment at 80 characters (or close - by putting the URL on a new line of comment.

rbradford · 2023-07-18T14:40:31Z

src/efi/mod.rs

@@ -614,7 +614,6 @@ pub extern "efiapi" fn install_configuration_table(guid: *mut Guid, table: *mut

    for entry in ct.iter_mut() {
        if entry.vendor_guid == unsafe { *guid } {
-            entry.vendor_table = table;


This should be a separate commit.

rbradford · 2023-07-18T14:41:19Z

src/efi/mod.rs

@@ -1115,6 +1114,8 @@ pub fn efi_exec(
    let mut ct_index = 0;

    // Populate with FDT table if present


Could you update the commit title to mark as aarch64 specific.

jongwu · 2023-07-19T05:57:03Z

Thanks @rbradford -, all comments addressed

rbradford

I'll let you have a go at improving the other commit messages in the series. Let me know if you want help.

rbradford · 2023-07-19T09:54:30Z

aarch64-unknown-none.ld

+    RHF:    [0x40600000-]
+   Assuming 2MB is enough to load payload.
+   The stack start is at the end of the RHF region. */
+ram_min = 0x40600000;


Your commit message should focus on why you're doing something. Generally there is no need to repeat what the code does.

e.g.

aarch64: Increase size of EFI payload In order to support the requirements of a larger GRUB binary extend the size of the memory used for loading the payload. Since the payload address is hardcoded to a location in RAM below that of where the RHF binary is loaded increase adjust the ram_min constant to handle that. This does not require any VMM changes as the load address in the PE binary will reflect this change.

Thanks @rbradford -, done.

In order to support the requirements of a larger GRUB binary extend the size of the memory used for loading the payload. Since the payload address is hardcoded to a location in RAM below that of where the RHF binary is loaded increase adjust the ram_min constant to handle that. This does not require any VMM changes as the load address in the PE binary will reflect this change. Fixes: cloud-hypervisor#261 Signed-off-by: Jianyong Wu <[email protected]>

Increasing the maximum number of memory allocations is required to support booting the version of GRUB used in Ubuntu 20.04 for AArch64. Fixes: cloud-hypervisor#261 Signed-off-by: Jianyong Wu <[email protected]>

This patch increases the size of the byte array to 12 bytes (so that the '.' can be included for files that are named the full 8 + 3 bytes. If the string presented is exactly 11 bytes then adjust it to correctly included the '.' separator. Fixes: cloud-hypervisor#261 Signed-off-by: Jianyong Wu <[email protected]>

The Linux kernel only enabled ACPI if the provided FDT table is a stub [1]. Although this is a violation of the boot Arm Base Boot Requirement 7.4.3 [2] which requires an FDT table (a stub would be sufficient) for simplicity the FDT table is simply skipped which does not cause issues. Fixes: cloud-hypervisor#261 Signed-off-by: Jianyong Wu <[email protected]> [1] https://github.com/torvalds/linux/blob/d528014517f2b0531862c02865b9d4c908019dc4/arch/arm64/kernel/acpi.c#L203 [2] https://developer.arm.com/documentation/den0044/latest

This variable will be immediately overwritten later in this function. Signed-off-by: Jianyong Wu <[email protected]>

rbradford

I worked on your commit messages @jongwu - please let me know if you're happy with those changes!

jongwu · 2023-07-21T00:55:47Z

@rbradford -, Great! Thanks! 😄

Since PR cloud-hypervisor#262 was merged, we can boot vanilla Ubuntu images using RHF from Cloud Hypervisor. This commit adds integration tests for aarch64. Signed-off-by: Akira Moroo <[email protected]>

Since PR #262 was merged, we can boot vanilla Ubuntu images using RHF from Cloud Hypervisor. This commit adds integration tests for aarch64. Signed-off-by: Akira Moroo <[email protected]>

jongwu force-pushed the let_ubuntu_go branch 2 times, most recently from 6886d07 to 19fea2c Compare July 6, 2023 06:14

rbradford requested a review from retrage July 6, 2023 07:44

jongwu force-pushed the let_ubuntu_go branch 3 times, most recently from 57dead0 to a088e25 Compare July 6, 2023 09:01

rbradford requested changes Jul 6, 2023

View reviewed changes

retrage reviewed Jul 9, 2023

View reviewed changes

src/efi/mod.rs Show resolved Hide resolved

aarch64-unknown-none.ld Outdated Show resolved Hide resolved

retrage reviewed Jul 9, 2023

View reviewed changes

src/fat.rs Outdated Show resolved Hide resolved

jongwu force-pushed the let_ubuntu_go branch from a088e25 to 056fc41 Compare July 17, 2023 08:33

retrage approved these changes Jul 18, 2023

View reviewed changes

rbradford reviewed Jul 18, 2023

View reviewed changes

jongwu force-pushed the let_ubuntu_go branch 2 times, most recently from 8e7f997 to 759881b Compare July 19, 2023 05:56

jongwu force-pushed the let_ubuntu_go branch 3 times, most recently from de7c4af to 117049c Compare July 19, 2023 07:00

rbradford reviewed Jul 19, 2023

View reviewed changes

jongwu force-pushed the let_ubuntu_go branch from 117049c to 895dca0 Compare July 20, 2023 03:12

jongwu added 5 commits July 20, 2023 15:21

efi/alloc: Bump maximum number of memory allocations

d327a57

Increasing the maximum number of memory allocations is required to support booting the version of GRUB used in Ubuntu 20.04 for AArch64. Fixes: cloud-hypervisor#261 Signed-off-by: Jianyong Wu <[email protected]>

efi: Remove useless assignment

ab9b25e

This variable will be immediately overwritten later in this function. Signed-off-by: Jianyong Wu <[email protected]>

rbradford force-pushed the let_ubuntu_go branch from 895dca0 to ab9b25e Compare July 20, 2023 14:21

rbradford approved these changes Jul 20, 2023

View reviewed changes

retrage merged commit 40d07f3 into cloud-hypervisor:main Jul 21, 2023

retrage mentioned this pull request Jul 22, 2023

tests: Add integration tests for aarch64 #267

Merged

retrage mentioned this pull request Mar 14, 2023

aarch64 Support #198

Open

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AArch64: let ubuntu 20.04+ boot from rust hypervisor firmware #262

AArch64: let ubuntu 20.04+ boot from rust hypervisor firmware #262

jongwu commented Jul 6, 2023 •

edited

Loading

rbradford Jul 6, 2023

jongwu Jul 6, 2023 •

edited

Loading

rbradford Jul 6, 2023

rbradford Jul 6, 2023

jongwu Jul 7, 2023

retrage Jul 9, 2023

jongwu Jul 17, 2023

retrage left a comment •

edited

Loading

retrage left a comment •

edited

Loading

rbradford left a comment •

edited

Loading

rbradford Jul 18, 2023

rbradford Jul 18, 2023

rbradford Jul 18, 2023

rbradford Jul 18, 2023

jongwu commented Jul 19, 2023

rbradford left a comment

rbradford Jul 19, 2023

jongwu Jul 20, 2023

rbradford left a comment

jongwu commented Jul 21, 2023

		@@ -1115,6 +1114,8 @@ pub fn efi_exec(
		let mut ct_index = 0;

		// Populate with FDT table if present

AArch64: let ubuntu 20.04+ boot from rust hypervisor firmware #262

AArch64: let ubuntu 20.04+ boot from rust hypervisor firmware #262

Conversation

jongwu commented Jul 6, 2023 • edited Loading

Choose a reason for hiding this comment

jongwu Jul 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

retrage left a comment • edited Loading

Choose a reason for hiding this comment

retrage left a comment • edited Loading

Choose a reason for hiding this comment

rbradford left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jongwu commented Jul 19, 2023

rbradford left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rbradford left a comment

Choose a reason for hiding this comment

jongwu commented Jul 21, 2023

jongwu commented Jul 6, 2023 •

edited

Loading

jongwu Jul 6, 2023 •

edited

Loading

retrage left a comment •

edited

Loading

retrage left a comment •

edited

Loading

rbradford left a comment •

edited

Loading