Skip to content
This repository has been archived by the owner on Apr 6, 2023. It is now read-only.

Create platform for QEMU's virt target #7

Closed
6 tasks done
JohnAZoidberg opened this issue Aug 12, 2020 · 19 comments
Closed
6 tasks done

Create platform for QEMU's virt target #7

JohnAZoidberg opened this issue Aug 12, 2020 · 19 comments
Assignees

Comments

@JohnAZoidberg
Copy link
Collaborator

JohnAZoidberg commented Aug 12, 2020

Status

  • Works with QEMU 5.2 and 6.0. Does NOT work with QEMU 5.0
  • Boot to UEFI Shell
  • Can access virtio (PCIe) disk
  • ResetSystem runtime call works
  • Figure out why we need to patch OpenSBI (@changab)
  • Boot Linux (@JohnAZoidberg)

Instructions for building and runninng: https://github.com/JohnAZoidberg/riscv-edk2-docker/tree/riscv-virt

WIP branches:

Original comment

The qemu-system-riscv64 -machine virt target of QEMU doesn't look like a SiFive board. It only has virtio peripherals, therefore our current platforms don't work.

Here we don't even need our own device tree, we can just take it from MROM because QEMU puts it there: https://github.com/qemu/qemu/blob/d0ed6a69d399ae193959225cdeaa9382746c91cc/hw/riscv/boot.c#L249

@JohnAZoidberg JohnAZoidberg self-assigned this Aug 12, 2020
@JohnAZoidberg
Copy link
Collaborator Author

JohnAZoidberg commented Feb 28, 2021

I've modified the U540 platform for QEMU's -machine virt.
It's booting to UEFI Shell, recognizes the virtio disk and starts Linux EFISTUB.
But Linux get's stuck failing to get a memory page:

[    0.548367] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    0.548876] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    0.549887] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.550328] ehci-pci: EHCI PCI platform driver
[    0.550881] ehci-platform: EHCI generic platform driver
[    0.551453] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.551891] ohci-pci: OHCI PCI platform driver
[    0.552492] ohci-platform: OHCI generic platform driver
[    0.558739] usbcore: registered new interface driver uas
[    0.559758] usbcore: registered new interface driver usb-storage
[    0.562438] mousedev: PS/2 mouse device common for all mice
[    0.566688] goldfish_rtc 101000.rtc: registered as rtc0
[    0.567787] goldfish_rtc 101000.rtc: setting system clock to 2021-02-28T04:41:04 UTC (1614487264)
[    0.572236] syscon-poweroff poweroff: pm_power_off already claimed 0000000052f031de sbi_shutdown
[    0.573156] syscon-poweroff: probe of poweroff failed with error -16
[    0.574958] EFI Variables Facility v0.08 2004-May-17
[    0.576667] Unable to handle kernel paging request at virtual address 0000002ed0ad7be8
[    0.578052] Oops [#1]
[    0.578380] Modules linked in:
[    0.578909] CPU: 0 PID: 35 Comm: kworker/u2:1 Not tainted 5.8.0-rc1-00011-gad3b29a94a7d #754
[    0.580066] Workqueue: efi_rts_wq efi_call_rts
[    0.580517] epc: ffffffe000706b20 ra : ffffffe000706b0e sp : ffffffe03830fda0
[    0.581021]  gp : ffffffe00106a3b8 tp : ffffffe0382e1300 t0 : ffffffe037d4d000
[    0.581503]  t1 : 0000000000001873 t2 : 0000000000000000 s0 : ffffffe03830fde0
[    0.581907]  s1 : ffffffe038a91d18 a0 : ffffffe0010471f8 a1 : ffffffe037d4cc00
[    0.582337]  a2 : ffffffe038a91d20 a3 : ffffffe001018a08 a4 : 0000000000000000
[    0.582755]  a5 : 0000002ed0ad7b98 a6 : ffffffe000aa3210 a7 : 0000000000000006
[    0.583146]  s2 : ffffffe037d4cc00 s3 : ffffffe038a91d20 s4 : 0000000000000022
[    0.583622]  s5 : 0000000000000000 s6 : 0000000000000000 s7 : ffffffe0010aede0
[    0.584034]  s8 : 0000000000000001 s9 : ffffffe00021a832 s10: ffffffffffffffff
[    0.584422]  s11: ffffffe03828ecc0 t3 : 00000000fa83b2da t4 : 000000000000642f
[    0.584810]  t5 : 0000000000000000 t6 : ffffffe001078447
[    0.585107] status: 0000000000000120 badaddr: 0000002ed0ad7be8 cause: 000000000000000d
[    0.586137] ---[ end trace 59fe67af50ca423e ]---
[    0.586590] note: kworker/u2:1[35] exited with preempt_count 1

The failure looks very much like the issue here: https://lists.denx.de/pipermail/u-boot/2020-July/421194.html
So perhaps we're doing something wrong with SetVirtualAddressMap.
I suspect it's got something to do with PCI though, since we could boot fine on the U540 without PCI.

@JohnAZoidberg
Copy link
Collaborator Author

JohnAZoidberg commented Feb 28, 2021

The branches are https://github.com/riscv/riscv-edk2/tree/riscv-virt-540-mod and https://github.com/riscv/riscv-edk2-platforms/tree/riscv-virt-540-mod.

OpenSBI (Silicon/RISC-V/ProcessorPkg/Library/RiscVOpensbiLib/opensbi) is at 937caee0833115f69d697ca190001ba0aa5c7368 but needs the following patch:

diff --git c/lib/utils/fdt/fdt_helper.c i/lib/utils/fdt/fdt_helper.c
index aec73a0..0ceca44 100644
--- c/lib/utils/fdt/fdt_helper.c
+++ i/lib/utils/fdt/fdt_helper.c
@@ -7,11 +7,13 @@
  */

 #include <libfdt.h>
+#include <libfdt_env.h>
 #include <sbi/riscv_asm.h>
 #include <sbi/sbi_console.h>
 #include <sbi/sbi_hartmask.h>
 #include <sbi/sbi_platform.h>
 #include <sbi/sbi_scratch.h>
+#include <sbi/sbi_string.h>
 #include <sbi_utils/fdt/fdt_helper.h>
 #include <sbi_utils/irqchip/plic.h>
 #include <sbi_utils/sys/clint.h>
@@ -123,7 +125,7 @@ int fdt_parse_hart_id(void *fdt, int cpu_offset, u32 *hartid)
        prop = fdt_getprop(fdt, cpu_offset, "device_type", &len);
        if (!prop || !len)
                return SBI_EINVAL;
-       if (strncmp (prop, "cpu", strlen ("cpu")))
+       if (sbi_strncmp (prop, "cpu", strlen ("cpu")))
                return SBI_EINVAL;

        val = fdt_getprop(fdt, cpu_offset, "reg", &len);

Building still works the same as U540, since I haven't split it out into its own platform yet:

build -a RISCV64 -p Platform/SiFive/U5SeriesPkg/FreedomU540HiFiveUnleashedBoard/U540.dsc -t GCC5 \
  && cp -v /Build/FreedomU540HiFiveUnleashed/DEBUG_GCC5/FV/U540.fd /docker/U540-virt.fd

Run it like this with your own disk image:

qemu-system-riscv64 -machine virt \
  -bios Build/FreedomU540HiFiveUnleashed/DEBUG_GCC5/FV/U540.fd \
  -m 2048 -nographic  \
  -device virtio-blk-device,drive=hd0 \
  -drive file=/home/zoid/Downloads/linux.iso,format=raw,id=hd0 \
  -device virtio-net-device,netdev=usernet \
  -netdev user,id=usernet,hostfwd=tcp::10000-:22

@JohnAZoidberg
Copy link
Collaborator Author

Virtio disk and virtio NIC work:

Shell> map -r
Mapping table
      FS0: Alias(s):F0:;BLK0:
          VenHw(837DCA9E-E874-4D82-B29A-23FE0E23D1E2,0080001000000000)
Shell> ifconfig -l

-----------------------------------------------------------------

name         : eth0
Media State  : Media present
policy       : static
mac addr     : 52:54:00:12:34:56

ipv4 address : 0.0.0.0

subnet mask  : 0.0.0.0

gateway: 0.0.0.0

  Routes (0 entries):

DNS server   : 

@JohnAZoidberg
Copy link
Collaborator Author

JohnAZoidberg commented Feb 28, 2021

@kallisti5
Copy link

kallisti5 commented Feb 28, 2021

Hm. EDK2 definitely boots... but:

qemu-system-riscv64 -machine virt -bios ~/Downloads/U540-virt-3a3c8579ec33335e3.fd -m 2048 -device virtio-blk-device,drive=hd0 -drive file=haiku-mmc.image,format=raw,id=hd0
  1. EFI/BOOT/BOOTRISCV64.EFI doesn't auto-boot
  2. Bootloader doesn't start

Both of these things work fine under u-boot

logs:

UEFI Interactive Shell v2.287477C2-69C7-11D2-8E39-00A0C969723B BEDE4BA0
EDK IIlProtocolInterface: 752F3136-4E16-4FDC-A22A-E5F46812F4CA BEDE7F98
UEFI v2.70 (EDK II, 0x00010000)008-7F9B-4F30-87AC-60C9FEF5DA4E BE6B1450
Mapping table
      FS0: Alias(s):HD0b:;BLK1:
          VenHw(837DCA9E-E874-4D82-B29A-23FE0E23D1E2,0080001000000000)/HD(1,MBR,0x00000000,0x4,0x10000)
     BLK0: Alias(s):
          VenHw(837DCA9E-E874-4D82-B29A-23FE0E23D1E2,0080001000000000)
     BLK2: Alias(s):
          VenHw(837DCA9E-E874-4D82-B29A-23FE0E23D1E2,0080001000000000)/HD(2,MBR,0x00000000,0x10004,0x96000)


Press ESC in 1 seconds to skip startup.nsh or any other key to continue.
Shell> fs0:
FS0:\> cd EFI/BOOTuccess
FS0:\EFI\BOOT\> FI\BOOT' Success
FS0:\EFI\BOOT\>  Success
FS0:\EFI\BOOT\> FI' Success
FS0:\EFI\BOOT\>  Success
FS0:\EFI\BOOT\> ' Success
FS0:\EFI\BOOT\> FI\BOOT' Success
FS0:\EFI\BOOT\> 
FS0:\EFI\BOOT\> 
FS0:\EFI\BOOT\> 
FS0:\EFI\BOOT\> BOOTRISCV64.EFI'.' Success
FSOpen: Open '\EFI\BOOT\BOOTRISCV64.EFI' Success
FSOpen: Open '\EFI\BOOT\BOOTRISCV64.EFI' Success
FSOpen: Open '\EFI\BOOT\BOOTRISCV64.EFI' Success
FSOpen: Open '\EFI\BOOT\BOOTRISCV64.EFI' Success
[Security] 3rd party image[0] is deferred to load before EndOfDxe: VenHw(837DCA9E-E874-4D82-B29A-23FE0E23D1E2,0080001000000000)/HD(1,MBR,0x00000000,0x4,0x10000)/\EFI\BOOT\BOOTRISCV64.EFI.pen: Open '..' Success
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B BEDA7940
Command Error Status: Unsupported
FSOpen: Open '.' Success
FSOpen: Open '\EFI' Success
FSOpen: Open '.' Success
FSOpen: Open '..' Success
FSOpen: Open '\EFI\BOOT' Success

Our nightly riscv64 images are available here if you want to test:

https://download.haiku-os.org/nightly-images/riscv64/

@JohnAZoidberg
Copy link
Collaborator Author

By the way, UEFI SCT (Self Certification Test) runs fine on the virt machine :)

@vlsunil
Copy link

vlsunil commented Apr 21, 2021

Hi @JohnAZoidberg,
Qemu has added High MMIO space (>4G) for PCIe recently. With your virt branch/prebuilt EDK2 image, it is failing to boot for this new MMIO range. Have you seen this?

ProcessPciHost: Config[0x30000000+0x10000000) Bus[0x0..0xFF] Io[0x0+0x10000)@0x3000000 Mem32[0x40000000+0x40000000)@0x0 Mem64[0x400000000+0x400000000)@0x0
CpuSetMemoryAttributes: Set memory attributes not supported yet
CpuSetMemoryAttributes: Set memory attributes not supported yet
RootBridge: PciRoot(0x0)
  Support/Attr: 70001 / 70001
    DmaAbove4G: Yes
NoExtConfSpace: No
     AllocAttr: 3 (CombineMemPMem Mem64Decode)
           Bus: 0 - FF Translation=0
            Io: 0 - FFFF Translation=0
           Mem: 40000000 - 7FFFFFFF Translation=0
    MemAbove4G: 400000000 - 7FFFFFFFF Translation=0
          PMem: FFFFFFFFFFFFFFFF - 0 Translation=0
   PMemAbove4G: FFFFFFFFFFFFFFFF - 0 Translation=0
CpuSetMemoryAttributes: Set memory attributes not supported yet

ASSERT_EFI_ERROR (Status = Not Found)
ASSERT [PciHostBridgeDxe] /edk2/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c(332): !EFI_ERROR (CheckStatus)

@kallisti5
Copy link

I've noticed qemu 6.x flat out crashes when loading the latest u-boot OpenSBI bootloader. Maybe also related to the High MMIO change you mentioned?

@vlsunil
Copy link

vlsunil commented Apr 22, 2021

@kallisti5 : I have not tried u-boot. But I expect some level of testing with u-boot done for the high mmio change. EDK2 virt support is not yet merged and hence I guess no one would have tested it.

@JohnAZoidberg
Copy link
Collaborator Author

No, haven't seen this. What version of QEMU?

@vlsunil
Copy link

vlsunil commented Apr 22, 2021

It is based on v6.0.0-rc4.

@vlsunil
Copy link

vlsunil commented Apr 29, 2021

Hi @JohnAZoidberg ,
I found the issue for above 4GB MMIO crash in edk2. Since it is branch, could you please review and merge below change to fix it? Basically increased the physical memory supported to 48 bits. Ideally, we should get it from the platform instead of hardcoding.

diff --git a/Platform/SiFive/U5SeriesPkg/FreedomU540HiFiveUnleashedBoard/Universal/Pei/PlatformPei/Platform.c b/Platform/SiFive/U5SeriesPkg/FreedomU540HiFiveUnleashedBoard/Universal/Pei/PlatformPei/Platform.c
index 6641e10f..3bb13548 100644
--- a/Platform/SiFive/U5SeriesPkg/FreedomU540HiFiveUnleashedBoard/Universal/Pei/PlatformPei/Platform.c
+++ b/Platform/SiFive/U5SeriesPkg/FreedomU540HiFiveUnleashedBoard/Universal/Pei/PlatformPei/Platform.c
@@ -199,7 +199,7 @@ MiscInitialization (
   // of IO space. (Side note: unlike other HOBs, the CPU HOB is needed during
   // S3 resume as well, so we build it unconditionally.)
   //
-  BuildCpuHob (32, 32);
+  BuildCpuHob (48, 32);^M
 }

@JohnAZoidberg
Copy link
Collaborator Author

Thanks! Would that work with QEMU 5.0 also?

@vlsunil
Copy link

vlsunil commented May 3, 2021

Was this branch working with 5.0? Without my change also, it doesn't work for me. It hangs after below message.

SecCoreStartUpWithStack: Initializing OpenSBI library for booting hart 0

@JohnAZoidberg
Copy link
Collaborator Author

Yes, with 5.2 it was working for me. But it needs a second patch for OpenSBI that I forgot to mention here: #14 (comment)

@JohnAZoidberg
Copy link
Collaborator Author

JohnAZoidberg commented May 3, 2021

I created a fork of OpenSBI to put our patches and updated the submodule to point to there.
No need for manually patching OpenSBI anymore.

Just do

git pull
git submodule sync
git submodule update --init

@JohnAZoidberg
Copy link
Collaborator Author

Thanks! Would that work with QEMU 5.0 also?

Ok, I can confirm that your proposed change works with 5.2 and 6.0. Thanks!
I pushed it to the branch.

@JohnAZoidberg
Copy link
Collaborator Author

This branch doesn't work with older QEMU.
It hangs directly after:

SecCoreStartUpWithStack: Initializing OpenSBI library for booting hart 0

so I don't think this is due to the change contributed by @vlsunil.

Only Ubuntu 21.04 has 5.2. Older version have only 5.0 or older.

@JohnAZoidberg
Copy link
Collaborator Author

I consider this done. It's not upstreamed but it's working well on riscv-virt-gh-actions branch.

Just build Platform/Qemu/RiscvVirt/RiscvVirt.dsc or download the RISCVVIRT.fd from the latest run of GitHub actoins.

Linux boots fine from ESP on virtio :)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants