Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,7 @@ tools/unit-tests/unit-image-hybrid
tools/unit-tests/unit-imx-rt-cache-align
tools/unit-tests/unit-update-disk-fit
tools/unit-tests/unit-update-disk-oob
tools/unit-tests/unit-update-disk-amp
tools/unit-tests/unit-update-ram-enc
tools/unit-tests/unit-update-ram-enc-nopart
tools/unit-tests/unit-va416x0-fram
Expand Down Expand Up @@ -692,6 +693,7 @@ tools/unit-tests/fdt_memrsv_extract.h
tools/unit-tests/hifive1_flash_write_extract.h
tools/unit-tests/mpfs250_wdt_extract.h
tools/unit-tests/unit-mpfs-wdt
tools/unit-tests/unit-mpfs-wdt-os-owned
tools/unit-tests/hifive1_hfrosccfg_extract.h
tools/unit-tests/keystore_api_extract.h
tools/unit-tests/keystore_emitted.c
Expand Down
4 changes: 4 additions & 0 deletions config/examples/polarfire_mpfs250.config
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,10 @@ GZIP?=1
# Use RISC-V assembly version of ECDSA and SHA
NO_ASM?=0

# Linux owns the MSS watchdogs: wolfBoot never refreshes them, since the first
# refresh locks the window the Linux mpfs_wdt driver programs at probe.
MPFS_WDT_NODES_KEEP?=1

# SDCard or eMMC support via SDHCI driver
DISK_SDCARD?=1
DISK_EMMC?=0
Expand Down
70 changes: 68 additions & 2 deletions docs/PolarFire.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ wolfBoot runs here either as a payload of Microchip's Hart Software Services or
- [Building wolfBoot](#building-wolfboot)
- [QSPI](#qspi)
- [HSS S-mode with L2-LIM (no DDR)](#hss-s-mode-with-l2-lim-no-ddr)
- [Linux beside a bare-metal hart (AMP)](#linux-beside-a-bare-metal-hart-amp)
- [M-mode (bare-metal eNVM boot)](#m-mode-bare-metal-envm-boot)
- [M-mode + DDR: booting Linux (minimal SBI)](#m-mode--ddr-booting-linux-minimal-sbi)
- [Boot mode 2: User Boot Loader stub](#boot-mode-2-user-boot-loader-stub)
Expand Down Expand Up @@ -313,7 +314,7 @@ These flags apply to `polarfire_mpfs250_m_qspi.config` and are added via `CFLAGS
| Flag | Default | Description |
|------|---------|-------------|
| `WATCHDOG` | undefined | Has no timeout to set on this target: the MSS watchdogs are armed by the bitstream and their window cannot be changed from software. See "Watchdog policy" below. wolfBoot refreshes all five in `hal_init()` and again in `hal_prepare_boot()` either way. |
| `MPFS_WDT_NODES_KEEP` (make variable, `=1`) | undefined | Linux boot only: keep the five MSS watchdog nodes enabled in the dtb and stop the parked E51 from refreshing them, so the OS owns the watchdogs. See "Watchdog policy" below. |
| `MPFS_WDT_NODES_KEEP` (make variable, `=1`) | undefined (`1` in `polarfire_mpfs250.config`) | Linux boot only: keep the five MSS watchdog nodes enabled in the dtb and stop the parked E51 from refreshing them, so the OS owns the watchdogs. In an S-mode build under HSS, wolfBoot never refreshes them. See "Watchdog policy" below. |
| `MPFS_QSPI_CLK_DIV` (make variable) | `QSPI_CLK_DIV_8` | QSPI SCK divider, a `QSPI_CLK_DIV_*` name from `hal/mpfs250.h`. See "PolarFire SoC QSPI". |
| `MPFS_SC_PROBE` (make variable) | `1` on M-mode builds | Read-only: at boot, report whether the System Controller serves the device serial number, the TRNG nonce and PUF emulation, with two stability checks (nonce differs across calls, PUF response repeats). Only service status and the two comparisons are printed, never a serial, nonce or PUF value. `MPFS_SC_PROBE=0` disables it. M-mode only. |
| `MPFS_SNVM_DUMP` (make variable, `=1`) | undefined | Read-only: at boot, print the admin word, state and a checksum of `MPFS_SNVM_DUMP_COUNT` sNVM pages from `MPFS_SNVM_DUMP_FIRST` (defaults 2 from 200, the keystore and the wrapped image key). Page contents are never printed. M-mode only. |
Expand Down Expand Up @@ -488,6 +489,67 @@ python3 tools/scripts/mpfs_qspi_prog.py /dev/ttyUSB1 \
to back the LIM SRAM region. See the M-mode section for a detailed explanation.
- UART output appears on MMUART1 (`/dev/ttyUSB1`), same as other S-mode configurations.

## Linux beside a bare-metal hart (AMP)

wolfBoot can boot Linux on U54_1-3 while U54_4 runs a separate bare-metal application, for example a firmware TPM. Linux must not use the application's DDR, and must not try to start its hart.

### M-mode wolfBoot (HSS replacement)

The application travels in the same signed FIT as the kernel, as an extra image, so the one signature covers both and it follows the A/B slot:

```
fwtpm-1 {
description = "bare-metal image for U54_4";
data = /incbin/("app.bin"); /* objcopy -O binary app.elf */
type = "firmware";
arch = "riscv";
compression = "none";
load = <0x91c00000>;
entry = <0x91c00000>;
hash-1 { algo = "sha256"; };
};
```

List it in the configuration (`loadables = "fwtpm-1";`) and build an M-mode SD or eMMC configuration with the hart, the image's memory and, optionally, the image's node name (`MPFS_AMP_IMAGE`, default `fwtpm-1`):

```sh
cp config/examples/polarfire_mpfs250_m.config .config
make MPFS_AMP_HART=4 MPFS_AMP_MEM_BASE=0x91C00000 MPFS_AMP_MEM_SIZE=0x400000
```

After verifying the FIT, wolfBoot copies the image to `MPFS_AMP_MEM_BASE` (refusing one larger than `MPFS_AMP_MEM_SIZE`) and refuses to boot if it is missing. Before releasing hart 1 it zeroes the L2 LIM with doubleword stores, since the LIM comes out of reset with unwritten ECC and HSS is not there to clear it, and starts the AMP hart in M-mode at the image's first byte with interrupts off; the image sets up its own trap vector and stack. The kernel's dtb gets a `/memreserve/` entry for the image's memory and the AMP hart's `cpu@` node disabled, and the SBI refuses `hart_start` for that hart. The parked E51 keeps refreshing all five watchdogs, the AMP hart's included, so `MPFS_WDT_NODES_KEEP` cannot be combined with `MPFS_AMP_HART`. If the application uses the System Controller mailbox, build with `MPFS_SCB_SMODE_DENY=1` so Linux's drivers stay off it.

The boot log shows the image loaded and the hart started:

```
AMP: 'fwtpm-1' loaded at 0x91C00000 (329728 bytes) for hart 4
FDT: Reserved AMP memory 0x91C00000, size 0x400000
Starting hart 4 in M-mode at 0x91C00000 (AMP)
Releasing hart 1 into S-mode at 0x80200000 (dtb=0x8A000000)
```

### wolfBoot as an HSS payload

HSS can start wolfBoot on U54_1-3 and the application on U54_4 from one payload:

```yaml
set-name: 'PolarFire-SoC-HSS::wolfBoot + bare metal'
hart-entry-points: {u54_1: '0x80000000', u54_2: '0x80000000', u54_3: '0x80000000', u54_4: '0x91C00000'}
payloads:
wolfboot.elf: {owner-hart: u54_1, secondary-hart: u54_2, secondary-hart: u54_3, priv-mode: prv_s, ancilliary-data: hal/mpfs.dtb}
app.elf: {exec-addr: '0x91C00000', owner-hart: u54_4, priv-mode: prv_m, skip-opensbi: true}
```

HSS passes the payload no device tree and its OpenSBI domain does not fence the other hart's memory off from it, so nothing tells Linux that the application's DDR is in use; Linux allocates it and overwrites the running application. Build wolfBoot with that region and it adds a `/memreserve/` entry for it to the kernel's dtb:

```sh
make MPFS_AMP_MEM_BASE=0x91C00000 MPFS_AMP_MEM_SIZE=0x400000 WOLFBOOT_RAMBOOT_MAX_SIZE=0x03C00000
```

wolfBoot stages the image at `WOLFBOOT_LOAD_ADDRESS` before checking its signature and refuses an image whose staging range would reach the AMP memory. With the example `WOLFBOOT_LOAD_ADDRESS=0x8E000000`, `WOLFBOOT_RAMBOOT_MAX_SIZE=0x03C00000` ends the staging window at `0x91C00000`.

The FIT's dtb should also mark the AMP hart's `cpu@` node `disabled`; otherwise Linux tries to start that hart through SBI and logs that it failed to.

## M-mode (bare-metal eNVM boot)

wolfBoot supports running directly in Machine Mode (M-mode) on PolarFire SoC, replacing the Hart
Expand Down Expand Up @@ -649,7 +711,11 @@ answers with a chip reset rather than an error (STATUS bit 2). An M-mode build r
after setting MVRP itself and never lands there, but a build running as an HSS payload
inherits the supervisor's window, so `mpfs_wdt_pet_all()` tests that bit per watchdog and
skips any refresh that would reset the board; the window is whole seconds wide and every
caller loops, so a skipped refresh costs nothing. After hand-off the parked
caller loops, so a skipped refresh costs nothing. An HSS payload that boots Linux needs
`MPFS_WDT_NODES_KEEP=1` (`polarfire_mpfs250.config` sets it), which stops wolfBoot refreshing
them at all: the first refresh locks TIME, MVRP and TRIGGER until reset, the Linux `mpfs_wdt`
driver programs those at probe, and a locked watchdog resets the board during the Linux boot.
After hand-off the parked
E51 acts as a monitor and refreshes all five watchdogs; the OS watchdog driver is disabled via
the dtb fixup so the two never conflict. `MPFS_WDT_NODES_KEEP=1` hands the watchdogs to the OS
instead: the dtb nodes stay enabled and the parked E51 stops refreshing, so the OS driver arms
Expand Down
130 changes: 128 additions & 2 deletions hal/mpfs250.c
Original file line number Diff line number Diff line change
Expand Up @@ -71,10 +71,12 @@ extern void sdhci_irq_handler(void);
* (40 MHz E51 reset, 150 MHz under HSS), raised when M-mode raises the PLL. */
uint32_t mpfs_apb_clk_hz = MSS_APB_AHB_CLK;

#if defined(WOLFBOOT_RISCV_MMODE) || !defined(MPFS_WDT_NODES_KEEP)
static const unsigned long mpfs_wdt_bases[MPFS_NUM_HARTS] = {
MSS_WDT_E51_BASE, MSS_WDT_U54_1_BASE, MSS_WDT_U54_2_BASE,
MSS_WDT_U54_3_BASE, MSS_WDT_U54_4_BASE
};
#endif

#ifdef WOLFBOOT_RISCV_MMODE
/* CPU MHz for the mcycle udelay(); seeded at the E51 reset clock and bumped
Expand Down Expand Up @@ -164,6 +166,15 @@ static void mpfs_signal_main_hart_started(void)
__asm__ volatile("fence iorw, iorw" ::: "memory");
}

#if defined(MPFS_AMP_HART) && \
(!defined(MPFS_DDR_INIT) || !defined(WOLFBOOT_MMODE_SMODE_BOOT))
#error MPFS_AMP_HART needs the M-mode Linux boot (MPFS_DDR_INIT and WOLFBOOT_MMODE_SMODE_BOOT)
#endif
#if defined(MPFS_AMP_HART) && \
((MPFS_AMP_HART) <= MPFS_FIRST_U54_HART || (MPFS_AMP_HART) > MPFS_LAST_U54_HART)
#error MPFS_AMP_HART must be a U54 other than the Linux boot hart
#endif

#if defined(MPFS_DDR_INIT) && defined(WOLFBOOT_MMODE_SMODE_BOOT)
/* Per-hart S-mode start mailboxes, consumed by secondary_hart_entry(). In the
* DTIM (uncached, hart-coherent) at +0x100; SBI shared state is at +0x000. */
Expand All @@ -174,6 +185,12 @@ typedef struct {
} mpfs_kernel_handoff_t;

#define MPFS_KERNEL_HANDOFF_MARKER 0x4C4E5858UL /* "LNXX" */
#ifdef MPFS_AMP_HART
#define MPFS_AMP_HANDOFF_MARKER 0x414D504DUL /* "AMPM" */

/* Entry of the AMP image, set once mpfs_amp_load() has placed it. */
static unsigned long mpfs_amp_entry;
#endif

#define mpfs_kernel_handoff \
((mpfs_kernel_handoff_t *)(0x01000000UL + 0x100UL))
Expand Down Expand Up @@ -206,6 +223,21 @@ void secondary_hart_entry(unsigned long hartid, HLS_DATA* hls)
riscv_mmode_to_smode(kentry, hartid, opq);
/* never returns */
}
#ifdef MPFS_AMP_HART
/* The AMP image stays in M-mode and sets up its own trap vector and
* stack; it must not take the IPI that released it. */
if (hartid == (unsigned long)MPFS_AMP_HART &&
mpfs_kernel_handoff[hartid].marker == MPFS_AMP_HANDOFF_MARKER) {
unsigned long aentry;
__asm__ volatile("fence r,rw" ::: "memory");
aentry = (unsigned long)mpfs_kernel_handoff[hartid].entry;
__asm__ volatile("csrw mie, zero");
CLINT_MSIP_REG(hartid) = 0;
__asm__ volatile("fence iorw, iorw" ::: "memory");
__asm__ volatile("fence.i" ::: "memory");
((void (*)(unsigned long))aentry)(hartid);
}
#endif
/* Sleep until the next IPI (e.g. a future SBI HSM hart_start),
* then clear it and re-check the mailbox. */
__asm__ volatile("wfi");
Expand Down Expand Up @@ -243,6 +275,11 @@ int sbi_hal_hart_start(unsigned long hartid, unsigned long saddr,
hartid > (unsigned long)MPFS_LAST_U54_HART) {
return -1;
}
#ifdef MPFS_AMP_HART
if (hartid == (unsigned long)MPFS_AMP_HART) {
return -1;
}
#endif
mpfs_kernel_handoff[hartid].entry = (uint64_t)saddr;
mpfs_kernel_handoff[hartid].opaque = (uint64_t)opaque;
__asm__ volatile("fence iorw, iorw" ::: "memory");
Expand Down Expand Up @@ -284,6 +321,30 @@ hal_smode_boot(unsigned long entry, unsigned long hartid, unsigned long dtb)
* clock driver ungates SUBBLK_CLOCK_CR but does not lift the soft reset. */
mpfs_mmc_os_release();

#ifdef MPFS_AMP_HART
if (mpfs_amp_entry == 0UL) {
wolfBoot_printf("AMP: no image loaded for hart %d\n", MPFS_AMP_HART);
wolfBoot_panic();
}
/* The LIM comes out of reset with unwritten ECC that only a doubleword
* store initialises; the AMP image and the OS share memory there. */
{
volatile uint64_t *lim = (volatile uint64_t *)MPFS_L2_LIM_BASE;
unsigned long n;
for (n = 0; n < MPFS_L2_LIM_SIZE / sizeof(uint64_t); n++) {
Comment thread
dgarske marked this conversation as resolved.
Resolved
lim[n] = 0;
}
}
mpfs_kernel_handoff[MPFS_AMP_HART].entry = (uint64_t)mpfs_amp_entry;
__asm__ volatile("fence iorw, iorw" ::: "memory");
mpfs_kernel_handoff[MPFS_AMP_HART].marker = MPFS_AMP_HANDOFF_MARKER;
__asm__ volatile("fence iorw, iorw" ::: "memory");
wolfBoot_printf("Starting hart %d in M-mode at 0x%lx (AMP)\n",
MPFS_AMP_HART, mpfs_amp_entry);
CLINT_MSIP_REG(MPFS_AMP_HART) = 0x01;
__asm__ volatile("fence iorw, iorw" ::: "memory");
#endif

mpfs_kernel_handoff[MPFS_FIRST_U54_HART].entry = (uint64_t)entry;
mpfs_kernel_handoff[MPFS_FIRST_U54_HART].opaque = (uint64_t)dtb;
__asm__ volatile("fence iorw, iorw" ::: "memory");
Expand Down Expand Up @@ -312,6 +373,29 @@ hal_smode_boot(unsigned long entry, unsigned long hartid, unsigned long dtb)
}
__builtin_unreachable();
}

#ifdef MPFS_AMP_HART
/* Places the AMP image at MPFS_AMP_MEM_BASE, bounded by MPFS_AMP_MEM_SIZE, and
* enters it at its first byte: the image is linked to run there. */
int mpfs_amp_load(void *fit)
{
int len = 0;
void *p;

p = fit_load_image_to((fdt_ctx *)fit, MPFS_AMP_IMAGE,
(void *)MPFS_AMP_MEM_BASE,
(uint32_t)MPFS_AMP_MEM_SIZE, &len);
if (p == NULL || len <= 0) {
wolfBoot_printf("AMP: no usable '%s' image in the FIT\n",
MPFS_AMP_IMAGE);
return -1;
}
mpfs_amp_entry = (unsigned long)MPFS_AMP_MEM_BASE;
wolfBoot_printf("AMP: '%s' loaded at 0x%lx (%d bytes) for hart %d\n",
MPFS_AMP_IMAGE, mpfs_amp_entry, len, MPFS_AMP_HART);
return 0;
}
#endif
#endif /* MPFS_DDR_INIT && WOLFBOOT_MMODE_SMODE_BOOT */

#endif /* WOLFBOOT_RISCV_MMODE */
Expand Down Expand Up @@ -1330,6 +1414,33 @@ static int mpfs_dts_fixup_inplace(void* dts_addr, uint32_t capacity)
#ifdef WOLFBOOT_UPDATE_DISK
(void)wolfBoot_disk_dts_fixup(&ctx);
#endif
#ifdef MPFS_AMP_MEM_BASE
#if (MPFS_AMP_MEM_SIZE) == 0
#error MPFS_AMP_MEM_SIZE must be non-zero
#endif
/* HSS passes the payload no dtb and does not fence another domain's
* memory off from it, so the OS learns of the AMP hart's region here. */
ret = fdt_add_mem_rsv(&ctx, (uint64_t)MPFS_AMP_MEM_BASE,
(uint64_t)MPFS_AMP_MEM_SIZE);
Comment thread
dgarske marked this conversation as resolved.
Resolved
Comment on lines +1423 to +1424
if (ret != 0) {
/* Linux would overwrite the running AMP image. */
wolfBoot_printf("FDT: Failed to reserve AMP memory (%d)\n", ret);
wolfBoot_panic();
}
wolfBoot_printf("FDT: Reserved AMP memory 0x%lx, size 0x%lx\n",
(unsigned long)MPFS_AMP_MEM_BASE, (unsigned long)MPFS_AMP_MEM_SIZE);
#endif
#ifdef MPFS_AMP_HART
/* Keeps Linux from trying to start the AMP hart through SBI HSM. */
off = fdt_find_node_offset(&ctx, -1, MPFS_AMP_CPU_NODE);
if (off >= 0) {
ret = fdt_fixup_str(&ctx, off, MPFS_AMP_CPU_NODE, "status",
"disabled");
}
if (off < 0 || ret != 0) {
wolfBoot_printf("FDT: Failed to disable %s\n", MPFS_AMP_CPU_NODE);
}
#endif

#if defined(MPFS_DDR_INIT) && defined(WOLFBOOT_MMODE_SMODE_BOOT) && \
(!defined(MPFS_WDT_NODES_KEEP) || defined(MPFS_SCB_SMODE_DENY))
Expand Down Expand Up @@ -1421,10 +1532,21 @@ static int mpfs_dts_fixup_inplace(void* dts_addr, uint32_t capacity)
* fixups above. */
int hal_dts_fixup(void* dts_addr, uint32_t capacity)
{
int ret;

if (dts_addr == NULL) {
return -1;
}
return mpfs_dts_fixup_inplace(dts_addr, capacity);
ret = mpfs_dts_fixup_inplace(dts_addr, capacity);
#ifdef MPFS_AMP_MEM_BASE
/* A pass that gave up before the reservation leaves the AMP image
* unprotected, so it must not boot. */
if (ret != 0) {
wolfBoot_printf("FDT: AMP memory not reserved\n");
wolfBoot_panic();
}
#endif
return ret;
}

void hal_prepare_boot(void)
Expand Down Expand Up @@ -1488,9 +1610,12 @@ static int mpfs_scb_wait_clear(uint32_t reg_offset, uint32_t mask,
}

/* A refresh above MVRP is the forbidden window and resets the chip, so each
* watchdog is tested: under a supervisor that owns the window, every one is. */
* watchdog is tested. An S-mode build with MPFS_WDT_NODES_KEEP refreshes
* none: the first write locks TIME, MVRP and TRIGGER until reset, and the OS
* watchdog driver has to program them at probe. */
void mpfs_wdt_pet_all(void)
{
#if defined(WOLFBOOT_RISCV_MMODE) || !defined(MPFS_WDT_NODES_KEEP)
unsigned long b;
unsigned int i;

Expand All @@ -1500,6 +1625,7 @@ void mpfs_wdt_pet_all(void)
MSS_WDT_REFRESH(b) = 0xDEADC0DEU;
}
}
#endif
}

#if defined(EXT_FLASH) || defined(MPFS_SBI_SC_FLASH)
Expand Down
20 changes: 20 additions & 0 deletions hal/mpfs250.h
Original file line number Diff line number Diff line change
Expand Up @@ -463,6 +463,10 @@ typedef struct {
#define MPFS_FIRST_U54_HART 1 /* First U54 is hart 1 */
#define MPFS_LAST_U54_HART 4 /* Last U54 is hart 4 */

/* L2 LIM left by WAY_ENABLE=0x0B: ways 12-15, 512 KB */
#define MPFS_L2_LIM_BASE 0x08000000UL
#define MPFS_L2_LIM_SIZE 0x00080000UL

/* Stack configuration per hart */
#ifndef STACK_SIZE_PER_HART
#define STACK_SIZE_PER_HART 8192
Expand Down Expand Up @@ -785,6 +789,22 @@ int mpfs_pdma_memcpy(void *dst, const void *src, uint32_t bytes);
* unconditionally in hal/mpfs250.c: the SDHCI and QSPI-programmer callers
* exist in the S-mode builds too, where MPFS_DDR_INIT is not set. */
void mpfs_wdt_pet_all(void);

#ifdef MPFS_AMP_HART
#define MPFS_AMP_STR_(x) #x
#define MPFS_AMP_STR(x) MPFS_AMP_STR_(x)
#define MPFS_AMP_CPU_NODE "cpu@" MPFS_AMP_STR(MPFS_AMP_HART)

/* hal_smode_boot() zeroes the whole L2 LIM after the AMP image is loaded. */
#if (MPFS_AMP_MEM_BASE < MPFS_L2_LIM_BASE + MPFS_L2_LIM_SIZE) && \
(MPFS_AMP_MEM_BASE + MPFS_AMP_MEM_SIZE > MPFS_L2_LIM_BASE)
#error "MPFS_AMP_MEM_BASE/MPFS_AMP_MEM_SIZE must not overlap the L2 LIM"
#endif

/* Load the AMP image from the verified FIT into MPFS_AMP_MEM_BASE for
* MPFS_AMP_HART to run in M-mode. 0 on success. */
int mpfs_amp_load(void *fit);
#endif
#endif /* __ASSEMBLER__ */

#if defined(EXT_FLASH) || defined(MPFS_SBI_SC_FLASH)
Expand Down
Loading
Loading