Physical Memory Manipulation

[EduardoBGrasel]

Hi,

I'm trying to manipulates de physical memory address inside bao to a certain pattern, is there a way to do this?

The entire address of a given guest (freeRTOS) or a task within a guest is in the format 0xXXXX1XXX;

I tried modify the functions pp_alloc in the core/mem.c and mem_map in core/mmu/mem.c, but bao freezes.

Platform: Raspberry Pi 4b
Demo: Dual-guest Linux+FreeRTOS

[josecm]

IIUC, you want to allocate only physical addresses with that format.

This is non-trivial as Bao always allocates physical memory contigously for a given regions. Even when coloring is enabled Bao allocates that region contigously in the allowed colors.

What you can do is add another allocator for that pattern, add that information to struct ppages and modify mem_map to modify something along the lines of what is done for the coloring case.

Let me know if this helps in any way.

[EduardoBGrasel]

Thank you for your response.

I tried this in two ways, the first one I only modify de pp_alloc and bao inicializes with the correct address in many cases, but yet theres a lot of address that doesn't follow the pattern and the guests doesn't start, so I modified the mem_alloc also. Nonetheless now bao freezes in #starting application on 0x200000.

Is it possible that I change the address that bao and the VMs uses to initialize? Is there a way to know these addresses?

[josecm]

tried this in two ways, the first one I only modify de pp_alloc and bao inicializes with the correct address in many cases

Have you also changed the map function? Because the mapping function assumes the physical pages struct passed to it represent a contigous memory region. If you don't change this, even if you allocate the correct pages in pp_alloc in won't work.

Note that you would somehow let the global allocator when to call your special allocator just for the address spaces you target.

Is it possible that I change the address that bao and the VMs uses to initialize? Is there a way to know these addresses?

Bao's virtual addresses are fixed for a given architecture. For the VMs is totally possible. It just depends on the configuration (entry field). But do you wish to assign these special addresses to the hypervisor also? That would be a much more complicated task as you would need to relocate the hypervisor after initializing the memory infrastructure.

[EduardoBGrasel]

Here is the pp_alloc modified, I only modify add a loop to only accept the address in the pattern that i am interested (the paddr_t candidate_addr).

the idea is to reallocate the page until the pattern fits, but after this the guests doesn't inicialize. I tried to modify the mem_map too, but no matter what i tried it always crashes bao, so i just made a simple printk to debug in it.

bool pp_alloc(struct page_pool *pool, size_t num_pages, bool aligned, struct ppages *ppages) {
    ppages->colors = 0;
    ppages->num_pages = 0;

    bool ok = false;
    if (num_pages == 0) return true;

    spin_lock(&pool->lock);

    size_t start = aligned ? pool->base / PAGE_SIZE % num_pages : 0;
    size_t curr = pool->last + ((pool->last + start) % num_pages);

    for (size_t i = 0; i < 2 && !ok; i++) {
        while (pool->free != 0) {
            ssize_t bit = bitmap_find_consec(pool->bitmap, pool->size, curr, num_pages, false);

            if (bit < 0) {
                curr = aligned ? (num_pages - ((pool->base / PAGE_SIZE) % num_pages)) % num_pages : 0;
                break;
            }

            paddr_t candidate_addr = pool->base + (bit * PAGE_SIZE);
            uint8_t bank_bits = (candidate_addr >> 12) & 0xF;

            if (bank_bits != 0b0101) { // just 0x5000
                curr = bit + 1;
                continue;
            }

            ppages->base = candidate_addr;
            ppages->num_pages = num_pages;
            bitmap_set_consecutive(pool->bitmap, bit, num_pages);
            pool->free -= num_pages;
            pool->last = bit + num_pages;
            ok = true;
            break;
        }
    }

    spin_unlock(&pool->lock);
    return ok;
}

bool mem_map(struct addr_space *as, vaddr_t va, struct ppages *ppages,
    size_t num_pages, mem_flags_t flags)
{
// Debug area
// paddr_t debug_paddr = 0;
// vaddr_t debug_vaddr = 0;
// size_t debug_pages = 0;
size_t static total = 0;

size_t count = 0;
pte_t *pte = NULL;
vaddr_t vaddr = va & ~(PAGE_SIZE - 1);

struct section *sec = mem_find_sec(as, vaddr);

if ((sec == NULL) || (sec != mem_find_sec(as, vaddr + num_pages * PAGE_SIZE - 1)))
return false;

spin_lock(&as->lock);
if (sec->shared) spin_lock(&sec->lock);

/**
* TODO check if entry is reserved. Unrolling mapping if something
* goes wrong.
*/

struct ppages temp_ppages;
if (ppages == NULL && !all_clrs(as->colors)) {
temp_ppages = mem_alloc_ppages(as->colors, num_pages, false);
if (temp_ppages.num_pages < num_pages) {
    ERROR("failed to alloc colored physical pages");
}
ppages = &temp_ppages;
}

if (ppages && !all_clrs(ppages->colors)) {
size_t index = 0;
mem_inflate_pt(as, vaddr, num_pages * PAGE_SIZE);
for (size_t i = 0; i < ppages->num_pages; i++) {
    pte = pt_get_pte(&as->pt, as->pt.dscr->lvls - 1, vaddr);
    index = pp_next_clr(ppages->base, index, ppages->colors);
    paddr_t paddr = ppages->base + (index * PAGE_SIZE);
    pte_set(pte, paddr, PTE_PAGE, flags);
    printk("VA: 0x%lx IPA Guest: 0x%lx -> RPA Bao: 0x%lx\n", va, vaddr, paddr);
    total++;
    vaddr += PAGE_SIZE;
    index++;
    //printk("%lu", total);
}
} else {
paddr_t paddr = ppages ? ppages->base : 0;
while (count < num_pages) {
    size_t lvl = 0;
    for (lvl = 0; lvl < as->pt.dscr->lvls; lvl++) {
        pte = pt_get_pte(&as->pt, lvl, vaddr);
        if (pt_lvl_terminal(&as->pt, lvl)) {
            if (pt_pte_mappable(as, pte, lvl, num_pages - count,
                                vaddr, ppages ? paddr : 0)) {
                break;
            } else if (!pte_valid(pte)) {
                mem_alloc_pt(as, pte, lvl, vaddr);
            } else if (!pte_table(&as->pt, pte, lvl)) {
                ERROR("trying to override previous mapping");
            }
        }
    }

    size_t entry = pt_getpteindex(&as->pt, pte, lvl);
    size_t nentries = pt_nentries(&as->pt, lvl);
    size_t lvlsz = pt_lvlsize(&as->pt, lvl);

    while ((entry < nentries) && (count < num_pages) &&
           (num_pages - count >= lvlsz / PAGE_SIZE)) {
        if (ppages == NULL) {
            struct ppages temp =
                mem_alloc_ppages(as->colors, lvlsz / PAGE_SIZE, true);
            if (temp.num_pages < lvlsz / PAGE_SIZE) {
                if (lvl == (as->pt.dscr->lvls - 1)) {
                    // TODO: free previously allocated pages
                    ERROR("failed to alloc physical pages");
                } else {
                    pte = pt_get_pte(&as->pt, lvl, vaddr);
                    if (!pte_valid(pte)) {
                        mem_alloc_pt(as, pte, lvl, vaddr);
                    }
                    break;
                }
            }
            paddr = temp.base;
        }
        pte_set(pte, paddr, pt_page_type(&as->pt, lvl), flags);
        printk("VA: 0x%lx IPA Guest: 0x%lx -> RPA Bao: 0x%lx\n", va, vaddr, paddr);
        vaddr += lvlsz;
        paddr += lvlsz;
        count += lvlsz / PAGE_SIZE;
        pte++;
        entry++;
        total++;
        //printk("total: %lu\n", total);
    }
}
}
fence_sync();

if (sec->shared) {
spin_unlock(&sec->lock);
// TODO tlb shootdown?
}
spin_unlock(&as->lock);

return true;
}

this is a part of the output that i've got:
VA: 0xfe8000002000 IPA Guest: 0xfe8000004000 -> RPA Bao: 0x2f7000
VA: 0x0 IPA Guest: 0x400000 -> RPA Bao: 0x8f5000
VA: 0x0 IPA Guest: 0x0 -> RPA Bao: 0xbf5000
VA: 0x0 IPA Guest: 0x600000 -> RPA Bao: 0xff5000
VA: 0xfe8000002000 IPA Guest: 0xfe8000002000 -> RPA Bao: 0x305000
VA: 0x0 IPA Guest: 0x200000 -> RPA Bao: 0x13f5000
VA: 0x0 IPA Guest: 0x800000 -> RPA Bao: 0x17f5000
VA: 0xfe8000002000 IPA Guest: 0xfe8000003000 -> RPA Bao: 0x306000
VA: 0x0 IPA Guest: 0x400000 -> RPA Bao: 0x1bf5000
VA: 0x0 IPA Guest: 0xa00000 -> RPA Bao: 0x1ff5000
VA: 0xfe8000002000 IPA Guest: 0xfe8000004000 -> RPA Bao: 0x307000
VA: 0x0 IPA Guest: 0x600000 -> RPA Bao: 0x23f5000
VA: 0x0 IPA Guest: 0xc00000 -> RPA Bao: 0x27f5000
VA: 0x0 IPA Guest: 0x0 -> RPA Bao: 0x2bf5000
VA: 0x0 IPA Guest: 0xe00000 -> RPA Bao: 0x33f5000
VA: 0x0 IPA Guest: 0x200000 -> RPA Bao: 0x37f5000
VA: 0x0 IPA Guest: 0x800000 -> RPA Bao: 0x2ff5000
VA: 0x0 IPA Guest: 0x400000 -> RPA Bao: 0x3ff5000
VA: 0x0 IPA Guest: 0xa00000 -> RPA Bao: 0x43f5000
VA: 0x0 IPA Guest: 0x600000 -> RPA Bao: 0x47f5000
VA: 0x0 IPA Guest: 0xc00000 -> RPA Bao: 0x4bf5000
VA: 0x0 IPA Guest: 0x1000000 -> RPA Bao: 0x3bf5000
VA: 0x0 IPA Guest: 0x800000 -> RPA Bao: 0x4ff5000
VA: 0x0 IPA Guest: 0xe00000 -> RPA Bao: 0x53f5000
VA: 0x0 IPA Guest: 0xa00000 -> RPA Bao: 0x5bf5000

the ones with the five as the after the three zeros in the end come form the pp_alloc.

can you tell me if you that see something is missing or maybe wrong in the logic?