FDP: segfault while reading physical memory in std::__atomic_base<bool>::store

[Wenzel]

Hi,

I'm facing a segfault while trying to read physical memory:

#0  std::__atomic_base<bool>::store (__m=std::memory_order_release, __i=false, this=0x370f1a00370f190) at /usr/include/c++/9/bits/atomic_base.h:397
#1  std::atomic<bool>::store (this=0x370f1a00370f190, __i=false, __m=std::memory_order_release) at /usr/include/c++/9/atomic:105
#2  0x00007ffff75a8be4 in (anonymous namespace)::ttas_spinlock_unlock (flag=0x370f1a00370f190) at FDP.cpp:97
#3  (anonymous namespace)::UnlockSHM (FDPShm=0x370f1a00370f190) at FDP.cpp:107
#4  RunCmdBuffer (pFDP=0x7ffff4bbf010, pDst=0x7ffff45bd04f, pSrc=0x7fffffffb9b0, szSize=24) at FDP.cpp:353
#5  0x00007ffff75a8d82 in FDP_ReadPhysicalMemoryInternal (pFDP=0x7ffff4bbf010, pDstBuffer=0x7ffff45bd04f "t", ReadSize=10485759, 
    PhysicalAddress=11534335) at FDP.cpp:399
#6  0x00007ffff75a8e1b in FDP_ReadPhysicalMemory (pFDP=0x7ffff4bbf010, 
    pDstBuffer=0x7ffff3bbd050 "\002\260\001\213~8\215M\370\210E\364\213E\364\211E\374\211]\370\213\207p\a", ReadSize=15728640, 
    PhysicalAddress=1048576) at FDP.cpp:413
#7  0x00007ffff769496e in fdp::FDP::read_physical_memory () from /lib/libmicrovmi.so
#8  0x00007ffff760d010 in <microvmi::driver::virtualbox::VBox as microvmi::api::Introspectable>::read_physical () from /lib/libmicrovmi.so
#9  0x00007ffff760b157 in microvmi_read_physical () from /lib/libmicrovmi.so
#10 0x00007ffff7e942e9 in DeviceMicrovmi_ReadContigious () from /home/mtarral/local/memprocfs/leechcore_device_microvmi.so
#11 0x00007ffff7c16260 in LcReadContigious_DeviceRead () from /home/mtarral/local/memprocfs/leechcore.so
#12 0x00007ffff7c1649d in LcReadContigious_Read () from /home/mtarral/local/memprocfs/leechcore.so
#13 0x00007ffff7c1670d in LcReadContigious_ReadScatterGather () from /home/mtarral/local/memprocfs/leechcore.so
#14 0x00007ffff7c16ca8 in LcReadScatter () from /home/mtarral/local/memprocfs/leechcore.so
#15 0x00007ffff7c1706e in LcRead () from /home/mtarral/local/memprocfs/leechcore.so
#16 0x00007ffff7ecde04 in VmmWinInit_DTB_FindValidate () from /home/mtarral/local/memprocfs/vmm.so
#17 0x00007ffff7eceae4 in VmmWinInit_TryInitialize () from /home/mtarral/local/memprocfs/vmm.so
#18 0x00007ffff7ecbc32 in VmmProcInitialize () from /home/mtarral/local/memprocfs/vmm.so
#19 0x00007ffff7eb04e6 in VMMDLL_InitializeEx () from /home/mtarral/local/memprocfs/vmm.so
#20 0x00007ffff7eb066d in VMMDLL_Initialize () from /home/mtarral/local/memprocfs/vmm.so
#21 0x000055555555894d in main ()

I initialized FDP with:

• FDP_OpenSHM()
• FDP_Init()
  https://github.com/Wenzel/fdp/blob/master/src/lib.rs#L92

The last call in FDP is here

My memory dump example is still working though:

But this time I'm trying to bridge libmicrovmi with MemProcFS.

I wanted to check with you if this was a bug somewhere in FDP ?

Thanks !

[Wenzel]

From what I can see from the LeechCore call, it's trying to read 0xf00000 bytes at physical address 0x100000

#10 0x00007ffff7e942e9 in DeviceMicrovmi_ReadContigious (ctxRC=0x7ffff3bbd010) at leechcore_device_microvmi.c:14
14        if (!microvmi_read_physical(driver, ctxRC->paBase, ctxRC->pb, ctxRC->cb, &bytes_read)) {
(gdb) driver
Undefined command: "driver".  Try "help".
(gdb) p driver
$1 = (void *) 0x555555568880
(gdb) l
9       {
10        // read contigious physical memory
11        PLC_CONTEXT ctxLC = ctxRC->ctxLC;
12        void* driver = ctxLC->hDevice;
13        uint64_t bytes_read = 0;
14        if (!microvmi_read_physical(driver, ctxRC->paBase, ctxRC->pb, ctxRC->cb, &bytes_read)) {
15          lcprintfvvv(ctxLC, "Failed to read physical memory at 0x%llx\n", ctxRC->paBase);
16        }
17        ctxRC->cbRead = (DWORD)bytes_read;
18      }
(gdb) p/x ctxRC->paBase
$2 = 0x100000
(gdb) p/x ctxRC->cb
$3 = 0xf00000

[Wenzel]

As a workaround I'm splitting the read by 4K, and it seems to work.
So the 0xf00000 read size was definitely the issue ! 😉

[bamiaux]

Icebox does split into aligned chunks before reading and we don't get segfaults.
However, reading the virtualbox implementation, it's supposed to support reading physical memory mapping multiple contiguous pages so something is definitely wrong