AARCH64 stack initialisation fails with newlib for qemu_cortex_a53

[AshwinKrishn]

Describe the bug
Unable to use newlib with qemu_cortex_a53. When using minilib things work fine. The moment I change to newlib, the booting fails.

Please also mention any information which could help others to understand
the problem you're facing:

• qemu_cortex_a53
• The code loses control in zeroing of bss in gdb. Something happens there.

To Reproduce
Steps to reproduce the behavior:

1. prj.conf
   CONFIG_STACK_CANARIES=n
   CONFIG_INIT_STACKS=y
   CONFIG_USERSPACE=y
   CONFIG_TEST_USERSPACE=y

2. west build -b qemu_cortex_a53 blackscholes_arm/ -DCMAKE_VERBOSE_MAKEFILE=ON

3. west build -t run

Expected behavior
Must be able to boot successfully.

Impact
Unable to test my main work because the work requires math functions and minimal lib does not provide. Newlib provides math functions.

Logs and console output
The boot up just freezes. It gets stuck in SCV trap.

Environment (please complete the following information):

• OS: Linux
• Toolchain Zephyr SDK
• west update latest

Additional context
prj.conf said as above.

[stephanosio]

@carlocaione It looks like something happens when CONFIG_INIT_STACKS is enabled with newlib.

[AshwinKrishn]

Looks like when __init_sys_init_malloc_prepare0 begins in user space initialization phase , it is stuck forever.

@stephanosio Should I disable the CONFIG_INIT_STACKS ?
Update : CONFIG_INIT_STACKS is not enabled. Removed and tested.

[stephanosio]

I have tested this with both tests/c_lib and samples/hello_world. Both boot fine with CONFIG_USERSPACE=y and CONFIG_NEWLIB_LIBC=y set; though, CONFIG_INIT_STACKS=y-related failure is only observed on tests/c_lib.

There seems to be an underlying arch port-level issue that is causing this (maybe yet another linker script alignment-related issue?). I do not have the bandwidth to analyse and fix this at the moment. I might get back to it later if this does not get resolved in time.

Meanwhile, please provide a proper instruction and/or branch we can use to reproduce this.

[AshwinKrishn]

Hello @stephanosio Thank you for your time. I have figured out where exactly the kernel_panic is called. It is being called from "zephyrproject_arm/zephyr/arch/arm64/core/mmu.c:48" . The arch_mem_map is unable to map memory.

These are the parameters of the function:

These are the critical memory configs
CONFIG_64BIT=y
CONFIG_SRAM_SIZE=2097152
CONFIG_SRAM_BASE_ADDRESS=0xa0000000
CONFIG_KERNEL_VM_BASE=0xa0000000
CONFIG_KERNEL_VM_OFFSET=0

I am using freshly downloaded Zephyr as of 02/24/2022 using updated using "west update".
This looks like Kconfig is allowing some wrong combination of RAM and VIRT addresses. I do not have complete expertise in this topic.

Requesting your kind help.

[AshwinKrishn]

Hello @carlocaione @stephanosio A small update , I increased the CONFIG_MAX_XLAT_TABLES to 4096 and the above trap went off.

I then went on and found another trapping location.

The memset is unable to access the memory. Meaning the mapping has not ben properly updated.

Looks like the mapping in arm64 needs some change.

Experts please weigh in.

[carlocaione]

uhm, something is wrong with A53 + newlib libc.

Also hello_world_user is failing with qemu_cortex_a53_smp when CONFIG_NEWLIB_LIBC=y.

SSERTION FAIL [(options & ((1UL << (2)))) == 0U || z_stack_is_user_capable(stack)] @ WEST_TOPDIR/zephyr/kernel/thread.c:514
        user thread 0x40034000 with kernel-only stack 0x40082000
E: ELR_ELn: 0x000000004000d2c0
E: ESR_ELn: 0x0000000056000002
E:   EC:  0x15 (Unknown)
E:   IL:  0x1
E:   ISS: 0x2
E: TPIDRRO: 0x0100000040034e20
E: x0:  0x0000000040034e20  x1:  0x0000000000000202
E: x2:  0x0000000000000000  x3:  0x00000000400846e9
E: x4:  0x0000000000000010  x5:  0x0000000000000004
E: x6:  0x0000000000000031  x7:  0x0000000040020e3a
E: x8:  0x0000000000000004  x9:  0x00000000ffffffff
E: x10: 0x0000000000000001  x11: 0x0000000000000000
E: x12: 0x0000000000000000  x13: 0x0000000000000000
E: x14: 0x0000000000000000  x15: 0x0000000000000000
E: x16: 0x0000000040010fec  x17: 0x0000000000000000
E: x18: 0x0000000000000000  x30: 0x0000000040013098
E: >>> ZEPHYR FATAL ERROR 4: Kernel panic on CPU 0
E: Current thread: 0x40034a20 (unknown)
E: Halting system

Not sure if related but that would be a nice coincidence.

[carlocaione]

This is an interesting bug. At least for the problem I reported the problem is that z_stack_is_user_capable(stack) is returning NULL.

In particular what's returning NULL is z_object_lookup():

static inline struct z_object *
z_object_lookup ( const char *str,  size_t len)
{
  if (len <= MAX_WORD_LENGTH && len >= MIN_WORD_LENGTH)
    {
       unsigned int key = hash((const char *)&str, len);

      if (key <= MAX_HASH_VALUE)
        if (len == sizeof(void *))
          {
             const char *s = wordlist[key].name;

            if (str == s)
              return &wordlist[key];
          }
    }
  return 0;
}

The (str == s) comparison is failing because while the str is correctly carrying the address of the stack, the value of s retrieved from the wordlist table is wrong.

In particular in s we found the address of the stack as it would have been without newlibc enabled.

[carlocaione]

This also happens with qemu_x86_64 so this is a more general issue. Opening a different issue to track this.

[stephanosio]

Closing this in favour of #42184 since that is the root issue.

[stephanosio]

@AshwinKrishn FYI, #42184 (comment)

Try doing a clean (pristine) build after configuration changes and see if that helps.