Segfault when enabling `LIBUKBOOT_MAINTHREAD` with some musl’s `pthread`

[shym]

Describe the bug

I’ve been testing the new feature to make sure unikernels can terminate gracefully but ran into a segfault when using musl’s pthread_mutex_lock on a ERRORCHECK mutex.
If I understand correctly, the issue is the fact that the code in main will run in a thread created directly using uk_sched_thread_create_fn2 instead of pthread_create and so musl’s tcb is never properly initialized. In particular the following example triggers a segfault when accessing its robust_list, as its head is NULL, when trying to lock the mutex.

(See #82 for another issue with the initialization of the TCB).

Steps to reproduce

I wrote a small reproducer in https://github.com/shym/unikraft-simple/tree/mutex.
In one glance, the repository content is just those 3 files:

• Kraftfile :

  spec: v0.6
  name: simple
  unikraft:
    version: staging
    kconfig:
    - CONFIG_LIBUKBOOT_MAINTHREAD=y
  libraries:
    musl: staging
  targets:
  - qemu/x86_64

• main.c:

  #include <pthread.h>
  #include <assert.h>
  
  int main() {
    pthread_mutex_t m;
    pthread_mutexattr_t attr;
  
    pthread_mutexattr_init(&attr);
    pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_ERRORCHECK);
    pthread_mutex_init(&m, &attr);
    pthread_mutexattr_destroy(&attr);
  
    pthread_mutex_lock(&m);
    // ^ Triggers a segfault
    pthread_mutex_unlock(&m);
  
    return 0;
  }

• Makefile.uk:

  $(eval $(call addlib,appsimple))
  
  APPSIMPLE_SRCS-y += $(APPSIMPLE_BASE)/main.c

and run:

kraft build
qemu-system-x86_64 -nographic -nodefaults -serial stdio -kernel .unikraft/build/simple_qemu-arm64

The log of the failure is there while it lasts.

Expected behavior

I expected the program to terminate normally.

Which platforms were you using or does this bug affect?

kvm

Relevant log output

$  qemu-system-x86_64 -nographic -nodefaults -serial stdio -kernel .unikraft/build/simple_qemu-x86_64
Booting from ROM..Powered by
o.   .o       _ _               __ _
Oo   Oo  ___ (_) | __ __  __ _ ' _) :_
oO   oO ' _ `| | |/ /  _)' _` | |_|  _)
oOo oOO| | | | |   (| | | (_) |  _) :_
 OoOoO ._, ._:_:_,\_._,  .__,_:_, \___)
                  Helene 0.18.0~3fd0499
[    0.107072] CRIT: [libkvmplat] <trace.c @   41> RIP: 000000000018b795 CS: 0008
[    0.108098] CRIT: [libkvmplat] <trace.c @   42> RSP: 0000000007f8ff20 SS: 0010 EFLAGS: 00000202
[    0.108376] CRIT: [libkvmplat] <trace.c @   44> RAX: 0000000000000000 RBX: 000000000024f180 RCX: 0000000007f8ffd0
[    0.108702] CRIT: [libkvmplat] <trace.c @   46> RDX: 0000000007f8ffb0 RSI: 0000000000000000 RDI: 000000000024f208
[    0.109029] CRIT: [libkvmplat] <trace.c @   48> RBP: 0000000007f8ff30 R08: 0000000000000000 R09: 0000000000000000
[    0.109350] CRIT: [libkvmplat] <trace.c @   50> R10: 0000000000000000 R11: 0000000000012028 R12: 0000000000000002
[    0.109672] CRIT: [libkvmplat] <trace.c @   52> R13: 00000000002383e0 R14: 0000000000000000 R15: 0000000007f8ffb0
[    0.110118] CRIT: [libkvmplat] <trace.c @   86> base is 0x7f8ff30 caller is 0x18b40f
[    0.110390] CRIT: [libkvmplat] <trace.c @   86> base is 0x7f8ff90 caller is 0x1b1bae
[    0.110815] CRIT: [libkvmplat] <trace.c @   86> base is 0x7f8ffe0 caller is 0x140772
[    0.111131] CRIT: [libkvmplat] <trace.c @   86> base is 0x7f90010 caller is 0
[    0.111472] CRIT: [libkvmplat] <trace.c @   66> 
[    0.111702] CRIT: [libkvmplat] <trace.c @   66> 7f8ff10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[    0.112156] CRIT: [libkvmplat] <trace.c @   66> 7f8ff20: 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00
[    0.112588] CRIT: [libkvmplat] <trace.c @   66> 7f8ff30: 90 ff f8 07 00 00 00 00 0f b4 18 00 00 00 00 00
[    0.113036] CRIT: [libkvmplat] <trace.c @   66> 7f8ff40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[    0.113498] CRIT: [libkvmplat] <trace.c @   66> 
[    0.113702] CRIT: [libkvmplat] <trace.c @   66> 7f8ff20: 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00
[    0.114120] CRIT: [libkvmplat] <trace.c @   66> 7f8ff30: 90 ff f8 07 00 00 00 00 0f b4 18 00 00 00 00 00
[    0.114540] CRIT: [libkvmplat] <trace.c @   66> 7f8ff40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[    0.114973] CRIT: [libkvmplat] <trace.c @   66> 7f8ff50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[    0.115424] CRIT: [libkvmplat] <trace.c @   66> 
[    0.115618] CRIT: [libkvmplat] <trace.c @   66> 18b780: 88 00 00 00 48 8d 4a 20 48 89 72 20 48 89 7a 18
[    0.116038] CRIT: [libkvmplat] <trace.c @   66> 18b790: 48 39 f7 74 04 48 89 4e f8 48 89 8b 88 00 00 00
[    0.116450] CRIT: [libkvmplat] <trace.c @   66> 18b7a0: 48 c7 83 98 00 00 00 00 00 00 00 85 c0 74 ba c7
[    0.116904] CRIT: [libkvmplat] <trace.c @   66> 18b7b0: 42 14 00 00 00 00 b8 82 00 00 00 e9 12 ff ff ff
[    0.117338] CRIT: [libkvmplat] <traps.c @  113> Crashing