Coredump occurred in nghttp3_conn_read_control function

[wanlebing]

application is a HTTP/3 server, pseudocode here：

loop {
    // 1. receive data from quic
    data = self.stream.poll_read(...);
    // 2. write data to nghttp3
    nghttp3_conn_read_stream(...);
}

1. the coredump：

#0  0x00007fe04dc4e49f in raise () from /lib64/libc.so.6
#1  0x00007fe04dc21f2e in abort () from /lib64/libc.so.6
#2  0x00007fe04dc21d39 in __assert_fail_base.cold.0 () from /lib64/libc.so.6
#3  0x00007fe04dc46e86 in __assert_fail () from /lib64/libc.so.6
#4  0x0000557d1e8baa09 in nghttp3_conn_read_control (conn=0x7fe008669f90, stream=0x7fe008134160, src=0x7fe00825fe74 "\200\017\a", srclen=7)

assert failed while reading control stream.

2. some related variables：

(gdb) i locals
p = 0x7fe00825fe7b "\276\237\363\235s\334AD\326)\252\272%\244\022B\250R\250\252ㄖP\035k\214\252\035TK8\200\017\a"
end = 0x7fe00825fe7b "\276\237\363\235s\334AD\326)\252\272%\244\022B\250R\250\252ㄖP\035k\214\252\035TK8\200\017\a"
rv = 0
rstate = 0x7fe008134370
rvint = 0x7fe008134370
nread = 2
nconsumed = 7
busy = 0
len = 0
pri_field_value = 0x0
pri_field_valuelen = 0
__PRETTY_FUNCTION__ = "nghttp3_conn_read_control"
__func__ = "nghttp3_conn_read_control"

(gdb) p /x *rstate
$2 = {rvint = {acc = 0x0, left = 0x0}, fr = {hd = {type = 0xf0700, length = 0x8}, data = {hd = {type = 0xf0700, length = 0x8}}, headers = {hd = {type = 0xf0700, length = 0x8}, 
      nva = 0x5c, nvlen = 0x0}, settings = {hd = {type = 0xf0700, length = 0x8}, niv = 0x5c, iv = {{id = 0x0, value = 0x0}}}, goaway = {hd = {type = 0xf0700, length = 0x8}, 
      id = 0x5c}, priority_update = {hd = {type = 0xf0700, length = 0x8}, pri_elem_id = 0x5c, {pri = {urgency = 0x0, inc = 0x0}, {data = 0x0, datalen = 0x0}}}}, left = 0x6, 
  state = 0x9}

we can see the problem, buffer is not enough:
rstate->left is 0x6
p equal end is 0x7fe00825fe7b

3. the orresponding source code:

      /* Fall through */
    case NGHTTP3_CTRL_STREAM_STATE_PRIORITY_UPDATE:
      /* We need to buffer Priority Field Value because it might be
         fragmented. */
      len = (size_t)nghttp3_min_int64(rstate->left, (int64_t)(end - p));
      assert(len > 0);

assert(len > 0) failed here, some possibilities：

1. no enough data in buf [src, src + srclen]
2. the HTTP/3 frame is fragmented
3. bugs in stream state machine
4. ...

need some guidance to figure out the problem, thanks in advance.

[tatsuhiro-t]

nghttp3 version?

[wanlebing]

(gdb) i args
conn = 0x7fe008669f90
stream = 0x7fe008134160
src = 0x7fe00825fe74 "\200\017\a"
srclen = 7

(gdb) i locals
p = 0x7fe00825fe7b "\276\237\363\235s\334AD\326)\252\272%\244\022B\250R\250\252ㄖP\035k\214\252\035TK8\200\017\a"
end = 0x7fe00825fe7b "\276\237\363\235s\334AD\326)\252\272%\244\022B\250R\250\252ㄖP\035k\214\252\035TK8\200\017\a"
rv = 0
rstate = 0x7fe008134370
rvint = 0x7fe008134370
nread = 2
nconsumed = 7
busy = 0
len = 0
pri_field_value = 0x0
pri_field_valuelen = 0
PRETTY_FUNCTION = "nghttp3_conn_read_control"
func = "nghttp3_conn_read_control"
(gdb) p /x *rvint
$5 = {acc = 0x0, left = 0x0}

nconsumed is srclen and p equal to end，means state NGHTTP3_CTRL_STREAM_STATE_PRIORITY_UPDATE is processed once and when NGHTTP3_CTRL_STREAM_STATE_PRIORITY_UPDATE is being processed twice within stream state machine cause the assert failed.

[tatsuhiro-t]

The latest version is v1.10.1.

[wanlebing]

The latest version is v1.10.1.

ok，i will try it，but I didn't see related updates between v1.8.0 and v1.10.1.

[wanlebing]

(gdb) f 4
#4 0x000055983d8bea59 in nghttp3_conn_read_control (conn=0x7f89f43772c0, stream=0x7f89f447a3e0, src=0x7f89f42f8326 "\200\017\a", srclen=7)
at /root/.cargo/git/checkouts/nghttp3-rs-bbe236760f62b866/269ed97/nghttp3-sys/nghttp3/lib/nghttp3_conn.c:983
983 in /root/.cargo/git/checkouts/nghttp3-rs-bbe236760f62b866/269ed97/nghttp3-sys/nghttp3/lib/nghttp3_conn.c
(gdb) p /x rstate->fr.priority_update.pri_elem_id
$8 = 0x54
(gdb) x/7xb src
0x7f89f42f8326: 0x80 0x0f 0x07 0x00 0x08 0x40 0x54

the src contains Type (i)、Length Prioritized Element ID，but not contains Priority Field Value.

as the source code, maybe the HTTP/3 PRIORITY_UPDATE Frame shoule be bufferd the total by nghttp3 or application, to avoid fragmented without Priority Field Value(end == p):

case NGHTTP3_CTRL_STREAM_STATE_PRIORITY_UPDATE:
  /* We need to buffer Priority Field Value because it might be
     fragmented. */
  len = (size_t)nghttp3_min_int64(rstate->left, (int64_t)(end - p));
  assert(len > 0);

[wanlebing]

The latest version is v1.10.1.

i have tried v1.10.1，the same coredump still exits.

[tatsuhiro-t]

at /root/.cargo/git/checkouts/nghttp3-rs-bbe236760f62b866/269ed97/nghttp3-sys/nghttp3/lib/nghttp3_conn.c:983

This suggests you are still using v1.8.0.

[wanlebing]

at /root/.cargo/git/checkouts/nghttp3-rs-bbe236760f62b866/269ed97/nghttp3-sys/nghttp3/lib/nghttp3_conn.c:983

v1.10.1 assert in line 981？i will reproduce it again, but there is little difference in nghttp3_conn_read_control between v1.10.1 and v1.8.0.

[wanlebing]

at /root/.cargo/git/checkouts/nghttp3-rs-bbe236760f62b866/269ed97/nghttp3-sys/nghttp3/lib/nghttp3_conn.c:983

This suggests you are still using v1.8.0.

after upgrade to v1.10.1, the crash disappeared，seems to be fixed by this commit ba42bd2