Skip to content

Shutting down postgres with more than one steampipe extension installed segfaults #870

@isaac-sec

Description

@isaac-sec

When I create a database with a single steampipe extension, shutting down the postgres process results in a clean shutdown:

2025-06-24 09:28:30.293 IDT [79488] LOG:  starting PostgreSQL 15.12 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 15.1.1 20250425, 64-bit
2025-06-24 09:28:30.293 IDT [79488] LOG:  listening on IPv6 address "::1", port 39649
2025-06-24 09:28:30.293 IDT [79488] LOG:  listening on IPv4 address "127.0.0.1", port 39649
2025-06-24 09:28:30.294 IDT [79491] LOG:  database system was shut down at 2025-06-24 09:28:30
2025-06-24 09:28:30.297 IDT [79488] LOG:  database system is ready to accept connections
2025/06/24 09:28:31 [INFO] Log level WARN
^C2025-06-24 09:28:32.423 IDT [79488] LOG:  received fast shutdown request
2025-06-24 09:28:32.424 IDT [79488] LOG:  aborting any active transactions
2025-06-24 09:28:32.424 IDT [79496] FATAL:  terminating connection due to administrator command
2025-06-24 09:28:32.425 IDT [79488] LOG:  background worker "logical replication launcher" (PID 79494) exited with exit code 1
2025-06-24 09:28:32.432 IDT [79489] LOG:  shutting down
2025-06-24 09:28:32.432 IDT [79489] LOG:  checkpoint starting: shutdown immediate
2025-06-24 09:28:32.433 IDT [79489] LOG:  checkpoint complete: wrote 138 buffers (0.8%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.002 s, sync=0.001 s, total=0.002 s; sync files=53, longest=0.001 s, average=0.000 s; distance=831 kB, estimate=831 kB
2025-06-24 09:28:32.436 IDT [79488] LOG:  database system is shut down

However, when started with two steampipe extensions or more, a shutdown results in a segfault:

2025-06-24 09:30:27.556 IDT [79606] LOG:  starting PostgreSQL 15.12 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 15.1.1 20250425, 64-bit
2025-06-24 09:30:27.557 IDT [79606] LOG:  listening on IPv6 address "::1", port 41897
2025-06-24 09:30:27.557 IDT [79606] LOG:  listening on IPv4 address "127.0.0.1", port 41897
2025-06-24 09:30:27.558 IDT [79610] LOG:  database system was shut down at 2025-06-24 09:30:27
2025-06-24 09:30:27.560 IDT [79606] LOG:  database system is ready to accept connections
2025/06/24 09:30:28 [INFO] Log level WARN
2025/06/24 09:30:28 [INFO] Log level WARN
^C2025-06-24 09:30:37.233 IDT [79606] LOG:  received fast shutdown request
2025-06-24 09:30:37.233 IDT [79606] LOG:  aborting any active transactions
2025-06-24 09:30:37.233 IDT [79614] FATAL:  terminating connection due to administrator command
2025-06-24 09:30:37.235 IDT [79606] LOG:  background worker "logical replication launcher" (PID 79613) exited with exit code 1
2025-06-24 09:30:38.202 IDT [79606] LOG:  server process (PID 79614) was terminated by signal 11: Segmentation fault
2025-06-24 09:30:38.202 IDT [79606] LOG:  terminating any other active server processes
2025-06-24 09:30:38.203 IDT [79606] LOG:  abnormal database system shutdown
2025-06-24 09:30:38.210 IDT [79606] LOG:  database system is shut down

Note that this is just with the extensions installed (using CREATE EXTENSION ...), and without any foreign servers configured. This happens locally, as well as when running in the postgres:15.13-bookworm container.

Process tree before the segmentation fault:

postgres,83756,83739
  ├─postgres,83757,83757
  ├─postgres,83758,83758
  ├─postgres,83760,83760
  ├─postgres,83761,83761
  ├─postgres,83762,83762
  └─postgres,83764,83764
      ├─{postgres},83765,83764
      ├─{postgres},83766,83764
      ├─{postgres},83767,83764
      ├─{postgres},83768,83764
      ├─{postgres},83769,83764
      ├─{postgres},83770,83764
      ├─{postgres},83771,83764
      ├─{postgres},83772,83764
      ├─{postgres},83773,83764
      ├─{postgres},83774,83764
      ├─{postgres},83775,83764
      ├─{postgres},83776,83764
      ├─{postgres},83777,83764
      ├─{postgres},83778,83764
      ├─{postgres},83779,83764
      ├─{postgres},83780,83764
      ├─{postgres},83781,83764
      ├─{postgres},83782,83764
      ├─{postgres},83783,83764
      ├─{postgres},83784,83764
      ├─{postgres},83785,83764
      ├─{postgres},83786,83764
      ├─{postgres},83787,83764
      ├─{postgres},83788,83764
      ├─{postgres},83789,83764
      ├─{postgres},83790,83764
      ├─{postgres},83791,83764
      ├─{postgres},83792,83764
      ├─{postgres},83793,83764
      └─{postgres},83794,83764

Backtrace of an attached GDB to 83764 (the process with the multiple children is always the one that segfaults):

(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00007f96153867da in ?? () from /usr/lib/libcrypto.so.3
#2  0x00007f9615389dc4 in OPENSSL_thread_stop () from /usr/lib/libcrypto.so.3
#3  0x00007f9615389e27 in OPENSSL_cleanup () from /usr/lib/libcrypto.so.3
#4  0x00007f9614a505e1 in ?? () from /usr/lib/libc.so.6
#5  0x00007f9614a506be in exit () from /usr/lib/libc.so.6
#6  0x0000555c082757b0 in proc_exit ()
#7  0x0000555c0840ea65 in errfinish ()
#8  0x0000555c07eb6a2f in ?? ()
#9  0x0000555c082982d8 in ProcessClientReadInterrupt ()
#10 0x0000555c081381e5 in secure_read ()
#11 0x0000555c08144521 in ?? ()
#12 0x0000555c082a1e65 in PostgresMain ()
#13 0x0000555c08206e83 in ?? ()
#14 0x0000555c08207e0f in PostmasterMain ()
#15 0x0000555c07ee0641 in main ()
#16 0x00007f9614a376b5 in ?? () from /usr/lib/libc.so.6
#17 0x00007f9614a37769 in __libc_start_main () from /usr/lib/libc.so.6
#18 0x0000555c07ee0c55 in _start ()

Possibly connected with #485.

Metadata

Metadata

Assignees

Labels

ext:needs-triageExternal issues that have been accepted and now need initial review/assessment/fix

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions