Node version v0.8.13 as leader gets stuck every 2-3 hours after restart

Describe the bug
My node doesn't run longer than 2-3 hours stable. I have tried tweaking my node-config to get an optimal result, I tried different configurations in order to solve this, but I don't think it's the node-config.

Mandatory Information

1. jcli --full-version jcli 0.8.13 (HEAD-241b3a59, release, linux [x86_64]) - [rustc 1.41.0 (5e1a79984 2020-01-27)];
2. jormungandr --full-version jormungandr 0.8.13 (HEAD-241b3a59, release, linux [x86_64]) - [rustc 1.41.0 (5e1a79984 2020-01-27)];

To Reproduce
Steps to reproduce the behavior:

1. When the process of jormungandr has stopped, I'll run it with start_leader with the following configuration.

log:
- output: stderr
  format: plain
  level: info
http_fetch_block0_service:
  - "https://github.com/input-output-hk/jormungandr-block0/raw/master/data/"
skip_bootstrap: false
bootstrap_from_trusted_peers: false
p2p:
  listen_address: "/ip4/0.0.0.0/tcp/17900"
  public_address: "/ip4/x.x.x.x/tcp/17900"
  topics_of_interest:
    blocks: high
    messages: high
  max_connections: 600
  max_bootstrap_attempts: 3
  max_unreachable_nodes_to_connect_per_event: 18
  gossip_interval: 5s
  policy:
    quarantine_duration: 10m
  trusted_peers:
    - address: "/ip4/3.231.168.222/tcp/3000"
      id: ff2aaaac6cab77d3fb72bf3cb9079246eca323c60b2fd68a
    - address: "/ip4/13.56.0.226/tcp/3000"
      id: 7ddf203c86a012e8863ef19d96aabba23d2445c492d86267
    - address: "/ip4/52.28.91.178/tcp/3000"
      id: 23b3ca09c644fe8098f64c24d75d9f79c8e058642e63a28c
    - address: "/ip4/3.125.75.156/tcp/3000"
      id: 22fb117f9f72f38b21bca5c0f069766c0d4327925d967791
    - address: "/ip4/13.112.181.42/tcp/3000"
      id: 52762c49a84699d43c96fdfe6de18079fb2512077d6aa5bc
    - address: "/ip4/13.114.196.228/tcp/3000"
      id: 7e1020c2e2107a849a8353876d047085f475c9bc646e42e9
    - address: "/ip4/52.8.15.52/tcp/3000"
      id: 18bf81a75e5b15a49b843a66f61602e14d4261fb5595b5f5
    - address: "/ip4/52.9.132.248/tcp/3000"
      id: 671a9e7a5c739532668511bea823f0f5c5557c99b813456c
    - address: "/ip4/3.125.183.71/tcp/3000"
      id: 9d15a9e2f1336c7acda8ced34e929f697dc24ea0910c3e67
    - address: "/ip4/18.184.35.137/tcp/3000"
      id: 06aa98b0ab6589f464d08911717115ef354161f0dc727858
    - address: "/ip4/18.182.115.51/tcp/3000"
      id: 8529e334a39a5b6033b698be2040b1089d8f67e0102e2575
    - address: "/ip4/3.115.154.161/tcp/3000"
      id: 35bead7d45b3b8bda5e74aa12126d871069e7617b7f4fe62
    - address: "/ip4/18.177.78.96/tcp/3000"
      id: fc89bff08ec4e054b4f03106f5312834abdf2fcb444610e9
    - address: "/ip4/52.9.77.197/tcp/3000"
      id: fcdf302895236d012635052725a0cdfc2e8ee394a1935b63
    - address: "/ip4/54.183.149.167/tcp/3000"
      id: df02383863ae5e14fea5d51a092585da34e689a73f704613
    - address: "/ip4/3.124.116.145/tcp/3000"
      id: 99cb10f53185fbef110472d45a36082905ee12df8a049b74
rest:
  listen: "127.0.0.1:3100"
storage: /home/ada/storage
explorer:
  enabled: false
mempool:
    pool_max_entries: 10000
    log_max_entries: 100000
leadership:
    logs_capacity: 4096

2. The node runs for some time and get's stuck very quickly and shoots up the amount of max_connections, that is usally a good indication that it's not registering any blocks.
3. In the logs I see the following stuck-notifier messages:

Mar 05 16:02:43.091 WARN blockchain is not moving up, system-date=82.37473, the last tip c19b40e7-000426db-82.24939 was 25068 seconds ago, task: stuck_notifier
Mar 05 16:03:43.093 WARN blockchain is not moving up, system-date=82.37503, the last tip c19b40e7-000426db-82.24939 was 25128 seconds ago, task: stuck_notifier
Mar 05 16:04:43.093 WARN blockchain is not moving up, system-date=82.37533, the last tip c19b40e7-000426db-82.24939 was 25188 seconds ago, task: stuck_notifier
Mar 05 16:05:43.091 WARN blockchain is not moving up, system-date=82.37563, the last tip c19b40e7-000426db-82.24939 was 25248 seconds ago, task: stuck_notifier
Mar 05 16:06:43.092 WARN blockchain is not moving up, system-date=82.37593, the last tip c19b40e7-000426db-82.24939 was 25308 seconds ago, task: stuck_notifier
Mar 05 16:07:43.092 WARN blockchain is not moving up, system-date=82.37623, the last tip c19b40e7-000426db-82.24939 was 25368 seconds ago, task: stuck_notifier
Mar 05 16:08:43.093 WARN blockchain is not moving up, system-date=82.37653, the last tip c19b40e7-000426db-82.24939 was 25428 seconds ago, task: stuck_notifier

4. With a restart-script that checks if the node is stuck, that process also get clogged as sometimes it stops at bootstrapping process.
5. As a result:

Has this node been scheduled to be leader?
---
- created_at_time: "2020-03-05T07:33:47.290214282+00:00"
  enclave_leader_id: 1
  finished_at_time: "2020-03-05T12:50:41.000850225+00:00"
  scheduled_at_date: "82.31711"
  scheduled_at_time: "2020-03-05T12:50:39+00:00"
  status:
    Rejected:
      reason: Failed to compute the schedule within time boundaries
  wake_at_time: "2020-03-05T12:50:39.001462736+00:00"

Expected behavior
A stable node as a leader, running on 4cpu, 16gb machine with ubuntu server and doesn't do anything else.

[cardanoinvest]

Yeah similar, everything after v0.8.9 is getting stuck with no error.

[bergr01]

Sometimes happens on latest master (as of 10 March, 2020) as well. Hanging is characterized by the timing out of the 'jcli rest v0 network stats get' command:

jcli rest v0 network stats get | wc
failed to make a REST request
|-> connection with node timed out
|-> http://127.0.0.1:8504/api/v0/network/stats: timed out

netstat data for the hung node:

1 CLOSE_WAIT
471 ESTABLISHED
2 LISTEN

no ERRO level messages in the log, the only WARN is the blockchain not moving - the only resolution so far seems to be a restart.

[ilhanu]

Same with the latest v0.8.14.

The node was up since 16:30 UTC, and started getting stuck at 89.40127

Mar 12 19:02:58.261 WARN blockchain is not moving up, system-date=89.42880, the last tip a7caa4b7-00048ca2-89.40127 was 5507 seconds ago, task: stuck_notifier
Mar 12 19:03:58.262 WARN blockchain is not moving up, system-date=89.42910, the last tip a7caa4b7-00048ca2-89.40127 was 5567 seconds ago, task: stuck_notifier
Mar 12 19:04:58.262 WARN blockchain is not moving up, system-date=89.42940, the last tip a7caa4b7-00048ca2-89.40127 was 5627 seconds ago, task: stuck_notifier
Mar 12 19:05:58.261 WARN blockchain is not moving up, system-date=89.42970, the last tip a7caa4b7-00048ca2-89.40127 was 5687 seconds ago, task: stuck_notifier
Mar 12 19:06:58.262 WARN blockchain is not moving up, system-date=89.43000, the last tip a7caa4b7-00048ca2-89.40127 was 5747 seconds ago, task: stuck_notifier
Mar 12 19:07:58.262 WARN blockchain is not moving up, system-date=89.43030, the last tip a7caa4b7-00048ca2-89.40127 was 5807 seconds ago, task: stuck_notifier
Mar 12 19:08:58.261 WARN blockchain is not moving up, system-date=89.43060, the last tip a7caa4b7-00048ca2-89.40127 was 5867 seconds ago, task: stuck_notifier
Mar 12 19:09:58.262 WARN blockchain is not moving up, system-date=89.43090, the last tip a7caa4b7-00048ca2-89.40127 was 5927 seconds ago, task: stuck_notifier
Mar 12 19:10:58.261 WARN blockchain is not moving up, system-date=89.43120, the last tip a7caa4b7-00048ca2-89.40127 was 5987 seconds ago, task: stuck_notifier
Mar 12 19:11:58.262 WARN blockchain is not moving up, system-date=89.43150, the last tip a7caa4b7-00048ca2-89.40127 was 6047 seconds ago, task: stuck_notifier
Mar 12 19:12:58.261 WARN blockchain is not moving up, system-date=89.43180, the last tip a7caa4b7-00048ca2-89.40127 was 6107 seconds ago, task: stuck_notifier

[bergr01]

Happening on 0.8.14 too - last time it happened the jormungandr process memory usage was at a whopping 3 GB RAM on a Linux box, and log file was filled with frequently repeating:

Mar 13 06:05:47.390 INFO create new branch with tip d3303226-000492c5-90.16469 | current-tip ba9806aa-00049307-90.17331, task: block
Mar 13 06:05:47.390 INFO create new branch with tip d4121349-000492c8-90.16482 | current-tip ba9806aa-00049307-90.17331, task: block
Mar 13 06:05:47.391 INFO create new branch with tip 26d198b0-000492ca-90.16515 | current-tip ba9806aa-00049307-90.17331, task: block
Mar 13 06:05:47.391 INFO create new branch with tip c693918a-000492cc-90.16541 | current-tip ba9806aa-00049307-90.17331, task: block
Mar 13 06:05:47.391 INFO create new branch with tip d8f8d24a-000492ce-90.16589 | current-tip ba9806aa-00049307-90.17331, task: block

Still on version v0.8.14, Had one block planned this morning. The chain stopped moving first at e16b61bf-0004ad4f-92.14177 after that 31563e60-0004ad50-92.16623 which was the slot that was planned, didn't get into the ledger and that is exactly from where it didn't move further.

The leader_logs:

---
- created_at_time: "2020-03-14T22:07:36.161218829+00:00"
  enclave_leader_id: 1
  finished_at_time: "2020-03-15T04:27:43.006138376+00:00"
  scheduled_at_date: "92.16623"
  scheduled_at_time: "2020-03-15T04:27:43+00:00"
  status:
    Block:
      block: 31563e60e57414c7c443979af2bc8782b90ff048a686fc505a1b017d5cd50be0
      chain_length: 306512
      parent: e16b61bf485a68575567f3e596d7686a588c654efa2682adb3a016804e69cdc2
  wake_at_time: "2020-03-15T04:27:43.001058026+00:00"

A long list of stuck notifiers:

Mar 15 07:22:28.539 WARN blockchain is not moving up, system-date=92.21865, the last tip 31563e60-0004ad50-92.16623 was 10485 seconds ago, task: stuck_notifier
Mar 15 07:23:28.541 WARN blockchain is not moving up, system-date=92.21895, the last tip 31563e60-0004ad50-92.16623 was 10545 seconds ago, task: stuck_notifier
Mar 15 07:24:28.542 WARN blockchain is not moving up, system-date=92.21925, the last tip 31563e60-0004ad50-92.16623 was 10605 seconds ago, task: stuck_notifier

[cardanoinvest]

Still on version v0.8.14

@NicolasDP You have any idea what might cause this, the same problems I run into with v0.8.15, is there anything I can provide to solve this issue or an action I can do that it doesn't fork every 2-3 hours.

Deleting the listen_address give me the following output

Mar 18 20:42:45.296 INFO loading from storage, currently at 5b8cd795-0004b12d-92.26295 processing=236.975512ms (94.79µs per block) ..., task: bootstrap
Mar 18 20:42:45.535 INFO loading from storage, currently at 547abab0-0004baf1-93.12515 processing=237.38194ms (94.952µs per block) ..., task: bootstrap
Mar 18 20:42:45.694 INFO loading from storage, currently at 563cb364-0004c4b5-93.41967 processing=156.763994ms (62.705µs per block) ..., task: bootstrap
Mar 18 20:42:45.930 INFO loading from storage, currently at 25001c4e-0004ce79-94.26665 processing=234.457225ms (93.782µs per block) ..., task: bootstrap
Mar 18 20:42:46.157 INFO loading from storage, currently at f6da637c-0004d83d-95.12545 processing=224.602863ms (89.841µs per block) ..., task: bootstrap
Mar 18 20:42:46.318 INFO loading from storage, currently at 85abf29e-0004e201-95.40833 processing=159.326441ms (63.73µs per block) ..., task: bootstrap
Mar 18 20:42:46.427 INFO Loaded from storage tip is : 434718b5-0004e3c1-96.2582, task: bootstrap
Mar 18 20:42:46.427 INFO getting peers from bootstrap peer 3.115.154.161:3000, peer_addr: 3.115.154.161:3000, task: bootstrap
Mar 18 20:42:46.951 INFO peer 3.115.154.161:3000 : peers known : 44, peer_addr: 3.115.154.161:3000, task: bootstrap
Mar 18 20:42:46.952 INFO adding 43 peers from peer, task: bootstrap
Mar 18 20:42:56.955 WARN unable to reach peer for initial bootstrap, reason: connection timed out, peer_addr: 94.254.35.248:3300, task: bootstrap
Mar 18 20:42:57.144 INFO pulling blocks starting from checkpoints: Checkpoints([Hash(Blake2b256(0x434718b593433be479e8d8cec7eb1cc2937e5a38873cf885494b3b4904c76903)), Hash(Blake2b256(0x3bae7cc052f8da7857027697c613c34bb0d57d52d898b9ebff477b4b06294d64)), Hash(Blake2b256(0xd90a40154cce286b59cb285ae024e6274057d1718beb61e2b48e4271b5ff0029)), Hash(Blake2b256(0xe5d1c1067fc8bae7170488d51f8f8d32660e6e59e7fdaef8259e1573bbb61ae2)), Hash(Blake2b256(0xfcbecde87f100de36617b1f5baf5ed4c424b2d391a0aa5eaa7ed50155fc702a7)), Hash(Blake2b256(0xe2fa7441080f1809b1fb0c4537d45e43bdcf539a14e3f0d01b62c1c908c8883a)), Hash(Blake2b256(0xe5235df4f9c45e33caabfaf97d65b13c900980a409c7d7968a4966e5d28dcfe1)), Hash(Blake2b256(0x117d504f5cd5896b7341ece6f2ddb03f03b96b63a887c725496870ae5750ce27)), Hash(Blake2b256(0x4ade08b6c6277c7e2f300a6f208c65df90f5063de459e128544f9624bacb10e9)), Hash(Blake2b256(0x490c1ee2ed74b837e728ca93f44ea3ef2eece10978bc963ec09dfccf678ddf36)), Hash(Blake2b256(0x651eacbfcbe62677340b9afe064ea410197b05af7d709b5d3c8d03311ab4b8b7)), Hash(Blake2b256(0xa4f493327653b78271b8a9dbd855011f7662b2108eb153762f882180473708fa)), Hash(Blake2b256(0xa907c0aaf9ba78a249e7427f4d31cec2ba883d639357466035587177d4b8cc4e)), Hash(Blake2b256(0x0ea6ced8c0955da110f912f93656f43b937b03ea542dc56539b158ea7d646b05)), Hash(Blake2b256(0x20a41514f1b3a670569cdfd4dea1770fe50ed234211a197bd46227c3834a76e8)), Hash(Blake2b256(0x4ec0cb36c64230e98992e5d8dd2e494b3e5b159e83fe02a8f2565603a02c4c36))]), peer_addr: 165.227.7.116:3000, task: bootstrap
Mar 18 20:42:57.617 INFO switching branch from 434718b5-0004e3c1-96.2582 to dd04733e-0004e3c4-96.2619, peer_addr: 165.227.7.116:3000, task: bootstrap
Mar 18 20:42:57.633 INFO initial bootstrap completed, peer_addr: 165.227.7.116:3000, task: bootstrap
Mar 18 20:42:57.646 INFO listening and accepting gRPC connections, local_addr: 94.210.207.154:3000, task: network
Mar 18 20:42:57.646 ERRO failed to listen for P2P connections at 94.210.207.154:3000, reason: failed to listen for connections on 94.210.207.154:3000: Cannot assign requested address (os error 99), task: network
Mar 18 20:42:57.646 INFO connecting to peer, node_id: 35bead7d45b3b8bda5e74aa12126d871069e7617b7f4fe62, peer_addr: 3.115.154.161:3000, task: network
Mar 18 20:42:57.646 INFO connecting to peer, node_id: ff2aaaac6cab77d3fb72bf3cb9079246eca323c60b2fd68a, peer_addr: 3.231.168.222:3000, task: network
Mar 18 20:42:57.646 INFO connecting to peer, node_id: 18bf81a75e5b15a49b843a66f61602e14d4261fb5595b5f5, peer_addr: 52.8.15.52:3000, task: network
Mar 18 20:42:57.646 INFO connecting to peer, node_id: 23b3ca09c644fe8098f64c24d75d9f79c8e058642e63a28c, peer_addr: 52.28.91.178:3000, task: network
Mar 18 20:42:57.646 INFO connecting to peer, node_id: 52762c49a84699d43c96fdfe6de18079fb2512077d6aa5bc, peer_addr: 13.112.181.42:3000, task: network
Mar 18 20:42:57.646 INFO connecting to peer, node_id: 22fb117f9f72f38b21bca5c0f069766c0d4327925d967791, peer_addr: 3.125.75.156:3000, task: network
Mar 18 20:42:57.646 INFO connecting to peer, node_id: 9d15a9e2f1336c7acda8ced34e929f697dc24ea0910c3e67, peer_addr: 3.125.183.71:3000, task: network
Mar 18 20:42:57.646 INFO connecting to peer, node_id: df02383863ae5e14fea5d51a092585da34e689a73f704613, peer_addr: 54.183.149.167:3000, task: network
Mar 18 20:42:57.654 INFO connecting to peer, node_id: 7e1020c2e2107a849a8353876d047085f475c9bc646e42e9, peer_addr: 13.114.196.228:3000, task: network
Mar 18 20:42:57.654 INFO connecting to peer, node_id: fc89bff08ec4e054b4f03106f5312834abdf2fcb444610e9, peer_addr: 18.177.78.96:3000, task: network
Mar 18 20:42:57.654 INFO connecting to peer, node_id: 06aa98b0ab6589f464d08911717115ef354161f0dc727858, peer_addr: 18.184.35.137:3000, task: network
Mar 18 20:42:57.654 INFO connecting to peer, node_id: 7ddf203c86a012e8863ef19d96aabba23d2445c492d86267, peer_addr: 13.56.0.226:3000, task: network
Mar 18 20:42:57.654 INFO connecting to peer, node_id: fcdf302895236d012635052725a0cdfc2e8ee394a1935b63, peer_addr: 52.9.77.197:3000, task: network
Mar 18 20:42:57.654 INFO connecting to peer, node_id: 99cb10f53185fbef110472d45a36082905ee12df8a049b74, peer_addr: 3.124.116.145:3000, task: network
Mar 18 20:42:57.654 INFO connecting to peer, node_id: 671a9e7a5c739532668511bea823f0f5c5557c99b813456c, peer_addr: 52.9.132.248:3000, task: network
Mar 18 20:42:57.654 INFO connecting to peer, node_id: 8529e334a39a5b6033b698be2040b1089d8f67e0102e2575, peer_addr: 18.182.115.51:3000, task: network
Mar 18 20:42:57.654 INFO service finished with error, task: network
Mar 18 20:42:57.654 CRIT Service has terminated with an error
A service has terminated with an error

[cardanoinvest]

Still on version v0.8.15

The stats of the node with getting stuck are better now, but it's showing its stuck in the logs, while it's not reall

Mar 22 13:20:47.625 WARN blockchain is not moving up, system-date=99.32615, the last tip 3e88c93b-000518c8-99.30796 was 3638 seconds ago, task: stuck_notifier
Mar 22 13:21:47.628 WARN blockchain is not moving up, system-date=99.32645, the last tip 701d4fda-000518cc-99.30836 was 3618 seconds ago, task: stuck_notifier
Mar 22 13:22:47.629 WARN blockchain is not moving up, system-date=99.32675, the last tip eef4aee4-000518cf-99.30866 was 3618 seconds ago, task: stuck_notifier
Mar 22 13:23:47.628 WARN blockchain is not moving up, system-date=99.32705, the last tip eef4aee4-000518cf-99.30866 was 3678 seconds ago, task: stuck_notifier
Mar 22 13:24:47.628 WARN blockchain is not moving up, system-date=99.32735, the last tip ba35413e-000518d2-99.30923 was 3624 seconds ago, task: stuck_notifier
Mar 22 13:25:47.629 WARN blockchain is not moving up, system-date=99.32765, the last tip c587d828-000518d6-99.30949 was 3632 seconds ago, task: stuck_notifier
Mar 22 13:26:47.628 WARN blockchain is not moving up, system-date=99.32795, the last tip c587d828-000518d6-99.30949 was 3692 seconds ago, task: stuck_notifier
Mar 22 13:27:26.356 INFO PushHeaders request failed, error: Error { code: Unavailable, source: Status { code: Unavailable, message: "the service is unavailable (oneshot canceled)" } }, node_id: 7a043f677b8404ca571a7bf4518ed6d0b22262f696ff0b31, peer_addr: 18.176.194.154:3000, task: network
Mar 22 13:27:47.628 WARN blockchain is not moving up, system-date=99.32825, the last tip 78828dc0-000518da-99.31019 was 3612 seconds ago, task: stuck_notifier
Mar 22 13:28:47.628 WARN blockchain is not moving up, system-date=99.32855, the last tip ef3fe601-000518dd-99.31045 was 3620 seconds ago, task: stuck_notifier
Mar 22 13:29:47.628 WARN blockchain is not moving up, system-date=99.32885, the last tip 6d623611-000518df-99.31076 was 3618 seconds ago, task: stuck_notifier
Mar 22 13:30:13.080 INFO PushHeaders request failed, error: Error { code: Unavailable, source: Status { code: Unavailable, message: "the service is unavailable (oneshot canceled)" } }, node_id: 000000369369369369369124875136936936936936921000, peer_addr: 202.92.217.26:3201, task: network
Mar 22 13:30:47.628 WARN blockchain is not moving up, system-date=99.32915, the last tip aceeadf7-000518e2-99.31097 was 3636 seconds ago, task: stuck_notifier
Mar 22 13:31:47.629 WARN blockchain is not moving up, system-date=99.32945, the last tip a870dba6-000518e6-99.31145 was 3600 seconds ago, task: stuck_notifier
Mar 22 13:32:47.628 WARN blockchain is not moving up, system-date=99.32975, the last tip e5da77de-000518e9-99.31174 was 3602 seconds ago, task: stuck_notifier
Mar 22 13:33:47.628 WARN blockchain is not moving up, system-date=99.33005, the last tip c6ee1ce1-000518eb-99.31197 was 3616 seconds ago, task: stuck_notifier
Mar 22 13:34:47.625 WARN blockchain is not moving up, system-date=99.33035, the last tip c6ee1ce1-000518eb-99.31197 was 3676 seconds ago, task: stuck_notifier
Mar 22 13:35:47.629 WARN blockchain is not moving up, system-date=99.33065, the last tip e6dbf9cf-000518ef-99.31242 was 3646 seconds ago, task: stuck_notifier
Mar 22 13:36:47.628 WARN blockchain is not moving up, system-date=99.33095, the last tip eb342b72-000518f4-99.31294 was 3602 seconds ago, task: stuck_notifier
Mar 22 13:37:47.628 WARN blockchain is not moving up, system-date=99.33125, the last tip b3420ec8-000518f8-99.31320 was 3610 seconds ago, task: stuck_notifier
Mar 22 13:38:47.629 WARN blockchain is not moving up, system-date=99.33155, the last tip 234cfe9f-000518fd-99.31354 was 3602 seconds ago, task: stuck_notifier

As stats it's giving me the following:

---

blockRecvCnt: 426
lastBlockContentSize: 0
lastBlockDate: "99.31361"
lastBlockFees: 0
lastBlockHash: 2743803b52715d35fd530dceaec811ea974101ba6ed4ec1d726e43551758bff5
lastBlockHeight: "334079"
lastBlockSum: 0
lastBlockTime: "2020-03-22T12:38:59+00:00"
lastBlockTx: 0
lastReceivedBlockTime: "2020-03-22T13:39:03+00:00"
nodeId: 753c9cbbf95dac8f6738042feaf9b507e05f2fcd5baea171
peerAvailableCnt: 3104
peerConnectedCnt: 133
peerQuarantinedCnt: 7128
peerTotalCnt: 10240
peerUnreachableCnt: 8
state: Running
txRecvCnt: 17
uptime: 8080
version: jormungandr 0.8.15-19f8eb49