Support config v2 for nemesis and stress tool

[kobzonega]

Changelog entry

Support config v2 for nemesis and stress tool via --yaml-config argument to path

Changelog category

• Not for changelog (changelog entry is not required)

Description for reviewers

How to run

sudo ./tool deploy_tools --cluster_path=/home/apkobzev/nemesis-2dc/databases.yaml --ydbd_path=/home/apkobzev/ydb/ydb/apps/ydbd/ydbd --yaml-config=/home/apkobzev/nemesis-2dc/config.yaml

sudo ./tool start_nemesis --cluster_path=/home/apkobzev/nemesis-2dc/databases.yaml --ydbd_path=/home/apkobzev/ydb/ydb/apps/ydbd/ydbd --yaml-config=/home/apkobzev/nemesis-2dc/config.yaml

config.yaml

metadata:
  kind: MainConfig
  cluster: ""
  version: 0

config:
  erasure: block-4-2 # default for static and dynamic groups
  default_disk_type: SSD # default disk type used for group creation of both kinds

  self_management_config:
    enabled: true # automatic management of static resources (static group, state storage, etc)

  bridge_config:
    piles:
    - name: r1
    - name: r2

  host_configs: # like host flavours in kube
  - drive:
    - path: /dev/disk/by-partlabel/kikimr_ssd_01
      type: SSD
    - path: /dev/disk/by-partlabel/kikimr_ssd_02
      type: SSD
    - path: /dev/disk/by-partlabel/kikimr_ssd_03
      type: SSD
    - path: /dev/disk/by-partlabel/kikimr_ssd_04
      type: SSD
    - path: /dev/disk/by-partlabel/kikimr_ssd_05
      type: SSD
    - path: /dev/disk/by-partlabel/kikimr_ssd_06
      type: SSD
    host_config_id: 1
  hosts:
  - {host: myt0-7691.search.yandex.net, location: {data_center: 'zone-a', rack:  1}, host_config_id: 1, bridge_pile_name: r1}
  - {host: myt0-7619.search.yandex.net, location: {data_center: 'zone-a', rack:  2}, host_config_id: 1, bridge_pile_name: r1}
  - {host: myt0-7612.search.yandex.net, location: {data_center: 'zone-a', rack:  3}, host_config_id: 1, bridge_pile_name: r1}
  - {host: myt0-7610.search.yandex.net, location: {data_center: 'zone-a', rack:  4}, host_config_id: 1, bridge_pile_name: r1}
  - {host: myt0-7564.search.yandex.net, location: {data_center: 'zone-a', rack:  5}, host_config_id: 1, bridge_pile_name: r1}
  - {host: myt0-7551.search.yandex.net, location: {data_center: 'zone-a', rack:  6}, host_config_id: 1, bridge_pile_name: r1}
  - {host: myt0-7548.search.yandex.net, location: {data_center: 'zone-a', rack:  7}, host_config_id: 1, bridge_pile_name: r1}
  - {host: myt0-7545.search.yandex.net, location: {data_center: 'zone-a', rack:  8}, host_config_id: 1, bridge_pile_name: r1}

  - {host: myt0-7542.search.yandex.net, location: {data_center: 'zone-b', rack:  1}, host_config_id: 1, bridge_pile_name: r2}
    #  - {host: myt0-7531.search.yandex.net, location: {data_center: 'zone-b', rack:  2}, host_config_id: 1, bridge_pile_name: r2}
  - {host: myt0-7635.search.yandex.net, location: {data_center: 'zone-b', rack:  2}, host_config_id: 1, bridge_pile_name: r2}
  - {host: myt0-7513.search.yandex.net, location: {data_center: 'zone-b', rack:  3}, host_config_id: 1, bridge_pile_name: r2}
  - {host: myt0-7512.search.yandex.net, location: {data_center: 'zone-b', rack:  4}, host_config_id: 1, bridge_pile_name: r2}
  - {host: myt0-7501.search.yandex.net, location: {data_center: 'zone-b', rack:  5}, host_config_id: 1, bridge_pile_name: r2}
  - {host: myt0-7461.search.yandex.net, location: {data_center: 'zone-b', rack:  6}, host_config_id: 1, bridge_pile_name: r2}
  - {host: myt0-7448.search.yandex.net, location: {data_center: 'zone-b', rack:  7}, host_config_id: 1, bridge_pile_name: r2}
  - {host: myt0-7442.search.yandex.net, location: {data_center: 'zone-b', rack:  8}, host_config_id: 1, bridge_pile_name: r2}

databases.yaml

domains:
  - domain_name: Root
    dynamic_slots: 16
    databases:
      - name: "testdb"
        storage_units:
          - count: 4
            kind: ssd
        compute_units:
          - count: 10
            kind: slot
            zone: any

nemesis.service systemd unit

[Unit]
Description=Nemesis
After=network-online.target
Wants=nemesis-autoconf.service
StartLimitInterval=10
StartLimitBurst=15

[Service]
Restart=always
RestartSec=1
User=kikimr
Environment=NEMESIS_USER=robot-nemesis
ExecStart=/Berkanavt/nemesis/bin/nemesis --ydb-cluster-template /Berkanavt/kikimr/cfg/databases.yaml --yaml-config /Berkanavt/kikimr/cfg/config.yaml --ydb-binary-path /Berkanavt/kikimr/bin/kikimr --enable-nemesis-list-filter-by-hostname --private-key-file /Berkanavt/nemesis/cfg/id_rsa
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=nemesis
SyslogFacility=daemon
SyslogLevel=err
LimitNOFILE=65536
LimitCORE=0
LimitMEMLOCK=32212254720

[Install]
WantedBy=multi-user.target

[github-actions]

⚪ 2025-07-07 08:57:26 UTC Pre-commit check linux-x86_64-release-asan for ee380de has started.
⚪ 2025-07-07 08:57:38 UTC Artifacts will be uploaded here
⚫ 2025-07-07 08:58:38 UTC Check cancelled

[github-actions]

⚪ 2025-07-07 08:57:58 UTC Pre-commit check linux-x86_64-relwithdebinfo for ee380de has started.
⚫ 2025-07-07 08:58:37 UTC Check cancelled

[github-actions]

🟢 2025-07-07 10:30:10 UTC The validation of the Pull Request description is successful.

[github-actions]

⚪ 2025-07-07 09:06:14 UTC Pre-commit check linux-x86_64-relwithdebinfo for 1b6ff94 has started.
⚪ 2025-07-07 09:06:18 UTC Artifacts will be uploaded here
⚪ 2025-07-07 09:08:57 UTC ya make is running...
⚫ 2025-07-07 09:13:17 UTC Check cancelled

[github-actions]

⚪ 2025-07-07 09:06:21 UTC Pre-commit check linux-x86_64-release-asan for 1b6ff94 has started.
⚪ 2025-07-07 09:06:32 UTC Artifacts will be uploaded here
⚪ 2025-07-07 09:09:14 UTC ya make is running...
⚫ 2025-07-07 09:13:18 UTC Check cancelled

[github-actions]

⚪ 2025-07-07 09:16:24 UTC Pre-commit check linux-x86_64-release-asan for 4305303 has started.
⚪ 2025-07-07 09:16:56 UTC Artifacts will be uploaded here
⚪ 2025-07-07 09:19:41 UTC ya make is running...
⚫ 2025-07-07 09:58:18 UTC Check cancelled

[github-actions]

⚪ 2025-07-07 09:19:29 UTC Pre-commit check linux-x86_64-relwithdebinfo for 4305303 has started.
⚪ 2025-07-07 09:19:41 UTC Artifacts will be uploaded here
⚪ 2025-07-07 09:22:19 UTC ya make is running...
🟡 2025-07-07 09:57:26 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
6386	6221	0	1	149	15

🟢 2025-07-07 09:58:14 UTC ydbd size 2.2 GiB changed* by 0 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash	main: 1e497ce	merge: 4305303	diff	diff %
ydbd size	2 387 893 864 Bytes	2 387 893 864 Bytes	0 Bytes	0.000%
ydbd stripped size	499 612 008 Bytes	499 612 008 Bytes	0 Bytes	0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}
⚫ 2025-07-07 09:58:16 UTC Check cancelled

[github-actions]

⚪ 2025-07-07 10:17:15 UTC Pre-commit check linux-x86_64-release-asan for 355cfee has started.
⚪ 2025-07-07 10:17:26 UTC Artifacts will be uploaded here
⚪ 2025-07-07 10:20:04 UTC ya make is running...
🟡 2025-07-07 11:14:28 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
2670	2521	0	23	117	9

🟢 2025-07-07 11:14:51 UTC Build successful.
🟢 2025-07-07 11:15:11 UTC ydbd size 3.9 GiB changed* by +5.5 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: 1e497ce	merge: 355cfee	diff	diff %
ydbd size	4 200 039 280 Bytes	4 200 044 888 Bytes	+5.5 KiB	+0.000%
ydbd stripped size	1 455 242 744 Bytes	1 455 242 360 Bytes	-384 Bytes	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

[github-actions]

⚪ 2025-07-07 10:17:16 UTC Pre-commit check linux-x86_64-relwithdebinfo for 355cfee has started.
⚪ 2025-07-07 10:17:27 UTC Artifacts will be uploaded here
⚪ 2025-07-07 10:20:08 UTC ya make is running...
🟡 2025-07-07 10:57:21 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
6386	6207	0	9	155	15

⚪ 2025-07-07 10:57:59 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-07-07 11:03:34 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
113 (only retried tests)	98	0	1	2	12

⚪ 2025-07-07 11:03:42 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-07-07 11:08:47 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
28 (only retried tests)	13	0	1	2	12

🟢 2025-07-07 11:08:54 UTC Build successful.
🟢 2025-07-07 11:09:08 UTC ydbd size 2.2 GiB changed* by -312 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash	main: 1e497ce	merge: 355cfee	diff	diff %
ydbd size	2 387 893 864 Bytes	2 387 893 552 Bytes	-312 Bytes	-0.000%
ydbd stripped size	499 612 008 Bytes	499 612 200 Bytes	+192 Bytes	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

[github-actions]

⚪ 2025-07-07 12:15:33 UTC Pre-commit check linux-x86_64-release-asan for d329390 has started.
⚪ 2025-07-07 12:15:45 UTC Artifacts will be uploaded here
⚪ 2025-07-07 12:18:30 UTC ya make is running...
🟡 2025-07-07 13:16:34 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
2684	2512	0	23	142	7

🟢 2025-07-07 13:16:57 UTC Build successful.
🟡 2025-07-07 13:17:18 UTC ydbd size 3.9 GiB changed* by +152.8 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: 2674df3	merge: d329390	diff	diff %
ydbd size	4 200 044 824 Bytes	4 200 201 328 Bytes	+152.8 KiB	+0.004%
ydbd stripped size	1 455 242 360 Bytes	1 455 292 632 Bytes	+49.1 KiB	+0.003%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

[github-actions]

⚪ 2025-07-07 12:16:20 UTC Pre-commit check linux-x86_64-relwithdebinfo for d329390 has started.
⚪ 2025-07-07 12:16:32 UTC Artifacts will be uploaded here
⚪ 2025-07-07 12:19:10 UTC ya make is running...
🟡 2025-07-07 12:49:35 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
6400	6242	0	1	143	14

⚪ 2025-07-07 12:50:13 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-07-07 12:58:29 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
32 (only retried tests)	18	0	0	2	12

🟢 2025-07-07 12:58:36 UTC Build successful.
🟢 2025-07-07 12:58:49 UTC ydbd size 2.2 GiB changed* by +60.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: 2674df3	merge: d329390	diff	diff %
ydbd size	2 387 893 536 Bytes	2 387 955 072 Bytes	+60.1 KiB	+0.003%
ydbd stripped size	499 612 200 Bytes	499 610 280 Bytes	-1.9 KiB	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}