Update stable-24-3: add metrics to describe table #8688
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
swalrus1
commented
Sep 3, 2024
swalrus1
commented
- Only members of ReleaseApprovers team can approve changes in this branch (Only members of ReleaseApprovers team can approve changes in this branch #6323)
- Initial commit for stable-24-3 (Initial commit for stable-24-3 #6321)
- fix query errors processing (fix query errors processing #6441) (fix query errors processing (#6441) #6451)
- Merge Handle empty GC commands correctly #6398 Fix VDisk replication stuck sensor bug #6405 from main into stable-24-3 (Merge #6398 #6405 from main into stable-24-3 #6448)
- Fix test_db_counters test (Fix test_db_counters test #6436)
- 24-3 Fix tests with EnableLocalDBBtreeIndex = false (24-3 Fix tests with EnableLocalDBBtreeIndex = false #6391)
- 24-3 Iterative B-Tree histograms builder (24-3 Iterative B-Tree histograms builder #6392)
- 24-3 Obtain UsedTableMemory from private cache stats, include historic indexes size (24-3 Obtain UsedTableMemory from private cache stats, include historic indexes size #6400)
- revert 19bed6c (revert 19bed6c19f373cc26a3b8d1d4e4d1ff1126a56ce #6469)
- 24-3: Fix bugs in coordinator state migration (24-3: Fix bugs in coordinator state migration #6460)
- Merge some fixes (Merge some fixes #6490)
- set predicate-extract config default (set predicate-extract config default #6456)
- 24-3: Fix bugs in: change exchange split, removing schema snapshots & emitting of resolved timestamps (24-3: Fix bugs in: change exchange split, removing schema snapshots & emitting of resolved timestamps #6616)
- 24.3: Fix null dereference in node broker (KIKIMR-21693) (KIKIMR-21693: Fix null dereference in node broker #6516) (24.3: Fix null dereference in node broker (KIKIMR-21693) (#6516) #6623)
- Use the whole TTableDescription to describe an indexImplTable in TIndexDescription (Use the whole TTableDescription to describe an indexImplTable in TIndexDescription #6280) (Use the whole TTableDescription to describe an indexImplTable in TIndexDescription (#6280) #6450)
- Changed memory calculation for TEvFreeItems (merge from main Changed memory calculation for TEvFreeItems #6625) (Changed memory calculation for TEvFreeItems (merge from main #6625) #6694)
- fix drain compatability (fix drain compatability #6613) (fix drain compatability (#6613) #6716)
- fix neighbours count (fix neighbours count #6409) (fix neighbours count (#6409) #6718)
- fix TabletsTotal counter (fix TabletsTotal counter #6492) (fix TabletsTotal counter (#6492) #6717)
- do not trigger emergency balancer when all nodes have high usage (do not trigger emergency balancer when all nodes have high usage #6532) (do not trigger emergency balancer when all nodes have high usage (#6532) #6720)
- 24-3: schemeshard: preserialize Table.SplitBoundary for describe result (24-3: schemeshard: preserialize Table.SplitBoundary for describe result #6847)
- 24-3: schemeshard: reject operations with too big local tx commit (schemeshard: reject operations with too big local tx commit #6760) (24-3: schemeshard: reject operations with too big local tx commit (#6760) #6849)
- 24-3: Fix read requirements on init (24-3: Fix read requirements on init #6948)
- build: refresh Embedded UI (v6.11.0) (build: refresh Embedded UI (v6.11.0) #6968)
- Extend DisabledOnSchemeShard FF for column tables on dedicated bases … (Extend DisabledOnSchemeShard FF for column tables on dedicated bases … #6962)
- Merge Fix LWTrace leaking HTTP request string to HTML page (Fix LWTrace leaking HTTP request string to HTML page #7011) (Fix LWTrace leaking HTTP request string to HTML page (#7011) #7014) (Merge Fix LWTrace leaking HTTP request string to HTML page (#7011) (#7014) #7024)
- deprioritise system tablets in balancer (deprioritise system tablets in balancer #6840) (deprioritise system tablets in balancer (#6840) #7047)
- use potential max thread count in local (use potential max thread count in local #7050)
- merge memory limitations features and staff to stable-24-3 (merge memory limitations features and staff to stable-24-3 #6803)
- Fix scope id handling (Fix scope id handling #7093)
- YQ-3345 support load cpu threshold (YQ-3345 support load cpu threshold #6790)
- YQ WM fixed build errors (YQ WM fixed build errors #7128)
- Disable modification in cs by default (Disable modification in cs by default #7161)
- CS improvements (CS improvements #7013)
- Dont create delete flags column if not necessary (Dont create delete flags column if not necessary #7169)
- 24-3: Enable/disable ssl connections, return connection_string in API (24-3: Enable/disable ssl connections, return connection_string in API #7147)
- 24-3: Check shard state at TTxCdcStreamEmitHeartbeats (24-3: Check shard state at TTxCdcStreamEmitHeartbeats #7148)
- 24-3: Allow streams on index tables, replicate index tables (24-3: Allow streams on index tables, replicate index tables #7150)
- add autopartitioning by load (add autopartitioning by load #7124)
- fix viewer redirects (fix viewer redirects #7103) (fix viewer redirects (#7103) #7120)
- 24-3 Freeze event spaces ids (24-3 Freeze event spaces ids #7192)
- Added metadata flag to track if stats for optimizer were loaded (Added metadata flag to track if stats for optimizer were loaded #7190)
- fix compaction intervals construction (fix compaction intervals construction #7176) (fix compaction intervals construction (#7176) #7208)
- add enable implicit params flag to table service config (add enable implicit params flag to table service config #6943)
- 24-3 Fix using uninitialized value error. (24-3 Fix using uninitialized value error. #7224)
- 24-3 [CBO] Warning FIX ([CBO] Warning FIX #6420) (24-3 [CBO] Warning FIX (#6420) #7241)
- fix segfault and memory leaks in CS (fix segfault and memory leaks in CS #7288)
- 24-3 Rescue asan tests (remove NO_EXPORT_DYNAMIC_SYMBOLS for them) (24-3 Rescue asan tests (remove NO_EXPORT_DYNAMIC_SYMBOLS for them) #7311)
- Enable sequences (Enable sequences #7297)
- YDB-7262 Add default permille ICB configuration for vdisk garbage compaction threshold (YDB-7262 Add default permille ICB configuration for vdisk garbage compaction threshold #7319)
- WM fixed bugs and performance (WM fixed bugs and performance #7254)
- hotfix for hotkeys (hotfix for hotkeys #7425)
- Fix actor system usage after it was freed (Fix actor system usage after it was freed #7316) (Fix actor system usage after it was freed (#7316) #7404)
- Disable vpatch 24-3 (Disable vpatch 24-3 #7320)
- PushLeftStage fix (PushLeftStage fix #7349)
- Ignore stored GCBarrierPreparation (Ignore stored GCBarrierPreparation #7473)
- add config option to specify TTL for user logins (add config option to specify TTL for user logins #7083) (add config option to specify TTL for user logins (#7083) #7486)
- fix for dead nodes always passing filter (fix for dead nodes always passing filter #6912) (fix for dead nodes always passing filter (#6912) #7522)
- stable-24-3: Enable COUNT in view queries (Enable COUNT in view queries #6820) (stable-24-3: Enable COUNT in view queries (#6820) #7523)
- Fix trim duration (change microseconds to milliseconds) (Fix trim duration (change microseconds to milliseconds) #7302)
- 24-3: Sinks improvements (24-3: Sinks improvements #6856)
- 24-3: CTAS fixes (24-3: CTAS fixes #6857)
- 24-3: Datashard + Columnshard Reads (24-3: Datashard + Columnshard Reads #6858)
- storage healthcheck fixes (fix missing static group issue #7212) (storage healthcheck fixes (#7212) #7394)
- control inflight pings in hive (control inflight pings in hive #6916) (control inflight pings in hive (#6916) #7238)
- Delete empty portions normalizer (Delete empty portions normalizer #7596) (Delete empty portions normalizer (#7596) #7600)
- fix suspended read session of a topic (fix suspended read session of a topic #7635)
- Allow logging AST with free arguments. (Allow logging AST with free arguments. #7573) (Allow logging AST with free arguments. (#7573) #7653)
- YDB-2568 Enable match_recognize in ydb / stable-24-3 (YDB-2568 Enable match_recognize in ydb / stable-24-3 #7540)
- 24-3: Fix unexpected read iterator stream reset (24-3: Fix unexpected read iterator stream reset #7710)
- Allow multiple join-broadcasts in single stage (Allow multiple join-broadcasts in single stage #7556) (Allow multiple join-broadcasts in single stage (#7556) #7713)
- [Http] Reply with structured issues when client accepts json data ([Http] Reply with structured issues when client accepts json data #7691)
- fix LockedPartitions without session (fix LockedPartitions without session #7736)
- fix query id for recompilation (fix query id for recompilation #7599)
- [CDC] Do not lose presition during float/double to json serialization… ([CDC] Do not lose presition during float/double to json serialization… #7737)
- Fix json float/double print format (Fix json float/double print format #7572) (Fix json float/double print format (#7572) #7776)
- Double free when accessing .AsBoxed() on Pod with 0 refcount (Double free when accessing .AsBoxed() on Pod with 0 refcount #7659) (Double free when accessing .AsBoxed() on Pod with 0 refcount (#7659) #7721)
- Fix segfault when UserToken == nullptr (Fix segfault when UserToken == nullptr #7849)
- Fix RU for kafka read (Fix RU for kafka read #7859)
- Fix optional columns (Fix optional columns #7875)
- dont use cpu for not abortable chunks (dont use cpu for not abortable chunks #7865) (dont use cpu for not abortable chunks (#7865) #7878)
- scan error processing on restore data (scan error processing on restore data #7813) (scan error processing on restore data (#7813) #7877)
- fix frozing of write session (fix frozing of write session #7958)
- Fix cs perf yaem (Fix cs perf yaem #8005)
- allow to configure min alloc size (allow to configure min alloc size #7951) (allow to configure min alloc size (#7951) #7996)
- enable UUID pk for 24.3 (enable UUID pk for 24.3 #8018)
- Merge Fix LWTrace leaking HTTP request string to HTML page (Fix LWTrace leaking HTTP request string to HTML page #7011) (Fix LWTrace leaking HTTP request string to HTML page (#7011) #7014) (Merge Fix LWTrace leaking HTTP request string to HTML page (#7011) (#7014) #7024)
- Enable DROP VIEW from a folder, bugfix (Enable DROP VIEW from a folder, bugfix #8066) (Enable DROP VIEW from a folder, bugfix (#8066) #8079)
- Fix describe consumer (Fix describe consumer #8100)
- Enabled statistics in stable (Enabled statistics in stable #8107)
- 24-3: Add basic dynconfig audit (24-3: Add basic dynconfig audit #8155)
- return implicit query params type into feature flag (return implicit query params type into feature flag #8063)
- 24-3: Allow to alter cdc topic's retention period (24-3: Allow to alter cdc topic's retention period #8289)
- PgWire auth with ApiKey (PgWire auth with ApiKey #8283)
- 24-3: Async replication: configurable default retention period (24-3: Async replication: configurable default retention period #8292)
- fix CORS headers (fix CORS headers #8303) (fix CORS headers (#8303) #8313)
- 24-3 Report index sizes by type (24-3 Report index sizes by type #8325)
- 24-3: Add sensors to Node Broker (Add sensors to Node Broker #8091) (24-3: Add sensors to Node Broker (#8091) #8311)
- asan fix
- ExecuteData set transaction mode always
- storage healthceck fixes (storage healthceck fixes #8460)
- 24-3: Enable CMS request priorities by default (Enable CMS request priorities by default #8380) (24-3: Enable CMS request priorities by default (#8380) #8413)
- Merge stable 24 3 (Merge stable 24 3 #8545)
- [Ldap] Merge ldap fixes to stable 24-3 ([Ldap] Merge ldap fixes to stable 24-3 #8326)
- 24-3: Reset __async_replica attr after changing REPLICATION_MODE to NONE (24-3: Reset __async_replica attr after changing REPLICATION_MODE to NONE #8620)
- Topics improvements for 24-3 (Topics improvements for 24-3 #8605)
- data query uses different local compute tasks param (24-3) (data query uses different local compute tasks param (24-3) #8633)
- fixes applied to sessions sysview (fixes applied to sessions sysview #8667)
- add missing table stats for column shards (add missing table stats for column shards #7140)
- Fix table stats aggregation for OLAP tables (Fix table stats aggregation for OLAP tables #8417)
- add missing monitoring metrics for column shards (add missing monitoring metrics for column shards #7314)
Co-authored-by: Tony-Romanov <150126326+Tony-Romanov@users.noreply.github.com>
… emitting of resolved timestamps (ydb-platform#6616)
…form#6625) (ydb-platform#6694) Co-authored-by: mregrock <mregrock@yandex.ru> Co-authored-by: Egor Kulin <mregrock@localhost.localdomain>
…lt (ydb-platform#6847) merged 83a86c2 from main Preserialize table's split boundaries the same way table partitions are. The size of both depend on the same variable: number of shards in the table, but TablePartitions was preserialized (and cached) while Table.SplitBoundaries wasn't. Preserializing all potentially huge parts of DescribeSchemeResult message before it gets to the interconnect saves interconnect actors additional serialization costs. And when partitioning of the huge tables goes through the period of a rapid change that additional serialization causes interconnect to overload. Single shortcoming though: preserialized SplitBoundary is not used (cannot be used) when boundaries of the index tables are requested through describe request on table index. KIKIMR-21686
…b-platform#6760) (ydb-platform#6849) merged ebee36a from main Add commit redo size check for successfully ignited operations as a precaution measure to avoid infinite loop of schemeshard hitting local tx commit redo size limit, restarting, attempting to propose persisted operation again, hitting commit redo size limit again, restarting and so on. This could happen with inherently massive operations such as copy-tables used as a starting step of database export/backup. Coping large number of tables with huge number of partitions can result in so large TTxOperationPropose local transaction that its size would hit the limit imposed by the tablet executor. Tablet violating that limit is considered broken and will be immediately stopped. See ydb/core/tablet_flat/flat_executor.cpp, NTabletFlatExecutor::TExecutor::ExecuteTransaction(). KIKIMR-21751
Co-authored-by: antonkovalenko <antonkovalenko@users.noreply.github.com>
Co-authored-by: kruall <kruall@ydb.tech>
Co-authored-by: ivanmorozov333 <ivanmorozov@ydb.tech>
Co-authored-by: Alexander Rutkovsky <alexvru@ydb.tech>
Co-authored-by: niksaveliev <nik@saveliev.me> Co-authored-by: Alek5andr-Kotov <akotov@ydb.tech> Co-authored-by: Sergey Veselov <sergeyveselov@ydb.tech> Co-authored-by: FloatingCrowbar <komels@ydb.tech>
|
🔴 Unable to merge your PR into the |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.