Skip to content

Commit a3ae600

Browse files
authored
[indexer-2] rework some indices (#14203)
## Description As realized GIN indices works poorly in our use case where order by tx_sequence_number is highly desired. This PR makes the following changes wrt schema/indices: 1. break down `tx_indices` table into individual tables `tx_seners`, `tx_recipient`, `tx_input_objects`, `tx_changed_objects` and `tx_calls`. Check their `up.sql` files for schema. 2. nuke the `sender` index on `events` table as it's GIN too (thanks to @gegaowp for pointing out we don't need a new table `event_sender` because `tx_senders` has what we what already. Because the addition of new tables, there is some changes in parallel commit in the ingestion path. Also note that in several places I write raw queries (but no SQL injection risk) for proof-of-concept purposes. If there is strong desire to use diesel ORM, I will try to make it work, but it will take a bit of time as i'm not fluent in diesel yet. ## Test Plan How did you test the new or updated feature? --- If your changes are not user-facing and not a breaking change, you can skip the following section. Otherwise, please indicate what changed, and then add to the Release Notes section as highlighted during the release process. ### Type of Change (Check all that apply) - [ ] protocol change - [ ] user-visible impact - [ ] breaking change for a client SDKs - [ ] breaking change for FNs (FN binary must upgrade) - [ ] breaking change for validators or node operators (must upgrade binaries) - [ ] breaking change for on-chain data layout - [ ] necessitate either a data wipe or data migration ### Release notes 1. break down `tx_indices` table into individual tables `tx_seners`, `tx_recipient`, `tx_input_objects`, `tx_changed_objects` and `tx_calls`. Check their `up.sql` files for schema. 2. nuke the `sender` index on `events` table as it's GIN too (thanks to @gegaowp for pointing out we don't need a new table `event_sender` because `tx_senders` has what we what already.
1 parent 537de69 commit a3ae600

File tree

18 files changed

+559
-184
lines changed

18 files changed

+559
-184
lines changed

crates/sui-indexer/migrations_v2/2023-08-19-044020_events/up.sql

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ CREATE TABLE events
1818
PRIMARY KEY(tx_sequence_number, event_sequence_number)
1919
);
2020

21-
CREATE INDEX events_senders ON events USING GIN(senders);
22-
CREATE INDEX events_package_module ON events (package, module);
23-
CREATE INDEX events_event_type ON events (event_type);
21+
CREATE INDEX events_package ON events (package, tx_sequence_number, event_sequence_number);
22+
CREATE INDEX events_package_module ON events (package, module, tx_sequence_number, event_sequence_number);
23+
CREATE INDEX events_event_type ON events (event_type text_pattern_ops, tx_sequence_number, event_sequence_number);
2424
CREATE INDEX events_checkpoint_sequence_number ON events (checkpoint_sequence_number);

crates/sui-indexer/migrations_v2/2023-08-19-044035_tx_indices/up.sql

Lines changed: 0 additions & 33 deletions
This file was deleted.
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
-- This file should undo anything in `up.sql`
2+
DROP TABLE IF EXISTS tx_recipients;
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
-- Your SQL goes here
2+
CREATE TABLE tx_recipients (
3+
tx_sequence_number BIGINT NOT NULL,
4+
-- SuiAddress in bytes.
5+
recipient BYTEA NOT NULL,
6+
PRIMARY KEY(recipient, tx_sequence_number)
7+
);
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
-- This file should undo anything in `up.sql`
2-
DROP TABLE IF EXISTS tx_indices;
2+
DROP TABLE IF EXISTS tx_senders;
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
-- Your SQL goes here
2+
CREATE TABLE tx_senders (
3+
tx_sequence_number BIGINT NOT NULL,
4+
-- SuiAddress in bytes.
5+
sender BYTEA NOT NULL,
6+
PRIMARY KEY(sender, tx_sequence_number)
7+
);
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
-- This file should undo anything in `up.sql`
2+
DROP TABLE IF EXISTS tx_input_objects;
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
-- Your SQL goes here
2+
CREATE TABLE tx_input_objects (
3+
tx_sequence_number BIGINT NOT NULL,
4+
-- Object ID in bytes.
5+
object_id BYTEA NOT NULL,
6+
PRIMARY KEY(object_id, tx_sequence_number)
7+
);
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
-- This file should undo anything in `up.sql`
2+
DROP TABLE IF EXISTS tx_changed_objects;
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
-- Your SQL goes here
2+
CREATE TABLE tx_changed_objects (
3+
tx_sequence_number BIGINT NOT NULL,
4+
-- Object Id in bytes.
5+
object_id BYTEA NOT NULL,
6+
PRIMARY KEY(object_id, tx_sequence_number)
7+
);

0 commit comments

Comments
 (0)