Skip to content

Conversation

siomari
Copy link
Contributor

@siomari siomari commented Sep 13, 2024

Description

Implementation of paginated query for epochs in GraphQL

Test plan

How did you test the new or updated feature?

Added a pagination.move test under the sui-graphql-e2e-tests crate.


Release notes

Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.

For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.

  • Protocol:
  • Nodes (Validators and Full nodes):
  • Indexer:
  • JSON-RPC:
  • GraphQL: Added support for paginated queries for epochs. This allows users to fetch epochs data in a paginated format.
  • CLI:
  • Rust SDK:
  • REST API:

Copy link

vercel bot commented Sep 13, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
sui-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 7, 2024 2:01pm
3 Skipped Deployments
Name Status Preview Comments Updated (UTC)
multisig-toolkit ⬜️ Ignored (Inspect) Visit Preview Oct 7, 2024 2:01pm
sui-kiosk ⬜️ Ignored (Inspect) Visit Preview Oct 7, 2024 2:01pm
sui-typescript-docs ⬜️ Ignored (Inspect) Visit Preview Oct 7, 2024 2:01pm

Copy link
Contributor

@wlmyng wlmyng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

more or less good to go, just one more testing idea

}
}

//# run-graphql --cursors {"c":5,"e":2}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about a checkpoint viewed at before 5, and trying to view an epoch beyond that checkpoint, and looking at the full state of epochs per that checkpoint? ie c=3, try to view e=4, c=3, query full epochs state starting from epoch 0

@wlmyng
Copy link
Contributor

wlmyng commented Sep 13, 2024

Actually.. I think we'll also need to create a new index on column first_checkpoint_id for table epochs to support this.
Otherwise this will be a sequential scan through epochs 😨

At the same time, it feels a bit extraneous to add an entire index just to support the initial query and filter condition. Can we get by with epoch_viewed_at instead of checkpoint_viewed_at? We should still pass around checkpoint_viewed_at for edge queries but when querying the epochs table, we'd use only the epoch_viewed_at.

@amnn
Copy link
Contributor

amnn commented Sep 14, 2024

Actually.. I think we'll also need to create a new index on column first_checkpoint_id for table epochs to support this.
Otherwise this will be a sequential scan through epochs

There's only ~500 of these so I think an index would be fine (cheap, and the easiest solution).

Note that we already use a similar query to do a consistent fetch on epochs -- I think it's also fine to do that in a follow-up PR, sequential scan over 500 records is peanuts.

Copy link
Contributor

@amnn amnn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this @siomari !! Apart from @wlmyng's comment about adding some more tests, I think this is good to go. I'll take care of adding the appropriate indices to epochs in another PR, so don't worry about that.

C: object(0,0)

task 1, line 7:
//# create-checkpoint
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: if the aim is just to create a checkpoint for the epoch to contain, that isn't necessary, advance-epoch also generates a checkpoint (as you can see, the checkpoints created by create-checkpoint seem to go up in 2's because of this).

Comment on lines 6 to 32
// TODO: Short term hack to get around indexer epoch issue
//# create-checkpoint

//# advance-epoch

// TODO: Short term hack to get around indexer epoch issue
//# create-checkpoint

//# advance-epoch


// TODO: Short term hack to get around indexer epoch issue
//# create-checkpoint

//# advance-epoch

//# create-checkpoint

//# advance-epoch

//# create-checkpoint

//# advance-epoch

//# create-checkpoint

//# advance-epoch
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per the earier comment

Suggested change
// TODO: Short term hack to get around indexer epoch issue
//# create-checkpoint
//# advance-epoch
// TODO: Short term hack to get around indexer epoch issue
//# create-checkpoint
//# advance-epoch
// TODO: Short term hack to get around indexer epoch issue
//# create-checkpoint
//# advance-epoch
//# create-checkpoint
//# advance-epoch
//# create-checkpoint
//# advance-epoch
//# create-checkpoint
//# advance-epoch
//# advance-epoch
//# advance-epoch
//# advance-epoch
//# advance-epoch
//# advance-epoch
//# advance-epoch

Copy link
Contributor

@amnn amnn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes look good @siomari ! There are some CI warnings to address:

  • Issue from clippy
  • Updating the GraphQL schema (run the test that failed in CI, and it will explain how to update the snapshot).
  • You will also need to rebase your PR, because we recently moved the E2E tests into a sub-directory called stable.

Once that's done, you're good to go!

@siomari siomari merged commit 0773e2f into main Oct 9, 2024
49 checks passed
@siomari siomari deleted the ms/query-epochs-graphql branch October 9, 2024 10:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants