Skip to content
This repository was archived by the owner on Jan 9, 2025. It is now read-only.

enable gzip compression for rpc responses #12

Open
wants to merge 149 commits into
base: master
Choose a base branch
from

Conversation

mschneider
Copy link

The changes are in the last commit, this PR also includes a bunch of unrelated changes, basically all commits until 18.0.0 release. 18 is the version uses in the official solana client.

The goal of this change is to allow rust programs that connect to solana via json rpc to use gzip compression which is offered by most rpc nodes through load balancers like HA proxy. It effectively reduces the cost of running these nodes, and improves performance for any kind of rust user program, especially when querying large / many accounts via gpa/gma requests. It has been tested extensively in production by us and other users for more than a year, so we are reasonably confident in using it.

We understand that jsonrpc is somewhat discontinued, but it's a small change that would benefit most rpc service providers as well as their heavy consumers.

CriesofCarrots and others added 30 commits December 8, 2018 07:39
* test: add make_request fn that returns value without pretty printing

* use encoding enum
* case insensitive charset matching

* added test

* revert vscode formatting, added e2e test

* use eq_ignore_ascii_case, match without spacing

* match without spacing
Updates the requirements on [parking_lot](https://github.com/Amanieu/parking_lot) to permit the latest version.
- [Release notes](https://github.com/Amanieu/parking_lot/releases)
- [Changelog](https://github.com/Amanieu/parking_lot/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Amanieu/parking_lot/commits)

Signed-off-by: dependabot[bot] <support@dependabot.com>
* Solve more issues with commas.

* Fix RPC naming.
* Fix thread_pool::Builder deprecation

* Fix reactor::Handle::current() deprecaction warning
* Derive Clone for request types

* Derive Clone for Response
* Empty rpc trait with proc macro attribute

* Generate skeleton to_delegate method

* Move delegates to core - is this the right place

* WIP: wrap rpc methods in to_delegate

* Generate basic to_delegate

* Add derive examples and tests

* Add async rpc method example

* Fix extern crate hygiene

* Add where clause bounds to to_delegate function

* WIP: Support one way serialisation for generics

* Add Serialize bounds to generic RPC return types

* Add Serialize bounds for RPC args

* Clean up imports

* Refactor and read aliases from rpc trait

* Wire up aliases

* Derive rpc with metadata associated type

* Refactor

* Rename trait level attribute to rpc

* Refactor: move entry point function into lib

* WIP: implement pubsub

* Implement pubsub macro

* Register pub/sub method aliases

* Remove delegates file prematurely copied to core

* Fix TCP tests?

* Inline rpc method registration

Removes need for Wrap* types

* Extract return type when creating RpcMethod

* Handle methods with no params

* WIP: Add support for optional trailing params

* Handle optional trailing params

* Bring back aliases

* Inline into_future to remove extraneous types/fns

* Refactor extra args for use in pubsub

* WIP: migrate pubsub delegate registration

* Extract special args from delegate

* Register sub/unsub using generated delgate closure

* WIP: move macro delegates to core and pubsub

* WIP: refactor to handle subscription method

* Handle pubsub error

* Add pub/sub IoDelegate

* Refactor and fix pubsub compiler errors

* Restore original delegates file and fix auto_args

I've copied and split the delegates file into core and pubsub for use
with proc macros, was originally reusing it in classic macros but
putting it back

* Wrap subscriber in typed subscriber

* Handle pubsub unsubscribe method

* Uncomment pubsub example

* Unsubscribe optional metadata

* Copy macro tests over and make them compile with derive

* Fix pubsub trailing test

* Rename to_delegate

* Remove errant printlns

* Copy and modify module docs from macros

* Fix derive doc test

* Mark subscribe/unsubscribe methods with attribute

* Generic type param with metadata example, update some error messages

* Refactor: extract a couple of functions for readability

* Add compiletests to test compile errors

* Remove test both for now

* Remove top level future imports

* Fix unsubscribe missing import

* Correct derive crate author

* Rust 2018: cargo fix --edition --package jsonrpc-derive

* Remove redundant `extern crate` rust 2018

* Fix doc tests for edition 2018

* Add missing_docs warning

* Replace &str to_string() with into()

* Move all magic strings to constants

* WIP

* Bring pubsub methods back into same trait

* Geoup pubsub methods by subscription name

* Refactor attribute parsing

* Update compilefail tests for new pubsub

* Fix combined rpc and pubsub trait

* Fix compilefail tests

* added some tests (failing)

* Remove multiple trailing params tests, will implement in later PR

* Fix parse of single param tuple with trailing comma

* Specify subscription name in missing annotation error

* Add tests for trailing args

* Handle single trailing param

* Replace extern crates in macro output

* Reduce recursion limit

* Add comment for `to_delegate` trait method

* Deprecate `to_delegate` method generated by jsonrpc-macros

* Update README with derive example

* Add pubsub example and rpc attribute docs

* Remove commented out code

* Bump major version
* support multiple trailing arguments

* Try to get the Windows build to work
* ci: add appveyor

* Try to get the Windows build to work
* Remove minihttp.

* Use crates.io version in README.
* core: cargo fix --edition

* core: edition = 2018

* http: cargo fix --edition

* http: edition = 2018

* ipc: edition 2018

* pubsub: edition 2018

* pubsub: cargo fix --edition-idioms --broken-code, then fixed compile errors

* server-utils: cargo fix --edition

* server-utils: edition 2018

* server-utils: cargo fix --edition-idioms --broken-code

* stdio: edition 2018

* stdio: edition 2018 idioms

* tcp: edition 2018

* tcp: edition 2018 idioms

* test: edition 2018

* test: edition 2018 idioms

* ws: edition 2018

* ws: edition 2018 idioms

* edition 2018: remove whitespaces from cargo fix

* core: edition 2018 idioms

* http: edition 2018 idioms

* derive: edition 2018 idioms

* ipc: edition 2018 idioms

* pubsub: edition 2018 idioms

* Remove more blank lines and unused

* Remove all `dyn` keywords for trait objects

* Remove anonymous lifetimes

* Replace `r#try` with `?`

* Remove `extern crate` in doctests

* Remove `extern crate` in READMEs

* Convert doctest to use derive instead of macros
* WIP: 2018 compile tests with one way serialization

* WIP: attempting to get traitbounds to work with compiletests

* Failing compile test for one way serialise Subscription type

* One way Serialize for generic type only used in Subscriber
* Add jsonrpc-derive details.

* Update docs and authors.

* Bump version.
* Fix rpc alias(es) key constant

* Bump derive patch
…ech#384)

* Fix rpc attribute error message

* Don't require pubsub dependency if no pubsub methods
* Use syn::Error when parsing attributes, error message constants

* Add compiletests for attribute parse errors

* Compile errors for invalid attribute parameters

* Replace anon fn args in compilefail examples with underscore

* Fix spaces and blank line

* Revert accidental change to macros auto_args

* Revert accidental macros changes

* Revert accidental change to pubsuc macros tests

* Fix build on rust 1.33-beta: remove Rpc impls

* Clone after filter

* Remove redundant static lifetime on constants

* Remove redundant braces
* Add example for request params

* Update params.rs

* Use serde_derive to import Deserialize macro
tomusdrw and others added 22 commits July 30, 2020 11:27
* Remove failure dependency.

* Remove needless format!.

* cargo fmt --all
* Use patched ws-rs

* Update settings

* Bump versions to 16.0.0
Signed-off-by: koushiro <koushiro.cqx@gmail.com>
…tech#594)

* Expose direct configuration of max in and out buffer capacity

* Correct argument duplicate typo"

* Add server builder tests

* rustfmt ws/src/server_builder.rs
* Update examples in top-level readme.

* cargo fmt --all

* Added docs.
* Fix typos

* Fix request.sender

* cargo fmt --all

Co-authored-by: Tomasz Drwięga <tomasz@parity.io>
paritytech#589) (paritytech#600)

* Update `local` client to support middleware (Kudos Seun) (paritytech#589)

* v15.0.1

* v15.0.1 => v15.1.0

* v15.0.1 => v15.1.0

* cargo fmt

* fix tests

* adds *_with_middleware methods

* remove Unpin bound, add documentation, cargo fmt

* 15.0.1

* bump to 15.1.0

Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>
Co-authored-by: Niklas <niklasadolfsson1@gmail.com>

* cargo fmt --all

Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>
Co-authored-by: Niklas <niklasadolfsson1@gmail.com>
* core: Prefer standard Future over futures' variant

* core: Make futures::executor convenience support optional

* core: Use only futures-util and (optionally) futures-executor

* core: Keep futures re-export for backcompat
* Bump minor version.

* Fix build warning.

* Fix ui tests.

* Remove windows runner since it's decomissioned.
* WIP: Update to Tokio 1.0

* ipc: Migrate remaining Unix test code to Tokio 1.0

* core-client: Don't depend on unused hyper/server feature

* http: Fix used feature set by hyper

* WIP: Bump to the transferred parity-tokio-ipc repo

* Use newly released version of parity-tokio-ipc

* Remove extra newline in Cargo.toml

* Refactor suspension slightly in SuspendableStream
* Remove ambiguity when using deps
* Fix doc and some linting hints
* Make tests more readable
* Run clippy during the CI checks
* Fix formatting
* A few more linting fixes
* Remove old comment
@CriesofCarrots
Copy link

We haven't used this fork in years, and I think moving the monorepo back to it is going to be a hard sell.
Is this something that paritytech would consider upstream?

@mschneider
Copy link
Author

mschneider commented Mar 17, 2023

We haven't used this fork in years, and I think moving the monorepo back to it is going to be a hard sell. Is this something that paritytech would consider upstream?

Parity is not merging anything either, the jsonrpc crate is discontinued. Given that solana is discontinuing jsonrpc alltogether a rewrite using the new jsonrpsee is not likely. I see the principle behind the decision, but I don't think it applies here, as there is no more changes merged upstream. A fork of this package to manage small performance / security issues is necessary.

@mvines
Copy link

mvines commented Mar 17, 2023

Given that solana is discontinuing jsonrpc alltogether

What makes you say this?

@mschneider
Copy link
Author

mschneider commented Mar 17, 2023 via email

@mvines
Copy link

mvines commented Mar 17, 2023

You surely misunderstood ;)

@mschneider
Copy link
Author

mschneider commented Mar 17, 2023 via email

@mvines
Copy link

mvines commented Mar 17, 2023

The JSONRPC interface will be around for years. Upgrading to jsonrpsee, if it's ready for production, is the right way IMO

@CriesofCarrots
Copy link

CriesofCarrots commented Mar 18, 2023

The JSONRPC interface will be around for years. Upgrading to jsonrpsee, if it's ready for production, is the right way IMO

Yeah, this. I would be a lot more eager to support an update to jsonrpsee than help maintain a forked dependency.
At a glance, it's hard to tell when jsonrpsee might be published, since their v1 milestone is closed: https://github.com/paritytech/jsonrpsee/milestone/2
But there is active development going on.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.