Node Public Interfaces Team Weekly Update - 2021-09-17

[frol]

Previous update is #4789

Brief Summary

Indexer for Explorer is playing a crucial role in NEAR ecosystem, and in the course of the last 2 weeks we had 4 major incidents on mainnet related to the downtime of Indexer for Explorer which we worked hard to resolve (all different root causes) and eliminate in the future.

We received a super positive feedback on Indexer-as-a-Service (a.k.a. webhooks for blockchain events) and currently collaborate with Pingbox on the implementation side.

near-jsonrpc-client-rs crate is on its way to get ready for the release! 🎉

Result

• NEAR internal infrastructure received stability and security improvements (lead by @chefsale and @mhalambek):

  • Internal alerting has been restructured to reduce lower-priority events
  • Audited CI services for proper use of secrets/tokens
  • Fixed Discourse misconfiguration

• Indexer for Explorer received stability improvements (lead by @khorolets in collaboration with @mhalambek and @chefsale):

  • 0.10.0 release of indexer-for-explorer with a helpful non-strict mode and ability to run indexer for specific number of blocks
    refactor: Strict/Non-strict mode improvements near-indexer-for-explorer#164
  • Introduced escape_json function in indexer-for-explorer to escape incoming json args to avoid attempts to store null-byte containing data to postgres
    fix: Add escape_json function to the serializer near-indexer-for-explorer#162
  • Introduced an indexer status endpoint to NEAR Explorer to improve alerts about problems
    feat: API indexer-status endpoint near-explorer#732
  • Created a fix to handle restored receipts issue from indexer perspective
    fix(indexer): Handle restored receipts by attaching them to the chunk #4827

• Explorer received UX improvements on Nodes (Validators) and Transaction Details pages (lead by @shelegdmitriy):

  • Fix caching of validators stake and improved their sorting. At the first place we sort 'active', 'joining' and 'leaving' validators by their stake and after then other types of validators will be sorted by proposed for next epoch if such or by current stake
    fix(backend): Fix caching of the validator stake near-explorer#728
  • Improved navigation between receipt-receiver-signer. For now you can navigate to receipt (in transaction) or signer or receiver (account page) from one place!
    fix(frontend): Add links to useful fields in receipts section near-explorer#731
  • Fix condition in displaying some data on validators page
    fix(frontend): Fix condition of TelemetryRow appearance near-explorer#734

• Explorer received stability and performance improvements to the data backing the Statistics page (lead by @telezhnaya):

  • Optimized some of the stats collection for Explorer
    Optimize Explorer stats near-indexer-for-explorer#137
    https://github.com/telezhnaya/near-analytics
    feat(backend): start using analytics DB for stats page near-explorer#735

• Sparked the discussion around FT/NFT observability standard (help is needed!)
  NFT NEP-171 require having minting/burning interface NEPs#254

• near-jsonrpc-client-rs crate is on its way to get ready for the release 🎉 (lead by @miraclx)

  • Use a trait, implemented on all Rpc*Request structs to simplify the design and prune boilerplate
    refactor: use traits, drop HTTP endpoint support, update docs near-jsonrpc-client-rs#6
  • Use module scoping to resolve the use of the same Rpc*Request structs by multiple RPC methods
    e.g while both broadcast_tx_async and broadcast_tx_commit use RpcBroadcastTransactionRequest, we proxy using RpcBroadcastTxAsyncRequest and RpcBroadcastTxCommitRequest respectively, both of which model RpcBroadcastTransactionRequest
  • Add Github workflow for generating rustdoc documentation. It's accessible on https://git.io/near-jsonrpc-client-rs

• nearup improvements:

  • Added support for custom genesis.json restarts and updates in nearup
    fix(bug): Guildnet nearup genesis is self hosted on openshard.io S3 i… nearup#184

Challenges & Blockers

• Lack of standardization of events for FT and NFT standards still block Explorer and Wallet on displaying the data to the end-users. Please go to NFT NEP-171 require having minting/burning interface NEPs#254 and share your thoughts about FT/NFT NEPs extensions
• Indexer for Explorer is considering various ideas on storage improvements since we won't be able to scale with PostgreSQL in the future

Action Items

• indexer

  • Improve performance and stability (Improve uptime near-indexer-for-explorer#161):
    • Investigate and fix start-from-interruption on indexer-for-explorer as it is too slow for no reason (@khorolets)
    • Decide on refactoring indexer-for-explorer to improve it's stability (@khorolets)

• explorer

  • get rid of some legacy code
    refactor(backend): Remove legacy code near-explorer#736

• near-jsonrpc-client-rs

  • get ready for the release on crates.io:
    • Remove client re-export from near-jsonrpc and relegate the near-jsonrpc-client to dev-dependency only. (@miraclx)
    • Look into Query ViewState with with non-existing account id and invalid prefix does not return error as expected #2793, since strict AccountId's introduction addresses this, cleanup the cases and close linked issues. (@miraclx)
    • Re-inspect the crates on nearcore and near-jsonrpc-client-rs to get near-jsonrpc-client-rs published. (@miraclx)