v0.19.1

• Detect reorgs during query execution (#1801).
• Annotate SQL queries with the GraphQL query ID that caused them (#1946).
• Fix potential deadlock caused by reentering the load manager semaphore (#1948).
• Fix fulltext query issue with optional and unset fields (#1937 via #1938).
• Fix build warnings with --release (#1949 via #1953).
• Dependency updates: async-trait, chrono, wasmparser.

v0.19.0

• Skip trace_filter on empty blocks (#1923).
• Ensure runtime hosts are unique to avoid double-counting, improve logging (#1904).
• Add administrative Postgres views (#1889).
• Limit the GraphQL skip argument in the same way as we limit first (#1912).
• Fix GraphQL fragment bugs (#1825).
• Don't crash node and show better error when multiple graph nodes are indexing the same subgraph (#1903).
• Add a query semaphore to allow to control the number of concurrent queries and subscription queries being executed (#1802).
• Call Ethereum contracts by block hash (#1905).
• Fix fetching the correct function ABI from the contract ABI (#1886).
• Add LFU cache for historical queries (#1878, #1879, #1891).
• Log GraphQL queries only once (#1873).
• Gracefully fail on a null block hash and encoding failures in the Ethereum adapter (#1872).
• Improve metrics by using labels more (#1868, ...)
• Log when decoding a contract call result fails to decode (#1842).
• Fix Ethereum node requirements parsing based on the manifest (#1834).
• Speed up queries that involve checking for inclusion in an array (#1820).
• Add better error message when blocking a query due to load management (#1822).
• Support multiple Ethereum nodes/endpoints per network, with different capabilities (#1810).
• Change how we index foreign keys (#1811).
• Add an experimental Ethereum node config file (#1819).
• Allow using GraphQL variables in block constraints (#1803).
• Add Solidity struct array / Ethereum tuple array support (#1815).
• Resolve subgraph names in a blocking task (#1797).
• Add environmen variable options for sensitive arguments (#1784).
• USe blocking task for store events (#1789).
• Refactor servers, log GraphQL panics (#1783).
• Remove excessive logging in the store (#1772).
• Add dynamic load management for GraphQL queries (#1762, #1773, #1774).
• Add ability to block certain queries (#1749, #1771).
• Log the complexity of each query executed (#1752).
• Add support for running against read-only Postgres replicas (#1746, #1748,
  #1753, #1750, #1754, #1860).
• Catch invalid opcode reverts on Geth (#1744).
• Optimize queries for single-object lookups (#1734).
• Increase the maximum number of blocking threads (#1742).
• Increase default JSON-RPC timeout (#1732).
• Ignore flaky network indexers tests (#1724).
• Change default max block range size to 1000 (#1727).
• Fixed aliased scalar fields (#1726).
• Fix issue inserting fulltext fields when all included field values are null (#1710).
• Remove frequent "GraphQL query served" log message (#1719).
• Fix bigDecimal.devidedBy (#1715).
• Optimize GraphQL execution, remove non-prefetch code (#1712, #1730, #1733, #1743, #1775).
• Add a query cache (#1708, #1709, #1747, #1751, #1777).
• Support the new Geth revert format (#1713).
• Switch WASM runtime from wasmi to wasmtime and cranelift (#1700).
• Avoid adding order by clauses for single-object lookups (#1703).
• Refactor chain head and store event listeners (#1693).
• Properly escape single quotes in strings for SQL queries (#1695).
• Revamp how Graph Node Docker image is built (#1644).
• Add BRIN indexes to speed up revert handling (#1683).
• Don't store chain head block in SubgraphDeployment entity (#1673).
• Allow varying block constraints across different GraphQL query fields (#1685).
• Handle database tables that have text columns where they should have enums (#1681).
• Make contract call cache collision-free (#1680).
• Fix a SQL query in cleanup_cached_blocks (#1672).
• Exit process when panicking in the notification listener (#1671).
• Rebase ethabi and web3 forks on top of upstream (#1662).
• Remove parity-wasm dependency (#1663).
• Normalize BigDecimal values, limit BigDecimal exponent (#1640).
• Strip nulls from strings (#1656).
• Fetch genesis block by number 0 instead of "earliest" (#1658).
• Speed up GraphQL query execution (#1648).
• Fetch event logs in parallel (#1646).
• Cheaper block polling (#1646).
• Improve indexing status API (#1609, #1655, #1659, #1718).
• Log Postgres contention again (#1643).
• Allow User-Agent in CORS headers (#1635).
• Docker: Increase startup wait timeouts (Postgres, IPFS) to 120s (#1634).
• Allow using Bytes for id fields (#1607).
• Increase Postgres connection pool size (#1620).
• Fix entities updated after being removed in the same block (#1632).
• Pass log_index to mappings in place of transaction_log_index (required for Geth).
• Don't return __typename to mappings (#1629).
• Log warnings after 10 successive failed eth_call requests. This makes it more visible when graph-node is not operating against an Ethereum archive node (#1606).
• Improve use of async/await across the codebase.
• Add Proof Of Indexing (POI).
• Add first implementation of subgraph grafting.
• Add integration test for handling Ganache reverts (#1590).
• Log all GraphQL and SQL queries performed by a node, controlled through the GRAPH_LOG_QUERY_TIMING environment variable (#1595).
• Fix loading more than 200 dynamic data sources (#1596).
• Fix fulltext schema validation (includes fields).
• Dependency updates: anyhow, async-trait, bs58, blake3, bytes, chrono, clap, crossbeam-channel derive_more, diesel-derive-enum, duct, ethabi, git-testament, hex-literal, hyper, indexmap, jsonrpc-core, mockall, once_cell, petgraph, reqwest, semver, serde, serde_json, slog-term, tokio, wasmparser.

v0.18.0

NOTE: JSONB storage is deprecated and will be removed in the next release. This only affects subgraphs that were deployed with a graph-node version before 0.16. Starting with this version, graph-node will print a warning for any subgraph that uses JSONB storage when that subgraph starts syncing. Please check your logs for this warning. You can remove the warning by redeploying the subgraph.

Feature: Fulltext Search (#1521)

A frequently requested feature has been support for more advanced text-based search, e.g. to power search fields in dApps. This release introduces a @fulltext directive on a new, reserved _Schema_ type to define fulltext search APIs that can then be used in queries. The example below shows how such an API can be defined in the subgraph schema:

type _Schema_
  @fulltext(
    name: "artistSearch"
    language: en
    algorithm: rank
    include: [
      {
        entity: "Artist"
        fields: [
          { name: "name" }
          { name: "bio" }
          { name: "genre" }
          { name: "promoCopy" }
        ]
      }
    ]
  )

This will add a special database column for Artist entities that can be used for fulltext search queries across all included entity fields, based on the tsvector and tsquery features provided by Postgres.

The @fulltext directive will also add an artistSearch field on the root query object to the generated subgraph GraphQL API, which can be used as follows:

{
  artistSearch(text: "breaks & electro & detroit") {
    id
    name
    bio
  }
}

For more information about the supported operators (like the & in the above query), please refer to the Postgres
documentation.

Feature: 3Box Profiles (#1574)

3Box has become a popular solution for integrating user profiles into dApps. Starting with this release, it is possible to fetch profile data for Ethereum addresses and DIDs. Example usage:

import { box } from '@graphprotocol/graph-ts'

let profile = box.profile("0xc8d807011058fcc0FB717dcd549b9ced09b53404")
if (profile !== null) {
  let name = profile.get("name")
  ...
}

let profileFromDid = box.profile(
  "id:3:bafyreia7db37k7epoc4qaifound6hk7swpwfkhudvdug4bgccjw6dh77ue"
)
...

Feature: Arweave Transaction Data (#1574)

This release enables accessing Arweave transaction data using Arweave transaction IDs:

import { arweave, json } from '@graphprotocol/graph-ts'

let data = arweave.transactionData(
  "W2czhcswOAe4TgL4Q8kHHqoZ1jbFBntUCrtamYX_rOU"
)

if (data !== null) {
  let data = json.fromBytes(data)
  ...
}

Feature: Data Source Context (#1404 via #1537)

Data source contexts allow passing extra configuration when creating a data source from a template. As an example, let's say a subgraph tracks exchanges that are associated with a particular trading pair, which is included in the NewExchange event. That information can be passed into the dynamically created data source, like so:

import { DataSourceContext } from '@graphprotocol/graph-ts'
import { Exchange } from '../generated/templates'

export function handleNewExchange(event: NewExchange): void {
  let context = new DataSourceContext()
  context.setString('tradingPair', event.params.tradingPair)
  Exchange.createWithContext(event.params.exchange, context)
}

Inside a mapping of the Exchange template, the context can then be accessed as follows:

import { dataSource } from '@graphprotocol/graph-ts'

...

let context = dataSource.context()
let tradingPair = context.getString('tradingPair')

There are setters and getters like setString and getString for all value types to make working with data source contexts convenient.

Feature: Error Handling for JSON Parsing (#1588 via #1578)

With contracts anchoring JSON data on IPFS on chain, there is no guarantee that this data is actually valid JSON. Until now, failure to parse JSON in subgraph mappings would fail the subgraph. This release adds a new json.try_fromBytes host export that allows subgraph to gracefully handle JSON parsing errors.

import { json } from '@graphprotocol/graph-ts'

export function handleSomeEvent(event: SomeEvent): void {
  // JSON data as bytes, e.g. retrieved from IPFS
  let data = ...

  // This returns a `Result<JSONValue, boolean>`, meaning that the error type is
  // just a boolean (true if there was an error, false if parsing succeeded).
  // The actual error message is logged automatically.
  let result = json.try_fromBytes(data)

  if (result.isOk) { // or: !result.isError
    // Do something with the JSON value
    let value = result.value
    ...
  } else {
    // Handle the error
   let error = result.error
   ...
  }
}

Ethereum

• Add support for calling overloaded contract functions (#48 via #1440).
• Add integration test for calling overloaded contract functions (#1441).
• Avoid eth_getLogs requests with block ranges too large for Ethereum nodes to handle (#1536).
• Simplify eth_getLogs fetching logic to reduce the risk of being rate limited by Ethereum nodes and the risk of overloading them (#1540).
• Retry JSON-RPC responses with a -32000 error (Alchemy uses this for timeouts) (#1539).
• Reduce block range size for trace_filter requests to prevent request timeouts out (#1547).
• Fix loading dynamically created data sources with topic0 event handlers from the database (#1580).
• Fix handling contract call reverts in newer versions of Ganache (#1591).

IPFS

• Add support for checking multiple IPFS nodes when fetching files (#1498).

GraphQL

• Use correct network when resolving block numbers in time travel queries (#1508).
• Fix enum field validation in subgraph schemas (#1495).
• Prevent WebSocket connections from hogging the blocking thread pool and freezing the node (#1522).

Database

• Switch subgraph metadata from JSONB to relational storage (#1394 via #1454, #1457, #1459).
• Clean up large notifications less frequently (#1505).
• Add metric for Postgres connection errors (#1484).
• Log SQL queries executed as part of the GraphQL API (#1465, #1466, #1468).
• Log entities returned by SQL queries (#1503).
• Fix several GraphQL prefetch / SQL query execution issues (#1523, #1524, #1526).
• Print deprecation warnings for JSONB subgraphs (#1527).
• Make sure reorg handling does not affect metadata of other subgraphs (#1538).

Performance

• Maintain an in-memory entity cache across blocks to speed up store.get (#1381 via #1416).
• Speed up revert handling by making use of cached blocks (#1449).
• Speed up simple queries by delaying building JSON objects for results (#1476).
• Resolve block numbers to hashes using cached blocks when possible (#1477).
• Improve GraphQL prefetching performance by using lateral joins (#1450 via #1483).
• Vastly reduce memory consumption when indexing data sources created from templates (#1494).

Misc

• Default to IPFS 0.4.23 in the Docker Compose setup (#1592).
• Support Elasticsearch endpoints without HTTP basic auth (#1576).
• Fix --version not reporting the current version (#967 via #1567).
• Convert more code to async/await and simplify async logic (#1558, #1560, #1571).
• Use lossy, more tolerant UTF-8 conversion when converting strings to bytes (#1541).
• Detect when a node is unresponsive and kill it (#1507).
• Dump core when exiting because of a fatal error (#1512).
• Update to futures 0.3 and tokio 0.2, enabling async/await (#1448).
• Log block and full transaction hash when handlers fail (#1496).
• Speed up network indexer tests (#1453).
• Fix Travis to always install Node.js 11.x. (#1588).
• Dependency updates: bytes, chrono, crossbeam-channel, ethabi, failure, futures, hex, hyper, indexmap, jsonrpc-http-server, num-bigint, priority-queue, reqwest, rust-web3, serde, serde_json, slog-async, slog-term, tokio, tokio-tungstenite, walkdir, url.

v0.17.1

• Reject time-travel queries with block numbers that are in the future (#1437).
• Fix setting fields to null being ignored in certain cases (#1428).
• Fix error in GraphQL execution caused by overly aggressive assertion in prefetch (#1422).
• Log GraphQL query variables (#1429).
• Dependency updates: serde.

v0.17.0

GraphQL query prefetching (#1386)

Previously referred to SQL query combination, this feature speeds up nested queries by a factor of about 10x by batching every GraphQL query level into a single Postgres SQL query. A query like

{
  bands(first: 100) {
    name
    musicians(first: 100) {
      name
    }
}

previously would result in one SQL query for the bands and 100 SQL queries for the musicians of each individual band. With query prefetching, only two SQL queries are executed: one for the bands and one for the musicians of all bands.

This is now enabled by default.

Time-travel queries (#1397)

It is now possible to run queries against the state of a subgraph at an arbitrary block height, and not just the latest block. So far, a query like transactions { id } would always return the data for the latest set of transactions, and rerunning it might return a different result if the subgraph had ingested more transactions in the mean time.

It is now possible to fix the block at which to query by passing an additional block argument to top-level fields:

{
  transactions(block: { number: 7000 }) {
    id
  }
}

will return the transactions as of the block with that number, and

{
  transactions(block: { hash: "0xdeadbeef" }) {
    id
  }
}

will return transactions as of that block hash. To make absolutely sure the query result is reproducible, the form with the hash should be used as which block is designated by number may change if the chain is reorganized (though using number far enough away from the chain head will be fine).

If a query does not explicitly specify a block, we continue to run it against the latest available data, so that the behavior for existing queries does not change.

Indexing performance

Work on improving the indexing performance continues in this release. The following changes have been implemented since v0.16.1.

• Add indexing metrics using Prometheus (#1338, #1360, #1361, #1362, #1374).
• Optimize store.set entity validation (#1384).
• Batch-load entities before applying changes (#1388).

Other changes

GraphQL

• Catch unknown fields in GraphQL queries (#1339).
• Log query execution time (#1402).
• Add query execution time metrics using Prometheus (#1403).

Ethereum

• Dynamically adjust block range sizes when scanning for triggers to send Ethereum nodes manageable requests (#1370).
• Fix reorg handling on short chains (#1377).
• Fix call handlers not being trigger in dynamic data sources (#1382).
• Limit JSON-RPC retries to avoid getting stuck on reorg'ed blocks (#1396).
• Ensure Ethereum triggers (blocks, calls, events) are always processed in the correct order (#1413).
• Add block ingestor metrics using Prometheus (#1401, #1403).

Mappings

• Reduce frequency of handler timeout checks to speed up indexing (#1353).
• Log Ethereum transaction hash when a handler fails (#1371).

Store

• Exit with error when losing the connection to Postgres (#1348). This allows to environments like Kubernetes to restart the nodes and therefore recover from lost Postgres connections.
• Allow unneeded old blocks to be deleted from the store periodically (#1408).

Misc

• Add missing ports to docker-compose.yml (#1385, thanks @Amxx!).
• Make startup asynchronous to avoid getting stuck connecting to Ethereum nodes (#1352, #1372).
• Detect invalid ETHEREUM_REORG_THRESHOLD values (#1366).
• Rename graph-datasource-ethereum crate to graph-chain-ethereum (#1368).
• Document call and block handlers in the manifest spec.
• Dependency updates: chrono, derive_more, git-testament, jsonrpc-http-server, num-traits, reqwest, serde_derive, serde_json, slog-term, uuid.

v0.16.1

• Use retrying IPFS resolver to load subgraph files (#1239).
• Fix parsing GraphQL block hashes and numbers (#1335).
• Allow to recover from database connection errors (#1348).
• Dependency updates: reqwest.

v0.16.0

Relational database schemas (#1020 via #1159)

Subgraph data is now stored in Postgres using relational schemas. Instead of a storing all entities of a subgraph in a single table using JSONB, each entity type gets a dedicated table, with each field being stored in its own column.

This has several advantages:

1. Reading and writing are faster, more scalable.
2. Entity data can be validated and enforced better, preventing bugs in subgraphs.
3. Introspecting the data is much easier when debugging.

The stricter validation of entities written to the store may cause subgraphs that work today to fail when being deployed. A note from @lutter, who implemented this feature:

As part of the switch to the relational storage scheme, subgraph entities are validated more strictly than they were in the past. In particular, the store will reject attempts to store entities that have null in fields that are marked as non-null in the GraphQL schema. If your subgraph reports such errors after you deploy a new version, you can address that either by marking the field as nullable in the GraphQL schema by leaving the ! off its type, or by changing your mappings to ensure that the field is always set to a non-null value.

Note: This only affects new subgraph deployments. Existing deployments will keep using the JSONB schema. On the hosted service we will slowly clean up unused deployments over time until we can drop the old JSONB-based schema entirely.

Start block for data sources (#1262, #1294, #1320)

Until now, subgraphs were always indexing from the genesis block. This involved scanning the entire chain for relevant blocks, logs and traces, even if the subgraph contracts were only deployed recently.

This was fine initially. However, with more expensive features—such as block and call handlers—being added and more advanced subgraphs being developed, it has often become desirable to skip irrelevant old blocks entirely to speed up indexing.

This is now possible. As of this release, data sources can specify an optional startBlock number in the manifest:

dataSources:
  - name: Gravity
    source:
      address: '0x2E645469f354BB4F5c8a05B3b30A929361cf77eC'
      abi: Gravity
      startBlock: 6000000

The subgraph will then start indexing from this block. If there are multiple data sources with or without start blocks, the earliest of these blocks is used as the starting point. The absence of a startBlock is equivalent to 0 (aka the genesis block).

Other changes

GraphQL

• Fix querying interfaces by id (#1204)
• Fix following interface references (#1303 via #1307)
• Support where filter on relationship fields (#1212)
• Validate @derivedFrom directives on subgraph schemas (#1247)

Ethereum

• Add an Ethereum contract call cache to speed up indexing (#1230 via #1270)
• Detect failed Solidity assertions in try_ calls (#1162)
• Detect Bad instruction fd errors in try_ calls (#1327)
• Allow recursive data source template creation within the same block (#1185)
• Handle Alchemy timeouts in eth_getLogs gracefully (#1242)
• Make eth_getLogs scanning more efficient (#1260)
• Only fetch transaction receipts when necessary (#1308, #1326)

Mappings

• Reduce memory usage and number of threads by using one thread per mapping instead of one per data source (#1281)

Database

• Allow subgraphs to make progress while Postgres is autovacuuming the subgraphs table (#1175)
• Use a single connection pool per node (#1232)
• Preload pg_stat_statements in Docker Compose setup (#1163)
• Fix default Postgres port in the README (#1181, thanks @akeem!)

Monitoring

• Add Prometheus instrumentation support (#1259 via #1264)

Configuration

• Add new GRAPH_IPFS_SUBGRAPH_LOADING_TIMEOUT environment variable for loading subgraph manifests from IPFS (#1091, #1238, #1244)
• Add optional limit for GraphQL/WS operations using the GRAPH_GRAPHQL_MAX_OPERATIONS_PER_CONNECTION environment variable (#1160)

Misc

• Don't skip the genesis block when indexing (#1295 via #1297)
• Allow Docker image to wait for other services using full URLs (#1275)
• Fix Docker builds by passing --locked to Cargo
• Support multiple Ethereum networks in Docker image by sperating networks in the ethereum environment variable with spaces (#1332)
• Fix yarn commands in README (#1282, thanks @ana0!)
• Update dependencies (bs58, chrono, clap, crossbeam-channel, graphql-parser, hex, hex-literal, hyper, indexmap, jsonrpc-core, jsonrpc-http-server, lru_time_cache, num-bigint, serde, serde_derive, serde_yaml)
• Remove dependencies on itertools and sentry

v0.15.1

• Fix WASM memory allocation bug (#1154).
• Fix incorrect contract call reversion detection caused by ambiguity between Ganache and Parity (#1155).

Note: Make sure to also read the release notes for v0.15.0.

v0.15.0

CAUTION: If you are running your own Graph Node, make sure to run 0.14.0 for a week or two and deploy a new version of your subgraphs at least once before switching to 0.15.0. The 0.14.0 release gradually removes the public.entities table in favor of per-subgraph entities tables. Switching to 0.15.0 too early may break existing subgraphs and require you to redeploy and reindex immediately.

Fallible contract calls (#1139)

Calls to contract functions can fail due to assertions in the contract. Until now there was no way to handle this in subgraphs gracefully. This release introduces new try_someContractFunction call variants that return a result object with reverted and value fields. These can then be used in mappings to handle call failures.

An example can be found in the documentation.

Top-level templates (#1085)

Data source templates have been moved to a top-level templates field in the manifest. The data source templates documentation has been updated accordingly.

Top-level templates simplify creating new data sources at runtime: templates can now be referred to from all data sources and can create new data sources from other templates as well.

This also affects subgraph validation and code generation in a few ways. While Graph Node >= 0.15.0 still allows subgraphs with nested templates to run, Graph CLI now rejects such subgraphs. Code generation now puts all generated template classes into a single templates.ts file.

For more details, see the original issue.

Docker changes

• Set default Ethereum network name to mainnet in docker-compose.yml (#1086).
• Add setup.sh script for Linux host IP detection. Run this before the docker-compose up and it will inject the host IP address into docker-compose.yml (#1123).

Other changes

• Add support for arrays of Ethereum tuples / Solidity structs (#1119).
• Optimize performance of serializing data from Rust to AssemblyScript (#1145).
• Support decoding Ethereum strings with broken UTF-8 encoding (#1152).
• Bump default IPFS timeout to 60s (#1091).
• Fetch blocks from Ethereum if loading them from the cache fails (#1108).
• Reduce memory usage of filtering blocks with call traces (#1110).
• Add GRAPH_TOKIO_THREAD_COUNT environment variable (#1117, see environment variable docs).
• Add GRAPH_NODE_ID environment variable as an alternative to --node-id (#1136, see environment variable docs).
• Fix block/call handlers from dynamic data sources not being loaded on node restart (#1087).
• Fix ingesting blocks with no transactions (#1099).
• Fix indexing the genesis block with Ganache (#1094).
• Fix skipping call traces without insufficient data (#1104).
• Fix Postgres indexes for string fields to support values of arbitrary length (#1138).
• Update codebase to Rust 1.37 (#1118).
• Update dependencies.

v0.14.0

v0.14.0