Looks like `strawberry.Streamable` isn't quite following the spec

[metanivek]

Describe the Bug

I'm testing strawberry.Streamable. My code looks analogous to what is in the docs: https://strawberry.rocks/docs/types/defer-and-stream.

But, unlike the docs, the response I get from the server does not follow the spec for path.

Instead of something like

"path": ["blogPost", "comments", 2]

I get

"path": ["blogPost", "comments"]

which looks like what it should be for @defer, not @stream.

System Information

❯ uv pip show strawberry-graphql
Name: strawberry-graphql
Version: 0.281.0

[patrick91]

@metanivek can you post a minimal reproduction? what version of graphql core are you using?

[metanivek]

@patrick91 sure!

Save the following script and use uv run to start server:

# /// script
# requires-python = ">=3.10"
# dependencies = [
#   "strawberry-graphql[asgi]==0.281.0",
#   "graphql-core>=3.3.0a9",
#   "uvicorn==0.35.0"
# ]
# ///
import asyncio
from typing import AsyncGenerator

import strawberry
from strawberry.schema.config import StrawberryConfig
from strawberry.asgi import GraphQL
import uvicorn


@strawberry.type
class Comment:
    id: strawberry.ID
    content: str
    author_name: str


@strawberry.type
class BlogPost:
    id: strawberry.ID
    title: str

    @strawberry.field
    async def comments(self) -> strawberry.Streamable[Comment]:
        for i in range(5):
            await asyncio.sleep(0.5)
            yield Comment(
                id=strawberry.ID(f"comment-{i}"),
                content=f"This is comment number {i}",
                author_name=f"User {i}",
            )


@strawberry.type
class Query:
    @strawberry.field
    async def blog_post(self) -> BlogPost:
        return BlogPost(id=strawberry.ID("post-1"), title="Streaming demo")


schema = strawberry.Schema(
    query=Query,
    config=StrawberryConfig(enable_experimental_incremental_execution=True),
)

app = GraphQL(schema)

if __name__ == "__main__":
    uvicorn.run(app, host="127.0.0.1", port=7000)

Run the following in graphiql: http://127.0.0.1:7000/graphql

query {
  blogPost {
    id
    title
    comments @stream(initialCount: 2) {
      id
      content
      authorName
    }
  }
}

Observe network traffic:

---
Content-Type: application/json; charset=utf-8
Content-Length: 346

{"data": {"blogPost": {"id": "post-1", "title": "Streaming demo", "comments": [{"id": "comment-0", "content": "This is comment number 0", "authorName": "User 0"}, {"id": "comment-1", "content": "This is comment number 1", "authorName": "User 1"}]}}, "hasNext": true, "pending": [{"id": "0", "path": ["blogPost", "comments"]}], "extensions": null}
---
Content-Type: application/json; charset=utf-8
Content-Length: 211

{"hasNext": true, "extensions": null, "incremental": [{"items": [{"id": "comment-2", "content": "This is comment number 2", "authorName": "User 2"}], "id": "0", "path": ["blogPost", "comments"], "label": null}]}
---
Content-Type: application/json; charset=utf-8
Content-Length: 211

{"hasNext": true, "extensions": null, "incremental": [{"items": [{"id": "comment-3", "content": "This is comment number 3", "authorName": "User 3"}], "id": "0", "path": ["blogPost", "comments"], "label": null}]}
---
Content-Type: application/json; charset=utf-8
Content-Length: 211

{"hasNext": true, "extensions": null, "incremental": [{"items": [{"id": "comment-4", "content": "This is comment number 4", "authorName": "User 4"}], "id": "0", "path": ["blogPost", "comments"], "label": null}]}
---
Content-Type: application/json; charset=utf-8
Content-Length: 66

{"hasNext": false, "extensions": null, "completed": [{"id": "0"}]}
-----

[patrick91]

thanks!

Pretty sure we need to update the logic here: https://github.com/strawberry-graphql/strawberry/blob/main/strawberry/http/async_base_view.py#L409-L434

I'll to read the spec again (if I can find it 😅) in the next day or two and send a fix

[erikwrede]

From the way I read the spec, we implement the specified behavior: https://github.com/graphql/graphql-spec/blob/8e9cbcb46384399f5ed73996fac008de1d394551/spec/Section%207%20--%20Response.md#incremental

Incremental List Result Format
[...]
The Incremental List Result's path can be determined using the prior Pending Result with the same id as this Incremental Result. The Incremental List Result's path is the same as the Pending Result's path.

and then also

Pending Result Format

[...]

Every Pending Result must contain an entry with the key path.

The key path in your case being ["blogPost", "comments"]. From these lines, I cannot deduct a requirement to append the index in the incremental responses. If anything, we are including too much detail by also including path and label in incremental - looking at spec examples and tests, they should only be in pending: https://github.com/graphql/graphql-js/blob/ec9c063b0355f4bba6cec45a4ee51c3a8560e1c9/src/execution/__tests__/stream-test.ts#L340-L389

@metanivek can you point us to the exact spec quote you base this issue upon?

[metanivek]

I based my original post on a few things. Collectively these corresponded to a "spec" in my head. Apologies for the loose usage of the word!

Strawberry documentation for defer and stream

graphql-yoga implementation. See test snapshot

This blog post https://graphql.org/blog/2020-12-08-defer-stream/

[erikwrede]

@metanivek no worries, the stream and defer spec is somewhat fluid since its still an RFC and the response format has changed overtime. Let me do some more research on what's the current Standard and get back here 😊

[erikwrede]

I asked on the discord, our behavior looks to be correct while yoga's is outdated.

[metanivek]

@erikwrede thanks!

I looked around a little more based on your response (for my own curiosity on the history), and found graphql/graphql-spec#1110 as a cross reference if anybody else stumbles upon this issue.

I think this can be closed now since Strawberry is already setup for the future (but the clients need to catch up). 😄