Pipeline streaming

[stellasia]

Description

Add a Pipeline.stream method that yields events about pipeline progress leveraging the callback mechanism already in place:

• Make pipeline.callbacks a list
• Implement a new Pipeline.stream method with the following logic:
  • Create a new callback that put events in an asyncio.Queue and append it to the callbcks list
  • Create asyncion task to run pipeline in background
  • Loop until the run task is done and the event queue is empty:
    • check run task status (done or pending)
    • yield event from queue if any

Type of Change

• New feature
• Bug fix
• Breaking change
• Documentation update
• Project configuration change

Complexity

Complexity: High

How Has This Been Tested?

• Unit tests
• E2E tests
• Manual tests

Checklist

The following requirements should have been met (depending on the changes in the branch):

• Documentation has been updated
• Unit tests have been updated
• E2E tests have been updated
• Examples have been updated
• New files have copyright header
• CLA (https://neo4j.com/developer/cla/) has been signed
• CHANGELOG.md updated if appropriate

[oskarhane]

Looks like a good change! 🍶