Skip to content

Implement in-thunk wait() with task continuations #391

@jpsamaroo

Description

@jpsamaroo

In order for a running Dagger task to wait on another task (such as when a task spawns a DAG dynamically and waits on its completion), we need to communicate to the scheduler that our task is going to temporarily sleep until a certain condition is met (namely that another task has completed). We currently hack around this by forcibly decreasing the current processor's pressure/occupancy by the amount that the yielding task consumes, which effectively allows another task to be run. This is generally the behavior that we want, but is not a very clean way to do it, and can introduce more bugs when the functionality needs to be tweaked.

A better approach would be to treat the yielding task as temporarily completed, and scheduling its continuation (the rest of its execution) as another task to be scheduled once a condition is reached. This would remove the task from view of the processor it was running on, which simplifies per-processor scheduling. This yielding mechanism would also be beneficial for increasing the number of low-occupancy tasks that can be scheduled at once, since we then have the ability to run another task while we wait for a waiting task to meet its condition.

The condition based-scheduling required for this feature would also be quite useful for other cases where we want to intentionally pause execution of a new or yielding task until same important constraint is loosened (such as waiting until more memory becomes available, waiting for data to become available on a Channel, etc.).

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions