When to specify stream for an operation?

[lbench]

When should or shouldn't a stream be specified for an operation?

[awni]

The primary use cases are:

• If you want to specify a device (like CPU or GPU) you can use the stream kwarg for that.
• If you want to run operations in parallel (CPU or GPU) you can use multiple streams. For really compute bound workloads it won't help much to use multiple streams on the GPU.

In general specifying a stream is a more advanced use case and OI would always start by running everything on the default stream of the default device (the GPU on Apple silicon), profile and optimize from there.