Working with OTEL in the reactive code - how does it work? Does it even work?

[jan-peremsky]

Our whole BE codebase is built on top of Quarkus.
Our code is fully reactive and command/event driven.
We use Azure CosmosDB as the persistence layer and as the event hub. CosmosDB driver is reactive as well but built on top of the Reactor framework.
Once we tried to produce OTEL based remote tracing info - problems arose.

@WithSpan does not work on the reactive (Uni<?> returning) methods - the instrumentation creates a Span but just for the pipeline construction phase. The Span is ended long before the subscription and pipeline-execution phase.
Calling CosmosDB Reactor pipelines (they use their own thread pools) messes up with Spans even more.
If we receive an asynchronous "command/event" from the CosmosDB event bus - there is no easy way to start a span in order to continue the execution trace.
We ended up creating a utility class which:

• can start or continue a trace by injecting it to the Vertx context
• start/end spans around Unis
• access active span on the reactive pipeline via the withContext and inject attributes and events
• wrap reactor based pipelines, so that the tracing info does not get lost

This works , however it is very fragile as every Quarkus change might render it broken.

The questions finally:

• is there a plan to support Span handling (creation, events, attributes) on the Mutiny pipeline?
• is there a plan to have an official way to start/continue a trace for non-quarkus libraries like CosmosDB?