Tracing for Production Applications

[kevin-m-kent]

Hi - I see there is useful output after each run tracing the activity of each node (inputs, outputs, execution time, etc) when I run the flow locally under .promptflow/chat.detail.json. I am interested in sending these traces to a place like Application Insights for my promptflow instance that I have deployed as an Azure web application. I see in the tracing and dev modules there are references to application insights but I can't quite figure out how one would go about sending these traces to a place like application insights. Are there any examples of this or guidance? Thank you.

[kevin-m-kent]

Is it as simple as setting APPLICATIONINSIGHTS_CONNECTION_STRING with devkit installed?

[kevin-m-kent]

ok thanks! I did this but I am not getting any traces to my instance. Should this work out of the box? Any particular promptflow version? (I'm on 1.5.0 at the moment).

[kevin-m-kent]

upgraded but I think I figured out why it isn't working, some issue with None values:

failed to record metrics: object of type 'NoneType' has no len()

seems it might be failing on this line -

promptflow/src/promptflow-core/promptflow/core/_serving/monitor/metrics.py

Line 216 in 6b9c5c6

if len(run.system_metrics) > 0:

maybe I should migrate this to an issue? Looking at the debug output it does look like some of my nodes have None for system_metrics.

[kevin-m-kent]

Opened #2644

[wxpjimmy]

@kevin-m-kent trace and metrics are two different things, the metrics issue won't impact your collected trace data, have you checked your trace data after upgrading to the latest PF package?

[kevin-m-kent]

yes I did and I wasn't getting any traces. And the error message I was getting was coming from the function I listed above. there is no dependency between record_tracing_metrics failing and the trace collection and upload path?