#GetHelp with Lab334: Evaluate and improve the quality and safety of your AI applications

[nitya]

About Lab334

⁉️ - Have a question, comment or insight to share about this lab? Leave us a comment on this thread. The content creators and instructors will monitor this and respond within a day or two. We will also capture the main responses in an FAQ on the repo.

---

Evaluations are a key tool in your AI Observability toolkit! This lab helps you skill up on the Azure AI Foundry evaluations SDK and portal features and capabilities, helping you assess the quality and safety of your AI applications. This discussion thread is for our in-venue AND self-guided learners to ask questions, share feedback, and report bugs or suggestions that can help us make this lab a better learning experience!.

Review The Slides

Learning Options

• Join A Lab Session At Build
• Self-Paced Lab At Home

Abstract

You’ve built a custom AI application grounded in your enterprise data. How do you ensure response quality and safety? Join us as we explore AI-assisted evaluation workflows with built-in and custom evaluators on Azure AI. Learn what each metric represents, then understand how to analyze the scores for your specific application. Learn why observability is key, and how the generated telemetry can be used both locally, and in the cloud, to help you assess and debug your application performance.

Outline

[amynic]

Welcome all to the Azure AI Foundry Developer Forum - excited for @nitya to use this space for all your questions and connections to our experts! - consider marking our forum as a favourite as you get more embedded into building with the Azure AI Foundry Developer forum https://aka.ms/azureaifoundry/forum

Also if your a Discord fan, join our 25K dev base over there too! https://aka.ms/azureaifoundry/discord

[mlindemu-oneoff]

In lab 03-safety-evaluators, section 7: the ProtectedMaterialEvaluator fails to recognize John Denver's "Take Me Home, Country Road" as protected material.

[nitya]

Thanks for capturing this for me @mlindemu-oneoff - have shared this with the right team and we are looking into it! Will update here when resolved. cc @luigiw

[nitya]

Thanks to everyone who attended this lab at #MSBuild. Now that the event is behind us, we'll close out this discussion. If you have any questions or comments please do post an issue instead. Happy Learning!