Udacity AI Product Manager Nanodegree Exercises

Welcome to the repository for the Udacity AI Product Manager Nano Degree exercises. This repository contains the necessary files and instructions for completing the exercises in this course.

Folder structure

exercises
    |_ exercise1-defining-success-metrics
    |   |_ starter
    |   |   |_ starter_files
    |   |_ solution
    |   |   |_ solution_files
    |_ exercise1-fairness-evaluation-report
    |   |_ solution
    |   |   |_ solution_files
    |_ README.md

Exercise 1: Defining Success Metrics

Objective

Develop a comprehensive plan to define, measure, and monitor success metrics for the Mango Oasis app, ensuring it provides accurate, helpful, and unbiased responses to customers interested in mango-related products.

Instructions

1. Understand the Requirements:

   • Review the Mango Oasis app's goals and user needs.
   • Identify key success metrics that align with these goals.

2. Define Success Metrics:

   • Define specific metrics such as Context Recall, Context Precision, Faithfulness, Answer Relevance, and any other relevant metrics.
   • Explain the importance of each metric and how it contributes to the overall success of the app.

3. Measure Success Metrics:

   • Use the provided dataset and code snippets to calculate the defined metrics.
   • Ensure the calculations are accurate and reflect the true performance of the Mango Oasis app.

4. Monitor Success Metrics:

   • Develop a plan to continuously monitor these metrics over time.
   • Suggest tools and techniques for ongoing evaluation and improvement.

5. Document Your Findings:

   • Create a detailed report summarizing your plan, the metrics defined, and the rationale behind choosing these metrics.
   • Include charts and visualizations to illustrate the current performance of the app based on the calculated metrics.

Deliverables

• Your findings
• A comprehensive report detailing the defined success metrics, measurement methods, and monitoring strategies.

Note: You do not need to run the code provided in the solution files. Your task is to review the generated report and understand the key findings and methodologies used.

Exercise 2: Fairness Evaluation Report

Objective

Analyze the results from a pre-run Fairness Evaluation Notebook and create a report that communicates the key findings, insights, and recommendations for mitigating bias in the Mango Oasis app.

Instructions

1. Review the Fairness Evaluation Notebook:

   • Load and review the provided notebook containing the results of the fairness evaluation.
   • Understand the metrics used and the results obtained.

2. Analyze the Results:

   • Summarize the key findings from the notebook.
   • Identify instances of bias and understand their potential impact on the app's performance.

3. Create a Fairness Evaluation Report:

   • Communicate the key findings in a clear and concise manner.
   • Provide insights into the causes of identified biases.
   • Suggest actionable recommendations for mitigating these biases.

4. Include Visualizations:

   • Use charts and graphs to highlight key points and make the report more engaging.
   • Ensure that visualizations are clear, accurate, and effectively support the report's findings.

Deliverables

• A detailed fairness evaluation report summarizing the key findings, insights, and recommendations for bias mitigation.
• Supporting visualizations that enhance the report's clarity and impact.

Note: The fairness evaluation notebook is pre-run, and you do not need to execute any code. Focus on reviewing the report and extracting key insights from the analysis.

Submission Guidelines

• Submit your completed exercises through the designated submission platform.

---

License

The contents of this repository are covered under Udacity License Terms