Evaluator Identifier for Relevance and Similarity are incorrect

[m-gheini]

• Package Name: azure-ai-evaluation
• Package Version: 1.5.0
• Operating System: Windows
• Python Version: 3.12.10

Describe the bug
For defining evaluators within cloud evaluation, we need to access evaluators by their ids. Right now id for Similarity and Relevance are incorrect and are not pointing to latest version of Asset ID used in Azure Foundry Evaluators. This will cause error during evaluation and not seeing results for these evaluators on evaluation tab on Foundry.

To Reproduce

Import Built in Evaluators:

from azure.ai.evaluation import (
    RelevanceEvaluator,
    SimilarityEvaluator,
)

Create a cloud evaluation with evaluators as below:

evaluators={
        "relevance": EvaluatorConfiguration(
            id=RelevanceEvaluator.id,
            init_params={
                "model_config": model_config
            }
        ),
        "similarity": EvaluatorConfiguration(
            id=SimilarityEvaluator.id,
            init_params={
                "model_config": model_config
            }
        )
    }

Expected behavior
Access built in evaluator by id for latest version and run evaluator. Also get results in Foundry Evaluation tab.

Additional context
There is already a PR open for Similarity Evaluator #40804

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @luigiw @needuv @singankit.

[kashifkhan]

Thank you for the feedback @m-gheini. Ill route this over to the right team who can help you further

[cmelo05]

JFYI, I updated the PR with the update for Relevance Evaluator too

[singankit]

Thanks for the feedback. There is work going on make this better. Will update once the work in complete.

[cmelo05]

@singankit just for my understanding, since there is currently work for this feature to be better, this ID fix won't be merged, correct?

[singankit]

@cmelo05 Thanks for the message. I will be reaching out to the service team and depending on the timeline a decision can be made to either merge the PR or wait for service change to roll out and then update.