Skip to content

Commit 668e0b0

Browse files
authored
Fix device for OpenVINO Runtime in whisper evaluator (#3994)
* Fix device format for OpenVINO Runtime * Update README.md with whisper evaluator
1 parent 272ffd6 commit 668e0b0

File tree

2 files changed

+10
-5
lines changed

2 files changed

+10
-5
lines changed

tools/accuracy_checker/accuracy_checker/evaluators/custom_evaluators/README.md

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -54,10 +54,15 @@ Optionally you can provide `module_config` section which contains config for cus
5454
Evaluator code: `<omz_dir>/tools/accuracy_checker/accuracy_checker/evaluators/custom_evaluators/sr_evaluator.py`.
5555

5656
* **Tacotron2 Evaluator** demonstrates how to evaluate custom Tacotron2 model for text to speech task.
57-
Evaluator code: `<omz_dir>/tools/accuracy_checker/accuracy_checker/evaluators/custom_evaluators/tacotron2_evaluator.py`
57+
Evaluator code: `<omz_dir>/tools/accuracy_checker/accuracy_checker/evaluators/custom_evaluators/tacotron2_evaluator.py`.
5858

5959
* **Open-NMT Evaluator** demonstrates how to evaluate Open-NMT model for text translation task.
60-
Evaluator code: `<omz_dir>/tools/accuracy_checker/accuracy_checker/evaluators/custom_evaluators/opennmt_encoder_decoder_generator_evaluator.py`
60+
Evaluator code: `<omz_dir>/tools/accuracy_checker/accuracy_checker/evaluators/custom_evaluators/opennmt_encoder_decoder_generator_evaluator.py`.
6161

6262
* **StableDiffusion Evaluator** demonstrates how to evaluate image generation for Stable Diffusion models family.
63-
Evaluator code: `<omz_dir>/tools/accuracy_checker/accuracy_checker/evaluators/custom_evaluators/stable_diffusion_evaluator.py`
63+
Evaluator code: `<omz_dir>/tools/accuracy_checker/accuracy_checker/evaluators/custom_evaluators/stable_diffusion_evaluator.py`.
64+
65+
* **Whisper Evaluator** demonstrates how to evaluate Whisper family models with various pipeline classes, including GenAIWhisperPipeline, HFWhisperPipeline, and OptimumWhisperPipeline.
66+
Evaluator code: `<omz_dir>/tools/accuracy_checker/accuracy_checker/evaluators/custom_evaluators/whisper_evaluator.py`.
67+
68+

tools/accuracy_checker/accuracy_checker/evaluators/custom_evaluators/whisper_evaluator.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
"""
2-
Copyright (c) 2024 Intel Corporation
2+
Copyright (c) 2024-2025 Intel Corporation
33
44
Licensed under the Apache License, Version 2.0 (the "License");
55
you may not use this file except in compliance with the License.
@@ -54,7 +54,7 @@ def from_configs(cls, config, delayed_model_loading=False, orig_config=None):
5454
dataset_config = config["datasets"]
5555
pipeline_class_name = config["pipeline_class"]
5656
if 'device' in config['launchers'][0]:
57-
config["_device"] = config['launchers'][0]['device']
57+
config["_device"] = config['launchers'][0]['device'].upper()
5858

5959
if pipeline_class_name not in cls.VALID_PIPELINE_CLASSES:
6060
raise ValueError(f"Invalid pipeline class name: {pipeline_class_name}. "

0 commit comments

Comments
 (0)