Skip to content

Commit c00e101

Browse files
Export experiment duration in seconds in CSV. (#2392)
* Export experiment duration in seconds in CSV. Signed-off-by: Weilin Xu <weilin.xu@intel.com> * Update CHANGELOG Signed-off-by: Weilin Xu <weilin.xu@intel.com> * Log fit and test durations separately. Signed-off-by: Weilin Xu <weilin.xu@intel.com> --------- Signed-off-by: Weilin Xu <weilin.xu@intel.com> Co-authored-by: Samet Akcay <samet.akcay@intel.com>
1 parent 0301d59 commit c00e101

File tree

2 files changed

+13
-0
lines changed

2 files changed

+13
-0
lines changed

CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
1515

1616
### Changed
1717

18+
- Add duration of experiments in seconds in the benchmark CSV result by [mzweilin](https://github.com/mzweilin) in https://github.com/openvinotoolkit/anomalib/pull/2392
19+
1820
### Deprecated
1921

2022
### Fixed

src/anomalib/pipelines/benchmark/job.py

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
# SPDX-License-Identifier: Apache-2.0
55

66
import logging
7+
import time
78
from datetime import datetime
89
from pathlib import Path
910
from tempfile import TemporaryDirectory
@@ -48,6 +49,7 @@ def run(
4849
task_id: int | None = None,
4950
) -> dict[str, Any]:
5051
"""Run the benchmark."""
52+
job_start_time = time.time()
5153
devices: str | list[int] = "auto"
5254
if task_id is not None:
5355
devices = [task_id]
@@ -59,8 +61,16 @@ def run(
5961
devices=devices,
6062
default_root_dir=temp_dir,
6163
)
64+
fit_start_time = time.time()
6265
engine.fit(self.model, self.datamodule)
66+
test_start_time = time.time()
6367
test_results = engine.test(self.model, self.datamodule)
68+
job_end_time = time.time()
69+
durations = {
70+
"job_duration": job_end_time - job_start_time,
71+
"fit_duration": test_start_time - fit_start_time,
72+
"test_duration": job_end_time - test_start_time,
73+
}
6474
# TODO(ashwinvaidya17): Restore throughput
6575
# https://github.com/openvinotoolkit/anomalib/issues/2054
6676
output = {
@@ -69,6 +79,7 @@ def run(
6979
"model": self.model.__class__.__name__,
7080
"data": self.datamodule.__class__.__name__,
7181
"category": self.datamodule.category,
82+
**durations,
7283
**test_results[0],
7384
}
7485
logger.info(f"Completed with result {output}")

0 commit comments

Comments
 (0)