Eval HOTA Bug

[xifen523]

I want to eval HOTA

• this is my file track_txt.txt, gt.txt

my code

import os
import numpy as np
import motmetrics as mm


def compute_motchallenge(gtSource, tSource):
    # `gt.txt` and `test.txt` should be prepared in MOT15 format
    df_gt = mm.io.loadtxt(gtSource)
    df_test = mm.io.loadtxt(tSource)
    # Require different thresholds for matching
    th_list = np.arange(0.05, 0.99, 0.05)
    res_list = mm.utils.compare_to_groundtruth_reweighting(df_gt, df_test, "iou", distth=th_list)
    return res_list

# `data_dir` is the directory containing the gt.txt and test.txt files



if __name__ == '__main__':
    track_txt = r'track_txt.txt'
    gt_txt= r'gt_txt.txt'

    acc = compute_motchallenge(gt_txt, track_txt)
    mh = mm.metrics.create()

    summary = mh.compute_many(
        acc,
        metrics=[
            "deta_alpha",
            "assa_alpha",
            "hota_alpha",
        ],
        generate_overall=True,  # `Overall` is the average we need only
    )
    strsummary = mm.io.render_summary(
        summary.iloc[[-1], :],  # Use list to preserve `DataFrame` type
        formatters=mh.formatters,
        namemap={"hota_alpha": "HOTA", "assa_alpha": "ASSA", "deta_alpha": "DETA"},
    )
    print(strsummary)
    """
    # data_dir=motmetrics/data/TUD-Campus
            DETA  ASSA  HOTA
    OVERALL 41.8% 36.9% 39.1%
    # data_dir=motmetrics/data/TUD-Stadtmitte
            DETA  ASSA  HOTA
    OVERALL 39.2% 40.9% 39.8%
    """

the error is ：

  File "c:/Users/lk/Desktop/python/track/ByteTrack/py-motmetrics/py_eval_HOTA copy.py", line 23, in <module>
    acc = compute_motchallenge(gt_txt, track_txt)
  File "c:/Users/lk/Desktop/python/track/ByteTrack/py-motmetrics/py_eval_HOTA copy.py", line 12, in compute_motchallenge
    res_list = mm.utils.compare_to_groundtruth_reweighting(df_gt, df_test, "iou", distth=th_list)
  File "c:\Users\lk\Desktop\python\track\ByteTrack\py-motmetrics\motmetrics\utils.py", line 156, in compare_to_groundtruth_reweighting      
    acc.update(oids, hids, 1 - weighted_dists, frameid=fid, similartiy_matrix=dists, th=th)
  File "c:\Users\lk\Desktop\python\track\ByteTrack\py-motmetrics\motmetrics\mot.py", line 217, in update
    self._append_to_events('RAW', oids[i], hids[j], dist_ij)
IndexError: index 1 is out of bounds for axis 0 with size 0

how to fix this bug

[jrichardson-tw]

Hi, I started playing around with the dev HOTA functions, and I ran into the same issue. Pretty much from what I can tell, there is a slight difference between the original compare_to_groundtruth function and the compare_to_groundtruth_reweighting variant specified for HOTA calculations which leads to this error. As we can see in the utils file where these functions are defined, the compare_to_groundtruth resets the dists variable to an empty matrix of dim (0, 0) (line 227) whereas the compare_to_groundtruth_reweighting variant only resets the weighted_dists variable (line 141) but not the dists variable. I'm not too sure if dists should be carried over between loop iterations yet, but the effect is that since dists only gets recalculated when len(oids) > 0 and len(hids) > 0 (line 148), this causes the parameters of the _append_to_events function called in update to attempt to index into hid or gid when they are empty.

Pretty much, this bug will occur any time there is an input frame which either has Gids and no Hids or v.v.