自定义评测指标

在 MMEval 中实现一个自定义评测指标，需要继承 BaseMetric 并且实现 add 和 compute_metric 方法。

在评测过程中，评测指标需要在调用 add 后更新 _results 以存储中间结果。在最后进行指标计算的时候，将会对 _results 进行进程同步后调用 compute_metric 进行指标的计算。

以实现 Accuracy 指标为例：

import numpy as np
from mmeval.core import BaseMetric

class Accuracy(BaseMetric):

    def add(self, predictions, labels):
        self._results.append((predictions, labels))

    def compute_metric(self, results):
        predictions = np.concatenate(
            [res[0] for res in results])
        labels = np.concatenate(
            [res[1] for res in results])
        correct = (predictions == labels)
        accuracy = sum(correct) / len(predictions)
        return {'accuracy': accuracy}

使用 Accuracy：

# stateless call
accuracy = Accuracy()
metric_results = accuracy(predictions=[1, 2, 3, 4], labels=[1, 2, 3, 1])
print(metric_results)
# {'accuracy': 0.75}

# Accumulate batch
for i in range(10):
    predicts = np.random.randint(0, 4, size=(10,))
    labels = predicts = np.random.randint(0, 4, size=(10,))
    accuracy.add(predicts, labels)

metric_results = accuracy.compute()
accuracy.reset()  # clear the intermediate results