WordAccuracy

class mmeval.metrics.WordAccuracy(mode: Union[str, Sequence[str]] = 'ignore_case_symbol', invalid_symbol: str = '[^A-Za-z0-9一-龥]', **kwargs)

Calculate the word level accuracy.

Parameters

• mode (str or list[str]) –

  Options are:

  • ’exact’: Accuracy at word level.

  • ’ignore_case’: Accuracy at word level, ignoring letter case.

  • ’ignore_case_symbol’: Accuracy at word level, ignoring letter case and symbol. (Default metric for academic evaluation)

  If mode is a list, then metrics in mode will be calculated separately. Defaults to ‘ignore_case_symbol’.

• invalid_symbol (str) – A regular expression to filter out invalid or not cared characters. Defaults to ‘[^A-Za-z0-9u4e00-u9fa5]’

• **kwargs – Keyword parameters passed to BaseMetric.

Examples

>>> from mmeval import WordAccuracy
>>> metric = WordAccuracy()
>>> metric(['hello', 'hello', 'hello'], ['hello', 'HELLO', '$HELLO$'])
{'ignore_case_symbol_accuracy': 1.0}
>>> metric = WordAccuracy(mode=['exact', 'ignore_case',
>>>                             'ignore_case_symbol'])
>>> metric(['hello', 'hello', 'hello'], ['hello', 'HELLO', '$HELLO$'])
{'accuracy': 0.333333333,
 'ignore_case_accuracy': 0.666666667,
 'ignore_case_symbol_accuracy': 1.0}

add(predictions: Sequence[str], groundtruths: Sequence[str]) → None

Process one batch of data and predictions.

Parameters

• predictions (list[str]) – The prediction texts.

• groundtruths (list[str]) – The ground truth texts.

compute_metric(results: List[Tuple[int, int, int]]) → Dict

Compute the metrics from processed results.

Parameters

results (list[float]) – The processed results of each batch.

Returns

Nested dicts as results. Provided keys are:

• accuracy (float): Accuracy at word level.

• ignore_case_accuracy (float): Accuracy at word level, ignoring letter case.

• ignore_case_symbol_accuracy (float): Accuracy at word level, ignoring letter case and symbol.

Return type

dict[str, float]