WordAccuracy¶
- class mmeval.metrics.WordAccuracy(mode: Union[str, Sequence[str]] = 'ignore_case_symbol', invalid_symbol: str = '[^A-Za-z0-9一-龥]', **kwargs)[源代码]¶
Calculate the word level accuracy.
- 参数
mode (str or list[str]) –
Options are:
’exact’: Accuracy at word level.
’ignore_case’: Accuracy at word level, ignoring letter case.
’ignore_case_symbol’: Accuracy at word level, ignoring letter case and symbol. (Default metric for academic evaluation)
If mode is a list, then metrics in mode will be calculated separately. Defaults to ‘ignore_case_symbol’.
invalid_symbol (str) – A regular expression to filter out invalid or not cared characters. Defaults to ‘[^A-Za-z0-9u4e00-u9fa5]’
**kwargs – Keyword parameters passed to
BaseMetric
.
实际案例
>>> from mmeval import WordAccuracy >>> metric = WordAccuracy() >>> metric(['hello', 'hello', 'hello'], ['hello', 'HELLO', '$HELLO$']) {'ignore_case_symbol_accuracy': 1.0} >>> metric = WordAccuracy(mode=['exact', 'ignore_case', >>> 'ignore_case_symbol']) >>> metric(['hello', 'hello', 'hello'], ['hello', 'HELLO', '$HELLO$']) {'accuracy': 0.333333333, 'ignore_case_accuracy': 0.666666667, 'ignore_case_symbol_accuracy': 1.0}
- add(predictions: Sequence[str], groundtruths: Sequence[str]) → None[源代码]¶
Process one batch of data and predictions.
- 参数
predictions (list[str]) – The prediction texts.
groundtruths (list[str]) – The ground truth texts.
- compute_metric(results: List[Tuple[int, int, int]]) → Dict[源代码]¶
Compute the metrics from processed results.
- 参数
results (list[float]) – The processed results of each batch.
- 返回
Nested dicts as results. Provided keys are:
accuracy (float): Accuracy at word level.
ignore_case_accuracy (float): Accuracy at word level, ignoring letter case.
ignore_case_symbol_accuracy (float): Accuracy at word level, ignoring letter case and symbol.
- 返回类型
dict[str, float]