Support Matrix¶
Supported distributed communication backends¶
MPI4Py | torch.distributed | Horovod | paddle.distributed | oneflow.comm |
---|---|---|---|---|
MPI4PyDist | TorchCPUDist TorchCUDADist |
TFHorovodDist | PaddleDist | OneFlowDist |
Supported metrics and ML frameworks¶
Note
The following table lists the metrics implemented by MMEval and the corresponding machine learning framework support. A check mark indicates that the data type of the corresponding framework (e.g. Tensor) can be directly passed for computation.
Note
MMEval tested with PyTorch 1.6+, TensorFlow 2.4+, Paddle 2.2+ and OneFlow 0.8+.
Metric | numpy.ndarray | torch.Tensor | tensorflow.Tensor | paddle.Tensor | oneflow.Tensor |
---|---|---|---|---|---|
Accuracy | ✔ | ✔ | ✔ | ✔ | ✔ |
SingleLabelMetric | ✔ | ✔ | ✔ | ||
MultiLabelMetric | ✔ | ✔ | ✔ | ||
AveragePrecision | ✔ | ✔ | ✔ | ||
MeanIoU | ✔ | ✔ | ✔ | ✔ | ✔ |
VOCMeanAP | ✔ | ||||
OIDMeanAP | ✔ | ||||
COCODetection | ✔ | ||||
ProposalRecall | ✔ | ||||
F1Score | ✔ | ✔ | ✔ | ||
HmeanIoU | ✔ | ||||
PCKAccuracy | ✔ | ||||
MpiiPCKAccuracy | ✔ | ||||
JhmdbPCKAccuracy | ✔ | ||||
EndPointError | ✔ | ✔ | ✔ | ||
AVAMeanAP | ✔ | ||||
StructuralSimilarity | ✔ | ||||
SignalNoiseRatio | ✔ | ||||
PeakSignalNoiseRatio | ✔ | ||||
MeanAbsoluteError | ✔ | ||||
MeanSquaredError | ✔ |