Support Matrix¶
Supported distributed communication backends¶
| MPI4Py | torch.distributed | Horovod | paddle.distributed | oneflow.comm |
|---|---|---|---|---|
| MPI4PyDist | TorchCPUDist TorchCUDADist |
TFHorovodDist | PaddleDist | OneFlowDist |
Supported metrics and ML frameworks¶
Note
The following table lists the metrics implemented by MMEval and the corresponding machine learning framework support. A check mark indicates that the data type of the corresponding framework (e.g. Tensor) can be directly passed for computation.
Note
MMEval tested with PyTorch 1.6+, TensorFlow 2.4+, Paddle 2.2+ and OneFlow 0.8+.
| Metric | numpy.ndarray | torch.Tensor | tensorflow.Tensor | paddle.Tensor | oneflow.Tensor |
|---|---|---|---|---|---|
| Accuracy | ✔ | ✔ | ✔ | ✔ | ✔ |
| SingleLabelMetric | ✔ | ✔ | ✔ | ||
| MultiLabelMetric | ✔ | ✔ | ✔ | ||
| AveragePrecision | ✔ | ✔ | ✔ | ||
| MeanIoU | ✔ | ✔ | ✔ | ✔ | ✔ |
| VOCMeanAP | ✔ | ||||
| OIDMeanAP | ✔ | ||||
| COCODetection | ✔ | ||||
| ProposalRecall | ✔ | ||||
| F1Score | ✔ | ✔ | ✔ | ||
| HmeanIoU | ✔ | ||||
| PCKAccuracy | ✔ | ||||
| MpiiPCKAccuracy | ✔ | ||||
| JhmdbPCKAccuracy | ✔ | ||||
| EndPointError | ✔ | ✔ | ✔ | ||
| AVAMeanAP | ✔ | ||||
| StructuralSimilarity | ✔ | ||||
| SignalNoiseRatio | ✔ | ||||
| PeakSignalNoiseRatio | ✔ | ||||
| MeanAbsoluteError | ✔ | ||||
| MeanSquaredError | ✔ |