TensorBaseDistBackend

class mmeval.core.dist_backends.TensorBaseDistBackend

A base backend of Tensor base distributed communication like PyTorch.

all_gather_object(obj: Any) → List[Any]

All gather the given object from the current process group and returns a list consisting gathered object of each process..

There are 3 steps to all gather a python object using Tensor distributed communication:

1. Serialize picklable python object to tensor.

2. All gather the tensor size and padding the tensor with the same size.

3. All gather the padded tensor and deserialize tensor to picklable python object.

Parameters

obj (any) – Any pickle-able python object for all gather.

Returns

A list of the all gathered object.

Return type

list

broadcast_object(obj: Any, src: int = 0) → Any

Broadcast the given object from source process to the current process group.

There are 3 steps to broadcast a python object use Tensor distributed communication:

1. Serialize picklable python object to tensor.

2. Broadcast the tensor size and padding the tensor with the same size.

3. Broadcast the padded tensor and deserialize tensor to picklable python object.

Parameters

• obj (any) – Any pickle-able python object for broadcast.

• src (int) – The source rank index.

Returns

The broadcast object.

Return type

any