AWS support for multiple high throughput EFA interfaces

[flowinh2o]

Hello. There are certain instance types that support multiple network cards such as the A100 p4d instance types that support up to 4 NICs. Is it possible to setup a node with max support network cards and EFA enabled. That way someone could take advantage of the 400Gb/s (4 x 100Gb) throughput between node for distributed GPU training?

[concretevitamin]

It should be possible. One starting point is to customize the node_config field in https://github.com/skypilot-org/skypilot/blob/master/sky/templates/aws-ray.yml.j2 to have something like: ray-project/ray#33586. Note the InterfaceType: efa part.

We don't have expertise in this area and would love some community support here.