...

I use trevis-40[1,2] as an example. But this will need to be done for other nodes with multiple interfaces of the same kind, MLX, OPA, ETH

The main difference is in the routing rules. The rules explicitly cause the route selection algorithm to look at the ib0 or ib1 routing tables based on the source prefix. Therefore any packet with a source address set to ib0 or ib1's IP address triggers the rules and is first matched against the route in the corresponding table. In this way messages are guaranteed to use the correct interface. (ib0 and ib1 are used as examples)

trevis-401

401 is the most complicated node in the cluster. It has 2 ETH, 2 OPA and 2 MLX interfaces. 

...