[cheezum]

Hi,

I tried to setup an RDMA cluster in Azure by using 2 A8 machines with SuSE 12 (default templates from Azure). The network interfaces are available but when I try to pingpong them the machines are not able to see each other.

Both machines are in the same Resource Group and when I use the ethernet connection on eth0 the machines can ping each other.
Also, when I run some basic diagnostics with ibv_dev and other tools, all shows OK and there is 1 port available. What I find weird is that the port is connected via ethernet and not via Infiniband.

So my question is: what do I need to do to at least pingpong both machines?

[arohner]

Hi Cheezum,

Ensure to use the SLES 12 SP1 for HPC. The RDMA Network seems still to have some bugs, but after a reboot of the node all devices are loaded correctly. In addition on this template there is already intel mpi installed. You may fallow these instructions:
https://docs.microsoft.com/en-us/azu...c-rdma-cluster

Good Luck &
best wishes
Adrian

[cheezum]

Hi Adrian,

Thanks for your response.
I totally forgot this thread.
In December I've had contact with some HPC guys from Microsoft and one of them had a background with ANSYS.

Guess what ....

There's a bug in ANSYS 17.2 (which we were using at that time) that hostnames longer than 12 characters didn't work. It wasn't a problem in 16 and it isn't in 18.0

That was one part of it.
The other one was the distro:
I already used the SLES 12.1 with HPC and I had no succes in using this distro.
Because the tech guys from Microsoft used examples with the CentOS 7.1 distro with HPC I decided to give that a shot.
And voila... Out of the box, it works. The only part that needs to be done extra is setting up de variables in the .bashrc so the MPI variables are loaded when a user is logging in.

Regarding ANSYS, the guys from MS have some great basic scripts available on the Github to setup a basic linux oriented cluster.
You can find them at https://github.com/tanewill/5clickTe...awANSYSCluster