I have had my network set up for a while with no issues, it consisted of a router going to two different switches. One is a 10GbE switch where all of my own devices are connected, and the other is a Cisco 1GbE switch that all the roommates devices and the guest AP is connected to. This has worked out well but I wanted to move my own PCs and server over to Infiniband after Skyl3r got me more interested in it. So I just hooked up the Infiniband switch yesterday to the network.
So right now my network looks like this:
I just got the Voltaire/Mellanox 4036E infiniband switch yesterday, and have moved all my computers and server over to it. I was able to SSH into the switch via PuTTY and configure it. I have full network connectivity right now, I can be on my computer and talk to the router just fine and change settings around and all that. However, I cannot get internet on any of the computers that are connected via Infiniband. They have network access to the router, but no internet access. So that is problem #1.
The way the 4036E switch works is that is has thirty four 40gb ports, and two 10GbE SFP+ ports. There is a built in subnet manager that translates the Infiniband traffic to the Ethernet interfaces and vise versa to allow Infiniband clients to talk to the rest of the network. This means there is no need for a separate gateway to bridge the traffic between network types. I know the subnet manager is working, because otherwise I would not be able to talk to the router from my computer right now. I am just using the default minhop subnet manager.
The router IP is set to 192.168.1.1, I set the Infiniband switch to 192.168.1.2. It turns out this is just setting the management interface's Ethernet port to that address. When I plugged the 10GbE SFP+ port from the infiniband switch into my netgear switch, the SFP+ port seems to have assigned itself 192.168.1.3. I assume it must choose the next sequential number after what the management interface is set to, as I did not assign it .3 myself nor is that within the range set for the DHCP server on the router. Either way, thats what it got and the router sees that address and I can talk to the router just fine.
When I plug the cable from my Infiniband HSA (same as an Ethernet NIC for all intents and purposes) to the switch, I am given a proper DHCP address. In this case I was assigned 192.168.1.163. I can also manually assign myself an ipv4 address in the adapter just like any regular ethernet adapter in Windows.
So as far as I can tell, the network seems to be functioning ok. I can ping every device on the network from my computer that is currently on Infiniband, and yet I still cant get internet access on it. All computers that are plugged in over regular Ethernet can get to the internet, and if I unplug my computer from the Infiniband switch and plug it in over Ethernet then my computer can get on the internet. So I am not really sure where the problem is, but I am hoping someone on this forum is knowledgeable enough in networking that they can steer me in the right direction and help me get this configured so the Infiniband connected PCs can get on the internet.
So right now my network looks like this:
I just got the Voltaire/Mellanox 4036E infiniband switch yesterday, and have moved all my computers and server over to it. I was able to SSH into the switch via PuTTY and configure it. I have full network connectivity right now, I can be on my computer and talk to the router just fine and change settings around and all that. However, I cannot get internet on any of the computers that are connected via Infiniband. They have network access to the router, but no internet access. So that is problem #1.
The way the 4036E switch works is that is has thirty four 40gb ports, and two 10GbE SFP+ ports. There is a built in subnet manager that translates the Infiniband traffic to the Ethernet interfaces and vise versa to allow Infiniband clients to talk to the rest of the network. This means there is no need for a separate gateway to bridge the traffic between network types. I know the subnet manager is working, because otherwise I would not be able to talk to the router from my computer right now. I am just using the default minhop subnet manager.
The router IP is set to 192.168.1.1, I set the Infiniband switch to 192.168.1.2. It turns out this is just setting the management interface's Ethernet port to that address. When I plugged the 10GbE SFP+ port from the infiniband switch into my netgear switch, the SFP+ port seems to have assigned itself 192.168.1.3. I assume it must choose the next sequential number after what the management interface is set to, as I did not assign it .3 myself nor is that within the range set for the DHCP server on the router. Either way, thats what it got and the router sees that address and I can talk to the router just fine.
When I plug the cable from my Infiniband HSA (same as an Ethernet NIC for all intents and purposes) to the switch, I am given a proper DHCP address. In this case I was assigned 192.168.1.163. I can also manually assign myself an ipv4 address in the adapter just like any regular ethernet adapter in Windows.
So as far as I can tell, the network seems to be functioning ok. I can ping every device on the network from my computer that is currently on Infiniband, and yet I still cant get internet access on it. All computers that are plugged in over regular Ethernet can get to the internet, and if I unplug my computer from the Infiniband switch and plug it in over Ethernet then my computer can get on the internet. So I am not really sure where the problem is, but I am hoping someone on this forum is knowledgeable enough in networking that they can steer me in the right direction and help me get this configured so the Infiniband connected PCs can get on the internet.
