Overclock.net › Forums › Specialty Builds › Servers › infiniband IPoIB problem
New Posts  All Forums:Forum Nav:

infiniband IPoIB problem

post #1 of 5
Thread Starter 
(posted this in the network subforum but i know people here are so good in explaining hard stuff in a simple way biggrin.gif )


I always wanted to test out infiniband, so I waited until a could find some cheap adapter on ebay. It took me couple of month to get them, but i have finally have them now.
My first problem was that the open fabric driver werent update for server 2012/win8. On their forum i found a couple of people that modified them to make them work, so i tested those driver and they worked.

my cards:

Cisco Infiniband HCA

20GB/S
0MB On-Board Memory
Flash Size: 2MB
RoHS: Yes
PCI-Express 1.0a, x8
DDR
Power: 4W
Tall Bracket
MT50N21B02 SFS-HCA-320-A1

Both my card are detected and running. I have a cx4 cable connecting them. My problem is when i try to ping from one side to the other, it doesnt work.

I have both card in connected mode. Only one of the two has the OpenSm service running. Now i am little bit confused after reading a lot of guide online. I set up a static ip for both my card that werent in the same range than my normal internet network. I have also put them on another subnet mask

internet network : 192.168.1.1....
infiniband : 11.4.3.204 for one and 11.4.3.205 for the other

subnet infiniband: 255.255.0.0
internet subnet : 255.255.255.0

I don't know if any of you have ever tested infiniband but i know some of you have datacenters at home lol so I hope someone can help me with this thumb.gif
My baby
(16 items)
 
home server
(7 items)
 
 
CPUMotherboardGraphicsHard Drive
i7 3770k DZ77GAL-70K evga gtx 970 Crucial m4 
Hard DriveHard DriveHard DriveCooling
W.Green seagates adata  ex360 xspc 
CoolingCoolingOSMonitor
RayStorm CPU WaterBlock 970  Win 8 Asus vh222h-p 
KeyboardPowerCaseMouse
Alienware Tx750 corsair 800d Naga 
CPUMotherboardRAMHard Drive
e3-1230v2 Gigabyte B75M-D3H Patriot Intel Extreme Masters PVI316G160C9K Seagate Barracuda 2TB 7200RPM 
OSPowerCase
windows server 2012 datacenter Corsair CX Series CX500 Fractal Design Define R4 
  hide details  
Reply
My baby
(16 items)
 
home server
(7 items)
 
 
CPUMotherboardGraphicsHard Drive
i7 3770k DZ77GAL-70K evga gtx 970 Crucial m4 
Hard DriveHard DriveHard DriveCooling
W.Green seagates adata  ex360 xspc 
CoolingCoolingOSMonitor
RayStorm CPU WaterBlock 970  Win 8 Asus vh222h-p 
KeyboardPowerCaseMouse
Alienware Tx750 corsair 800d Naga 
CPUMotherboardRAMHard Drive
e3-1230v2 Gigabyte B75M-D3H Patriot Intel Extreme Masters PVI316G160C9K Seagate Barracuda 2TB 7200RPM 
OSPowerCase
windows server 2012 datacenter Corsair CX Series CX500 Fractal Design Define R4 
  hide details  
Reply
post #2 of 5
@OP

I can't help you directly, but you could also try asking the same question over on [H]ardforum, in the "SSDs & Data Storage" section. A number of people (e.g. packetboy) over there have deployed Infiniband in production systems and have had success. smile.gif
Mythica
(14 items)
 
  
CPUMotherboardGraphicsRAM
Intel i3 530 Gigabyte GA-H55M-D2H Palit nVidia GT430 Corsair Dominator 4GB TW3X4G1333C9A 
Hard DriveHard DriveOSMonitor
Western Digital Scorpio Black Samsung HD204UI Linux Mint 17 HP L1800 
KeyboardPowerCaseMouse
Logitech Basic Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
Mythica
(14 items)
 
  
CPUMotherboardGraphicsRAM
Intel i3 530 Gigabyte GA-H55M-D2H Palit nVidia GT430 Corsair Dominator 4GB TW3X4G1333C9A 
Hard DriveHard DriveOSMonitor
Western Digital Scorpio Black Samsung HD204UI Linux Mint 17 HP L1800 
KeyboardPowerCaseMouse
Logitech Basic Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
post #3 of 5
Thread Starter 
ok thanks a lot
My baby
(16 items)
 
home server
(7 items)
 
 
CPUMotherboardGraphicsHard Drive
i7 3770k DZ77GAL-70K evga gtx 970 Crucial m4 
Hard DriveHard DriveHard DriveCooling
W.Green seagates adata  ex360 xspc 
CoolingCoolingOSMonitor
RayStorm CPU WaterBlock 970  Win 8 Asus vh222h-p 
KeyboardPowerCaseMouse
Alienware Tx750 corsair 800d Naga 
CPUMotherboardRAMHard Drive
e3-1230v2 Gigabyte B75M-D3H Patriot Intel Extreme Masters PVI316G160C9K Seagate Barracuda 2TB 7200RPM 
OSPowerCase
windows server 2012 datacenter Corsair CX Series CX500 Fractal Design Define R4 
  hide details  
Reply
My baby
(16 items)
 
home server
(7 items)
 
 
CPUMotherboardGraphicsHard Drive
i7 3770k DZ77GAL-70K evga gtx 970 Crucial m4 
Hard DriveHard DriveHard DriveCooling
W.Green seagates adata  ex360 xspc 
CoolingCoolingOSMonitor
RayStorm CPU WaterBlock 970  Win 8 Asus vh222h-p 
KeyboardPowerCaseMouse
Alienware Tx750 corsair 800d Naga 
CPUMotherboardRAMHard Drive
e3-1230v2 Gigabyte B75M-D3H Patriot Intel Extreme Masters PVI316G160C9K Seagate Barracuda 2TB 7200RPM 
OSPowerCase
windows server 2012 datacenter Corsair CX Series CX500 Fractal Design Define R4 
  hide details  
Reply
post #4 of 5
Thread Starter 
Ok after reading a couple of posts on that forum (thank again parityboy), i have everything working.

Just some simple network transfer speed from ssd to a raid 10 server

Gigabit network


vs

Infiniband network
My baby
(16 items)
 
home server
(7 items)
 
 
CPUMotherboardGraphicsHard Drive
i7 3770k DZ77GAL-70K evga gtx 970 Crucial m4 
Hard DriveHard DriveHard DriveCooling
W.Green seagates adata  ex360 xspc 
CoolingCoolingOSMonitor
RayStorm CPU WaterBlock 970  Win 8 Asus vh222h-p 
KeyboardPowerCaseMouse
Alienware Tx750 corsair 800d Naga 
CPUMotherboardRAMHard Drive
e3-1230v2 Gigabyte B75M-D3H Patriot Intel Extreme Masters PVI316G160C9K Seagate Barracuda 2TB 7200RPM 
OSPowerCase
windows server 2012 datacenter Corsair CX Series CX500 Fractal Design Define R4 
  hide details  
Reply
My baby
(16 items)
 
home server
(7 items)
 
 
CPUMotherboardGraphicsHard Drive
i7 3770k DZ77GAL-70K evga gtx 970 Crucial m4 
Hard DriveHard DriveHard DriveCooling
W.Green seagates adata  ex360 xspc 
CoolingCoolingOSMonitor
RayStorm CPU WaterBlock 970  Win 8 Asus vh222h-p 
KeyboardPowerCaseMouse
Alienware Tx750 corsair 800d Naga 
CPUMotherboardRAMHard Drive
e3-1230v2 Gigabyte B75M-D3H Patriot Intel Extreme Masters PVI316G160C9K Seagate Barracuda 2TB 7200RPM 
OSPowerCase
windows server 2012 datacenter Corsair CX Series CX500 Fractal Design Define R4 
  hide details  
Reply
post #5 of 5
@OP

Glad I could help. Those transfer speeds look sweet. biggrin.gif
Mythica
(14 items)
 
  
CPUMotherboardGraphicsRAM
Intel i3 530 Gigabyte GA-H55M-D2H Palit nVidia GT430 Corsair Dominator 4GB TW3X4G1333C9A 
Hard DriveHard DriveOSMonitor
Western Digital Scorpio Black Samsung HD204UI Linux Mint 17 HP L1800 
KeyboardPowerCaseMouse
Logitech Basic Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
Mythica
(14 items)
 
  
CPUMotherboardGraphicsRAM
Intel i3 530 Gigabyte GA-H55M-D2H Palit nVidia GT430 Corsair Dominator 4GB TW3X4G1333C9A 
Hard DriveHard DriveOSMonitor
Western Digital Scorpio Black Samsung HD204UI Linux Mint 17 HP L1800 
KeyboardPowerCaseMouse
Logitech Basic Thermaltake ToughPower 850W Lian-Li PC-A04B Logitech Trackman Wheel 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Servers
Overclock.net › Forums › Specialty Builds › Servers › infiniband IPoIB problem