Hi everyone, glad to be on this community.
I have two Mellanox M2401G Infiniband switches, which are the Dell blade switches for M1000e.
They seem to work, but I have some issues here and maybe someone has a few seconds to look and guide me in the right direction...
Switches are connected to fabric C, and the servers have a dual port ConnectX Infiniband card (MT25418 based, 0HX271, DEL08C0000001)
Here is the card info:
[root@dev1 ~]# ibstat
CA 'mlx4_0'
CA type: MT25418
Number of ports: 2
Firmware version: 2.7.0
Hardware version: a0
Node GUID: 0x00188b9097fe45f9
System image GUID: 0x00188b9097fe45fc
Port 1:
State: Active
Physical state: LinkUp
Rate: 20
Base lid: 1
LMC: 0
SM lid: 1
Capability mask: 0x0251086a
Port GUID: 0x00188b9097fe45fa
Link layer: InfiniBand
Port 2:
State: Active
Physical state: LinkUp
Rate: 20
Base lid: 3
LMC: 0
SM lid: 3
Capability mask: 0x0251086a
Port GUID: 0x00188b9097fe45fb
Link layer: InfiniBand
[root@dev1 ~]#
Now, since it's a dual port card I enabled opensm for both guids
14801 ? Sl 0:02 /usr/sbin/opensm -B -g 0x00188b9097fe45fa --subnet_prefix 0xfe80000000000000 14821 ? Sl 0:02 /usr/sbin/opensm -B -g 0x00188b9097fe45fb --subnet_prefix 0xfe80000000000001
Seems to work, however, I an only see one switch, never both, although 1st switch is connected to port1 and second switch to port2.
The switch definitely exists and if I power off the 1st one I can see the second one.
[root@dev1 ~]# mst start
Starting MST (Mellanox Software Tools) driver set
Loading MST PCI module - Success
Loading MST PCI configuration module - Success
Create devices
unable to initialize libusb: -99
[root@dev1 ~]# mst ib add
-I- Discovering the fabric - Running: ibnetdiscover
-I- Added 2 in-band devices
[root@dev1 ~]# mst status
MST modules:
------------
MST PCI module loaded
MST PCI configuration module loaded
MST devices:
------------
/dev/mst/mt25418_pciconf0 - PCI configuration cycles access.
bus:dev.fn=13:00.0 addr.reg=88 data.reg=92
Chip revision is: A0
/dev/mst/mt25418_pci_cr0 - PCI direct access.
bus:dev.fn=13:00.0 bar=0xfd200000 size=0x100000
Chip revision is: A0
Inband devices:
-------------------
/dev/mst/CA_MT25418_MT25408_ConnectX_Mellanox_Technologies_lid-0x0001
/dev/mst/SW_MT47396_0x2c90200410f0a_lid-0x0002
How can I see both switches to create a failover scenario? am I missing something here?
Thank you