You are correct. The current HPC-X does not build with thread support.
However, HPC-X does come with the source and instructions to build, so it can be compiled with the thread support separately if the application does require to use thread support from the HPC-X MPI library.
Hi Catalin,
can you attach a screenshot or the full switch console log?
Thanks,
I hadn't done my homework. I thought the tar ball in HPC-X was the regular OpenMPI only, but now that I looked I realize that the yalla driver is included.
Thanks for pointing it out.
/Nils
Checking Image at f4000000 ...
Bad trap at PC: 7ff7e7fc, SR: 29000, vector=1300
NIP: 7FF7E7FC XER: 00000000 LR: 7FF6C6A0 REGS: 7fe55c78 TRAP: 1300 DEAR: F4000000
MSR: 00029000 EE: 1 PR: 0 FP: 0 ME: 1 IR/DR: 00
GPR00: 7FF6C6A0 7FE55D68 7FE55F24 F4000000 00000000 00000017 00000017 30F46FC9
GPR08: 7FF5E40C 27051956 04F790D6 3B9ACA0C 0000000F 0001C4EC 7FFAF800 00000000
GPR16: 7FFA3DD4 7FFABEEC 00000000 00000000 00000000 00000000 00000000 00000001
GPR24: 00000000 7FE5B688 00000000 00000000 7FE5B738 00000002 7FFAFF70 F4000000
Call backtrace:
00000002 7FF6C6A0 7FF6CA7C 7FF7E30C 7FF7DA10 7FF7DB80 7FF806B0
7FF5DF68 7FF5C710
Exception
U-Boot 2009.01 SX_PPC_M460EX SX_3.2.0330-82-EMC ppc (Feb 27 2013 - 12:13:42)
CPU: AMCC PowerPC 460EX Rev. B at 1000 MHz (PLB=166, OPB=83, EBC=83 MHz)
Security/Kasumi support
Bootstrap Option H - Boot ROM Location I2C (Addr 0x52)
Internal PCI arbiter disabled
32 kB I-Cache 32 kB D-Cache
Board: Mellanox PPC460EX Board
FDEF: No
I2C: ready
DRAM: 2 GB (ECC enabled, 333 MHz, CL3)
FLASH: 16 MB
NAND: 1024 MiB
PCI: Bus Dev VenId DevId Class Int
PCIE0: link is not up.
PCIE1: successfully set as root-complex
01 00 15b3 c738 0c06 00
Net: ppc_4xx_eth0, ppc_4xx_eth1
Catalin Bostan
mobile:+40788232025
IT consultant
PRO SYS -bussines hardware solutions
VMware VCP4/VCP5;
Microsoft MCITP-VA;
Microsoft MCITP-EA;
SNIA-SCSP;
SNIA/CompTIA-Storage+;
Intel Technology Provider Expert 2011/2012/2013/2014
Thanks Catalin,
I think we need to replace the switch.
Please open an RMA request at:
I've got a couple of ConnectX-2 and ConnectX VPI cards, and the QSFP to CX4 cables I purchased seem to not understand that the other side is 10Gigabit ethernet. What cabling would I need to make those ports understand that they're connected to a 10GigE switch?
Current: (HCA) QSFP -> CX4 (passive) CX4 -> XFP (Force10 E600)
I've attempted the above with a variety of different card driver settings (on linux) and none of them work (inbox and mlnx_ofed) - ibstatus always reports 'state: 1: DOWN' and 'phys state: 3: Disabled'. I attempted to email support@mellanox, but since I don't have an active service contract they referred me here.
Does anybody have a working ConnectX-2 VPI to XFP 10GbE setup? If so, what cables/adapters did you use?
TIA
Hi, I have a cluster with two nodes with the name mic01 and mic03, which are installed Redhat 6.3, and mic01 mounted from mic03. I install the MLNX_OFED driver MLNX_OFED_LINUX-2.4-1.0.4-rhel6.3-x86_64.iso in mic03 with the command from user manual:
mount -o ro,loop MLNX_OFED_LINUX-2.4-1.0.4-rhel6.3-x86_64.iso /mnt
and then ./mlnxofedinstall in /mnt. The driver is successfully installed in mic03(the ib status is active).
However when I install MLNX_OFED in mic01 with the same method above, it shows the information after ./mlnxofedinstall:
Starting MLNX_OFED_LINUX-2.4-1.0.4 installation ...
Installing mlnx-ofa_kernel RPM
Preparing... ##################################################
mlnx-ofa_kernel ##################################################
Installing kmod-mlnx-ofa_kernel 2.4 RPM
Preparing... ##################################################
kmod-mlnx-ofa_kernel ##################################################
Installing mlnx-ofa_kernel-devel RPM
Preparing... ##################################################
mlnx-ofa_kernel-devel ##################################################
Installing kmod-kernel-mft-mlnx 3.8.0 RPM
Preparing... ##################################################
kmod-kernel-mft-mlnx ##################################################
Installing knem-mlnx RPM
Failed to install knem-mlnx 1.1.1.90mlnx RPM
Collecting debug info...
See /tmp/MLNX_OFED_LINUX-2.4-1.0.4.13023.logs/knem-mlnx-1.1.1.90mlnx.rpminstall.err.log
And it failed to install knem-mlnx 1.1.1.90mlnx RPM. In the log, it shows the error:
error: unpacking of archive failed on file /opt/knem-1.1.1.90mlnx: cpio: chown failed - Inappropriate ioctl for device
Where did I have a mistake? Can anyone help me with the problem?
I will appreciate it if you help me with this Thank you!
Hi,
I have just now plugged in a SFP into a new test setup (connectX-3 card) and plugged the other end of the fibre cable into a Cisco switch. Here's what the setup looks like right now by way of the self test -
# hca_self_test.ofed
---- Performing Adapter Device Self Test ----
Number of CAs Detected ................. 1
PCI Device Check ....................... PASS
Kernel Arch ............................ x86_64
Host Driver Version .................... MLNX_OFED_LINUX-2.3-2.0.0 (OFED-2.3-2.0.0): 2.6.32-504.el6.x86_64
Host Driver RPM Check .................. PASS
Firmware on CA #0 NIC .................. v2.32.5100
Firmware Check on CA #0 (NIC) .......... PASS
Host Driver Initialization ............. PASS
Number of CA Ports Active .............. 1
Port State of Port #1 on CA #0 (NIC)..... DOWN (Ethernet)
Port State of Port #2 on CA #0 (NIC)..... UP 1X QDR (Ethernet)
Error Counter Check on CA #0 (NIC)...... NA (Eth ports)
Kernel Syslog Check .................... PASS
Node GUID on CA #0 (NIC) ............... 00:02:c9:03:00:3c:98:40
------------------ DONE ---------------------
At this point you probably think - cool it works. Port one on the card does not.
If I move the SFP to port one (no cable plugged in) I immediately get a flashing green light\LED. So I looked up the manual. It states:
Green - physical link
• Constant on indicates a good physical link
• If neither LED is lit, then the physical link has not been established
mmmmmm
So what does a flashing green LED mean?
Our adapter card is based on the MT23108 InfiniHost device. It appears that the last release to support this was Mellanox OFED 1.5.2-2.1. Unfortunately, the MLNX_OFED Download Center archive only goes back to 1.5.3-3.0.0, which does not support this device. Is it possible to get access to a 1.5.2-2.1 iso download that we can use for installation? Thanks again for the help.
Is there no problem when you make a host profile with iSER?
Any add-ons or other upgrades that add PPPoE support? If not from Mellanox maybe 3th party?
Thank you for your input!
Anyone else up for some of the other questions ?
Hope so!
Not able to create host profile with iSER enabled. Error: Unknown.
Thank you for your reply.
Let's wait patch for this problems...:)
I found in an older connectX-2 pdf this statement.
Blinking indicates a problem with the Physical link
Which is odd for me as it does this with the SFP plugged in only no cable connected. This is different from the behaviour of the other port. When I move the SFP with no cable plugged in into it there is no light at all, and of course it is find with the cable plugged in.
The other port blinks with or without cable - the exact same cable that works in the other port........
Hi All,
I had a look around the forum and looks like one of the question is very close to what I'm think about.
I'm looking to add a few Nutanix Nodes to my network, which connect via 10Gbe SFP+, currently I don't have any 10GbE infrastructure.
So I was thinking of purchasing an SX1012 Switch.
With two of the break out 40GbE Connectors, that connect into the Nutanix nodes.
But my question is, How do I get the SX1012 to connect to my Dell PowerConnect 2824? The 2824 has 2 x 1GbE SFP connection and 24 1Gb RJ45.
Could I use on of the
| MPN: MC3208411-T | Mellanox MC3208411-T Module ETH 1GbE 1Gb/s SFP Base-T Up To 100m |
and connect to the 2824 using RJ45?
Thanks
Chris
Hi Chris,
I think you can use the below:
SX1012 - QSA - MC3208411-T - twister pair cable -> 2824
you would need to lower the port speed on the SX1012 to 1000 Mbps and your good to go.
The QSA ( QSFP to SFP adapater) PN is MAM1Q00A-QSA
Hi eddie,
Thanks for the response, just to clarify, Is
The QSA ( QSFP to SFP adapater) PN is MAM1Q00A-QSA
Actually the QSFP+ to SFP+
Thanks
Chris
Yes
Great, Thanks Eddie
Chris