Situation: Link is not UP on each end, when port speed is 40G Ethernet.
Two SX1012 switches connected by ~2500 meters of single mode fiber. Transcievers are QSFP+ LR4 40GE OPNEXT. Models:
TRQ5B20MN-LF069
TRQ5B20MN-LF100
These transceivers support this ~2500 meter cable length.
Light readings verified and acceptable on both ends of the cable. They are within proper range, not saturating (too hot) and not below sensitivity threshold (too weak).
Switch software is latest version - 3.4.3002.
Switch configurations are proper.
Root cause of link issue: The OPNEXT transceivers aren't working, at 40GE, in these Mellanox switches, with this fiber length.
Fix:
Mellanox recommends using this transceiver:
Module QSFP MC2210511-LR4 Active Optical Module 40GigE+FDR10
10km SM
wavelengths: 1270, 1290, 1310 and 1330nm,
Standard MSA Singlemode Transceiver for 10Gigabit Ethernet and 40Gigabit Ethernet with LC connector.
Use with MetroX MTX6XX0 long haul platforms, SX1012, SX1710, SX1036/1700 ports 1,3,33,35 and SX1024 ports 50,52,54,56,58,60.
Mellanox guaranties this module functionality only in a Mellanox End-to-End solution.
Mellanox doesn't officially support third party LR4 or SR4 modules, though third party transceivers may or may not work in various solutions.
Note: Due to a hardware limitation, SwitchX-2 units do support LR-4 modules, but SwitchX models do not. These switches referenced in this discussion are in fact SwitchX-2 models.
Additionally, LR4 modules will be supported by ConnectX-3 adapters and by the Spectrum family of switches (starting next software release with ETA December 2015).