There shouldn't be an issue communicating with a ConnectX3 VPI.
If your ConnectX3 VPI CX4 design supports switching between Infiniband and Ethernet (as the VPI label implies) - there shouldn't be an issue caused by the medium (CX4 to QSFP), when connecting to a MCX354A-FCBT.
Please note - You need to configure both link partners to work in the same protocol (Eth or IB).
Regards.