Both ports showing Down/Polling.
you need to check your physical wires/connections. you might using the wrong combinations of HCA/Cable/Switch.
what do you have there for cables?
try the following:
1) run opensm on your switch and take a cable - loopback between two ports, see if the link comes up (if not, then the cable is no good)
2) run opensm daemon on the server and loopback between the HCA port 1 and 2 - same as the above.