Hello Michael,
From what I see the configuration you are describing should be workable. When you described your issue you said that the connections became confused and stalled at times. I feel you were implying below this that the issue came from using both InfiniBand and Ethernet on one card.
Could you elaborate on this error?
Is there a specific output you are seeing?
What is the traffic like on these ports during this error condition?
Is it when both cards are putting traffic on egress on both ports, or ingress, or a mix?
I noticed you are using the RHEL6.3 Community OFED, have you tried your success with our driver?
Could you provide the ibv_devinfo output for your machines? I would like to see the PSID of your cards.