Found a similar comment on Google, but the recommended ethtool -K <ethx> gro off.
That did not work, so went through and turned off all offloading in the driver, used ethtool with the following
tso, gro, gso, and lro
With all of these I can use the 10G interface in Virtualbox, have not tried yet to determine exactly which one is the culprit or if I need them all.