Quantcast
Channel: Mellanox Interconnect Community: Message List
Viewing all 6226 articles
Browse latest View live

Re: MLNX_OFED_LINUX installtion Error: Error: The current MLNX_OFED_LINUX is intended for rhel6.4

$
0
0

Can you check this dir?

 

# ls -l /dev/infiniband/

total 0

crw-rw---- 1 root root 231,  64 Jan  6 18:57 issm0

crw-rw---- 1 root root 231,  65 Jan  6 18:57 issm1

crw-rw-rw- 1 root root  10,  56 Dec 28 00:17 rdma_cm

crw-rw-rw- 1 root root 231, 224 Dec 28 00:17 ucm0

crw-rw-rw- 1 root root 231,   0 Jan  6 18:57 umad0

crw-rw-rw- 1 root root 231,   1 Jan  6 18:57 umad1

crw-rw-rw- 1 root root 231, 192 Dec 28 00:17 uverbs0

 

Those are created by udev and the rules should look like this.

If those rules are missing or have wrong permissions, that might be the cause.

 

# cat /etc/udev/rules.d/90-ib.rules

KERNEL=="umad*", NAME="infiniband/%k" MODE="0666"

KERNEL=="issm*", NAME="infiniband/%k"

KERNEL=="ucm*", NAME="infiniband/%k", MODE="0666"

KERNEL=="uverbs*", NAME="infiniband/%k", MODE="0666"

KERNEL=="ucma", NAME="infiniband/%k", MODE="0666"

KERNEL=="rdma_cm", NAME="infiniband/%k", MODE="0666"


Re: MLNX_OFED_LINUX installtion Error: Error: The current MLNX_OFED_LINUX is intended for rhel6.4

$
0
0

Hi here is the output:

 

# ls -l /dev/infiniband/
total 0
crw-rw-rw-. 1 root root 10, 56 Jan  1 10:33 rdma_cm

 

and this:

 

# cat /etc/udev/rules.d/90-ib.rules
KERNEL=="umad*", NAME="infiniband/%k" MODE="0666"
KERNEL=="issm*", NAME="infiniband/%k"
KERNEL=="ucm*", NAME="infiniband/%k", MODE="0666"
KERNEL=="uverbs*", NAME="infiniband/%k", MODE="0666"
KERNEL=="ucma", NAME="infiniband/%k", MODE="0666"
KERNEL=="rdma_cm", NAME="infiniband/%k", MODE="0666"

 

 

What to do next ?

Re: MLNX_OFED_LINUX installtion Error: Error: The current MLNX_OFED_LINUX is intended for rhel6.4

$
0
0

Stop openibd

Turn debug on for udev.

 

udevadm control --log-priority debug

udevadm control --reload-rules

 

Start openibd

 

And search for 90-ib.rules in /var/log/messages

These are logs when it works, but you should see errors that can give use some clues

 

Jan  6 23:08:40 my-host udevd-work[11143]: MODE 0666 /etc/udev/rules.d/90-ib.rules:3

Jan  6 23:08:40 my-host udevd-work[11143]: NAME 'infiniband/ucm0' /etc/udev/rules.d/90-ib.rules:3

Jan  6 23:08:40 my-host udevd-work[11143]: create db link (infiniband/ucm0 char/231:224)

Jan  6 23:08:40 my-host udevd-work[11143]: creating device node '/dev/infiniband/ucm0', devnum=231:224, mode=0666, uid=0, gid=0

Jan  6 23:08:40 my-host udevd-work[11143]: preserve file '/dev/infiniband/ucm0', because it has correct dev_t

Jan  6 23:08:40 my-host udevd-work[11143]: preserve permissions /dev/infiniband/ucm0, 020666, uid=0, gid=0

Jan  6 23:08:40 my-host udevd-work[11143]: creating symlink '/dev/char/231:224' to '../infiniband/ucm0'

Re: ports active but LEDs not lighting up

$
0
0

Thanks for your advice. It's mechanical problem. The light guiding pipes that suppose to transmits light from LED on a switch mainboard to front panel indicators had been detached from the enclosure (perhaps due to bad shipment handling).

Re: Upgrading 4036, media full

$
0
0

IB.PNG.pngHi inbusiness,

 

As the text you just quoted, we have image 2.1 and try to upgrade to 3.0. I'm aware of the above steps that you outline, but since the available space is inadequate (above image) for where it inflates the tar file, this is where I get the error message from the (first image). Obviously since it cant inflate the packages, it is a corrupt file. You can see why this is an issue, as the image we upgrade to is 97M in size and the presumed disk location only is 100M. Inflation then isn't possible. Had it used the 256M partition, it would have been flying. Update firmware Chassis, gives me no control over the location, to place the files. It has its own logic.

 

Magnus

Re: Upgrading 4036, media full

$
0
0

4036's U-Boot systex is very unique.

 

If you want set up some parameter don't use "="

 

If you setup server ip you must type below.

 

server ip 192.168.0.1

(NOT server ip = 192.168.0.1)


If you use "=" that mean delete that parameter.

That's all.

 

And may I have a question?

This is my firmware upgrade test screenshot.

upgrade_tgz.JPG.jpg

Why your upgrade file name is upgrade.tgz?

Normally upgrade_all.tgz has all firmware and software.

Re: Upgrading 4036, media full

$
0
0

In what order?

 

 

 

>tftpboot server ip 10.0.21.15 ? Don’t work… can you check your syntax?

 

 

 

tftpboot server ip 10.0.21.15 comes close, but dont start transfer but gives this output: tftpboot- boot image via network using TFTP protocol… then what?

Does VMA support SR-IOV?

$
0
0

Hi,

 

I am trying VMA in a Linux kvm Virtual Machine, with SR-IOV, MLNX_OFED_LINUX-2.0-3.0.0 and fw 2.30.3000

I have log_num_mgm_entry_size=-1 in modprobe but it doesn't seem to matter, because the value in /sys/module/mlx4_core/parameters/log_num_mgm_entry_size is always 10 (the default one). In hypervisor the /sys param is -1.

 

Also, what does the Error about QP mean?

UDP seems to be working but vma_stats does not show any offloaded packets.

TCP doesn't work at all, the client fails to connect to the server.

 

VMA INFO   : VMA_VERSION: 6.4.11-0 Release built on 2013-09-08-09:51:56

VMA INFO   : Cmd Line: sockperf server --load-vma -i 192.168.1.58

VMA INFO   : Log Level                      3                          [VMA_TRACELEVEL]

VMA INFO   : MTU                            2044                       [VMA_MTU]

VMA INFO   : ---------------------------------------------------------------------------

VMA WARNING: ***********************************************************************************************

VMA WARNING: * VMA will not operate properly while flow steering option is disabled!                       *

VMA WARNING: * Please restart your VMA applications after running the following:                           *

VMA WARNING: * WARNING: the following steps will restart your network interface!                           *

VMA WARNING: * 1. "echo options mlx4_core log_num_mgm_entry_size=-1 > /etc/modprobe.d/mlx4_core.conf"    *

VMA WARNING: * 2. "/etc/init.d/openibd restart"                                                          *

VMA WARNING: * Read more about the Flow Steering support in the VMA's User Manual                          *

VMA WARNING: ***********************************************************************************************

VMA WARNING: ndv94:configure() Mismatch between interface mlx0 MTU=1500 and VMA_MTU=2044. Make sure VMA_MTU and all offloaded interfaces MTUs match.

sockperf: == version #2.5.231 ==

VMA ERROR  : rfs[0x26fb470]:186:create_ibv_flow() Create of QP flow ID failed with flow dst:192.168.1.58:11111, src:0.0.0.0:0, protocol:UDP

 

BTW, on bare metal VMA works quite nice.

 

Thanks,

sorin


Re: Upgrading 4036, media full

$
0
0

Hi!

1. You must configure 4036's ip address, subnet mask, gateway.

2. You must configure tftp server's ip address, too.

 

then...

 

If you in U-Boot console, you can find tftp boot command by help+enter

 

Before boot from tftp, you must check current environment, too!

Re: XenServer support

$
0
0

Hello Fiat,

following your word, I have tried to compile OFED v 1.5.4 into xenserver environment, but:

 

- ./mlnx_add_kernel_support.sh stop with error: "kernel-ib was not created", but kernel_ib and kernel_ib_devel rpm are created and I found these into subdir /root-of-mlx-tmp-work/MLNX_OFED_SRC-1.5.3-4.0.42/RPMS/centos-release-5-7.el5.centos/i686/

 

At the end of my test, I have get compiled package "kernel-ib" and "ofed-script" and I have installed these into xenserver.

During boot somes modules aren't loaded, error displayed on openibd startup are:

Loading Mellanox MLX4 HCA driver:                      [FAILED]
Loading Mellanox MLX4_EN HCA driver:                   [FAILED]
Loading HCA driver and Access Layer:                   [FAILED]

 

 

BUT, if I try "service network restart", system UP my ib0 device.

 

Now, I have try with OFED 1.5.4 for centos 5.7.

But, is correct?

Which version ofofedshould be used to have "compiles fine"?


Many thanks,

 

Enrico

Re: XenServer support

$
0
0

It looks like you are trying to install OFED 1.5.3, not 1.5.4

 

If you are using connectx2/3 cards you can simply install OFED 2.0

 

If you are using the older 10Gb cards, you'll need to install 1.5.4. (1.5.3 may work, but I have not tried to get this working). There are some conflicts you will need to resolve first in the ofa_kernel tree.

 

 

Remove line 9 (definition of 'IS_ERR_OR_NULL') from:

ofa_kernel-1.5.4.1/kernel_addons/backport/2.6.32/include/linux/err.h

 

Remove 'netif_set_real_num_tx_queues' function definition from:

ofa_kernel-1.5.4.1/kernel_addons/backport/2.6.32/include/linux/netdevice.h

 

simply run ./install.pl --all and the RPM's will be built.

 

Dave

Re: XenServer support

$
0
0

Dave, many thanks for your answer!

 

Sorry, but I'm very confused with OFED version.

 

I have downloaded from mellanox website a last available 1.5 and it are a 1.5.3

 

1.5.4 are unavailable.

 

For version 2.0, only available are under Mellanox website, in openfabrics.org are only 1.5 and 3.5!

 

Now, how I can get good version?

 

- 1.5.4 from openfabrics.org

- 2.0 from mellanox.com

 

It's correct?

Re: XenServer support

Re: el5.10 ofed build problem.

$
0
0

I'm having the same problem. Did you fix it yet?

 

Thanks,

Koji

Re: mlnx-ofed-kernel installation failed!

$
0
0

Hi Andre,

 

quick question, if im trying to use GPUDirect RMDA between two Nvidia Kepler K20c cards in two different machine, which infiniband driver should i use? Currently we are have ConnectX-3 FDR infiniband 40gigE in our machine.

 

thanks.


multiport IB card pros

$
0
0

Hi,

 

I am wondering if I should invest in single port or 2 port adapters.

What are advantages of multiport IB cards?

Do they offer something similar to any form of Ethernet Link Aggregation?

Can you point me to a doc describing it?

 

In other words, what are advatages of having

Storage with 2 or 4 IB port card ===== IB switch ===== multiple clients with single port HCA

vs.

Storage with 2 or 4 IB port card ===== IB switch ===== multiple clients with 2-port HCA

vs.

Storage with 1 IB port card ------ IB switch ===== multiple clients with single port HCA

Re: Upgrading 4036, media full

$
0
0

Hi inbusiness,

 

 

 

The syntax was wrong and I never figured it out. I have a Master degree in networking, but thanks for your advice in having IP addresses on both ends  of the layer 2 link….

 

 

 

I solved the problem with a workaround in Linux. Only from 2.1 it was a problem. Rest was just the normal procedure.

 

 

 

Thank you for all your effort.

 

 

 

Magnus

Re: mlnx-ofed-kernel installation failed!

Re: multiport IB card pros

$
0
0

It gives you High Availability.

For storage for example, with iser on the client (initiator) side if you create 2 iser interfaces, one on ib0 and one on ib1:

iscsiadm --mode iface

default tcp,<empty>,<empty>,<empty>,<empty>

iser iser,<empty>,<empty>,<empty>,<empty>

iser-0 iser,<empty>,<empty>,ib0,<empty>

iser-1 iser,<empty>,<empty>,ib1,<empty>

 

and then login to the same target over each interface

scsiadm -m node -p 192.168.1.10 -T iser-target --login -I iser-0

scsiadm -m node -p 192.168.1.10 -T iser-target --login -I iser-1

 

you will get two block devices, e.g. /dev/sdf and /dev/sdg. lsscsi should list them.

 

Then you put them in multipath and you get /dev/mapper/mpathN devices which you can fdisk/mkfs or create logical volumes like on any other block devices. multipath will route IO over iser-0 and iser-1. The default algorithm is round-robin.

 

On the server (target) side 2 interfaces will also give you HA. But it's also important that the target side network does not become a bottleneck. If you have 2x40Gb ports in the target server and 80 clients then each client will be limited to 1Gbps if they all do IO simultaneously. So you should go for 4 or more ports in the target.

Are ConnectX-3 VPI adapters compatible with the Red Hat MRG (i.e. real-time patch)?

$
0
0

I would like to use the ConnectX-3 VPI adapters on a Scientific Linux system with the real-time patch instaled.  This is functionally equivalent to a RHEL system with Red Hat MRG.  Has anybody successfully used these adapters on a Linux system with the real-time patch installed?

 

Thanks,

Malcolm Smith

Viewing all 6226 articles
Browse latest View live