This is a page tracks issues that we run into while testing the Multi-Rail Feature

Problem List

• with 17 interfaces trying to discover on the 17th interface returns an error "no route to host". This is an intermittent issue. Attempting to reproduce
• With 17 interface discovered "show peer" hangs
  • When the rc from the kernel is not 0. The structure is not copied out of the kernel to user space. The code depends on that in order to pass the new size if the data to be copied out is too big for the buffer passed in by the user. Since that doesn't happen when rc == -E2BIG, user space code gets into an infinite loop sending IOCTLs to the kernel

  • from libcfs_ioctl()   145 »·······»·······»·······if (err == 0) { 146 »·······»·······»·······»·······if (copy_to_user(uparam, hdr, hdr->ioc_len)) 147 »·······»·······»·······»·······»·······err = -EFAULT; 148 »·······»·······»·······} 149 »·······»·······»·······break; The buffer is only copied to user space if the ioctl handler returns 0. Not really sure if it's safe to change that.

• "lnetctl discover" command hangs with discovery off. This happened once, so an intermittent issue. Will try to reproduce.
• "lnetctl discover" discovers the peer even with discovery off.

• Doug: I configured a Parallels VM with 16 interfaces (won't let me do 17 as 16 is a limit).  When I "lctl network configure" with no YAML or module parameters, I get this error from ksocklnd: "Mar 7 14:01:16 centos-7 kernel: LNet: 5111:0:(socklnd.c:2652:ksocknal_enumerate_interfaces()) Ignoring interface virbr0 (too many interfaces)".

• Doug: When discovering a node with 16 interfaces via "lnetctl discover --nid", it works, but I am seeing this log: 

  Mar 7 15:48:04 centos-7 kernel: LNetError: 24769:0:(peer.c:1726:lnet_peer_push_event()) Push Put from unknown 0@<0:0> (source 0@<0:0>)

• Doug: Tried the command "lnetctl set discovery" (no args) and got a core dump.
• Doug: How do I query the discovery setting?  There is no "lnetctl show discovery" (should there be?).  I tried "lnetctl set discovery" with no parameters and that core dumped (see previous bullet).  From a usability perspective, there is no obvious way to get this information.

• Doug: When I enter "lnetctl set" to see what options I can set, I get this: 

  [root@centos-7 ~]# lnetctl set set {tiny_buffers | small_buffers | large_buffers | routing}

  It does not mention "discovery" at all.

Usability Issues

• Doug: How do I query the discovery setting?  There is no "lnetctl show discovery" (should there be?).  I tried "lnetctl set discovery" with no parameters and that core dumped (see previous section).  From a usability perspective, there is no obvious way to get this information.
• Doug: I understand that Chris Morrone pushed for us to have "lnetctl set <key> <value>".  That breaks from the original paradigm to follow which was done with the Linux "ip" command.  It was designed to be: "ip <object> <action> <optional params>".  So, it would be more logical to have: "lnetctl discover set 0/1" than "lnetctl set discovery 0/1".  Then starting discovery can be: "lnetctl discover start <nids>".  Looking for discovery status can be: "lnetctl discovery show".  I found myself guessing at these commands as I have given here and had no idea to look at "set".