...
Performance needs to be taken into account with this feature. It is not feasible to traverse the policy lists on every send operation. This will add unnecessary overhead. When rules are applied they have to be "falttenedflattened" to the constructs they impact. For example, a Network Rule is added as follows: o2ib priority 0. This rule gives priority for using o2ib network for sending. A priority field in the network will be added. This will be set to 0 for the o2ib network. As we traverse the networks in the selection algorithm, which is part of the current code, the priority field will be compared. This is a more optimal approach than examining the policies on every network to see if it matches or not.
...
A rule can be uniquely identified using the matching rule or an internal ID which assigned by the LNet module when a rule is added and returned to the user space when they are returned as a result of a show command.
cfg-100, cfg-105, cfg-110, cfg-115, cfg-120, cfg-125, cfg-130, cfg-135, cfg-140, cfg-160, cfg-165
lnetctl Interface
Structures
| Code Block |
|---|
/* This is a common structure which describes an expression */
struct lnet_match_expr {
__u32 lme_start;
__u32 lme_end;
__u32 lme_incr;
char lme_r_expr[0];
};
struct lnet_selection_descriptor {
enum selection_type lsd_type;
char *lsd_pattern1;
char *lsd_pattern2;
union {
__u32 lsda_priority;
} lsd_action_u;
};
/*
* lustre_lnet_add_selection
* Delete the peer NIDs. If all peer NIDs of a peer are deleted
* then the peer is deleted
*
* selection - describes the selection policy rule
* seq_no - sequence number of the command
* err_rc - YAML structure of the resultant return code
*/
int lustre_lnet_add_selection(struct selection_descriptor *selection, int seq_no, struct cYAML **er_rc); |
cfg-100, cfg-105, cfg-110, cfg-115, cfg-120, cfg-125, cfg-130, cfg-135, cfg-140, cfg-160, cfg-165
lnetctl Interface
# Adding a network priority rule. If the NI under the network doesn't have
# an explicit priority set, it'll inherit the network priority:
lnetctl > selection net # Adding a network priority rule. If the NI under the network doesn't have
# an explicit priority set, it'll inherit the network priority:
lnetctl > selection net [add | del | show] -h
Usage: selection net add --net <network name> --priority <priority>
WHERE:
selection net add: add a selection rule based on the network priority
--net: network string (e.g. o2ib or o2ib* or o2ib[1,2])
--priority: Rule priority
Usage: selection net del --net <network name> [--id <rule id>]
WHERE:
selection net del: delete a selection rule given the network patter or the id. If both
are provided they need to match or an error is returned.
--net: network string (e.g. o2ib or o2ib* or o2ib[1,2])
--id: ID assigned to the rule returned by the show command.
Usage: selection net show [--net <network name>]
WHERE:
selection net show: show selection rules and filter on network name if provided.
--net: network string (e.g. o2ib or o2ib* or o2ib[1,2])
# Add a NID priority rule. All NIDs added that match this pattern shall be assigned
# the identified priority. When the selection algorithm runs it shall prefer NIDs with
# higher priority.
lnetctl > selection nid [add | del | show] -h
Usage: selection nid net add --nid <NID> net <network name> --priority <priority>
WHERE:
selection nid net add: add a selection rule based on the nid patternnetwork priority
--nid: nid pattern which follows the same syntax as ip2netnet: network string (e.g. o2ib or o2ib* or o2ib[1,2])
--priority: Rule priority
Usage: selection nid net del --nid <NID> net <network name> [--id <rule id>]
WHERE:
selection nid net del: delete a selection rule given the nid network patter or the id. If both
are provided they need to match or an error is returned.
--nid: nid pattern which follows the same syntax as ip2netnet: network string (e.g. o2ib or o2ib* or o2ib[1,2])
--id: ID assigned to the rule returned by the show command.
Usage: selection nid net show [--nid <NID>net <network name>]
WHERE:
selection nid net show: show selection rules and filter on NID pattern network name if provided.
--nid: nid pattern which follows the same syntax as ip2net
# Adding point to point rule. This creates an association between a local NI and a remote
# NID, and assigns a priority to this relationship so that it's preferred when selecting a pathway..
lnetctl > selection peer net: network string (e.g. o2ib or o2ib* or o2ib[1,2])
# Add a NID priority rule. All NIDs added that match this pattern shall be assigned
# the identified priority. When the selection algorithm runs it shall prefer NIDs with
# higher priority.
lnetctl > selection nid [add | del | show] -h
Usage: selection peer nid add --local <NID> --remote nid <NID> --priority <priority>
WHERE:
selection peer nid add: add a selection rule based on local to remote pathway
--local: nid pattern which follows the same syntax as ip2netthe nid pattern
--remotenid: nid pattern which follows the same syntax as ip2net
--priority: Rule priority
Usage: selection peer nid del --local <NID> --remote <NID> nid <NID> [--id <ID><rule id>]
WHERE:
selection peer nid del: delete a selection rule based on local to remote NID pattern or idgiven the nid patter or the id. If both
are provided they need to match or an error is returned.
--local: nid pattern which follows the same syntax as ip2net
--remote: nid pattern which follows the same syntax as ip2net
--id: ID of assigned to the rule as provided returned by the show command.
Usage: selection peer nid show [--local <NID>] [--remote nid <NID>]
WHERE:
selection peer nid show: show selection rules and filter on NID patterns pattern if provided.
--local: nid pattern which follows the same syntax as ip2net
--remote: nid pattern which follows the same syntax as ip2net
# the output will be of the same YAML format as the input described below.
|
YAML Syntax
...
# Adding point to point rule. This creates an association between a local NI and a remote
# NID, and assigns a priority to this relationship so that it's preferred when selecting a pathway..
lnetctl > selection peer [add | del | show] -h
Usage: selection peer add --local <NID> --remote <NID> --priority <priority>
WHERE:
selection peer add: add a selection rule based on local to remote pathway
--local: nid pattern which follows the same syntax as ip2net
--remote: nid pattern which follows the same syntax as ip2net
--priority: Rule priority
Usage: selection peer del --local <NID> --remote <NID> --id <ID>
WHERE:
selection peer del: delete a selection rule based on local to remote NID pattern or id
--local: nid pattern which follows the same syntax as ip2net
--remote: nid pattern which follows the same syntax as ip2net
--id: ID of the rule as provided by the show command.
Usage: selection peer show [--local <NID>] [--remote <NID>]
WHERE:
selection peer show: show selection rules and filter on NID patterns if provided.
--local: nid pattern which follows the same syntax as ip2net
--remote: nid pattern which follows the same syntax as ip2net
# the output will be of the same YAML format as the input described below.
|
YAML Syntax
Each selection rule will translate into a separate IOCLT to the kernel.
# Configuring Network rules
selection:
- type: net
net: <net name or pattern. e.g. o2ib1, o2ib*, o2ib[1,2]>
priority: <Unsigned integer where 0 is the highest priority>
# Configuring NID rules:
selection:
- type: nid
nid: <a NID pattern as described in the Lustre Manual ip2net syntax>
priority: <Unsigned integer where 0 is the highest priority>
# Configuring Point-to-Point rules.
selection:
- type: peer
local: <a NID pattern as described in the Lustre Manual ip2net syntax>
remote: <a NID pattern as described in the Lustre Manual ip2net syntax>
priority: <Unsigned integer where 0 is the highest priority>
# to delete the rules, there are two options:
# 1. Whenever a rule is added it will be assigned a unique ID. Show command will display the
# unique ID. The unique ID must be explicitly identified in the delete command.
# 2. The rule is matched in the kernel based on the matching rule, unique identifier.
# This means that there can not exist two rules that have the exact matching criteria
# Both options shall be supported.
|
Flattening rules
Rules will have a serialize and deserialize APIs. The serialize API will flatten the rules into a contiguous buffer that will be sent to the kernel. On the kernel side the rules will be deserialzed to be stored and queried. When the userspace queries the rules, the rules are serialized and sent up to user space, which deserializes it and prints it in a YAML format.
# Configuring Network rules
selection:
- type: net
net: <net name or pattern. e.g. o2ib1, o2ib*, o2ib[1,2]>
priority: <Unsigned integer where 0 is the highest priority>
# Configuring NID rules:
selection:
- type: nid
nid: <a NID pattern as described in the Lustre Manual ip2net syntax>
priority: <Unsigned integer where 0 is the highest priority>
# Configuring Point-to-Point rules.
selection:
- type: peer
local: <a NID pattern as described in the Lustre Manual ip2net syntax>
remote: <a NID pattern as described in the Lustre Manual ip2net syntax>
priority: <Unsigned integer where 0 is the highest priority>
# to delete the rules, there are two options:
# 1. Whenever a rule is added it will be assigned a unique ID. Show command will display the
# unique ID. The unique ID must be explicitly identified in the delete command.
# 2. The rule is matched in the kernel based on the matching rule, unique identifier.
# This means that there can not exist two rules that have the exact matching criteria
# Both options shall be supported.
|
Flattening rules
Rules will have a serialize and deserialize APIs. The serialize API will flatten the rules into a contiguous buffer that will be sent to the kernel. On the kernel side the rules will be deserialzed to be stored and queried. When the userspace queries the rules, the rules are serialized and sent up to user space, which deserializes it and prints it in a YAML format.
DLC API
| Code Block |
|---|
/* This is a common structure which describes an expression */
struct lnet_match_expr {
__u32 lme_start;
__u32 lme_end;
__u32 lme_incr;
char lme_r_expr[0];
};
struct lnet_selection_descriptor {
enum selection_type lsd_type;
char *lsd_pattern1;
char *lsd_pattern2;
union {
__u32 lsda_priority;
} lsd_action_u;
};
/*
* lustre_lnet_add_selection
* Delete the peer NIDs. If all peer NIDs of a peer are deleted
* then the peer is deleted
*
* selection - describes the selection policy rule
* seq_no - sequence number of the command
* err_rc - YAML structure of the resultant return code
*/
int lustre_lnet_add_selection(struct selection_descriptor *selection, int seq_no, struct cYAML **er_rc); |
|---|
|
Selection Policies
There are four different types of rules that this HLD will address:
...
