Testing SR-IOV¶
NFVbench supports SR-IOV with the PVP and PVVP packet flow. Most use cases for SR-IOV only require single VNF chains (NxPVP). Daisy chaining VNFs with SR-IOV (PVVP) requires selecting either SR-IOV for the middle network or a fast vswitch (using the standard OVS for that purpose works but would be a serious bottleneck)
Instructions below refer to the PVP or PVVP use cases. For external chains using SR-IOV, select the VLAN tagging option that corresponds to the external chains SR-IOV setting.
Pre-requisites¶
To test SR-IOV you need to have compute nodes configured to support one or more SR-IOV interfaces (also knows as PF or physical function) and you need OpenStack to be configured to support SR-IOV. You will also need to know: - the name of the physical networks associated to the SR-IOV interfaces (this is a configuration in Nova compute) - the VLAN range that can be used on the switch ports that are wired to the SR-IOV ports. Such switch ports are normally configured in trunk mode with a range of VLAN ids enabled on that port
For example, in the case of 2 SR-IOV ports per compute node, 2 physical networks are generally configured in OpenStack with a distinct name. The VLAN range to use is is also allocated and reserved by the network administrator and in coordination with the corresponding top of rack switch port configuration.
Configuration¶
To enable SR-IOV test, you will need to provide the following configuration options to NFVbench (in the configuration file). This example instructs NFVbench to create the left and right networks of a PVP packet flow to run on 2 SRIOV ports named “phys_sriov0” and “phys_sriov1” using resp. segmentation_id 2000 and 2001:
sriov: true
internal_networks:
left:
segmentation_id: 2000
physical_network: phys_sriov0
right:
segmentation_id: 2001
physical_network: phys_sriov1
The segmentation ID fields must be different. In the case of PVVP, the middle network also needs to be provisioned properly. The same physical network can also be shared by the virtual networks but with different segmentation IDs.
Multi-Chaining¶
The above configuration works for multi-chaining and shared network (“service_chain_shared_net” set to true). In that case all VNFs will share the same left and right network/VLAN.
In the case of non shared network (“service_chain_shared_net” set to false), the segmentation_id fields must contain a list of distinct VLANs to use for each chain. Example of configuration for 3 chains:
sriov: true
internal_networks:
left:
segmentation_id: [2000, 2001, 2002]
physical_network: phys_sriov0
right:
segmentation_id: [2100, 2101, 2102]
physical_network: phys_sriov1
Alternatively it is also possible to specify different physnets per chain:
sriov: true
internal_networks:
left:
segmentation_id: [2000, 2001, 2002]
physical_network: [phys_sriov0, phys_sriov2, phys_sriov4]
right:
segmentation_id: [2100, 2101, 2102]
physical_network: [phys_sriov1, phys_srviov3, phys_sriov5]
NFVbench cores with SR-IOV¶
The default core count for NFVbench/TRex may not be sufficient for higher throughput line cards (greater than 10Gbps). This will result in warning messages such as:
INFO WARNING: There is a significant difference between requested TX rate (119047618) and actual TX rate (38897379).
The traffic generator may not have sufficient CPU to achieve the requested TX rate.
In that case it is recommended to try allocating more cores to TRex using the cores property in the configuration file, for example to set to 8 cores:
cores: 8
It is also advisable to increase the number of vcpus in the VMs:
VM Flavor for SR-IOV and NIC NUMA socket placement¶
Because SR-IOV throughput uses a lot of CPU in the VM, it is recommended to increase the vcpu count, for example to 4 vcpus:
flavor:
# Number of vCPUs for the flavor
vcpus: 4
# Memory for the flavor in MB
ram: 8192
# Size of local disk in GB
disk: 0
extra_specs:
"hw:cpu_policy": dedicated
If the 2 selected ports reside on NICs that are on different NUMA sockets, you will need to explicitly tell Nova to use 2 numa nodes in the flavor used for the VMs in order to satisfy the filters, for example:
flavor:
# Number of vCPUs for the flavor
vcpus: 4
# Memory for the flavor in MB
ram: 8192
# Size of local disk in GB
disk: 0
extra_specs:
"hw:cpu_policy": dedicated
"hw:numa_nodes": 2
Failure to do so might cause the VM creation to fail with the Nova error “Instance creation error: Insufficient compute resources: Requested instance NUMA topology together with requested PCI devices cannot fit the given host NUMA topology.”