| Message ID | 1512734518-103757-1-git-send-email-mark.b.kavanagh@intel.com |
|---|---|
| Headers | show |
| Series | netdev-dpdk: support multi-segment mbufs | expand |
Thank you Mark & Michael Qiu for this work. I had looked at the patch series and have some high level comments as inline. I would like to test these patch on some of the test scenarios. so please send a proper patch series once the DPDK17.11 support is added in OVS. Regards _Sugesh > -----Original Message----- > From: Kavanagh, Mark B > Sent: Friday, December 8, 2017 12:02 PM > To: dev@openvswitch.org; qiudayu@chinac.com > Cc: Stokes, Ian <ian.stokes@intel.com>; Loftus, Ciara <ciara.loftus@intel.com>; > santosh.shukla@caviumnetworks.com; Chandran, Sugesh > <sugesh.chandran@intel.com>; Kavanagh, Mark B > <mark.b.kavanagh@intel.com> > Subject: [ovs-dev][RFC PATCH V4 0/9] netdev-dpdk: support multi-segment > mbufs > > Overview > ======== > This patchset introduces support for multi-segment mbufs to OvS-DPDK. > Multi-segment mbufs are typically used when the size of an mbuf is insufficient > to contain the entirety of a packet's data. Instead, the data is split across > numerous mbufs, each carrying a portion, or 'segment', of the packet data. > mbufs are chained via their 'next' > attribute (an mbuf pointer). > > Use Cases > ========= > i. Handling oversized (guest-originated) frames, which are marked > for hardware accelration/offload (TSO, for example). > > Packets which originate from a non-DPDK source may be marked for > offload; as such, they may be larger than the permitted ingress > interface's MTU, and may be stored in an oversized dp-packet. In > order to transmit such packets over a DPDK port, their contents > must be copied to a DPDK mbuf (via dpdk_do_tx_copy). However, in > its current implementation, that function only copies data into > a single mbuf; if the space available in the mbuf is exhausted, > but not all packet data has been copied, then it is lost. > Similarly, when cloning a DPDK mbuf, it must be considered > whether that mbuf contains multiple segments. Both issues are > resolved within this patchset. > > ii. Handling jumbo frames. > > While OvS already supports jumbo frames, it does so by increasing > mbuf size, such that the entirety of a jumbo frame may be handled > in a single mbuf. This is certainly the preferred, and most > performant approach (and remains the default). However, it places > high demands on system memory; multi-segment mbufs may be > prefereable for systems which are memory-constrained. > > Enabling multi-segment mbufs > ============================ > Multi-segment and single-segment mbufs are mutually exclusive, and the user > must decide on which approach to adopt on init. The introduction of a new > OVSDB field, 'dpdk-multi-seg-mbufs', facilitates this. > > This is a global boolean value, which determines how jumbo frames are > represented across all DPDK ports. In the absence of a user-supplied value, > 'dpdk-multi-seg-mbufs' defaults to false, i.e. multi-segment mbufs must be > explicitly enabled / single-segment mbufs remain the default. > > Setting the field is identical to setting existing DPDK-specific OVSDB > fields: > > ovs-vsctl set Open_vSwitch . other_config:dpdk-init=true > ovs-vsctl set Open_vSwitch . other_config:dpdk-lcore-mask=0x10 > ovs-vsctl set Open_vSwitch . other_config:dpdk-socket-mem=4096,0 > ==> ovs-vsctl set Open_vSwitch . other_config:dpdk-multi-seg-mbufs=true > > > Notes > ===== > This patchset was generated and built against: > - Current HEAD of OvS master[1] + DPDK v17.11 support patchset[2] > - DPDK v17.11 > > [1] 159cc1f ("datapath-windows: Correct endianness for deleting zone.") [2] > https://mail.openvswitch.org/pipermail/ovs-dev/2017-December/341786.html > > --- > > v4: - restructure patchset > - account for 128B ARM cacheline when sizing mbufs > > Mark Kavanagh (5): > netdev-dpdk: fix mbuf sizing > dp-packet: fix reset_packet for multi-seg mbufs > dp-packet: fix put_uninit for multi-seg mbufs > dp-packet: init specific mbuf fields to 0 > netdev-dpdk: support multi-segment jumbo frames > > Michael Qiu (4): > dp-packet: Fix data_len issue with multi-seg mbufs > dp-packet: copy mbuf info for packet copy > dp-packet: copy data from multi-seg. DPDK mbuf > netdev-dpdk: copy large packet to multi-seg. mbufs > > NEWS | 1 + > lib/dp-packet.c | 51 ++++++++++++++++- > lib/dp-packet.h | 26 ++++----- > lib/dpdk.c | 7 +++ > lib/netdev-dpdk.c | 156 ++++++++++++++++++++++++++++++++++++++++++--- > ------ > lib/netdev-dpdk.h | 1 + > vswitchd/vswitch.xml | 20 +++++++ > 7 files changed, 220 insertions(+), 42 deletions(-) > > -- > 1.9.3
Thanks Sugesh, I'll rebase today and send an updated version that addresses your comments. Best, Mark >-----Original Message----- >From: Chandran, Sugesh >Sent: Friday, December 8, 2017 6:10 PM >To: Kavanagh, Mark B <mark.b.kavanagh@intel.com>; dev@openvswitch.org; >qiudayu@chinac.com >Cc: Stokes, Ian <ian.stokes@intel.com>; Loftus, Ciara ><ciara.loftus@intel.com>; santosh.shukla@caviumnetworks.com >Subject: RE: [ovs-dev][RFC PATCH V4 0/9] netdev-dpdk: support multi-segment >mbufs > >Thank you Mark & Michael Qiu for this work. > >I had looked at the patch series and have some high level comments as inline. > >I would like to test these patch on some of the test scenarios. >so please send a proper patch series once the DPDK17.11 support is added in >OVS. > >Regards >_Sugesh > > >> -----Original Message----- >> From: Kavanagh, Mark B >> Sent: Friday, December 8, 2017 12:02 PM >> To: dev@openvswitch.org; qiudayu@chinac.com >> Cc: Stokes, Ian <ian.stokes@intel.com>; Loftus, Ciara ><ciara.loftus@intel.com>; >> santosh.shukla@caviumnetworks.com; Chandran, Sugesh >> <sugesh.chandran@intel.com>; Kavanagh, Mark B >> <mark.b.kavanagh@intel.com> >> Subject: [ovs-dev][RFC PATCH V4 0/9] netdev-dpdk: support multi-segment >> mbufs >> >> Overview >> ======== >> This patchset introduces support for multi-segment mbufs to OvS-DPDK. >> Multi-segment mbufs are typically used when the size of an mbuf is >insufficient >> to contain the entirety of a packet's data. Instead, the data is split >across >> numerous mbufs, each carrying a portion, or 'segment', of the packet data. >> mbufs are chained via their 'next' >> attribute (an mbuf pointer). >> >> Use Cases >> ========= >> i. Handling oversized (guest-originated) frames, which are marked >> for hardware accelration/offload (TSO, for example). >> >> Packets which originate from a non-DPDK source may be marked for >> offload; as such, they may be larger than the permitted ingress >> interface's MTU, and may be stored in an oversized dp-packet. In >> order to transmit such packets over a DPDK port, their contents >> must be copied to a DPDK mbuf (via dpdk_do_tx_copy). However, in >> its current implementation, that function only copies data into >> a single mbuf; if the space available in the mbuf is exhausted, >> but not all packet data has been copied, then it is lost. >> Similarly, when cloning a DPDK mbuf, it must be considered >> whether that mbuf contains multiple segments. Both issues are >> resolved within this patchset. >> >> ii. Handling jumbo frames. >> >> While OvS already supports jumbo frames, it does so by increasing >> mbuf size, such that the entirety of a jumbo frame may be handled >> in a single mbuf. This is certainly the preferred, and most >> performant approach (and remains the default). However, it places >> high demands on system memory; multi-segment mbufs may be >> prefereable for systems which are memory-constrained. >> >> Enabling multi-segment mbufs >> ============================ >> Multi-segment and single-segment mbufs are mutually exclusive, and the user >> must decide on which approach to adopt on init. The introduction of a new >> OVSDB field, 'dpdk-multi-seg-mbufs', facilitates this. >> >> This is a global boolean value, which determines how jumbo frames are >> represented across all DPDK ports. In the absence of a user-supplied value, >> 'dpdk-multi-seg-mbufs' defaults to false, i.e. multi-segment mbufs must be >> explicitly enabled / single-segment mbufs remain the default. >> >> Setting the field is identical to setting existing DPDK-specific OVSDB >> fields: >> >> ovs-vsctl set Open_vSwitch . other_config:dpdk-init=true >> ovs-vsctl set Open_vSwitch . other_config:dpdk-lcore-mask=0x10 >> ovs-vsctl set Open_vSwitch . other_config:dpdk-socket-mem=4096,0 >> ==> ovs-vsctl set Open_vSwitch . other_config:dpdk-multi-seg-mbufs=true >> >> >> Notes >> ===== >> This patchset was generated and built against: >> - Current HEAD of OvS master[1] + DPDK v17.11 support patchset[2] >> - DPDK v17.11 >> >> [1] 159cc1f ("datapath-windows: Correct endianness for deleting zone.") [2] >> https://mail.openvswitch.org/pipermail/ovs-dev/2017-December/341786.html >> >> --- >> >> v4: - restructure patchset >> - account for 128B ARM cacheline when sizing mbufs >> >> Mark Kavanagh (5): >> netdev-dpdk: fix mbuf sizing >> dp-packet: fix reset_packet for multi-seg mbufs >> dp-packet: fix put_uninit for multi-seg mbufs >> dp-packet: init specific mbuf fields to 0 >> netdev-dpdk: support multi-segment jumbo frames >> >> Michael Qiu (4): >> dp-packet: Fix data_len issue with multi-seg mbufs >> dp-packet: copy mbuf info for packet copy >> dp-packet: copy data from multi-seg. DPDK mbuf >> netdev-dpdk: copy large packet to multi-seg. mbufs >> >> NEWS | 1 + >> lib/dp-packet.c | 51 ++++++++++++++++- >> lib/dp-packet.h | 26 ++++----- >> lib/dpdk.c | 7 +++ >> lib/netdev-dpdk.c | 156 ++++++++++++++++++++++++++++++++++++++++++--- >> ------ >> lib/netdev-dpdk.h | 1 + >> vswitchd/vswitch.xml | 20 +++++++ >> 7 files changed, 220 insertions(+), 42 deletions(-) >> >> -- >> 1.9.3
Overview ======== This patchset introduces support for multi-segment mbufs to OvS-DPDK. Multi-segment mbufs are typically used when the size of an mbuf is insufficient to contain the entirety of a packet's data. Instead, the data is split across numerous mbufs, each carrying a portion, or 'segment', of the packet data. mbufs are chained via their 'next' attribute (an mbuf pointer). Use Cases ========= i. Handling oversized (guest-originated) frames, which are marked for hardware accelration/offload (TSO, for example). Packets which originate from a non-DPDK source may be marked for offload; as such, they may be larger than the permitted ingress interface's MTU, and may be stored in an oversized dp-packet. In order to transmit such packets over a DPDK port, their contents must be copied to a DPDK mbuf (via dpdk_do_tx_copy). However, in its current implementation, that function only copies data into a single mbuf; if the space available in the mbuf is exhausted, but not all packet data has been copied, then it is lost. Similarly, when cloning a DPDK mbuf, it must be considered whether that mbuf contains multiple segments. Both issues are resolved within this patchset. ii. Handling jumbo frames. While OvS already supports jumbo frames, it does so by increasing mbuf size, such that the entirety of a jumbo frame may be handled in a single mbuf. This is certainly the preferred, and most performant approach (and remains the default). However, it places high demands on system memory; multi-segment mbufs may be prefereable for systems which are memory-constrained. Enabling multi-segment mbufs ============================ Multi-segment and single-segment mbufs are mutually exclusive, and the user must decide on which approach to adopt on init. The introduction of a new OVSDB field, 'dpdk-multi-seg-mbufs', facilitates this. This is a global boolean value, which determines how jumbo frames are represented across all DPDK ports. In the absence of a user-supplied value, 'dpdk-multi-seg-mbufs' defaults to false, i.e. multi-segment mbufs must be explicitly enabled / single-segment mbufs remain the default. Setting the field is identical to setting existing DPDK-specific OVSDB fields: ovs-vsctl set Open_vSwitch . other_config:dpdk-init=true ovs-vsctl set Open_vSwitch . other_config:dpdk-lcore-mask=0x10 ovs-vsctl set Open_vSwitch . other_config:dpdk-socket-mem=4096,0 ==> ovs-vsctl set Open_vSwitch . other_config:dpdk-multi-seg-mbufs=true Notes ===== This patchset was generated and built against: - Current HEAD of OvS master[1] + DPDK v17.11 support patchset[2] - DPDK v17.11 [1] 159cc1f ("datapath-windows: Correct endianness for deleting zone.") [2] https://mail.openvswitch.org/pipermail/ovs-dev/2017-December/341786.html --- v4: - restructure patchset - account for 128B ARM cacheline when sizing mbufs Mark Kavanagh (5): netdev-dpdk: fix mbuf sizing dp-packet: fix reset_packet for multi-seg mbufs dp-packet: fix put_uninit for multi-seg mbufs dp-packet: init specific mbuf fields to 0 netdev-dpdk: support multi-segment jumbo frames Michael Qiu (4): dp-packet: Fix data_len issue with multi-seg mbufs dp-packet: copy mbuf info for packet copy dp-packet: copy data from multi-seg. DPDK mbuf netdev-dpdk: copy large packet to multi-seg. mbufs NEWS | 1 + lib/dp-packet.c | 51 ++++++++++++++++- lib/dp-packet.h | 26 ++++----- lib/dpdk.c | 7 +++ lib/netdev-dpdk.c | 156 ++++++++++++++++++++++++++++++++++++++++++--------- lib/netdev-dpdk.h | 1 + vswitchd/vswitch.xml | 20 +++++++ 7 files changed, 220 insertions(+), 42 deletions(-)