| Message ID | 1586870575-12919-1-git-send-email-u9012063@gmail.com |
|---|---|
| State | Accepted |
| Delegated to: | Ilya Maximets |
| Headers | show |
| Series | [ovs-dev,PATCHv5] netdev-afxdp: Add interrupt mode netdev class. | expand |
On Tue, Apr 14, 2020 at 6:23 AM William Tu <u9012063@gmail.com> wrote: > > The patch adds a new netdev class 'afxdp-nonpmd' to enable afxdp > interrupt mode. This is similar to 'type=afxdp', except that the > is_pmd field is set to false. As a result, the packet processing > is handled by main thread, not pmd thread. This avoids burning > the CPU to always 100% when there is no traffic. > > Signed-off-by: William Tu <u9012063@gmail.com> > --- > v5: > - add NETDEV_AFXDP_CLASS_COMMON > v4: > - Previously crash and fix it with qid = qid % netdev_n_txq(netdev) > Now remove it because Ilya's fix: > "dpif-netdev: Force port reconfiguration to change dynamic_txqs." > --- > NEWS | 3 +++ > lib/netdev-linux.c | 37 +++++++++++++++++++++++-------------- > lib/netdev-provider.h | 1 + > lib/netdev.c | 1 + > tests/system-afxdp.at | 23 +++++++++++++++++++++++ > 5 files changed, 51 insertions(+), 14 deletions(-) > > diff --git a/NEWS b/NEWS > index 70bd17584594..6db2d993ffdb 100644 > --- a/NEWS > +++ b/NEWS > @@ -10,6 +10,9 @@ Post-v2.13.0 > * Deprecated DPDK ring ports (dpdkr) are no longer supported. > - Linux datapath: > * Support for kernel versions up to 5.5.x. > + - AF_XDP: > + * New netdev class 'afxdp-nonpmd' for netdev-afxdp to save CPU cycles > + by enabling interrupt mode. > > > v2.13.0 - 14 Feb 2020 > diff --git a/lib/netdev-linux.c b/lib/netdev-linux.c > index ff045cb1290b..1d7ed0145c48 100644 > --- a/lib/netdev-linux.c > +++ b/lib/netdev-linux.c > @@ -3599,24 +3599,33 @@ const struct netdev_class netdev_internal_class = { > }; > > #ifdef HAVE_AF_XDP > +#define NETDEV_AFXDP_CLASS_COMMON \ > + .construct = netdev_afxdp_construct, \ > + .destruct = netdev_afxdp_destruct, \ > + .get_stats = netdev_afxdp_get_stats, \ > + .get_custom_stats = netdev_afxdp_get_custom_stats, \ > + .get_status = netdev_linux_get_status, \ > + .set_config = netdev_afxdp_set_config, \ > + .get_config = netdev_afxdp_get_config, \ > + .reconfigure = netdev_afxdp_reconfigure, \ > + .get_numa_id = netdev_linux_get_numa_id, \ > + .send = netdev_afxdp_batch_send, \ > + .rxq_construct = netdev_afxdp_rxq_construct, \ > + .rxq_destruct = netdev_afxdp_rxq_destruct, \ > + .rxq_recv = netdev_afxdp_rxq_recv > + > const struct netdev_class netdev_afxdp_class = { > NETDEV_LINUX_CLASS_COMMON, > + NETDEV_AFXDP_CLASS_COMMON, > .type = "afxdp", > .is_pmd = true, > - .init = netdev_afxdp_init, > - .construct = netdev_afxdp_construct, > - .destruct = netdev_afxdp_destruct, > - .get_stats = netdev_afxdp_get_stats, > - .get_custom_stats = netdev_afxdp_get_custom_stats, > - .get_status = netdev_linux_get_status, > - .set_config = netdev_afxdp_set_config, > - .get_config = netdev_afxdp_get_config, > - .reconfigure = netdev_afxdp_reconfigure, > - .get_numa_id = netdev_linux_get_numa_id, > - .send = netdev_afxdp_batch_send, > - .rxq_construct = netdev_afxdp_rxq_construct, > - .rxq_destruct = netdev_afxdp_rxq_destruct, > - .rxq_recv = netdev_afxdp_rxq_recv, > +}; > + > +const struct netdev_class netdev_afxdp_nonpmd_class = { > + NETDEV_LINUX_CLASS_COMMON, > + NETDEV_AFXDP_CLASS_COMMON, > + .type = "afxdp-nonpmd", > + .is_pmd = false, > }; > #endif > > diff --git a/lib/netdev-provider.h b/lib/netdev-provider.h > index 6f509424bc81..d9503adb0fb6 100644 > --- a/lib/netdev-provider.h > +++ b/lib/netdev-provider.h > @@ -850,6 +850,7 @@ extern const struct netdev_class netdev_tap_class; > > #ifdef HAVE_AF_XDP > extern const struct netdev_class netdev_afxdp_class; > +extern const struct netdev_class netdev_afxdp_nonpmd_class; > #endif > #ifdef __cplusplus > } > diff --git a/lib/netdev.c b/lib/netdev.c > index 8c44eee8e98a..90962eec66cf 100644 > --- a/lib/netdev.c > +++ b/lib/netdev.c > @@ -154,6 +154,7 @@ netdev_initialize(void) > netdev_register_flow_api_provider(&netdev_offload_tc); > #ifdef HAVE_AF_XDP > netdev_register_provider(&netdev_afxdp_class); > + netdev_register_provider(&netdev_afxdp_nonpmd_class); > #endif > #endif > #if defined(__FreeBSD__) || defined(__NetBSD__) > diff --git a/tests/system-afxdp.at b/tests/system-afxdp.at > index e4451624f882..0d09906fb6c8 100644 > --- a/tests/system-afxdp.at > +++ b/tests/system-afxdp.at > @@ -22,3 +22,26 @@ AT_CHECK([grep "ovs-p0: could not set configuration" ovs-vswitchd.log | wc -l], > OVS_TRAFFIC_VSWITCHD_STOP(["/ovs-p0: Too big 'n_rxq'/d > /ovs-p0: could not set configuration/d"]) > AT_CLEANUP > + > + > +AT_SETUP([AF_XDP - ping between pmd and non-pmd ports]) > +AT_KEYWORDS([afxdp nonpmd]) > +OVS_TRAFFIC_VSWITCHD_START() > + > +AT_CHECK([ovs-ofctl add-flow br0 "actions=normal"]) > + > +ADD_NAMESPACES(at_ns0, at_ns1) > +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24") > +ADD_VETH(p1, at_ns1, br0, "10.1.1.2/24") > + > +AT_CHECK([ovs-vsctl del-port ovs-p0]) > +AT_CHECK([ovs-vsctl add-port br0 ovs-p0 -- \ > + set interface ovs-p0 type=afxdp-nonpmd options:n_rxq=1], > + [0], [], [stderr]) > + > +NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 -w 2 10.1.1.2 | FORMAT_PING], [0], [dnl > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > +]) > + > +OVS_TRAFFIC_VSWITCHD_STOP > +AT_CLEANUP > -- > 2.7.4 > Hi Ilya, Do you think this version looks good? Thanks William
On 4/27/20 5:54 PM, William Tu wrote: > On Tue, Apr 14, 2020 at 6:23 AM William Tu <u9012063@gmail.com> wrote: >> >> The patch adds a new netdev class 'afxdp-nonpmd' to enable afxdp >> interrupt mode. This is similar to 'type=afxdp', except that the >> is_pmd field is set to false. As a result, the packet processing >> is handled by main thread, not pmd thread. This avoids burning >> the CPU to always 100% when there is no traffic. >> >> Signed-off-by: William Tu <u9012063@gmail.com> >> --- >> v5: >> - add NETDEV_AFXDP_CLASS_COMMON >> v4: >> - Previously crash and fix it with qid = qid % netdev_n_txq(netdev) >> Now remove it because Ilya's fix: >> "dpif-netdev: Force port reconfiguration to change dynamic_txqs." >> --- >> NEWS | 3 +++ >> lib/netdev-linux.c | 37 +++++++++++++++++++++++-------------- >> lib/netdev-provider.h | 1 + >> lib/netdev.c | 1 + >> tests/system-afxdp.at | 23 +++++++++++++++++++++++ >> 5 files changed, 51 insertions(+), 14 deletions(-) >> >> diff --git a/NEWS b/NEWS >> index 70bd17584594..6db2d993ffdb 100644 >> --- a/NEWS >> +++ b/NEWS >> @@ -10,6 +10,9 @@ Post-v2.13.0 >> * Deprecated DPDK ring ports (dpdkr) are no longer supported. >> - Linux datapath: >> * Support for kernel versions up to 5.5.x. >> + - AF_XDP: >> + * New netdev class 'afxdp-nonpmd' for netdev-afxdp to save CPU cycles >> + by enabling interrupt mode. >> >> >> v2.13.0 - 14 Feb 2020 >> diff --git a/lib/netdev-linux.c b/lib/netdev-linux.c >> index ff045cb1290b..1d7ed0145c48 100644 >> --- a/lib/netdev-linux.c >> +++ b/lib/netdev-linux.c >> @@ -3599,24 +3599,33 @@ const struct netdev_class netdev_internal_class = { >> }; >> >> #ifdef HAVE_AF_XDP >> +#define NETDEV_AFXDP_CLASS_COMMON \ >> + .construct = netdev_afxdp_construct, \ >> + .destruct = netdev_afxdp_destruct, \ >> + .get_stats = netdev_afxdp_get_stats, \ >> + .get_custom_stats = netdev_afxdp_get_custom_stats, \ >> + .get_status = netdev_linux_get_status, \ >> + .set_config = netdev_afxdp_set_config, \ >> + .get_config = netdev_afxdp_get_config, \ >> + .reconfigure = netdev_afxdp_reconfigure, \ >> + .get_numa_id = netdev_linux_get_numa_id, \ >> + .send = netdev_afxdp_batch_send, \ >> + .rxq_construct = netdev_afxdp_rxq_construct, \ >> + .rxq_destruct = netdev_afxdp_rxq_destruct, \ >> + .rxq_recv = netdev_afxdp_rxq_recv >> + >> const struct netdev_class netdev_afxdp_class = { >> NETDEV_LINUX_CLASS_COMMON, >> + NETDEV_AFXDP_CLASS_COMMON, >> .type = "afxdp", >> .is_pmd = true, >> - .init = netdev_afxdp_init, >> - .construct = netdev_afxdp_construct, >> - .destruct = netdev_afxdp_destruct, >> - .get_stats = netdev_afxdp_get_stats, >> - .get_custom_stats = netdev_afxdp_get_custom_stats, >> - .get_status = netdev_linux_get_status, >> - .set_config = netdev_afxdp_set_config, >> - .get_config = netdev_afxdp_get_config, >> - .reconfigure = netdev_afxdp_reconfigure, >> - .get_numa_id = netdev_linux_get_numa_id, >> - .send = netdev_afxdp_batch_send, >> - .rxq_construct = netdev_afxdp_rxq_construct, >> - .rxq_destruct = netdev_afxdp_rxq_destruct, >> - .rxq_recv = netdev_afxdp_rxq_recv, >> +}; >> + >> +const struct netdev_class netdev_afxdp_nonpmd_class = { >> + NETDEV_LINUX_CLASS_COMMON, >> + NETDEV_AFXDP_CLASS_COMMON, >> + .type = "afxdp-nonpmd", >> + .is_pmd = false, >> }; >> #endif >> >> diff --git a/lib/netdev-provider.h b/lib/netdev-provider.h >> index 6f509424bc81..d9503adb0fb6 100644 >> --- a/lib/netdev-provider.h >> +++ b/lib/netdev-provider.h >> @@ -850,6 +850,7 @@ extern const struct netdev_class netdev_tap_class; >> >> #ifdef HAVE_AF_XDP >> extern const struct netdev_class netdev_afxdp_class; >> +extern const struct netdev_class netdev_afxdp_nonpmd_class; >> #endif >> #ifdef __cplusplus >> } >> diff --git a/lib/netdev.c b/lib/netdev.c >> index 8c44eee8e98a..90962eec66cf 100644 >> --- a/lib/netdev.c >> +++ b/lib/netdev.c >> @@ -154,6 +154,7 @@ netdev_initialize(void) >> netdev_register_flow_api_provider(&netdev_offload_tc); >> #ifdef HAVE_AF_XDP >> netdev_register_provider(&netdev_afxdp_class); >> + netdev_register_provider(&netdev_afxdp_nonpmd_class); >> #endif >> #endif >> #if defined(__FreeBSD__) || defined(__NetBSD__) >> diff --git a/tests/system-afxdp.at b/tests/system-afxdp.at >> index e4451624f882..0d09906fb6c8 100644 >> --- a/tests/system-afxdp.at >> +++ b/tests/system-afxdp.at >> @@ -22,3 +22,26 @@ AT_CHECK([grep "ovs-p0: could not set configuration" ovs-vswitchd.log | wc -l], >> OVS_TRAFFIC_VSWITCHD_STOP(["/ovs-p0: Too big 'n_rxq'/d >> /ovs-p0: could not set configuration/d"]) >> AT_CLEANUP >> + >> + >> +AT_SETUP([AF_XDP - ping between pmd and non-pmd ports]) >> +AT_KEYWORDS([afxdp nonpmd]) >> +OVS_TRAFFIC_VSWITCHD_START() >> + >> +AT_CHECK([ovs-ofctl add-flow br0 "actions=normal"]) >> + >> +ADD_NAMESPACES(at_ns0, at_ns1) >> +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24") >> +ADD_VETH(p1, at_ns1, br0, "10.1.1.2/24") >> + >> +AT_CHECK([ovs-vsctl del-port ovs-p0]) >> +AT_CHECK([ovs-vsctl add-port br0 ovs-p0 -- \ >> + set interface ovs-p0 type=afxdp-nonpmd options:n_rxq=1], >> + [0], [], [stderr]) >> + >> +NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 -w 2 10.1.1.2 | FORMAT_PING], [0], [dnl >> +3 packets transmitted, 3 received, 0% packet loss, time 0ms >> +]) >> + >> +OVS_TRAFFIC_VSWITCHD_STOP >> +AT_CLEANUP >> -- >> 2.7.4 >> > Hi Ilya, > Do you think this version looks good? I actually had a plan to check it a bit later today. Best regards, Ilya Maximets.
On 4/14/20 3:22 PM, William Tu wrote: > The patch adds a new netdev class 'afxdp-nonpmd' to enable afxdp > interrupt mode. This is similar to 'type=afxdp', except that the > is_pmd field is set to false. As a result, the packet processing > is handled by main thread, not pmd thread. This avoids burning > the CPU to always 100% when there is no traffic. > > Signed-off-by: William Tu <u9012063@gmail.com> Thanks! Applied to master. Best regards, Ilya Maximets.
diff --git a/NEWS b/NEWS index 70bd17584594..6db2d993ffdb 100644 --- a/NEWS +++ b/NEWS @@ -10,6 +10,9 @@ Post-v2.13.0 * Deprecated DPDK ring ports (dpdkr) are no longer supported. - Linux datapath: * Support for kernel versions up to 5.5.x. + - AF_XDP: + * New netdev class 'afxdp-nonpmd' for netdev-afxdp to save CPU cycles + by enabling interrupt mode. v2.13.0 - 14 Feb 2020 diff --git a/lib/netdev-linux.c b/lib/netdev-linux.c index ff045cb1290b..1d7ed0145c48 100644 --- a/lib/netdev-linux.c +++ b/lib/netdev-linux.c @@ -3599,24 +3599,33 @@ const struct netdev_class netdev_internal_class = { }; #ifdef HAVE_AF_XDP +#define NETDEV_AFXDP_CLASS_COMMON \ + .construct = netdev_afxdp_construct, \ + .destruct = netdev_afxdp_destruct, \ + .get_stats = netdev_afxdp_get_stats, \ + .get_custom_stats = netdev_afxdp_get_custom_stats, \ + .get_status = netdev_linux_get_status, \ + .set_config = netdev_afxdp_set_config, \ + .get_config = netdev_afxdp_get_config, \ + .reconfigure = netdev_afxdp_reconfigure, \ + .get_numa_id = netdev_linux_get_numa_id, \ + .send = netdev_afxdp_batch_send, \ + .rxq_construct = netdev_afxdp_rxq_construct, \ + .rxq_destruct = netdev_afxdp_rxq_destruct, \ + .rxq_recv = netdev_afxdp_rxq_recv + const struct netdev_class netdev_afxdp_class = { NETDEV_LINUX_CLASS_COMMON, + NETDEV_AFXDP_CLASS_COMMON, .type = "afxdp", .is_pmd = true, - .init = netdev_afxdp_init, - .construct = netdev_afxdp_construct, - .destruct = netdev_afxdp_destruct, - .get_stats = netdev_afxdp_get_stats, - .get_custom_stats = netdev_afxdp_get_custom_stats, - .get_status = netdev_linux_get_status, - .set_config = netdev_afxdp_set_config, - .get_config = netdev_afxdp_get_config, - .reconfigure = netdev_afxdp_reconfigure, - .get_numa_id = netdev_linux_get_numa_id, - .send = netdev_afxdp_batch_send, - .rxq_construct = netdev_afxdp_rxq_construct, - .rxq_destruct = netdev_afxdp_rxq_destruct, - .rxq_recv = netdev_afxdp_rxq_recv, +}; + +const struct netdev_class netdev_afxdp_nonpmd_class = { + NETDEV_LINUX_CLASS_COMMON, + NETDEV_AFXDP_CLASS_COMMON, + .type = "afxdp-nonpmd", + .is_pmd = false, }; #endif diff --git a/lib/netdev-provider.h b/lib/netdev-provider.h index 6f509424bc81..d9503adb0fb6 100644 --- a/lib/netdev-provider.h +++ b/lib/netdev-provider.h @@ -850,6 +850,7 @@ extern const struct netdev_class netdev_tap_class; #ifdef HAVE_AF_XDP extern const struct netdev_class netdev_afxdp_class; +extern const struct netdev_class netdev_afxdp_nonpmd_class; #endif #ifdef __cplusplus } diff --git a/lib/netdev.c b/lib/netdev.c index 8c44eee8e98a..90962eec66cf 100644 --- a/lib/netdev.c +++ b/lib/netdev.c @@ -154,6 +154,7 @@ netdev_initialize(void) netdev_register_flow_api_provider(&netdev_offload_tc); #ifdef HAVE_AF_XDP netdev_register_provider(&netdev_afxdp_class); + netdev_register_provider(&netdev_afxdp_nonpmd_class); #endif #endif #if defined(__FreeBSD__) || defined(__NetBSD__) diff --git a/tests/system-afxdp.at b/tests/system-afxdp.at index e4451624f882..0d09906fb6c8 100644 --- a/tests/system-afxdp.at +++ b/tests/system-afxdp.at @@ -22,3 +22,26 @@ AT_CHECK([grep "ovs-p0: could not set configuration" ovs-vswitchd.log | wc -l], OVS_TRAFFIC_VSWITCHD_STOP(["/ovs-p0: Too big 'n_rxq'/d /ovs-p0: could not set configuration/d"]) AT_CLEANUP + + +AT_SETUP([AF_XDP - ping between pmd and non-pmd ports]) +AT_KEYWORDS([afxdp nonpmd]) +OVS_TRAFFIC_VSWITCHD_START() + +AT_CHECK([ovs-ofctl add-flow br0 "actions=normal"]) + +ADD_NAMESPACES(at_ns0, at_ns1) +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24") +ADD_VETH(p1, at_ns1, br0, "10.1.1.2/24") + +AT_CHECK([ovs-vsctl del-port ovs-p0]) +AT_CHECK([ovs-vsctl add-port br0 ovs-p0 -- \ + set interface ovs-p0 type=afxdp-nonpmd options:n_rxq=1], + [0], [], [stderr]) + +NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 -w 2 10.1.1.2 | FORMAT_PING], [0], [dnl +3 packets transmitted, 3 received, 0% packet loss, time 0ms +]) + +OVS_TRAFFIC_VSWITCHD_STOP +AT_CLEANUP
The patch adds a new netdev class 'afxdp-nonpmd' to enable afxdp interrupt mode. This is similar to 'type=afxdp', except that the is_pmd field is set to false. As a result, the packet processing is handled by main thread, not pmd thread. This avoids burning the CPU to always 100% when there is no traffic. Signed-off-by: William Tu <u9012063@gmail.com> --- v5: - add NETDEV_AFXDP_CLASS_COMMON v4: - Previously crash and fix it with qid = qid % netdev_n_txq(netdev) Now remove it because Ilya's fix: "dpif-netdev: Force port reconfiguration to change dynamic_txqs." --- NEWS | 3 +++ lib/netdev-linux.c | 37 +++++++++++++++++++++++-------------- lib/netdev-provider.h | 1 + lib/netdev.c | 1 + tests/system-afxdp.at | 23 +++++++++++++++++++++++ 5 files changed, 51 insertions(+), 14 deletions(-)