Message ID | 1513728002-7643-1-git-send-email-shannon.nelson@oracle.com |
---|---|
Headers | show |
Series | ixgbe: Add ipsec offload | expand |
On 2017/12/20 7:59, Shannon Nelson wrote: > This is an implementation of the ipsec hardware offload feature for > the ixgbe driver and Intel's 10Gbe series NICs: x540, x550, 82599. Hi, Nelson I notice that the ipsec feature is based on x540, x550, 82599. But this ixgbe driver will also work with 82598. Does this ipsec feature also work with 82598? Thanks a lot. Zhu Yanjun > These patches apply to net-next v4.14 as well as Jeff Kirsher's next-queue > v4.15-rc1-206-ge47375b. > > The ixgbe NICs support ipsec offload for 1024 Rx and 1024 Tx Security > Associations (SAs), using up to 128 inbound IP addresses, and using the > rfc4106(gcm(aes)) encryption. This code does not yet support IPv6, > checksum offload, or TSO in conjunction with the ipsec offload - those > will be added in the future. > > This code shows improvements in both packet throughput and CPU utilization. > For example, here are some quicky numbers that show the magnitude of the > performance gain on a single run of "iperf -c <dest>" with the ipsec > offload on both ends of a point-to-point connection: > > 9.4 Gbps - normal case > 7.6 Gbps - ipsec with offload > 343 Mbps - ipsec no offload > > To set up a similar test case, you first need to be sure you have a recent > version of iproute2 that supports the ipsec offload tag, probably something > from ip 4.12 or newer would be best. I have a shell script that builds > up the appropriate commands for me, but here are the resulting commands > for all tcp traffic between 14.0.0.52 and 14.0.0.70: > > For the left side (14.0.0.52): > ip x p add dir out src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp tmpl \ > proto esp src 14.0.0.52 dst 14.0.0.70 spi 0x07 mode transport reqid 0x07 > ip x p add dir in src 14.0.0.70/24 dst 14.0.0.52/24 proto tcp tmpl \ > proto esp dst 14.0.0.52 src 14.0.0.70 spi 0x07 mode transport reqid 0x07 > ip x s add proto esp src 14.0.0.52 dst 14.0.0.70 spi 0x07 mode transport \ > reqid 0x07 replay-window 32 \ > aead 'rfc4106(gcm(aes))' 0x44434241343332312423222114131211f4f3f2f1 128 \ > sel src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp offload dev eth4 dir out > ip x s add proto esp dst 14.0.0.52 src 14.0.0.70 spi 0x07 mode transport \ > reqid 0x07 replay-window 32 \ > aead 'rfc4106(gcm(aes))' 0x44434241343332312423222114131211f4f3f2f1 128 \ > sel src 14.0.0.70/24 dst 14.0.0.52/24 proto tcp offload dev eth4 dir in > > For the right side (14.0.0.70): > ip x p add dir out src 14.0.0.70/24 dst 14.0.0.52/24 proto tcp tmpl \ > proto esp src 14.0.0.70 dst 14.0.0.52 spi 0x07 mode transport reqid 0x07 > ip x p add dir in src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp tmpl \ > proto esp dst 14.0.0.70 src 14.0.0.52 spi 0x07 mode transport reqid 0x07 > ip x s add proto esp src 14.0.0.70 dst 14.0.0.52 spi 0x07 mode transport \ > reqid 0x07 replay-window 32 \ > aead 'rfc4106(gcm(aes))' 0x44434241343332312423222114131211f4f3f2f1 128 \ > sel src 14.0.0.70/24 dst 14.0.0.52/24 proto tcp offload dev eth4 dir out > ip x s add proto esp dst 14.0.0.70 src 14.0.0.52 spi 0x07 mode transport \ > reqid 0x07 replay-window 32 \ > aead 'rfc4106(gcm(aes))' 0x44434241343332312423222114131211f4f3f2f1 128 \ > sel src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp offload dev eth4 dir in > > In both cases, the command "ip x s flush ; ip x p flush" will clean > it all out and remove the offloads. > > Lastly, thanks to Alex Duyck for his early comments. > > Please see the individual patches for specific update info. > > v3: fixes after comments from those wonderfully pesky kbuild robots > v2: fixes after comments from Alex > > Shannon Nelson (10): > ixgbe: clean up ipsec defines > ixgbe: add ipsec register access routines > ixgbe: add ipsec engine start and stop routines > ixgbe: add ipsec data structures > ixgbe: add ipsec offload add and remove SA > ixgbe: restore offloaded SAs after a reset > ixgbe: process the Rx ipsec offload > ixgbe: process the Tx ipsec offload > ixgbe: ipsec offload stats > ixgbe: register ipsec offload with the xfrm subsystem > > drivers/net/ethernet/intel/ixgbe/Makefile | 1 + > drivers/net/ethernet/intel/ixgbe/ixgbe.h | 33 +- > drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c | 2 + > drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c | 923 +++++++++++++++++++++++ > drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.h | 92 +++ > drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c | 4 +- > drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 39 +- > drivers/net/ethernet/intel/ixgbe/ixgbe_type.h | 22 +- > 8 files changed, 1093 insertions(+), 23 deletions(-) > create mode 100644 drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c > create mode 100644 drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.h >
On 2017/12/21 14:39, Yanjun Zhu wrote: > > > On 2017/12/20 7:59, Shannon Nelson wrote: >> This is an implementation of the ipsec hardware offload feature for >> the ixgbe driver and Intel's 10Gbe series NICs: x540, x550, 82599. > Hi, Nelson > > I notice that the ipsec feature is based on x540, x550, 82599. But > this ixgbe driver > will also work with 82598. > > Does this ipsec feature also work with 82598? Sorry. I mean, after these ipsec patches are applied, whether ipsec offload enabled or not, can this ixgbe driver still work well with 82598? Zhu Yanjun > > Thanks a lot. > Zhu Yanjun >> These patches apply to net-next v4.14 as well as Jeff Kirsher's >> next-queue >> v4.15-rc1-206-ge47375b. >> >> The ixgbe NICs support ipsec offload for 1024 Rx and 1024 Tx Security >> Associations (SAs), using up to 128 inbound IP addresses, and using the >> rfc4106(gcm(aes)) encryption. This code does not yet support IPv6, >> checksum offload, or TSO in conjunction with the ipsec offload - those >> will be added in the future. >> >> This code shows improvements in both packet throughput and CPU >> utilization. >> For example, here are some quicky numbers that show the magnitude of the >> performance gain on a single run of "iperf -c <dest>" with the ipsec >> offload on both ends of a point-to-point connection: >> >> 9.4 Gbps - normal case >> 7.6 Gbps - ipsec with offload >> 343 Mbps - ipsec no offload >> >> To set up a similar test case, you first need to be sure you have a >> recent >> version of iproute2 that supports the ipsec offload tag, probably >> something >> from ip 4.12 or newer would be best. I have a shell script that builds >> up the appropriate commands for me, but here are the resulting commands >> for all tcp traffic between 14.0.0.52 and 14.0.0.70: >> >> For the left side (14.0.0.52): >> ip x p add dir out src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp tmpl \ >> proto esp src 14.0.0.52 dst 14.0.0.70 spi 0x07 mode transport >> reqid 0x07 >> ip x p add dir in src 14.0.0.70/24 dst 14.0.0.52/24 proto tcp tmpl \ >> proto esp dst 14.0.0.52 src 14.0.0.70 spi 0x07 mode transport >> reqid 0x07 >> ip x s add proto esp src 14.0.0.52 dst 14.0.0.70 spi 0x07 mode >> transport \ >> reqid 0x07 replay-window 32 \ >> aead 'rfc4106(gcm(aes))' >> 0x44434241343332312423222114131211f4f3f2f1 128 \ >> sel src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp offload dev >> eth4 dir out >> ip x s add proto esp dst 14.0.0.52 src 14.0.0.70 spi 0x07 mode >> transport \ >> reqid 0x07 replay-window 32 \ >> aead 'rfc4106(gcm(aes))' >> 0x44434241343332312423222114131211f4f3f2f1 128 \ >> sel src 14.0.0.70/24 dst 14.0.0.52/24 proto tcp offload dev >> eth4 dir in >> For the right side (14.0.0.70): >> ip x p add dir out src 14.0.0.70/24 dst 14.0.0.52/24 proto tcp tmpl \ >> proto esp src 14.0.0.70 dst 14.0.0.52 spi 0x07 mode transport >> reqid 0x07 >> ip x p add dir in src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp tmpl \ >> proto esp dst 14.0.0.70 src 14.0.0.52 spi 0x07 mode transport >> reqid 0x07 >> ip x s add proto esp src 14.0.0.70 dst 14.0.0.52 spi 0x07 mode >> transport \ >> reqid 0x07 replay-window 32 \ >> aead 'rfc4106(gcm(aes))' >> 0x44434241343332312423222114131211f4f3f2f1 128 \ >> sel src 14.0.0.70/24 dst 14.0.0.52/24 proto tcp offload dev >> eth4 dir out >> ip x s add proto esp dst 14.0.0.70 src 14.0.0.52 spi 0x07 mode >> transport \ >> reqid 0x07 replay-window 32 \ >> aead 'rfc4106(gcm(aes))' >> 0x44434241343332312423222114131211f4f3f2f1 128 \ >> sel src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp offload dev >> eth4 dir in >> >> In both cases, the command "ip x s flush ; ip x p flush" will clean >> it all out and remove the offloads. >> >> Lastly, thanks to Alex Duyck for his early comments. >> >> Please see the individual patches for specific update info. >> >> v3: fixes after comments from those wonderfully pesky kbuild robots >> v2: fixes after comments from Alex >> >> Shannon Nelson (10): >> ixgbe: clean up ipsec defines >> ixgbe: add ipsec register access routines >> ixgbe: add ipsec engine start and stop routines >> ixgbe: add ipsec data structures >> ixgbe: add ipsec offload add and remove SA >> ixgbe: restore offloaded SAs after a reset >> ixgbe: process the Rx ipsec offload >> ixgbe: process the Tx ipsec offload >> ixgbe: ipsec offload stats >> ixgbe: register ipsec offload with the xfrm subsystem >> >> drivers/net/ethernet/intel/ixgbe/Makefile | 1 + >> drivers/net/ethernet/intel/ixgbe/ixgbe.h | 33 +- >> drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c | 2 + >> drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c | 923 >> +++++++++++++++++++++++ >> drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.h | 92 +++ >> drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c | 4 +- >> drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 39 +- >> drivers/net/ethernet/intel/ixgbe/ixgbe_type.h | 22 +- >> 8 files changed, 1093 insertions(+), 23 deletions(-) >> create mode 100644 drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c >> create mode 100644 drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.h >> > >
On 12/20/2017 11:09 PM, Yanjun Zhu wrote: > On 2017/12/21 14:39, Yanjun Zhu wrote: >> On 2017/12/20 7:59, Shannon Nelson wrote: >>> This is an implementation of the ipsec hardware offload feature for >>> the ixgbe driver and Intel's 10Gbe series NICs: x540, x550, 82599. >> Hi, Nelson >> >> I notice that the ipsec feature is based on x540, x550, 82599. But >> this ixgbe driver >> will also work with 82598. >> >> Does this ipsec feature also work with 82598? > Sorry. I mean, after these ipsec patches are applied, whether ipsec > offload enabled or not, > can this ixgbe driver still work well with 82598? Hmm... I don't have one to test on, but I suspect the 82598 might not be happy with this. I'll send a followup patch to catch this case. Thanks! sln > > Zhu Yanjun >> >> Thanks a lot. >> Zhu Yanjun >>> These patches apply to net-next v4.14 as well as Jeff Kirsher's >>> next-queue >>> v4.15-rc1-206-ge47375b. >>> >>> The ixgbe NICs support ipsec offload for 1024 Rx and 1024 Tx Security >>> Associations (SAs), using up to 128 inbound IP addresses, and using the >>> rfc4106(gcm(aes)) encryption. This code does not yet support IPv6, >>> checksum offload, or TSO in conjunction with the ipsec offload - those >>> will be added in the future. >>> >>> This code shows improvements in both packet throughput and CPU >>> utilization. >>> For example, here are some quicky numbers that show the magnitude of the >>> performance gain on a single run of "iperf -c <dest>" with the ipsec >>> offload on both ends of a point-to-point connection: >>> >>> 9.4 Gbps - normal case >>> 7.6 Gbps - ipsec with offload >>> 343 Mbps - ipsec no offload >>> >>> To set up a similar test case, you first need to be sure you have a >>> recent >>> version of iproute2 that supports the ipsec offload tag, probably >>> something >>> from ip 4.12 or newer would be best. I have a shell script that builds >>> up the appropriate commands for me, but here are the resulting commands >>> for all tcp traffic between 14.0.0.52 and 14.0.0.70: >>> >>> For the left side (14.0.0.52): >>> ip x p add dir out src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp tmpl \ >>> proto esp src 14.0.0.52 dst 14.0.0.70 spi 0x07 mode transport >>> reqid 0x07 >>> ip x p add dir in src 14.0.0.70/24 dst 14.0.0.52/24 proto tcp tmpl \ >>> proto esp dst 14.0.0.52 src 14.0.0.70 spi 0x07 mode transport >>> reqid 0x07 >>> ip x s add proto esp src 14.0.0.52 dst 14.0.0.70 spi 0x07 mode >>> transport \ >>> reqid 0x07 replay-window 32 \ >>> aead 'rfc4106(gcm(aes))' >>> 0x44434241343332312423222114131211f4f3f2f1 128 \ >>> sel src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp offload dev >>> eth4 dir out >>> ip x s add proto esp dst 14.0.0.52 src 14.0.0.70 spi 0x07 mode >>> transport \ >>> reqid 0x07 replay-window 32 \ >>> aead 'rfc4106(gcm(aes))' >>> 0x44434241343332312423222114131211f4f3f2f1 128 \ >>> sel src 14.0.0.70/24 dst 14.0.0.52/24 proto tcp offload dev >>> eth4 dir in >>> For the right side (14.0.0.70): >>> ip x p add dir out src 14.0.0.70/24 dst 14.0.0.52/24 proto tcp tmpl \ >>> proto esp src 14.0.0.70 dst 14.0.0.52 spi 0x07 mode transport >>> reqid 0x07 >>> ip x p add dir in src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp tmpl \ >>> proto esp dst 14.0.0.70 src 14.0.0.52 spi 0x07 mode transport >>> reqid 0x07 >>> ip x s add proto esp src 14.0.0.70 dst 14.0.0.52 spi 0x07 mode >>> transport \ >>> reqid 0x07 replay-window 32 \ >>> aead 'rfc4106(gcm(aes))' >>> 0x44434241343332312423222114131211f4f3f2f1 128 \ >>> sel src 14.0.0.70/24 dst 14.0.0.52/24 proto tcp offload dev >>> eth4 dir out >>> ip x s add proto esp dst 14.0.0.70 src 14.0.0.52 spi 0x07 mode >>> transport \ >>> reqid 0x07 replay-window 32 \ >>> aead 'rfc4106(gcm(aes))' >>> 0x44434241343332312423222114131211f4f3f2f1 128 \ >>> sel src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp offload dev >>> eth4 dir in >>> >>> In both cases, the command "ip x s flush ; ip x p flush" will clean >>> it all out and remove the offloads. >>> >>> Lastly, thanks to Alex Duyck for his early comments. >>> >>> Please see the individual patches for specific update info. >>> >>> v3: fixes after comments from those wonderfully pesky kbuild robots >>> v2: fixes after comments from Alex >>> >>> Shannon Nelson (10): >>> ixgbe: clean up ipsec defines >>> ixgbe: add ipsec register access routines >>> ixgbe: add ipsec engine start and stop routines >>> ixgbe: add ipsec data structures >>> ixgbe: add ipsec offload add and remove SA >>> ixgbe: restore offloaded SAs after a reset >>> ixgbe: process the Rx ipsec offload >>> ixgbe: process the Tx ipsec offload >>> ixgbe: ipsec offload stats >>> ixgbe: register ipsec offload with the xfrm subsystem >>> >>> drivers/net/ethernet/intel/ixgbe/Makefile | 1 + >>> drivers/net/ethernet/intel/ixgbe/ixgbe.h | 33 +- >>> drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c | 2 + >>> drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c | 923 >>> +++++++++++++++++++++++ >>> drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.h | 92 +++ >>> drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c | 4 +- >>> drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 39 +- >>> drivers/net/ethernet/intel/ixgbe/ixgbe_type.h | 22 +- >>> 8 files changed, 1093 insertions(+), 23 deletions(-) >>> create mode 100644 drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c >>> create mode 100644 drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.h >>> >> >> >