| Message ID | 20190729165918.92933-1-ppenkov.kernel@gmail.com |
|---|---|
| Headers | show |
| Series | Introduce a BPF helper to generate SYN cookies | expand |
On Mon, Jul 29, 2019 at 09:59:12AM -0700, Petar Penkov wrote: > From: Petar Penkov <ppenkov@google.com> > > This patch series introduces a BPF helper function that allows generating SYN > cookies from BPF. Currently, this helper is enabled at both the TC hook and the > XDP hook. > > The first two patches in the series add/modify several TCP helper functions to > allow for SKB-less operation, as is the case at the XDP hook. > > The third patch introduces the bpf_tcp_gen_syncookie helper function which > generates a SYN cookie for either XDP or TC programs. The return value of > this function contains both the MSS value, encoded in the cookie, and the > cookie itself. > > The last three patches sync tools/ and add a test. > > Performance evaluation: > I sent 10Mpps to a fixed port on a host with 2 10G bonded Mellanox 4 NICs from > random IPv6 source addresses. Without XDP I observed 7.2Mpps (syn-acks) being > sent out if the IPv6 packets carry 20 bytes of TCP options or 7.6Mpps if they > carry no options. If I attached a simple program that checks if a packet is > IPv6/TCP/SYN, looks up the socket, issues a cookie, and sends it back out after > swapping src/dest, recomputing the checksum, and setting the ACK flag, I > observed 10Mpps being sent back out. Is it 10m because trafic gen is 10m? What is cpu utilization at this rate? Is it cpu or nic limited if you crank up the syn flood? Original 7M with all cores or single core? The patch set looks good to me. I'd like Eric to review it one more time before applying.
On Mon, Jul 29, 2019 at 1:48 PM Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote: > > On Mon, Jul 29, 2019 at 09:59:12AM -0700, Petar Penkov wrote: > > From: Petar Penkov <ppenkov@google.com> > > > > This patch series introduces a BPF helper function that allows generating SYN > > cookies from BPF. Currently, this helper is enabled at both the TC hook and the > > XDP hook. > > > > The first two patches in the series add/modify several TCP helper functions to > > allow for SKB-less operation, as is the case at the XDP hook. > > > > The third patch introduces the bpf_tcp_gen_syncookie helper function which > > generates a SYN cookie for either XDP or TC programs. The return value of > > this function contains both the MSS value, encoded in the cookie, and the > > cookie itself. > > > > The last three patches sync tools/ and add a test. > > > > Performance evaluation: > > I sent 10Mpps to a fixed port on a host with 2 10G bonded Mellanox 4 NICs from > > random IPv6 source addresses. Without XDP I observed 7.2Mpps (syn-acks) being > > sent out if the IPv6 packets carry 20 bytes of TCP options or 7.6Mpps if they > > carry no options. If I attached a simple program that checks if a packet is > > IPv6/TCP/SYN, looks up the socket, issues a cookie, and sends it back out after > > swapping src/dest, recomputing the checksum, and setting the ACK flag, I > > observed 10Mpps being sent back out. > Thank you for reviewing this so quickly! > Is it 10m because trafic gen is 10m? Yes, I believe so. > What is cpu utilization at this rate? > Is it cpu or nic limited if you crank up the syn flood? > Original 7M with all cores or single core? My receiver was configured with 16rx queues and 16 cores. 7M all cores are at 100% so I believe this case is CPU limited. At XDP, all cores are at roughly 40%. I couldn't reliably generate higher SYN flood rate than that, and the highest numbers I could see for XDP did not go past 10.65Mpps with ~42% utilization on each core. I think I am hitting a NIC limit here since the CPUs are free. > > The patch set looks good to me. > I'd like Eric to review it one more time before applying. >
On Mon, Jul 29, 2019 at 4:46 PM Petar Penkov <ppenkov@google.com> wrote: >> > > What is cpu utilization at this rate? > > Is it cpu or nic limited if you crank up the syn flood? > > Original 7M with all cores or single core? > My receiver was configured with 16rx queues and 16 cores. 7M all cores > are at 100% so I believe this case is CPU limited. At XDP, all cores > are at roughly 40%. I couldn't reliably generate higher SYN flood rate > than that, and the highest numbers I could see for XDP did not go past > 10.65Mpps with ~42% utilization on each core. I think I am hitting a > NIC limit here since the CPUs are free. Applied. Thanks!
From: Petar Penkov <ppenkov@google.com> This patch series introduces a BPF helper function that allows generating SYN cookies from BPF. Currently, this helper is enabled at both the TC hook and the XDP hook. The first two patches in the series add/modify several TCP helper functions to allow for SKB-less operation, as is the case at the XDP hook. The third patch introduces the bpf_tcp_gen_syncookie helper function which generates a SYN cookie for either XDP or TC programs. The return value of this function contains both the MSS value, encoded in the cookie, and the cookie itself. The last three patches sync tools/ and add a test. Performance evaluation: I sent 10Mpps to a fixed port on a host with 2 10G bonded Mellanox 4 NICs from random IPv6 source addresses. Without XDP I observed 7.2Mpps (syn-acks) being sent out if the IPv6 packets carry 20 bytes of TCP options or 7.6Mpps if they carry no options. If I attached a simple program that checks if a packet is IPv6/TCP/SYN, looks up the socket, issues a cookie, and sends it back out after swapping src/dest, recomputing the checksum, and setting the ACK flag, I observed 10Mpps being sent back out. Changes since v1: 1/ Added performance numbers to the cover letter 2/ Patch 2: Refactored a bit to fix compilation issues 3/ Patch 3: Changed ENOTSUPP to EOPNOTSUPP at Toke's suggestion Changes since RFC: 1/ Cookie is returned in host order at Alexei's suggestion 2/ If cookies are not enabled via a sysctl, the helper function returns -ENOENT instead of -EINVAL at Lorenz's suggestion 3/ Fixed documentation to properly reflect that MSS is 16 bits at Lorenz's suggestion 4/ BPF helper requires TCP length to match ->doff field, rather than to simply be no more than 20 bytes at Eric and Alexei's suggestion 5/ Packet type is looked up from the packet version field, rather than from the socket. v4 packets are rejected on v6-only sockets but should work with dual stack listeners at Eric's suggestion 6/ Removed unnecessary `net` argument from helper function in patch 2 at Lorenz's suggestion 7/ Changed test to only pass MSS option so we can convince the verifier that the memory access is not out of bounds Note that 7/ below illustrates the verifier might need to be extended to allow passing a variable tcph->doff to the helper function like below: __u32 thlen = tcph->doff * 4; if (thlen < sizeof(*tcph)) return; __s64 cookie = bpf_tcp_gen_syncookie(sk, ipv4h, 20, tcph, thlen); Petar Penkov (6): tcp: tcp_syn_flood_action read port from socket tcp: add skb-less helpers to retrieve SYN cookie bpf: add bpf_tcp_gen_syncookie helper bpf: sync bpf.h to tools/ selftests/bpf: bpf_tcp_gen_syncookie->bpf_helpers selftests/bpf: add test for bpf_tcp_gen_syncookie include/net/tcp.h | 10 +++ include/uapi/linux/bpf.h | 30 ++++++- net/core/filter.c | 73 +++++++++++++++++ net/ipv4/tcp_input.c | 81 +++++++++++++++++-- net/ipv4/tcp_ipv4.c | 15 ++++ net/ipv6/tcp_ipv6.c | 15 ++++ tools/include/uapi/linux/bpf.h | 37 ++++++++- tools/testing/selftests/bpf/bpf_helpers.h | 3 + .../bpf/progs/test_tcp_check_syncookie_kern.c | 48 +++++++++-- .../selftests/bpf/test_tcp_check_syncookie.sh | 3 + .../bpf/test_tcp_check_syncookie_user.c | 61 ++++++++++++-- 11 files changed, 354 insertions(+), 22 deletions(-)