From patchwork Mon Jun 3 13:19:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Fijalkowski X-Patchwork-Id: 1109256 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="UUFHSAoo"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 45HbHV3zKLz9s7h for ; Mon, 3 Jun 2019 23:19:18 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728505AbfFCNTR (ORCPT ); Mon, 3 Jun 2019 09:19:17 -0400 Received: from mail-lj1-f196.google.com ([209.85.208.196]:41118 "EHLO mail-lj1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727403AbfFCNTQ (ORCPT ); Mon, 3 Jun 2019 09:19:16 -0400 Received: by mail-lj1-f196.google.com with SMTP id s21so5520367lji.8 for ; Mon, 03 Jun 2019 06:19:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=bp+nVoB4HtDC82RcKiB+JgTW5jisIiXnKI12mHtf1Uc=; b=UUFHSAoofzWitUSdDlbj/1Ipu/QHWdiRLAx/izKKfJiNGJdUSAQAqX5dARkyZJigkT /G8X0IIGlQzs5YnWSucUbO+08ibhaeDBk7L4si9uLLd65PSMHMv1miLAtm+JJg6xB7JK Oq0KaQpjuTU9zM76qzdrTNG0cPMSEhBbx2Mw8/oZfVnYNZf9eCuSkzgl1rroBqms8d2x Y6eKix2ycmG9tH70JMZIv24stXlTY/cUh2d3zMc3vrVi6JJuGzioMAtLS6EeOf1SS6Pu ct4KLtIKnn608agsCK6+7990KhthSiL7sCzchCF83LKBL/7osLe9lyWJ+Y5dg3AImT8l rGpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=bp+nVoB4HtDC82RcKiB+JgTW5jisIiXnKI12mHtf1Uc=; b=by6jhZ/PBZAL/1ABa+lwhzCMzQzJyZkXgyOwzlH0672HCOHk88i7AhVrTQlaIzhCiY /i4Hja/Fdeg40n2JggMpdxDpZmoQwERpFhTuI22o+27HCJaGtFXY+iSYRybLNvvHUsJm WN6KMo/yaFwk0A6DdyvSVQO7qwKXY1+Jc7DB91cj6JZd+cDes+00WILB7QbtPXarR8Ny oFrtOVlqN4zzPVDHQ0kfxrNNOKo337XgoRGrmMTPKpgsvYI8Y2C0HZmd6NMUoziS1+g0 O9e0OmVlNDRN8JvzHxlyXyFJY9psFOu63f8OlX0oXgo+xKrvwArLYYU4k9DVybfsc5X7 b5SA== X-Gm-Message-State: APjAAAW0bL4G+FCsQFAyubk0DE0wri7Vr4oDyLKICQSzn44tu1UA2Url 1FgYt0AQmjQpl402L1Ca4tA= X-Google-Smtp-Source: APXvYqzl4UGy7VjsQAdf6pDOupzzT6kT7apYaeaVa/EBHG2PQojkFtRu2YOxFsFeiShmLVbNIMvz/w== X-Received: by 2002:a2e:4710:: with SMTP id u16mr14239223lja.41.1559567954943; Mon, 03 Jun 2019 06:19:14 -0700 (PDT) Received: from localhost.localdomain (host-185-93-94-143.ip-point.pl. [185.93.94.143]) by smtp.gmail.com with ESMTPSA id 20sm577808ljf.21.2019.06.03.06.19.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 03 Jun 2019 06:19:14 -0700 (PDT) From: Maciej Fijalkowski X-Google-Original-From: Maciej Fijalkowski To: magnus.karlsson@intel.com, bjorn.topel@intel.com, netdev@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, jakub.kicinski@netronome.com, jonathan.lemon@gmail.com, songliubraving@fb.com Subject: [RFC PATCH bpf-next 1/4] libbpf: fill the AF_XDP fill queue before bind() call Date: Mon, 3 Jun 2019 15:19:04 +0200 Message-Id: <20190603131907.13395-2-maciej.fijalkowski@intel.com> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20190603131907.13395-1-maciej.fijalkowski@intel.com> References: <20190603131907.13395-1-maciej.fijalkowski@intel.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Let's get into the driver via ndo_bpf with command set to XDP_SETUP_UMEM with fill queue that already contains some available entries that can be used by Rx driver rings. Things worked in such way on old version of xdpsock (that lacked libbpf support) and there's no particular reason for having this preparation done after bind(). Signed-off-by: Maciej Fijalkowski Signed-off-by: Krzysztof Kazimierczak --- samples/bpf/xdpsock_user.c | 15 --------------- tools/lib/bpf/xsk.c | 19 ++++++++++++++++++- 2 files changed, 18 insertions(+), 16 deletions(-) diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c index d08ee1ab7bb4..e9dceb09b6d1 100644 --- a/samples/bpf/xdpsock_user.c +++ b/samples/bpf/xdpsock_user.c @@ -296,8 +296,6 @@ static struct xsk_socket_info *xsk_configure_socket(struct xsk_umem_info *umem) struct xsk_socket_config cfg; struct xsk_socket_info *xsk; int ret; - u32 idx; - int i; xsk = calloc(1, sizeof(*xsk)); if (!xsk) @@ -318,19 +316,6 @@ static struct xsk_socket_info *xsk_configure_socket(struct xsk_umem_info *umem) if (ret) exit_with_error(-ret); - ret = xsk_ring_prod__reserve(&xsk->umem->fq, - XSK_RING_PROD__DEFAULT_NUM_DESCS, - &idx); - if (ret != XSK_RING_PROD__DEFAULT_NUM_DESCS) - exit_with_error(-ret); - for (i = 0; - i < XSK_RING_PROD__DEFAULT_NUM_DESCS * - XSK_UMEM__DEFAULT_FRAME_SIZE; - i += XSK_UMEM__DEFAULT_FRAME_SIZE) - *xsk_ring_prod__fill_addr(&xsk->umem->fq, idx++) = i; - xsk_ring_prod__submit(&xsk->umem->fq, - XSK_RING_PROD__DEFAULT_NUM_DESCS); - return xsk; } diff --git a/tools/lib/bpf/xsk.c b/tools/lib/bpf/xsk.c index 38667b62f1fe..57dda1389870 100644 --- a/tools/lib/bpf/xsk.c +++ b/tools/lib/bpf/xsk.c @@ -529,7 +529,8 @@ int xsk_socket__create(struct xsk_socket **xsk_ptr, const char *ifname, struct xdp_mmap_offsets off; struct xsk_socket *xsk; socklen_t optlen; - int err; + int err, i; + u32 idx; if (!umem || !xsk_ptr || !rx || !tx) return -EFAULT; @@ -632,6 +633,22 @@ int xsk_socket__create(struct xsk_socket **xsk_ptr, const char *ifname, } xsk->tx = tx; + err = xsk_ring_prod__reserve(umem->fill, + XSK_RING_PROD__DEFAULT_NUM_DESCS, + &idx); + if (err != XSK_RING_PROD__DEFAULT_NUM_DESCS) { + err = -errno; + goto out_mmap_tx; + } + + for (i = 0; + i < XSK_RING_PROD__DEFAULT_NUM_DESCS * + XSK_UMEM__DEFAULT_FRAME_SIZE; + i += XSK_UMEM__DEFAULT_FRAME_SIZE) + *xsk_ring_prod__fill_addr(umem->fill, idx++) = i; + xsk_ring_prod__submit(umem->fill, + XSK_RING_PROD__DEFAULT_NUM_DESCS); + sxdp.sxdp_family = PF_XDP; sxdp.sxdp_ifindex = xsk->ifindex; sxdp.sxdp_queue_id = xsk->queue_id; From patchwork Mon Jun 3 13:19:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Fijalkowski X-Patchwork-Id: 1109257 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="n5DYoJfo"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 45HbHX4YZ1z9s1c for ; Mon, 3 Jun 2019 23:19:20 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727823AbfFCNTT (ORCPT ); Mon, 3 Jun 2019 09:19:19 -0400 Received: from mail-lj1-f194.google.com ([209.85.208.194]:43578 "EHLO mail-lj1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726336AbfFCNTR (ORCPT ); Mon, 3 Jun 2019 09:19:17 -0400 Received: by mail-lj1-f194.google.com with SMTP id 16so2575828ljv.10 for ; Mon, 03 Jun 2019 06:19:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=/C1LT1bx3g7WPx/wKiHahB+UY59S2DI+l7zWgRA5mDU=; b=n5DYoJfo1fWUEjlq+bJ8mx76Y6mIqLD1a9es8GAd/O2Vs52MUrskIgT3VnKyYJDGv4 9Auk+V956RIBnngMPo+qG7EJewtzLYeFAeJu9jqCBDHHsMnjhEN/XiHHNj58qDLYP5MS bCVLYVH55wcRGfK3hy1hNZDLyGEXaFrtXNRNOqfvPIQLEj/JgU4uWX5Dxv1p7xw3b5RN Szv0PRKsytBEi/XPHMPDZzNUPgQIwfcOFnQgiJLYgYr/EpTC3UuMPo1CASSqLQCKMQnd m9cmWrQjl+5O+zu+YH/AO0nVjwetI+dXtPgMqI5FeeUjK+f6pYusoSZ2pxkSBBYgCQTH ZrtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=/C1LT1bx3g7WPx/wKiHahB+UY59S2DI+l7zWgRA5mDU=; b=S0gaO30FdZ5gM/CAWf1UiRYXaK+Kf+zcTpI6BkuBJCw+d71zKZhlXVRWFh+3XZJxMS V4oquFzas8b38m7EU2ykzFziWbSQ2sA9DlY2xm8UCl6ks38/TUxgkUUTuQbhtbIF0oJH NUeMCBsgT9EN/Y9XhR2dN3EfIdymYL8lvHM74uVxoofDg/0cdvEdPWR+zAHs4ezMMmB4 Ec+rPA81TloDCNMoF5rQlasND9sEXQut4M9RBa/r4hHF0ihS+8i5gM8WQQCql+zAOwgu k2eW3npGptBwg8/5cy1T0NKyIPKEr65xLLTgA+DtJrF1rF36Kns8tNgBqw4ZyFT2JAgW 85GA== X-Gm-Message-State: APjAAAXJUHUztGBAFzwDDkPFsGU6kPJpCb369bwh0uEglQscRF6xloJ3 No2DQQB03FnfXuqJCdjnVe0= X-Google-Smtp-Source: APXvYqyl27RsMHsn8uAd4WywLvEDh6D6FtEIJxrmY2VjfLbm2cNjed+ATHwQDYltvkrWsWhf4cKLTQ== X-Received: by 2002:a2e:3a1a:: with SMTP id h26mr13749921lja.156.1559567956088; Mon, 03 Jun 2019 06:19:16 -0700 (PDT) Received: from localhost.localdomain (host-185-93-94-143.ip-point.pl. [185.93.94.143]) by smtp.gmail.com with ESMTPSA id 20sm577808ljf.21.2019.06.03.06.19.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 03 Jun 2019 06:19:15 -0700 (PDT) From: Maciej Fijalkowski X-Google-Original-From: Maciej Fijalkowski To: magnus.karlsson@intel.com, bjorn.topel@intel.com, netdev@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, jakub.kicinski@netronome.com, jonathan.lemon@gmail.com, songliubraving@fb.com Subject: [RFC PATCH bpf-next 2/4] libbpf: check for channels.max_{t, r}x in xsk_get_max_queues Date: Mon, 3 Jun 2019 15:19:05 +0200 Message-Id: <20190603131907.13395-3-maciej.fijalkowski@intel.com> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20190603131907.13395-1-maciej.fijalkowski@intel.com> References: <20190603131907.13395-1-maciej.fijalkowski@intel.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org When it comes down to ethtool's get channels API, various drivers are reporting the queue count in two ways - they are setting max_combined or max_tx/max_rx fields. When creating the eBPF maps for xsk socket, this API is used so that we have an entries in maps per each queue. In case where driver (mlx4, ice) reports queues in max_tx/max_rx, we end up with eBPF maps with single entries, so it's not possible to attach an AF_XDP socket onto queue other than 0 - xsk_set_bpf_maps() would try to call bpf_map_update_elem() with key set to xsk->queue_id. To fix this, let's look for channels.max_{t,r}x as well in xsk_get_max_queues. Signed-off-by: Maciej Fijalkowski --- tools/lib/bpf/xsk.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/tools/lib/bpf/xsk.c b/tools/lib/bpf/xsk.c index 57dda1389870..514ab3fb06f4 100644 --- a/tools/lib/bpf/xsk.c +++ b/tools/lib/bpf/xsk.c @@ -339,21 +339,23 @@ static int xsk_get_max_queues(struct xsk_socket *xsk) ifr.ifr_data = (void *)&channels; strncpy(ifr.ifr_name, xsk->ifname, IFNAMSIZ); err = ioctl(fd, SIOCETHTOOL, &ifr); - if (err && errno != EOPNOTSUPP) { - ret = -errno; - goto out; - } + close(fd); + + if (err && errno != EOPNOTSUPP) + return -errno; - if (channels.max_combined == 0 || errno == EOPNOTSUPP) + if (channels.max_combined) + ret = channels.max_combined; + else if (channels.max_rx && channels.max_tx) + ret = min(channels.max_rx, channels.max_tx); + else if (channels.max_combined == 0 || errno == EOPNOTSUPP) /* If the device says it has no channels, then all traffic * is sent to a single stream, so max queues = 1. */ ret = 1; else - ret = channels.max_combined; + ret = -1; -out: - close(fd); return ret; } From patchwork Mon Jun 3 13:19:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Fijalkowski X-Patchwork-Id: 1109259 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="f58EisfL"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 45HbHb67kCz9s7h for ; Mon, 3 Jun 2019 23:19:23 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728529AbfFCNTW (ORCPT ); Mon, 3 Jun 2019 09:19:22 -0400 Received: from mail-lj1-f194.google.com ([209.85.208.194]:42465 "EHLO mail-lj1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727403AbfFCNTU (ORCPT ); Mon, 3 Jun 2019 09:19:20 -0400 Received: by mail-lj1-f194.google.com with SMTP id t28so5041039lje.9 for ; Mon, 03 Jun 2019 06:19:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=daJ7DQFGqdxbmrFgyxA8+fMwjZ7EZmTJaFWyZhbuPPI=; b=f58EisfL3R0On8w6XI8fk4pktzS2e2WC0fdh1q5/SSwC2j0Zv8D68ZtqZnXqPxIfwJ JX7SuqkyniOJe2i5WjRHkIYpw8KLbsCoRCiUGRsvBBhHmbCQvIwHozzFKfHjewXVgy+B RuOKXu5N0H/lJXyVTnI9cX2QdcE79v1woS0p7CyyT8oz5bt7g4SLxcWiEB++lO4CTgPw EJn6sek+eEumPRvx8MCH1hg8zUUfVFVjP3TtgS1KCmNeDJQyk1Zeqslzew0Uwo3k/cjp IH4TIfsXkg7WqqWczf3fyQFftywyg2lM36xINDphfL8I/qYbRBXkfl6+wDTKr02iILm4 3Vpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=daJ7DQFGqdxbmrFgyxA8+fMwjZ7EZmTJaFWyZhbuPPI=; b=DfaYWNPLkSQ67IVBzbMmAQTLG7LOhkenFI6YEXxMCAnGOOC55u+TXor04h4zIwQHqA 3QABRZrfPtwUwlKXmSOdRCBFrqAHEFXB7ls6REZ/coT+hH6woFbw7JjZpZn4P/lt7r+Z eENGJX7x1IsQxNLxk/4QYoOB9/YnRzJ1nRsSX6DB2CoLYmyv1PiNL24FWh94BQt1w6s+ XKU188C27cN1MkEht0RXnAKeCArPTcerwjpgKccFXs82mjKpstdNATgSu5bdxFFneZn/ 6KpKfNJduPVt/yU2YkvqrVGMXm3YCW47x0i6lNYd7drSdY+/JzhiIUuHFgOmlI/7W8x1 75cQ== X-Gm-Message-State: APjAAAW11oLONMSf1PWO7hhHTjpvkJ+z1pKpMD2fDdouZLOXVjMEgnIr TIpXMKb2aGqwxx30KGY5sa0= X-Google-Smtp-Source: APXvYqzgs4funjynq2k3D2uouY4yWO9HADzQyLUTMgTEaoIeGg+C/DZkcVqxZ2MoGmr1fsiMq/rMbg== X-Received: by 2002:a2e:998b:: with SMTP id w11mr6493752lji.179.1559567957055; Mon, 03 Jun 2019 06:19:17 -0700 (PDT) Received: from localhost.localdomain (host-185-93-94-143.ip-point.pl. [185.93.94.143]) by smtp.gmail.com with ESMTPSA id 20sm577808ljf.21.2019.06.03.06.19.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 03 Jun 2019 06:19:16 -0700 (PDT) From: Maciej Fijalkowski X-Google-Original-From: Maciej Fijalkowski To: magnus.karlsson@intel.com, bjorn.topel@intel.com, netdev@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, jakub.kicinski@netronome.com, jonathan.lemon@gmail.com, songliubraving@fb.com Subject: [RFC PATCH bpf-next 3/4] libbpf: move xdp program removal to libbpf Date: Mon, 3 Jun 2019 15:19:06 +0200 Message-Id: <20190603131907.13395-4-maciej.fijalkowski@intel.com> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20190603131907.13395-1-maciej.fijalkowski@intel.com> References: <20190603131907.13395-1-maciej.fijalkowski@intel.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Since xsk support in libbpf loads the xdp program interface, make it also responsible for its removal. Store the prog id in xsk_socket_config so when removing the program we are still able to compare the current program id with the id from the attachment time and make a decision onward. While at it, remove the socket/umem in xdpsock's error path. Signed-off-by: Maciej Fijalkowski --- samples/bpf/xdpsock_user.c | 33 ++++++++++----------------------- tools/lib/bpf/xsk.c | 32 ++++++++++++++++++++++++++++++++ tools/lib/bpf/xsk.h | 1 + 3 files changed, 43 insertions(+), 23 deletions(-) diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c index e9dceb09b6d1..123862b16dd4 100644 --- a/samples/bpf/xdpsock_user.c +++ b/samples/bpf/xdpsock_user.c @@ -68,7 +68,6 @@ static int opt_queue; static int opt_poll; static int opt_interval = 1; static u32 opt_xdp_bind_flags; -static __u32 prog_id; struct xsk_umem_info { struct xsk_ring_prod fq; @@ -170,22 +169,6 @@ static void *poller(void *arg) return NULL; } -static void remove_xdp_program(void) -{ - __u32 curr_prog_id = 0; - - if (bpf_get_link_xdp_id(opt_ifindex, &curr_prog_id, opt_xdp_flags)) { - printf("bpf_get_link_xdp_id failed\n"); - exit(EXIT_FAILURE); - } - if (prog_id == curr_prog_id) - bpf_set_link_xdp_fd(opt_ifindex, -1, opt_xdp_flags); - else if (!curr_prog_id) - printf("couldn't find a prog id on a given interface\n"); - else - printf("program on interface changed, not removing\n"); -} - static void int_exit(int sig) { struct xsk_umem *umem = xsks[0]->umem->umem; @@ -195,7 +178,6 @@ static void int_exit(int sig) dump_stats(); xsk_socket__delete(xsks[0]->xsk); (void)xsk_umem__delete(umem); - remove_xdp_program(); exit(EXIT_SUCCESS); } @@ -206,7 +188,16 @@ static void __exit_with_error(int error, const char *file, const char *func, fprintf(stderr, "%s:%s:%i: errno: %d/\"%s\"\n", file, func, line, error, strerror(error)); dump_stats(); - remove_xdp_program(); + + if (xsks[0]->xsk) + xsk_socket__delete(xsks[0]->xsk); + + if (xsks[0]->umem) { + struct xsk_umem *umem = xsks[0]->umem->umem; + + (void)xsk_umem__delete(umem); + } + exit(EXIT_FAILURE); } @@ -312,10 +303,6 @@ static struct xsk_socket_info *xsk_configure_socket(struct xsk_umem_info *umem) if (ret) exit_with_error(-ret); - ret = bpf_get_link_xdp_id(opt_ifindex, &prog_id, opt_xdp_flags); - if (ret) - exit_with_error(-ret); - return xsk; } diff --git a/tools/lib/bpf/xsk.c b/tools/lib/bpf/xsk.c index 514ab3fb06f4..e28bedb0b078 100644 --- a/tools/lib/bpf/xsk.c +++ b/tools/lib/bpf/xsk.c @@ -259,6 +259,8 @@ int xsk_umem__create(struct xsk_umem **umem_ptr, void *umem_area, __u64 size, static int xsk_load_xdp_prog(struct xsk_socket *xsk) { static const int log_buf_size = 16 * 1024; + struct bpf_prog_info info = {}; + __u32 info_len = sizeof(info); char log_buf[log_buf_size]; int err, prog_fd; @@ -321,6 +323,14 @@ static int xsk_load_xdp_prog(struct xsk_socket *xsk) return err; } + err = bpf_obj_get_info_by_fd(prog_fd, &info, &info_len); + if (err) { + pr_warning("can't get prog info - %s\n", strerror(errno)); + close(prog_fd); + return err; + } + xsk->config.prog_id = info.id; + xsk->prog_fd = prog_fd; return 0; } @@ -483,6 +493,25 @@ static int xsk_set_bpf_maps(struct xsk_socket *xsk) return err; } +static void xsk_remove_xdp_prog(struct xsk_socket *xsk) +{ + __u32 prog_id = xsk->config.prog_id; + __u32 curr_prog_id = 0; + int err; + + err = bpf_get_link_xdp_id(xsk->ifindex, &curr_prog_id, + xsk->config.xdp_flags); + if (err) + return; + + if (prog_id == curr_prog_id) + bpf_set_link_xdp_fd(xsk->ifindex, -1, xsk->config.xdp_flags); + else if (!curr_prog_id) + pr_warning("couldn't find a prog id on a given interface\n"); + else + pr_warning("program on interface changed, not removing\n"); +} + static int xsk_setup_xdp_prog(struct xsk_socket *xsk) { __u32 prog_id = 0; @@ -506,6 +535,7 @@ static int xsk_setup_xdp_prog(struct xsk_socket *xsk) err = xsk_lookup_bpf_maps(xsk); if (err) goto out_load; + xsk->config.prog_id = prog_id; } err = xsk_set_bpf_maps(xsk); @@ -744,6 +774,8 @@ void xsk_socket__delete(struct xsk_socket *xsk) } + xsk_remove_xdp_prog(xsk); + xsk->umem->refcount--; /* Do not close an fd that also has an associated umem connected * to it. diff --git a/tools/lib/bpf/xsk.h b/tools/lib/bpf/xsk.h index 82ea71a0f3ec..e1b23e9432c9 100644 --- a/tools/lib/bpf/xsk.h +++ b/tools/lib/bpf/xsk.h @@ -186,6 +186,7 @@ struct xsk_socket_config { __u32 tx_size; __u32 libbpf_flags; __u32 xdp_flags; + __u32 prog_id; __u16 bind_flags; }; From patchwork Mon Jun 3 13:19:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Fijalkowski X-Patchwork-Id: 1109258 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="kVwr+JpN"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 45HbHb2pFNz9s1c for ; Mon, 3 Jun 2019 23:19:22 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728523AbfFCNTV (ORCPT ); Mon, 3 Jun 2019 09:19:21 -0400 Received: from mail-lf1-f65.google.com ([209.85.167.65]:41017 "EHLO mail-lf1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728512AbfFCNTU (ORCPT ); Mon, 3 Jun 2019 09:19:20 -0400 Received: by mail-lf1-f65.google.com with SMTP id 136so2045876lfa.8 for ; Mon, 03 Jun 2019 06:19:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=LeBa2Av8Q3e3e0gop1RqyOgipHnUTdOBWLST+LfMBZo=; b=kVwr+JpNPbnORWl2440+rFIkNFkxHijSXQ8N0yKhmdjM74znKnUapTv/KM89fUl10j Kp2QnBwdZZCQ1TDD5q+3ytVy+btOQPLCqOCtRRBXeDg+zT574J2frXm1yl87auU/KG6v 0/aDB+qDl0o28yhBT+p8N8W/h/OEcM1tuKXyg1JRcWJAL7ukNcKvkkvmPgpRpZoM+mY5 y5JuIkAoitn7R8Xq80QPC5mX9i7Mm5meR4XX8BuXARkuVEgbiDoKICnaSkGYOjPzPdV8 GmSgmTnj4W6/4DIHvF9BkTpx1bSsK9iMxHq57ubWxUZxFEs5I2BPFr6YuU9p10/K3sKy c5/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=LeBa2Av8Q3e3e0gop1RqyOgipHnUTdOBWLST+LfMBZo=; b=PHMWSGd2PHvPPRr4+SkcIsoWGQ2OIHCciIAtkBI0dRAy5DSBorHVp6pI/MuneGsl/J Hbp8FeX6qh1QGuYkB5FskJGW+WEG38ct1uNS4k5mL3RjF+kPyvj5uNiY6wUwbC5cRffS Iucb4tWLhzc5Uae/oGZdxpnUZ4OLphrgz8er8shpWI56eP46c7upOAjKh7uNG2mxabE7 S1Gr9Bz11Yz7ri1v5BXQx3sDeo+K0Cb7AYN2NW/P8tvtDwqRVbEBA7Clq2A/VzthSx/i IX6ZR8VC9D+C9fOHc7BoEICarTDq+t1KjiBjjQD5hikW6e65AdlmVmVfUYR++du6E3yJ CZyw== X-Gm-Message-State: APjAAAX/vBiwjcGcoDFyMC2DkSWN6TkKBU/6XeabTP7duaaBe7/Tr5CR qUVUW96gBHeT4LOVlKXkRKUaeaHF X-Google-Smtp-Source: APXvYqx6clVHDRhpd5nlJnpgzDsnshvEgXl3IxVdDDguCQ7RA0xDp+vzIdLdHnR+Bqwkp8d2dhhr/Q== X-Received: by 2002:ac2:5231:: with SMTP id i17mr13962040lfl.39.1559567958183; Mon, 03 Jun 2019 06:19:18 -0700 (PDT) Received: from localhost.localdomain (host-185-93-94-143.ip-point.pl. [185.93.94.143]) by smtp.gmail.com with ESMTPSA id 20sm577808ljf.21.2019.06.03.06.19.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 03 Jun 2019 06:19:17 -0700 (PDT) From: Maciej Fijalkowski X-Google-Original-From: Maciej Fijalkowski To: magnus.karlsson@intel.com, bjorn.topel@intel.com, netdev@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, jakub.kicinski@netronome.com, jonathan.lemon@gmail.com, songliubraving@fb.com Subject: [RFC PATCH bpf-next 4/4] libbpf: don't remove eBPF resources when other xsks are present Date: Mon, 3 Jun 2019 15:19:07 +0200 Message-Id: <20190603131907.13395-5-maciej.fijalkowski@intel.com> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20190603131907.13395-1-maciej.fijalkowski@intel.com> References: <20190603131907.13395-1-maciej.fijalkowski@intel.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org In case where multiple xsk sockets are attached to a single interface and one of them gets detached, the eBPF maps and program are removed. This should not happen as the rest of xsksocks are still using these resources. In order to fix that, let's have an additional eBPF map with a single entry that will be used as a xsks count. During the xsk_socket__delete, remove the resources only when this count is equal to 0. This map is not being accessed from eBPF program, so the verifier is not associating it with the prog, which in turn makes bpf_obj_get_info_by_fd not reporting this map in nr_map_ids field of struct bpf_prog_info. The described behaviour brings the need to have this map pinned, so in case when socket is being created and the libbpf detects the presence of bpf resources, it will be able to access that map. Signed-off-by: Maciej Fijalkowski --- tools/lib/bpf/xsk.c | 59 +++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 51 insertions(+), 8 deletions(-) diff --git a/tools/lib/bpf/xsk.c b/tools/lib/bpf/xsk.c index e28bedb0b078..88d2c931ad14 100644 --- a/tools/lib/bpf/xsk.c +++ b/tools/lib/bpf/xsk.c @@ -44,6 +44,8 @@ #define PF_XDP AF_XDP #endif +#define XSKS_CNT_MAP_PATH "/sys/fs/bpf/xsks_cnt_map" + struct xsk_umem { struct xsk_ring_prod *fill; struct xsk_ring_cons *comp; @@ -65,6 +67,7 @@ struct xsk_socket { int prog_fd; int qidconf_map_fd; int xsks_map_fd; + int xsks_cnt_map_fd; __u32 queue_id; char ifname[IFNAMSIZ]; }; @@ -372,7 +375,7 @@ static int xsk_get_max_queues(struct xsk_socket *xsk) static int xsk_create_bpf_maps(struct xsk_socket *xsk) { int max_queues; - int fd; + int fd, ret; max_queues = xsk_get_max_queues(xsk); if (max_queues < 0) @@ -392,6 +395,24 @@ static int xsk_create_bpf_maps(struct xsk_socket *xsk) } xsk->xsks_map_fd = fd; + fd = bpf_create_map_name(BPF_MAP_TYPE_ARRAY, "xsks_cnt_map", + sizeof(int), sizeof(int), 1, 0); + if (fd < 0) { + close(xsk->qidconf_map_fd); + close(xsk->xsks_map_fd); + return fd; + } + + ret = bpf_obj_pin(fd, XSKS_CNT_MAP_PATH); + if (ret < 0) { + pr_warning("pinning map failed; is bpffs mounted?\n"); + close(xsk->qidconf_map_fd); + close(xsk->xsks_map_fd); + close(fd); + return ret; + } + xsk->xsks_cnt_map_fd = fd; + return 0; } @@ -456,8 +477,10 @@ static int xsk_lookup_bpf_maps(struct xsk_socket *xsk) close(fd); } + xsk->xsks_cnt_map_fd = bpf_obj_get(XSKS_CNT_MAP_PATH); err = 0; - if (xsk->qidconf_map_fd < 0 || xsk->xsks_map_fd < 0) { + if (xsk->qidconf_map_fd < 0 || xsk->xsks_map_fd < 0 || + xsk->xsks_cnt_map_fd < 0) { err = -ENOENT; xsk_delete_bpf_maps(xsk); } @@ -467,17 +490,25 @@ static int xsk_lookup_bpf_maps(struct xsk_socket *xsk) return err; } -static void xsk_clear_bpf_maps(struct xsk_socket *xsk) +static void xsk_clear_bpf_maps(struct xsk_socket *xsk, long *xsks_cnt_ptr) { + long xsks_cnt, key = 0; int qid = false; bpf_map_update_elem(xsk->qidconf_map_fd, &xsk->queue_id, &qid, 0); bpf_map_delete_elem(xsk->xsks_map_fd, &xsk->queue_id); + bpf_map_lookup_elem(xsk->xsks_cnt_map_fd, &key, &xsks_cnt); + if (xsks_cnt) + xsks_cnt--; + bpf_map_update_elem(xsk->xsks_cnt_map_fd, &key, &xsks_cnt, 0); + if (xsks_cnt_ptr) + *xsks_cnt_ptr = xsks_cnt; } static int xsk_set_bpf_maps(struct xsk_socket *xsk) { int qid = true, fd = xsk->fd, err; + long xsks_cnt, key = 0; err = bpf_map_update_elem(xsk->qidconf_map_fd, &xsk->queue_id, &qid, 0); if (err) @@ -487,9 +518,18 @@ static int xsk_set_bpf_maps(struct xsk_socket *xsk) if (err) goto out; + err = bpf_map_lookup_elem(xsk->xsks_cnt_map_fd, &key, &xsks_cnt); + if (err) + goto out; + + xsks_cnt++; + err = bpf_map_update_elem(xsk->xsks_cnt_map_fd, &key, &xsks_cnt, 0); + if (err) + goto out; + return 0; out: - xsk_clear_bpf_maps(xsk); + xsk_clear_bpf_maps(xsk, NULL); return err; } @@ -752,13 +792,18 @@ void xsk_socket__delete(struct xsk_socket *xsk) size_t desc_sz = sizeof(struct xdp_desc); struct xdp_mmap_offsets off; socklen_t optlen; + long xsks_cnt; int err; if (!xsk) return; - xsk_clear_bpf_maps(xsk); - xsk_delete_bpf_maps(xsk); + xsk_clear_bpf_maps(xsk, &xsks_cnt); + unlink(XSKS_CNT_MAP_PATH); + if (!xsks_cnt) { + xsk_delete_bpf_maps(xsk); + xsk_remove_xdp_prog(xsk); + } optlen = sizeof(off); err = getsockopt(xsk->fd, SOL_XDP, XDP_MMAP_OFFSETS, &off, &optlen); @@ -774,8 +819,6 @@ void xsk_socket__delete(struct xsk_socket *xsk) } - xsk_remove_xdp_prog(xsk); - xsk->umem->refcount--; /* Do not close an fd that also has an associated umem connected * to it.