From patchwork Sat Sep 21 16:52:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Bianconi X-Patchwork-Id: 1988155 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=GiIYxOJh; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4X9wLS1w2Bz1xsQ for ; Sun, 22 Sep 2024 02:53:20 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 623B6405AB; Sat, 21 Sep 2024 16:53:15 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id MYaB-mzWnajJ; Sat, 21 Sep 2024 16:53:14 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.34; helo=ash.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 5CA724061E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1726937594; bh=R2j7ow6x5+86gY3cUf6G5NOxtJaUORVrYYb/W8aw3Y0=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=GiIYxOJhlLeeuPhGcYBjQjKgIu3g47/N5thIhUhNtxMrh3WOFha3PX9R0fZVSYFKO Ot6Sjk4kWuWnKHA1cbMrVkqwTEEfd2VieYbCWHK8jUjr27NA6LOUfDRAcx4MpHM5SM NzcAdDB29NaHhQBx1gJACZz/NgY15+k00GRWV6hN5DuGaT64W+HsnOvqi7mK13sEY0 OD4ws1qFGZfSJWCD/fi/OiN7PtUkINVYmE+WOp/rAjFRjIx5kfNKDlzsfkCxB88nNg 1TBVcp6EyeDKkYLUVyLjt4S/TeQligA9vQlbaqBq8BvxsispW0uNe12sLFyJe+e7eZ irJ3IoMw/AnGQ== Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp4.osuosl.org (Postfix) with ESMTP id 5CA724061E; Sat, 21 Sep 2024 16:53:14 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 068FF1BF373 for ; Sat, 21 Sep 2024 16:53:13 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id C4AF440672 for ; Sat, 21 Sep 2024 16:53:11 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id SVlyLivP2hth for ; Sat, 21 Sep 2024 16:53:10 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2604:1380:4641:c500::1; helo=dfw.source.kernel.org; envelope-from=lorenzo@kernel.org; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp4.osuosl.org BC110405AB DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org BC110405AB Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by smtp4.osuosl.org (Postfix) with ESMTPS id BC110405AB for ; Sat, 21 Sep 2024 16:53:10 +0000 (UTC) Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 302285C5712; Sat, 21 Sep 2024 16:53:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 57C12C4CEC2; Sat, 21 Sep 2024 16:53:09 +0000 (UTC) From: Lorenzo Bianconi To: bpf@vger.kernel.org Date: Sat, 21 Sep 2024 18:52:57 +0200 Message-ID: <0e22c103ecafc5c79c6a27107348765913647afa.1726935917.git.lorenzo@kernel.org> X-Mailer: git-send-email 2.46.1 In-Reply-To: References: MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1726937589; bh=ySUc9WSYN+irTtZy0COsP164L2CZaBehasfOZt7YPEc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MTRNUrss/XvTAGiHVye1+lkmMl/JhzE5NDleFz32ZSOYX/4+hwvbG64f5CfQwxZqt yXDQc/zVoVkH5PsJ0zP/4EZ6sidAUpHLW6ngeCXitVI+/4Cu2l5f/XbppsX8gYWGQ/ OBTjyuFgXwIkDMTnDHD6dmu9P82TBm3/4UjcYUOijbO35xjHRp+zO8OUIiEhU34OT1 PTv1HO+ml8E9B6f8atHHrlwkNdjW6D8V4PaIRKWURSNVhL2X7xijTFfg8JJsMjemS5 cbyNcO43H1xweWC7X7z24+YQQi9XxwgrW7fJ232JGE4+qChhzY/xxlPvnLTSWT/oc1 d9I9tL+rfR6GA== X-Mailman-Original-Authentication-Results: smtp4.osuosl.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org X-Mailman-Original-Authentication-Results: smtp4.osuosl.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=MTRNUrss Subject: [Intel-wired-lan] [RFC bpf-next 1/4] net: xdp: Add xdp_rx_meta structure X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: toke@toke.dk, mst@redhat.com, jasowang@redhat.com, ast@kernel.org, edumazet@google.com, anthony.l.nguyen@intel.com, sdf@google.com, daniel@iogearbox.net, przemyslaw.kitszel@intel.com, john.fastabend@gmail.com, intel-wired-lan@lists.osuosl.org, kuba@kernel.org, pabeni@redhat.com, hawk@kernel.org, alexandre.torgue@foss.st.com, netdev@vger.kernel.org, tariqt@nvidia.com, aleksander.lobakin@intel.com, mcoquelin.stm32@gmail.com, saeedm@nvidia.com, davem@davemloft.net Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Introduce xdp_rx_meta structure as a container to store the already supported xdp rx hints (rx_hash and rx_vlan) in the xdp_buff and xdp_frame structures (the rx_timestamp will be added to the skb_shared_info area). Rely on the xdp_buff/xdp_frame flag field to indicate what kind of rx hints have been populated by the driver. This is a preliminary patch to preserve xdp rx hints running XDP_REDIRECT. Signed-off-by: Lorenzo Bianconi --- include/net/xdp.h | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/include/net/xdp.h b/include/net/xdp.h index e6770dd40c917..5e08b58a2a10f 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -70,12 +70,27 @@ struct xdp_txq_info { struct net_device *dev; }; +struct xdp_rx_meta { + struct { + u32 val; + u32 type; + } hash; + struct { + __be16 proto; + u16 tci; + } vlan; +}; + enum xdp_buff_flags { XDP_FLAGS_HAS_FRAGS = BIT(0), /* non-linear xdp buff */ XDP_FLAGS_FRAGS_PF_MEMALLOC = BIT(1), /* xdp paged memory is under * pressure */ + XDP_FLAGS_META_RX_HASH = BIT(2), /* hw rx hash */ + XDP_FLAGS_META_RX_VLAN = BIT(3), /* hw rx vlan */ }; +#define XDP_FLAGS_META_RX (XDP_FLAGS_META_RX_HASH | \ + XDP_FLAGS_META_RX_VLAN) struct xdp_buff { void *data; @@ -86,6 +101,7 @@ struct xdp_buff { struct xdp_txq_info *txq; u32 frame_sz; /* frame size to deduce data_hard_end/reserved tailroom*/ u32 flags; /* supported values defined in xdp_buff_flags */ + struct xdp_rx_meta rx_meta; /* rx hw metadata */ }; static __always_inline bool xdp_buff_has_frags(struct xdp_buff *xdp) @@ -113,6 +129,11 @@ static __always_inline void xdp_buff_set_frag_pfmemalloc(struct xdp_buff *xdp) xdp->flags |= XDP_FLAGS_FRAGS_PF_MEMALLOC; } +static __always_inline bool xdp_buff_has_rx_meta(struct xdp_buff *xdp) +{ + return !!(xdp->flags & XDP_FLAGS_META_RX); +} + static __always_inline void xdp_init_buff(struct xdp_buff *xdp, u32 frame_sz, struct xdp_rxq_info *rxq) { @@ -175,6 +196,7 @@ struct xdp_frame { struct net_device *dev_rx; /* used by cpumap */ u32 frame_sz; u32 flags; /* supported values defined in xdp_buff_flags */ + struct xdp_rx_meta rx_meta; /* rx hw metadata */ }; static __always_inline bool xdp_frame_has_frags(struct xdp_frame *frame) @@ -187,6 +209,11 @@ static __always_inline bool xdp_frame_is_frag_pfmemalloc(struct xdp_frame *frame return !!(frame->flags & XDP_FLAGS_FRAGS_PF_MEMALLOC); } +static __always_inline bool xdp_frame_has_rx_meta(struct xdp_frame *frame) +{ + return !!(frame->flags & XDP_FLAGS_META_RX); +} + #define XDP_BULK_QUEUE_SIZE 16 struct xdp_frame_bulk { int count; @@ -257,6 +284,8 @@ void xdp_convert_frame_to_buff(struct xdp_frame *frame, struct xdp_buff *xdp) xdp->data_meta = frame->data - frame->metasize; xdp->frame_sz = frame->frame_sz; xdp->flags = frame->flags; + if (xdp_frame_has_rx_meta(frame)) + xdp->rx_meta = frame->rx_meta; } static inline @@ -284,6 +313,8 @@ int xdp_update_frame_from_buff(struct xdp_buff *xdp, xdp_frame->metasize = metasize; xdp_frame->frame_sz = xdp->frame_sz; xdp_frame->flags = xdp->flags; + if (xdp_buff_has_rx_meta(xdp)) + xdp_frame->rx_meta = xdp->rx_meta; return 0; } From patchwork Sat Sep 21 16:52:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Bianconi X-Patchwork-Id: 1988156 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=XbF4T6An; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4X9wLV3Lccz1xsQ for ; Sun, 22 Sep 2024 02:53:22 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 7732940672; Sat, 21 Sep 2024 16:53:18 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id B7yS_voesoNi; Sat, 21 Sep 2024 16:53:16 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.34; helo=ash.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 0E4974064B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1726937596; bh=25DbdkuOEdqnoc8SijsqgVyQHYj82ciiUr/jWcJogSM=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=XbF4T6AnuGoqH4cUtfCkf5p6cfh2SMbxPUkab5YW0NbiFPAEKI64qWzh/UOQ8mjtM ktEcByz7Hf7NQ5DxqYpA3zFLrXjlXZZthGRks1B2LzSWVHgUaxDH3l0ZRKVjcAnJ3Z oyZNDy+l3//uzGHqRDA0+qj11QMC9dGBgD5IdW2lJZWvl974TGN8TeZXUs/PRbOUOe CgMQKX12eVd15r3fzlqLmxnGMsP+JLYsb4DPk4bcXJEx9p96Zhbt3iSOQ76qfIxXei dtuJndmSJ9N9dXoggxrhYWHurVVs3Ta4s7F+75mWtnuiPfeJYGzwLrxHJAuGvhxi2T OXqj7X+SBvjfQ== Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp4.osuosl.org (Postfix) with ESMTP id 0E4974064B; Sat, 21 Sep 2024 16:53:16 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id DEA7C1BF373 for ; Sat, 21 Sep 2024 16:53:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id CD1C7400D0 for ; Sat, 21 Sep 2024 16:53:14 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id WawHalq5Ky_D for ; Sat, 21 Sep 2024 16:53:13 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2604:1380:4641:c500::1; helo=dfw.source.kernel.org; envelope-from=lorenzo@kernel.org; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp2.osuosl.org 6BF8C40186 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 6BF8C40186 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by smtp2.osuosl.org (Postfix) with ESMTPS id 6BF8C40186 for ; Sat, 21 Sep 2024 16:53:13 +0000 (UTC) Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id AEFB45C5719; Sat, 21 Sep 2024 16:53:08 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D82A8C4CEC2; Sat, 21 Sep 2024 16:53:11 +0000 (UTC) From: Lorenzo Bianconi To: bpf@vger.kernel.org Date: Sat, 21 Sep 2024 18:52:58 +0200 Message-ID: <01ce17910fdd7b693c23132663fa884d5ec7f440.1726935917.git.lorenzo@kernel.org> X-Mailer: git-send-email 2.46.1 In-Reply-To: References: MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1726937592; bh=FtECcRJ9XXZ3jnvA5WKYiudLQt/ewvPA8UCT9rYB6eQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QkBYEes6AI7AnET4LlfzKJfKdkZ+nZyYEzbAV0SyGCSHvLg77jwOlWSFFSAwlWmXZ ryQvlxKYJDn8T3n+XSQxKYDRCPodVDD3FWFqJrT0Y9GfjfOuGdPVRTFfVf/CRCjoeV Z6BQ7mjQRzJDdEX1HrrwcFvJD8Gx2iDH+ceTYMuaVeWSNnKeSG0CDM9QbJTevv7GWK ggt/GThXtZaXBNrEO/IC9FJeOzb/dBujbHhE14y0sN9/97XjTMWqXDynMVYLSYiJbp kg9CcjP9ott+U53Cmp8y5Qxy5bFu5E/vLIzbusZx6MXzHkLRTsXZbb6vIORCp6Lr3B RKDmA85MRm5Lw== X-Mailman-Original-Authentication-Results: smtp2.osuosl.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org X-Mailman-Original-Authentication-Results: smtp2.osuosl.org; dkim=pass (2048-bit key, unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=QkBYEes6 Subject: [Intel-wired-lan] [RFC bpf-next 2/4] net: xdp: Update rx_hash of xdp_rx_meta struct running xmo_rx_hash callback X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: toke@toke.dk, mst@redhat.com, jasowang@redhat.com, ast@kernel.org, edumazet@google.com, anthony.l.nguyen@intel.com, sdf@google.com, daniel@iogearbox.net, przemyslaw.kitszel@intel.com, john.fastabend@gmail.com, intel-wired-lan@lists.osuosl.org, kuba@kernel.org, pabeni@redhat.com, hawk@kernel.org, alexandre.torgue@foss.st.com, netdev@vger.kernel.org, tariqt@nvidia.com, aleksander.lobakin@intel.com, mcoquelin.stm32@gmail.com, saeedm@nvidia.com, davem@davemloft.net Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Set rx_hash in xdp_rx_meta struct of xdp_buff/xdp_frame according to the value reported by the hw. Update the xmo_rx_hash callback of xdp_metadata_ops for the following drivers: - ice - igc - mlx4 - mlx5 - veth - virtio_net Set rx_hash value/type reported by the hw converting the xdp_frame into a sk_buff. Signed-off-by: Lorenzo Bianconi --- drivers/net/ethernet/intel/ice/ice_txrx_lib.c | 3 +++ drivers/net/ethernet/intel/igc/igc_main.c | 2 ++ drivers/net/ethernet/mellanox/mlx4/en_rx.c | 1 + .../net/ethernet/mellanox/mlx5/core/en/xdp.c | 2 ++ drivers/net/veth.c | 1 + drivers/net/virtio_net.c | 3 ++- include/net/xdp.h | 14 +++++++++++++ net/core/xdp.c | 20 ++++++++++++++++++- 8 files changed, 44 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_txrx_lib.c b/drivers/net/ethernet/intel/ice/ice_txrx_lib.c index 2719f0e20933f..e4b051a8d99c7 100644 --- a/drivers/net/ethernet/intel/ice/ice_txrx_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_txrx_lib.c @@ -520,12 +520,15 @@ static int ice_xdp_rx_hash(const struct xdp_md *ctx, u32 *hash, enum xdp_rss_hash_type *rss_type) { const struct ice_xdp_buff *xdp_ext = (void *)ctx; + struct xdp_buff *xdp = (void *)&(xdp_ext->xdp_buff); *hash = ice_get_rx_hash(xdp_ext->eop_desc); *rss_type = ice_xdp_rx_hash_type(xdp_ext->eop_desc); if (!likely(*hash)) return -ENODATA; + xdp_set_rx_meta_hash(xdp, *hash, *rss_type); + return 0; } diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index dfd6c00b4205d..ed22a7a70695e 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -6776,12 +6776,14 @@ static int igc_xdp_rx_hash(const struct xdp_md *_ctx, u32 *hash, enum xdp_rss_hash_type *rss_type) { const struct igc_xdp_buff *ctx = (void *)_ctx; + struct xdp_buff *xdp = (void *)&(ctx->xdp); if (!(ctx->xdp.rxq->dev->features & NETIF_F_RXHASH)) return -ENODATA; *hash = le32_to_cpu(ctx->rx_desc->wb.lower.hi_dword.rss); *rss_type = igc_xdp_rss_type[igc_rss_type(ctx->rx_desc)]; + xdp_set_rx_meta_hash(xdp, *hash, *rss_type); return 0; } diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c index 15c57e9517e9a..ef6c687866f9d 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c @@ -708,6 +708,7 @@ int mlx4_en_xdp_rx_hash(const struct xdp_md *ctx, u32 *hash, if (cqe->ipv6_ext_mask) xht |= XDP_RSS_L3_DYNHDR; } + xdp_set_rx_meta_hash(&(_ctx->xdp), *hash, xht); *rss_type = xht; return 0; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c index 4610621a340e5..92fb98397751a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c @@ -238,6 +238,7 @@ static int mlx5e_xdp_rx_hash(const struct xdp_md *ctx, u32 *hash, enum xdp_rss_hash_type *rss_type) { const struct mlx5e_xdp_buff *_ctx = (void *)ctx; + struct xdp_buff *xdp = (void *)&(_ctx->xdp); const struct mlx5_cqe64 *cqe = _ctx->cqe; u32 hash_type, l4_type, ip_type, lookup; @@ -252,6 +253,7 @@ static int mlx5e_xdp_rx_hash(const struct xdp_md *ctx, u32 *hash, l4_type = FIELD_GET(CQE_RSS_HTYPE_L4, hash_type); lookup = ip_type | l4_type; *rss_type = mlx5_xdp_rss_type[lookup]; + xdp_set_rx_meta_hash(xdp, *hash, *rss_type); return 0; } diff --git a/drivers/net/veth.c b/drivers/net/veth.c index 426e68a950672..cc8e90d330456 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -1633,6 +1633,7 @@ static int veth_xdp_rx_hash(const struct xdp_md *ctx, u32 *hash, *hash = skb_get_hash(skb); *rss_type = skb->l4_hash ? XDP_RSS_TYPE_L4_ANY : XDP_RSS_TYPE_NONE; + xdp_set_rx_meta_hash(&(_ctx->xdp), *hash, *rss_type); return 0; } diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index 3f10c72743e94..1d1ada8eeda0e 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -6250,8 +6250,8 @@ virtnet_xdp_rss_type[VIRTIO_NET_HASH_REPORT_MAX_TABLE] = { static int virtnet_xdp_rx_hash(const struct xdp_md *_ctx, u32 *hash, enum xdp_rss_hash_type *rss_type) { - const struct xdp_buff *xdp = (void *)_ctx; struct virtio_net_hdr_v1_hash *hdr_hash; + struct xdp_buff *xdp = (void *)_ctx; struct virtnet_info *vi; u16 hash_report; @@ -6267,6 +6267,7 @@ static int virtnet_xdp_rx_hash(const struct xdp_md *_ctx, u32 *hash, *rss_type = virtnet_xdp_rss_type[hash_report]; *hash = __le32_to_cpu(hdr_hash->hash_value); + xdp_set_rx_meta_hash(xdp, *hash, *rss_type); return 0; } diff --git a/include/net/xdp.h b/include/net/xdp.h index 5e08b58a2a10f..e1c344eb7e686 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -214,6 +214,11 @@ static __always_inline bool xdp_frame_has_rx_meta(struct xdp_frame *frame) return !!(frame->flags & XDP_FLAGS_META_RX); } +static __always_inline bool xdp_frame_has_rx_meta_hash(struct xdp_frame *frame) +{ + return !!(frame->flags & XDP_FLAGS_META_RX_HASH); +} + #define XDP_BULK_QUEUE_SIZE 16 struct xdp_frame_bulk { int count; @@ -504,6 +509,15 @@ struct xdp_metadata_ops { u16 *vlan_tci); }; +static __always_inline void +xdp_set_rx_meta_hash(struct xdp_buff *xdp, u32 hash, + enum xdp_rss_hash_type rss_type) +{ + xdp->rx_meta.hash.val = hash; + xdp->rx_meta.hash.type = rss_type; + xdp->flags |= XDP_FLAGS_META_RX_HASH; +} + #ifdef CONFIG_NET u32 bpf_xdp_metadata_kfunc_id(int id); bool bpf_dev_bound_kfunc_id(u32 btf_id); diff --git a/net/core/xdp.c b/net/core/xdp.c index bcc5551c6424b..e2f4d01cf84cf 100644 --- a/net/core/xdp.c +++ b/net/core/xdp.c @@ -594,6 +594,23 @@ int xdp_alloc_skb_bulk(void **skbs, int n_skb, gfp_t gfp) } EXPORT_SYMBOL_GPL(xdp_alloc_skb_bulk); +static void xdp_set_skb_rx_hash_from_meta(struct xdp_frame *frame, + struct sk_buff *skb) +{ + enum pkt_hash_types hash_type = PKT_HASH_TYPE_NONE; + + if (!xdp_frame_has_rx_meta_hash(frame)) + return; + + if (frame->rx_meta.hash.type & XDP_RSS_TYPE_L4_ANY) + hash_type = PKT_HASH_TYPE_L4; + else if (frame->rx_meta.hash.type & (XDP_RSS_TYPE_L3_IPV4 | + XDP_RSS_TYPE_L3_IPV6)) + hash_type = PKT_HASH_TYPE_L3; + + skb_set_hash(skb, frame->rx_meta.hash.val, hash_type); +} + struct sk_buff *__xdp_build_skb_from_frame(struct xdp_frame *xdpf, struct sk_buff *skb, struct net_device *dev) @@ -634,9 +651,10 @@ struct sk_buff *__xdp_build_skb_from_frame(struct xdp_frame *xdpf, /* Essential SKB info: protocol and skb->dev */ skb->protocol = eth_type_trans(skb, dev); + xdp_set_skb_rx_hash_from_meta(xdpf, skb); + /* Optional SKB info, currently missing: * - HW checksum info (skb->ip_summed) - * - HW RX hash (skb_set_hash) * - RX ring dev queue index (skb_record_rx_queue) */ From patchwork Sat Sep 21 16:52:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Bianconi X-Patchwork-Id: 1988157 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=XXUs6Ao4; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4X9wLX3sZnz1xsQ for ; Sun, 22 Sep 2024 02:53:24 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 06BDC405D9; Sat, 21 Sep 2024 16:53:21 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id nvIr3LJ-bSk7; Sat, 21 Sep 2024 16:53:20 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.34; helo=ash.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org EF9C7406E5 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1726937600; bh=E8x7wlQNHT+KDlk0X8Xbi+HpU0S/SVjNfgmeaAxGY7s=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=XXUs6Ao4DLQxDgCx8zj4Yhb4XIH01+bQxiuE7dWxJ569RHJB1Zkt4nTqE8foQcgMA PiYH782Sr7ZIBeur6O8HchQIeiadS8+GUpZgxcVcGlnbrLrY67HoZLU33wEclGQA4P XVbshDKwlfloyTUHRuwiZvFaSMb/1IH1R97Zxv5ftO95GLEllEbTs7mwakMHNbtpmv U7hRtSOxL5uloUj2bC+YVBn0NRMVKvPLkr9lpzbLU9aSUrmhzr7+SYMItoOlCy4NkK pAwJN4uqYo8tOYgp2E8TCsGnZAsC8r/b82rTLD4BcXun/XasU7az0R7KxvKSxdqBh2 UjuBcLdyjdpnA== Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp4.osuosl.org (Postfix) with ESMTP id EF9C7406E5; Sat, 21 Sep 2024 16:53:19 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 9C1FF1BF373 for ; Sat, 21 Sep 2024 16:53:18 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 86E854064B for ; Sat, 21 Sep 2024 16:53:18 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id N0mV9n6sTiT5 for ; Sat, 21 Sep 2024 16:53:16 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2604:1380:4641:c500::1; helo=dfw.source.kernel.org; envelope-from=lorenzo@kernel.org; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp4.osuosl.org 2B7354074B DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 2B7354074B Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by smtp4.osuosl.org (Postfix) with ESMTPS id 2B7354074B for ; Sat, 21 Sep 2024 16:53:16 +0000 (UTC) Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 894955C503A; Sat, 21 Sep 2024 16:53:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AE731C4CEC2; Sat, 21 Sep 2024 16:53:14 +0000 (UTC) From: Lorenzo Bianconi To: bpf@vger.kernel.org Date: Sat, 21 Sep 2024 18:52:59 +0200 Message-ID: <26256149e5331a69ba9574907ac570a7d2d2e382.1726935917.git.lorenzo@kernel.org> X-Mailer: git-send-email 2.46.1 In-Reply-To: References: MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1726937595; bh=3jw9I0lwiO4TTcm1AqiwzBTLqWxZoiHdGD9kSgY0iRA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=j0X8UTg+dz4aJfP1I6iIW7aF1XJpGoSGCwGaAeJTO5ai5LeUVOzuRBz92hrC1ul4n vhna9qluD4JeHV8sO7y6An8tK2s4jJCXSkvCusq7cSaOoEwfXOp6xItbaE8MeS9m1V mspRJ1n12fJgRMfE3Ati8nRHMlQNbxo9C8rdrwTs4M+8f3hG9+BiDKwAC5jvklwa3a IMHLqb2hZtv5ugwJ8Y2nfP4HRUr1OmufHjs/FNNPSGWWtnJjHJtpNbUOwUOW7KNLKp V/CEUThSY6Bex1C3eaMEm6AvHE0nd8LwRxgdFNWJU/b1cJbhsMFW7Tep6sDb+LTusA VUWSEHFgc3Kzg== X-Mailman-Original-Authentication-Results: smtp4.osuosl.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org X-Mailman-Original-Authentication-Results: smtp4.osuosl.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=j0X8UTg+ Subject: [Intel-wired-lan] [RFC bpf-next 3/4] net: xdp: Update rx_vlan of xdp_rx_meta struct running xmo_rx_vlan_tag callback X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: toke@toke.dk, mst@redhat.com, jasowang@redhat.com, ast@kernel.org, edumazet@google.com, anthony.l.nguyen@intel.com, sdf@google.com, daniel@iogearbox.net, przemyslaw.kitszel@intel.com, john.fastabend@gmail.com, intel-wired-lan@lists.osuosl.org, kuba@kernel.org, pabeni@redhat.com, hawk@kernel.org, alexandre.torgue@foss.st.com, netdev@vger.kernel.org, tariqt@nvidia.com, aleksander.lobakin@intel.com, mcoquelin.stm32@gmail.com, saeedm@nvidia.com, davem@davemloft.net Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Set vlan_proto and vlan_tci in xdp_rx_meta struct of xdp_buff/xdp_frame according to the value reported by the hw. Update the xmo_rx_vlan_tag callback of xdp_metadata_ops for the following drivers: - ice - mlx5 - veth Set rx vlan_{prot, tci} reported by the hw converting the xdp_frame into a sk_buff. Signed-off-by: Lorenzo Bianconi --- drivers/net/ethernet/intel/ice/ice_txrx_lib.c | 3 +++ drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c | 3 +++ drivers/net/veth.c | 3 +++ include/net/xdp.h | 14 ++++++++++++++ net/core/xdp.c | 3 +++ 5 files changed, 26 insertions(+) diff --git a/drivers/net/ethernet/intel/ice/ice_txrx_lib.c b/drivers/net/ethernet/intel/ice/ice_txrx_lib.c index e4b051a8d99c7..74dabe5b0c35c 100644 --- a/drivers/net/ethernet/intel/ice/ice_txrx_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_txrx_lib.c @@ -545,6 +545,7 @@ static int ice_xdp_rx_vlan_tag(const struct xdp_md *ctx, __be16 *vlan_proto, u16 *vlan_tci) { const struct ice_xdp_buff *xdp_ext = (void *)ctx; + struct xdp_buff *xdp = (void *)&(xdp_ext->xdp_buff); *vlan_proto = xdp_ext->pkt_ctx->vlan_proto; if (!*vlan_proto) @@ -554,6 +555,8 @@ static int ice_xdp_rx_vlan_tag(const struct xdp_md *ctx, __be16 *vlan_proto, if (!*vlan_tci) return -ENODATA; + xdp_set_rx_meta_vlan(xdp, *vlan_proto, *vlan_tci); + return 0; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c index 92fb98397751a..d3b7eee031470 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c @@ -262,6 +262,7 @@ static int mlx5e_xdp_rx_vlan_tag(const struct xdp_md *ctx, __be16 *vlan_proto, u16 *vlan_tci) { const struct mlx5e_xdp_buff *_ctx = (void *)ctx; + struct xdp_buff *xdp = (void *)&(_ctx->xdp); const struct mlx5_cqe64 *cqe = _ctx->cqe; if (!cqe_has_vlan(cqe)) @@ -269,6 +270,8 @@ static int mlx5e_xdp_rx_vlan_tag(const struct xdp_md *ctx, __be16 *vlan_proto, *vlan_proto = htons(ETH_P_8021Q); *vlan_tci = be16_to_cpu(cqe->vlan_info); + xdp_set_rx_meta_vlan(xdp, *vlan_proto, *vlan_tci); + return 0; } diff --git a/drivers/net/veth.c b/drivers/net/veth.c index cc8e90d330456..3a4b81104a6bd 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -1642,6 +1642,7 @@ static int veth_xdp_rx_vlan_tag(const struct xdp_md *ctx, __be16 *vlan_proto, u16 *vlan_tci) { const struct veth_xdp_buff *_ctx = (void *)ctx; + struct xdp_buff *xdp = (void *)&(_ctx->xdp); const struct sk_buff *skb = _ctx->skb; int err; @@ -1653,6 +1654,8 @@ static int veth_xdp_rx_vlan_tag(const struct xdp_md *ctx, __be16 *vlan_proto, return err; *vlan_proto = skb->vlan_proto; + xdp_set_rx_meta_vlan(xdp, skb->vlan_proto, *vlan_tci); + return err; } diff --git a/include/net/xdp.h b/include/net/xdp.h index e1c344eb7e686..2ffaad806b9ed 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -219,6 +219,11 @@ static __always_inline bool xdp_frame_has_rx_meta_hash(struct xdp_frame *frame) return !!(frame->flags & XDP_FLAGS_META_RX_HASH); } +static __always_inline bool xdp_frame_has_rx_meta_vlan(struct xdp_frame *frame) +{ + return !!(frame->flags & XDP_FLAGS_META_RX_VLAN); +} + #define XDP_BULK_QUEUE_SIZE 16 struct xdp_frame_bulk { int count; @@ -518,6 +523,15 @@ xdp_set_rx_meta_hash(struct xdp_buff *xdp, u32 hash, xdp->flags |= XDP_FLAGS_META_RX_HASH; } +static __always_inline void +xdp_set_rx_meta_vlan(struct xdp_buff *xdp, __be16 vlan_proto, + u16 vlan_tci) +{ + xdp->rx_meta.vlan.proto = vlan_proto; + xdp->rx_meta.vlan.tci = vlan_tci; + xdp->flags |= XDP_FLAGS_META_RX_VLAN; +} + #ifdef CONFIG_NET u32 bpf_xdp_metadata_kfunc_id(int id); bool bpf_dev_bound_kfunc_id(u32 btf_id); diff --git a/net/core/xdp.c b/net/core/xdp.c index e2f4d01cf84cf..84d6b134f8e97 100644 --- a/net/core/xdp.c +++ b/net/core/xdp.c @@ -652,6 +652,9 @@ struct sk_buff *__xdp_build_skb_from_frame(struct xdp_frame *xdpf, skb->protocol = eth_type_trans(skb, dev); xdp_set_skb_rx_hash_from_meta(xdpf, skb); + if (xdp_frame_has_rx_meta_vlan(xdpf)) + __vlan_hwaccel_put_tag(skb, xdpf->rx_meta.vlan.proto, + xdpf->rx_meta.vlan.tci); /* Optional SKB info, currently missing: * - HW checksum info (skb->ip_summed) From patchwork Sat Sep 21 16:53:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Bianconi X-Patchwork-Id: 1988158 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=Dgd1h1vc; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4X9wLb3cxVz1xsQ for ; Sun, 22 Sep 2024 02:53:27 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 1FA77408BF; Sat, 21 Sep 2024 16:53:22 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id hlW3ajFBvbtS; Sat, 21 Sep 2024 16:53:21 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.34; helo=ash.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 2210D406E5 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1726937601; bh=soiQrtq0V4Qe4eTruM9Am81qj70ME8Y0+HCj2f7poHM=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=Dgd1h1vcVybt/SQs9/rRZp2QR5rTMNLoGNzfgcuIIVRaPu4a8PkzZA06gKNfeZuGu LB9NB6HoKpagBVe2odbzQyF0H5ZxZr+e6qP0BKOhpCtBEMy0mRV9IcN3nMP6im3Liz cclGUUrOSCoy8ZeVClvaYg5jB8EwUpyefspZq4mE98rbeSEq/EVonBrELg2RuQW1z6 wzv6PzN8intkzj3G9Ok6qz1n7Riyy3pzw+cf/0c9o7wprusXZMuxKAV4K3EmUP3uUD 5P+eHrOiuK631gyy6HrhBSOnCiG4SNQmCiAyAWefjcsfCtRS6/5kwdUKBbi1TxwyeS hqcop8saUa+9w== Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp4.osuosl.org (Postfix) with ESMTP id 2210D406E5; Sat, 21 Sep 2024 16:53:21 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 11B5F1BF373 for ; Sat, 21 Sep 2024 16:53:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 0056C60637 for ; Sat, 21 Sep 2024 16:53:20 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id g1EUOBvowA_B for ; Sat, 21 Sep 2024 16:53:19 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2604:1380:4641:c500::1; helo=dfw.source.kernel.org; envelope-from=lorenzo@kernel.org; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org 1258C60633 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 1258C60633 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by smtp3.osuosl.org (Postfix) with ESMTPS id 1258C60633 for ; Sat, 21 Sep 2024 16:53:18 +0000 (UTC) Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 6052F5C58C7; Sat, 21 Sep 2024 16:53:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 87C17C4CEC6; Sat, 21 Sep 2024 16:53:17 +0000 (UTC) From: Lorenzo Bianconi To: bpf@vger.kernel.org Date: Sat, 21 Sep 2024 18:53:00 +0200 Message-ID: <4683b6c75e4ee0842a9d24faf4a2ca4e2be45223.1726935918.git.lorenzo@kernel.org> X-Mailer: git-send-email 2.46.1 In-Reply-To: References: MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1726937597; bh=pH4e8OUiYnFljQThlJQzDja0lcvpvuiTePZIdXTOrJQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=K0equYT9i7Jy9RpGQCgVuzSQBbcqp9vIAJfk9v23yFjyAcbMN0MQXzbr7WX3iLmtc CNLR5ZFUCTRCtgnQ4HrZppgBz3hKrsX/VzeHjUztnvAn0eJs+fra1QGs5jigeBzgu7 xGCkK2c3+Mne5mNOe1TFZLmE+V3o6XHGS3McUAVCqAI59wTIoRa5UU0BqoLQYDR2Xb FsT1XyPmV9SrgPHvsWcpSQHLrBRXSP2zEnDpYG5OWIWqsB+scUCaPf0iVPCqdSRkRI dgIUrFGKaP2tv2/jOpUN3eqppefmWzQl8gxpckI8MGHwja4ZB65zuB8Im7FjCPtWz6 5hxc/Ka7/bjug== X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dmarc=pass (p=quarantine dis=none) header.from=kernel.org X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key, unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=K0equYT9 Subject: [Intel-wired-lan] [RFC bpf-next 4/4] net: xdp: Update rx timestamp of xdp_rx_meta struct running xmo_rx_timestamp callback X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: toke@toke.dk, mst@redhat.com, jasowang@redhat.com, ast@kernel.org, edumazet@google.com, anthony.l.nguyen@intel.com, sdf@google.com, daniel@iogearbox.net, przemyslaw.kitszel@intel.com, john.fastabend@gmail.com, intel-wired-lan@lists.osuosl.org, kuba@kernel.org, pabeni@redhat.com, hawk@kernel.org, alexandre.torgue@foss.st.com, netdev@vger.kernel.org, tariqt@nvidia.com, aleksander.lobakin@intel.com, mcoquelin.stm32@gmail.com, saeedm@nvidia.com, davem@davemloft.net Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Set rx_timestamp in xdp_rx_meta struct of xdp_buff/xdp_frame according to the value reported by the hw. Update the xmo_rx_timestamp callback of xdp_metadata_ops for the following drivers: - ice - igc - mlx5 - mlx4 - stmmac - veth Set rx_timestamp reported by the hw converting the xdp_frame into a sk_buff. Signed-off-by: Lorenzo Bianconi --- drivers/net/ethernet/intel/ice/ice_txrx_lib.c | 3 +++ drivers/net/ethernet/intel/igc/igc_main.c | 3 +++ drivers/net/ethernet/mellanox/mlx4/en_rx.c | 2 ++ .../net/ethernet/mellanox/mlx5/core/en/xdp.c | 3 +++ .../net/ethernet/stmicro/stmmac/stmmac_main.c | 3 +++ drivers/net/veth.c | 5 +++++ include/net/xdp.h | 20 +++++++++++++++++++ net/core/xdp.c | 6 ++++++ 8 files changed, 45 insertions(+) diff --git a/drivers/net/ethernet/intel/ice/ice_txrx_lib.c b/drivers/net/ethernet/intel/ice/ice_txrx_lib.c index 74dabe5b0c35c..d5ff77784d756 100644 --- a/drivers/net/ethernet/intel/ice/ice_txrx_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_txrx_lib.c @@ -489,12 +489,15 @@ void ice_finalize_xdp_rx(struct ice_tx_ring *xdp_ring, unsigned int xdp_res, static int ice_xdp_rx_hw_ts(const struct xdp_md *ctx, u64 *ts_ns) { const struct ice_xdp_buff *xdp_ext = (void *)ctx; + struct xdp_buff *xdp = (void *)&(xdp_ext->xdp_buff); *ts_ns = ice_ptp_get_rx_hwts(xdp_ext->eop_desc, xdp_ext->pkt_ctx); if (!*ts_ns) return -ENODATA; + xdp_set_rx_meta_ts(xdp, *ts_ns); + return 0; } diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index ed22a7a70695e..240c52f0c6ec4 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -6795,7 +6795,10 @@ static int igc_xdp_rx_timestamp(const struct xdp_md *_ctx, u64 *timestamp) struct igc_inline_rx_tstamps *tstamp = ctx->rx_ts; if (igc_test_staterr(ctx->rx_desc, IGC_RXDADV_STAT_TSIP)) { + struct xdp_buff *xdp = (void *)&(ctx->xdp); + *timestamp = igc_ptp_rx_pktstamp(adapter, tstamp->timer0); + xdp_set_rx_meta_ts(xdp, *timestamp); return 0; } diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c index ef6c687866f9d..dc714b3d9631c 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c @@ -681,6 +681,8 @@ int mlx4_en_xdp_rx_timestamp(const struct xdp_md *ctx, u64 *timestamp) *timestamp = mlx4_en_get_hwtstamp(_ctx->mdev, mlx4_en_get_cqe_ts(_ctx->cqe)); + xdp_set_rx_meta_ts(&(_ctx->xdp), *timestamp); + return 0; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c index d3b7eee031470..507322e0c8dd0 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c @@ -178,12 +178,15 @@ mlx5e_xmit_xdp_buff(struct mlx5e_xdpsq *sq, struct mlx5e_rq *rq, static int mlx5e_xdp_rx_timestamp(const struct xdp_md *ctx, u64 *timestamp) { const struct mlx5e_xdp_buff *_ctx = (void *)ctx; + struct xdp_buff *xdp = (void *)&(_ctx->xdp); if (unlikely(!mlx5e_rx_hw_stamp(_ctx->rq->tstamp))) return -ENODATA; *timestamp = mlx5e_cqe_ts_to_ns(_ctx->rq->ptp_cyc2time, _ctx->rq->clock, get_cqe_ts(_ctx->cqe)); + xdp_set_rx_meta_ts(xdp, *timestamp); + return 0; } diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c index f3a1b179aaeac..4da3200816eda 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c @@ -7458,9 +7458,12 @@ static int stmmac_xdp_rx_timestamp(const struct xdp_md *_ctx, u64 *timestamp) /* Check if timestamp is available */ if (stmmac_get_rx_timestamp_status(priv, desc, ndesc, priv->adv_ts)) { + struct xdp_buff *xdp = (void *)&(ctx->xdp); + stmmac_get_timestamp(priv, desc_contains_ts, priv->adv_ts, &ns); ns -= priv->plat->cdc_error_adj; *timestamp = ns_to_ktime(ns); + xdp_set_rx_meta_ts(xdp, *timestamp); return 0; } diff --git a/drivers/net/veth.c b/drivers/net/veth.c index 3a4b81104a6bd..3eaba1709a915 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -1619,6 +1619,11 @@ static int veth_xdp_rx_timestamp(const struct xdp_md *ctx, u64 *timestamp) return -ENODATA; *timestamp = skb_hwtstamps(_ctx->skb)->hwtstamp; + /* Here we just need to set the XDP_FLAGS_META_RX_TS bit in the + * xdp_buff flags. + */ + xdp_buff_set_rx_meta_ts_flag(&(_ctx->xdp)); + return 0; } diff --git a/include/net/xdp.h b/include/net/xdp.h index 2ffaad806b9ed..c30c01b23709e 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -88,6 +88,7 @@ enum xdp_buff_flags { */ XDP_FLAGS_META_RX_HASH = BIT(2), /* hw rx hash */ XDP_FLAGS_META_RX_VLAN = BIT(3), /* hw rx vlan */ + XDP_FLAGS_META_RX_TS = BIT(4), /* hw rx timestamp */ }; #define XDP_FLAGS_META_RX (XDP_FLAGS_META_RX_HASH | \ XDP_FLAGS_META_RX_VLAN) @@ -134,6 +135,11 @@ static __always_inline bool xdp_buff_has_rx_meta(struct xdp_buff *xdp) return !!(xdp->flags & XDP_FLAGS_META_RX); } +static __always_inline void xdp_buff_set_rx_meta_ts_flag(struct xdp_buff *xdp) +{ + xdp->flags |= XDP_FLAGS_META_RX_TS; +} + static __always_inline void xdp_init_buff(struct xdp_buff *xdp, u32 frame_sz, struct xdp_rxq_info *rxq) { @@ -224,6 +230,11 @@ static __always_inline bool xdp_frame_has_rx_meta_vlan(struct xdp_frame *frame) return !!(frame->flags & XDP_FLAGS_META_RX_VLAN); } +static __always_inline bool xdp_frame_has_rx_meta_ts(struct xdp_frame *frame) +{ + return !!(frame->flags & XDP_FLAGS_META_RX_TS); +} + #define XDP_BULK_QUEUE_SIZE 16 struct xdp_frame_bulk { int count; @@ -532,6 +543,15 @@ xdp_set_rx_meta_vlan(struct xdp_buff *xdp, __be16 vlan_proto, xdp->flags |= XDP_FLAGS_META_RX_VLAN; } +static __always_inline void xdp_set_rx_meta_ts(struct xdp_buff *xdp, u64 ts) +{ + struct skb_shared_info *sinfo = xdp_get_shared_info_from_buff(xdp); + struct skb_shared_hwtstamps *shwt = &sinfo->hwtstamps; + + shwt->hwtstamp = ts; + xdp->flags |= XDP_FLAGS_META_RX_TS; +} + #ifdef CONFIG_NET u32 bpf_xdp_metadata_kfunc_id(int id); bool bpf_dev_bound_kfunc_id(u32 btf_id); diff --git a/net/core/xdp.c b/net/core/xdp.c index 84d6b134f8e97..5c9efcf1436d0 100644 --- a/net/core/xdp.c +++ b/net/core/xdp.c @@ -619,11 +619,15 @@ struct sk_buff *__xdp_build_skb_from_frame(struct xdp_frame *xdpf, unsigned int headroom, frame_size; void *hard_start; u8 nr_frags; + u64 ts; /* xdp frags frame */ if (unlikely(xdp_frame_has_frags(xdpf))) nr_frags = sinfo->nr_frags; + if (unlikely(xdp_frame_has_rx_meta_ts(xdpf))) + ts = sinfo->hwtstamps.hwtstamp; + /* Part of headroom was reserved to xdpf */ headroom = sizeof(*xdpf) + xdpf->headroom; @@ -655,6 +659,8 @@ struct sk_buff *__xdp_build_skb_from_frame(struct xdp_frame *xdpf, if (xdp_frame_has_rx_meta_vlan(xdpf)) __vlan_hwaccel_put_tag(skb, xdpf->rx_meta.vlan.proto, xdpf->rx_meta.vlan.tci); + if (unlikely(xdp_frame_has_rx_meta_ts(xdpf))) + skb_hwtstamps(skb)->hwtstamp = ts; /* Optional SKB info, currently missing: * - HW checksum info (skb->ip_summed)