From patchwork Wed Dec 4 11:42:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Wander Lairson Costa X-Patchwork-Id: 2018215 X-Patchwork-Delegate: anthony.l.nguyen@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=dVuPG88I; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Y3Fyz53LFz1yR0 for ; Wed, 4 Dec 2024 22:43:39 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 1A9856069C; Wed, 4 Dec 2024 11:43:38 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id m8pjrqZe1fnY; Wed, 4 Dec 2024 11:43:37 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.142; helo=lists1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 70CB1606CC DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1733312617; bh=YjLiIgA1heSf6FeldDRqyic0hKMQLJvylr1VcRItvD4=; h=From:To:Cc:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From; b=dVuPG88IQqwzrk2jisa5A2oxsbV2cu/nIIbyCZZ1AO30/ZPSVz3eiXfVNzovZOleb HOKVzDydlg01wWuLTreNS8dH9o76SIqaVrXvN7FwlwB+Nf+4DarHJGcwqetswwl+4b 35UFX++rYfeMp2uKzurLqbpwlhM4Q1X8I24AhMQuHmQglMQh92MPZN6xRHrIYDcx6i jAgW9qhe8Rju4YqI6IZJ/KUKbrJEgCei2TapjwYc0MyQtb29i1AWP4I8kv3AkcpUrL zUuWMBhaCDl+4MeGhyDJpT18PtT+VVire7r4DTLuJwHNiuM8nznFTUiQrxYCAT3IS3 KGqBa2ek0ODbw== Received: from lists1.osuosl.org (lists1.osuosl.org [140.211.166.142]) by smtp3.osuosl.org (Postfix) with ESMTP id 70CB1606CC; Wed, 4 Dec 2024 11:43:37 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists1.osuosl.org (Postfix) with ESMTP id CC3E31DD1 for ; Wed, 4 Dec 2024 11:43:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id AC5EC606CC for ; Wed, 4 Dec 2024 11:43:35 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id byJU6lqQUBZM for ; Wed, 4 Dec 2024 11:43:34 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=wander@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org B4A9D6075D DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org B4A9D6075D Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id B4A9D6075D for ; Wed, 4 Dec 2024 11:43:33 +0000 (UTC) Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-439-CrQUMUH4NIapA6bZIKoFvA-1; Wed, 04 Dec 2024 06:43:29 -0500 X-MC-Unique: CrQUMUH4NIapA6bZIKoFvA-1 X-Mimecast-MFC-AGG-ID: CrQUMUH4NIapA6bZIKoFvA Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D233B1955F68; Wed, 4 Dec 2024 11:43:26 +0000 (UTC) Received: from wcosta-thinkpadt14gen4.rmtbr.csb (unknown [10.22.88.52]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B95643000197; Wed, 4 Dec 2024 11:43:20 +0000 (UTC) From: Wander Lairson Costa To: Tony Nguyen , Przemek Kitszel , Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Sebastian Andrzej Siewior , Clark Williams , Steven Rostedt , Auke Kok , Jeff Garzik , intel-wired-lan@lists.osuosl.org (moderated list:INTEL ETHERNET DRIVERS), netdev@vger.kernel.org (open list:NETWORKING DRIVERS), linux-kernel@vger.kernel.org (open list), linux-rt-devel@lists.linux.dev (open list:Real-time Linux (PREEMPT_RT):Keyword:PREEMPT_RT) Cc: Wander Lairson Costa Date: Wed, 4 Dec 2024 08:42:24 -0300 Message-ID: <20241204114229.21452-2-wander@redhat.com> In-Reply-To: <20241204114229.21452-1-wander@redhat.com> References: <20241204114229.21452-1-wander@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1733312612; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YjLiIgA1heSf6FeldDRqyic0hKMQLJvylr1VcRItvD4=; b=CXG5Oz80r9ohMkYVtSjngwwuEyVEJ5+svV5vkCi+ulgNzp/NyxibFt0duWOiWUjyDBPFRx eKxPKeA7S27P1VYnhXNAV6pmjixZPBSy9JGs1oc7oB5jCCH4DZHP4nBidRogVZ35FZoLYr clNEG/JBFPlFY97E/gko/4tU6Ivu7r8= X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (1024-bit key, unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=CXG5Oz80 Subject: [Intel-wired-lan] [PATCH iwl-net 1/4] igb: narrow scope of vfs_lock in SR-IOV cleanup X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" The adapter->vfs_lock currently protects critical sections shared between igb_disable_sriov() and igb_msg_task(). Since igb_msg_task() — which is invoked solely by the igb_msix_other() ISR—only proceeds when adapter->vfs_allocated_count > 0, we can reduce the lock scope further. By moving the assignment adapter->vfs_allocated_count = 0 to the start of the cleanup code in igb_disable_sriov(), we can restrict the spinlock protection solely to this assignment. This change removes kfree() calls from within the locked section, simplifying lock management. Once kfree() is outside the vfs_lock scope, it becomes possible to safely convert vfs_lock to a raw_spin_lock. Signed-off-by: Wander Lairson Costa Tested-by: Rafal Romanowski --- drivers/net/ethernet/intel/igb/igb_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c index 08578980b6518..4ca25660e876e 100644 --- a/drivers/net/ethernet/intel/igb/igb_main.c +++ b/drivers/net/ethernet/intel/igb/igb_main.c @@ -3708,12 +3708,12 @@ static int igb_disable_sriov(struct pci_dev *pdev, bool reinit) msleep(500); } spin_lock_irqsave(&adapter->vfs_lock, flags); + adapter->vfs_allocated_count = 0; + spin_unlock_irqrestore(&adapter->vfs_lock, flags); kfree(adapter->vf_mac_list); adapter->vf_mac_list = NULL; kfree(adapter->vf_data); adapter->vf_data = NULL; - adapter->vfs_allocated_count = 0; - spin_unlock_irqrestore(&adapter->vfs_lock, flags); wr32(E1000_IOVCTL, E1000_IOVCTL_REUSE_VFQ); wrfl(); msleep(100); From patchwork Wed Dec 4 11:42:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wander Lairson Costa X-Patchwork-Id: 2018216 X-Patchwork-Delegate: anthony.l.nguyen@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=9wKMYc7l; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Y3Fz969Mmz1yR0 for ; Wed, 4 Dec 2024 22:43:49 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 1B6CC4070B; Wed, 4 Dec 2024 11:43:48 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id wDAD5eEQHqhV; Wed, 4 Dec 2024 11:43:46 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.142; helo=lists1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 2A392407BA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1733312626; bh=qOTYSXLeLG/VXXfQqMnchfUvQ+U6U+p7WeEdqM7zheA=; h=From:To:Cc:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From; b=9wKMYc7lCU8teW/ePYJZT1eJFNfnAyi5EtVnhGYEmi6nkLSMUDVckOz3iHJTcmYVj HZbHa8fzAREsHsnZqQNNSRNPNwOTC2lDMhi3N4h182lAj0tZXhB2jPIkT7r5WWzI1U 13RSQjwYoniJBNYGwnlmEjHYncqK4owlDkb3vI9sphQS4lfZLWB3lh71mLzYcWc+v4 pr0eqPVmYBEoOZAa0LQNplsuwXepriMlR7Pjs1TQ1sV7TIZSAIqZPFpdOqVNNFBtqc pLaZLtCaOfMgurn66kzjqNH9+u+4DbS2wRa68kN8CywrAJby8GlFNTEuntqG9iLwQe OlIQfGPEA0QYw== Received: from lists1.osuosl.org (lists1.osuosl.org [140.211.166.142]) by smtp4.osuosl.org (Postfix) with ESMTP id 2A392407BA; Wed, 4 Dec 2024 11:43:46 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists1.osuosl.org (Postfix) with ESMTP id 272811DD1 for ; Wed, 4 Dec 2024 11:43:45 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 1444E405F0 for ; Wed, 4 Dec 2024 11:43:45 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id tqJ6Uo0XYy4T for ; Wed, 4 Dec 2024 11:43:44 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=wander@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp2.osuosl.org E264C405BF DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org E264C405BF Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id E264C405BF for ; Wed, 4 Dec 2024 11:43:43 +0000 (UTC) Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-317-wgdQ46f1ORGI3g8c15s_sQ-1; Wed, 04 Dec 2024 06:43:37 -0500 X-MC-Unique: wgdQ46f1ORGI3g8c15s_sQ-1 X-Mimecast-MFC-AGG-ID: wgdQ46f1ORGI3g8c15s_sQ Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B5D851953940; Wed, 4 Dec 2024 11:43:34 +0000 (UTC) Received: from wcosta-thinkpadt14gen4.rmtbr.csb (unknown [10.22.88.52]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id CC95C3000197; Wed, 4 Dec 2024 11:43:27 +0000 (UTC) From: Wander Lairson Costa To: Tony Nguyen , Przemek Kitszel , Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Sebastian Andrzej Siewior , Clark Williams , Steven Rostedt , Jeff Garzik , Auke Kok , intel-wired-lan@lists.osuosl.org (moderated list:INTEL ETHERNET DRIVERS), netdev@vger.kernel.org (open list:NETWORKING DRIVERS), linux-kernel@vger.kernel.org (open list), linux-rt-devel@lists.linux.dev (open list:Real-time Linux (PREEMPT_RT):Keyword:PREEMPT_RT) Cc: Wander Lairson Costa , Clark Williams Date: Wed, 4 Dec 2024 08:42:25 -0300 Message-ID: <20241204114229.21452-3-wander@redhat.com> In-Reply-To: <20241204114229.21452-1-wander@redhat.com> References: <20241204114229.21452-1-wander@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1733312622; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qOTYSXLeLG/VXXfQqMnchfUvQ+U6U+p7WeEdqM7zheA=; b=ge5PeAkojdiuZAB9Ws4HY0FtglYH5Lr73dvv+G0iyh0QrkrG5h+xOY+K2AEq19441f+TRe hZjPzKJsy8ZPqQUId+XAqojKGgDzcQy9DhSpdjiWjIM48WqoaYpdhiQ7fhEGazp9cxCLot fKS2RTvNzYG/WFB9jgFfaNFN7o0H/VE= X-Mailman-Original-Authentication-Results: smtp2.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com X-Mailman-Original-Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ge5PeAko Subject: [Intel-wired-lan] [PATCH iwl-net 2/4] igb: introduce raw vfs_lock to igb_adapter X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" This change adds a raw_spinlock for the vfs_lock to the igb_adapter structure, enabling its use in both interrupt and preemptible contexts. This is essential for upcoming modifications to split igb_msg_task() into interrupt-safe and preemptible-safe parts. The motivation for this change stems from the need to modify igb_msix_other() to run in interrupt context under PREEMPT_RT. Currently, igb_msg_task() contains a code path that invokes kcalloc() with the GFP_ATOMIC flag. However, on PREEMPT_RT, GFP_ATOMIC is not honored, making it unsafe to call allocation functions in interrupt context. By introducing this raw spinlock, we can safely acquire the lock in both contexts, paving the way for the necessary restructuring of igb_msg_task(). Signed-off-by: Wander Lairson Costa Suggested-by: Clark Williams Tested-by: Rafal Romanowski --- drivers/net/ethernet/intel/igb/igb.h | 4 ++ drivers/net/ethernet/intel/igb/igb_main.c | 51 ++++++++++++++++++++--- 2 files changed, 50 insertions(+), 5 deletions(-) diff --git a/drivers/net/ethernet/intel/igb/igb.h b/drivers/net/ethernet/intel/igb/igb.h index 3c2dc7bdebb50..d50c22f09d0f8 100644 --- a/drivers/net/ethernet/intel/igb/igb.h +++ b/drivers/net/ethernet/intel/igb/igb.h @@ -666,6 +666,10 @@ struct igb_adapter { struct vf_mac_filter *vf_mac_list; /* lock for VF resources */ spinlock_t vfs_lock; +#ifdef CONFIG_PREEMPT_RT + /* Used to lock VFS in interrupt context under PREEMPT_RT */ + raw_spinlock_t raw_vfs_lock; +#endif }; /* flags controlling PTP/1588 function */ diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c index 4ca25660e876e..9b4235ec226df 100644 --- a/drivers/net/ethernet/intel/igb/igb_main.c +++ b/drivers/net/ethernet/intel/igb/igb_main.c @@ -3657,6 +3657,47 @@ static int igb_probe(struct pci_dev *pdev, const struct pci_device_id *ent) return err; } +#ifdef CONFIG_PREEMPT_RT +static __always_inline void vfs_lock_init(struct igb_adapter *adapter) +{ + spin_lock_init(&adapter->vfs_lock); + raw_spin_lock_init(&adapter->raw_vfs_lock); +} + +static __always_inline void vfs_lock_irqsave(struct igb_adapter *adapter, + unsigned long *flags) +{ + /* + * Remember that under PREEMPT_RT spin_lock_irqsave + * ignores the flags parameter + */ + spin_lock_irqsave(&adapter->vfs_lock, *flags); + raw_spin_lock_irqsave(&adapter->raw_vfs_lock, *flags); +} + +static __always_inline void vfs_unlock_irqrestore(struct igb_adapter *adapter, + unsigned long flags) +{ + raw_spin_unlock_irqrestore(&adapter->raw_vfs_lock, flags); + spin_unlock_irqrestore(&adapter->vfs_lock, flags); +} +#else +static __always_inline void vfs_lock_init(struct igb_adapter *adapter) +{ + spin_lock_init(&adapter->vfs_lock); +} + +static __always_inline void vfs_lock_irqsave(struct igb_adapter *adapter, unsigned long *flags) +{ + spin_lock_irqsave(&adapter->vfs_lock, *flags); +} + +static __always_inline void vfs_unlock_irqrestore(struct igb_adapter *adapter, unsigned long flags) +{ + spin_unlock_irqrestore(&adapter->vfs_lock, flags); +} +#endif + #ifdef CONFIG_PCI_IOV static int igb_sriov_reinit(struct pci_dev *dev) { @@ -3707,9 +3748,9 @@ static int igb_disable_sriov(struct pci_dev *pdev, bool reinit) pci_disable_sriov(pdev); msleep(500); } - spin_lock_irqsave(&adapter->vfs_lock, flags); + vfs_lock_irqsave(adapter, &flags); adapter->vfs_allocated_count = 0; - spin_unlock_irqrestore(&adapter->vfs_lock, flags); + vfs_unlock_irqrestore(adapter, flags); kfree(adapter->vf_mac_list); adapter->vf_mac_list = NULL; kfree(adapter->vf_data); @@ -4042,7 +4083,7 @@ static int igb_sw_init(struct igb_adapter *adapter) spin_lock_init(&adapter->stats64_lock); /* init spinlock to avoid concurrency of VF resources */ - spin_lock_init(&adapter->vfs_lock); + vfs_lock_init(adapter); #ifdef CONFIG_PCI_IOV switch (hw->mac.type) { case e1000_82576: @@ -8035,7 +8076,7 @@ static void igb_msg_task(struct igb_adapter *adapter) unsigned long flags; u32 vf; - spin_lock_irqsave(&adapter->vfs_lock, flags); + vfs_lock_irqsave(adapter, &flags); for (vf = 0; vf < adapter->vfs_allocated_count; vf++) { /* process any reset requests */ if (!igb_check_for_rst(hw, vf)) @@ -8049,7 +8090,7 @@ static void igb_msg_task(struct igb_adapter *adapter) if (!igb_check_for_ack(hw, vf)) igb_rcv_ack_from_vf(adapter, vf); } - spin_unlock_irqrestore(&adapter->vfs_lock, flags); + vfs_unlock_irqrestore(adapter, flags); } /** From patchwork Wed Dec 4 11:42:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wander Lairson Costa X-Patchwork-Id: 2018217 X-Patchwork-Delegate: anthony.l.nguyen@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=8nzHwQ98; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Y3FzH3tyVz1yR0 for ; Wed, 4 Dec 2024 22:43:55 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 064AD40340; Wed, 4 Dec 2024 11:43:54 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id T2WWgqqAW_L5; Wed, 4 Dec 2024 11:43:51 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.142; helo=lists1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org AD756407CD DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1733312631; bh=gAAUCxkswU4jgn9KaJsquRaa+KOEK5hfScrof0xevFg=; h=From:To:Cc:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From; b=8nzHwQ98lGTd9X/AHoJTi3YnjbuDgfB/FZjBd8Le8o7PaczUwpdJkxWeMk/ZmrjUc Pz+XZNYPWYqrpSohMsAPKGvn7etduXW/flA7qoVMSkmQsop5cJljHmCmAAHEYPuyK3 84jpryjMUjKYYe8xb1UIPCEAnbPP2N5INspfz7HWBnzpElDDPiNy79YWW4986VpZ8S YSBX2UhuMjL2UtvWWr111Ratq1iTbzul2vCqGXuJvxwe27m9zPg5V9RX2Nw8mr6Art 5B9Cn/5RTwUmGo1/LoeiJap60L4ptmofwICA5qoqHkRNmgDYWT6FmidkU6t2APl3d3 A9Zu9ok42F0nQ== Received: from lists1.osuosl.org (lists1.osuosl.org [140.211.166.142]) by smtp4.osuosl.org (Postfix) with ESMTP id AD756407CD; Wed, 4 Dec 2024 11:43:51 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists1.osuosl.org (Postfix) with ESMTP id D2B4A1DAA for ; Wed, 4 Dec 2024 11:43:49 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id C165B6069C for ; Wed, 4 Dec 2024 11:43:49 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id yaNr7he7JBic for ; Wed, 4 Dec 2024 11:43:49 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=wander@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org EAA9A6075D DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org EAA9A6075D Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id EAA9A6075D for ; Wed, 4 Dec 2024 11:43:48 +0000 (UTC) Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-177-Nb3gPDAUN-20-dKbI4VyIA-1; Wed, 04 Dec 2024 06:43:44 -0500 X-MC-Unique: Nb3gPDAUN-20-dKbI4VyIA-1 X-Mimecast-MFC-AGG-ID: Nb3gPDAUN-20-dKbI4VyIA Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 15FA91955D83; Wed, 4 Dec 2024 11:43:42 +0000 (UTC) Received: from wcosta-thinkpadt14gen4.rmtbr.csb (unknown [10.22.88.52]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D0B133000197; Wed, 4 Dec 2024 11:43:35 +0000 (UTC) From: Wander Lairson Costa To: Tony Nguyen , Przemek Kitszel , Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Sebastian Andrzej Siewior , Clark Williams , Steven Rostedt , Auke Kok , Jeff Garzik , intel-wired-lan@lists.osuosl.org (moderated list:INTEL ETHERNET DRIVERS), netdev@vger.kernel.org (open list:NETWORKING DRIVERS), linux-kernel@vger.kernel.org (open list), linux-rt-devel@lists.linux.dev (open list:Real-time Linux (PREEMPT_RT):Keyword:PREEMPT_RT) Cc: Wander Lairson Costa Date: Wed, 4 Dec 2024 08:42:26 -0300 Message-ID: <20241204114229.21452-4-wander@redhat.com> In-Reply-To: <20241204114229.21452-1-wander@redhat.com> References: <20241204114229.21452-1-wander@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1733312628; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gAAUCxkswU4jgn9KaJsquRaa+KOEK5hfScrof0xevFg=; b=KjVH8kMkZGIU/BGSWhrumj+CNMKK0lmaOSroUwK1v+5gDO9fbXKaufXELN5pxOwunThVvw 8N5JTBLHpr0QLxx+CTLOZt6kGGVfoJbJNuqkU/bZYb0waK10I1bbQmc4ZgDCZ5FVl+1BTL EW/6FeEwzzgj6mPYcDKYRE7brHTuVJk= X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KjVH8kMk Subject: [Intel-wired-lan] [PATCH iwl-net 3/4] igb: split igb_msg_task() X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" From the perspective of PREEMPT_RT, igb_msg_task() invokes functions that are a mix of IRQ-safe and non-IRQ-safe operations. To address this, we separate igb_msg_task() into distinct IRQ-safe and preemptible-safe components. This is a preparatory step for upcoming commits, where the igb_msix_other interrupt handler will be split into IRQ and threaded handlers, each invoking the appropriate part of the newly divided igb_msg_task(). Signed-off-by: Wander Lairson Costa Tested-by: Rafal Romanowski --- drivers/net/ethernet/intel/igb/igb_main.c | 88 +++++++++++++++++++++-- 1 file changed, 81 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c index 9b4235ec226df..5828831fd29c2 100644 --- a/drivers/net/ethernet/intel/igb/igb_main.c +++ b/drivers/net/ethernet/intel/igb/igb_main.c @@ -149,6 +149,8 @@ static int igb_vlan_rx_kill_vid(struct net_device *, __be16, u16); static void igb_restore_vlan(struct igb_adapter *); static void igb_rar_set_index(struct igb_adapter *, u32); static void igb_ping_all_vfs(struct igb_adapter *); +static void igb_msg_task_irq_safe(struct igb_adapter *adapter); +static void igb_msg_task_preemptible_safe(struct igb_adapter *adapter); static void igb_msg_task(struct igb_adapter *); static void igb_vmm_control(struct igb_adapter *); static int igb_set_vf_mac(struct igb_adapter *, int, unsigned char *); @@ -3681,6 +3683,30 @@ static __always_inline void vfs_unlock_irqrestore(struct igb_adapter *adapter, raw_spin_unlock_irqrestore(&adapter->raw_vfs_lock, flags); spin_unlock_irqrestore(&adapter->vfs_lock, flags); } + +static __always_inline void vfs_spin_lock_irqsave(struct igb_adapter *adapter, + unsigned long *flags) +{ + spin_lock_irqsave(&adapter->vfs_lock, *flags); +} + +static __always_inline void vfs_spin_unlock_irqrestore(struct igb_adapter *adapter, + unsigned long flags) +{ + spin_unlock_irqrestore(&adapter->vfs_lock, flags); +} + +static __always_inline void vfs_raw_spin_lock_irqsave(struct igb_adapter *adapter, + unsigned long *flags) +{ + raw_spin_lock_irqsave(&adapter->raw_vfs_lock, *flags); +} + +static __always_inline void vfs_raw_spin_unlock_irqrestore(struct igb_adapter *adapter, + unsigned long flags) +{ + raw_spin_unlock_irqrestore(&adapter->raw_vfs_lock, flags); +} #else static __always_inline void vfs_lock_init(struct igb_adapter *adapter) { @@ -3696,6 +3722,30 @@ static __always_inline void vfs_unlock_irqrestore(struct igb_adapter *adapter, u { spin_unlock_irqrestore(&adapter->vfs_lock, flags); } + +static __always_inline void vfs_spin_lock_irqsave(struct igb_adapter *adapter, + unsigned long *flags) +{ + spin_lock_irqsave(&adapter->vfs_lock, *flags); +} + +static __always_inline void vfs_spin_unlock_irqrestore(struct igb_adapter *adapter, + unsigned long flags) +{ + spin_unlock_irqrestore(&adapter->vfs_lock, flags); +} + +static __always_inline void vfs_raw_spin_lock_irqsave(struct igb_adapter *adapter, + unsigned long *flags) +{ + spin_lock_irqsave(&adapter->vfs_lock, *flags); +} + +static __always_inline void vfs_raw_spin_unlock_irqrestore(struct igb_adapter *adapter, + unsigned long flags) +{ + spin_unlock_irqrestore(&adapter->vfs_lock, flags); +} #endif #ifdef CONFIG_PCI_IOV @@ -8070,27 +8120,51 @@ static void igb_rcv_msg_from_vf(struct igb_adapter *adapter, u32 vf) igb_unlock_mbx(hw, vf); } -static void igb_msg_task(struct igb_adapter *adapter) +/* + * Note: the split of irq and preempible safe parts of igb_msg_task() + * only makes sense under PREEMPT_RT. + * The root cause of igb_rcv_msg_from_vf() is not IRQ safe is because + * it calls kcalloc with GFP_ATOMIC, but GFP_ATOMIC is not IRQ safe + * in PREEMPT_RT. + */ +static void igb_msg_task_irq_safe(struct igb_adapter *adapter) { struct e1000_hw *hw = &adapter->hw; unsigned long flags; u32 vf; - vfs_lock_irqsave(adapter, &flags); + vfs_raw_spin_lock_irqsave(adapter, &flags); for (vf = 0; vf < adapter->vfs_allocated_count; vf++) { /* process any reset requests */ if (!igb_check_for_rst(hw, vf)) igb_vf_reset_event(adapter, vf); - /* process any messages pending */ - if (!igb_check_for_msg(hw, vf)) - igb_rcv_msg_from_vf(adapter, vf); - /* process any acks */ if (!igb_check_for_ack(hw, vf)) igb_rcv_ack_from_vf(adapter, vf); } - vfs_unlock_irqrestore(adapter, flags); + vfs_raw_spin_unlock_irqrestore(adapter, flags); +} + +static void igb_msg_task_preemptible_safe(struct igb_adapter *adapter) +{ + struct e1000_hw *hw = &adapter->hw; + unsigned long flags; + u32 vf; + + vfs_spin_lock_irqsave(adapter, &flags); + for (vf = 0; vf < adapter->vfs_allocated_count; vf++) { + /* process any messages pending */ + if (!igb_check_for_msg(hw, vf)) + igb_rcv_msg_from_vf(adapter, vf); + } + vfs_spin_unlock_irqrestore(adapter, flags); +} + +static __always_inline void igb_msg_task(struct igb_adapter *adapter) +{ + igb_msg_task_irq_safe(adapter); + igb_msg_task_preemptible_safe(adapter); } /** From patchwork Wed Dec 4 11:42:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wander Lairson Costa X-Patchwork-Id: 2018218 X-Patchwork-Delegate: anthony.l.nguyen@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=6MWEEy3x; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Y3FzQ10MRz1yR0 for ; Wed, 4 Dec 2024 22:44:01 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id A5B1B8175C; Wed, 4 Dec 2024 11:44:00 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id Y8bbXOnS0NVb; Wed, 4 Dec 2024 11:43:58 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.142; helo=lists1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 96DAA8175A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1733312638; bh=h4OzJ2Yicwkoa/Gn8s9Kz6ryvpeCxefm3yl1+gEE2Ro=; h=From:To:Cc:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From; b=6MWEEy3xJ72vNT91yI7pHcfMAsmD+HimXfpvRE+g9wLf2Z9jr7ln5XHNaWnRxXKO7 EOcHpD3WPWmZzblTawxxT+7UgS6hatBxwE+iOeruYyYPTBDEAqXWqSspNfIE37s+wY SyYi474t1Z/69SFfA2ghH7T5V8B5TPOvT7ly7ZrQwBNHaTIKmk190ME32KecZQY4UD i10eWo2fgrd3oD+hguKDFtoNgco7Z8i0EoVZ8tbsRboM1g/1tqOBPxM0kmVGy1B1sb B0BhgrJ9DSJxhQ3tHQY6N71zqz3LPevzlp/sor9NAZKjqETPKL81te+gaCBRCAza1O LJwx9Z6xKjaKA== Received: from lists1.osuosl.org (lists1.osuosl.org [140.211.166.142]) by smtp1.osuosl.org (Postfix) with ESMTP id 96DAA8175A; Wed, 4 Dec 2024 11:43:58 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists1.osuosl.org (Postfix) with ESMTP id 153A51DAA for ; Wed, 4 Dec 2024 11:43:57 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id E881E405F0 for ; Wed, 4 Dec 2024 11:43:56 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id n4I0NH-hAyLY for ; Wed, 4 Dec 2024 11:43:56 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=wander@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp2.osuosl.org BBC94405BF DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org BBC94405BF Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id BBC94405BF for ; Wed, 4 Dec 2024 11:43:55 +0000 (UTC) Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-602-5TspYdiBP0ab5yCLjwMkAw-1; Wed, 04 Dec 2024 06:43:52 -0500 X-MC-Unique: 5TspYdiBP0ab5yCLjwMkAw-1 X-Mimecast-MFC-AGG-ID: 5TspYdiBP0ab5yCLjwMkAw Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2DED91955F3F; Wed, 4 Dec 2024 11:43:49 +0000 (UTC) Received: from wcosta-thinkpadt14gen4.rmtbr.csb (unknown [10.22.88.52]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 04D023000197; Wed, 4 Dec 2024 11:43:42 +0000 (UTC) From: Wander Lairson Costa To: Tony Nguyen , Przemek Kitszel , Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Sebastian Andrzej Siewior , Clark Williams , Steven Rostedt , Auke Kok , Jeff Garzik , intel-wired-lan@lists.osuosl.org (moderated list:INTEL ETHERNET DRIVERS), netdev@vger.kernel.org (open list:NETWORKING DRIVERS), linux-kernel@vger.kernel.org (open list), linux-rt-devel@lists.linux.dev (open list:Real-time Linux (PREEMPT_RT):Keyword:PREEMPT_RT) Cc: Wander Lairson Costa , Yuying Ma Date: Wed, 4 Dec 2024 08:42:27 -0300 Message-ID: <20241204114229.21452-5-wander@redhat.com> In-Reply-To: <20241204114229.21452-1-wander@redhat.com> References: <20241204114229.21452-1-wander@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1733312634; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h4OzJ2Yicwkoa/Gn8s9Kz6ryvpeCxefm3yl1+gEE2Ro=; b=HHvXOgc1/b/NFH/OO8tNaZE7NPxwB2kztncaIRbALSq64vgWJeqp/ANNt2vzHuyejHrDrZ Gznu2M7o7NXNnhiRfvhtKTzArSFpfRq0fSjQcpSukiNf/y9/0DsSlNsi2qlO++ZG8oKykg pofYVRa05cxeLb8QyiTMAlrn7QeO+js= X-Mailman-Original-Authentication-Results: smtp2.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com X-Mailman-Original-Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=HHvXOgc1 Subject: [Intel-wired-lan] [PATCH iwl-net 4/4] igb: fix igb_msix_other() handling for PREEMPT_RT X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" During testing of SR-IOV, Red Hat QE encountered an issue where the ip link up command intermittently fails for the igbvf interfaces when using the PREEMPT_RT variant. Investigation revealed that e1000_write_posted_mbx returns an error due to the lack of an ACK from e1000_poll_for_ack. The underlying issue arises from the fact that IRQs are threaded by default under PREEMPT_RT. While the exact hardware details are not available, it appears that the IRQ handled by igb_msix_other must be processed before e1000_poll_for_ack times out. However, e1000_write_posted_mbx is called with preemption disabled, leading to a scenario where the IRQ is serviced only after the failure of e1000_write_posted_mbx. Commit 338c4d3902fe ("igb: Disable threaded IRQ for igb_msix_other") forced the ISR to run in a non-threaded context. However, Sebastian observed that some functions called within the ISR acquire locks that may sleep. In the previous two patches, we managed to make igb_msg_mask() safe to call from an interrupt context. In this commit, we move most of the ISR handling to an interrupt context, leaving non IRQ safe code to be called from the thread context under PREEMPT_RT. Reproducer: ipaddr_vlan=3 nic_test=ens14f0 vf=${nic_test}v0 # The main testing steps: while true; do ip link set ${nic_test} mtu 1500 ip link set ${vf} mtu 1500 ip link set $vf up # 3. set vlan and ip for VF ip link set ${nic_test} vf 0 vlan ${ipaddr_vlan} ip addr add 172.30.${ipaddr_vlan}.1/24 dev ${vf} ip addr add 2021:db8:${ipaddr_vlan}::1/64 dev ${vf} # 4. check the link state for VF and PF ip link show ${nic_test} if ! ip link show $vf | grep 'state UP'; then echo 'Error found' break fi ip link set $vf down done You can also reproduce it more reliably by setting nr_cpus=1 in the kernel command line. Fixes: 9d5c824399de ("igb: PCI-Express 82575 Gigabit Ethernet driver") Signed-off-by: Wander Lairson Costa Reported-by: Yuying Ma Tested-by: Rafal Romanowski --- drivers/net/ethernet/intel/igb/igb_main.c | 35 ++++++++++++++++------- 1 file changed, 24 insertions(+), 11 deletions(-) diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c index 5828831fd29c2..b2894cebe2c9e 100644 --- a/drivers/net/ethernet/intel/igb/igb_main.c +++ b/drivers/net/ethernet/intel/igb/igb_main.c @@ -131,6 +131,7 @@ static void igb_set_uta(struct igb_adapter *adapter, bool set); static irqreturn_t igb_intr(int irq, void *); static irqreturn_t igb_intr_msi(int irq, void *); static irqreturn_t igb_msix_other(int irq, void *); +static irqreturn_t igb_msix_other_threaded(int irq, void *); static irqreturn_t igb_msix_ring(int irq, void *); #ifdef CONFIG_IGB_DCA static void igb_update_dca(struct igb_q_vector *); @@ -151,7 +152,6 @@ static void igb_rar_set_index(struct igb_adapter *, u32); static void igb_ping_all_vfs(struct igb_adapter *); static void igb_msg_task_irq_safe(struct igb_adapter *adapter); static void igb_msg_task_preemptible_safe(struct igb_adapter *adapter); -static void igb_msg_task(struct igb_adapter *); static void igb_vmm_control(struct igb_adapter *); static int igb_set_vf_mac(struct igb_adapter *, int, unsigned char *); static void igb_flush_mac_table(struct igb_adapter *); @@ -908,8 +908,9 @@ static int igb_request_msix(struct igb_adapter *adapter) struct net_device *netdev = adapter->netdev; int i, err = 0, vector = 0, free_vector = 0; - err = request_irq(adapter->msix_entries[vector].vector, - igb_msix_other, 0, netdev->name, adapter); + err = request_threaded_irq(adapter->msix_entries[vector].vector, + igb_msix_other, igb_msix_other_threaded, + IRQF_NO_THREAD, netdev->name, adapter); if (err) goto err_out; @@ -7113,9 +7114,27 @@ static irqreturn_t igb_msix_other(int irq, void *data) igb_check_wvbr(adapter); } - /* Check for a mailbox event */ + /* Check for a mailbox event (interrupt safe part) */ if (icr & E1000_ICR_VMMB) - igb_msg_task(adapter); + igb_msg_task_irq_safe(adapter); + + adapter->test_icr = icr; + + if (!IS_ENABLED(CONFIG_PREEMPT_RT)) + return igb_msix_other_threaded(irq, data); + + return IRQ_WAKE_THREAD; +} + +static irqreturn_t igb_msix_other_threaded(int irq, void *data) +{ + struct igb_adapter *adapter = data; + struct e1000_hw *hw = &adapter->hw; + u32 icr = adapter->test_icr; + + /* Check for a mailbox event (preempible safe part) */ + if (icr & E1000_ICR_VMMB) + igb_msg_task_preemptible_safe(adapter); if (icr & E1000_ICR_LSC) { hw->mac.get_link_status = 1; @@ -8161,12 +8180,6 @@ static void igb_msg_task_preemptible_safe(struct igb_adapter *adapter) vfs_spin_unlock_irqrestore(adapter, flags); } -static __always_inline void igb_msg_task(struct igb_adapter *adapter) -{ - igb_msg_task_irq_safe(adapter); - igb_msg_task_preemptible_safe(adapter); -} - /** * igb_set_uta - Set unicast filter table address * @adapter: board private structure