From patchwork Tue Oct 22 11:29:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philip Cox X-Patchwork-Id: 2000450 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=185.125.189.65; helo=lists.ubuntu.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=patchwork.ozlabs.org) Received: from lists.ubuntu.com (lists.ubuntu.com [185.125.189.65]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XXqjR2xzzz1xvv for ; Tue, 22 Oct 2024 22:30:19 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=lists.ubuntu.com) by lists.ubuntu.com with esmtp (Exim 4.86_2) (envelope-from ) id 1t3D5V-0006Ys-9j; Tue, 22 Oct 2024 11:30:09 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by lists.ubuntu.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1t3D5S-0006WI-EE for kernel-team@lists.ubuntu.com; Tue, 22 Oct 2024 11:30:06 +0000 Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 40C6B3FA67 for ; Tue, 22 Oct 2024 11:30:06 +0000 (UTC) Received: by mail-qk1-f200.google.com with SMTP id af79cd13be357-7ac8f684cdaso1075978885a.3 for ; Tue, 22 Oct 2024 04:30:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729596604; x=1730201404; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sM9gbvGsgjCR4092q2LFDzTKMjA2KqZTqCySYm3aI2M=; b=H0kn3Nm4rVQ9dMIYncTLWlGIYJqCl9ObzuneA5dkGvXqCIoUgsl+UsADKYHwBK6l9X PStJ9bWZ8+cviyU1KDcrK3uhMEZh3jFWi2pgNCFZ9VAJ7bfJy7X80Y0haJ/NgsKqDm6U nV7mCcinJnAueba+DosBOb/dmLB6P0ACyxeSiMApZ3C64Y1/xgQtJSyJyREsCgOxs602 wvz4ldUIHv6j1nks/psfciLoNWJbks677fSzuHOk4RSxVr3jecZUAGBae/QvorEjBYIs QHQZx5H+OpBkwC0Nn9NkZgV7EXGnRqTxaJbs+/iOLF3Cm9haBl7hcMX6Kx8yV+esW3/N fnOQ== X-Gm-Message-State: AOJu0Yxtd7qu9OYPKN4kLTArHsBGqtiRTR+aUjWvf6o5YxJNdbpyRKsb tb4orIhmE5x6uqi/ppe+FoihrFU6mka3/kpvWrDMSeYtFHBit/GfUr3Rqt4vTKUAR3cJa3fiwLy +H0sv8hPQGwBuQG8BEeqfUuyJP1z6r0J7gTcZB18tJqosyLpe28u5X5mMfyhtHM737Oad+tgL6k 8IqSE3ivA/9g== X-Received: by 2002:a05:620a:31a0:b0:7b1:48e0:de3 with SMTP id af79cd13be357-7b175237df0mr402130485a.58.1729596603764; Tue, 22 Oct 2024 04:30:03 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHOa++DYEVO6Q87539mCPAx6MpqyrEWWJ2GUd9DmM9F2oFyFgE5Lnv7ynvd8m+kbjQh/0/41A== X-Received: by 2002:a05:620a:31a0:b0:7b1:48e0:de3 with SMTP id af79cd13be357-7b175237df0mr402126485a.58.1729596603352; Tue, 22 Oct 2024 04:30:03 -0700 (PDT) Received: from cox.home.arpa ([174.88.10.82]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7b165a883f1sm268426685a.130.2024.10.22.04.29.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Oct 2024 04:29:59 -0700 (PDT) From: Philip Cox To: kernel-team@lists.ubuntu.com Subject: [SRU][AWS][J][[PATCH 2/2] UBUNTU SAUCE: (no-up) linux/ena: Add NUMA aware interrupt allocation Date: Tue, 22 Oct 2024 07:29:33 -0400 Message-Id: <20241022112935.23113-5-philip.cox@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241022112935.23113-1-philip.cox@canonical.com> References: <20241022112935.23113-1-philip.cox@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" BugLink: https://bugs.launchpad.net/bugs/2085159 This patch implements an improved allocation strategy for IO interrupts, taking into account the physical device NUMA node. Also, it replaces the use of a deprecated API (irq_set_affinity_hint) which enforces affinity with a newer API (irq_update_affinity_hint) which only hints at the preferred CPU mask. Signed-off-by: Osama Abboud (content derived from https://github.com/amzn/amzn-drivers/commit/ed7754cdf6ca0d4fb3d76906abc17718cc3ffa23) Signed-off-by: Philip Cox --- drivers/net/ethernet/amazon/ena/ena_netdev.c | 21 +++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c index 9de2ea7dbeec..51334c3a1d26 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c @@ -2181,16 +2181,20 @@ static void ena_setup_mgmnt_intr(struct ena_adapter *adapter) static void ena_setup_io_intr(struct ena_adapter *adapter) { + const struct cpumask *affinity = cpu_online_mask; + int irq_idx, i, cpu, io_queue_count, node; struct net_device *netdev; - int irq_idx, i, cpu; - int io_queue_count; netdev = adapter->netdev; io_queue_count = adapter->num_io_queues + adapter->xdp_num_queues; + node = dev_to_node(adapter->ena_dev->dmadev); + + if (node != NUMA_NO_NODE) + affinity = cpumask_of_node(node); for (i = 0; i < io_queue_count; i++) { irq_idx = ENA_IO_IRQ_IDX(i); - cpu = i % num_online_cpus(); + cpu = cpumask_local_spread(i, node); snprintf(adapter->irq_tbl[irq_idx].name, ENA_IRQNAME_SIZE, "%s-Tx-Rx-%d", netdev->name, i); @@ -2200,8 +2204,7 @@ static void ena_setup_io_intr(struct ena_adapter *adapter) pci_irq_vector(adapter->pdev, irq_idx); adapter->irq_tbl[irq_idx].cpu = cpu; - cpumask_set_cpu(cpu, - &adapter->irq_tbl[irq_idx].affinity_hint_mask); + cpumask_copy(&adapter->irq_tbl[irq_idx].affinity_hint_mask, affinity); } } @@ -2224,7 +2227,7 @@ static int ena_request_mgmnt_irq(struct ena_adapter *adapter) "Set affinity hint of mgmnt irq.to 0x%lx (irq vector: %d)\n", irq->affinity_hint_mask.bits[0], irq->vector); - irq_set_affinity_hint(irq->vector, &irq->affinity_hint_mask); + irq_update_affinity_hint(irq->vector, &irq->affinity_hint_mask); return rc; } @@ -2257,7 +2260,7 @@ static int ena_request_io_irq(struct ena_adapter *adapter) "Set affinity hint of irq. index %d to 0x%lx (irq vector: %d)\n", i, irq->affinity_hint_mask.bits[0], irq->vector); - irq_set_affinity_hint(irq->vector, &irq->affinity_hint_mask); + irq_update_affinity_hint(irq->vector, &irq->affinity_hint_mask); } return rc; @@ -2277,7 +2280,7 @@ static void ena_free_mgmnt_irq(struct ena_adapter *adapter) irq = &adapter->irq_tbl[ENA_MGMNT_IRQ_IDX]; synchronize_irq(irq->vector); - irq_set_affinity_hint(irq->vector, NULL); + irq_update_affinity_hint(irq->vector, NULL); free_irq(irq->vector, irq->data); } @@ -2296,7 +2299,7 @@ static void ena_free_io_irq(struct ena_adapter *adapter) for (i = ENA_IO_IRQ_FIRST_IDX; i < ENA_MAX_MSIX_VEC(io_queue_count); i++) { irq = &adapter->irq_tbl[i]; - irq_set_affinity_hint(irq->vector, NULL); + irq_update_affinity_hint(irq->vector, NULL); free_irq(irq->vector, irq->data); } }