| Message ID | 20240723132958.41320-2-marek.vasut+renesas@mailbox.org |
|---|---|
| State | Handled Elsewhere |
| Headers | show |
| Series | genirq/msi: Silence set affinity failed warning | expand |
On 7/23/24 10:27 PM, Marek Vasut wrote: > Various PCIe controllers that mux MSIs onto single IRQ line produce these > "IRQ%d: set affinity failed" warnings when entering suspend. This has been > discussed before [1] [2] and an example test case is included at the end > of this commit message. > > Controller drivers which create MSI IRQ domain with MSI_FLAG_USE_DEF_CHIP_OPS > flag set and which do not override the .irq_set_affinity irqchip callback get > assigned default .irq_set_affinity = msi_domain_set_affinity() callback. That > is not desired on controllers where it is not possible to set affinity of each > MSI IRQ line to a specific CPU core due to hardware limitation. > > Introduce dedicated flag MSI_FLAG_NO_AFFINITY, which keeps .irq_set_affinity > unset in case the controller driver did not assign the callback. This way, the > migrate_one_irq() code in cpuhotplug.c can exit right away, without printing > the aforementioned warning. The .irq_set_affinity implementations which only > return -EINVAL can be removed from multiple controller drivers. > > ``` > $ grep 25 /proc/interrupts > 25: 0 0 0 0 0 0 0 0 PCIe MSI 0 Edge PCIe PME > > $ echo core > /sys/power/pm_test ; echo mem > /sys/power/state > ... > Disabling non-boot CPUs ... > IRQ25: set affinity failed(-22). <---------- This is being silenced here > psci: CPU7 killed (polled 4 ms) > ... > ``` > > [1] https://lore.kernel.org/all/d4a6eea3c5e33a3a4056885419df95a7@kernel.org/ > [2] https://lore.kernel.org/all/5f4947b18bf381615a37aa81c2242477@kernel.org/ > > Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org> Looks good to me. Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
diff --git a/include/linux/msi.h b/include/linux/msi.h index 9449797638255..b10093c4d00ea 100644 --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -554,6 +554,8 @@ enum { MSI_FLAG_MSIX_CONTIGUOUS = (1 << 19), /* PCI/MSI-X vectors can be dynamically allocated/freed post MSI-X enable */ MSI_FLAG_PCI_MSIX_ALLOC_DYN = (1 << 20), + /* PCI MSIs cannot be steered separately to CPU cores */ + MSI_FLAG_NO_AFFINITY = (1 << 21), }; /** diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c index 5fa0547ece0c4..ca6e2ae6d6fc0 100644 --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -832,7 +832,7 @@ static void msi_domain_update_chip_ops(struct msi_domain_info *info) struct irq_chip *chip = info->chip; BUG_ON(!chip || !chip->irq_mask || !chip->irq_unmask); - if (!chip->irq_set_affinity) + if (!chip->irq_set_affinity && !(info->flags & MSI_FLAG_NO_AFFINITY)) chip->irq_set_affinity = msi_domain_set_affinity; }
Various PCIe controllers that mux MSIs onto single IRQ line produce these "IRQ%d: set affinity failed" warnings when entering suspend. This has been discussed before [1] [2] and an example test case is included at the end of this commit message. Controller drivers which create MSI IRQ domain with MSI_FLAG_USE_DEF_CHIP_OPS flag set and which do not override the .irq_set_affinity irqchip callback get assigned default .irq_set_affinity = msi_domain_set_affinity() callback. That is not desired on controllers where it is not possible to set affinity of each MSI IRQ line to a specific CPU core due to hardware limitation. Introduce dedicated flag MSI_FLAG_NO_AFFINITY, which keeps .irq_set_affinity unset in case the controller driver did not assign the callback. This way, the migrate_one_irq() code in cpuhotplug.c can exit right away, without printing the aforementioned warning. The .irq_set_affinity implementations which only return -EINVAL can be removed from multiple controller drivers. ``` $ grep 25 /proc/interrupts 25: 0 0 0 0 0 0 0 0 PCIe MSI 0 Edge PCIe PME $ echo core > /sys/power/pm_test ; echo mem > /sys/power/state ... Disabling non-boot CPUs ... IRQ25: set affinity failed(-22). <---------- This is being silenced here psci: CPU7 killed (polled 4 ms) ... ``` [1] https://lore.kernel.org/all/d4a6eea3c5e33a3a4056885419df95a7@kernel.org/ [2] https://lore.kernel.org/all/5f4947b18bf381615a37aa81c2242477@kernel.org/ Signed-off-by: Marek Vasut <marek.vasut+renesas@mailbox.org> --- Cc: "Krzysztof Wilczyński" <kw@linux.com> Cc: "Pali Rohár" <pali@kernel.org> Cc: "Uwe Kleine-König" <u.kleine-koenig@pengutronix.de> Cc: Aleksandr Mishin <amishin@t-argos.ru> Cc: Anna-Maria Behnsen <anna-maria@linutronix.de> Cc: Anup Patel <apatel@ventanamicro.com> Cc: Bjorn Helgaas <bhelgaas@google.com> Cc: Broadcom internal kernel review list <bcm-kernel-feedback-list@broadcom.com> Cc: Daire McNamara <daire.mcnamara@microchip.com> Cc: Damien Le Moal <dlemoal@kernel.org> Cc: Florian Fainelli <florian.fainelli@broadcom.com> Cc: Hou Zhiqiang <Zhiqiang.Hou@nxp.com> Cc: Jianjun Wang <jianjun.wang@mediatek.com> Cc: Jim Quinlan <jim2101024@gmail.com> Cc: Jingoo Han <jingoohan1@gmail.com> Cc: Jisheng Zhang <Jisheng.Zhang@synaptics.com> Cc: Jon Hunter <jonathanh@nvidia.com> Cc: Jonathan Derrick <jonathan.derrick@linux.dev> Cc: Jonathan Hunter <jonathanh@nvidia.com> Cc: Joyce Ooi <joyce.ooi@intel.com> Cc: Karthikeyan Mitran <m.karthikeyan@mobiveil.co.in> Cc: Kishon Vijay Abraham I <kishon@kernel.org> Cc: Koichiro Den <den@valinux.co.jp> Cc: Lorenzo Pieralisi <lpieralisi@kernel.org> Cc: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> Cc: Marc Zyngier <maz@kernel.org> Cc: Michal Simek <michal.simek@amd.com> Cc: Nicolas Saenz Julienne <nsaenz@kernel.org> Cc: Niklas Cassel <cassel@kernel.org> Cc: Nipun Gupta <nipun.gupta@amd.com> Cc: Nirmal Patel <nirmal.patel@linux.intel.com> Cc: Rob Herring <robh@kernel.org> Cc: Ryder Lee <ryder.lee@mediatek.com> Cc: Shivamurthy Shastri <shivamurthy.shastri@linutronix.de> Cc: Siddharth Vadapalli <s-vadapalli@ti.com> Cc: Thierry Reding <thierry.reding@gmail.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com> Cc: linux-arm-kernel@lists.infradead.org Cc: linux-mediatek@lists.infradead.org Cc: linux-pci@vger.kernel.org Cc: linux-renesas-soc@vger.kernel.org Cc: linux-rpi-kernel@lists.infradead.org Cc: linux-tegra@vger.kernel.org --- V2: - Introduce MSI_FLAG_NO_AFFINITY to inhibit assignment of msi_domain_set_affinity() V3: - Replace MSI_FLAG_USE_DEF_CHIP_OPS_NOAFF with MSI_FLAG_NO_AFFINITY and make MSI_FLAG_NO_AFFINITY into separate flag - Update commit message - Rebase on current linux-next - Use genirq/msi: subject prefix which is likely better fit now - Split off R-Car part of the patch V4: No change --- include/linux/msi.h | 2 ++ kernel/irq/msi.c | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-)