From patchwork Wed Feb 16 15:29:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Lane X-Patchwork-Id: 1593783 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=sKmw5jvA; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JzMMY56Cvz9sFw for ; Thu, 17 Feb 2022 02:29:45 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1nKMFP-0003aA-I8; Wed, 16 Feb 2022 15:29:39 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1nKMFL-0003ZI-LJ for kernel-team@lists.ubuntu.com; Wed, 16 Feb 2022 15:29:35 +0000 Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id 1BEFF4060A for ; Wed, 16 Feb 2022 15:29:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1645025375; bh=Gpg7ZdSHCrVM1SamCCe8cI/PCwVeAwhay8b5Rbfkjmw=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=sKmw5jvAJRDCsABETl7OT8bFxG1tflFVXyy6euw2/CntgPnMx4Qha7yaIXvbRWziO snW9Bh9wRNtzfPIB64whmxqk+2XFIjpJANU3Va+u1snjG0DLetot7nuFhe3dnSzRBF 6Tban7EplDHd5H/MBuGQZTaYMKNpFeACZWxFrBD5pXiaRk9Gb7JZShKQtXyxt/yefK 6RRKpFahGoqfNtqTM4p6KUXzWs0evWszQknC9USqz+3nxVZGShW30MWGULCdM7PeAc 6JK+vRUoF0eHLX5kPE+j00hNnBfMkjwzcKWGaoKYEJ6HFw7T+FpBq7KNjdM95QoD4K xn5xMCBgJGsGQ== Received: by mail-qk1-f197.google.com with SMTP id q5-20020a05620a0d8500b004738c1b48beso1421923qkl.7 for ; Wed, 16 Feb 2022 07:29:35 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Gpg7ZdSHCrVM1SamCCe8cI/PCwVeAwhay8b5Rbfkjmw=; b=h56eWtaQrYZw/+0iZfA4AKEm/jHKGJFvyN/N2bpTo8H58ZqaiX5B/YgQuqSFXu8E+t is5byJQ0bgMh/ZEqccrF9xNh/Of10PUnycLvIamd62rUVUc25lVHzx7FsfrKL3ZkYFkt 5QdxWw2NeArbqw8rmgyzKaWXVA91NLsW3w9vu/C9t2gw7QZC2iDvizJPMFevGsWWU3Qu eQO+CemszFnSqix5nK9b4r/Ei+45nOZ+DQ9e0AoyA58ExS1NziABiGuQkZTSMslCCYFw e8+JOAvoagk0YvYUbij8fHR/iWjOCDO2JPCvYGiU2Pp9GyhiLVH5ERyHpxYiWggp7AP2 zj0g== X-Gm-Message-State: AOAM5324BasWGlqdJlnkl+eKLWv6gMd5TSlRPoffRAe6aFh5hD8rqLL3 l2W/UYbBEDFUhoZCe0GgpO6oW169io4YHG3BjL8LToZ7uH/aP6dW1x7ExQ5Hjs6yOuhzXb6atuh Hj6HcjbeCGhvwH0kRUmG0YDW1azMaeCgZHhmRwScUtw== X-Received: by 2002:ad4:4b2b:0:b0:42c:2fd7:5705 with SMTP id s11-20020ad44b2b000000b0042c2fd75705mr2252419qvw.4.1645025373642; Wed, 16 Feb 2022 07:29:33 -0800 (PST) X-Google-Smtp-Source: ABdhPJy6eJSie+kUjlFWUUlzFjyuU3bckxpttJfTTWo6V23j+SfSheB89kWVW//RPvw5WPcVHW4z2g== X-Received: by 2002:ad4:4b2b:0:b0:42c:2fd7:5705 with SMTP id s11-20020ad44b2b000000b0042c2fd75705mr2252397qvw.4.1645025373201; Wed, 16 Feb 2022 07:29:33 -0800 (PST) Received: from localhost.localdomain ([71.217.161.186]) by smtp.gmail.com with ESMTPSA id de43sm18286342qkb.4.2022.02.16.07.29.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Feb 2022 07:29:32 -0800 (PST) From: Jeff Lane To: kernel-team@lists.ubuntu.com Subject: [PATCH 1/1] PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU Date: Wed, 16 Feb 2022 10:29:29 -0500 Message-Id: <20220216152929.2145997-2-jeffrey.lane@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220216152929.2145997-1-jeffrey.lane@canonical.com> References: <20220216152929.2145997-1-jeffrey.lane@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Adrian Huang BugLink: https://bugs.launchpad.net/bugs/1937295 When enabling VMD in BIOS setup (Ice Lake Processor: Whitley platform), the host OS cannot boot successfully with the following error message: nvme nvme0: I/O 12 QID 0 timeout, completion polled nvme nvme0: Shutdown timeout set to 6 seconds DMAR: DRHD: handling fault status reg 2 DMAR: [INTR-REMAP] Request device [0x00:0x00.5] fault index 0xa00 [fault reason 0x25] Blocked a compatibility format interrupt request The request device is the VMD controller: # lspci -s 0000:00.5 -nn 0000:00:00.5 RAID bus controller [0104]: Intel Corporation Volume Management Device NVMe RAID Controller [8086:28c0] (rev 04) `git bisect` points to this offending commit ee81ee84f873 ("PCI: vmd: Disable MSI-X remapping when possible"), which disables VMD MSI remapping. The IOMMU hardware blocks the compatibility format interrupt request because Interrupt Remapping Enable Status (IRES) and Extended Interrupt Mode Enable (EIME) are enabled. Please refer to section "5.1.4 Interrupt-Remapping Hardware Operation" in Intel VT-d spec. To fix the issue, VMD driver still enables the interrupt remapping irrespective of VMD_FEAT_CAN_BYPASS_MSI_REMAP if the IOMMU subsystem enables the interrupt remapping. Test configuration is shown as follows: * Two VMD controllers 1. 8086:28c0 (Whitley's VMD) 2. 8086:201d (Purley's VMD: The issue does not appear in this controller. Just make sure if any side effect occurs.) * w/wo intremap=off Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=214219 Link: https://lore.kernel.org/r/20210901124047.1615-1-adrianhuang0701@gmail.com Signed-off-by: Adrian Huang Signed-off-by: Lorenzo Pieralisi Reviewed-by: Jon Derrick Cc: Jon Derrick Cc: Nirmal Patel Cc: Joerg Roedel (cherry picked from commit 2565e5b69c44b4e42469afea3cc5a97e74d1ed45) Signed-off-by: Jeff Lane --- drivers/pci/controller/vmd.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c index e3fcdfec58b3..db72932d049f 100644 --- a/drivers/pci/controller/vmd.c +++ b/drivers/pci/controller/vmd.c @@ -6,6 +6,7 @@ #include #include +#include #include #include #include @@ -710,7 +711,8 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features) * acceptable because the guest is usually CPU-limited and MSI * remapping doesn't become a performance bottleneck. */ - if (!(features & VMD_FEAT_CAN_BYPASS_MSI_REMAP) || + if (iommu_capable(vmd->dev->dev.bus, IOMMU_CAP_INTR_REMAP) || + !(features & VMD_FEAT_CAN_BYPASS_MSI_REMAP) || offset[0] || offset[1]) { ret = vmd_alloc_irqs(vmd); if (ret)