From patchwork Mon Oct 31 12:33:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 1697175 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=daynix-com.20210112.gappssmtp.com header.i=@daynix-com.20210112.gappssmtp.com header.a=rsa-sha256 header.s=20210112 header.b=5sW7IDud; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N1CL140tZz23l6 for ; Mon, 31 Oct 2022 23:35:37 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1opTzu-0003kb-8e; Mon, 31 Oct 2022 08:34:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1opTz0-000313-Jp for qemu-devel@nongnu.org; Mon, 31 Oct 2022 08:33:46 -0400 Received: from mail-pl1-x632.google.com ([2607:f8b0:4864:20::632]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1opTyx-00049O-Br for qemu-devel@nongnu.org; Mon, 31 Oct 2022 08:33:37 -0400 Received: by mail-pl1-x632.google.com with SMTP id d24so10613799pls.4 for ; Mon, 31 Oct 2022 05:33:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Nr0YW9yWzkpPk77jiuUxzZwbjjkwdTQ1TcUuJ89L3jM=; b=5sW7IDud1o1MLCdzFhhAEoviI4jujWiGEtqNBKU1SY2A0ZCuyx8kOfZDmVNZYsPHA6 ip4OBQnYUhYtNPV1A1EyE5aRgi9HeXUeGfa+7f4M1ZR7AX85tu8DcD9qNqR+pEJvqkaU 6LJhdurILctxIfkmztQB52ysjEpj0YU0KB/78sPimt0Cewyw/Pey0AI53jO73xzdeJDK noHT6aqSBM2kwHysEIQlIp3A9F5pijjJOY8YepKBnInF7YyOEhogMIEOsdVPPtGyZqdl NUK5PkcQ7aI3FYD+1X2awM3BOj/lb5q8V27I6N9RwXup/WuTt+aYGYpwDJ4KAoK2ntUr Cnrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Nr0YW9yWzkpPk77jiuUxzZwbjjkwdTQ1TcUuJ89L3jM=; b=eqJuduySgWxML5GxfitC4/hkGyexlKdo8pRPhTTYgdPn2vZO9whEPVco7yPv6hS0qc z2ZHcLrSCmm+53D9Yoaq7GQ1Cqx52PEvh+IL6yUV+vNOFmrtJgycjwclvCNtXrVEmY1W meUBokD6JRog3uW0/kU0m8TJptLGuqh+bMp4/DeAORleq8ZxbZHXQ9pYJWp8lyHMJR/5 qkKxjPaut7gqZiOXOTxR+0Fl7iSHOZ0s/0POUogekgJew1f4brA6DjOEomlKt5m3wsUC ksQcbbhZsgt0VNY+yVWPcuh8tvOB9rc6Ch/DJ0s9CU894YPyRv+CVtr2ljMhxKPSPkRG 7lCQ== X-Gm-Message-State: ACrzQf2I/J3N3ACsuagAc+OV+6q5aF4Iu4aKX5wKP4H1O664ab/fGbV+ 6hZAcSORTl0kBV9lCJ06ukugjHntbWwDvbP6 X-Google-Smtp-Source: AMsMyM42Vkgpe0miKhu7+oszB6wdJr+B9dITmRbEtAkS3LntApdGcWgPUFa/uQu8PxSpF3FWoyYXAg== X-Received: by 2002:a17:902:b10d:b0:187:29fe:bda0 with SMTP id q13-20020a170902b10d00b0018729febda0mr2209432plr.16.1667219613728; Mon, 31 Oct 2022 05:33:33 -0700 (PDT) Received: from fedora.flets-east.jp ([2400:4050:c360:8200:8ae8:3c4:c0da:7419]) by smtp.gmail.com with ESMTPSA id m10-20020a6545ca000000b0042b5095b7b4sm4093810pgr.5.2022.10.31.05.33.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Oct 2022 05:33:33 -0700 (PDT) From: Akihiko Odaki To: Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, qemu-arm@nongnu.org, "Michael S . Tsirkin" , Marcel Apfelbaum , Gerd Hoffmann , Paolo Bonzini , Richard Henderson , Eduardo Habkost , John Snow , Dmitry Fleytman , Jason Wang , Stefan Weil , Keith Busch , Klaus Jensen , Peter Maydell , Andrey Smirnov , Paul Burton , Aleksandar Rikalo , Yan Vugenfirer , Yuri Benditovich , Akihiko Odaki Subject: [PATCH v6 01/17] hw/vfio/pci: Ensure MSI and MSI-X do not overlap Date: Mon, 31 Oct 2022 21:33:03 +0900 Message-Id: <20221031123319.21532-2-akihiko.odaki@daynix.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221031123319.21532-1-akihiko.odaki@daynix.com> References: <20221031123319.21532-1-akihiko.odaki@daynix.com> MIME-Version: 1.0 Received-SPF: none client-ip=2607:f8b0:4864:20::632; envelope-from=akihiko.odaki@daynix.com; helo=mail-pl1-x632.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org+incoming=patchwork.ozlabs.org@nongnu.org pci_add_capability() checks whether capabilities overlap, and notifies its caller so that it can properly handle the case. However, in the most cases, the capabilities actually never overlap, and the interface incurred extra error handling code, which is often incorrect or suboptimal. For such cases, pci_add_capability() can simply abort the execution if the capabilities actually overlap since it should be a programming error. This change handles the other cases: hw/vfio/pci depends on the check to decide MSI and MSI-X capabilities overlap with another. As they are quite an exceptional and hw/vfio/pci knows much about PCI capabilities, adding code specific to the cases to hw/vfio/pci still results in less code than having error handling code everywhere in total. Signed-off-by: Akihiko Odaki --- hw/vfio/pci.c | 61 ++++++++++++++++++++++++++++++++++++++++++--------- hw/vfio/pci.h | 3 +++ 2 files changed, 54 insertions(+), 10 deletions(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 939dcc3d4a..c7e3ef95a7 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -1278,23 +1278,42 @@ static void vfio_disable_interrupts(VFIOPCIDevice *vdev) } } -static int vfio_msi_setup(VFIOPCIDevice *vdev, int pos, Error **errp) +static void vfio_msi_early_setup(VFIOPCIDevice *vdev, Error **errp) { uint16_t ctrl; - bool msi_64bit, msi_maskbit; - int ret, entries; - Error *err = NULL; + uint8_t pos; + + pos = pci_find_capability(&vdev->pdev, PCI_CAP_ID_MSI); + if (!pos) { + return; + } if (pread(vdev->vbasedev.fd, &ctrl, sizeof(ctrl), vdev->config_offset + pos + PCI_CAP_FLAGS) != sizeof(ctrl)) { error_setg_errno(errp, errno, "failed reading MSI PCI_CAP_FLAGS"); - return -errno; + return; } - ctrl = le16_to_cpu(ctrl); + vdev->msi_pos = pos; + vdev->msi_ctrl = le16_to_cpu(ctrl); - msi_64bit = !!(ctrl & PCI_MSI_FLAGS_64BIT); - msi_maskbit = !!(ctrl & PCI_MSI_FLAGS_MASKBIT); - entries = 1 << ((ctrl & PCI_MSI_FLAGS_QMASK) >> 1); + vdev->msi_cap_size = 0xa; + if ((vdev->msi_ctrl & PCI_MSI_FLAGS_MASKBIT)) { + vdev->msi_cap_size += 0xa; + } + if ((vdev->msi_ctrl & PCI_MSI_FLAGS_64BIT)) { + vdev->msi_cap_size += 0x4; + } +} + +static int vfio_msi_setup(VFIOPCIDevice *vdev, int pos, Error **errp) +{ + bool msi_64bit, msi_maskbit; + int ret, entries; + Error *err = NULL; + + msi_64bit = !!(vdev->msi_ctrl & PCI_MSI_FLAGS_64BIT); + msi_maskbit = !!(vdev->msi_ctrl & PCI_MSI_FLAGS_MASKBIT); + entries = 1 << ((vdev->msi_ctrl & PCI_MSI_FLAGS_QMASK) >> 1); trace_vfio_msi_setup(vdev->vbasedev.name, pos); @@ -1306,7 +1325,6 @@ static int vfio_msi_setup(VFIOPCIDevice *vdev, int pos, Error **errp) error_propagate_prepend(errp, err, "msi_init failed: "); return ret; } - vdev->msi_cap_size = 0xa + (msi_maskbit ? 0xa : 0) + (msi_64bit ? 0x4 : 0); return 0; } @@ -1524,6 +1542,7 @@ static void vfio_msix_early_setup(VFIOPCIDevice *vdev, Error **errp) pba = le32_to_cpu(pba); msix = g_malloc0(sizeof(*msix)); + msix->pos = pos; msix->table_bar = table & PCI_MSIX_FLAGS_BIRMASK; msix->table_offset = table & ~PCI_MSIX_FLAGS_BIRMASK; msix->pba_bar = pba & PCI_MSIX_FLAGS_BIRMASK; @@ -2025,6 +2044,22 @@ static int vfio_add_std_cap(VFIOPCIDevice *vdev, uint8_t pos, Error **errp) } } + if (cap_id != PCI_CAP_ID_MSI && + range_covers_byte(vdev->msi_pos, vdev->msi_cap_size, pos)) { + error_setg(errp, + "A capability overlaps with MSI (%" PRIu8 " in [%" PRIu8 ", %" PRIu8 "))", + pos, vdev->msi_pos, vdev->msi_pos + vdev->msi_cap_size); + return -EINVAL; + } + + if (cap_id != PCI_CAP_ID_MSIX && vdev->msix && + range_covers_byte(vdev->msix->pos, MSIX_CAP_LENGTH, pos)) { + error_setg(errp, + "A capability overlaps with MSI-X (%" PRIu8 " in [%" PRIu8 ", %" PRIu8 "))", + pos, vdev->msix->pos, vdev->msix->pos + MSIX_CAP_LENGTH); + return -EINVAL; + } + /* Scale down size, esp in case virt caps were added above */ size = MIN(size, vfio_std_cap_max_size(pdev, pos)); @@ -3037,6 +3072,12 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) vfio_bars_prepare(vdev); + vfio_msi_early_setup(vdev, &err); + if (err) { + error_propagate(errp, err); + goto error; + } + vfio_msix_early_setup(vdev, &err); if (err) { error_propagate(errp, err); diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index 7c236a52f4..9ae0278058 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -107,6 +107,7 @@ enum { /* Cache of MSI-X setup */ typedef struct VFIOMSIXInfo { + uint8_t pos; uint8_t table_bar; uint8_t pba_bar; uint16_t entries; @@ -128,6 +129,8 @@ struct VFIOPCIDevice { unsigned int rom_size; off_t rom_offset; /* Offset of ROM region within device fd */ void *rom; + uint8_t msi_pos; + uint16_t msi_ctrl; int msi_cap_size; VFIOMSIVector *msi_vectors; VFIOMSIXInfo *msix;