From patchwork Mon Mar 28 14:42:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Parri X-Patchwork-Id: 1610221 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Ki4xV6Fb; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by bilbo.ozlabs.org (Postfix) with ESMTP id 4KRwRV4QHwz9sCD for ; Tue, 29 Mar 2022 01:43:18 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244002AbiC1Oo5 (ORCPT ); Mon, 28 Mar 2022 10:44:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39350 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244001AbiC1Oox (ORCPT ); Mon, 28 Mar 2022 10:44:53 -0400 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C09935F4FF; Mon, 28 Mar 2022 07:43:11 -0700 (PDT) Received: by mail-ej1-x629.google.com with SMTP id yy13so29179494ejb.2; Mon, 28 Mar 2022 07:43:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=k6Z2QAh69D2f3QVduoGx7amlvw7Kf+1kJe9caoLYo90=; b=Ki4xV6FbK/Bb9MVx3DDLR9ANFfqVPIk219k3dd+EjsBILtkGxKDryZivr6PqnEpi76 hK/NogkvQF+7BD+pjn3/0QiIaJkI9CSfSXAFQILezgs3rL5gPonOMuEDsMEpt2U6iBy0 Fjw7hksxTES7jI8Bvz0nGgXGKSfHuS+pYJHTFP10/ZatEJ07vsNP6gqiy1+ibRgP4ibk ovAzkumrDJiGfDz5roq9GIdxVcQuWGoUxowlSzckHcP13HY7xbAQ9b6Z5pGuXDtYvo6Z d69XBwKKlCumiJi/QdW2HE30Z53PFbSzsNA8MNg9RGxi+8sUAl/Y6AEyW8/fLGJkIBnb YnaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=k6Z2QAh69D2f3QVduoGx7amlvw7Kf+1kJe9caoLYo90=; b=4mn9+CAxFKgl7/kfbCauiFVU4EKCo2G+hvXvUbVDuwLcz5mtp4obPxjg67+WKvflpC CUS0XHnp3oUj+K/o0L5u/VKwz8n486KrUpjHKVqAuTGCOxJNs9o6usyN4F+pr3yGhUx/ EPkRumjh6jGj1/sNKkMeNZfCDvSwlMM54jrSIF47D2Yo8B1XV+iqsgx5dACodTtV9G3K nmCmoc+s1VES6h3wlADxm4CzZ1p95NRR/y36zRVgKJMjaQnIv96+8DVoglpGlsLoD0Gy zat7kPTwfE/dO1Cd/GYHK8FTKISQbh+if7V12EsUys/S2Jf1bckIGE5HBVRwXiug347A kQpg== X-Gm-Message-State: AOAM531O5N0ixjds0yp3VgnwlKJ1XXG/8npk7Y129gdX3Mii+DI4uzCs UA9gE5EmiU11roSntrKt/O0= X-Google-Smtp-Source: ABdhPJwGwFyvK6uc931lKjO3/W0EPQcKDDaKNpIRB8yfeV8wwR1+sZsLc1txexrOi7g5qj5gVPZxlw== X-Received: by 2002:a17:906:2bd7:b0:6cd:f89d:c828 with SMTP id n23-20020a1709062bd700b006cdf89dc828mr28190599ejg.232.1648478590143; Mon, 28 Mar 2022 07:43:10 -0700 (PDT) Received: from anparri.mshome.net (host-82-59-4-232.retail.telecomitalia.it. [82.59.4.232]) by smtp.gmail.com with ESMTPSA id g2-20020aa7c842000000b0041314b98872sm7008983edt.22.2022.03.28.07.43.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Mar 2022 07:43:09 -0700 (PDT) From: "Andrea Parri (Microsoft)" To: KY Srinivasan , Haiyang Zhang , Stephen Hemminger , Wei Liu , Dexuan Cui , Michael Kelley , Wei Hu , Lorenzo Pieralisi , Rob Herring , Krzysztof Wilczynski , Bjorn Helgaas Cc: linux-hyperv@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, "Andrea Parri (Microsoft)" Subject: [RFC PATCH 1/4] Drivers: hv: vmbus: Remove special code for unsolicited messages Date: Mon, 28 Mar 2022 16:42:41 +0200 Message-Id: <20220328144244.100228-2-parri.andrea@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220328144244.100228-1-parri.andrea@gmail.com> References: <20220328144244.100228-1-parri.andrea@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org vmbus_requestor has included code for handling unsolicited messages since its introduction with commit e8b7db38449ac ("Drivers: hv: vmbus: Add vmbus_requestor data structure for VMBus hardening"); such code was motivated by the early use of vmbus_requestor from storvsc. Since storvsc moved to a tag-based mechanism to generate/retrieve request IDs with commit bf5fd8cae3c8f ("scsi: storvsc: Use blk_mq_unique_tag() to generate requestIDs"), the special handling of unsolicited messages in vmbus_requestor is not useful and can be removed. Signed-off-by: Andrea Parri (Microsoft) Reviewed-by: Michael Kelley --- drivers/hv/channel.c | 13 +------------ 1 file changed, 1 insertion(+), 12 deletions(-) diff --git a/drivers/hv/channel.c b/drivers/hv/channel.c index dc5c35210c16a..a253eee3aeb1a 100644 --- a/drivers/hv/channel.c +++ b/drivers/hv/channel.c @@ -1243,11 +1243,7 @@ u64 vmbus_next_request_id(struct vmbus_channel *channel, u64 rqst_addr) spin_unlock_irqrestore(&rqstor->req_lock, flags); - /* - * Cannot return an ID of 0, which is reserved for an unsolicited - * message from Hyper-V. - */ - return current_id + 1; + return current_id; } EXPORT_SYMBOL_GPL(vmbus_next_request_id); @@ -1268,15 +1264,8 @@ u64 vmbus_request_addr(struct vmbus_channel *channel, u64 trans_id) if (!channel->rqstor_size) return VMBUS_NO_RQSTOR; - /* Hyper-V can send an unsolicited message with ID of 0 */ - if (!trans_id) - return trans_id; - spin_lock_irqsave(&rqstor->req_lock, flags); - /* Data corresponding to trans_id is stored at trans_id - 1 */ - trans_id--; - /* Invalid trans_id */ if (trans_id >= rqstor->size || !test_bit(trans_id, rqstor->req_bitmap)) { spin_unlock_irqrestore(&rqstor->req_lock, flags); From patchwork Mon Mar 28 14:42:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Parri X-Patchwork-Id: 1610222 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=VwlVKLd/; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by bilbo.ozlabs.org (Postfix) with ESMTP id 4KRwRW0Mhyz9sDX for ; Tue, 29 Mar 2022 01:43:19 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244008AbiC1Oo5 (ORCPT ); Mon, 28 Mar 2022 10:44:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39672 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235440AbiC1Oo4 (ORCPT ); Mon, 28 Mar 2022 10:44:56 -0400 Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [IPv6:2a00:1450:4864:20::634]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 461E3261F; Mon, 28 Mar 2022 07:43:14 -0700 (PDT) Received: by mail-ej1-x634.google.com with SMTP id yy13so29179834ejb.2; Mon, 28 Mar 2022 07:43:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Kph0VowbW7jN5384Zdt/0e4BwkBNNpSO83jQo1/O5Oo=; b=VwlVKLd/IhOAMrpouDw2zXjM7Hw7smfMJI7DKfJ4b6lIUN4la92Q7L1DWY5+MvhRlD Ar5ouEPtXHrgXLUXQSSovjG1ylAmPtb8ygAUxmq6Dvh9tXkm6rV2bIwYPT+J4tpB3Lp7 wlfUPcu2xy+T3HNn/uLGX3Gff6VuRY2z/mi6hkmaV1/X41zTfBvBNCftFc8T+iVmIOk5 W6aMroxQksMcB3ZpjPyhewK4cwOOIPkq5/FI27Um77rQynXUGi282LquHlMjAFRlHZ2c D4JvS/A+aLhRqIVme3IqYhCKD4QAwH53/StDREafZ4IPQ8hLvAAAESgWi52CnAoYStVr t86w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Kph0VowbW7jN5384Zdt/0e4BwkBNNpSO83jQo1/O5Oo=; b=2qPn7tWYTb4YqPdGH/xXPs7B5D1MjwdMvlyTdwylzj35uGtkErSTBGd8iYP3rcX4r/ CNcuyC5aLLB9hAv7F8MpElioxg1/MngNsSF6a6c76gi1lW1F/6xRC8kFjPHjNiueAR83 r1g9oA/WwoEK0qjnLh0bQm9iQ053EXn38qSiQUIu3EG0F2JYhJwYevRV+fLwQXSrWe99 Wr5aigghiRheVRNvZLo0E2FNHExLTEP6pQS/J8IChLSi29V5tF+XTdpqidtge3H15OXD LRFe7QKmNd0qBKCzpfhe/049OIDCzU5Nxhlv5i5aLNO6lKBb6xaYW5kPH9Aoz3T6LrUs bJnA== X-Gm-Message-State: AOAM53218QzfSJ2PBiUFwgWPCMIG4CTuckth68MOH/yamy8MouYBRz+Y eX4/ldQrCXqsaSiNCcOnEx8= X-Google-Smtp-Source: ABdhPJwCqESW17eriXR9TX/L14+0d5lkvW0vbdPQ7JEsHTwUQx8whIKnLLab5xez/7d7mSvwrAGK9A== X-Received: by 2002:a17:907:7e85:b0:6e0:dd95:9fc6 with SMTP id qb5-20020a1709077e8500b006e0dd959fc6mr13175214ejc.256.1648478592626; Mon, 28 Mar 2022 07:43:12 -0700 (PDT) Received: from anparri.mshome.net (host-82-59-4-232.retail.telecomitalia.it. [82.59.4.232]) by smtp.gmail.com with ESMTPSA id g2-20020aa7c842000000b0041314b98872sm7008983edt.22.2022.03.28.07.43.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Mar 2022 07:43:12 -0700 (PDT) From: "Andrea Parri (Microsoft)" To: KY Srinivasan , Haiyang Zhang , Stephen Hemminger , Wei Liu , Dexuan Cui , Michael Kelley , Wei Hu , Lorenzo Pieralisi , Rob Herring , Krzysztof Wilczynski , Bjorn Helgaas Cc: linux-hyperv@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, "Andrea Parri (Microsoft)" Subject: [RFC PATCH 2/4] PCI: hv: Use vmbus_requestor to generate transaction IDs for VMbus hardening Date: Mon, 28 Mar 2022 16:42:42 +0200 Message-Id: <20220328144244.100228-3-parri.andrea@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220328144244.100228-1-parri.andrea@gmail.com> References: <20220328144244.100228-1-parri.andrea@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Currently, pointers to guest memory are passed to Hyper-V as transaction IDs in hv_pci. In the face of errors or malicious behavior in Hyper-V, hv_pci should not expose or trust the transaction IDs returned by Hyper-V to be valid guest memory addresses. Instead, use small integers generated by vmbus_requestor as request (transaction) IDs. Suggested-by: Michael Kelley Signed-off-by: Andrea Parri (Microsoft) --- drivers/pci/controller/pci-hyperv.c | 30 +++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index ae0bc2fee4ca8..9f963a46b8298 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -91,6 +91,9 @@ static enum pci_protocol_version_t pci_protocol_versions[] = { /* space for 32bit serial number as string */ #define SLOT_NAME_SIZE 11 +/* Size of requestor for VMbus */ +#define HV_PCI_RQSTOR_SIZE 64 + /* * Message Types */ @@ -1407,7 +1410,7 @@ static void hv_int_desc_free(struct hv_pci_dev *hpdev, int_pkt->wslot.slot = hpdev->desc.win_slot.slot; int_pkt->int_desc = *int_desc; vmbus_sendpacket(hpdev->hbus->hdev->channel, int_pkt, sizeof(*int_pkt), - (unsigned long)&ctxt.pkt, VM_PKT_DATA_INBAND, 0); + 0, VM_PKT_DATA_INBAND, 0); kfree(int_desc); } @@ -2649,7 +2652,7 @@ static void hv_eject_device_work(struct work_struct *work) ejct_pkt->message_type.type = PCI_EJECTION_COMPLETE; ejct_pkt->wslot.slot = hpdev->desc.win_slot.slot; vmbus_sendpacket(hbus->hdev->channel, ejct_pkt, - sizeof(*ejct_pkt), (unsigned long)&ctxt.pkt, + sizeof(*ejct_pkt), 0, VM_PKT_DATA_INBAND, 0); /* For the get_pcichild() in hv_pci_eject_device() */ @@ -2696,8 +2699,9 @@ static void hv_pci_onchannelcallback(void *context) const int packet_size = 0x100; int ret; struct hv_pcibus_device *hbus = context; + struct vmbus_channel *chan = hbus->hdev->channel; u32 bytes_recvd; - u64 req_id; + u64 req_id, req_addr; struct vmpacket_descriptor *desc; unsigned char *buffer; int bufferlen = packet_size; @@ -2743,11 +2747,13 @@ static void hv_pci_onchannelcallback(void *context) switch (desc->type) { case VM_PKT_COMP: - /* - * The host is trusted, and thus it's safe to interpret - * this transaction ID as a pointer. - */ - comp_packet = (struct pci_packet *)req_id; + req_addr = chan->request_addr_callback(chan, req_id); + if (req_addr == VMBUS_RQST_ERROR) { + dev_warn_ratelimited(&hbus->hdev->device, + "Invalid request ID\n"); + break; + } + comp_packet = (struct pci_packet *)req_addr; response = (struct pci_response *)buffer; comp_packet->completion_func(comp_packet->compl_ctxt, response, @@ -3419,6 +3425,10 @@ static int hv_pci_probe(struct hv_device *hdev, goto free_dom; } + hdev->channel->next_request_id_callback = vmbus_next_request_id; + hdev->channel->request_addr_callback = vmbus_request_addr; + hdev->channel->rqstor_size = HV_PCI_RQSTOR_SIZE; + ret = vmbus_open(hdev->channel, pci_ring_size, pci_ring_size, NULL, 0, hv_pci_onchannelcallback, hbus); if (ret) @@ -3749,6 +3759,10 @@ static int hv_pci_resume(struct hv_device *hdev) hbus->state = hv_pcibus_init; + hdev->channel->next_request_id_callback = vmbus_next_request_id; + hdev->channel->request_addr_callback = vmbus_request_addr; + hdev->channel->rqstor_size = HV_PCI_RQSTOR_SIZE; + ret = vmbus_open(hdev->channel, pci_ring_size, pci_ring_size, NULL, 0, hv_pci_onchannelcallback, hbus); if (ret) From patchwork Mon Mar 28 14:42:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Parri X-Patchwork-Id: 1610225 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=CP+jLK3X; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by bilbo.ozlabs.org (Postfix) with ESMTP id 4KRwRs4JsFz9sCD for ; Tue, 29 Mar 2022 01:43:37 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244018AbiC1OpJ (ORCPT ); Mon, 28 Mar 2022 10:45:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39892 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244017AbiC1Oo6 (ORCPT ); Mon, 28 Mar 2022 10:44:58 -0400 Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [IPv6:2a00:1450:4864:20::634]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9BC695DE56; Mon, 28 Mar 2022 07:43:17 -0700 (PDT) Received: by mail-ej1-x634.google.com with SMTP id bq8so15198731ejb.10; Mon, 28 Mar 2022 07:43:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UMTqeBE3FIxp8HTVbYedcaXO9nqf4rXvpBaCDQ8h5rQ=; b=CP+jLK3XyRPt1s6LHqva5mG7v49WF71cPG7ErelvKANJ2MvpGpqbPTU3JJyZ+wMuYg roi1o0Kafl4dUQBQ95pGBSA94E4xl8L4gBvGcOFEF7dkPPtAnhz9e4T6OCA9VFw6sN+l FRhoTVfmpzWq9F79aiCqQBak2WPCwzEthUcAZPFmoyOgUCXhLewkvfgaKl65lmUUbiwN BmVukxngnR4hdrzP6pr7OuszH0G56FvJG0hsMDylPA4shj/HeGhpyBVwr0h5eDdYXDfA egcA2l7VBsL+OXjatrquWYZJYLRviuWILY6wS9k9HgBVh2g/pbcyTsR8TcmX0nZCz5Kh vPrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UMTqeBE3FIxp8HTVbYedcaXO9nqf4rXvpBaCDQ8h5rQ=; b=8GkgeOOCg+cFdsqZgcJp60HDXgIqrmiNJedODvxVXBxkNJfHefhKSWhZfUK/jAC6db HPOxAl5wKn9PMS3mJzGOkUIoumFg9phnQAg9c4d8jioEk28c7GLo0ir5cvbNEeVlJTEw rgPCcYb1mdn4yZMSESpGaHpG6Uz8farP8LToAKtJzeWsV+pfoX0BTS4oHFXN3PBof0NL pDMJaGz+j/1tEyFMiYDAn6FdXSYfr1MmYiJ3f5/92dzcaPgeo1g2ICMgaPv+Ckt7TMYz HY/rZ/9kGwr1XuRevObwo3kFVFVykwDYfsoloNIMvYQ4TskWEh3CW2J8Ivz36KofAiV4 QWqg== X-Gm-Message-State: AOAM533/TYW4YI8g8QDJt/hh/3OvTlCTlmxgzLWZafeGdybuo3iu2D9Y IlypUnNhN/blzfbvxM0FE8I= X-Google-Smtp-Source: ABdhPJw7+9S0v7Xhf9hyaSMHMFA0ERycLJFI8Zejti6gd8HlXDWNirXQ/nL4Vr/YSz2PIjP+SVGURg== X-Received: by 2002:a17:907:6089:b0:6db:a3d7:3fa9 with SMTP id ht9-20020a170907608900b006dba3d73fa9mr29289697ejc.593.1648478594893; Mon, 28 Mar 2022 07:43:14 -0700 (PDT) Received: from anparri.mshome.net (host-82-59-4-232.retail.telecomitalia.it. [82.59.4.232]) by smtp.gmail.com with ESMTPSA id g2-20020aa7c842000000b0041314b98872sm7008983edt.22.2022.03.28.07.43.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Mar 2022 07:43:14 -0700 (PDT) From: "Andrea Parri (Microsoft)" To: KY Srinivasan , Haiyang Zhang , Stephen Hemminger , Wei Liu , Dexuan Cui , Michael Kelley , Wei Hu , Lorenzo Pieralisi , Rob Herring , Krzysztof Wilczynski , Bjorn Helgaas Cc: linux-hyperv@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, "Andrea Parri (Microsoft)" Subject: [RFC PATCH 3/4] Drivers: hv: vmbus: Introduce vmbus_sendpacket_getid() Date: Mon, 28 Mar 2022 16:42:43 +0200 Message-Id: <20220328144244.100228-4-parri.andrea@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220328144244.100228-1-parri.andrea@gmail.com> References: <20220328144244.100228-1-parri.andrea@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org The function can be used to send a VMbus packet and retrieve the corresponding transaction ID. It will be used by hv_pci. No functional change. Suggested-by: Michael Kelley Signed-off-by: Andrea Parri (Microsoft) --- drivers/hv/channel.c | 38 ++++++++++++++++++++++++++++++++------ drivers/hv/hyperv_vmbus.h | 2 +- drivers/hv/ring_buffer.c | 4 +++- include/linux/hyperv.h | 7 +++++++ 4 files changed, 43 insertions(+), 8 deletions(-) diff --git a/drivers/hv/channel.c b/drivers/hv/channel.c index a253eee3aeb1a..3eaa41c7ce15f 100644 --- a/drivers/hv/channel.c +++ b/drivers/hv/channel.c @@ -1022,11 +1022,13 @@ void vmbus_close(struct vmbus_channel *channel) EXPORT_SYMBOL_GPL(vmbus_close); /** - * vmbus_sendpacket() - Send the specified buffer on the given channel + * vmbus_sendpacket_getid() - Send the specified buffer on the given channel * @channel: Pointer to vmbus_channel structure * @buffer: Pointer to the buffer you want to send the data from. * @bufferlen: Maximum size of what the buffer holds. * @requestid: Identifier of the request + * @trans_id: Identifier of the transaction associated to this request, if + * the send is successful; undefined, otherwise. * @type: Type of packet that is being sent e.g. negotiate, time * packet etc. * @flags: 0 or VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED @@ -1036,8 +1038,8 @@ EXPORT_SYMBOL_GPL(vmbus_close); * * Mainly used by Hyper-V drivers. */ -int vmbus_sendpacket(struct vmbus_channel *channel, void *buffer, - u32 bufferlen, u64 requestid, +int vmbus_sendpacket_getid(struct vmbus_channel *channel, void *buffer, + u32 bufferlen, u64 requestid, u64 *trans_id, enum vmbus_packet_type type, u32 flags) { struct vmpacket_descriptor desc; @@ -1063,7 +1065,31 @@ int vmbus_sendpacket(struct vmbus_channel *channel, void *buffer, bufferlist[2].iov_base = &aligned_data; bufferlist[2].iov_len = (packetlen_aligned - packetlen); - return hv_ringbuffer_write(channel, bufferlist, num_vecs, requestid); + return hv_ringbuffer_write(channel, bufferlist, num_vecs, requestid, trans_id); +} +EXPORT_SYMBOL(vmbus_sendpacket_getid); + +/** + * vmbus_sendpacket() - Send the specified buffer on the given channel + * @channel: Pointer to vmbus_channel structure + * @buffer: Pointer to the buffer you want to send the data from. + * @bufferlen: Maximum size of what the buffer holds. + * @requestid: Identifier of the request + * @type: Type of packet that is being sent e.g. negotiate, time + * packet etc. + * @flags: 0 or VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED + * + * Sends data in @buffer directly to Hyper-V via the vmbus. + * This will send the data unparsed to Hyper-V. + * + * Mainly used by Hyper-V drivers. + */ +int vmbus_sendpacket(struct vmbus_channel *channel, void *buffer, + u32 bufferlen, u64 requestid, + enum vmbus_packet_type type, u32 flags) +{ + return vmbus_sendpacket_getid(channel, buffer, bufferlen, + requestid, NULL, type, flags); } EXPORT_SYMBOL(vmbus_sendpacket); @@ -1122,7 +1148,7 @@ int vmbus_sendpacket_pagebuffer(struct vmbus_channel *channel, bufferlist[2].iov_base = &aligned_data; bufferlist[2].iov_len = (packetlen_aligned - packetlen); - return hv_ringbuffer_write(channel, bufferlist, 3, requestid); + return hv_ringbuffer_write(channel, bufferlist, 3, requestid, NULL); } EXPORT_SYMBOL_GPL(vmbus_sendpacket_pagebuffer); @@ -1160,7 +1186,7 @@ int vmbus_sendpacket_mpb_desc(struct vmbus_channel *channel, bufferlist[2].iov_base = &aligned_data; bufferlist[2].iov_len = (packetlen_aligned - packetlen); - return hv_ringbuffer_write(channel, bufferlist, 3, requestid); + return hv_ringbuffer_write(channel, bufferlist, 3, requestid, NULL); } EXPORT_SYMBOL_GPL(vmbus_sendpacket_mpb_desc); diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h index 3a1f007b678a0..64c0b9cbe183b 100644 --- a/drivers/hv/hyperv_vmbus.h +++ b/drivers/hv/hyperv_vmbus.h @@ -181,7 +181,7 @@ void hv_ringbuffer_cleanup(struct hv_ring_buffer_info *ring_info); int hv_ringbuffer_write(struct vmbus_channel *channel, const struct kvec *kv_list, u32 kv_count, - u64 requestid); + u64 requestid, u64 *trans_id); int hv_ringbuffer_read(struct vmbus_channel *channel, void *buffer, u32 buflen, u32 *buffer_actual_len, diff --git a/drivers/hv/ring_buffer.c b/drivers/hv/ring_buffer.c index 71efacb909659..c8561c80c460c 100644 --- a/drivers/hv/ring_buffer.c +++ b/drivers/hv/ring_buffer.c @@ -283,7 +283,7 @@ void hv_ringbuffer_cleanup(struct hv_ring_buffer_info *ring_info) /* Write to the ring buffer. */ int hv_ringbuffer_write(struct vmbus_channel *channel, const struct kvec *kv_list, u32 kv_count, - u64 requestid) + u64 requestid, u64 *trans_id) { int i; u32 bytes_avail_towrite; @@ -354,6 +354,8 @@ int hv_ringbuffer_write(struct vmbus_channel *channel, } desc = hv_get_ring_buffer(outring_info) + old_write; desc->trans_id = (rqst_id == VMBUS_NO_RQSTOR) ? requestid : rqst_id; + if (trans_id) + *trans_id = desc->trans_id; /* Set previous packet start */ prev_indices = hv_get_ring_bufferindices(outring_info); diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h index fe2e0179ed51e..a7cb596d893b1 100644 --- a/include/linux/hyperv.h +++ b/include/linux/hyperv.h @@ -1161,6 +1161,13 @@ extern int vmbus_open(struct vmbus_channel *channel, extern void vmbus_close(struct vmbus_channel *channel); +extern int vmbus_sendpacket_getid(struct vmbus_channel *channel, + void *buffer, + u32 bufferLen, + u64 requestid, + u64 *trans_id, + enum vmbus_packet_type type, + u32 flags); extern int vmbus_sendpacket(struct vmbus_channel *channel, void *buffer, u32 bufferLen, From patchwork Mon Mar 28 14:42:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Parri X-Patchwork-Id: 1610224 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=O7nQGlk/; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by bilbo.ozlabs.org (Postfix) with ESMTP id 4KRwRr2LNwz9sCD for ; Tue, 29 Mar 2022 01:43:36 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244063AbiC1OpL (ORCPT ); Mon, 28 Mar 2022 10:45:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40966 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244035AbiC1OpJ (ORCPT ); Mon, 28 Mar 2022 10:45:09 -0400 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2EE295FF02; Mon, 28 Mar 2022 07:43:19 -0700 (PDT) Received: by mail-ed1-x536.google.com with SMTP id h4so9429087edr.3; Mon, 28 Mar 2022 07:43:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pgQsnHWSKcUjSr0c0J8SsSRoM3VXhq2y61ybhNS+8g0=; b=O7nQGlk/gZojEGd5R9rpAToHuyXorMSSly1Ywl67lXXgV1Rq5gudAJf/TIafmwVEMf xBtvQDt4dmEZu7Oj5/BofdfDWkOTUr+GlS/A5ZrIEcx8cQeAW/BIcGrshxQ5t5HrIC6Q etkBf8srwhJyQV1x/HkKWHapueODt4cg9QNEhvg396VE3AsDyyxh7PdFlFFZVaFCtaij OK5AXy+0ULZPtxtt1n6f1IygWdlflDGvWssi/9vhtVnl8ApWLHsgt4x1zWXWadT53S1K 7o7WEmUU4PbzhNve/1D88qZC09a6sv5UTn99kD5qXwhHQ2LJ8Znw/eorGyRm1fdanvxv zuvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pgQsnHWSKcUjSr0c0J8SsSRoM3VXhq2y61ybhNS+8g0=; b=z7pOaGtry8JQRSXA68uPJJIwrsN+d6/D6LAvT4DddK+Cpq/XWdwL99PP//ok1fnxHr QHx3wz/+9crhBxOVg2NY9CP8UztxI3PzsHOyqwBnltywd1HlnpVwy4PtHyKEU5eOomIN EIyWnxHast6nWozcl0dyOYmEzbbWHFnb5RClfXIVyBJdOqC24JVdG6izTHrRDIfIIXLa K6FSRWTPZQHmFEEnD7WbmP5KkOtiWhTp+8pnejXGxZkSNzoOyjIuWtfos3MRJVGQ7c/h aldj246uwr42EivnMSAlBvbA4kPQHztBrrkrR3XX8hptAh5xfktTs3Z6QpfZK0vCsFGp Ycnw== X-Gm-Message-State: AOAM532rpKQjfDe7+u9dfRtYEyPIdV/T81+9upMSOLYUDjOsRVAibhcw Yj/5mLVKLi00WU7SjtS7rxk7GqHRw5Hftweo X-Google-Smtp-Source: ABdhPJzulsjImQ7v6i6fPCBD9QIlnNfqUl8dIT0nJbMsdaHHW4UE9A3BrJMM4sKQmu/+AcmmhAb1hw== X-Received: by 2002:a05:6402:270b:b0:419:3383:7a9f with SMTP id y11-20020a056402270b00b0041933837a9fmr16617576edd.191.1648478597530; Mon, 28 Mar 2022 07:43:17 -0700 (PDT) Received: from anparri.mshome.net (host-82-59-4-232.retail.telecomitalia.it. [82.59.4.232]) by smtp.gmail.com with ESMTPSA id g2-20020aa7c842000000b0041314b98872sm7008983edt.22.2022.03.28.07.43.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Mar 2022 07:43:17 -0700 (PDT) From: "Andrea Parri (Microsoft)" To: KY Srinivasan , Haiyang Zhang , Stephen Hemminger , Wei Liu , Dexuan Cui , Michael Kelley , Wei Hu , Lorenzo Pieralisi , Rob Herring , Krzysztof Wilczynski , Bjorn Helgaas Cc: linux-hyperv@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, "Andrea Parri (Microsoft)" Subject: [RFC PATCH 4/4] PCI: hv: Fix synchronization between channel callback and hv_compose_msi_msg() Date: Mon, 28 Mar 2022 16:42:44 +0200 Message-Id: <20220328144244.100228-5-parri.andrea@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220328144244.100228-1-parri.andrea@gmail.com> References: <20220328144244.100228-1-parri.andrea@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Dexuan wrote: "[...] when we disable AccelNet, the host PCI VSP driver sends a PCI_EJECT message first, and the channel callback may set hpdev->state to hv_pcichild_ejecting on a different CPU. This can cause hv_compose_msi_msg() to exit from the loop and 'return', and the on-stack variable 'ctxt' is invalid. Now, if the response message from the host arrives, the channel callback will try to access the invalid 'ctxt' variable, and this may cause a crash." Schematically: Hyper-V sends PCI_EJECT msg hv_pci_onchannelcallback() state = hv_pcichild_ejecting hv_compose_msi_msg() alloc and init comp_pkt state == hv_pcichild_ejecting Hyper-V sends VM_PKT_COMP msg hv_pci_onchannelcallback() retrieve address of comp_pkt 'free' comp_pkt and return comp_pkt->completion_func() Dexuan also showed how the crash can be triggered after introducing suitable delays in the driver code, thus validating the 'assumption' that the host can still normally respond to the guest's compose_msi request after the host has started to eject the PCI device. Fix the synchronization by leveraging the requestor lock as follows: - Before 'return'-ing in hv_compose_msi_msg(), remove the ID (while holding the requestor lock) associated to the completion packet. - Retrieve the address *and call ->completion_func() within a same (requestor) critical section in hv_pci_onchannelcallback(). Fixes: de0aa7b2f97d3 ("PCI: hv: Fix 2 hang issues in hv_compose_msi_msg()") Reported-by: Wei Hu Reported-by: Dexuan Cui Suggested-by: Michael Kelley Signed-off-by: Andrea Parri (Microsoft) --- drivers/pci/controller/pci-hyperv.c | 83 ++++++++++++++++++++++++++--- 1 file changed, 77 insertions(+), 6 deletions(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index 9f963a46b8298..8876b318173f0 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -1662,6 +1662,55 @@ static u32 hv_compose_msi_req_v3( return sizeof(*int_pkt); } +/* As in vmbus_request_addr() but without the requestor lock */ +static u64 __hv_pci_request_addr(struct vmbus_channel *channel, u64 trans_id) +{ + struct vmbus_requestor *rqstor = &channel->requestor; + u64 req_addr; + + if (trans_id >= rqstor->size || + !test_bit(trans_id, rqstor->req_bitmap)) + return VMBUS_RQST_ERROR; + + req_addr = rqstor->req_arr[trans_id]; + rqstor->req_arr[trans_id] = rqstor->next_request_id; + rqstor->next_request_id = trans_id; + + bitmap_clear(rqstor->req_bitmap, trans_id, 1); + + return req_addr; +} + +/* + * Clear/remove @trans_id from @channel's requestor, provided the memory + * address stored at @trans_id equals @rqst_addr. + */ +static void hv_pci_request_addr_match(struct vmbus_channel *channel, + u64 trans_id, u64 rqst_addr) +{ + struct vmbus_requestor *rqstor = &channel->requestor; + unsigned long flags; + u64 req_addr; + + spin_lock_irqsave(&rqstor->req_lock, flags); + + if (trans_id >= rqstor->size || + !test_bit(trans_id, rqstor->req_bitmap)) { + spin_unlock_irqrestore(&rqstor->req_lock, flags); + return; + } + + req_addr = rqstor->req_arr[trans_id]; + if (req_addr == rqst_addr) { + rqstor->req_arr[trans_id] = rqstor->next_request_id; + rqstor->next_request_id = trans_id; + + bitmap_clear(rqstor->req_bitmap, trans_id, 1); + } + + spin_unlock_irqrestore(&rqstor->req_lock, flags); +} + /** * hv_compose_msi_msg() - Supplies a valid MSI address/data * @data: Everything about this MSI @@ -1691,7 +1740,7 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) struct pci_create_interrupt3 v3; } int_pkts; } __packed ctxt; - + u64 trans_id; u32 size; int ret; @@ -1753,10 +1802,10 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) goto free_int_desc; } - ret = vmbus_sendpacket(hpdev->hbus->hdev->channel, &ctxt.int_pkts, - size, (unsigned long)&ctxt.pci_pkt, - VM_PKT_DATA_INBAND, - VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED); + ret = vmbus_sendpacket_getid(hpdev->hbus->hdev->channel, &ctxt.int_pkts, + size, (unsigned long)&ctxt.pci_pkt, + &trans_id, VM_PKT_DATA_INBAND, + VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED); if (ret) { dev_err(&hbus->hdev->device, "Sending request for interrupt failed: 0x%x", @@ -1835,6 +1884,16 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) enable_tasklet: tasklet_enable(&channel->callback_event); + /* + * The completion packet on the stack becomes invalid after 'return'; + * remove the ID from the VMbus requestor if the identifier is still + * mapped to/associated with the packet. (The identifier could have + * been 're-used', i.e., already removed and (re-)mapped.) + * + * Cf. hv_pci_onchannelcallback(). + */ + hv_pci_request_addr_match(channel, trans_id, + (unsigned long)&ctxt.pci_pkt); free_int_desc: kfree(int_desc); drop_reference: @@ -2700,6 +2759,8 @@ static void hv_pci_onchannelcallback(void *context) int ret; struct hv_pcibus_device *hbus = context; struct vmbus_channel *chan = hbus->hdev->channel; + struct vmbus_requestor *rqstor = &chan->requestor; + unsigned long flags; u32 bytes_recvd; u64 req_id, req_addr; struct vmpacket_descriptor *desc; @@ -2747,17 +2808,27 @@ static void hv_pci_onchannelcallback(void *context) switch (desc->type) { case VM_PKT_COMP: - req_addr = chan->request_addr_callback(chan, req_id); + spin_lock_irqsave(&rqstor->req_lock, flags); + req_addr = __hv_pci_request_addr(chan, req_id); if (req_addr == VMBUS_RQST_ERROR) { + spin_unlock_irqrestore(&rqstor->req_lock, flags); dev_warn_ratelimited(&hbus->hdev->device, "Invalid request ID\n"); break; } comp_packet = (struct pci_packet *)req_addr; response = (struct pci_response *)buffer; + /* + * Call ->completion_func() within the critical section to make + * sure that the packet pointer is still valid during the call: + * here 'valid' means that there's a task still waiting for the + * completion, and that the packet data is still on the waiting + * task's stack. Cf. hv_compose_msi_msg(). + */ comp_packet->completion_func(comp_packet->compl_ctxt, response, bytes_recvd); + spin_unlock_irqrestore(&rqstor->req_lock, flags); break; case VM_PKT_DATA_INBAND: