From patchwork Wed Jul 12 18:01:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Gardner X-Patchwork-Id: 1806917 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=b+/PF5ex; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4R1QYR173Wz20cD for ; Thu, 13 Jul 2023 04:02:02 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1qJe9x-0001fo-DZ; Wed, 12 Jul 2023 18:01:53 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1qJe9u-0001fY-OQ for kernel-team@lists.ubuntu.com; Wed, 12 Jul 2023 18:01:50 +0000 Received: from mail-pl1-f200.google.com (mail-pl1-f200.google.com [209.85.214.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 4259C3F438 for ; Wed, 12 Jul 2023 18:01:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1689184910; bh=ZSiW0uzGIozF4k+aDxrJCR+61Q0fzCfTSebV1cb/O+Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=b+/PF5exQzsmjNBWUYTB4NvDJnFbjAgOQ54R3AGF3dPsNBJS8KH5wZeNG9qK6D2cA aCYPl8i9p87PZIGDOd8vZAO6H2JGDy47p5QehEB2vuiKSZ8pxMBSI8SrWpSPyXLBKp CU7SzGLH3d+ndHaMH7VH+1WHHxwUkNKUWPyHiJ2yE95IOXLmfCSPKDqXJhqEOzEtJY 96AExkElV+YnvxL89EBm9Jb2Tgw9CQhOAg1uLav+hk2RNSr2aRTpluvk8pt+oCjYuG gCeW6EeqsItiiw/d2OrPirHKR90vYFWiARUV+Jsq4HedYOagrTV3ei5SUHfhgjl/Gn PlhV9PdBoFhKw== Received: by mail-pl1-f200.google.com with SMTP id d9443c01a7336-1b9de8fbfc9so30027715ad.1 for ; Wed, 12 Jul 2023 11:01:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689184908; x=1691776908; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZSiW0uzGIozF4k+aDxrJCR+61Q0fzCfTSebV1cb/O+Q=; b=PslZ7OJcPhnnqTIYe4Ipvs/y7FkQt5rjRTeyUUctxhLLm0d1MBBg7Gnoo0lLkxlI1B N+A3UFCuWtvrUFCLAip7UgLjfZMCugqaXc7hDyTh7XoYOQlRnOfHAEpIj170TuHtZRzo YPqLrrPUNl+61Rk5XhQVUQ/wcsWj/+oDtDfRzmwJ4LNAHgd78S123oNRbMMT7R1wvGIX GBd0juLqqofiCJBx6dLgd88eRrxZ7boeg/iYIZGhIvsBia4L0Pvo3ugvChlEUcZbgjpy Sl1l1WfRpf/QcX9VXmmVnF5ilYIZUS/ld4EZ2YFtYooQChw76GsDDAOLoUbHZUTi8wWa rukw== X-Gm-Message-State: ABy/qLad+li0qKqXia5MGPzeJM7zzViAedcaOLSB69pSG9cMxPx/hDBO HZWpDgM2Z8JC2m9LBPsfCj4yKu7DFoSRd3gzCuUCJYku7Z5+AdFSMeWwfLzs4i6fJXA81ACsBdm P455Ec9GtGFWI6Xn4VMfFwKw7WarKXQwyQtyqMbkkV1Xzd6fx8w== X-Received: by 2002:a17:902:c94c:b0:1b2:4852:9a5f with SMTP id i12-20020a170902c94c00b001b248529a5fmr18950553pla.54.1689184908429; Wed, 12 Jul 2023 11:01:48 -0700 (PDT) X-Google-Smtp-Source: APBJJlFU4970ra/ziW3eQ/eizWhA+BjGKFy16X32lRKvcOfJcciFDkj8zFGZhFRtw2wyE99Q/CojPg== X-Received: by 2002:a17:902:c94c:b0:1b2:4852:9a5f with SMTP id i12-20020a170902c94c00b001b248529a5fmr18950527pla.54.1689184908110; Wed, 12 Jul 2023 11:01:48 -0700 (PDT) Received: from smtp.gmail.com (174-045-099-030.res.spectrum.com. [174.45.99.30]) by smtp.gmail.com with ESMTPSA id n19-20020a170902969300b001b0358848b0sm4276551plp.161.2023.07.12.11.01.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Jul 2023 11:01:47 -0700 (PDT) From: Tim Gardner To: kernel-team@lists.ubuntu.com Subject: [PATCH 2/6] swiotlb: fix the deadlock in swiotlb_do_find_slots Date: Wed, 12 Jul 2023 12:01:37 -0600 Message-Id: <20230712180141.713006-3-tim.gardner@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230712180141.713006-1-tim.gardner@canonical.com> References: <20230712180141.713006-1-tim.gardner@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: "GuoRui.Yu" BugLink: https://bugs.launchpad.net/bugs/2026736 In general, if swiotlb is sufficient, the logic of index = wrap_area_index(mem, index + 1) is fine, it will quickly take a slot and release the area->lock; But if swiotlb is insufficient and the device has min_align_mask requirements, such as NVME, we may not be able to satisfy index == wrap and exit the loop properly. In this case, other kernel threads will not be able to acquire the area->lock and release the slot, resulting in a deadlock. The current implementation of wrap_area_index does not involve a modulo operation, so adjusting the wrap to ensure the loop ends is not trivial. Introduce a new variable to record the number of loops and exit the loop after completing the traversal. Backtraces: Other CPUs are waiting this core to exit the swiotlb_do_find_slots loop. [10199.924391] RIP: 0010:swiotlb_do_find_slots+0x1fe/0x3e0 [10199.924403] Call Trace: [10199.924404] [10199.924405] swiotlb_tbl_map_single+0xec/0x1f0 [10199.924407] swiotlb_map+0x5c/0x260 [10199.924409] ? nvme_pci_setup_prps+0x1ed/0x340 [10199.924411] dma_direct_map_page+0x12e/0x1c0 [10199.924413] nvme_map_data+0x304/0x370 [10199.924415] nvme_prep_rq.part.0+0x31/0x120 [10199.924417] nvme_queue_rq+0x77/0x1f0 ... [ 9639.596311] NMI backtrace for cpu 48 [ 9639.596336] Call Trace: [ 9639.596337] [ 9639.596338] _raw_spin_lock_irqsave+0x37/0x40 [ 9639.596341] swiotlb_do_find_slots+0xef/0x3e0 [ 9639.596344] swiotlb_tbl_map_single+0xec/0x1f0 [ 9639.596347] swiotlb_map+0x5c/0x260 [ 9639.596349] dma_direct_map_sg+0x7a/0x280 [ 9639.596352] __dma_map_sg_attrs+0x30/0x70 [ 9639.596355] dma_map_sgtable+0x1d/0x30 [ 9639.596356] nvme_map_data+0xce/0x370 ... [ 9639.595665] NMI backtrace for cpu 50 [ 9639.595682] Call Trace: [ 9639.595682] [ 9639.595683] _raw_spin_lock_irqsave+0x37/0x40 [ 9639.595686] swiotlb_release_slots.isra.0+0x86/0x180 [ 9639.595688] dma_direct_unmap_sg+0xcf/0x1a0 [ 9639.595690] nvme_unmap_data.part.0+0x43/0xc0 Fixes: 1f221a0d0dbf ("swiotlb: respect min_align_mask") Signed-off-by: GuoRui.Yu Signed-off-by: Xiaokang Hu Signed-off-by: Christoph Hellwig (cherry picked from commit 7c3940bf81e5664cdb50c3fedfec8f0a756a34fb) Signed-off-by: Tim Gardner --- kernel/dma/swiotlb.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index cb13374db6a4..9e1f88cebb17 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -585,8 +585,8 @@ static int swiotlb_do_find_slots(struct device *dev, int area_index, unsigned int iotlb_align_mask = dma_get_min_align_mask(dev) & ~(IO_TLB_SIZE - 1); unsigned int nslots = nr_slots(alloc_size), stride; - unsigned int index, wrap, count = 0, i; unsigned int offset = swiotlb_align_offset(dev, orig_addr); + unsigned int index, slots_checked, count = 0, i; unsigned long flags; unsigned int slot_base; unsigned int slot_index; @@ -609,15 +609,16 @@ static int swiotlb_do_find_slots(struct device *dev, int area_index, goto not_found; slot_base = area_index * mem->area_nslabs; - index = wrap = wrap_area_index(mem, ALIGN(area->index, stride)); + index = wrap_area_index(mem, ALIGN(area->index, stride)); - do { + for (slots_checked = 0; slots_checked < mem->area_nslabs; ) { slot_index = slot_base + index; if (orig_addr && (slot_addr(tbl_dma_addr, slot_index) & iotlb_align_mask) != (orig_addr & iotlb_align_mask)) { index = wrap_area_index(mem, index + 1); + slots_checked++; continue; } @@ -633,7 +634,8 @@ static int swiotlb_do_find_slots(struct device *dev, int area_index, goto found; } index = wrap_area_index(mem, index + stride); - } while (index != wrap); + slots_checked += stride; + } not_found: spin_unlock_irqrestore(&area->lock, flags);