From patchwork Fri Jun 14 20:57:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: dann frazier X-Patchwork-Id: 1116275 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45QXxs3s5Fz9s9G; Sat, 15 Jun 2019 06:58:08 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1hbtGs-0003cw-VG; Fri, 14 Jun 2019 20:58:02 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.86_2) (envelope-from ) id 1hbtGr-0003cS-LE for kernel-team@lists.ubuntu.com; Fri, 14 Jun 2019 20:58:01 +0000 Received: from mail-io1-f70.google.com ([209.85.166.70]) by youngberry.canonical.com with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.76) (envelope-from ) id 1hbtGr-00052G-6t for kernel-team@lists.ubuntu.com; Fri, 14 Jun 2019 20:58:01 +0000 Received: by mail-io1-f70.google.com with SMTP id x24so4173629ioh.16 for ; Fri, 14 Jun 2019 13:58:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=TNc9HAu3IeHng3n6jEshLyUVuoWTbi/ejCj0BUePs3o=; b=bk1xx9aRIPIIxEtRIYrzJxjzKSv6JskBbbNK1npTyix1Itgp/4oyoJQkMda3GLteqI FHrrYumDVEpt3aZvS1IHGuQGvhuNPQeVWJ70DbdhBtdgt8dp1fk2wrHj/5mEow63LFCd Yn9sWFtwDTqh8pRCaKIEr3NCOqk6LvXz5L2xzWBV15VbA2oQdcSmVFUnvSF3bw4Z71qi EagwItyxZoDiGTtVYou2T44BdQrLJvLwW9GajuoTxWdyL7X/wONKQb29u3p15/+zpyXf abCq39gYOwxehXUL6nuqNwyl/21po0AWprzypPz+dckuERiPEis+VkF9wvUyRQYi8Vv3 GFrA== X-Gm-Message-State: APjAAAVEwjG3qGh0u8f6g31qMR2tHgHSqrZO6xGncoCJyr03766p9AXp ZRrAV3jihjQUEX4mnoCnyK8VTP8g81tSE7VU//oIbcujmYhYWdLOCIeNGZuVGxZoaKHy0GVt12j heBSsj/41TrJFd2O4uJOgURr5DhbUi2nz3B0tNmCCUw== X-Received: by 2002:a6b:2c96:: with SMTP id s144mr46562199ios.57.1560545880075; Fri, 14 Jun 2019 13:58:00 -0700 (PDT) X-Google-Smtp-Source: APXvYqy2zqgL6i8CDabWRvizfpD0BAgU7nIIbgjPhcckehp5uyieKM88hBUvCY4ss9xpCqQSemYAMw== X-Received: by 2002:a6b:2c96:: with SMTP id s144mr46562180ios.57.1560545879784; Fri, 14 Jun 2019 13:57:59 -0700 (PDT) Received: from xps13.canonical.com (c-71-56-235-36.hsd1.co.comcast.net. [71.56.235.36]) by smtp.gmail.com with ESMTPSA id r139sm5291834iod.61.2019.06.14.13.57.59 for (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 14 Jun 2019 13:57:59 -0700 (PDT) From: dann frazier To: kernel-team@lists.ubuntu.com Subject: [PATCH][SRU Disco] iommu/iova: Separate atomic variables to improve performance Date: Fri, 14 Jun 2019 14:57:48 -0600 Message-Id: <20190614205748.19844-1-dann.frazier@canonical.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Jinyu Qi BugLink: https://bugs.launchpad.net/bugs/1832909 In struct iova_domain, there are three atomic variables, the former two are about TLB flush counters which use atomic_add operation, anoter is used to flush timer that use cmpxhg operation. These variables are in the same cache line, so it will cause some performance loss under the condition that many cores call queue_iova function, Let's isolate the two type atomic variables to different cache line to reduce cache line conflict. Cc: Joerg Roedel Signed-off-by: Jinyu Qi Signed-off-by: Joerg Roedel (cherry picked from commit 14bd9a607f9082e7b5690c27e69072f2aeae0de4) Signed-off-by: dann frazier Acked-by: Stefan Bader Acked-by: Connor Kuehl Acked-by: Connor Kuehl --- include/linux/iova.h | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/include/linux/iova.h b/include/linux/iova.h index 0b93bf96693ef..28a5128405f82 100644 --- a/include/linux/iova.h +++ b/include/linux/iova.h @@ -76,6 +76,14 @@ struct iova_domain { unsigned long start_pfn; /* Lower limit for this domain */ unsigned long dma_32bit_pfn; unsigned long max32_alloc_size; /* Size of last failed allocation */ + struct iova_fq __percpu *fq; /* Flush Queue */ + + atomic64_t fq_flush_start_cnt; /* Number of TLB flushes that + have been started */ + + atomic64_t fq_flush_finish_cnt; /* Number of TLB flushes that + have been finished */ + struct iova anchor; /* rbtree lookup anchor */ struct iova_rcache rcaches[IOVA_RANGE_CACHE_MAX_SIZE]; /* IOVA range caches */ @@ -85,14 +93,6 @@ struct iova_domain { iova_entry_dtor entry_dtor; /* IOMMU driver specific destructor for iova entry */ - struct iova_fq __percpu *fq; /* Flush Queue */ - - atomic64_t fq_flush_start_cnt; /* Number of TLB flushes that - have been started */ - - atomic64_t fq_flush_finish_cnt; /* Number of TLB flushes that - have been finished */ - struct timer_list fq_timer; /* Timer to regularily empty the flush-queues */ atomic_t fq_timer_on; /* 1 when timer is active, 0