From patchwork Fri Mar 29 05:46:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexey Kardashevskiy X-Patchwork-Id: 1069166 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ozlabs.ru Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 44VrMs6l17z9sQs for ; Fri, 29 Mar 2019 16:46:49 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728651AbfC2Fqt (ORCPT ); Fri, 29 Mar 2019 01:46:49 -0400 Received: from ozlabs.ru ([107.173.13.209]:45197 "EHLO ozlabs.ru" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726251AbfC2Fqt (ORCPT ); Fri, 29 Mar 2019 01:46:49 -0400 Received: from fstn1-p1.ozlabs.ibm.com (localhost [IPv6:::1]) by ozlabs.ru (Postfix) with ESMTP id A23CAAE8003F; Fri, 29 Mar 2019 01:46:47 -0400 (EDT) From: Alexey Kardashevskiy To: linuxppc-dev@lists.ozlabs.org Cc: Alexey Kardashevskiy , David Gibson , kvm-ppc@vger.kernel.org, "Aneesh Kumar K.V" , Paul Mackerras Subject: [PATCH kernel 1/2] powerpc/mm_iommu: Prepare for less locking Date: Fri, 29 Mar 2019 16:46:40 +1100 Message-Id: <20190329054641.48597-2-aik@ozlabs.ru> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190329054641.48597-1-aik@ozlabs.ru> References: <20190329054641.48597-1-aik@ozlabs.ru> Sender: kvm-ppc-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org The next patch will reduce amount of time spent under locks. This adds mm_iommu_find() to see if the region is already registered. This removes a rather ugly union from the mm_iommu_table_group_mem_t struct and keeps the hack local in mm_iommu_do_alloc(). This makes pageshift and hpas local and assigns them late as soon this moves to a helper. This should cause no behavioral change. Signed-off-by: Alexey Kardashevskiy --- arch/powerpc/mm/mmu_context_iommu.c | 82 +++++++++++++++-------------- 1 file changed, 43 insertions(+), 39 deletions(-) diff --git a/arch/powerpc/mm/mmu_context_iommu.c b/arch/powerpc/mm/mmu_context_iommu.c index e7a9c4f6bfca..6b351c79713b 100644 --- a/arch/powerpc/mm/mmu_context_iommu.c +++ b/arch/powerpc/mm/mmu_context_iommu.c @@ -35,18 +35,8 @@ struct mm_iommu_table_group_mem_t { atomic64_t mapped; unsigned int pageshift; u64 ua; /* userspace address */ - u64 entries; /* number of entries in hpas/hpages[] */ - /* - * in mm_iommu_get we temporarily use this to store - * struct page address. - * - * We need to convert ua to hpa in real mode. Make it - * simpler by storing physical address. - */ - union { - struct page **hpages; /* vmalloc'ed */ - phys_addr_t *hpas; - }; + u64 entries; /* number of entries in hpas */ + phys_addr_t *hpas; #define MM_IOMMU_TABLE_INVALID_HPA ((uint64_t)-1) u64 dev_hpa; /* Device memory base address */ }; @@ -91,26 +81,36 @@ bool mm_iommu_preregistered(struct mm_struct *mm) } EXPORT_SYMBOL_GPL(mm_iommu_preregistered); +/* Must be called with &mem_list_mutex held */ +static bool mm_iommu_find(struct mm_struct *mm, unsigned long ua, + unsigned long entries) +{ + struct mm_iommu_table_group_mem_t *mem; + + list_for_each_entry_rcu(mem, &mm->context.iommu_group_mem_list, next) { + /* Overlap? */ + if ((mem->ua < (ua + (entries << PAGE_SHIFT))) && + (ua < (mem->ua + (mem->entries << PAGE_SHIFT)))) + return true; + } + return false; +} + static long mm_iommu_do_alloc(struct mm_struct *mm, unsigned long ua, unsigned long entries, unsigned long dev_hpa, struct mm_iommu_table_group_mem_t **pmem) { struct mm_iommu_table_group_mem_t *mem; long i, ret, locked_entries = 0; - unsigned int pageshift; + unsigned int pageshift, mem_pageshift; + struct page **hpages; + phys_addr_t *hpas; mutex_lock(&mem_list_mutex); - list_for_each_entry_rcu(mem, &mm->context.iommu_group_mem_list, - next) { - /* Overlap? */ - if ((mem->ua < (ua + (entries << PAGE_SHIFT))) && - (ua < (mem->ua + - (mem->entries << PAGE_SHIFT)))) { - ret = -EINVAL; - goto unlock_exit; - } - + if (mm_iommu_find(mm, ua, entries)) { + ret = -EINVAL; + goto unlock_exit; } if (dev_hpa == MM_IOMMU_TABLE_INVALID_HPA) { @@ -128,58 +128,60 @@ static long mm_iommu_do_alloc(struct mm_struct *mm, unsigned long ua, } if (dev_hpa != MM_IOMMU_TABLE_INVALID_HPA) { - mem->pageshift = __ffs(dev_hpa | (entries << PAGE_SHIFT)); + mem_pageshift = __ffs(dev_hpa | (entries << PAGE_SHIFT)); + hpas = NULL; mem->dev_hpa = dev_hpa; goto good_exit; } mem->dev_hpa = MM_IOMMU_TABLE_INVALID_HPA; - /* - * For a starting point for a maximum page size calculation - * we use @ua and @entries natural alignment to allow IOMMU pages - * smaller than huge pages but still bigger than PAGE_SIZE. - */ - mem->pageshift = __ffs(ua | (entries << PAGE_SHIFT)); - mem->hpas = vzalloc(array_size(entries, sizeof(mem->hpas[0]))); - if (!mem->hpas) { + hpages = vzalloc(array_size(entries, sizeof(hpages[0]))); + if (!hpages) { kfree(mem); ret = -ENOMEM; goto unlock_exit; } down_read(&mm->mmap_sem); - ret = get_user_pages_longterm(ua, entries, FOLL_WRITE, mem->hpages, NULL); + ret = get_user_pages_longterm(ua, entries, FOLL_WRITE, hpages, NULL); up_read(&mm->mmap_sem); if (ret != entries) { /* free the reference taken */ for (i = 0; i < ret; i++) - put_page(mem->hpages[i]); + put_page(hpages[i]); - vfree(mem->hpas); + vfree(hpages); kfree(mem); ret = -EFAULT; goto unlock_exit; } + /* + * For a starting point for a maximum page size calculation + * we use @ua and @entries natural alignment to allow IOMMU pages + * smaller than huge pages but still bigger than PAGE_SIZE. + */ + mem_pageshift = __ffs(ua | (entries << PAGE_SHIFT)); + hpas = (phys_addr_t *) hpages; pageshift = PAGE_SHIFT; for (i = 0; i < entries; ++i) { - struct page *page = mem->hpages[i]; + struct page *page = hpages[i]; /* * Allow to use larger than 64k IOMMU pages. Only do that * if we are backed by hugetlb. */ - if ((mem->pageshift > PAGE_SHIFT) && PageHuge(page)) { + if ((mem_pageshift > PAGE_SHIFT) && PageHuge(page)) { struct page *head = compound_head(page); pageshift = compound_order(head) + PAGE_SHIFT; } - mem->pageshift = min(mem->pageshift, pageshift); + mem_pageshift = min(mem_pageshift, pageshift); /* * We don't need struct page reference any more, switch * to physical address. */ - mem->hpas[i] = page_to_pfn(page) << PAGE_SHIFT; + hpas[i] = page_to_pfn(page) << PAGE_SHIFT; } good_exit: @@ -188,6 +190,8 @@ static long mm_iommu_do_alloc(struct mm_struct *mm, unsigned long ua, mem->used = 1; mem->ua = ua; mem->entries = entries; + mem->hpas = hpas; + mem->pageshift = mem_pageshift; *pmem = mem; list_add_rcu(&mem->next, &mm->context.iommu_group_mem_list);