From patchwork Wed Dec 5 12:37:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Ivanov X-Patchwork-Id: 1008191 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.infradead.org (client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=linux-um-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=cambridgegreys.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="HM1xtnfs"; dkim-atps=neutral Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 438yvP0LXQz9sCh for ; Wed, 5 Dec 2018 23:38:25 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=TJW9tXpaSpdvsViezq4qXjylBDRn9AZs/Fc2lc+yoew=; b=HM1 xtnfs8dxDjN/Ed46oeMNGorTORev7lIOccTSa6zP+fyZqiWEJfSJHfs6mX+M3gYfVfDp5nXz6FjXF vBFVA9Oizh7Nf1EGIZz/RBGjN38yvxgbwQeqxbqf8ekztGCyeFuWrN0g8IJE1LIIxW8asy2J4t4OE nZQhYGVXCvqpV0N5OcNuh1L0KEaxSewsbaKDGvzAfJQAkEORG78qVtCA+vcQeZKC2jxH8IHTDFVCw be5bQTfi4mOjJkDuhzAC4or2iWtxvfuG0l9ulWI46l3GOhgmdoZDqRHms1y1cSNrU9RA1bG6Uo8a3 txac/SIaLOzqopokD+MJca6PHrMmDzg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1gUWRS-0007rJ-18; Wed, 05 Dec 2018 12:38:14 +0000 Received: from ivanoab6.miniserver.com ([5.153.251.140] helo=www.kot-begemot.co.uk) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1gUWRM-0007oD-0K for linux-um@lists.infradead.org; Wed, 05 Dec 2018 12:38:12 +0000 Received: from [192.168.17.6] (helo=smaug.kot-begemot.co.uk) by www.kot-begemot.co.uk with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1gUWR6-00062E-Ef; Wed, 05 Dec 2018 12:37:52 +0000 Received: from amistad.kot-begemot.co.uk ([192.168.3.89]) by smaug.kot-begemot.co.uk with esmtp (Exim 4.89) (envelope-from ) id 1gUWQy-0004gQ-SB; Wed, 05 Dec 2018 12:37:46 +0000 From: anton.ivanov@cambridgegreys.com To: linux-um@lists.infradead.org Subject: [PATCH 1/3] um: Optimize TLB operations v2 Date: Wed, 5 Dec 2018 12:37:39 +0000 Message-Id: <20181205123741.15456-1-anton.ivanov@cambridgegreys.com> X-Mailer: git-send-email 2.11.0 X-Spam-Score: -1.0 X-Spam-Score: -1.0 X-Clacks-Overhead: GNU Terry Pratchett X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20181205_043809_285948_15560DA8 X-CRM114-Status: GOOD ( 11.27 ) X-Spam-Score: 0.0 (/) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (0.0 points) pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: linux-um@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: richard@nod.at, Anton Ivanov MIME-Version: 1.0 Sender: "linux-um" Errors-To: linux-um-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org From: Anton Ivanov Make the code to merge mmap/munmap/mprotect operations in tlb.c common for userspace and kernel. Kernel tlb operations can now be merged as well. Signed-off-by: Anton Ivanov --- arch/um/kernel/tlb.c | 85 +++++++++++++++++++++++++++++++++++----------------- 1 file changed, 57 insertions(+), 28 deletions(-) diff --git a/arch/um/kernel/tlb.c b/arch/um/kernel/tlb.c index 37508b190106..b7f7a60a0928 100644 --- a/arch/um/kernel/tlb.c +++ b/arch/um/kernel/tlb.c @@ -37,17 +37,19 @@ struct host_vm_change { } mprotect; } u; } ops[1]; + int userspace; int index; - struct mm_id *id; + struct mm_struct *mm; void *data; int force; }; -#define INIT_HVC(mm, force) \ +#define INIT_HVC(mm, force, userspace) \ ((struct host_vm_change) \ { .ops = { { .type = NONE } }, \ - .id = &mm->context.id, \ + .mm = mm, \ .data = NULL, \ + .userspace = userspace, \ .index = 0, \ .force = force }) @@ -68,18 +70,40 @@ static int do_ops(struct host_vm_change *hvc, int end, op = &hvc->ops[i]; switch (op->type) { case MMAP: - ret = map(hvc->id, op->u.mmap.addr, op->u.mmap.len, - op->u.mmap.prot, op->u.mmap.fd, - op->u.mmap.offset, finished, &hvc->data); + if (hvc->userspace) + ret = map(&hvc->mm->context.id, op->u.mmap.addr, + op->u.mmap.len, op->u.mmap.prot, + op->u.mmap.fd, + op->u.mmap.offset, finished, + &hvc->data); + else + map_memory(op->u.mmap.addr, op->u.mmap.offset, + op->u.mmap.len, 1, 1, 1); break; case MUNMAP: - ret = unmap(hvc->id, op->u.munmap.addr, - op->u.munmap.len, finished, &hvc->data); + if (hvc->userspace) + ret = unmap(&hvc->mm->context.id, + op->u.munmap.addr, + op->u.munmap.len, finished, + &hvc->data); + else + ret = os_unmap_memory( + (void *) op->u.munmap.addr, + op->u.munmap.len); + break; case MPROTECT: - ret = protect(hvc->id, op->u.mprotect.addr, - op->u.mprotect.len, op->u.mprotect.prot, - finished, &hvc->data); + if (hvc->userspace) + ret = protect(&hvc->mm->context.id, + op->u.mprotect.addr, + op->u.mprotect.len, + op->u.mprotect.prot, + finished, &hvc->data); + else + ret = os_protect_memory( + (void *) op->u.mprotect.addr, + op->u.mprotect.len, + 1, 1, 1); break; default: printk(KERN_ERR "Unknown op type %d in do_ops\n", @@ -100,9 +124,12 @@ static int add_mmap(unsigned long virt, unsigned long phys, unsigned long len, { __u64 offset; struct host_vm_op *last; - int fd, ret = 0; + int fd = -1, ret = 0; - fd = phys_mapping(phys, &offset); + if (hvc->userspace) + fd = phys_mapping(phys, &offset); + else + offset = phys; if (hvc->index != 0) { last = &hvc->ops[hvc->index - 1]; if ((last->type == MMAP) && @@ -277,9 +304,9 @@ void fix_range_common(struct mm_struct *mm, unsigned long start_addr, pgd_t *pgd; struct host_vm_change hvc; unsigned long addr = start_addr, next; - int ret = 0; + int ret = 0, userspace = 1; - hvc = INIT_HVC(mm, force); + hvc = INIT_HVC(mm, force, userspace); pgd = pgd_offset(mm, addr); do { next = pgd_addr_end(addr, end_addr); @@ -314,9 +341,11 @@ static int flush_tlb_kernel_range_common(unsigned long start, unsigned long end) pmd_t *pmd; pte_t *pte; unsigned long addr, last; - int updated = 0, err; + int updated = 0, err = 0, force = 0, userspace = 0; + struct host_vm_change hvc; mm = &init_mm; + hvc = INIT_HVC(mm, force, userspace); for (addr = start; addr < end;) { pgd = pgd_offset(mm, addr); if (!pgd_present(*pgd)) { @@ -325,8 +354,7 @@ static int flush_tlb_kernel_range_common(unsigned long start, unsigned long end) last = end; if (pgd_newpage(*pgd)) { updated = 1; - err = os_unmap_memory((void *) addr, - last - addr); + err = add_munmap(addr, last - addr, &hvc); if (err < 0) panic("munmap failed, errno = %d\n", -err); @@ -342,8 +370,7 @@ static int flush_tlb_kernel_range_common(unsigned long start, unsigned long end) last = end; if (pud_newpage(*pud)) { updated = 1; - err = os_unmap_memory((void *) addr, - last - addr); + err = add_munmap(addr, last - addr, &hvc); if (err < 0) panic("munmap failed, errno = %d\n", -err); @@ -359,8 +386,7 @@ static int flush_tlb_kernel_range_common(unsigned long start, unsigned long end) last = end; if (pmd_newpage(*pmd)) { updated = 1; - err = os_unmap_memory((void *) addr, - last - addr); + err = add_munmap(addr, last - addr, &hvc); if (err < 0) panic("munmap failed, errno = %d\n", -err); @@ -372,22 +398,25 @@ static int flush_tlb_kernel_range_common(unsigned long start, unsigned long end) pte = pte_offset_kernel(pmd, addr); if (!pte_present(*pte) || pte_newpage(*pte)) { updated = 1; - err = os_unmap_memory((void *) addr, - PAGE_SIZE); + err = add_munmap(addr, PAGE_SIZE, &hvc); if (err < 0) panic("munmap failed, errno = %d\n", -err); if (pte_present(*pte)) - map_memory(addr, - pte_val(*pte) & PAGE_MASK, - PAGE_SIZE, 1, 1, 1); + err = add_mmap(addr, pte_val(*pte) & PAGE_MASK, + PAGE_SIZE, 0, &hvc); } else if (pte_newprot(*pte)) { updated = 1; - os_protect_memory((void *) addr, PAGE_SIZE, 1, 1, 1); + err = add_mprotect(addr, PAGE_SIZE, 0, &hvc); } addr += PAGE_SIZE; } + if (!err) + err = do_ops(&hvc, hvc.index, 1); + + if (err < 0) + panic("flush_tlb_kernel failed, errno = %d\n", err); return updated; } From patchwork Wed Dec 5 12:37:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Ivanov X-Patchwork-Id: 1008190 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.infradead.org (client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=linux-um-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=cambridgegreys.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="cL7Hn/Jj"; dkim-atps=neutral Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 438yvN3gjRz9sDB for ; Wed, 5 Dec 2018 23:38:24 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=rXCIFYDDPwHPBuRArc04itqpB4YXBv8re/VwMIbqjHc=; b=cL7Hn/JjJKB4YY6gieqs8qQS+w tKy1o0LQXa9s/vxXgXR55qjRpihDCR0megaUQ8hutMjSfhzIi7Gnla13FqPociuIWcfbnbJ8qCyWN auNBKmuGTcHCz0e2j+LFzWKZTCDdMzy3aGTbDkfN3OgbKqrxtFoyEqGFyKipyLw9yPT76oPVd7mwl AO72PPGkWyU9cUObLaAf0jrZpNoCBY5XHM9zZHsiN8kNaZCOenlNzpQkgMeFEEl7zLz/7/bsRbh2I ZwH4QdD12JIFvmYEzozPsJwr1Hx6TbZMEce/3QBChsxOK8C0XURTqclLamtFSQU3UZBSLpDD9IPGA AyrrPWcw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1gUWRQ-0007qh-Qa; Wed, 05 Dec 2018 12:38:12 +0000 Received: from ivanoab6.miniserver.com ([5.153.251.140] helo=www.kot-begemot.co.uk) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1gUWRM-0007oF-1O for linux-um@lists.infradead.org; Wed, 05 Dec 2018 12:38:10 +0000 Received: from [192.168.17.6] (helo=smaug.kot-begemot.co.uk) by www.kot-begemot.co.uk with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1gUWR7-00062H-GX; Wed, 05 Dec 2018 12:37:53 +0000 Received: from amistad.kot-begemot.co.uk ([192.168.3.89]) by smaug.kot-begemot.co.uk with esmtp (Exim 4.89) (envelope-from ) id 1gUWR0-0004gQ-RH; Wed, 05 Dec 2018 12:37:48 +0000 From: anton.ivanov@cambridgegreys.com To: linux-um@lists.infradead.org Subject: [PATCH 2/3] um: Skip TLB flushing where not needed Date: Wed, 5 Dec 2018 12:37:40 +0000 Message-Id: <20181205123741.15456-2-anton.ivanov@cambridgegreys.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181205123741.15456-1-anton.ivanov@cambridgegreys.com> References: <20181205123741.15456-1-anton.ivanov@cambridgegreys.com> X-Spam-Score: -1.0 X-Spam-Score: -1.0 X-Clacks-Overhead: GNU Terry Pratchett X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20181205_043809_252181_0EBFDBD9 X-CRM114-Status: UNSURE ( 8.34 ) X-CRM114-Notice: Please train this message. X-Spam-Score: 0.0 (/) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (0.0 points) pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: linux-um@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: richard@nod.at, Anton Ivanov MIME-Version: 1.0 Sender: "linux-um" Errors-To: linux-um-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org From: Anton Ivanov Skip TLB flushing for all cases where it is not needed, not just flush_tlb_mm_range Signed-off-by: Anton Ivanov --- arch/um/kernel/tlb.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/arch/um/kernel/tlb.c b/arch/um/kernel/tlb.c index b7f7a60a0928..9ca902df243a 100644 --- a/arch/um/kernel/tlb.c +++ b/arch/um/kernel/tlb.c @@ -520,6 +520,13 @@ pte_t *addr_pte(struct task_struct *task, unsigned long addr) void flush_tlb_all(void) { + /* + * Don't bother flushing if this address space is about to be + * destroyed. + */ + if (atomic_read(¤t->mm->mm_users) == 0) + return; + flush_tlb_mm(current->mm); } @@ -541,6 +548,13 @@ void __flush_tlb_one(unsigned long addr) static void fix_range(struct mm_struct *mm, unsigned long start_addr, unsigned long end_addr, int force) { + /* + * Don't bother flushing if this address space is about to be + * destroyed. + */ + if (atomic_read(&mm->mm_users) == 0) + return; + fix_range_common(mm, start_addr, end_addr, force); } @@ -556,13 +570,6 @@ EXPORT_SYMBOL(flush_tlb_range); void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start, unsigned long end) { - /* - * Don't bother flushing if this address space is about to be - * destroyed. - */ - if (atomic_read(&mm->mm_users) == 0) - return; - fix_range(mm, start, end, 0); }