From patchwork Mon Jan 19 15:44:42 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Leroy X-Patchwork-Id: 430570 X-Patchwork-Delegate: scottwood@freescale.com Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id E11AD140185 for ; Tue, 20 Jan 2015 02:45:24 +1100 (AEDT) Received: from ozlabs.org (ozlabs.org [103.22.144.67]) by lists.ozlabs.org (Postfix) with ESMTP id A3DAC1A10BF for ; Tue, 20 Jan 2015 02:45:24 +1100 (AEDT) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Received: from mailhub1.si.c-s.fr (pegase1.c-s.fr [93.17.236.30]) by lists.ozlabs.org (Postfix) with ESMTP id 0E9801A0D0E for ; Tue, 20 Jan 2015 02:44:46 +1100 (AEDT) Received: from localhost (mailhub1-int [192.168.12.234]) by localhost (Postfix) with ESMTP id CD4A91C8252; Mon, 19 Jan 2015 16:44:43 +0100 (CET) X-Virus-Scanned: amavisd-new at c-s.fr Received: from mailhub1.si.c-s.fr ([192.168.12.234]) by localhost (mailhub1.c-s.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DiU1nC5aXPR9; Mon, 19 Jan 2015 16:44:43 +0100 (CET) Received: from messagerie.si.c-s.fr (messagerie [192.168.25.192]) by pegase1.c-s.fr (Postfix) with ESMTP id 9E7151C8227; Mon, 19 Jan 2015 16:44:43 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 8D358C73C5; Mon, 19 Jan 2015 16:44:43 +0100 (CET) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id cZaJofN2e6-9; Mon, 19 Jan 2015 16:44:43 +0100 (CET) Received: from PO10863.localdomain (unknown [172.25.231.75]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 43DE6C73C4; Mon, 19 Jan 2015 16:44:43 +0100 (CET) Received: by localhost.localdomain (Postfix, from userid 0) id 8E9E11A5E85; Mon, 19 Jan 2015 16:44:42 +0100 (CET) From: Christophe Leroy To: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , scottwood@freescale.com Subject: [PATCH v2] powerpc/8xx: reduce pressure on TLB due to context switches Message-Id: <20150119154442.8E9E11A5E85@localhost.localdomain> Date: Mon, 19 Jan 2015 16:44:42 +0100 (CET) Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" For nohash powerpc, when we run out of contexts, contexts are freed by stealing used contexts in-turn. When a victim has been selected, the associated TLB entries are freed using _tlbil_pid(). Unfortunatly, on the PPC 8xx, _tlbil_pid() does a tlbia, hence flushes ALL TLB entries and not only the one linked to the stolen context. Therefore, as implented today, at each task switch requiring a new context, all entries are flushed. This patch modifies the implementation so that when running out of contexts, all contexts get freed at once, hence dividing the number of calls to tlbia by 16. Signed-off-by: Christophe Leroy --- v1->v2: removed blank line and added comment arch/powerpc/mm/mmu_context_nohash.c | 43 +++++++++++++++++++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/mm/mmu_context_nohash.c b/arch/powerpc/mm/mmu_context_nohash.c index 928ebe7..dcbc231 100644 --- a/arch/powerpc/mm/mmu_context_nohash.c +++ b/arch/powerpc/mm/mmu_context_nohash.c @@ -52,12 +52,15 @@ #include #include +#include "mmu_decl.h" + static unsigned int first_context, last_context; static unsigned int next_context, nr_free_contexts; static unsigned long *context_map; static unsigned long *stale_map[NR_CPUS]; static struct mm_struct **context_mm; static DEFINE_RAW_SPINLOCK(context_lock); +static bool no_selective_tlbil; #define CTX_MAP_SIZE \ (sizeof(unsigned long) * (last_context / BITS_PER_LONG + 1)) @@ -133,6 +136,38 @@ static unsigned int steal_context_smp(unsigned int id) } #endif /* CONFIG_SMP */ +static unsigned int steal_all_contexts(void) +{ + struct mm_struct *mm; + int cpu = smp_processor_id(); + unsigned int id; + + for (id = first_context; id <= last_context; id++) { + /* Pick up the victim mm */ + mm = context_mm[id]; + + pr_hardcont(" | steal %d from 0x%p", id, mm); + + /* Mark this mm as having no context anymore */ + mm->context.id = MMU_NO_CONTEXT; + if (id != first_context) { + context_mm[id] = NULL; + __clear_bit(id, context_map); +#ifdef DEBUG_MAP_CONSISTENCY + mm->context.active = 0; +#endif + } + __clear_bit(id, stale_map[cpu]); + } + + /* Flush the TLB for all contexts (not to be used on SMP) */ + _tlbil_all(); + + nr_free_contexts = last_context - first_context; + + return first_context; +} + /* Note that this will also be called on SMP if all other CPUs are * offlined, which means that it may be called for cpu != 0. For * this to work, we somewhat assume that CPUs that are onlined @@ -241,7 +276,10 @@ void switch_mmu_context(struct mm_struct *prev, struct mm_struct *next) goto stolen; } #endif /* CONFIG_SMP */ - id = steal_context_up(id); + if (no_selective_tlbil) + id = steal_all_contexts(); + else + id = steal_context_up(id); goto stolen; } nr_free_contexts--; @@ -407,12 +445,15 @@ void __init mmu_context_init(void) if (mmu_has_feature(MMU_FTR_TYPE_8xx)) { first_context = 0; last_context = 15; + no_selective_tlbil = true; } else if (mmu_has_feature(MMU_FTR_TYPE_47x)) { first_context = 1; last_context = 65535; + no_selective_tlbil = false; } else { first_context = 1; last_context = 255; + no_selective_tlbil = false; } #ifdef DEBUG_CLAMP_LAST_CONTEXT