From patchwork Thu Jun 8 15:36:08 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 773350 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3wk8qK35s2z9s76 for ; Fri, 9 Jun 2017 01:44:13 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="fxFx/xFH"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3wk8qK21tBzDqSX for ; Fri, 9 Jun 2017 01:44:13 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="fxFx/xFH"; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Received: from mail-pg0-x242.google.com (mail-pg0-x242.google.com [IPv6:2607:f8b0:400e:c05::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3wk8fM1dkkzDqP3 for ; Fri, 9 Jun 2017 01:36:27 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="fxFx/xFH"; dkim-atps=neutral Received: by mail-pg0-x242.google.com with SMTP id v18so4951224pgb.3 for ; Thu, 08 Jun 2017 08:36:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=jS+LmJ+D70v88H+sHE+VSQgZWlRiURwEthgT3CM64rE=; b=fxFx/xFHI5vC3WXAMYRwHsXWbSJt/Rg4Xy57OzcsHuKL9em2EJyApzuWBu7wI/Mg6s nMlWtcDMPlmwyzAUDsfkbU0DLqwapf+ciXlwcYh9TVueQPV1ClEyGA4hcCKmNi/T4ZQd vGg4xGpJHf/XDjfmhMkIGhztyMoB6bKFnT39hi8PTK5IQ7DkhuhRuFbJtNjQItWjS30R XYIi/XJ3zMZoXDoASgLdVzN4bCb/RJBIn+n6mDJhuaC2soUQDXxC7qdBJqE6dL1ZWdpA uyoyufThD/ewbKFiQzdCJv53bGuS4I9Icyt/qH3mTiV7j23BmzUKDVndRuKKodjqYiya OZzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=jS+LmJ+D70v88H+sHE+VSQgZWlRiURwEthgT3CM64rE=; b=hDKZz+TfZGhf8A3qLlUO9fthty6YEG7W8lNxzkSU/WqXZWtwDSaGS5rX/QUQpPiy/y P7sgqnuA0kUISvSADksm5T+MHTzdpQJAV2dO5sSxHcKbtdYt2QXfegHc3sCh/XcfhH4z knDu0q3YJJFkL2GBCxkLBEjffPBGRODLIGqkoRKs2p6G/nQX6Dksqp/jrO7NnOGJfdWJ tdodqqxV2D5q4FB0oN1jTEWvsFRdImzqNWD5jNZzV+u5i4/uxH86kH8EJqPzRI9zNCDW fyDkuRfJu/RE3dO9igfE/Xcg6zZbKEszkbVBrQFg48LzD7wsJvkJk/8FP4LF82Mqenzv DDzw== X-Gm-Message-State: AODbwcBWy3kmrS5JaY9/ghFbLqID5lNF3Qy0kIRZJ2/QV2t66/8n2jf6 FKebfeNICTJGorKV X-Received: by 10.99.114.11 with SMTP id n11mr38549734pgc.4.1496936185239; Thu, 08 Jun 2017 08:36:25 -0700 (PDT) Received: from roar.au.ibm.com (14-202-185-133.tpgi.com.au. [14.202.185.133]) by smtp.gmail.com with ESMTPSA id q194sm10360445pfq.56.2017.06.08.08.36.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 08 Jun 2017 08:36:24 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH 3/4] powerpc/64: context switch an hwsync instruction can be avoided Date: Fri, 9 Jun 2017 01:36:08 +1000 Message-Id: <20170608153609.19217-3-npiggin@gmail.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170608153609.19217-1-npiggin@gmail.com> References: <20170608153609.19217-1-npiggin@gmail.com> X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Zijlstra , Nicholas Piggin Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" The hwsync in the context switch code to prevent MMIO access being reordered from the point of view of a single process if it gets migrated to a different CPU is not required because there is an hwsync performed earlier in the context switch path. Comment this so it's clear enough if anything changes on the scheduler or the powerpc sides. Remove the hwsync from _switch. This improves context switch performance by 2-3% on POWER8. Cc: Peter Zijlstra Signed-off-by: Nicholas Piggin Acked-by: Peter Zijlstra (Intel) --- arch/powerpc/include/asm/barrier.h | 5 +++++ arch/powerpc/kernel/entry_64.S | 23 +++++++++++++++++------ 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/arch/powerpc/include/asm/barrier.h b/arch/powerpc/include/asm/barrier.h index c0deafc212b8..25d42bd3f114 100644 --- a/arch/powerpc/include/asm/barrier.h +++ b/arch/powerpc/include/asm/barrier.h @@ -74,6 +74,11 @@ do { \ ___p1; \ }) +/* + * This must resolve to hwsync on SMP for the context switch path. + * See _switch, and core scheduler context switch memory ordering + * comments. + */ #define smp_mb__before_spinlock() smp_mb() #include diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S index 273a35926534..fb143859cc68 100644 --- a/arch/powerpc/kernel/entry_64.S +++ b/arch/powerpc/kernel/entry_64.S @@ -512,13 +512,24 @@ _GLOBAL(_switch) std r23,_CCR(r1) std r1,KSP(r3) /* Set old stack pointer */ -#ifdef CONFIG_SMP - /* We need a sync somewhere here to make sure that if the - * previous task gets rescheduled on another CPU, it sees all - * stores it has performed on this one. + /* + * On SMP kernels, care must be taken because a task may be + * scheduled off CPUx and on to CPUy. Memory ordering must be + * considered. + * + * Cacheable stores on CPUx will be visible when the task is + * scheduled on CPUy by virtue of the core scheduler barriers + * (see "Notes on Program-Order guarantees on SMP systems." in + * kernel/sched/core.c). + * + * Uncacheable stores in the case of involuntary preemption must + * be taken care of. The smp_mb__before_spin_lock() in __schedule() + * is implemented as hwsync on powerpc, which orders MMIO too. So + * long as there is an hwsync in the context switch path, it will + * be executed on the source CPU after the task has performed + * all MMIO ops on that CPU, and on the destination CPU before the + * task performs any MMIO ops there. */ - sync -#endif /* CONFIG_SMP */ /* * The kernel context switch path must contain a spin_lock,