From patchwork Fri May 13 21:15:41 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Christoph Lameter (Ampere)" X-Patchwork-Id: 95514 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 57D9DB6F00 for ; Sat, 14 May 2011 07:16:05 +1000 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759598Ab1EMVPs (ORCPT ); Fri, 13 May 2011 17:15:48 -0400 Received: from smtp104.prem.mail.ac4.yahoo.com ([76.13.13.43]:34326 "HELO smtp104.prem.mail.ac4.yahoo.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1755483Ab1EMVPp (ORCPT ); Fri, 13 May 2011 17:15:45 -0400 Received: (qmail 46783 invoked from network); 13 May 2011 21:15:44 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1305321344; bh=EBv3oPT0R6WuSoREryQgT3swVMZjZJfqd8Garc8evZI=; h=Received:X-Yahoo-SMTP:X-YMail-OSG:X-Yahoo-Newman-Property:Received:Date:From:X-X-Sender:To:cc:Subject:In-Reply-To:Message-ID:References:User-Agent:MIME-Version:Content-Type; b=JPObjCBLFIITiLpVPSBYjkAXqEzQ7WD6tQi1gbqXFYw7MYPDC1wlB/TTOERlTCSEOA7n8QVWu4qKJugD2UMuwiuPWanAW4zluQ5YKoWWGZnhjIO9aUharU8oHkJJ1C4cqdFF/llmaPtlqudRttlXsnfEZqy88zhx/o9wKqU0y3c= Received: from router.home (cl@99.30.10.212 with plain) by smtp104.prem.mail.ac4.yahoo.com with SMTP; 13 May 2011 14:15:44 -0700 PDT X-Yahoo-SMTP: _Dag8S.swBC1p4FJKLCXbs8NQzyse1SYSgnAbY0- X-YMail-OSG: p3L4NzwVM1kez_2Ifn9t5njmqzXLRaoFa4tZJG.8jKIzIsL nn4hSGa99OKojpLp1qvL3aVukNVHQNQDcWcQm4oDUvbjouyrOy_I5OKnIfIg E2uxUY.mEAUySP7ymXAUjStCxziEnH54XlwrrIeNEkOEQWSAGgVoNOH0KwVW g4BdeIQTIKdYhNbv8C4q9tbwbTunMJTSfNXGRL_6rtr050R46cuUWozGBSuB GLFTc1ieaHSuE7ByN3J5jh17AhKweiYr7jgotoOuxtRBQ0SvXtURSVAt5v94 .KDxPxOwrIRzfAeSWcdYLoa_0z1LbqgIAnihtFPTr6646Jdu9in6MqHhosW2 LGz4G5bdpH.e6ZXvBlatORUda X-Yahoo-Newman-Property: ymail-3 Received: from cl (helo=localhost) by router.home with local-esmtp (Exim 4.71) (envelope-from ) id 1QKzi3-0000FZ-IR; Fri, 13 May 2011 16:15:43 -0500 Date: Fri, 13 May 2011 16:15:41 -0500 (CDT) From: Christoph Lameter X-X-Sender: cl@router.home To: Eric Dumazet cc: Vegard Nossum , Pekka Enberg , casteyde.christian@free.fr, Andrew Morton , netdev@vger.kernel.org, bugzilla-daemon@bugzilla.kernel.org, bugme-daemon@bugzilla.kernel.org Subject: [PATCH] slub: Make CONFIG_PAGE_ALLOC work with new fastpath In-Reply-To: <1305083543.2437.39.camel@edumazet-laptop> Message-ID: References: <1303183217.4152.49.camel@edumazet-laptop> <1303244270.2756.3.camel@edumazet-laptop> <4DC90D7D.9030808@cs.helsinki.fi> <1305022632.2614.18.camel@edumazet-laptop> <4DC91137.4030109@cs.helsinki.fi> <1305047682.2758.1.camel@edumazet-laptop> <1305050754.2758.12.camel@edumazet-laptop> <1305055948.2437.13.camel@edumazet-laptop> <1305057989.2437.18.camel@edumazet-laptop> <1305060353.2437.26.camel@edumazet-laptop> <1305083543.2437.39.camel@edumazet-laptop> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org I ran this with hackbench under KVM with CONFIG_DEBUG_PAGEALLOC and I saw no failures. Subject: slub: Make CONFIG_PAGE_ALLOC work with new fastpath Fastpath can do a speculative access to a page that CONFIG_PAGE_ALLOC may have marked as invalid to retrieve the pointer to the next free object. Use probe_kernel_read in that case in order not to cause a page fault. Signed-off-by: Christoph Lameter Reported-by: Eric Dumazet Acked-by: Eric Dumazet --- mm/slub.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Index: linux-2.6/mm/slub.c =================================================================== --- linux-2.6.orig/mm/slub.c 2011-05-10 14:31:28.000000000 -0500 +++ linux-2.6/mm/slub.c 2011-05-10 14:31:35.000000000 -0500 @@ -261,6 +261,18 @@ static inline void *get_freepointer(stru return *(void **)(object + s->offset); } +static inline void *get_freepointer_safe(struct kmem_cache *s, void *object) +{ + void *p; + +#ifdef CONFIG_DEBUG_PAGEALLOC + probe_kernel_read(&p, (void **)(object + s->offset), sizeof(p)); +#else + p = get_freepointer(s, object); +#endif + return p; +} + static inline void set_freepointer(struct kmem_cache *s, void *object, void *fp) { *(void **)(object + s->offset) = fp; @@ -1933,7 +1945,7 @@ redo: if (unlikely(!irqsafe_cpu_cmpxchg_double( s->cpu_slab->freelist, s->cpu_slab->tid, object, tid, - get_freepointer(s, object), next_tid(tid)))) { + get_freepointer_safe(s, object), next_tid(tid)))) { note_cmpxchg_failure("slab_alloc", s, tid); goto redo;