From patchwork Wed May 14 09:43:29 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hu Tao X-Patchwork-Id: 348698 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 54424140080 for ; Wed, 14 May 2014 19:50:35 +1000 (EST) Received: from localhost ([::1]:50525 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WkVpU-0005nw-Rw for incoming@patchwork.ozlabs.org; Wed, 14 May 2014 05:50:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33348) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WkVlC-0005wj-Pz for qemu-devel@nongnu.org; Wed, 14 May 2014 05:46:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WkVl7-0006Mu-KC for qemu-devel@nongnu.org; Wed, 14 May 2014 05:46:02 -0400 Received: from [59.151.112.132] (port=27911 helo=heian.cn.fujitsu.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WkVl6-0006Gd-I6 for qemu-devel@nongnu.org; Wed, 14 May 2014 05:45:57 -0400 X-IronPort-AV: E=Sophos;i="4.97,1051,1389715200"; d="scan'208";a="30507083" Received: from unknown (HELO edo.cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 14 May 2014 17:43:13 +0800 Received: from G08CNEXCHPEKD01.g08.fujitsu.local (localhost.localdomain [127.0.0.1]) by edo.cn.fujitsu.com (8.14.3/8.13.1) with ESMTP id s4E9jiao002818; Wed, 14 May 2014 17:45:45 +0800 Received: from G08FNSTD100614.fnst.cn.fujitsu.com (10.167.226.102) by G08CNEXCHPEKD01.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.146.2; Wed, 14 May 2014 17:45:55 +0800 From: Hu Tao To: Date: Wed, 14 May 2014 17:43:29 +0800 Message-ID: X-Mailer: git-send-email 1.8.5.2.229.g4448466 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.167.226.102] X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 59.151.112.132 Cc: Paolo Bonzini , Marcelo Tosatti , Igor Mammedov Subject: [Qemu-devel] [PATCH v3.2 25/31] hostmem: add properties for NUMA memory policy X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Signed-off-by: Hu Tao [Raise errors on setting properties if !CONFIG_NUMA. Add BUILD_BUG_ON checks. - Paolo] Signed-off-by: Paolo Bonzini Signed-off-by: Marcelo Tosatti Signed-off-by: Hu Tao Signed-off-by: Hu Tao --- backends/hostmem.c | 125 ++++++++++++++++++++++++++++++++++++++++++++++- include/sysemu/hostmem.h | 4 ++ qapi-schema.json | 20 ++++++++ 3 files changed, 148 insertions(+), 1 deletion(-) diff --git a/backends/hostmem.c b/backends/hostmem.c index 738bb31..e9f69c8 100644 --- a/backends/hostmem.c +++ b/backends/hostmem.c @@ -10,11 +10,20 @@ * See the COPYING file in the top-level directory. */ #include "sysemu/hostmem.h" -#include "sysemu/sysemu.h" #include "qapi/visitor.h" +#include "qapi-types.h" +#include "qapi-visit.h" #include "qapi/qmp/qerror.h" #include "qom/object_interfaces.h" +#ifdef CONFIG_NUMA +#include +QEMU_BUILD_BUG_ON(HOST_MEM_POLICY_DEFAULT != MPOL_DEFAULT); +QEMU_BUILD_BUG_ON(HOST_MEM_POLICY_PREFERRED != MPOL_PREFERRED); +QEMU_BUILD_BUG_ON(HOST_MEM_POLICY_BIND != MPOL_BIND); +QEMU_BUILD_BUG_ON(HOST_MEM_POLICY_INTERLEAVE != MPOL_INTERLEAVE); +#endif + static void host_memory_backend_get_size(Object *obj, Visitor *v, void *opaque, const char *name, Error **errp) @@ -49,6 +58,84 @@ host_memory_backend_set_size(Object *obj, Visitor *v, void *opaque, backend->size = value; } +static void +get_host_nodes(Object *obj, Visitor *v, void *opaque, const char *name, + Error **errp) +{ + HostMemoryBackend *backend = MEMORY_BACKEND(obj); + uint16List *host_nodes = NULL; + uint16List **node = &host_nodes; + unsigned long value; + + value = find_first_bit(backend->host_nodes, MAX_NODES); + if (value == MAX_NODES) { + return; + } + + *node = g_malloc0(sizeof(**node)); + (*node)->value = value; + node = &(*node)->next; + + do { + value = find_next_bit(backend->host_nodes, MAX_NODES, value + 1); + if (value == MAX_NODES) { + break; + } + + *node = g_malloc0(sizeof(**node)); + (*node)->value = value; + node = &(*node)->next; + } while (true); + + visit_type_uint16List(v, &host_nodes, name, errp); +} + +static void +set_host_nodes(Object *obj, Visitor *v, void *opaque, const char *name, + Error **errp) +{ +#ifdef CONFIG_NUMA + HostMemoryBackend *backend = MEMORY_BACKEND(obj); + uint16List *l = NULL; + + visit_type_uint16List(v, &l, name, errp); + + while (l) { + bitmap_set(backend->host_nodes, l->value, 1); + l = l->next; + } +#else + error_setg(errp, "NUMA node binding are not supported by this QEMU"); +#endif +} + +static void +get_policy(Object *obj, Visitor *v, void *opaque, const char *name, + Error **errp) +{ + HostMemoryBackend *backend = MEMORY_BACKEND(obj); + int policy = backend->policy; + + visit_type_enum(v, &policy, HostMemPolicy_lookup, NULL, name, errp); +} + +static void +set_policy(Object *obj, Visitor *v, void *opaque, const char *name, + Error **errp) +{ + HostMemoryBackend *backend = MEMORY_BACKEND(obj); + int policy; + + visit_type_enum(v, &policy, HostMemPolicy_lookup, NULL, name, errp); + backend->policy = policy; + +#ifndef CONFIG_NUMA + if (policy != HOST_MEM_POLICY_DEFAULT) { + error_setg(errp, "NUMA policies are not supported by this QEMU"); + } +#endif +} + static bool host_memory_backend_get_merge(Object *obj, Error **errp) { HostMemoryBackend *backend = MEMORY_BACKEND(obj); @@ -159,6 +246,12 @@ static void host_memory_backend_initfn(Object *obj) object_property_add(obj, "size", "int", host_memory_backend_get_size, host_memory_backend_set_size, NULL, NULL, NULL); + object_property_add(obj, "host-nodes", "int", + get_host_nodes, + set_host_nodes, NULL, NULL, NULL); + object_property_add(obj, "policy", "str", + get_policy, + set_policy, NULL, NULL, NULL); } static void host_memory_backend_finalize(Object *obj) @@ -200,6 +293,36 @@ host_memory_backend_memory_init(UserCreatable *uc, Error **errp) if (!backend->dump) { qemu_madvise(ptr, sz, QEMU_MADV_DONTDUMP); } +#ifdef CONFIG_NUMA + unsigned long maxnode = find_last_bit(backend->host_nodes, MAX_NODES); + + /* check for invalid host-nodes and policies and give more verbose + * error messages than mbind(). */ + if (maxnode != MAX_NODES && backend->policy == MPOL_DEFAULT) { + error_setg(errp, "host-nodes must be empty for policy default," + " or you should explicitly specify a policy other" + " than default"); + return; + } else if (maxnode == MAX_NODES && backend->policy != MPOL_DEFAULT) { + error_setg(errp, "host-nodes must be set for policy %s", + HostMemPolicy_lookup[backend->policy]); + return; + } + + /* This is a workaround for a long standing bug in Linux' + * mbind implementation, which cuts off the last specified + * node. + */ + if (mbind(ptr, sz, backend->policy, backend->host_nodes, maxnode + 2, 0)) { + error_setg_errno(errp, errno, + "cannot bind memory to host NUMA nodes"); + return; + } +#endif + /* Preallocate memory after the NUMA policy has been instantiated. + * This is necessary to guarantee memory is allocated with + * specified NUMA policy in place. + */ if (backend->prealloc) { os_mem_prealloc(memory_region_get_fd(&backend->mr), ptr, sz); } diff --git a/include/sysemu/hostmem.h b/include/sysemu/hostmem.h index 819b72d..4e96298 100644 --- a/include/sysemu/hostmem.h +++ b/include/sysemu/hostmem.h @@ -12,8 +12,10 @@ #ifndef QEMU_HOSTMEM_H #define QEMU_HOSTMEM_H +#include "sysemu/sysemu.h" /* for MAX_NODES */ #include "qom/object.h" #include "exec/memory.h" +#include "qemu/bitmap.h" #define TYPE_MEMORY_BACKEND "memory" #define MEMORY_BACKEND(obj) \ @@ -52,6 +54,8 @@ struct HostMemoryBackend { uint64_t size; bool merge, dump; bool prealloc, force_prealloc; + DECLARE_BITMAP(host_nodes, MAX_NODES); + HostMemPolicy policy; MemoryRegion mr; }; diff --git a/qapi-schema.json b/qapi-schema.json index 935fa22..d6aa71e 100644 --- a/qapi-schema.json +++ b/qapi-schema.json @@ -4737,3 +4737,23 @@ '*cpus': ['uint16'], '*mem': 'size', '*memdev': 'str' }} + +## +# @HostMemPolicy +# +# Host memory policy types +# +# @default: restore default policy, remove any nondefault policy +# +# @preferred: set the preferred host nodes for allocation +# +# @bind: a strict policy that restricts memory allocation to the +# host nodes specified +# +# @interleave: memory allocations are interleaved across the set +# of host nodes specified +# +# Since 2.1 +## +{ 'enum': 'HostMemPolicy', + 'data': [ 'default', 'preferred', 'bind', 'interleave' ] }