From patchwork Tue Dec 22 10:42:51 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 560010 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 05956140BB7 for ; Tue, 22 Dec 2015 23:42:02 +1100 (AEDT) Received: from localhost ([::1]:50044 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aBMGN-0001il-TL for incoming@patchwork.ozlabs.org; Tue, 22 Dec 2015 07:41:59 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41499) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aBMDk-0005F4-0s for qemu-devel@nongnu.org; Tue, 22 Dec 2015 07:39:17 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aBMDh-00006u-R2 for qemu-devel@nongnu.org; Tue, 22 Dec 2015 07:39:15 -0500 Received: from [59.151.112.132] (port=17493 helo=heian.cn.fujitsu.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aBMDg-00005q-Qv for qemu-devel@nongnu.org; Tue, 22 Dec 2015 07:39:13 -0500 X-IronPort-AV: E=Sophos;i="5.20,346,1444665600"; d="scan'208";a="1856277" Received: from bogon (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 22 Dec 2015 18:43:05 +0800 Received: from G08CNEXCHPEKD02.g08.fujitsu.local (unknown [10.167.33.83]) by cn.fujitsu.com (Postfix) with ESMTP id 98D9D4092578; Tue, 22 Dec 2015 18:42:47 +0800 (CST) Received: from G08FNSTD140215.g08.fujitsu.local (10.167.226.56) by G08CNEXCHPEKD02.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.181.6; Tue, 22 Dec 2015 18:42:47 +0800 From: Zhang Chen To: qemu devel , Jason Wang , Stefan Hajnoczi Date: Tue, 22 Dec 2015 18:42:51 +0800 Message-ID: <1450780978-19123-4-git-send-email-zhangchen.fnst@cn.fujitsu.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1450780978-19123-1-git-send-email-zhangchen.fnst@cn.fujitsu.com> References: <1450780978-19123-1-git-send-email-zhangchen.fnst@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.226.56] X-yoursite-MailScanner-ID: 98D9D4092578.AFD84 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: zhangchen.fnst@cn.fujitsu.com X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 59.151.112.132 Cc: Li Zhijian , Gui jianfeng , "eddie.dong" , "Dr. David Alan Gilbert" , Huang peng , Gong lei , jan.kiszka@siemens.com, Zhang Chen , Yang Hongyang , zhanghailiang Subject: [Qemu-devel] [RFC PATCH v2 03/10] Colo-proxy: add colo-proxy framework X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: zhangchen Colo-proxy is a plugin of qemu netfilter like filter-buffer and dump Signed-off-by: zhangchen Signed-off-by: zhanghailiang --- net/Makefile.objs | 1 + net/colo-proxy.c | 240 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ net/colo-proxy.h | 24 ++++++ 3 files changed, 265 insertions(+) create mode 100644 net/colo-proxy.c create mode 100644 net/colo-proxy.h diff --git a/net/Makefile.objs b/net/Makefile.objs index 5fa2f97..95670f2 100644 --- a/net/Makefile.objs +++ b/net/Makefile.objs @@ -15,3 +15,4 @@ common-obj-$(CONFIG_VDE) += vde.o common-obj-$(CONFIG_NETMAP) += netmap.o common-obj-y += filter.o common-obj-y += filter-buffer.o +common-obj-y += colo-proxy.o diff --git a/net/colo-proxy.c b/net/colo-proxy.c new file mode 100644 index 0000000..2e37c45 --- /dev/null +++ b/net/colo-proxy.c @@ -0,0 +1,240 @@ +/* + * COarse-grain LOck-stepping Virtual Machines for Non-stop Service (COLO) + * (a.k.a. Fault Tolerance or Continuous Replication) + * + * Copyright (c) 2015 HUAWEI TECHNOLOGIES CO., LTD. + * Copyright (c) 2015 FUJITSU LIMITED + * Copyright (c) 2015 Intel Corporation + * + * Author: Zhang Chen + * + * This work is licensed under the terms of the GNU GPL, version 2 or + * later. See the COPYING file in the top-level directory. + */ + +#include "net/filter.h" +#include "net/queue.h" +#include "qemu-common.h" +#include "qemu/iov.h" +#include "qapi/qmp/qerror.h" +#include "qapi-visit.h" +#include "qom/object.h" +#include "qemu/sockets.h" +#include "qemu/main-loop.h" +#include "qemu/jhash.h" +#include "qemu/coroutine.h" +#include "net/eth.h" +#include "slirp/slirp.h" +#include "slirp/slirp_config.h" +#include "slirp/ip.h" +#include "net/net.h" +#include "qemu/error-report.h" +#include "net/colo-proxy.h" +#include "trace.h" +#include + +#define FILTER_COLO_PROXY(obj) \ + OBJECT_CHECK(COLOProxyState, (obj), TYPE_FILTER_COLO_PROXY) + +#define TYPE_FILTER_COLO_PROXY "colo-proxy" +#define PRIMARY_MODE "primary" +#define SECONDARY_MODE "secondary" + +/* + + |COLOProxyState++ + | | + +---------------+ +---------------+ +---------------+ + |conn list +--->conn +--------->conn | + +---------------+ +---------------+ +---------------+ + | | | | | | + +---------------+ +---v----+ +---v----+ +---v----+ +---v----+ + |primary | |secondary |primary | |secondary + |packet | |packet + |packet | |packet + + +--------+ +--------+ +--------+ +--------+ + | | | | + +---v----+ +---v----+ +---v----+ +---v----+ + |primary | |secondary |primary | |secondary + |packet | |packet + |packet | |packet + + +--------+ +--------+ +--------+ +--------+ + | | | | + +---v----+ +---v----+ +---v----+ +---v----+ + |primary | |secondary |primary | |secondary + |packet | |packet + |packet | |packet + + +--------+ +--------+ +--------+ +--------+ + + +*/ + +typedef struct COLOProxyState { + NetFilterState parent_obj; + NetQueue *incoming_queue;/* guest normal net queue */ + NetFilterDirection direction; /* packet direction */ + /* colo mode (primary or secondary) */ + int colo_mode; + /* primary colo connect address(192.168.0.100:12345) + * or secondary listening address(:12345) + */ + char *addr; + int sockfd; + + /* connection list: the packet belonged to this NIC + * could be found in this list. + * element type: Connection + */ + GQueue conn_list; + int status; /* proxy is running or not */ + ssize_t hashtable_size; /* proxy current hash size */ + QemuEvent need_compare_ev; /* notify compare thread */ + QemuThread thread; /* compare thread, a thread for each NIC */ + +} COLOProxyState; + +enum { + COLO_PROXY_NONE, /* colo proxy is not started */ + COLO_PROXY_RUNNING, /* colo proxy is running */ + COLO_PROXY_DONE, /* colo proxyis done(failover) */ +}; + +/* save all the connections of a vm instance in this table */ +GHashTable *colo_conn_hash; +static bool colo_do_checkpoint; +static ssize_t hashtable_max_size; + +static ssize_t colo_proxy_receive_iov(NetFilterState *nf, + NetClientState *sender, + unsigned flags, + const struct iovec *iov, + int iovcnt, + NetPacketSent *sent_cb) +{ + /* + * We return size when buffer a packet, the sender will take it as + * a already sent packet, so sent_cb should not be called later. + * + */ + COLOProxyState *s = FILTER_COLO_PROXY(nf); + ssize_t ret = 0; + + if (s->status != COLO_PROXY_RUNNING) { + /* proxy is not started or failovered */ + return 0; + } + + if (s->colo_mode == COLO_MODE_PRIMARY) { + /* colo_proxy_primary_handler */ + } else { + /* colo_proxy_secondary_handler */ + } + return iov_size(iov, iovcnt); +} + +static void colo_proxy_cleanup(NetFilterState *nf) +{ + COLOProxyState *s = FILTER_COLO_PROXY(nf); + close(s->sockfd); + s->sockfd = -1; + qemu_event_destroy(&s->need_compare_ev); +} + +static void colo_proxy_setup(NetFilterState *nf, Error **errp) +{ + COLOProxyState *s = FILTER_COLO_PROXY(nf); + + if (!s->addr) { + error_setg(errp, "filter colo_proxy needs 'addr' property set!"); + return; + } + + if (nf->direction != NET_FILTER_DIRECTION_ALL) { + error_setg(errp, "colo need queue all packet," + "please startup colo-proxy with queue=all\n"); + return; + } + + s->sockfd = -1; + s->hashtable_size = 0; + colo_do_checkpoint = false; + qemu_event_init(&s->need_compare_ev, false); + + s->incoming_queue = qemu_new_net_queue(qemu_netfilter_pass_to_next, nf); + colo_conn_hash = g_hash_table_new_full(connection_key_hash, + connection_key_equal, + g_free, + connection_destroy); + g_queue_init(&s->conn_list); +} + +static void colo_proxy_class_init(ObjectClass *oc, void *data) +{ + NetFilterClass *nfc = NETFILTER_CLASS(oc); + + nfc->setup = colo_proxy_setup; + nfc->cleanup = colo_proxy_cleanup; + nfc->receive_iov = colo_proxy_receive_iov; +} + +static int colo_proxy_get_mode(Object *obj, Error **errp) +{ + COLOProxyState *s = FILTER_COLO_PROXY(obj); + + return s->colo_mode; +} + +static void +colo_proxy_set_mode(Object *obj, int mode, Error **errp) +{ + COLOProxyState *s = FILTER_COLO_PROXY(obj); + + s->colo_mode = mode; +} + +static char *colo_proxy_get_addr(Object *obj, Error **errp) +{ + COLOProxyState *s = FILTER_COLO_PROXY(obj); + + return g_strdup(s->addr); +} + +static void +colo_proxy_set_addr(Object *obj, const char *value, Error **errp) +{ + COLOProxyState *s = FILTER_COLO_PROXY(obj); + g_free(s->addr); + s->addr = g_strdup(value); + if (!s->addr) { + error_setg(errp, "colo_proxy needs 'addr'" + "property set!"); + return; + } +} + +static void colo_proxy_init(Object *obj) +{ + object_property_add_enum(obj, "mode", "COLOMode", COLOMode_lookup, + colo_proxy_get_mode, colo_proxy_set_mode, NULL); + object_property_add_str(obj, "addr", colo_proxy_get_addr, + colo_proxy_set_addr, NULL); +} + +static void colo_proxy_fini(Object *obj) +{ + COLOProxyState *s = FILTER_COLO_PROXY(obj); + g_free(s->addr); +} + +static const TypeInfo colo_proxy_info = { + .name = TYPE_FILTER_COLO_PROXY, + .parent = TYPE_NETFILTER, + .class_init = colo_proxy_class_init, + .instance_init = colo_proxy_init, + .instance_finalize = colo_proxy_fini, + .instance_size = sizeof(COLOProxyState), +}; + +static void register_types(void) +{ + type_register_static(&colo_proxy_info); +} + +type_init(register_types); diff --git a/net/colo-proxy.h b/net/colo-proxy.h new file mode 100644 index 0000000..affc117 --- /dev/null +++ b/net/colo-proxy.h @@ -0,0 +1,24 @@ +/* + * COarse-grain LOck-stepping Virtual Machines for Non-stop Service (COLO) + * (a.k.a. Fault Tolerance or Continuous Replication) + * + * Copyright (c) 2015 HUAWEI TECHNOLOGIES CO., LTD. + * Copyright (c) 2015 FUJITSU LIMITED + * Copyright (c) 2015 Intel Corporation + * + * Author: Zhang Chen + * + * This work is licensed under the terms of the GNU GPL, version 2 or + * later. See the COPYING file in the top-level directory. + */ + + +#ifndef QEMU_COLO_PROXY_H +#define QEMU_COLO_PROXY_H + +int colo_proxy_start(int mode); +void colo_proxy_stop(int mode); +int colo_proxy_do_checkpoint(int mode); +bool colo_proxy_query_checkpoint(void); + +#endif /* QEMU_COLO_PROXY_H */