[V12,02/10] colo-compare: introduce colo compare initialization

This a COLO net ascii figure:

 Primary qemu                                                           Secondary qemu
+--------------------------------------------------------------+       +----------------------------------------------------------------+
| +----------------------------------------------------------+ |       |  +-----------------------------------------------------------+ |
| |                                                          | |       |  |                                                           | |
| |                        guest                             | |       |  |                        guest                              | |
| |                                                          | |       |  |                                                           | |
| +-------^--------------------------+-----------------------+ |       |  +---------------------+--------+----------------------------+ |
|         |                          |                         |       |                        ^        |                              |
|         |                          |                         |       |                        |        |                              |
|         |  +------------------------------------------------------+  |                        |        |                              |
|netfilter|  |                       |                         |    |  |   netfilter            |        |                              |
| +----------+ +----------------------------+                  |    |  |  +-----------------------------------------------------------+ |
| |       |  |                       |      |        out       |    |  |  |                     |        |  filter excute order       | |
| |       |  |          +-----------------------------+        |    |  |  |                     |        | +------------------->      | |
| |       |  |          |            |      |         |        |    |  |  |                     |        |   TCP                      | |
| | +-----+--+-+  +-----v----+ +-----v----+ |pri +----+----+sec|    |  |  | +------------+  +---+----+---v+rewriter++  +------------+ | |
| | |          |  |          | |          | |in  |         |in |    |  |  | |            |  |        |              |  |            | | |
| | |  filter  |  |  filter  | |  filter  +------>  colo   <------+ +-------->  filter   +--> adjust |   adjust     +-->   filter   | | |
| | |  mirror  |  |redirector| |redirector| |    | compare |   |  |    |  | | redirector |  | ack    |   seq        |  | redirector | | |
| | |          |  |          | |          | |    |         |   |  |    |  | |            |  |        |              |  |            | | |
| | +----^-----+  +----+-----+ +----------+ |    +---------+   |  |    |  | +------------+  +--------+--------------+  +---+--------+ | |
| |      |   tx        |   rx           rx  |                  |  |    |  |            tx                        all       |  rx      | |
| |      |             |                    |                  |  |    |  +-----------------------------------------------------------+ |
| |      |             +--------------+     |                  |  |    |                                                   |            |
| |      |   filter excute order      |     |                  |  |    |                                                   |            |
| |      |  +---------------->        |     |                  |  +--------------------------------------------------------+            |
| +-----------------------------------------+                  |       |                                                                |
|        |                            |                        |       |                                                                |
+--------------------------------------------------------------+       +----------------------------------------------------------------+
         |guest receive               | guest send
         |                            |
+--------+----------------------------v------------------------+
|                                                              |                          NOTE: filter direction is rx/tx/all
|                         tap                                  |                          rx:receive packets sent to the netdev
|                                                              |                          tx:receive packets sent by the netdev
+--------------------------------------------------------------+

In COLO-compare, we do packet comparing job.
Packets coming from the primary char indev will be sent to outdev.
Packets coming from the secondary char dev will be dropped after comparing.
colo-comapre need two input chardev and one output chardev:
primary_in=chardev1-id (source: primary send packet)
secondary_in=chardev2-id (source: secondary send packet)
outdev=chardev3-id

usage:

primary:
-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown
-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66
-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait
-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait
-chardev socket,id=compare0-0,host=3.3.3.3,port=9001
-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait
-chardev socket,id=compare_out0,host=3.3.3.3,port=9005
-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0
-object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out
-object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0
-object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0

secondary:
-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown
-device e1000,netdev=hn0,mac=52:a4:00:12:78:66
-chardev socket,id=red0,host=3.3.3.3,port=9003
-chardev socket,id=red1,host=3.3.3.3,port=9004
-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0
-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1

Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
---
 net/Makefile.objs  |   1 +
 net/colo-compare.c | 284 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 qemu-options.hx    |  39 ++++++++
 vl.c               |   3 +-
 4 files changed, 326 insertions(+), 1 deletion(-)
 create mode 100644 net/colo-compare.c

On 2016年08月17日 16:10, Zhang Chen wrote:
> This a COLO net ascii figure:
>
>   Primary qemu                                                           Secondary qemu
> +--------------------------------------------------------------+       +----------------------------------------------------------------+
> | +----------------------------------------------------------+ |       |  +-----------------------------------------------------------+ |
> | |                                                          | |       |  |                                                           | |
> | |                        guest                             | |       |  |                        guest                              | |
> | |                                                          | |       |  |                                                           | |
> | +-------^--------------------------+-----------------------+ |       |  +---------------------+--------+----------------------------+ |
> |         |                          |                         |       |                        ^        |                              |
> |         |                          |                         |       |                        |        |                              |
> |         |  +------------------------------------------------------+  |                        |        |                              |
> |netfilter|  |                       |                         |    |  |   netfilter            |        |                              |
> | +----------+ +----------------------------+                  |    |  |  +-----------------------------------------------------------+ |
> | |       |  |                       |      |        out       |    |  |  |                     |        |  filter excute order       | |
> | |       |  |          +-----------------------------+        |    |  |  |                     |        | +------------------->      | |
> | |       |  |          |            |      |         |        |    |  |  |                     |        |   TCP                      | |
> | | +-----+--+-+  +-----v----+ +-----v----+ |pri +----+----+sec|    |  |  | +------------+  +---+----+---v+rewriter++  +------------+ | |
> | | |          |  |          | |          | |in  |         |in |    |  |  | |            |  |        |              |  |            | | |
> | | |  filter  |  |  filter  | |  filter  +------>  colo   <------+ +-------->  filter   +--> adjust |   adjust     +-->   filter   | | |
> | | |  mirror  |  |redirector| |redirector| |    | compare |   |  |    |  | | redirector |  | ack    |   seq        |  | redirector | | |
> | | |          |  |          | |          | |    |         |   |  |    |  | |            |  |        |              |  |            | | |
> | | +----^-----+  +----+-----+ +----------+ |    +---------+   |  |    |  | +------------+  +--------+--------------+  +---+--------+ | |
> | |      |   tx        |   rx           rx  |                  |  |    |  |            tx                        all       |  rx      | |
> | |      |             |                    |                  |  |    |  +-----------------------------------------------------------+ |
> | |      |             +--------------+     |                  |  |    |                                                   |            |
> | |      |   filter excute order      |     |                  |  |    |                                                   |            |
> | |      |  +---------------->        |     |                  |  +--------------------------------------------------------+            |
> | +-----------------------------------------+                  |       |                                                                |
> |        |                            |                        |       |                                                                |
> +--------------------------------------------------------------+       +----------------------------------------------------------------+
>           |guest receive               | guest send
>           |                            |
> +--------+----------------------------v------------------------+
> |                                                              |                          NOTE: filter direction is rx/tx/all
> |                         tap                                  |                          rx:receive packets sent to the netdev
> |                                                              |                          tx:receive packets sent by the netdev
> +--------------------------------------------------------------+

It's better to add a doc under docs to explain this configuration in 
detail on top of this series.

> In COLO-compare, we do packet comparing job.
> Packets coming from the primary char indev will be sent to outdev.
> Packets coming from the secondary char dev will be dropped after comparing.
> colo-comapre need two input chardev and one output chardev:
> primary_in=chardev1-id (source: primary send packet)
> secondary_in=chardev2-id (source: secondary send packet)
> outdev=chardev3-id
>
> usage:
>
> primary:
> -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown
> -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66
> -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
> -chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait
> -chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait
> -chardev socket,id=compare0-0,host=3.3.3.3,port=9001
> -chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait
> -chardev socket,id=compare_out0,host=3.3.3.3,port=9005
> -object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0
> -object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out
> -object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0
> -object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0
>
> secondary:
> -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown
> -device e1000,netdev=hn0,mac=52:a4:00:12:78:66
> -chardev socket,id=red0,host=3.3.3.3,port=9003
> -chardev socket,id=red1,host=3.3.3.3,port=9004
> -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0
> -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1
>
> Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
> Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
> ---
>   net/Makefile.objs  |   1 +
>   net/colo-compare.c | 284 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>   qemu-options.hx    |  39 ++++++++
>   vl.c               |   3 +-
>   4 files changed, 326 insertions(+), 1 deletion(-)
>   create mode 100644 net/colo-compare.c
>
> diff --git a/net/Makefile.objs b/net/Makefile.objs
> index b7c22fd..ba92f73 100644
> --- a/net/Makefile.objs
> +++ b/net/Makefile.objs
> @@ -16,3 +16,4 @@ common-obj-$(CONFIG_NETMAP) += netmap.o
>   common-obj-y += filter.o
>   common-obj-y += filter-buffer.o
>   common-obj-y += filter-mirror.o
> +common-obj-y += colo-compare.o
> diff --git a/net/colo-compare.c b/net/colo-compare.c
> new file mode 100644
> index 0000000..cdc3e0e
> --- /dev/null
> +++ b/net/colo-compare.c
> @@ -0,0 +1,284 @@
> +/*
> + * COarse-grain LOck-stepping Virtual Machines for Non-stop Service (COLO)
> + * (a.k.a. Fault Tolerance or Continuous Replication)
> + *
> + * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD.
> + * Copyright (c) 2016 FUJITSU LIMITED
> + * Copyright (c) 2016 Intel Corporation
> + *
> + * Author: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or
> + * later.  See the COPYING file in the top-level directory.
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qemu/error-report.h"
> +#include "qemu-common.h"
> +#include "qapi/qmp/qerror.h"
> +#include "qapi/error.h"
> +#include "net/net.h"
> +#include "net/vhost_net.h"

Looks unnecessary.

> +#include "qom/object_interfaces.h"
> +#include "qemu/iov.h"
> +#include "qom/object.h"
> +#include "qemu/typedefs.h"
> +#include "net/queue.h"
> +#include "sysemu/char.h"
> +#include "qemu/sockets.h"
> +#include "qapi-visit.h"
> +
> +#define TYPE_COLO_COMPARE "colo-compare"
> +#define COLO_COMPARE(obj) \
> +    OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE)
> +
> +#define COMPARE_READ_LEN_MAX NET_BUFSIZE
> +
> +typedef struct CompareState {
> +    Object parent;
> +
> +    char *pri_indev;
> +    char *sec_indev;
> +    char *outdev;
> +    CharDriverState *chr_pri_in;
> +    CharDriverState *chr_sec_in;
> +    CharDriverState *chr_out;
> +    QTAILQ_ENTRY(CompareState) next;

This looks not used in this series but in commit "colo-compare and 
filter-rewriter work with colo-frame". We'd better delay the introducing 
to that patch.

> +    SocketReadState pri_rs;
> +    SocketReadState sec_rs;
> +} CompareState;
> +
> +typedef struct CompareClass {
> +    ObjectClass parent_class;
> +} CompareClass;
> +
> +typedef struct CompareChardevProps {
> +    bool is_socket;
> +    bool is_unix;
> +} CompareChardevProps;
> +
> +static char *compare_get_pri_indev(Object *obj, Error **errp)
> +{
> +    CompareState *s = COLO_COMPARE(obj);
> +
> +    return g_strdup(s->pri_indev);
> +}
> +
> +static void compare_set_pri_indev(Object *obj, const char *value, Error **errp)
> +{
> +    CompareState *s = COLO_COMPARE(obj);
> +
> +    g_free(s->pri_indev);
> +    s->pri_indev = g_strdup(value);
> +}
> +
> +static char *compare_get_sec_indev(Object *obj, Error **errp)
> +{
> +    CompareState *s = COLO_COMPARE(obj);
> +
> +    return g_strdup(s->sec_indev);
> +}
> +
> +static void compare_set_sec_indev(Object *obj, const char *value, Error **errp)
> +{
> +    CompareState *s = COLO_COMPARE(obj);
> +
> +    g_free(s->sec_indev);
> +    s->sec_indev = g_strdup(value);
> +}
> +
> +static char *compare_get_outdev(Object *obj, Error **errp)
> +{
> +    CompareState *s = COLO_COMPARE(obj);
> +
> +    return g_strdup(s->outdev);
> +}
> +
> +static void compare_set_outdev(Object *obj, const char *value, Error **errp)
> +{
> +    CompareState *s = COLO_COMPARE(obj);
> +
> +    g_free(s->outdev);
> +    s->outdev = g_strdup(value);
> +}
> +
> +static void compare_pri_rs_finalize(SocketReadState *pri_rs)
> +{
> +    /* if packet_enqueue pri pkt failed we will send unsupported packet */
> +}
> +
> +static void compare_sec_rs_finalize(SocketReadState *sec_rs)
> +{
> +    /* if packet_enqueue sec pkt failed we will notify trace */
> +}
> +
> +static int compare_chardev_opts(void *opaque,
> +                                const char *name, const char *value,
> +                                Error **errp)
> +{
> +    CompareChardevProps *props = opaque;
> +
> +    if (strcmp(name, "backend") == 0 && strcmp(value, "socket") == 0) {
> +        props->is_socket = true;
> +    } else if (strcmp(name, "host") == 0) {

Typo? net_vhost_chardev_opts() did:

     } else if (strcmp(name, "path") == 0) {
         props->is_unix = true;
     }



> +        props->is_unix = true;
> +    } else if (strcmp(name, "port") == 0) {
> +    } else if (strcmp(name, "server") == 0) {
> +    } else if (strcmp(name, "wait") == 0) {
> +    } else {
> +        error_setg(errp,
> +                   "COLO-compare does not support a chardev with option %s=%s",
> +                   name, value);
> +        return -1;
> +    }
> +    return 0;
> +}
> +
> +/*
> + * called from the main thread on the primary
> + * to setup colo-compare.
> + */
> +static void colo_compare_complete(UserCreatable *uc, Error **errp)
> +{
> +    CompareState *s = COLO_COMPARE(uc);
> +    CompareChardevProps props;
> +
> +    if (!s->pri_indev || !s->sec_indev || !s->outdev) {
> +        error_setg(errp, "colo compare needs 'primary_in' ,"
> +                   "'secondary_in','outdev' property set");
> +        return;
> +    } else if (!strcmp(s->pri_indev, s->outdev) ||
> +               !strcmp(s->sec_indev, s->outdev) ||
> +               !strcmp(s->pri_indev, s->sec_indev)) {
> +        error_setg(errp, "'indev' and 'outdev' could not be same "
> +                   "for compare module");
> +        return;
> +    }
> +
> +    s->chr_pri_in = qemu_chr_find(s->pri_indev);
> +    if (s->chr_pri_in == NULL) {
> +        error_setg(errp, "Primary IN Device '%s' not found",
> +                   s->pri_indev);
> +        return;
> +    }
> +
> +    /* inspect chardev opts */
> +    memset(&props, 0, sizeof(props));
> +    if (qemu_opt_foreach(s->chr_pri_in->opts, compare_chardev_opts, &props, errp)) {
> +        return;
> +    }
> +
> +    if (!props.is_socket || !props.is_unix) {
> +        error_setg(errp, "chardev \"%s\" is not a unix socket",
> +                   s->pri_indev);
> +        return;
> +    }
> +
> +    s->chr_sec_in = qemu_chr_find(s->sec_indev);
> +    if (s->chr_sec_in == NULL) {
> +        error_setg(errp, "Secondary IN Device '%s' not found",
> +                   s->sec_indev);
> +        return;
> +    }
> +
> +    memset(&props, 0, sizeof(props));
> +    if (qemu_opt_foreach(s->chr_sec_in->opts, compare_chardev_opts, &props, errp)) {
> +        return;
> +    }
> +
> +    if (!props.is_socket || !props.is_unix) {
> +        error_setg(errp, "chardev \"%s\" is not a unix socket",
> +                   s->sec_indev);

I believe tcp socket is also supported?

> +        return;
> +    }
> +
> +    s->chr_out = qemu_chr_find(s->outdev);
> +    if (s->chr_out == NULL) {
> +        error_setg(errp, "OUT Device '%s' not found", s->outdev);
> +        return;
> +    }
> +
> +    memset(&props, 0, sizeof(props));
> +    if (qemu_opt_foreach(s->chr_out->opts, compare_chardev_opts, &props, errp)) {
> +        return;
> +    }
> +
> +    if (!props.is_socket || !props.is_unix) {
> +        error_setg(errp, "chardev \"%s\" is not a unix socket",
> +                   s->outdev);

Ditto, and there's code duplication, please introduce a helper to do above.

> +        return;
> +    }
> +
> +    qemu_chr_fe_claim_no_fail(s->chr_pri_in);
> +
> +    qemu_chr_fe_claim_no_fail(s->chr_sec_in);
> +
> +    qemu_chr_fe_claim_no_fail(s->chr_out);
> +
> +    net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize);
> +    net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize);
> +
> +    return;
> +}
> +
> +static void colo_compare_class_init(ObjectClass *oc, void *data)
> +{
> +    UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
> +
> +    ucc->complete = colo_compare_complete;
> +}
> +
> +static void colo_compare_init(Object *obj)
> +{
> +    object_property_add_str(obj, "primary_in",
> +                            compare_get_pri_indev, compare_set_pri_indev,
> +                            NULL);
> +    object_property_add_str(obj, "secondary_in",
> +                            compare_get_sec_indev, compare_set_sec_indev,
> +                            NULL);
> +    object_property_add_str(obj, "outdev",
> +                            compare_get_outdev, compare_set_outdev,
> +                            NULL);
> +}
> +
> +static void colo_compare_finalize(Object *obj)
> +{
> +    CompareState *s = COLO_COMPARE(obj);
> +
> +    if (s->chr_pri_in) {
> +        qemu_chr_add_handlers(s->chr_pri_in, NULL, NULL, NULL, NULL);
> +        qemu_chr_fe_release(s->chr_pri_in);
> +    }
> +    if (s->chr_sec_in) {
> +        qemu_chr_add_handlers(s->chr_sec_in, NULL, NULL, NULL, NULL);
> +        qemu_chr_fe_release(s->chr_sec_in);
> +    }
> +    if (s->chr_out) {
> +        qemu_chr_fe_release(s->chr_out);
> +    }
> +
> +    g_free(s->pri_indev);
> +    g_free(s->sec_indev);
> +    g_free(s->outdev);
> +}
> +
> +static const TypeInfo colo_compare_info = {
> +    .name = TYPE_COLO_COMPARE,
> +    .parent = TYPE_OBJECT,
> +    .instance_size = sizeof(CompareState),
> +    .instance_init = colo_compare_init,
> +    .instance_finalize = colo_compare_finalize,
> +    .class_size = sizeof(CompareClass),
> +    .class_init = colo_compare_class_init,
> +    .interfaces = (InterfaceInfo[]) {
> +        { TYPE_USER_CREATABLE },
> +        { }
> +    }
> +};
> +
> +static void register_types(void)
> +{
> +    type_register_static(&colo_compare_info);
> +}
> +
> +type_init(register_types);
> diff --git a/qemu-options.hx b/qemu-options.hx
> index 587de8f..33d5d0b 100644
> --- a/qemu-options.hx
> +++ b/qemu-options.hx
> @@ -3866,6 +3866,45 @@ Dump the network traffic on netdev @var{dev} to the file specified by
>   The file format is libpcap, so it can be analyzed with tools such as tcpdump
>   or Wireshark.
>   
> +@item -object colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid},
> +outdev=@var{chardevid}
> +
> +Colo-compare gets packet from primary_in@var{chardevid} and secondary_in@var{chardevid}, than compare primary packet with
> +secondary packet. If the packet same, we will output primary

s/If the packet same/If the packets are same/.

> +packet to outdev@var{chardevid}, else we will notify colo-frame
> +do checkpoint and send primary packet to outdev@var{chardevid}.
> +
> +we can use it with the help of filter-mirror and filter-redirector.

s/we/We/ and looks like colo compare must be used with the help of 
mirror and redirector?

> +
> +@example
> +
> +primary:
> +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown
> +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66
> +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
> +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait
> +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait
> +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001
> +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait
> +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005
> +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0
> +-object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out
> +-object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0
> +-object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0
> +
> +secondary:
> +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown
> +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66
> +-chardev socket,id=red0,host=3.3.3.3,port=9003
> +-chardev socket,id=red1,host=3.3.3.3,port=9004
> +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0
> +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1
> +
> +@end example
> +
> +If you want to know the detail of above command line, you can read
> +the colo-compare git log.
> +
>   @item -object secret,id=@var{id},data=@var{string},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}]
>   @item -object secret,id=@var{id},file=@var{filename},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}]
>   
> diff --git a/vl.c b/vl.c
> index cbe51ac..c6b9a6f 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -2865,7 +2865,8 @@ static bool object_create_initial(const char *type)
>       if (g_str_equal(type, "filter-buffer") ||
>           g_str_equal(type, "filter-dump") ||
>           g_str_equal(type, "filter-mirror") ||
> -        g_str_equal(type, "filter-redirector")) {
> +        g_str_equal(type, "filter-redirector") ||
> +        g_str_equal(type, "colo-compare")) {
>           return false;
>       }
>

On 2016/8/31 15:53, Jason Wang wrote:
>
>
> On 2016年08月17日 16:10, Zhang Chen wrote:
>> This a COLO net ascii figure:
>>
>>    Primary qemu                                                           Secondary qemu
>> +--------------------------------------------------------------+       +----------------------------------------------------------------+
>> | +----------------------------------------------------------+ |       |  +-----------------------------------------------------------+ |
>> | |                                                          | |       |  |                                                           | |
>> | |                        guest                             | |       |  |                        guest                              | |
>> | |                                                          | |       |  |                                                           | |
>> | +-------^--------------------------+-----------------------+ |       |  +---------------------+--------+----------------------------+ |
>> |         |                          |                         |       |                        ^        |                              |
>> |         |                          |                         |       |                        |        |                              |
>> |         |  +------------------------------------------------------+  |                        |        |                              |
>> |netfilter|  |                       |                         |    |  |   netfilter            |        |                              |
>> | +----------+ +----------------------------+                  |    |  |  +-----------------------------------------------------------+ |
>> | |       |  |                       |      |        out       |    |  |  |                     |        |  filter excute order       | |
>> | |       |  |          +-----------------------------+        |    |  |  |                     |        | +------------------->      | |
>> | |       |  |          |            |      |         |        |    |  |  |                     |        |   TCP                      | |
>> | | +-----+--+-+  +-----v----+ +-----v----+ |pri +----+----+sec|    |  |  | +------------+  +---+----+---v+rewriter++  +------------+ | |
>> | | |          |  |          | |          | |in  |         |in |    |  |  | |            |  |        |              |  |            | | |
>> | | |  filter  |  |  filter  | |  filter  +------>  colo   <------+ +-------->  filter   +--> adjust |   adjust     +-->   filter   | | |
>> | | |  mirror  |  |redirector| |redirector| |    | compare |   |  |    |  | | redirector |  | ack    |   seq        |  | redirector | | |
>> | | |          |  |          | |          | |    |         |   |  |    |  | |            |  |        |              |  |            | | |
>> | | +----^-----+  +----+-----+ +----------+ |    +---------+   |  |    |  | +------------+  +--------+--------------+  +---+--------+ | |
>> | |      |   tx        |   rx           rx  |                  |  |    |  |            tx                        all       |  rx      | |
>> | |      |             |                    |                  |  |    |  +-----------------------------------------------------------+ |
>> | |      |             +--------------+     |                  |  |    |                                                   |            |
>> | |      |   filter excute order      |     |                  |  |    |                                                   |            |
>> | |      |  +---------------->        |     |                  |  +--------------------------------------------------------+            |
>> | +-----------------------------------------+                  |       |                                                                |
>> |        |                            |                        |       |                                                                |
>> +--------------------------------------------------------------+       +----------------------------------------------------------------+
>>            |guest receive               | guest send
>>            |                            |
>> +--------+----------------------------v------------------------+
>> |                                                              |                          NOTE: filter direction is rx/tx/all
>> |                         tap                                  |                          rx:receive packets sent to the netdev
>> |                                                              |                          tx:receive packets sent by the netdev
>> +--------------------------------------------------------------+
>
> It's better to add a doc under docs to explain this configuration in
> detail on top of this series.
>

Agreed! I'm adding a new document to introduce COLO (COLO-FT.txt).
Since COLO proxy is only used in COLO, and IMHO, it is unnecessary to add
a special documentation to describe this, we can add it into the COLO-FT.txt
after it is been merged ...

>> In COLO-compare, we do packet comparing job.
>> Packets coming from the primary char indev will be sent to outdev.
>> Packets coming from the secondary char dev will be dropped after comparing.
>> colo-comapre need two input chardev and one output chardev:
>> primary_in=chardev1-id (source: primary send packet)
>> secondary_in=chardev2-id (source: secondary send packet)
>> outdev=chardev3-id
>>
>> usage:
>>
>> primary:
>> -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown
>> -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66
>> -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
>> -chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait
>> -chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait
>> -chardev socket,id=compare0-0,host=3.3.3.3,port=9001
>> -chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait
>> -chardev socket,id=compare_out0,host=3.3.3.3,port=9005
>> -object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0
>> -object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out
>> -object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0
>> -object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0
>>
>> secondary:
>> -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown
>> -device e1000,netdev=hn0,mac=52:a4:00:12:78:66
>> -chardev socket,id=red0,host=3.3.3.3,port=9003
>> -chardev socket,id=red1,host=3.3.3.3,port=9004
>> -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0
>> -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1
>>
>> Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
>> Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
>> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
>> ---
>>    net/Makefile.objs  |   1 +
>>    net/colo-compare.c | 284 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>>    qemu-options.hx    |  39 ++++++++
>>    vl.c               |   3 +-
>>    4 files changed, 326 insertions(+), 1 deletion(-)
>>    create mode 100644 net/colo-compare.c
>>
>> diff --git a/net/Makefile.objs b/net/Makefile.objs
>> index b7c22fd..ba92f73 100644
>> --- a/net/Makefile.objs
>> +++ b/net/Makefile.objs
>> @@ -16,3 +16,4 @@ common-obj-$(CONFIG_NETMAP) += netmap.o
>>    common-obj-y += filter.o
>>    common-obj-y += filter-buffer.o
>>    common-obj-y += filter-mirror.o
>> +common-obj-y += colo-compare.o
>> diff --git a/net/colo-compare.c b/net/colo-compare.c
>> new file mode 100644
>> index 0000000..cdc3e0e
>> --- /dev/null
>> +++ b/net/colo-compare.c
>> @@ -0,0 +1,284 @@
>> +/*
>> + * COarse-grain LOck-stepping Virtual Machines for Non-stop Service (COLO)
>> + * (a.k.a. Fault Tolerance or Continuous Replication)
>> + *
>> + * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD.
>> + * Copyright (c) 2016 FUJITSU LIMITED
>> + * Copyright (c) 2016 Intel Corporation
>> + *
>> + * Author: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
>> + *
>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>> + * later.  See the COPYING file in the top-level directory.
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include "qemu/error-report.h"
>> +#include "qemu-common.h"
>> +#include "qapi/qmp/qerror.h"
>> +#include "qapi/error.h"
>> +#include "net/net.h"
>> +#include "net/vhost_net.h"
>
> Looks unnecessary.
>
>> +#include "qom/object_interfaces.h"
>> +#include "qemu/iov.h"
>> +#include "qom/object.h"
>> +#include "qemu/typedefs.h"
>> +#include "net/queue.h"
>> +#include "sysemu/char.h"
>> +#include "qemu/sockets.h"
>> +#include "qapi-visit.h"
>> +
>> +#define TYPE_COLO_COMPARE "colo-compare"
>> +#define COLO_COMPARE(obj) \
>> +    OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE)
>> +
>> +#define COMPARE_READ_LEN_MAX NET_BUFSIZE
>> +
>> +typedef struct CompareState {
>> +    Object parent;
>> +
>> +    char *pri_indev;
>> +    char *sec_indev;
>> +    char *outdev;
>> +    CharDriverState *chr_pri_in;
>> +    CharDriverState *chr_sec_in;
>> +    CharDriverState *chr_out;
>> +    QTAILQ_ENTRY(CompareState) next;
>
> This looks not used in this series but in commit "colo-compare and
> filter-rewriter work with colo-frame". We'd better delay the introducing
> to that patch.
>
>> +    SocketReadState pri_rs;
>> +    SocketReadState sec_rs;
>> +} CompareState;
>> +
>> +typedef struct CompareClass {
>> +    ObjectClass parent_class;
>> +} CompareClass;
>> +
>> +typedef struct CompareChardevProps {
>> +    bool is_socket;
>> +    bool is_unix;
>> +} CompareChardevProps;
>> +
>> +static char *compare_get_pri_indev(Object *obj, Error **errp)
>> +{
>> +    CompareState *s = COLO_COMPARE(obj);
>> +
>> +    return g_strdup(s->pri_indev);
>> +}
>> +
>> +static void compare_set_pri_indev(Object *obj, const char *value, Error **errp)
>> +{
>> +    CompareState *s = COLO_COMPARE(obj);
>> +
>> +    g_free(s->pri_indev);
>> +    s->pri_indev = g_strdup(value);
>> +}
>> +
>> +static char *compare_get_sec_indev(Object *obj, Error **errp)
>> +{
>> +    CompareState *s = COLO_COMPARE(obj);
>> +
>> +    return g_strdup(s->sec_indev);
>> +}
>> +
>> +static void compare_set_sec_indev(Object *obj, const char *value, Error **errp)
>> +{
>> +    CompareState *s = COLO_COMPARE(obj);
>> +
>> +    g_free(s->sec_indev);
>> +    s->sec_indev = g_strdup(value);
>> +}
>> +
>> +static char *compare_get_outdev(Object *obj, Error **errp)
>> +{
>> +    CompareState *s = COLO_COMPARE(obj);
>> +
>> +    return g_strdup(s->outdev);
>> +}
>> +
>> +static void compare_set_outdev(Object *obj, const char *value, Error **errp)
>> +{
>> +    CompareState *s = COLO_COMPARE(obj);
>> +
>> +    g_free(s->outdev);
>> +    s->outdev = g_strdup(value);
>> +}
>> +
>> +static void compare_pri_rs_finalize(SocketReadState *pri_rs)
>> +{
>> +    /* if packet_enqueue pri pkt failed we will send unsupported packet */
>> +}
>> +
>> +static void compare_sec_rs_finalize(SocketReadState *sec_rs)
>> +{
>> +    /* if packet_enqueue sec pkt failed we will notify trace */
>> +}
>> +
>> +static int compare_chardev_opts(void *opaque,
>> +                                const char *name, const char *value,
>> +                                Error **errp)
>> +{
>> +    CompareChardevProps *props = opaque;
>> +
>> +    if (strcmp(name, "backend") == 0 && strcmp(value, "socket") == 0) {
>> +        props->is_socket = true;
>> +    } else if (strcmp(name, "host") == 0) {
>
> Typo? net_vhost_chardev_opts() did:
>
>       } else if (strcmp(name, "path") == 0) {
>           props->is_unix = true;
>       }
>
>
>
>> +        props->is_unix = true;
>> +    } else if (strcmp(name, "port") == 0) {
>> +    } else if (strcmp(name, "server") == 0) {
>> +    } else if (strcmp(name, "wait") == 0) {
>> +    } else {
>> +        error_setg(errp,
>> +                   "COLO-compare does not support a chardev with option %s=%s",
>> +                   name, value);
>> +        return -1;
>> +    }
>> +    return 0;
>> +}
>> +
>> +/*
>> + * called from the main thread on the primary
>> + * to setup colo-compare.
>> + */
>> +static void colo_compare_complete(UserCreatable *uc, Error **errp)
>> +{
>> +    CompareState *s = COLO_COMPARE(uc);
>> +    CompareChardevProps props;
>> +
>> +    if (!s->pri_indev || !s->sec_indev || !s->outdev) {
>> +        error_setg(errp, "colo compare needs 'primary_in' ,"
>> +                   "'secondary_in','outdev' property set");
>> +        return;
>> +    } else if (!strcmp(s->pri_indev, s->outdev) ||
>> +               !strcmp(s->sec_indev, s->outdev) ||
>> +               !strcmp(s->pri_indev, s->sec_indev)) {
>> +        error_setg(errp, "'indev' and 'outdev' could not be same "
>> +                   "for compare module");
>> +        return;
>> +    }
>> +
>> +    s->chr_pri_in = qemu_chr_find(s->pri_indev);
>> +    if (s->chr_pri_in == NULL) {
>> +        error_setg(errp, "Primary IN Device '%s' not found",
>> +                   s->pri_indev);
>> +        return;
>> +    }
>> +
>> +    /* inspect chardev opts */
>> +    memset(&props, 0, sizeof(props));
>> +    if (qemu_opt_foreach(s->chr_pri_in->opts, compare_chardev_opts, &props, errp)) {
>> +        return;
>> +    }
>> +
>> +    if (!props.is_socket || !props.is_unix) {
>> +        error_setg(errp, "chardev \"%s\" is not a unix socket",
>> +                   s->pri_indev);
>> +        return;
>> +    }
>> +
>> +    s->chr_sec_in = qemu_chr_find(s->sec_indev);
>> +    if (s->chr_sec_in == NULL) {
>> +        error_setg(errp, "Secondary IN Device '%s' not found",
>> +                   s->sec_indev);
>> +        return;
>> +    }
>> +
>> +    memset(&props, 0, sizeof(props));
>> +    if (qemu_opt_foreach(s->chr_sec_in->opts, compare_chardev_opts, &props, errp)) {
>> +        return;
>> +    }
>> +
>> +    if (!props.is_socket || !props.is_unix) {
>> +        error_setg(errp, "chardev \"%s\" is not a unix socket",
>> +                   s->sec_indev);
>
> I believe tcp socket is also supported?
>
>> +        return;
>> +    }
>> +
>> +    s->chr_out = qemu_chr_find(s->outdev);
>> +    if (s->chr_out == NULL) {
>> +        error_setg(errp, "OUT Device '%s' not found", s->outdev);
>> +        return;
>> +    }
>> +
>> +    memset(&props, 0, sizeof(props));
>> +    if (qemu_opt_foreach(s->chr_out->opts, compare_chardev_opts, &props, errp)) {
>> +        return;
>> +    }
>> +
>> +    if (!props.is_socket || !props.is_unix) {
>> +        error_setg(errp, "chardev \"%s\" is not a unix socket",
>> +                   s->outdev);
>
> Ditto, and there's code duplication, please introduce a helper to do above.
>
>> +        return;
>> +    }
>> +
>> +    qemu_chr_fe_claim_no_fail(s->chr_pri_in);
>> +
>> +    qemu_chr_fe_claim_no_fail(s->chr_sec_in);
>> +
>> +    qemu_chr_fe_claim_no_fail(s->chr_out);
>> +
>> +    net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize);
>> +    net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize);
>> +
>> +    return;
>> +}
>> +
>> +static void colo_compare_class_init(ObjectClass *oc, void *data)
>> +{
>> +    UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
>> +
>> +    ucc->complete = colo_compare_complete;
>> +}
>> +
>> +static void colo_compare_init(Object *obj)
>> +{
>> +    object_property_add_str(obj, "primary_in",
>> +                            compare_get_pri_indev, compare_set_pri_indev,
>> +                            NULL);
>> +    object_property_add_str(obj, "secondary_in",
>> +                            compare_get_sec_indev, compare_set_sec_indev,
>> +                            NULL);
>> +    object_property_add_str(obj, "outdev",
>> +                            compare_get_outdev, compare_set_outdev,
>> +                            NULL);
>> +}
>> +
>> +static void colo_compare_finalize(Object *obj)
>> +{
>> +    CompareState *s = COLO_COMPARE(obj);
>> +
>> +    if (s->chr_pri_in) {
>> +        qemu_chr_add_handlers(s->chr_pri_in, NULL, NULL, NULL, NULL);
>> +        qemu_chr_fe_release(s->chr_pri_in);
>> +    }
>> +    if (s->chr_sec_in) {
>> +        qemu_chr_add_handlers(s->chr_sec_in, NULL, NULL, NULL, NULL);
>> +        qemu_chr_fe_release(s->chr_sec_in);
>> +    }
>> +    if (s->chr_out) {
>> +        qemu_chr_fe_release(s->chr_out);
>> +    }
>> +
>> +    g_free(s->pri_indev);
>> +    g_free(s->sec_indev);
>> +    g_free(s->outdev);
>> +}
>> +
>> +static const TypeInfo colo_compare_info = {
>> +    .name = TYPE_COLO_COMPARE,
>> +    .parent = TYPE_OBJECT,
>> +    .instance_size = sizeof(CompareState),
>> +    .instance_init = colo_compare_init,
>> +    .instance_finalize = colo_compare_finalize,
>> +    .class_size = sizeof(CompareClass),
>> +    .class_init = colo_compare_class_init,
>> +    .interfaces = (InterfaceInfo[]) {
>> +        { TYPE_USER_CREATABLE },
>> +        { }
>> +    }
>> +};
>> +
>> +static void register_types(void)
>> +{
>> +    type_register_static(&colo_compare_info);
>> +}
>> +
>> +type_init(register_types);
>> diff --git a/qemu-options.hx b/qemu-options.hx
>> index 587de8f..33d5d0b 100644
>> --- a/qemu-options.hx
>> +++ b/qemu-options.hx
>> @@ -3866,6 +3866,45 @@ Dump the network traffic on netdev @var{dev} to the file specified by
>>    The file format is libpcap, so it can be analyzed with tools such as tcpdump
>>    or Wireshark.
>>
>> +@item -object colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid},
>> +outdev=@var{chardevid}
>> +
>> +Colo-compare gets packet from primary_in@var{chardevid} and secondary_in@var{chardevid}, than compare primary packet with
>> +secondary packet. If the packet same, we will output primary
>
> s/If the packet same/If the packets are same/.
>
>> +packet to outdev@var{chardevid}, else we will notify colo-frame
>> +do checkpoint and send primary packet to outdev@var{chardevid}.
>> +
>> +we can use it with the help of filter-mirror and filter-redirector.
>
> s/we/We/ and looks like colo compare must be used with the help of
> mirror and redirector?
>
>> +
>> +@example
>> +
>> +primary:
>> +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown
>> +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66
>> +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
>> +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait
>> +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait
>> +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001
>> +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait
>> +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005
>> +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0
>> +-object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out
>> +-object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0
>> +-object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0
>> +
>> +secondary:
>> +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown
>> +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66
>> +-chardev socket,id=red0,host=3.3.3.3,port=9003
>> +-chardev socket,id=red1,host=3.3.3.3,port=9004
>> +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0
>> +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1
>> +
>> +@end example
>> +
>> +If you want to know the detail of above command line, you can read
>> +the colo-compare git log.
>> +
>>    @item -object secret,id=@var{id},data=@var{string},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}]
>>    @item -object secret,id=@var{id},file=@var{filename},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}]
>>
>> diff --git a/vl.c b/vl.c
>> index cbe51ac..c6b9a6f 100644
>> --- a/vl.c
>> +++ b/vl.c
>> @@ -2865,7 +2865,8 @@ static bool object_create_initial(const char *type)
>>        if (g_str_equal(type, "filter-buffer") ||
>>            g_str_equal(type, "filter-dump") ||
>>            g_str_equal(type, "filter-mirror") ||
>> -        g_str_equal(type, "filter-redirector")) {
>> +        g_str_equal(type, "filter-redirector") ||
>> +        g_str_equal(type, "colo-compare")) {
>>            return false;
>>        }
>>
>
>
> .
>

On 08/31/2016 03:53 PM, Jason Wang wrote:
>
>
> On 2016年08月17日 16:10, Zhang Chen wrote:
>> This a COLO net ascii figure:
>>
>>   Primary qemu Secondary qemu
>> +--------------------------------------------------------------+ 
>> +----------------------------------------------------------------+
>> | +----------------------------------------------------------+ 
>> |       | 
>> +-----------------------------------------------------------+ |
>> | |                                                          | 
>> |       | |                                                           
>> | |
>> | |                        guest                             | 
>> |       |  | guest                              | |
>> | |                                                          | 
>> |       | |                                                           
>> | |
>> | +-------^--------------------------+-----------------------+ 
>> |       | 
>> +---------------------+--------+----------------------------+ |
>> |         |                          | |       
>> |                        ^ |                              |
>> |         |                          | |       
>> |                        | |                              |
>> |         | +------------------------------------------------------+ 
>> |                        |        | |
>> |netfilter|  |                       | |    |  |   
>> netfilter            | |                              |
>> | +----------+ +----------------------------+ |    |  | 
>> +-----------------------------------------------------------+ |
>> | |       |  |                       |      |        out |    |  |  
>> |                     |        |  filter excute order       | |
>> | |       |  |          +-----------------------------+ |    |  |  
>> |                     |        | +------------------->      | |
>> | |       |  |          |            |      |         | |    |  |  
>> |                     |        | TCP                      | |
>> | | +-----+--+-+  +-----v----+ +-----v----+ |pri +----+----+sec|    
>> |  |  | +------------+ +---+----+---v+rewriter++  +------------+ | |
>> | | |          |  |          | |          | |in  |         |in |    
>> |  |  | |            |  |        |              | |            | | |
>> | | |  filter  |  |  filter  | |  filter  +------>  colo <------+ 
>> +-------->  filter   +--> adjust | adjust     +-->   filter   | | |
>> | | |  mirror  |  |redirector| |redirector| |    | compare | |  |    
>> |  | | redirector |  | ack    |   seq        |  | redirector | | |
>> | | |          |  |          | |          | |    |         | |  |    
>> |  | |            |  |        |              | |            | | |
>> | | +----^-----+  +----+-----+ +----------+ |    +---------+ |  |    
>> |  | +------------+  +--------+--------------+ +---+--------+ | |
>> | |      |   tx        |   rx           rx  | |  |    |  |            
>> tx                        all       | rx      | |
>> | |      |             |                    | |  |    | 
>> +-----------------------------------------------------------+ |
>> | |      |             +--------------+     | |  |    | |            |
>> | |      |   filter excute order      |     | |  |    | |            |
>> | |      |  +---------------->        | |                  | 
>> +--------------------------------------------------------+ |
>> | +-----------------------------------------+ | | |
>> |        |                            | | | |
>> +--------------------------------------------------------------+ 
>> +----------------------------------------------------------------+
>>           |guest receive               | guest send
>>           |                            |
>> +--------+----------------------------v------------------------+
>> | |                          NOTE: filter direction is rx/tx/all
>> |                         tap |                          rx:receive 
>> packets sent to the netdev
>> | |                          tx:receive packets sent by the netdev
>> +--------------------------------------------------------------+
>
> It's better to add a doc under docs to explain this configuration in 
> detail on top of this series.
>

As you say, Am I add /docs/colo-proxy.txt to explain it or add this in 
hailiang's COLO-FT.txt after merge?

>> In COLO-compare, we do packet comparing job.
>> Packets coming from the primary char indev will be sent to outdev.
>> Packets coming from the secondary char dev will be dropped after 
>> comparing.
>> colo-comapre need two input chardev and one output chardev:
>> primary_in=chardev1-id (source: primary send packet)
>> secondary_in=chardev2-id (source: secondary send packet)
>> outdev=chardev3-id
>>
>> usage:
>>
>> primary:
>> -netdev 
>> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown
>> -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66
>> -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
>> -chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait
>> -chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait
>> -chardev socket,id=compare0-0,host=3.3.3.3,port=9001
>> -chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait
>> -chardev socket,id=compare_out0,host=3.3.3.3,port=9005
>> -object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0
>> -object 
>> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out
>> -object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0
>> -object 
>> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0
>>
>> secondary:
>> -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down 
>> script=/etc/qemu-ifdown
>> -device e1000,netdev=hn0,mac=52:a4:00:12:78:66
>> -chardev socket,id=red0,host=3.3.3.3,port=9003
>> -chardev socket,id=red1,host=3.3.3.3,port=9004
>> -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0
>> -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1
>>
>> Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
>> Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
>> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
>> ---
>>   net/Makefile.objs  |   1 +
>>   net/colo-compare.c | 284 
>> +++++++++++++++++++++++++++++++++++++++++++++++++++++
>>   qemu-options.hx    |  39 ++++++++
>>   vl.c               |   3 +-
>>   4 files changed, 326 insertions(+), 1 deletion(-)
>>   create mode 100644 net/colo-compare.c
>>
>> diff --git a/net/Makefile.objs b/net/Makefile.objs
>> index b7c22fd..ba92f73 100644
>> --- a/net/Makefile.objs
>> +++ b/net/Makefile.objs
>> @@ -16,3 +16,4 @@ common-obj-$(CONFIG_NETMAP) += netmap.o
>>   common-obj-y += filter.o
>>   common-obj-y += filter-buffer.o
>>   common-obj-y += filter-mirror.o
>> +common-obj-y += colo-compare.o
>> diff --git a/net/colo-compare.c b/net/colo-compare.c
>> new file mode 100644
>> index 0000000..cdc3e0e
>> --- /dev/null
>> +++ b/net/colo-compare.c
>> @@ -0,0 +1,284 @@
>> +/*
>> + * COarse-grain LOck-stepping Virtual Machines for Non-stop Service 
>> (COLO)
>> + * (a.k.a. Fault Tolerance or Continuous Replication)
>> + *
>> + * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD.
>> + * Copyright (c) 2016 FUJITSU LIMITED
>> + * Copyright (c) 2016 Intel Corporation
>> + *
>> + * Author: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
>> + *
>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>> + * later.  See the COPYING file in the top-level directory.
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include "qemu/error-report.h"
>> +#include "qemu-common.h"
>> +#include "qapi/qmp/qerror.h"
>> +#include "qapi/error.h"
>> +#include "net/net.h"
>> +#include "net/vhost_net.h"
>
> Looks unnecessary.

I will remove it.

>
>> +#include "qom/object_interfaces.h"
>> +#include "qemu/iov.h"
>> +#include "qom/object.h"
>> +#include "qemu/typedefs.h"
>> +#include "net/queue.h"
>> +#include "sysemu/char.h"
>> +#include "qemu/sockets.h"
>> +#include "qapi-visit.h"
>> +
>> +#define TYPE_COLO_COMPARE "colo-compare"
>> +#define COLO_COMPARE(obj) \
>> +    OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE)
>> +
>> +#define COMPARE_READ_LEN_MAX NET_BUFSIZE
>> +
>> +typedef struct CompareState {
>> +    Object parent;
>> +
>> +    char *pri_indev;
>> +    char *sec_indev;
>> +    char *outdev;
>> +    CharDriverState *chr_pri_in;
>> +    CharDriverState *chr_sec_in;
>> +    CharDriverState *chr_out;
>> +    QTAILQ_ENTRY(CompareState) next;
>
> This looks not used in this series but in commit "colo-compare and 
> filter-rewriter work with colo-frame". We'd better delay the 
> introducing to that patch.

OK~ I got your point.

>
>> +    SocketReadState pri_rs;
>> +    SocketReadState sec_rs;
>> +} CompareState;
>> +
>> +typedef struct CompareClass {
>> +    ObjectClass parent_class;
>> +} CompareClass;
>> +
>> +typedef struct CompareChardevProps {
>> +    bool is_socket;
>> +    bool is_unix;
>> +} CompareChardevProps;
>> +
>> +static char *compare_get_pri_indev(Object *obj, Error **errp)
>> +{
>> +    CompareState *s = COLO_COMPARE(obj);
>> +
>> +    return g_strdup(s->pri_indev);
>> +}
>> +
>> +static void compare_set_pri_indev(Object *obj, const char *value, 
>> Error **errp)
>> +{
>> +    CompareState *s = COLO_COMPARE(obj);
>> +
>> +    g_free(s->pri_indev);
>> +    s->pri_indev = g_strdup(value);
>> +}
>> +
>> +static char *compare_get_sec_indev(Object *obj, Error **errp)
>> +{
>> +    CompareState *s = COLO_COMPARE(obj);
>> +
>> +    return g_strdup(s->sec_indev);
>> +}
>> +
>> +static void compare_set_sec_indev(Object *obj, const char *value, 
>> Error **errp)
>> +{
>> +    CompareState *s = COLO_COMPARE(obj);
>> +
>> +    g_free(s->sec_indev);
>> +    s->sec_indev = g_strdup(value);
>> +}
>> +
>> +static char *compare_get_outdev(Object *obj, Error **errp)
>> +{
>> +    CompareState *s = COLO_COMPARE(obj);
>> +
>> +    return g_strdup(s->outdev);
>> +}
>> +
>> +static void compare_set_outdev(Object *obj, const char *value, Error 
>> **errp)
>> +{
>> +    CompareState *s = COLO_COMPARE(obj);
>> +
>> +    g_free(s->outdev);
>> +    s->outdev = g_strdup(value);
>> +}
>> +
>> +static void compare_pri_rs_finalize(SocketReadState *pri_rs)
>> +{
>> +    /* if packet_enqueue pri pkt failed we will send unsupported 
>> packet */
>> +}
>> +
>> +static void compare_sec_rs_finalize(SocketReadState *sec_rs)
>> +{
>> +    /* if packet_enqueue sec pkt failed we will notify trace */
>> +}
>> +
>> +static int compare_chardev_opts(void *opaque,
>> +                                const char *name, const char *value,
>> +                                Error **errp)
>> +{
>> +    CompareChardevProps *props = opaque;
>> +
>> +    if (strcmp(name, "backend") == 0 && strcmp(value, "socket") == 0) {
>> +        props->is_socket = true;
>> +    } else if (strcmp(name, "host") == 0) {
>
> Typo? net_vhost_chardev_opts() did:
>
>     } else if (strcmp(name, "path") == 0) {
>         props->is_unix = true;
>     }
>
>

No, In colo-compare we use chardev like this:

-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait

If we only use "path" here will trigger a error.
Should I add anthor "path" here?


>
>> +        props->is_unix = true;
>> +    } else if (strcmp(name, "port") == 0) {
>> +    } else if (strcmp(name, "server") == 0) {
>> +    } else if (strcmp(name, "wait") == 0) {
>> +    } else {
>> +        error_setg(errp,
>> +                   "COLO-compare does not support a chardev with 
>> option %s=%s",
>> +                   name, value);
>> +        return -1;
>> +    }
>> +    return 0;
>> +}
>> +
>> +/*
>> + * called from the main thread on the primary
>> + * to setup colo-compare.
>> + */
>> +static void colo_compare_complete(UserCreatable *uc, Error **errp)
>> +{
>> +    CompareState *s = COLO_COMPARE(uc);
>> +    CompareChardevProps props;
>> +
>> +    if (!s->pri_indev || !s->sec_indev || !s->outdev) {
>> +        error_setg(errp, "colo compare needs 'primary_in' ,"
>> +                   "'secondary_in','outdev' property set");
>> +        return;
>> +    } else if (!strcmp(s->pri_indev, s->outdev) ||
>> +               !strcmp(s->sec_indev, s->outdev) ||
>> +               !strcmp(s->pri_indev, s->sec_indev)) {
>> +        error_setg(errp, "'indev' and 'outdev' could not be same "
>> +                   "for compare module");
>> +        return;
>> +    }
>> +
>> +    s->chr_pri_in = qemu_chr_find(s->pri_indev);
>> +    if (s->chr_pri_in == NULL) {
>> +        error_setg(errp, "Primary IN Device '%s' not found",
>> +                   s->pri_indev);
>> +        return;
>> +    }
>> +
>> +    /* inspect chardev opts */
>> +    memset(&props, 0, sizeof(props));
>> +    if (qemu_opt_foreach(s->chr_pri_in->opts, compare_chardev_opts, 
>> &props, errp)) {
>> +        return;
>> +    }
>> +
>> +    if (!props.is_socket || !props.is_unix) {
>> +        error_setg(errp, "chardev \"%s\" is not a unix socket",
>> +                   s->pri_indev);
>> +        return;
>> +    }
>> +
>> +    s->chr_sec_in = qemu_chr_find(s->sec_indev);
>> +    if (s->chr_sec_in == NULL) {
>> +        error_setg(errp, "Secondary IN Device '%s' not found",
>> +                   s->sec_indev);
>> +        return;
>> +    }
>> +
>> +    memset(&props, 0, sizeof(props));
>> +    if (qemu_opt_foreach(s->chr_sec_in->opts, compare_chardev_opts, 
>> &props, errp)) {
>> +        return;
>> +    }
>> +
>> +    if (!props.is_socket || !props.is_unix) {
>> +        error_setg(errp, "chardev \"%s\" is not a unix socket",
>> +                   s->sec_indev);
>
> I believe tcp socket is also supported?

If I understand correctly, "tcp socket" in here is the "-chardev socket".
I will rename "unix socket" to "tcp socket".

>
>> +        return;
>> +    }
>> +
>> +    s->chr_out = qemu_chr_find(s->outdev);
>> +    if (s->chr_out == NULL) {
>> +        error_setg(errp, "OUT Device '%s' not found", s->outdev);
>> +        return;
>> +    }
>> +
>> +    memset(&props, 0, sizeof(props));
>> +    if (qemu_opt_foreach(s->chr_out->opts, compare_chardev_opts, 
>> &props, errp)) {
>> +        return;
>> +    }
>> +
>> +    if (!props.is_socket || !props.is_unix) {
>> +        error_setg(errp, "chardev \"%s\" is not a unix socket",
>> +                   s->outdev);
>
> Ditto, and there's code duplication, please introduce a helper to do 
> above.

I don't understand what the "helper"?
In here we check each chardev, will I change to "goto error;" ?

>
>> +        return;
>> +    }
>> +
>> +    qemu_chr_fe_claim_no_fail(s->chr_pri_in);
>> +
>> +    qemu_chr_fe_claim_no_fail(s->chr_sec_in);
>> +
>> +    qemu_chr_fe_claim_no_fail(s->chr_out);
>> +
>> +    net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize);
>> +    net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize);
>> +
>> +    return;
>> +}
>> +
>> +static void colo_compare_class_init(ObjectClass *oc, void *data)
>> +{
>> +    UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
>> +
>> +    ucc->complete = colo_compare_complete;
>> +}
>> +
>> +static void colo_compare_init(Object *obj)
>> +{
>> +    object_property_add_str(obj, "primary_in",
>> +                            compare_get_pri_indev, 
>> compare_set_pri_indev,
>> +                            NULL);
>> +    object_property_add_str(obj, "secondary_in",
>> +                            compare_get_sec_indev, 
>> compare_set_sec_indev,
>> +                            NULL);
>> +    object_property_add_str(obj, "outdev",
>> +                            compare_get_outdev, compare_set_outdev,
>> +                            NULL);
>> +}
>> +
>> +static void colo_compare_finalize(Object *obj)
>> +{
>> +    CompareState *s = COLO_COMPARE(obj);
>> +
>> +    if (s->chr_pri_in) {
>> +        qemu_chr_add_handlers(s->chr_pri_in, NULL, NULL, NULL, NULL);
>> +        qemu_chr_fe_release(s->chr_pri_in);
>> +    }
>> +    if (s->chr_sec_in) {
>> +        qemu_chr_add_handlers(s->chr_sec_in, NULL, NULL, NULL, NULL);
>> +        qemu_chr_fe_release(s->chr_sec_in);
>> +    }
>> +    if (s->chr_out) {
>> +        qemu_chr_fe_release(s->chr_out);
>> +    }
>> +
>> +    g_free(s->pri_indev);
>> +    g_free(s->sec_indev);
>> +    g_free(s->outdev);
>> +}
>> +
>> +static const TypeInfo colo_compare_info = {
>> +    .name = TYPE_COLO_COMPARE,
>> +    .parent = TYPE_OBJECT,
>> +    .instance_size = sizeof(CompareState),
>> +    .instance_init = colo_compare_init,
>> +    .instance_finalize = colo_compare_finalize,
>> +    .class_size = sizeof(CompareClass),
>> +    .class_init = colo_compare_class_init,
>> +    .interfaces = (InterfaceInfo[]) {
>> +        { TYPE_USER_CREATABLE },
>> +        { }
>> +    }
>> +};
>> +
>> +static void register_types(void)
>> +{
>> +    type_register_static(&colo_compare_info);
>> +}
>> +
>> +type_init(register_types);
>> diff --git a/qemu-options.hx b/qemu-options.hx
>> index 587de8f..33d5d0b 100644
>> --- a/qemu-options.hx
>> +++ b/qemu-options.hx
>> @@ -3866,6 +3866,45 @@ Dump the network traffic on netdev @var{dev} 
>> to the file specified by
>>   The file format is libpcap, so it can be analyzed with tools such 
>> as tcpdump
>>   or Wireshark.
>>   +@item -object 
>> colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid},
>> +outdev=@var{chardevid}
>> +
>> +Colo-compare gets packet from primary_in@var{chardevid} and 
>> secondary_in@var{chardevid}, than compare primary packet with
>> +secondary packet. If the packet same, we will output primary
>
> s/If the packet same/If the packets are same/.

OK.

>
>> +packet to outdev@var{chardevid}, else we will notify colo-frame
>> +do checkpoint and send primary packet to outdev@var{chardevid}.
>> +
>> +we can use it with the help of filter-mirror and filter-redirector.
>
> s/we/We/ and looks like colo compare must be used with the help of 
> mirror and redirector?

Currently yes.

>
>> +
>> +@example
>> +
>> +primary:
>> +-netdev 
>> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown
>> +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66
>> +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
>> +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait
>> +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait
>> +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001
>> +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait
>> +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005
>> +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0
>> +-object 
>> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out
>> +-object 
>> filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0
>> +-object 
>> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0
>> +
>> +secondary:
>> +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down 
>> script=/etc/qemu-ifdown
>> +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66
>> +-chardev socket,id=red0,host=3.3.3.3,port=9003
>> +-chardev socket,id=red1,host=3.3.3.3,port=9004
>> +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0
>> +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1
>> +
>> +@end example
>> +
>> +If you want to know the detail of above command line, you can read
>> +the colo-compare git log.
>> +
>>   @item -object 
>> secret,id=@var{id},data=@var{string},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}]
>>   @item -object 
>> secret,id=@var{id},file=@var{filename},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}]
>>   diff --git a/vl.c b/vl.c
>> index cbe51ac..c6b9a6f 100644
>> --- a/vl.c
>> +++ b/vl.c
>> @@ -2865,7 +2865,8 @@ static bool object_create_initial(const char 
>> *type)
>>       if (g_str_equal(type, "filter-buffer") ||
>>           g_str_equal(type, "filter-dump") ||
>>           g_str_equal(type, "filter-mirror") ||
>> -        g_str_equal(type, "filter-redirector")) {
>> +        g_str_equal(type, "filter-redirector") ||
>> +        g_str_equal(type, "colo-compare")) {
>>           return false;
>>       }
>
>
>
> .
>

On 2016年08月31日 17:03, Zhang Chen wrote:
>
>
> On 08/31/2016 03:53 PM, Jason Wang wrote:
>>
>>
>> On 2016年08月17日 16:10, Zhang Chen wrote:
>>> This a COLO net ascii figure:
>>>
>>>   Primary qemu Secondary qemu
>>> +--------------------------------------------------------------+ 
>>> +----------------------------------------------------------------+
>>> | +----------------------------------------------------------+ 
>>> |       | 
>>> +-----------------------------------------------------------+ |
>>> | |                                                          | 
>>> |       | 
>>> |                                                           | |
>>> | |                        guest                             | 
>>> |       |  | guest                              | |
>>> | |                                                          | 
>>> |       | 
>>> |                                                           | |
>>> | +-------^--------------------------+-----------------------+ 
>>> |       | 
>>> +---------------------+--------+----------------------------+ |
>>> |         |                          | | |                        ^ 
>>> |                              |
>>> |         |                          | | |                        | 
>>> |                              |
>>> |         | +------------------------------------------------------+ 
>>> |                        |        | |
>>> |netfilter|  |                       | |    |  | 
>>> netfilter            | |                              |
>>> | +----------+ +----------------------------+ |    |  | 
>>> +-----------------------------------------------------------+ |
>>> | |       |  |                       |      |        out | |  |  
>>> |                     |        |  filter excute order       | |
>>> | |       |  |          +-----------------------------+ | |  |  
>>> |                     |        | +------------------->      | |
>>> | |       |  |          |            |      |         | | |  |  
>>> |                     |        | TCP                      | |
>>> | | +-----+--+-+  +-----v----+ +-----v----+ |pri +----+----+sec|    
>>> |  |  | +------------+ +---+----+---v+rewriter++  +------------+ | |
>>> | | |          |  |          | |          | |in  |         |in |    
>>> |  |  | |            |  |        |              | |            | | |
>>> | | |  filter  |  |  filter  | |  filter  +------>  colo <------+ 
>>> +-------->  filter   +--> adjust | adjust     +-->   filter   | | |
>>> | | |  mirror  |  |redirector| |redirector| |    | compare | |  |    
>>> |  | | redirector |  | ack    |   seq        |  | redirector | | |
>>> | | |          |  |          | |          | |    |         | |  |    
>>> |  | |            |  |        |              | |            | | |
>>> | | +----^-----+  +----+-----+ +----------+ |    +---------+ |  |    
>>> |  | +------------+  +--------+--------------+ +---+--------+ | |
>>> | |      |   tx        |   rx           rx  | |  |    | |            
>>> tx                        all       | rx      | |
>>> | |      |             |                    | |  |    | 
>>> +-----------------------------------------------------------+ |
>>> | |      |             +--------------+     | |  |    | |            |
>>> | |      |   filter excute order      |     | |  |    | |            |
>>> | |      |  +---------------->        | | | 
>>> +--------------------------------------------------------+ |
>>> | +-----------------------------------------+ | | |
>>> |        |                            | | | |
>>> +--------------------------------------------------------------+ 
>>> +----------------------------------------------------------------+
>>>           |guest receive               | guest send
>>>           |                            |
>>> +--------+----------------------------v------------------------+
>>> | |                          NOTE: filter direction is rx/tx/all
>>> |                         tap | rx:receive packets sent to the netdev
>>> | |                          tx:receive packets sent by the netdev
>>> +--------------------------------------------------------------+
>>
>> It's better to add a doc under docs to explain this configuration in 
>> detail on top of this series.
>>
>
> As you say, Am I add /docs/colo-proxy.txt to explain it or add this in 
> hailiang's COLO-FT.txt after merge?
>
>>> In COLO-compare, we do packet comparing job.
>>> Packets coming from the primary char indev will be sent to outdev.
>>> Packets coming from the secondary char dev will be dropped after 
>>> comparing.
>>> colo-comapre need two input chardev and one output chardev:
>>> primary_in=chardev1-id (source: primary send packet)
>>> secondary_in=chardev2-id (source: secondary send packet)
>>> outdev=chardev3-id
>>>
>>> usage:
>>>
>>> primary:
>>> -netdev 
>>> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown
>>> -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66
>>> -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
>>> -chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait
>>> -chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait
>>> -chardev socket,id=compare0-0,host=3.3.3.3,port=9001
>>> -chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait
>>> -chardev socket,id=compare_out0,host=3.3.3.3,port=9005
>>> -object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0
>>> -object 
>>> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out
>>> -object 
>>> filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0
>>> -object 
>>> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0
>>>
>>> secondary:
>>> -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down 
>>> script=/etc/qemu-ifdown
>>> -device e1000,netdev=hn0,mac=52:a4:00:12:78:66
>>> -chardev socket,id=red0,host=3.3.3.3,port=9003
>>> -chardev socket,id=red1,host=3.3.3.3,port=9004
>>> -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0
>>> -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1
>>>
>>> Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
>>> Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
>>> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
>>> ---
>>>   net/Makefile.objs  |   1 +
>>>   net/colo-compare.c | 284 
>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>   qemu-options.hx    |  39 ++++++++
>>>   vl.c               |   3 +-
>>>   4 files changed, 326 insertions(+), 1 deletion(-)
>>>   create mode 100644 net/colo-compare.c
>>>
>>> diff --git a/net/Makefile.objs b/net/Makefile.objs
>>> index b7c22fd..ba92f73 100644
>>> --- a/net/Makefile.objs
>>> +++ b/net/Makefile.objs
>>> @@ -16,3 +16,4 @@ common-obj-$(CONFIG_NETMAP) += netmap.o
>>>   common-obj-y += filter.o
>>>   common-obj-y += filter-buffer.o
>>>   common-obj-y += filter-mirror.o
>>> +common-obj-y += colo-compare.o
>>> diff --git a/net/colo-compare.c b/net/colo-compare.c
>>> new file mode 100644
>>> index 0000000..cdc3e0e
>>> --- /dev/null
>>> +++ b/net/colo-compare.c
>>> @@ -0,0 +1,284 @@
>>> +/*
>>> + * COarse-grain LOck-stepping Virtual Machines for Non-stop Service 
>>> (COLO)
>>> + * (a.k.a. Fault Tolerance or Continuous Replication)
>>> + *
>>> + * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD.
>>> + * Copyright (c) 2016 FUJITSU LIMITED
>>> + * Copyright (c) 2016 Intel Corporation
>>> + *
>>> + * Author: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
>>> + *
>>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>>> + * later.  See the COPYING file in the top-level directory.
>>> + */
>>> +
>>> +#include "qemu/osdep.h"
>>> +#include "qemu/error-report.h"
>>> +#include "qemu-common.h"
>>> +#include "qapi/qmp/qerror.h"
>>> +#include "qapi/error.h"
>>> +#include "net/net.h"
>>> +#include "net/vhost_net.h"
>>
>> Looks unnecessary.
>
> I will remove it.
>
>>
>>> +#include "qom/object_interfaces.h"
>>> +#include "qemu/iov.h"
>>> +#include "qom/object.h"
>>> +#include "qemu/typedefs.h"
>>> +#include "net/queue.h"
>>> +#include "sysemu/char.h"
>>> +#include "qemu/sockets.h"
>>> +#include "qapi-visit.h"
>>> +
>>> +#define TYPE_COLO_COMPARE "colo-compare"
>>> +#define COLO_COMPARE(obj) \
>>> +    OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE)
>>> +
>>> +#define COMPARE_READ_LEN_MAX NET_BUFSIZE
>>> +
>>> +typedef struct CompareState {
>>> +    Object parent;
>>> +
>>> +    char *pri_indev;
>>> +    char *sec_indev;
>>> +    char *outdev;
>>> +    CharDriverState *chr_pri_in;
>>> +    CharDriverState *chr_sec_in;
>>> +    CharDriverState *chr_out;
>>> +    QTAILQ_ENTRY(CompareState) next;
>>
>> This looks not used in this series but in commit "colo-compare and 
>> filter-rewriter work with colo-frame". We'd better delay the 
>> introducing to that patch.
>
> OK~ I got your point.
>
>>
>>> +    SocketReadState pri_rs;
>>> +    SocketReadState sec_rs;
>>> +} CompareState;
>>> +
>>> +typedef struct CompareClass {
>>> +    ObjectClass parent_class;
>>> +} CompareClass;
>>> +
>>> +typedef struct CompareChardevProps {
>>> +    bool is_socket;
>>> +    bool is_unix;
>>> +} CompareChardevProps;
>>> +
>>> +static char *compare_get_pri_indev(Object *obj, Error **errp)
>>> +{
>>> +    CompareState *s = COLO_COMPARE(obj);
>>> +
>>> +    return g_strdup(s->pri_indev);
>>> +}
>>> +
>>> +static void compare_set_pri_indev(Object *obj, const char *value, 
>>> Error **errp)
>>> +{
>>> +    CompareState *s = COLO_COMPARE(obj);
>>> +
>>> +    g_free(s->pri_indev);
>>> +    s->pri_indev = g_strdup(value);
>>> +}
>>> +
>>> +static char *compare_get_sec_indev(Object *obj, Error **errp)
>>> +{
>>> +    CompareState *s = COLO_COMPARE(obj);
>>> +
>>> +    return g_strdup(s->sec_indev);
>>> +}
>>> +
>>> +static void compare_set_sec_indev(Object *obj, const char *value, 
>>> Error **errp)
>>> +{
>>> +    CompareState *s = COLO_COMPARE(obj);
>>> +
>>> +    g_free(s->sec_indev);
>>> +    s->sec_indev = g_strdup(value);
>>> +}
>>> +
>>> +static char *compare_get_outdev(Object *obj, Error **errp)
>>> +{
>>> +    CompareState *s = COLO_COMPARE(obj);
>>> +
>>> +    return g_strdup(s->outdev);
>>> +}
>>> +
>>> +static void compare_set_outdev(Object *obj, const char *value, 
>>> Error **errp)
>>> +{
>>> +    CompareState *s = COLO_COMPARE(obj);
>>> +
>>> +    g_free(s->outdev);
>>> +    s->outdev = g_strdup(value);
>>> +}
>>> +
>>> +static void compare_pri_rs_finalize(SocketReadState *pri_rs)
>>> +{
>>> +    /* if packet_enqueue pri pkt failed we will send unsupported 
>>> packet */
>>> +}
>>> +
>>> +static void compare_sec_rs_finalize(SocketReadState *sec_rs)
>>> +{
>>> +    /* if packet_enqueue sec pkt failed we will notify trace */
>>> +}
>>> +
>>> +static int compare_chardev_opts(void *opaque,
>>> +                                const char *name, const char *value,
>>> +                                Error **errp)
>>> +{
>>> +    CompareChardevProps *props = opaque;
>>> +
>>> +    if (strcmp(name, "backend") == 0 && strcmp(value, "socket") == 
>>> 0) {
>>> +        props->is_socket = true;
>>> +    } else if (strcmp(name, "host") == 0) {
>>
>> Typo? net_vhost_chardev_opts() did:
>>
>>     } else if (strcmp(name, "path") == 0) {
>>         props->is_unix = true;
>>     }
>>
>>
>
> No, In colo-compare we use chardev like this:
>
> -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
>
> If we only use "path" here will trigger a error.
> Should I add anthor "path" here?

If I understand the code correctly, "is_unix" means "is unix domain 
socket"? If yes, according to the help:

-chardev 
socket,id=id[,host=host],port=port[,to=to][,ipv4][,ipv6][,nodelay][,reconnect=seconds]
[,server][,nowait][,telnet][,reconnect=seconds][,mux=on|off]
          [,logfile=PATH][,logappend=on|off][,tls-creds=ID] (tcp)
-chardev 
socket,id=id,path=path[,server][,nowait][,telnet][,reconnect=seconds]
          [,mux=on|off][,logfile=PATH][,logappend=on|off] (unix)

"host" will not be used for UNIX domain socket.

And if UNIX domain socket is not supported, there's probably no need to 
differentiate it from other types.

>
>
>>
>>> +        props->is_unix = true;
>>> +    } else if (strcmp(name, "port") == 0) {
>>> +    } else if (strcmp(name, "server") == 0) {
>>> +    } else if (strcmp(name, "wait") == 0) {
>>> +    } else {
>>> +        error_setg(errp,
>>> +                   "COLO-compare does not support a chardev with 
>>> option %s=%s",
>>> +                   name, value);
>>> +        return -1;
>>> +    }
>>> +    return 0;
>>> +}
>>> +
>>> +/*
>>> + * called from the main thread on the primary
>>> + * to setup colo-compare.
>>> + */
>>> +static void colo_compare_complete(UserCreatable *uc, Error **errp)
>>> +{
>>> +    CompareState *s = COLO_COMPARE(uc);
>>> +    CompareChardevProps props;
>>> +
>>> +    if (!s->pri_indev || !s->sec_indev || !s->outdev) {
>>> +        error_setg(errp, "colo compare needs 'primary_in' ,"
>>> +                   "'secondary_in','outdev' property set");
>>> +        return;
>>> +    } else if (!strcmp(s->pri_indev, s->outdev) ||
>>> +               !strcmp(s->sec_indev, s->outdev) ||
>>> +               !strcmp(s->pri_indev, s->sec_indev)) {
>>> +        error_setg(errp, "'indev' and 'outdev' could not be same "
>>> +                   "for compare module");
>>> +        return;
>>> +    }
>>> +
>>> +    s->chr_pri_in = qemu_chr_find(s->pri_indev);
>>> +    if (s->chr_pri_in == NULL) {
>>> +        error_setg(errp, "Primary IN Device '%s' not found",
>>> +                   s->pri_indev);
>>> +        return;
>>> +    }
>>> +
>>> +    /* inspect chardev opts */
>>> +    memset(&props, 0, sizeof(props));
>>> +    if (qemu_opt_foreach(s->chr_pri_in->opts, compare_chardev_opts, 
>>> &props, errp)) {
>>> +        return;
>>> +    }
>>> +
>>> +    if (!props.is_socket || !props.is_unix) {
>>> +        error_setg(errp, "chardev \"%s\" is not a unix socket",
>>> +                   s->pri_indev);
>>> +        return;
>>> +    }
>>> +
>>> +    s->chr_sec_in = qemu_chr_find(s->sec_indev);
>>> +    if (s->chr_sec_in == NULL) {
>>> +        error_setg(errp, "Secondary IN Device '%s' not found",
>>> +                   s->sec_indev);
>>> +        return;
>>> +    }
>>> +
>>> +    memset(&props, 0, sizeof(props));
>>> +    if (qemu_opt_foreach(s->chr_sec_in->opts, compare_chardev_opts, 
>>> &props, errp)) {
>>> +        return;
>>> +    }
>>> +
>>> +    if (!props.is_socket || !props.is_unix) {
>>> +        error_setg(errp, "chardev \"%s\" is not a unix socket",
>>> +                   s->sec_indev);
>>
>> I believe tcp socket is also supported?
>
> If I understand correctly, "tcp socket" in here is the "-chardev socket".
> I will rename "unix socket" to "tcp socket".
>
>>
>>> +        return;
>>> +    }
>>> +
>>> +    s->chr_out = qemu_chr_find(s->outdev);
>>> +    if (s->chr_out == NULL) {
>>> +        error_setg(errp, "OUT Device '%s' not found", s->outdev);
>>> +        return;
>>> +    }
>>> +
>>> +    memset(&props, 0, sizeof(props));
>>> +    if (qemu_opt_foreach(s->chr_out->opts, compare_chardev_opts, 
>>> &props, errp)) {
>>> +        return;
>>> +    }
>>> +
>>> +    if (!props.is_socket || !props.is_unix) {
>>> +        error_setg(errp, "chardev \"%s\" is not a unix socket",
>>> +                   s->outdev);
>>
>> Ditto, and there's code duplication, please introduce a helper to do 
>> above.
>
> I don't understand what the "helper"?
> In here we check each chardev, will I change to "goto error;" ?

A helper to avoid the code duplication for socket type inspection for 
pri_in,scr_in and chr_out.

>
>>
>>> +        return;
>>> +    }
>>> +
>>> +    qemu_chr_fe_claim_no_fail(s->chr_pri_in);
>>> +
>>> +    qemu_chr_fe_claim_no_fail(s->chr_sec_in);
>>> +
>>> +    qemu_chr_fe_claim_no_fail(s->chr_out);
>>> +
>>> +    net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize);
>>> +    net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize);
>>> +
>>> +    return;
>>> +}
>>> +
>>> +static void colo_compare_class_init(ObjectClass *oc, void *data)
>>> +{
>>> +    UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
>>> +
>>> +    ucc->complete = colo_compare_complete;
>>> +}
>>> +
>>> +static void colo_compare_init(Object *obj)
>>> +{
>>> +    object_property_add_str(obj, "primary_in",
>>> +                            compare_get_pri_indev, 
>>> compare_set_pri_indev,
>>> +                            NULL);
>>> +    object_property_add_str(obj, "secondary_in",
>>> +                            compare_get_sec_indev, 
>>> compare_set_sec_indev,
>>> +                            NULL);
>>> +    object_property_add_str(obj, "outdev",
>>> +                            compare_get_outdev, compare_set_outdev,
>>> +                            NULL);
>>> +}
>>> +
>>> +static void colo_compare_finalize(Object *obj)
>>> +{
>>> +    CompareState *s = COLO_COMPARE(obj);
>>> +
>>> +    if (s->chr_pri_in) {
>>> +        qemu_chr_add_handlers(s->chr_pri_in, NULL, NULL, NULL, NULL);
>>> +        qemu_chr_fe_release(s->chr_pri_in);
>>> +    }
>>> +    if (s->chr_sec_in) {
>>> +        qemu_chr_add_handlers(s->chr_sec_in, NULL, NULL, NULL, NULL);
>>> +        qemu_chr_fe_release(s->chr_sec_in);
>>> +    }
>>> +    if (s->chr_out) {
>>> +        qemu_chr_fe_release(s->chr_out);
>>> +    }
>>> +
>>> +    g_free(s->pri_indev);
>>> +    g_free(s->sec_indev);
>>> +    g_free(s->outdev);
>>> +}
>>> +
>>> +static const TypeInfo colo_compare_info = {
>>> +    .name = TYPE_COLO_COMPARE,
>>> +    .parent = TYPE_OBJECT,
>>> +    .instance_size = sizeof(CompareState),
>>> +    .instance_init = colo_compare_init,
>>> +    .instance_finalize = colo_compare_finalize,
>>> +    .class_size = sizeof(CompareClass),
>>> +    .class_init = colo_compare_class_init,
>>> +    .interfaces = (InterfaceInfo[]) {
>>> +        { TYPE_USER_CREATABLE },
>>> +        { }
>>> +    }
>>> +};
>>> +
>>> +static void register_types(void)
>>> +{
>>> +    type_register_static(&colo_compare_info);
>>> +}
>>> +
>>> +type_init(register_types);
>>> diff --git a/qemu-options.hx b/qemu-options.hx
>>> index 587de8f..33d5d0b 100644
>>> --- a/qemu-options.hx
>>> +++ b/qemu-options.hx
>>> @@ -3866,6 +3866,45 @@ Dump the network traffic on netdev @var{dev} 
>>> to the file specified by
>>>   The file format is libpcap, so it can be analyzed with tools such 
>>> as tcpdump
>>>   or Wireshark.
>>>   +@item -object 
>>> colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid},
>>> +outdev=@var{chardevid}
>>> +
>>> +Colo-compare gets packet from primary_in@var{chardevid} and 
>>> secondary_in@var{chardevid}, than compare primary packet with
>>> +secondary packet. If the packet same, we will output primary
>>
>> s/If the packet same/If the packets are same/.
>
> OK.
>
>>
>>> +packet to outdev@var{chardevid}, else we will notify colo-frame
>>> +do checkpoint and send primary packet to outdev@var{chardevid}.
>>> +
>>> +we can use it with the help of filter-mirror and filter-redirector.
>>
>> s/we/We/ and looks like colo compare must be used with the help of 
>> mirror and redirector?
>
> Currently yes.

Then please change the doc here.

Thanks

>
>>
>>> +
>>> +@example
>>> +
>>> +primary:
>>> +-netdev 
>>> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown
>>> +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66
>>> +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
>>> +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait
>>> +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait
>>> +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001
>>> +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait
>>> +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005
>>> +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0
>>> +-object 
>>> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out
>>> +-object 
>>> filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0
>>> +-object 
>>> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0
>>> +
>>> +secondary:
>>> +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down 
>>> script=/etc/qemu-ifdown
>>> +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66
>>> +-chardev socket,id=red0,host=3.3.3.3,port=9003
>>> +-chardev socket,id=red1,host=3.3.3.3,port=9004
>>> +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0
>>> +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1
>>> +
>>> +@end example
>>> +
>>> +If you want to know the detail of above command line, you can read
>>> +the colo-compare git log.
>>> +
>>>   @item -object 
>>> secret,id=@var{id},data=@var{string},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}]
>>>   @item -object 
>>> secret,id=@var{id},file=@var{filename},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}]
>>>   diff --git a/vl.c b/vl.c
>>> index cbe51ac..c6b9a6f 100644
>>> --- a/vl.c
>>> +++ b/vl.c
>>> @@ -2865,7 +2865,8 @@ static bool object_create_initial(const char 
>>> *type)
>>>       if (g_str_equal(type, "filter-buffer") ||
>>>           g_str_equal(type, "filter-dump") ||
>>>           g_str_equal(type, "filter-mirror") ||
>>> -        g_str_equal(type, "filter-redirector")) {
>>> +        g_str_equal(type, "filter-redirector") ||
>>> +        g_str_equal(type, "colo-compare")) {
>>>           return false;
>>>       }
>>
>>
>>
>> .
>>
>

On 08/31/2016 05:20 PM, Jason Wang wrote:
>
>
> On 2016年08月31日 17:03, Zhang Chen wrote:
>>
>>
>> On 08/31/2016 03:53 PM, Jason Wang wrote:
>>>
>>>
>>> On 2016年08月17日 16:10, Zhang Chen wrote:
>>>> This a COLO net ascii figure:
>>>>
>>>>   Primary qemu Secondary qemu
>>>> +--------------------------------------------------------------+ 
>>>> +----------------------------------------------------------------+
>>>> | +----------------------------------------------------------+ 
>>>> |       | 
>>>> +-----------------------------------------------------------+ |
>>>> | | | |       | | | |
>>>> | |                        guest | |       |  | 
>>>> guest                              | |
>>>> | | | |       | | | |
>>>> | +-------^--------------------------+-----------------------+ 
>>>> |       | 
>>>> +---------------------+--------+----------------------------+ |
>>>> |         |                          | | |                        ^ 
>>>> |                              |
>>>> |         |                          | | |                        | 
>>>> |                              |
>>>> |         | 
>>>> +------------------------------------------------------+ 
>>>> |                        |        | |
>>>> |netfilter|  |                       | |    |  | 
>>>> netfilter            | |                              |
>>>> | +----------+ +----------------------------+ |    |  | 
>>>> +-----------------------------------------------------------+ |
>>>> | |       |  |                       |      |        out | |  |  
>>>> |                     |        |  filter excute order       | |
>>>> | |       |  |          +-----------------------------+ | | |  
>>>> |                     |        | +------------------->      | |
>>>> | |       |  |          |            |      |         | | | |  
>>>> |                     |        | TCP | |
>>>> | | +-----+--+-+  +-----v----+ +-----v----+ |pri +----+----+sec|    
>>>> |  |  | +------------+ +---+----+---v+rewriter++  +------------+ | |
>>>> | | |          |  |          | |          | |in  | |in |    |  |  | 
>>>> |            |  |        |              | |            | | |
>>>> | | |  filter  |  |  filter  | |  filter  +------>  colo <------+ 
>>>> +-------->  filter   +--> adjust | adjust     +-->   filter   | | |
>>>> | | |  mirror  |  |redirector| |redirector| |    | compare | |  
>>>> |    |  | | redirector |  | ack    |   seq        |  | redirector | 
>>>> | |
>>>> | | |          |  |          | |          | |    |         | |  
>>>> |    |  | |            |  |        |              | |            | | |
>>>> | | +----^-----+  +----+-----+ +----------+ |    +---------+ |  
>>>> |    |  | +------------+  +--------+--------------+ +---+--------+ | |
>>>> | |      |   tx        |   rx           rx  | |  |    | 
>>>> |            tx                        all       | rx      | |
>>>> | |      |             |                    | |  |    | 
>>>> +-----------------------------------------------------------+ |
>>>> | |      |             +--------------+     | |  |    | |            |
>>>> | |      |   filter excute order      |     | |  |    | |            |
>>>> | |      |  +---------------->        | | | 
>>>> +--------------------------------------------------------+ |
>>>> | +-----------------------------------------+ | | |
>>>> |        |                            | | | |
>>>> +--------------------------------------------------------------+ 
>>>> +----------------------------------------------------------------+
>>>>           |guest receive               | guest send
>>>>           |                            |
>>>> +--------+----------------------------v------------------------+
>>>> | |                          NOTE: filter direction is rx/tx/all
>>>> |                         tap | rx:receive packets sent to the netdev
>>>> | |                          tx:receive packets sent by the netdev
>>>> +--------------------------------------------------------------+
>>>
>>> It's better to add a doc under docs to explain this configuration in 
>>> detail on top of this series.
>>>
>>
>> As you say, Am I add /docs/colo-proxy.txt to explain it or add this 
>> in hailiang's COLO-FT.txt after merge?

Can you give me a way for doc?


>>
>>>> In COLO-compare, we do packet comparing job.
>>>> Packets coming from the primary char indev will be sent to outdev.
>>>> Packets coming from the secondary char dev will be dropped after 
>>>> comparing.
>>>> colo-comapre need two input chardev and one output chardev:
>>>> primary_in=chardev1-id (source: primary send packet)
>>>> secondary_in=chardev2-id (source: secondary send packet)
>>>> outdev=chardev3-id
>>>>
>>>> usage:
>>>>
>>>> primary:
>>>> -netdev 
>>>> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown
>>>> -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66
>>>> -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
>>>> -chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait
>>>> -chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait
>>>> -chardev socket,id=compare0-0,host=3.3.3.3,port=9001
>>>> -chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait
>>>> -chardev socket,id=compare_out0,host=3.3.3.3,port=9005
>>>> -object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0
>>>> -object 
>>>> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out
>>>> -object 
>>>> filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0
>>>> -object 
>>>> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0
>>>>
>>>> secondary:
>>>> -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down 
>>>> script=/etc/qemu-ifdown
>>>> -device e1000,netdev=hn0,mac=52:a4:00:12:78:66
>>>> -chardev socket,id=red0,host=3.3.3.3,port=9003
>>>> -chardev socket,id=red1,host=3.3.3.3,port=9004
>>>> -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0
>>>> -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1
>>>>
>>>> Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
>>>> Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
>>>> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
>>>> ---
>>>>   net/Makefile.objs  |   1 +
>>>>   net/colo-compare.c | 284 
>>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>   qemu-options.hx    |  39 ++++++++
>>>>   vl.c               |   3 +-
>>>>   4 files changed, 326 insertions(+), 1 deletion(-)
>>>>   create mode 100644 net/colo-compare.c
>>>>
>>>> diff --git a/net/Makefile.objs b/net/Makefile.objs
>>>> index b7c22fd..ba92f73 100644
>>>> --- a/net/Makefile.objs
>>>> +++ b/net/Makefile.objs
>>>> @@ -16,3 +16,4 @@ common-obj-$(CONFIG_NETMAP) += netmap.o
>>>>   common-obj-y += filter.o
>>>>   common-obj-y += filter-buffer.o
>>>>   common-obj-y += filter-mirror.o
>>>> +common-obj-y += colo-compare.o
>>>> diff --git a/net/colo-compare.c b/net/colo-compare.c
>>>> new file mode 100644
>>>> index 0000000..cdc3e0e
>>>> --- /dev/null
>>>> +++ b/net/colo-compare.c
>>>> @@ -0,0 +1,284 @@
>>>> +/*
>>>> + * COarse-grain LOck-stepping Virtual Machines for Non-stop 
>>>> Service (COLO)
>>>> + * (a.k.a. Fault Tolerance or Continuous Replication)
>>>> + *
>>>> + * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD.
>>>> + * Copyright (c) 2016 FUJITSU LIMITED
>>>> + * Copyright (c) 2016 Intel Corporation
>>>> + *
>>>> + * Author: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
>>>> + *
>>>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>>>> + * later.  See the COPYING file in the top-level directory.
>>>> + */
>>>> +
>>>> +#include "qemu/osdep.h"
>>>> +#include "qemu/error-report.h"
>>>> +#include "qemu-common.h"
>>>> +#include "qapi/qmp/qerror.h"
>>>> +#include "qapi/error.h"
>>>> +#include "net/net.h"
>>>> +#include "net/vhost_net.h"
>>>
>>> Looks unnecessary.
>>
>> I will remove it.
>>
>>>
>>>> +#include "qom/object_interfaces.h"
>>>> +#include "qemu/iov.h"
>>>> +#include "qom/object.h"
>>>> +#include "qemu/typedefs.h"
>>>> +#include "net/queue.h"
>>>> +#include "sysemu/char.h"
>>>> +#include "qemu/sockets.h"
>>>> +#include "qapi-visit.h"
>>>> +
>>>> +#define TYPE_COLO_COMPARE "colo-compare"
>>>> +#define COLO_COMPARE(obj) \
>>>> +    OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE)
>>>> +
>>>> +#define COMPARE_READ_LEN_MAX NET_BUFSIZE
>>>> +
>>>> +typedef struct CompareState {
>>>> +    Object parent;
>>>> +
>>>> +    char *pri_indev;
>>>> +    char *sec_indev;
>>>> +    char *outdev;
>>>> +    CharDriverState *chr_pri_in;
>>>> +    CharDriverState *chr_sec_in;
>>>> +    CharDriverState *chr_out;
>>>> +    QTAILQ_ENTRY(CompareState) next;
>>>
>>> This looks not used in this series but in commit "colo-compare and 
>>> filter-rewriter work with colo-frame". We'd better delay the 
>>> introducing to that patch.
>>
>> OK~ I got your point.
>>
>>>
>>>> +    SocketReadState pri_rs;
>>>> +    SocketReadState sec_rs;
>>>> +} CompareState;
>>>> +
>>>> +typedef struct CompareClass {
>>>> +    ObjectClass parent_class;
>>>> +} CompareClass;
>>>> +
>>>> +typedef struct CompareChardevProps {
>>>> +    bool is_socket;
>>>> +    bool is_unix;
>>>> +} CompareChardevProps;
>>>> +
>>>> +static char *compare_get_pri_indev(Object *obj, Error **errp)
>>>> +{
>>>> +    CompareState *s = COLO_COMPARE(obj);
>>>> +
>>>> +    return g_strdup(s->pri_indev);
>>>> +}
>>>> +
>>>> +static void compare_set_pri_indev(Object *obj, const char *value, 
>>>> Error **errp)
>>>> +{
>>>> +    CompareState *s = COLO_COMPARE(obj);
>>>> +
>>>> +    g_free(s->pri_indev);
>>>> +    s->pri_indev = g_strdup(value);
>>>> +}
>>>> +
>>>> +static char *compare_get_sec_indev(Object *obj, Error **errp)
>>>> +{
>>>> +    CompareState *s = COLO_COMPARE(obj);
>>>> +
>>>> +    return g_strdup(s->sec_indev);
>>>> +}
>>>> +
>>>> +static void compare_set_sec_indev(Object *obj, const char *value, 
>>>> Error **errp)
>>>> +{
>>>> +    CompareState *s = COLO_COMPARE(obj);
>>>> +
>>>> +    g_free(s->sec_indev);
>>>> +    s->sec_indev = g_strdup(value);
>>>> +}
>>>> +
>>>> +static char *compare_get_outdev(Object *obj, Error **errp)
>>>> +{
>>>> +    CompareState *s = COLO_COMPARE(obj);
>>>> +
>>>> +    return g_strdup(s->outdev);
>>>> +}
>>>> +
>>>> +static void compare_set_outdev(Object *obj, const char *value, 
>>>> Error **errp)
>>>> +{
>>>> +    CompareState *s = COLO_COMPARE(obj);
>>>> +
>>>> +    g_free(s->outdev);
>>>> +    s->outdev = g_strdup(value);
>>>> +}
>>>> +
>>>> +static void compare_pri_rs_finalize(SocketReadState *pri_rs)
>>>> +{
>>>> +    /* if packet_enqueue pri pkt failed we will send unsupported 
>>>> packet */
>>>> +}
>>>> +
>>>> +static void compare_sec_rs_finalize(SocketReadState *sec_rs)
>>>> +{
>>>> +    /* if packet_enqueue sec pkt failed we will notify trace */
>>>> +}
>>>> +
>>>> +static int compare_chardev_opts(void *opaque,
>>>> +                                const char *name, const char *value,
>>>> +                                Error **errp)
>>>> +{
>>>> +    CompareChardevProps *props = opaque;
>>>> +
>>>> +    if (strcmp(name, "backend") == 0 && strcmp(value, "socket") == 
>>>> 0) {
>>>> +        props->is_socket = true;
>>>> +    } else if (strcmp(name, "host") == 0) {
>>>
>>> Typo? net_vhost_chardev_opts() did:
>>>
>>>     } else if (strcmp(name, "path") == 0) {
>>>         props->is_unix = true;
>>>     }
>>>
>>>
>>
>> No, In colo-compare we use chardev like this:
>>
>> -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
>>
>> If we only use "path" here will trigger a error.
>> Should I add anthor "path" here?
>
> If I understand the code correctly, "is_unix" means "is unix domain 
> socket"? If yes, according to the help:
>
> -chardev 
> socket,id=id[,host=host],port=port[,to=to][,ipv4][,ipv6][,nodelay][,reconnect=seconds]
> [,server][,nowait][,telnet][,reconnect=seconds][,mux=on|off]
>          [,logfile=PATH][,logappend=on|off][,tls-creds=ID] (tcp)
> -chardev 
> socket,id=id,path=path[,server][,nowait][,telnet][,reconnect=seconds]
>          [,mux=on|off][,logfile=PATH][,logappend=on|off] (unix)
>
> "host" will not be used for UNIX domain socket.
>
> And if UNIX domain socket is not supported, there's probably no need 
> to differentiate it from other types.

OK, I will remove the "is_unix" in next version.


>
>>
>>
>>>
>>>> +        props->is_unix = true;
>>>> +    } else if (strcmp(name, "port") == 0) {
>>>> +    } else if (strcmp(name, "server") == 0) {
>>>> +    } else if (strcmp(name, "wait") == 0) {
>>>> +    } else {
>>>> +        error_setg(errp,
>>>> +                   "COLO-compare does not support a chardev with 
>>>> option %s=%s",
>>>> +                   name, value);
>>>> +        return -1;
>>>> +    }
>>>> +    return 0;
>>>> +}
>>>> +
>>>> +/*
>>>> + * called from the main thread on the primary
>>>> + * to setup colo-compare.
>>>> + */
>>>> +static void colo_compare_complete(UserCreatable *uc, Error **errp)
>>>> +{
>>>> +    CompareState *s = COLO_COMPARE(uc);
>>>> +    CompareChardevProps props;
>>>> +
>>>> +    if (!s->pri_indev || !s->sec_indev || !s->outdev) {
>>>> +        error_setg(errp, "colo compare needs 'primary_in' ,"
>>>> +                   "'secondary_in','outdev' property set");
>>>> +        return;
>>>> +    } else if (!strcmp(s->pri_indev, s->outdev) ||
>>>> +               !strcmp(s->sec_indev, s->outdev) ||
>>>> +               !strcmp(s->pri_indev, s->sec_indev)) {
>>>> +        error_setg(errp, "'indev' and 'outdev' could not be same "
>>>> +                   "for compare module");
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    s->chr_pri_in = qemu_chr_find(s->pri_indev);
>>>> +    if (s->chr_pri_in == NULL) {
>>>> +        error_setg(errp, "Primary IN Device '%s' not found",
>>>> +                   s->pri_indev);
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    /* inspect chardev opts */
>>>> +    memset(&props, 0, sizeof(props));
>>>> +    if (qemu_opt_foreach(s->chr_pri_in->opts, 
>>>> compare_chardev_opts, &props, errp)) {
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    if (!props.is_socket || !props.is_unix) {
>>>> +        error_setg(errp, "chardev \"%s\" is not a unix socket",
>>>> +                   s->pri_indev);
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    s->chr_sec_in = qemu_chr_find(s->sec_indev);
>>>> +    if (s->chr_sec_in == NULL) {
>>>> +        error_setg(errp, "Secondary IN Device '%s' not found",
>>>> +                   s->sec_indev);
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    memset(&props, 0, sizeof(props));
>>>> +    if (qemu_opt_foreach(s->chr_sec_in->opts, 
>>>> compare_chardev_opts, &props, errp)) {
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    if (!props.is_socket || !props.is_unix) {
>>>> +        error_setg(errp, "chardev \"%s\" is not a unix socket",
>>>> +                   s->sec_indev);
>>>
>>> I believe tcp socket is also supported?
>>
>> If I understand correctly, "tcp socket" in here is the "-chardev 
>> socket".
>> I will rename "unix socket" to "tcp socket".
>>
>>>
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    s->chr_out = qemu_chr_find(s->outdev);
>>>> +    if (s->chr_out == NULL) {
>>>> +        error_setg(errp, "OUT Device '%s' not found", s->outdev);
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    memset(&props, 0, sizeof(props));
>>>> +    if (qemu_opt_foreach(s->chr_out->opts, compare_chardev_opts, 
>>>> &props, errp)) {
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    if (!props.is_socket || !props.is_unix) {
>>>> +        error_setg(errp, "chardev \"%s\" is not a unix socket",
>>>> +                   s->outdev);
>>>
>>> Ditto, and there's code duplication, please introduce a helper to do 
>>> above.
>>
>> I don't understand what the "helper"?
>> In here we check each chardev, will I change to "goto error;" ?
>
> A helper to avoid the code duplication for socket type inspection for 
> pri_in,scr_in and chr_out.

I got it~~
I will add it in next version.


>
>>
>>>
>>>> +        return;
>>>> +    }
>>>> +
>>>> +    qemu_chr_fe_claim_no_fail(s->chr_pri_in);
>>>> +
>>>> +    qemu_chr_fe_claim_no_fail(s->chr_sec_in);
>>>> +
>>>> +    qemu_chr_fe_claim_no_fail(s->chr_out);
>>>> +
>>>> +    net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize);
>>>> +    net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize);
>>>> +
>>>> +    return;
>>>> +}
>>>> +
>>>> +static void colo_compare_class_init(ObjectClass *oc, void *data)
>>>> +{
>>>> +    UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
>>>> +
>>>> +    ucc->complete = colo_compare_complete;
>>>> +}
>>>> +
>>>> +static void colo_compare_init(Object *obj)
>>>> +{
>>>> +    object_property_add_str(obj, "primary_in",
>>>> +                            compare_get_pri_indev, 
>>>> compare_set_pri_indev,
>>>> +                            NULL);
>>>> +    object_property_add_str(obj, "secondary_in",
>>>> +                            compare_get_sec_indev, 
>>>> compare_set_sec_indev,
>>>> +                            NULL);
>>>> +    object_property_add_str(obj, "outdev",
>>>> +                            compare_get_outdev, compare_set_outdev,
>>>> +                            NULL);
>>>> +}
>>>> +
>>>> +static void colo_compare_finalize(Object *obj)
>>>> +{
>>>> +    CompareState *s = COLO_COMPARE(obj);
>>>> +
>>>> +    if (s->chr_pri_in) {
>>>> +        qemu_chr_add_handlers(s->chr_pri_in, NULL, NULL, NULL, NULL);
>>>> +        qemu_chr_fe_release(s->chr_pri_in);
>>>> +    }
>>>> +    if (s->chr_sec_in) {
>>>> +        qemu_chr_add_handlers(s->chr_sec_in, NULL, NULL, NULL, NULL);
>>>> +        qemu_chr_fe_release(s->chr_sec_in);
>>>> +    }
>>>> +    if (s->chr_out) {
>>>> +        qemu_chr_fe_release(s->chr_out);
>>>> +    }
>>>> +
>>>> +    g_free(s->pri_indev);
>>>> +    g_free(s->sec_indev);
>>>> +    g_free(s->outdev);
>>>> +}
>>>> +
>>>> +static const TypeInfo colo_compare_info = {
>>>> +    .name = TYPE_COLO_COMPARE,
>>>> +    .parent = TYPE_OBJECT,
>>>> +    .instance_size = sizeof(CompareState),
>>>> +    .instance_init = colo_compare_init,
>>>> +    .instance_finalize = colo_compare_finalize,
>>>> +    .class_size = sizeof(CompareClass),
>>>> +    .class_init = colo_compare_class_init,
>>>> +    .interfaces = (InterfaceInfo[]) {
>>>> +        { TYPE_USER_CREATABLE },
>>>> +        { }
>>>> +    }
>>>> +};
>>>> +
>>>> +static void register_types(void)
>>>> +{
>>>> +    type_register_static(&colo_compare_info);
>>>> +}
>>>> +
>>>> +type_init(register_types);
>>>> diff --git a/qemu-options.hx b/qemu-options.hx
>>>> index 587de8f..33d5d0b 100644
>>>> --- a/qemu-options.hx
>>>> +++ b/qemu-options.hx
>>>> @@ -3866,6 +3866,45 @@ Dump the network traffic on netdev @var{dev} 
>>>> to the file specified by
>>>>   The file format is libpcap, so it can be analyzed with tools such 
>>>> as tcpdump
>>>>   or Wireshark.
>>>>   +@item -object 
>>>> colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid},
>>>> +outdev=@var{chardevid}
>>>> +
>>>> +Colo-compare gets packet from primary_in@var{chardevid} and 
>>>> secondary_in@var{chardevid}, than compare primary packet with
>>>> +secondary packet. If the packet same, we will output primary
>>>
>>> s/If the packet same/If the packets are same/.
>>
>> OK.
>>
>>>
>>>> +packet to outdev@var{chardevid}, else we will notify colo-frame
>>>> +do checkpoint and send primary packet to outdev@var{chardevid}.
>>>> +
>>>> +we can use it with the help of filter-mirror and filter-redirector.
>>>
>>> s/we/We/ and looks like colo compare must be used with the help of 
>>> mirror and redirector?
>>
>> Currently yes.
>
> Then please change the doc here.

s/We can use it/We must use it.

Thanks
Zhang Chen

>
> Thanks
>
>>
>>>
>>>> +
>>>> +@example
>>>> +
>>>> +primary:
>>>> +-netdev 
>>>> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown
>>>> +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66
>>>> +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
>>>> +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait
>>>> +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait
>>>> +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001
>>>> +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait
>>>> +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005
>>>> +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0
>>>> +-object 
>>>> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out
>>>> +-object 
>>>> filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0
>>>> +-object 
>>>> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0
>>>> +
>>>> +secondary:
>>>> +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down 
>>>> script=/etc/qemu-ifdown
>>>> +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66
>>>> +-chardev socket,id=red0,host=3.3.3.3,port=9003
>>>> +-chardev socket,id=red1,host=3.3.3.3,port=9004
>>>> +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0
>>>> +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1
>>>> +
>>>> +@end example
>>>> +
>>>> +If you want to know the detail of above command line, you can read
>>>> +the colo-compare git log.
>>>> +
>>>>   @item -object 
>>>> secret,id=@var{id},data=@var{string},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}]
>>>>   @item -object 
>>>> secret,id=@var{id},file=@var{filename},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}]
>>>>   diff --git a/vl.c b/vl.c
>>>> index cbe51ac..c6b9a6f 100644
>>>> --- a/vl.c
>>>> +++ b/vl.c
>>>> @@ -2865,7 +2865,8 @@ static bool object_create_initial(const char 
>>>> *type)
>>>>       if (g_str_equal(type, "filter-buffer") ||
>>>>           g_str_equal(type, "filter-dump") ||
>>>>           g_str_equal(type, "filter-mirror") ||
>>>> -        g_str_equal(type, "filter-redirector")) {
>>>> +        g_str_equal(type, "filter-redirector") ||
>>>> +        g_str_equal(type, "colo-compare")) {
>>>>           return false;
>>>>       }
>>>
>>>
>>>
>>> .
>>>
>>
>
>
>
> .
>

On 08/31/2016 05:39 PM, Zhang Chen wrote:
>
>
> On 08/31/2016 05:20 PM, Jason Wang wrote:
>>
>>
>> On 2016年08月31日 17:03, Zhang Chen wrote:
>>>
>>>
>>> On 08/31/2016 03:53 PM, Jason Wang wrote:
>>>>
>>>>
>>>> On 2016年08月17日 16:10, Zhang Chen wrote:
>>>>> This a COLO net ascii figure:
>>>>>
>>>>>   Primary qemu Secondary qemu
>>>>> +--------------------------------------------------------------+ 
>>>>> +----------------------------------------------------------------+
>>>>> | +----------------------------------------------------------+ 
>>>>> |       | 
>>>>> +-----------------------------------------------------------+ |
>>>>> | | | |       | | | |
>>>>> | |                        guest | |       |  | 
>>>>> guest                              | |
>>>>> | | | |       | | | |
>>>>> | +-------^--------------------------+-----------------------+ 
>>>>> |       | 
>>>>> +---------------------+--------+----------------------------+ |
>>>>> |         |                          | | |                        
>>>>> ^ | |
>>>>> |         |                          | | |                        
>>>>> | | |
>>>>> |         | 
>>>>> +------------------------------------------------------+ 
>>>>> |                        |        | |
>>>>> |netfilter|  |                       | |    |  | 
>>>>> netfilter            | |                              |
>>>>> | +----------+ +----------------------------+ |    |  | 
>>>>> +-----------------------------------------------------------+ |
>>>>> | |       |  |                       |      |        out | |  |  
>>>>> |                     |        |  filter excute order       | |
>>>>> | |       |  |          +-----------------------------+ | | |  
>>>>> |                     |        | +------------------->      | |
>>>>> | |       |  |          |            |      |         | | | |  
>>>>> |                     |        | TCP | |
>>>>> | | +-----+--+-+  +-----v----+ +-----v----+ |pri 
>>>>> +----+----+sec|    |  |  | +------------+ 
>>>>> +---+----+---v+rewriter++  +------------+ | |
>>>>> | | |          |  |          | |          | |in  | |in |    |  |  
>>>>> | |            |  |        |              | |            | | |
>>>>> | | |  filter  |  |  filter  | |  filter  +------> colo <------+ 
>>>>> +-------->  filter   +--> adjust | adjust     +-->   filter   | | |
>>>>> | | |  mirror  |  |redirector| |redirector| |    | compare | |  
>>>>> |    |  | | redirector |  | ack    |   seq        | | redirector | 
>>>>> | |
>>>>> | | |          |  |          | |          | |    | | |  |    |  | 
>>>>> |            |  |        |              | |            | | |
>>>>> | | +----^-----+  +----+-----+ +----------+ | +---------+ |  |    
>>>>> |  | +------------+ +--------+--------------+ +---+--------+ | |
>>>>> | |      |   tx        |   rx           rx  | |  |    | 
>>>>> |            tx                        all       | rx | |
>>>>> | |      |             |                    | |  |    | 
>>>>> +-----------------------------------------------------------+ |
>>>>> | |      |             +--------------+     | |  |    | 
>>>>> |            |
>>>>> | |      |   filter excute order      |     | |  |    | 
>>>>> |            |
>>>>> | |      |  +---------------->        | | | 
>>>>> +--------------------------------------------------------+ |
>>>>> | +-----------------------------------------+ | | |
>>>>> |        |                            | | | |
>>>>> +--------------------------------------------------------------+ 
>>>>> +----------------------------------------------------------------+
>>>>>           |guest receive               | guest send
>>>>>           |                            |
>>>>> +--------+----------------------------v------------------------+
>>>>> | |                          NOTE: filter direction is rx/tx/all
>>>>> |                         tap | rx:receive packets sent to the netdev
>>>>> | |                          tx:receive packets sent by the netdev
>>>>> +--------------------------------------------------------------+
>>>>
>>>> It's better to add a doc under docs to explain this configuration 
>>>> in detail on top of this series.
>>>>
>>>
>>> As you say, Am I add /docs/colo-proxy.txt to explain it or add this 
>>> in hailiang's COLO-FT.txt after merge?
>
> Can you give me a way for doc?
>
>

Sorry, I misunderstand hailiang's mean, I will add /docs/colo-proxy.txt 
in next version.

Thanks
Zhang Chen

>>>
>>>>> In COLO-compare, we do packet comparing job.
>>>>> Packets coming from the primary char indev will be sent to outdev.
>>>>> Packets coming from the secondary char dev will be dropped after 
>>>>> comparing.
>>>>> colo-comapre need two input chardev and one output chardev:
>>>>> primary_in=chardev1-id (source: primary send packet)
>>>>> secondary_in=chardev2-id (source: secondary send packet)
>>>>> outdev=chardev3-id
>>>>>
>>>>> usage:
>>>>>
>>>>> primary:
>>>>> -netdev 
>>>>> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown 
>>>>>
>>>>> -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66
>>>>> -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
>>>>> -chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait
>>>>> -chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait
>>>>> -chardev socket,id=compare0-0,host=3.3.3.3,port=9001
>>>>> -chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait
>>>>> -chardev socket,id=compare_out0,host=3.3.3.3,port=9005
>>>>> -object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0
>>>>> -object 
>>>>> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out
>>>>> -object 
>>>>> filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0
>>>>> -object 
>>>>> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0
>>>>>
>>>>> secondary:
>>>>> -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down 
>>>>> script=/etc/qemu-ifdown
>>>>> -device e1000,netdev=hn0,mac=52:a4:00:12:78:66
>>>>> -chardev socket,id=red0,host=3.3.3.3,port=9003
>>>>> -chardev socket,id=red1,host=3.3.3.3,port=9004
>>>>> -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0
>>>>> -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1
>>>>>
>>>>> Signed-off-by: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
>>>>> Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
>>>>> Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
>>>>> ---
>>>>>   net/Makefile.objs  |   1 +
>>>>>   net/colo-compare.c | 284 
>>>>> +++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>   qemu-options.hx    |  39 ++++++++
>>>>>   vl.c               |   3 +-
>>>>>   4 files changed, 326 insertions(+), 1 deletion(-)
>>>>>   create mode 100644 net/colo-compare.c
>>>>>
>>>>> diff --git a/net/Makefile.objs b/net/Makefile.objs
>>>>> index b7c22fd..ba92f73 100644
>>>>> --- a/net/Makefile.objs
>>>>> +++ b/net/Makefile.objs
>>>>> @@ -16,3 +16,4 @@ common-obj-$(CONFIG_NETMAP) += netmap.o
>>>>>   common-obj-y += filter.o
>>>>>   common-obj-y += filter-buffer.o
>>>>>   common-obj-y += filter-mirror.o
>>>>> +common-obj-y += colo-compare.o
>>>>> diff --git a/net/colo-compare.c b/net/colo-compare.c
>>>>> new file mode 100644
>>>>> index 0000000..cdc3e0e
>>>>> --- /dev/null
>>>>> +++ b/net/colo-compare.c
>>>>> @@ -0,0 +1,284 @@
>>>>> +/*
>>>>> + * COarse-grain LOck-stepping Virtual Machines for Non-stop 
>>>>> Service (COLO)
>>>>> + * (a.k.a. Fault Tolerance or Continuous Replication)
>>>>> + *
>>>>> + * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD.
>>>>> + * Copyright (c) 2016 FUJITSU LIMITED
>>>>> + * Copyright (c) 2016 Intel Corporation
>>>>> + *
>>>>> + * Author: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
>>>>> + *
>>>>> + * This work is licensed under the terms of the GNU GPL, version 
>>>>> 2 or
>>>>> + * later.  See the COPYING file in the top-level directory.
>>>>> + */
>>>>> +
>>>>> +#include "qemu/osdep.h"
>>>>> +#include "qemu/error-report.h"
>>>>> +#include "qemu-common.h"
>>>>> +#include "qapi/qmp/qerror.h"
>>>>> +#include "qapi/error.h"
>>>>> +#include "net/net.h"
>>>>> +#include "net/vhost_net.h"
>>>>
>>>> Looks unnecessary.
>>>
>>> I will remove it.
>>>
>>>>
>>>>> +#include "qom/object_interfaces.h"
>>>>> +#include "qemu/iov.h"
>>>>> +#include "qom/object.h"
>>>>> +#include "qemu/typedefs.h"
>>>>> +#include "net/queue.h"
>>>>> +#include "sysemu/char.h"
>>>>> +#include "qemu/sockets.h"
>>>>> +#include "qapi-visit.h"
>>>>> +
>>>>> +#define TYPE_COLO_COMPARE "colo-compare"
>>>>> +#define COLO_COMPARE(obj) \
>>>>> +    OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE)
>>>>> +
>>>>> +#define COMPARE_READ_LEN_MAX NET_BUFSIZE
>>>>> +
>>>>> +typedef struct CompareState {
>>>>> +    Object parent;
>>>>> +
>>>>> +    char *pri_indev;
>>>>> +    char *sec_indev;
>>>>> +    char *outdev;
>>>>> +    CharDriverState *chr_pri_in;
>>>>> +    CharDriverState *chr_sec_in;
>>>>> +    CharDriverState *chr_out;
>>>>> +    QTAILQ_ENTRY(CompareState) next;
>>>>
>>>> This looks not used in this series but in commit "colo-compare and 
>>>> filter-rewriter work with colo-frame". We'd better delay the 
>>>> introducing to that patch.
>>>
>>> OK~ I got your point.
>>>
>>>>
>>>>> +    SocketReadState pri_rs;
>>>>> +    SocketReadState sec_rs;
>>>>> +} CompareState;
>>>>> +
>>>>> +typedef struct CompareClass {
>>>>> +    ObjectClass parent_class;
>>>>> +} CompareClass;
>>>>> +
>>>>> +typedef struct CompareChardevProps {
>>>>> +    bool is_socket;
>>>>> +    bool is_unix;
>>>>> +} CompareChardevProps;
>>>>> +
>>>>> +static char *compare_get_pri_indev(Object *obj, Error **errp)
>>>>> +{
>>>>> +    CompareState *s = COLO_COMPARE(obj);
>>>>> +
>>>>> +    return g_strdup(s->pri_indev);
>>>>> +}
>>>>> +
>>>>> +static void compare_set_pri_indev(Object *obj, const char *value, 
>>>>> Error **errp)
>>>>> +{
>>>>> +    CompareState *s = COLO_COMPARE(obj);
>>>>> +
>>>>> +    g_free(s->pri_indev);
>>>>> +    s->pri_indev = g_strdup(value);
>>>>> +}
>>>>> +
>>>>> +static char *compare_get_sec_indev(Object *obj, Error **errp)
>>>>> +{
>>>>> +    CompareState *s = COLO_COMPARE(obj);
>>>>> +
>>>>> +    return g_strdup(s->sec_indev);
>>>>> +}
>>>>> +
>>>>> +static void compare_set_sec_indev(Object *obj, const char *value, 
>>>>> Error **errp)
>>>>> +{
>>>>> +    CompareState *s = COLO_COMPARE(obj);
>>>>> +
>>>>> +    g_free(s->sec_indev);
>>>>> +    s->sec_indev = g_strdup(value);
>>>>> +}
>>>>> +
>>>>> +static char *compare_get_outdev(Object *obj, Error **errp)
>>>>> +{
>>>>> +    CompareState *s = COLO_COMPARE(obj);
>>>>> +
>>>>> +    return g_strdup(s->outdev);
>>>>> +}
>>>>> +
>>>>> +static void compare_set_outdev(Object *obj, const char *value, 
>>>>> Error **errp)
>>>>> +{
>>>>> +    CompareState *s = COLO_COMPARE(obj);
>>>>> +
>>>>> +    g_free(s->outdev);
>>>>> +    s->outdev = g_strdup(value);
>>>>> +}
>>>>> +
>>>>> +static void compare_pri_rs_finalize(SocketReadState *pri_rs)
>>>>> +{
>>>>> +    /* if packet_enqueue pri pkt failed we will send unsupported 
>>>>> packet */
>>>>> +}
>>>>> +
>>>>> +static void compare_sec_rs_finalize(SocketReadState *sec_rs)
>>>>> +{
>>>>> +    /* if packet_enqueue sec pkt failed we will notify trace */
>>>>> +}
>>>>> +
>>>>> +static int compare_chardev_opts(void *opaque,
>>>>> +                                const char *name, const char *value,
>>>>> +                                Error **errp)
>>>>> +{
>>>>> +    CompareChardevProps *props = opaque;
>>>>> +
>>>>> +    if (strcmp(name, "backend") == 0 && strcmp(value, "socket") 
>>>>> == 0) {
>>>>> +        props->is_socket = true;
>>>>> +    } else if (strcmp(name, "host") == 0) {
>>>>
>>>> Typo? net_vhost_chardev_opts() did:
>>>>
>>>>     } else if (strcmp(name, "path") == 0) {
>>>>         props->is_unix = true;
>>>>     }
>>>>
>>>>
>>>
>>> No, In colo-compare we use chardev like this:
>>>
>>> -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
>>>
>>> If we only use "path" here will trigger a error.
>>> Should I add anthor "path" here?
>>
>> If I understand the code correctly, "is_unix" means "is unix domain 
>> socket"? If yes, according to the help:
>>
>> -chardev 
>> socket,id=id[,host=host],port=port[,to=to][,ipv4][,ipv6][,nodelay][,reconnect=seconds]
>> [,server][,nowait][,telnet][,reconnect=seconds][,mux=on|off]
>>          [,logfile=PATH][,logappend=on|off][,tls-creds=ID] (tcp)
>> -chardev 
>> socket,id=id,path=path[,server][,nowait][,telnet][,reconnect=seconds]
>>          [,mux=on|off][,logfile=PATH][,logappend=on|off] (unix)
>>
>> "host" will not be used for UNIX domain socket.
>>
>> And if UNIX domain socket is not supported, there's probably no need 
>> to differentiate it from other types.
>
> OK, I will remove the "is_unix" in next version.
>
>
>>
>>>
>>>
>>>>
>>>>> +        props->is_unix = true;
>>>>> +    } else if (strcmp(name, "port") == 0) {
>>>>> +    } else if (strcmp(name, "server") == 0) {
>>>>> +    } else if (strcmp(name, "wait") == 0) {
>>>>> +    } else {
>>>>> +        error_setg(errp,
>>>>> +                   "COLO-compare does not support a chardev with 
>>>>> option %s=%s",
>>>>> +                   name, value);
>>>>> +        return -1;
>>>>> +    }
>>>>> +    return 0;
>>>>> +}
>>>>> +
>>>>> +/*
>>>>> + * called from the main thread on the primary
>>>>> + * to setup colo-compare.
>>>>> + */
>>>>> +static void colo_compare_complete(UserCreatable *uc, Error **errp)
>>>>> +{
>>>>> +    CompareState *s = COLO_COMPARE(uc);
>>>>> +    CompareChardevProps props;
>>>>> +
>>>>> +    if (!s->pri_indev || !s->sec_indev || !s->outdev) {
>>>>> +        error_setg(errp, "colo compare needs 'primary_in' ,"
>>>>> +                   "'secondary_in','outdev' property set");
>>>>> +        return;
>>>>> +    } else if (!strcmp(s->pri_indev, s->outdev) ||
>>>>> +               !strcmp(s->sec_indev, s->outdev) ||
>>>>> +               !strcmp(s->pri_indev, s->sec_indev)) {
>>>>> +        error_setg(errp, "'indev' and 'outdev' could not be same "
>>>>> +                   "for compare module");
>>>>> +        return;
>>>>> +    }
>>>>> +
>>>>> +    s->chr_pri_in = qemu_chr_find(s->pri_indev);
>>>>> +    if (s->chr_pri_in == NULL) {
>>>>> +        error_setg(errp, "Primary IN Device '%s' not found",
>>>>> +                   s->pri_indev);
>>>>> +        return;
>>>>> +    }
>>>>> +
>>>>> +    /* inspect chardev opts */
>>>>> +    memset(&props, 0, sizeof(props));
>>>>> +    if (qemu_opt_foreach(s->chr_pri_in->opts, 
>>>>> compare_chardev_opts, &props, errp)) {
>>>>> +        return;
>>>>> +    }
>>>>> +
>>>>> +    if (!props.is_socket || !props.is_unix) {
>>>>> +        error_setg(errp, "chardev \"%s\" is not a unix socket",
>>>>> +                   s->pri_indev);
>>>>> +        return;
>>>>> +    }
>>>>> +
>>>>> +    s->chr_sec_in = qemu_chr_find(s->sec_indev);
>>>>> +    if (s->chr_sec_in == NULL) {
>>>>> +        error_setg(errp, "Secondary IN Device '%s' not found",
>>>>> +                   s->sec_indev);
>>>>> +        return;
>>>>> +    }
>>>>> +
>>>>> +    memset(&props, 0, sizeof(props));
>>>>> +    if (qemu_opt_foreach(s->chr_sec_in->opts, 
>>>>> compare_chardev_opts, &props, errp)) {
>>>>> +        return;
>>>>> +    }
>>>>> +
>>>>> +    if (!props.is_socket || !props.is_unix) {
>>>>> +        error_setg(errp, "chardev \"%s\" is not a unix socket",
>>>>> +                   s->sec_indev);
>>>>
>>>> I believe tcp socket is also supported?
>>>
>>> If I understand correctly, "tcp socket" in here is the "-chardev 
>>> socket".
>>> I will rename "unix socket" to "tcp socket".
>>>
>>>>
>>>>> +        return;
>>>>> +    }
>>>>> +
>>>>> +    s->chr_out = qemu_chr_find(s->outdev);
>>>>> +    if (s->chr_out == NULL) {
>>>>> +        error_setg(errp, "OUT Device '%s' not found", s->outdev);
>>>>> +        return;
>>>>> +    }
>>>>> +
>>>>> +    memset(&props, 0, sizeof(props));
>>>>> +    if (qemu_opt_foreach(s->chr_out->opts, compare_chardev_opts, 
>>>>> &props, errp)) {
>>>>> +        return;
>>>>> +    }
>>>>> +
>>>>> +    if (!props.is_socket || !props.is_unix) {
>>>>> +        error_setg(errp, "chardev \"%s\" is not a unix socket",
>>>>> +                   s->outdev);
>>>>
>>>> Ditto, and there's code duplication, please introduce a helper to 
>>>> do above.
>>>
>>> I don't understand what the "helper"?
>>> In here we check each chardev, will I change to "goto error;" ?
>>
>> A helper to avoid the code duplication for socket type inspection for 
>> pri_in,scr_in and chr_out.
>
> I got it~~
> I will add it in next version.
>
>
>>
>>>
>>>>
>>>>> +        return;
>>>>> +    }
>>>>> +
>>>>> +    qemu_chr_fe_claim_no_fail(s->chr_pri_in);
>>>>> +
>>>>> +    qemu_chr_fe_claim_no_fail(s->chr_sec_in);
>>>>> +
>>>>> +    qemu_chr_fe_claim_no_fail(s->chr_out);
>>>>> +
>>>>> +    net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize);
>>>>> +    net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize);
>>>>> +
>>>>> +    return;
>>>>> +}
>>>>> +
>>>>> +static void colo_compare_class_init(ObjectClass *oc, void *data)
>>>>> +{
>>>>> +    UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
>>>>> +
>>>>> +    ucc->complete = colo_compare_complete;
>>>>> +}
>>>>> +
>>>>> +static void colo_compare_init(Object *obj)
>>>>> +{
>>>>> +    object_property_add_str(obj, "primary_in",
>>>>> +                            compare_get_pri_indev, 
>>>>> compare_set_pri_indev,
>>>>> +                            NULL);
>>>>> +    object_property_add_str(obj, "secondary_in",
>>>>> +                            compare_get_sec_indev, 
>>>>> compare_set_sec_indev,
>>>>> +                            NULL);
>>>>> +    object_property_add_str(obj, "outdev",
>>>>> +                            compare_get_outdev, compare_set_outdev,
>>>>> +                            NULL);
>>>>> +}
>>>>> +
>>>>> +static void colo_compare_finalize(Object *obj)
>>>>> +{
>>>>> +    CompareState *s = COLO_COMPARE(obj);
>>>>> +
>>>>> +    if (s->chr_pri_in) {
>>>>> +        qemu_chr_add_handlers(s->chr_pri_in, NULL, NULL, NULL, 
>>>>> NULL);
>>>>> +        qemu_chr_fe_release(s->chr_pri_in);
>>>>> +    }
>>>>> +    if (s->chr_sec_in) {
>>>>> +        qemu_chr_add_handlers(s->chr_sec_in, NULL, NULL, NULL, 
>>>>> NULL);
>>>>> +        qemu_chr_fe_release(s->chr_sec_in);
>>>>> +    }
>>>>> +    if (s->chr_out) {
>>>>> +        qemu_chr_fe_release(s->chr_out);
>>>>> +    }
>>>>> +
>>>>> +    g_free(s->pri_indev);
>>>>> +    g_free(s->sec_indev);
>>>>> +    g_free(s->outdev);
>>>>> +}
>>>>> +
>>>>> +static const TypeInfo colo_compare_info = {
>>>>> +    .name = TYPE_COLO_COMPARE,
>>>>> +    .parent = TYPE_OBJECT,
>>>>> +    .instance_size = sizeof(CompareState),
>>>>> +    .instance_init = colo_compare_init,
>>>>> +    .instance_finalize = colo_compare_finalize,
>>>>> +    .class_size = sizeof(CompareClass),
>>>>> +    .class_init = colo_compare_class_init,
>>>>> +    .interfaces = (InterfaceInfo[]) {
>>>>> +        { TYPE_USER_CREATABLE },
>>>>> +        { }
>>>>> +    }
>>>>> +};
>>>>> +
>>>>> +static void register_types(void)
>>>>> +{
>>>>> +    type_register_static(&colo_compare_info);
>>>>> +}
>>>>> +
>>>>> +type_init(register_types);
>>>>> diff --git a/qemu-options.hx b/qemu-options.hx
>>>>> index 587de8f..33d5d0b 100644
>>>>> --- a/qemu-options.hx
>>>>> +++ b/qemu-options.hx
>>>>> @@ -3866,6 +3866,45 @@ Dump the network traffic on netdev 
>>>>> @var{dev} to the file specified by
>>>>>   The file format is libpcap, so it can be analyzed with tools 
>>>>> such as tcpdump
>>>>>   or Wireshark.
>>>>>   +@item -object 
>>>>> colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid},
>>>>> +outdev=@var{chardevid}
>>>>> +
>>>>> +Colo-compare gets packet from primary_in@var{chardevid} and 
>>>>> secondary_in@var{chardevid}, than compare primary packet with
>>>>> +secondary packet. If the packet same, we will output primary
>>>>
>>>> s/If the packet same/If the packets are same/.
>>>
>>> OK.
>>>
>>>>
>>>>> +packet to outdev@var{chardevid}, else we will notify colo-frame
>>>>> +do checkpoint and send primary packet to outdev@var{chardevid}.
>>>>> +
>>>>> +we can use it with the help of filter-mirror and filter-redirector.
>>>>
>>>> s/we/We/ and looks like colo compare must be used with the help of 
>>>> mirror and redirector?
>>>
>>> Currently yes.
>>
>> Then please change the doc here.
>
> s/We can use it/We must use it.
>
> Thanks
> Zhang Chen
>
>>
>> Thanks
>>
>>>
>>>>
>>>>> +
>>>>> +@example
>>>>> +
>>>>> +primary:
>>>>> +-netdev 
>>>>> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown 
>>>>>
>>>>> +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66
>>>>> +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait
>>>>> +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait
>>>>> +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait
>>>>> +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001
>>>>> +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait
>>>>> +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005
>>>>> +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0
>>>>> +-object 
>>>>> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out
>>>>> +-object 
>>>>> filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0
>>>>> +-object 
>>>>> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0
>>>>> +
>>>>> +secondary:
>>>>> +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down 
>>>>> script=/etc/qemu-ifdown
>>>>> +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66
>>>>> +-chardev socket,id=red0,host=3.3.3.3,port=9003
>>>>> +-chardev socket,id=red1,host=3.3.3.3,port=9004
>>>>> +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0
>>>>> +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1
>>>>> +
>>>>> +@end example
>>>>> +
>>>>> +If you want to know the detail of above command line, you can read
>>>>> +the colo-compare git log.
>>>>> +
>>>>>   @item -object 
>>>>> secret,id=@var{id},data=@var{string},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}]
>>>>>   @item -object 
>>>>> secret,id=@var{id},file=@var{filename},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}]
>>>>>   diff --git a/vl.c b/vl.c
>>>>> index cbe51ac..c6b9a6f 100644
>>>>> --- a/vl.c
>>>>> +++ b/vl.c
>>>>> @@ -2865,7 +2865,8 @@ static bool object_create_initial(const char 
>>>>> *type)
>>>>>       if (g_str_equal(type, "filter-buffer") ||
>>>>>           g_str_equal(type, "filter-dump") ||
>>>>>           g_str_equal(type, "filter-mirror") ||
>>>>> -        g_str_equal(type, "filter-redirector")) {
>>>>> +        g_str_equal(type, "filter-redirector") ||
>>>>> +        g_str_equal(type, "colo-compare")) {
>>>>>           return false;
>>>>>       }
>>>>
>>>>
>>>>
>>>> .
>>>>
>>>
>>
>>
>>
>> .
>>
>