From patchwork Fri Feb 19 04:20:52 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Zhou X-Patchwork-Id: 585021 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from archives.nicira.com (li376-54.members.linode.com [96.126.127.54]) by ozlabs.org (Postfix) with ESMTP id 7645E1402A0 for ; Fri, 19 Feb 2016 15:21:27 +1100 (AEDT) Received: from archives.nicira.com (localhost [127.0.0.1]) by archives.nicira.com (Postfix) with ESMTP id 6ACB210B1F; Thu, 18 Feb 2016 20:21:13 -0800 (PST) X-Original-To: dev@openvswitch.com Delivered-To: dev@openvswitch.com Received: from mail-pa0-f41.google.com (mail-pa0-f41.google.com [209.85.220.41]) by archives.nicira.com (Postfix) with ESMTPS id 438E010B15 for ; Thu, 18 Feb 2016 20:21:12 -0800 (PST) Received: by mail-pa0-f41.google.com with SMTP id fl4so43212817pad.0 for ; Thu, 18 Feb 2016 20:21:12 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=C6HaMX4CkG4y3hPdw4f49TceLIqmoeEvh3ExziYbn08=; b=atfSjpnktTE4gNTMjVMEMENExxcBKH+CGRmLmTRiKQjRWYn+4tIeuqLMECCEnePWSj Q08BqEAMA3/Uy/EfynetKZD81CU+OwiyLAsc1E/bLiDSIPOwi9Dz6tcmsjdM/tdz/Ye0 OQBNf9+/E/zMsl0qT87Avud4IB7aIOGdViHLIMd02ZeKogqm73uJ8QUnFE1RVQz2GI5q tg1+hzHDzzbVkGVX+CYC57bLiJISV+W9obSey5YDGGqQN1KgLKVsy+TRi7D+ooWJPxbz JQ/BN9cmsP4rp1vU6dYLCcENymxjPq4pS7+Stoae+czRkqDhpe4E8+X2AeaR0sxrB5up EG0A== X-Gm-Message-State: AG10YOSVjHNLTVjlS0wQc7JuTcO6BlbVex4SAMtFi1Ee3XcypPElry0GeowRxwlb0h8vZQ== X-Received: by 10.66.152.231 with SMTP id vb7mr15357994pab.132.1455855671855; Thu, 18 Feb 2016 20:21:11 -0800 (PST) Received: from ubuntu.localdomain ([208.91.1.34]) by smtp.gmail.com with ESMTPSA id yl1sm13745228pac.35.2016.02.18.20.21.10 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 18 Feb 2016 20:21:10 -0800 (PST) From: Andy Zhou To: dev@openvswitch.com Date: Thu, 18 Feb 2016 20:20:52 -0800 Message-Id: <1455855656-112840-3-git-send-email-azhou@ovn.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1455855656-112840-1-git-send-email-azhou@ovn.org> References: <1455855656-112840-1-git-send-email-azhou@ovn.org> Subject: [ovs-dev] [poll group RFC 2/6] lib: Introduce poll group provider class X-BeenThere: dev@openvswitch.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: dev-bounces@openvswitch.org Sender: "dev" Poll group is a new poll class that sits between application and the stream class. Poll group compliments the poll loop facility and the stream class to make main loop more efficient when dealing with large number of current connections. See comments in poll-group-provider.h for more details. Signed-off-by: Andy Zhou --- lib/automake.mk | 3 + lib/poll-group-provider.h | 153 ++++++++++++++++++++++++++++++++ lib/poll-group.c | 221 ++++++++++++++++++++++++++++++++++++++++++++++ lib/poll-group.h | 51 +++++++++++ 4 files changed, 428 insertions(+) create mode 100644 lib/poll-group-provider.h create mode 100644 lib/poll-group.c create mode 100644 lib/poll-group.h diff --git a/lib/automake.mk b/lib/automake.mk index 27a1669..dfda1bf 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -197,6 +197,9 @@ lib_libopenvswitch_la_SOURCES = \ lib/perf-counter.c \ lib/pktbuf.c \ lib/pktbuf.h \ + lib/poll-group.c \ + lib/poll-group.h \ + lib/poll-group-provider.h \ lib/poll-loop.c \ lib/poll-loop.h \ lib/process.c \ diff --git a/lib/poll-group-provider.h b/lib/poll-group-provider.h new file mode 100644 index 0000000..0875670 --- /dev/null +++ b/lib/poll-group-provider.h @@ -0,0 +1,153 @@ +/* + * Copyright (c) 2016 Nicira, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef POLL_GROUP_PROVIDER_H +#define POLL_GROUP_PROVIDER_H 1 + +#include +#include +#include "util.h" + +struct poll_group; +/* + * Poll group: An application aware poll interface + * + * Poll group implements a higher level interface than those offered by + * poll loop and aims to improve efficiency with high number concurrent + * connections. + * + * Poll group sits between the application and the stream layer. Same kind + * of application sessions can ask their individual streams to join + * a common poll group, which is created by the application. + * In addition, the application session is required to provide an + * unique pointer that is only associated with the session, called + * 'caller_event' when joining a poll group. Each poll group then + * in term registers with the main poll loop. + * + * When poll_block() wakes up, each poll group delivers only a set of + * 'caller_event's that correspond to the sessions that are responsible + * for poll_block waken up. Thus, the main loop can avoid interrogate all + * sessions which is required when using poll loop directly. + * + * On Linux, Poll group maps well into the epoll(2) facility. Each + * poll group maps directly to an epoll_fd that is created + * with epoll_create() system call. epoll implementation can provide + * additional performance benefits by waking up the main loop less + * frequently. + * + * Poll group is designed to be closely integrated with the stream class. + * An newly created application session can join a poll group at any time + * during its life cycle. Internally, a stream only turns on 'poll group' + * when a stream is fully connected. Stream connection set up and tear down + * is still managed by the poll loop, but those mode switches are managed + * internally with the stream class, transparent to the application. + * + * An application session can always leave poll group at any time, or + * simply never join one. In both cases, the stream will fall back to + * use poll loop directly. + */ + +struct poll_group_class { + /* Prefix of a poll group name, e.g. "default", "epoll" . */ + const char *name; + + /* The following APIs are required to be implemented in any + * class. + */ + + /* Create a new poll group. */ + struct poll_group *(*create)(const char *name); + + /* Shut down a poll group, release memory and other resources. */ + void (*close)(struct poll_group *group); + + /* 'join' allows a new 'fd' to join a poll group. 'caller_event' are user + * provided pointer that is opaque to poll group. Those pointers + * are passed back to the user when poll_group_get_events() are + * called. */ + int (*join)(struct poll_group *group, int fd, void *caller_event); + + /* 'updates' allows application to inform whether the 'fd' is + * interested to be waken up when 'fd' is ready for transmission. + */ + int (*update)(struct poll_group *group, int fd, bool write, + void *caller_event); + + /* 'leave' is the opposite of 'join'. 'fd' will no longer be watched + * by poll group. */ + int (*leave)(struct poll_group *group, int fd); + + /* 'poll_wait' is poll group's interface to poll loop, so that the + * poll group itself can be waken up by poll loop. */ + void (*poll_wait)(struct poll_group *group); + + /* The following APIs are optional and can be NULL. */ + + /* 'get_events', if implemented is used for an implementation to fill + * up the poll_group's 'events' buffer, using poll_group_notify(). + * It will be invoked when poll_group_get_events() is called. */ + void (*get_events)(struct poll_group *group); +}; + +static inline bool +poll_group_class_is_valid(const struct poll_group_class *class) +{ + return (class->name && class->create && class->close + && class->join && class->update && class->leave + && class->poll_wait); +} + +/* poll_group. + * + * This structure should be treated as opaque by implementation. */ +struct poll_group { + const struct poll_group_class *class; + char *name; /* Name of the poll group, useful for logging. */ + size_t n_joined; /* Total number of joined streams. */ + + + /* Data storage provide for implementing poll_group_get_events(). + * The size of **events buffer will be managed such that its size can + * accommodate at least up to 'n_events' of void pointer array. + * + * The events are filled by 'poll_group_notify()', in sequential order. + * 'n_events' are used to track the number of events added by + * poll_group_notify(). + * + * 'poll_group_get_events()' allows caller to retrieve the 'events' + * array and 'n_evnets' stored. However, this call in one short; it + * resets 'n_events'. The next poll_group_notify() call will start + * to add 'caller_event' into an empty array. + */ + void **events; /* Events buffer array, stores 'call_events' + pointers user provided to poll_group_join() */ + size_t n_events; /* Number of events received from + poll_group_notify() so far, but has not been + retrieved by poll_group_get_events() yet. */ + size_t n_allocated; /* Used internally for x2nrealloc(). */ +}; + +static inline void +poll_group_assert_class(const struct poll_group *group, + const struct poll_group_class *class) +{ + ovs_assert(group->class == class); +} + +void poll_group_init(struct poll_group *, const char *name, + const struct poll_group_class *); + +#endif /* poll-group-provider.h */ diff --git a/lib/poll-group.c b/lib/poll-group.c new file mode 100644 index 0000000..547867d --- /dev/null +++ b/lib/poll-group.c @@ -0,0 +1,221 @@ +/* + * Copyright (c) 2016 Nicira, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include +#include +#include +#include +#include +#include +#include "dynamic-string.h" +#include "poll-loop.h" +#include "util.h" +#include "openvswitch/vlog.h" +#include "poll-group.h" +#include "poll-group-provider.h" + +VLOG_DEFINE_THIS_MODULE(poll_group); + +static void add_caller_event(struct poll_group *group, void *caller_event); + +static const struct poll_group_class * +poll_group_select_class(const char *class_name OVS_UNUSED, size_t len OVS_UNUSED) +{ + /* Nothing has being impemented yet. */ + return NULL; +} + +static int +poll_group_is_valid(struct poll_group *group) +{ + const struct poll_group_class *class; + if (!group) { + return PG_ERROR_NO_GROUP; + } + + class = group->class; + if (!class) { + return PG_ERROR_NO_CLASS; + } + + if (!poll_group_class_is_valid(class)) { + return PG_ERROR_INVALID_CLASS; + } + + return 0; +} + + +const char * +poll_group_get_name(const struct poll_group *group) +{ + return group->name; +} + +void +poll_group_init(struct poll_group *group, + const char *name, + const struct poll_group_class *class) +{ + group->name = xstrdup(name); + group->class = class; + group->n_joined = 0; + + group->events = NULL; + group->n_events = 0; + group->n_allocated = 0; +} + +/* poll_group name is specified in tow parts ":", + * The class is used by 'poll_group_select_class' to select + * poll group implementation. 'name' is the name of the poll group + * mostly for logging purpose. */ +struct poll_group * +poll_group_create(const char *name) +{ + const struct poll_group_class *class; + size_t prefix_len; + struct poll_group *group = NULL; + + prefix_len = strcspn(name, ":"); + class = poll_group_select_class(name, prefix_len); + + if (poll_group_class_is_valid(class)) { + const char *group_name; + if (name[prefix_len] == '\0') { + group_name = name; + } else { + group_name = &name[prefix_len + 1]; + } + group = class->create(group_name); + } + + return group; +} + +void +poll_group_close(struct poll_group *group) +{ + int ret = poll_group_is_valid(group); + + if (ret) { + group->class->close(group); + group->n_allocated = 0; + group->n_allocated = 0; + group->n_joined = 0; + + free(group->events); + group->events = NULL; + + free(group->name); + group->name = NULL; + } + + free(group); +} + +int +poll_group_join(struct poll_group *group, int fd, void *caller_event) +{ + int ret = poll_group_is_valid(group); + + if (!ret) { + ret = group->class->join(group, fd, caller_event); + if (!ret) { + group->n_joined++; + } + } + + return ret; +} + +int +poll_group_update(struct poll_group *group, int fd, bool write, + void *caller_event) +{ + int ret = poll_group_is_valid(group); + + if (!ret) { + ret = group->class->update(group, fd, write, caller_event); + } + + return ret; +} + +int +poll_group_leave(struct poll_group *group, int fd) +{ + int ret = poll_group_is_valid(group); + + if (!ret) { + ret = group->class->leave(group, fd); + if (!ret) { + group->n_joined--; + } + } + + return ret; +} + +void +poll_group_poll_wait(struct poll_group *group) +{ + int ret = poll_group_is_valid(group); + + if (!ret) { + group->n_events = 0; + group->class->poll_wait(group); + } +} + +void +poll_group_notify(struct poll_group *group, void *caller_event) +{ + if (group && caller_event) { + add_caller_event(group, caller_event); + } +} + +void +poll_group_get_events(struct poll_group *group, void ***caller_event, + size_t *n) +{ + if (group && group->class->get_events) { + group->class->get_events(group); + } + *n = group->n_events; + *caller_event = group->events; +} + + +static void +add_caller_event(struct poll_group *group, void *caller_event) +{ + size_t n_events = group->n_events; + size_t n_allocated = group->n_allocated; + + ovs_assert(n_events <= group->n_joined); + + if (n_events >= n_allocated) { + void **events = group->events; + events = x2nrealloc(events, &group->n_allocated, sizeof events[0]); + group->events = events; + } + + group->events[group->n_events++] = caller_event; + +} diff --git a/lib/poll-group.h b/lib/poll-group.h new file mode 100644 index 0000000..e03cde7 --- /dev/null +++ b/lib/poll-group.h @@ -0,0 +1,51 @@ +/* + * Copyright (c) 2016 Nicira, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef POLL_GROUP_H +#define POLL_GROUP_H 1 + +#include +#include +#include +#include +#include "openvswitch/types.h" + +struct poll_group; + +enum { + PG_ERROR_NO_ERROR, /* Success */ + PG_ERROR_NO_GROUP, + PG_ERROR_NO_CLASS, + PG_ERROR_INVALID_CLASS, + PG_ERROR_DUPLICATED_FD, + PG_ERROR_FD_NOT_FOUND, +}; + +const char *poll_group_get_name(const struct poll_group *); +struct poll_group *poll_group_create(const char *name); +void poll_group_close(struct poll_group *group); + +int poll_group_join(struct poll_group *group, int fd, void *caller_event); +int poll_group_update(struct poll_group *group, int fd, bool write, + void *caller_event); +int poll_group_leave(struct poll_group *group, int fd); + +void poll_group_poll_wait(struct poll_group *group); + +void poll_group_notify(struct poll_group *, void *caller_event); +void poll_group_get_events(struct poll_group *, void ***caller_event, size_t *n); + +#endif /* poll-group.h */