From patchwork Mon Mar 19 08:24:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guoshuai Li X-Patchwork-Id: 887604 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=dtdream.com Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 404Tdj0RfKz9sWJ for ; Mon, 19 Mar 2018 19:25:12 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 5D669DCF; Mon, 19 Mar 2018 08:25:09 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 4F551DCF for ; Mon, 19 Mar 2018 08:25:07 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from smtp2203-239.mail.aliyun.com (smtp2203-239.mail.aliyun.com [121.197.203.239]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 194E02F5 for ; Mon, 19 Mar 2018 08:25:03 +0000 (UTC) X-Alimail-AntiSpam: AC=CONTINUE; BC=0.07444197|-1; CH=green; FP=0|0|0|0|0|-1|-1|-1; HT=e02c03291; MF=ligs@dtdream.com; NM=1; PH=DS; RN=2; RT=2; SR=0; TI=SMTPD_---.BMdiKmF_1521447895; Received: from localhost.localdomain(mailfrom:ligs@dtdream.com fp:222.128.6.212) by smtp.aliyun-inc.com(10.147.41.121); Mon, 19 Mar 2018 16:24:57 +0800 From: Guoshuai Li To: ovs-dev@openvswitch.org Date: Mon, 19 Mar 2018 16:24:41 +0800 Message-Id: <20180319082441.11120-1-ligs@dtdream.com> X-Mailer: git-send-email 2.13.2.windows.1 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v2] ovn-controller: support MAC_Binding aging X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Add the MAC_Binding aging. The default aging time is 20 minutes. Send the ARP request at 10(1*20/2), 13(2*20/3), 15(3*20/4) minutes. If no ARP reply is received within 20 minutes, the MAC_Binding column will be deleted. Sync a MAC_Binding cache in the ovn-controller where lport redirect chassis, to records timestamps and ARP send times. Time traversal cache to send ARP requests or aging. Signed-off-by: Guoshuai Li --- v2: Reconstruction and Fix code for check chassis. --- ovn/controller/pinctrl.c | 353 +++++++++++++++++++++++++++++++++++++++++++++-- tests/ovn.at | 70 ++++++++++ 2 files changed, 410 insertions(+), 13 deletions(-) diff --git a/ovn/controller/pinctrl.c b/ovn/controller/pinctrl.c index e8bf1c698..1e5ad4d94 100644 --- a/ovn/controller/pinctrl.c +++ b/ovn/controller/pinctrl.c @@ -51,6 +51,7 @@ #include "timeval.h" #include "vswitch-idl.h" #include "lflow.h" +#include "unixctl.h" VLOG_DEFINE_THIS_MODULE(pinctrl); @@ -70,6 +71,25 @@ static void run_put_mac_bindings(struct controller_ctx *); static void wait_put_mac_bindings(struct controller_ctx *); static void flush_put_mac_bindings(void); +static void init_aging_mac_bindings(void); +static void destroy_aging_mac_bindings(void); +static void aging_mac_bindings_wait(void); +static void aging_mac_bindings_run(struct controller_ctx *, + const struct ovsrec_bridge *, + const struct sbrec_chassis *, + const struct chassis_index *, + struct hmap *, + struct sset *); +static void init_aging_mac_binding(uint32_t, uint32_t, const char *); +static void list_aging_mac_binding_cache(struct unixctl_conn *conn, + int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, + void *arg OVS_UNUSED); +static void set_aging_mac_binding_time(struct unixctl_conn *conn, + int argc OVS_UNUSED, + const char *argv[], + void *arg OVS_UNUSED); + static void init_send_garps(void); static void destroy_send_garps(void); static void send_garp_wait(void); @@ -107,6 +127,7 @@ pinctrl_init(void) init_put_mac_bindings(); init_send_garps(); init_ipv6_ras(); + init_aging_mac_bindings(); } static ovs_be32 @@ -1166,6 +1187,8 @@ pinctrl_run(struct controller_ctx *ctx, send_garp_run(ctx, br_int, chassis, chassis_index, local_datapaths, active_tunnels); send_ipv6_ras(ctx, local_datapaths); + aging_mac_bindings_run(ctx, br_int, chassis, chassis_index, + local_datapaths, active_tunnels); } /* Table of ipv6_ra_state structures, keyed on logical port name */ @@ -1467,6 +1490,7 @@ pinctrl_wait(struct controller_ctx *ctx) rconn_recv_wait(swconn); send_garp_wait(); ipv6_ra_wait(); + aging_mac_bindings_wait(); } void @@ -1476,6 +1500,7 @@ pinctrl_destroy(void) destroy_put_mac_bindings(); destroy_send_garps(); destroy_ipv6_ras(); + destroy_aging_mac_bindings(); } /* Implementation of the "put_arp" and "put_nd" OVN actions. These @@ -1567,6 +1592,9 @@ pinctrl_handle_put_mac_binding(const struct flow *md, } pmb->timestamp = time_msec(); pmb->mac = headers->dl_src; + + /* init aging mac_binding timestamp and arp_send_count */ + init_aging_mac_binding(dp_key, port_key, ip_s); } static void @@ -1776,30 +1804,27 @@ send_garp_delete(const char *lport) free(garp); } -static long long int -send_garp(struct garp_data *garp, long long int current_time) +static void +send_arp_request(const struct eth_addr arp_sha, ovs_be32 arp_spa, + ovs_be32 arp_tpa, int ofport, int tag) { - if (current_time < garp->announce_time) { - return garp->announce_time; - } - - /* Compose a GARP request packet. */ + /* Compose a ARP request packet. */ uint64_t packet_stub[128 / 8]; struct dp_packet packet; dp_packet_use_stub(&packet, packet_stub, sizeof packet_stub); - compose_arp(&packet, ARP_OP_REQUEST, garp->ea, eth_addr_zero, - true, garp->ipv4, garp->ipv4); + compose_arp(&packet, ARP_OP_REQUEST, arp_sha, eth_addr_zero, + true, arp_spa, arp_tpa); - /* Compose a GARP request packet's vlan if exist. */ - if (garp->tag >= 0) { - eth_push_vlan(&packet, htons(ETH_TYPE_VLAN), htons(garp->tag)); + /* Compose a ARP request packet's vlan if exist. */ + if (tag >= 0) { + eth_push_vlan(&packet, htons(ETH_TYPE_VLAN), htons(tag)); } /* Compose actions. The garp request is output on localnet ofport. */ uint64_t ofpacts_stub[4096 / 8]; struct ofpbuf ofpacts = OFPBUF_STUB_INITIALIZER(ofpacts_stub); enum ofp_version version = rconn_get_version(swconn); - ofpact_put_OUTPUT(&ofpacts)->port = garp->ofport; + ofpact_put_OUTPUT(&ofpacts)->port = ofport; struct ofputil_packet_out po = { .packet = dp_packet_data(&packet), @@ -1813,6 +1838,17 @@ send_garp(struct garp_data *garp, long long int current_time) queue_msg(ofputil_encode_packet_out(&po, proto)); dp_packet_uninit(&packet); ofpbuf_uninit(&ofpacts); +} + +static long long int +send_garp(struct garp_data *garp, long long int current_time) +{ + if (current_time < garp->announce_time) { + return garp->announce_time; + } + + send_arp_request(garp->ea, garp->ipv4, garp->ipv4, garp->ofport, + garp->tag); /* Set the next announcement. At most 5 announcements are sent for a * vif. */ @@ -2161,6 +2197,297 @@ send_garp_run(struct controller_ctx *ctx, sset_destroy(&nat_ip_keys); } +/* Buffered "aging_mac_binding" operation. */ +struct aging_mac_binding { + /* Key. */ + uint32_t dp_key; + uint32_t port_key; + char ip_s[INET6_ADDRSTRLEN + 1]; /* Destination Ipv4 address. */ + int arp_send_count; + long long int announce_time; /* Next announcement in ms. */ + struct eth_addr ea; /* Source Ethernet address. */ + ovs_be32 ipv4; /* Source Ipv4 address. */ + ofp_port_t ofport; /* ofport used to output this packet. */ + int tag; /* VLAN tag of this ARP packet, or -1. */ +}; + +/* Contains "struct aging_mac_binding"s. + The cache for mac_bindings */ +static struct shash aging_mac_bindings; + +/* Next aging mac binding time announcement in ms. */ +static long long int next_wait_time; + +static long long int base_reachable_time = 20 * 60 * 1000; + +static void +init_aging_mac_bindings(void) +{ + shash_init(&aging_mac_bindings); + next_wait_time = LLONG_MAX; + unixctl_command_register("mac-binding-cache-list", "", 0, 0, + list_aging_mac_binding_cache, NULL); + unixctl_command_register("set-mac-binding-aging-time", "MSECS", 1, 1, + set_aging_mac_binding_time, NULL); +} + +static void +insert_aging_mac_bindings(char *name, int64_t dp_key, int64_t port_key, + const char *ip_s, struct eth_addr ea, ovs_be32 ipv4, + ofp_port_t ofport, int tag) +{ + struct aging_mac_binding *amb = xmalloc(sizeof *amb); + amb->dp_key = dp_key; + amb->port_key = port_key; + ovs_strlcpy(amb->ip_s, ip_s, INET6_ADDRSTRLEN); + amb->arp_send_count = 0; + amb->announce_time = time_msec(); + amb->ea = ea; + amb->ipv4 = ipv4; + amb->ofport = ofport; + amb->tag = tag; + shash_add(&aging_mac_bindings, name, amb); +} + +static void +destroy_aging_mac_bindings(void) +{ + shash_destroy_free_data(&aging_mac_bindings); +} + +static void +aging_mac_bindings_wait(void) +{ + poll_timer_wait_until(next_wait_time); +} + +static void +remove_mac_bindings(const struct controller_ctx *ctx, + const char *logical_port, const char * ip_s) +{ + const struct sbrec_mac_binding *b, *n; + SBREC_MAC_BINDING_FOR_EACH_SAFE (b, n, ctx->ovnsb_idl) { + if (!strcmp(b->logical_port, logical_port) + && !strcmp(b->ip, ip_s)) { + sbrec_mac_binding_delete(b); + VLOG_INFO("logical_port:%s ip:%s MAC_Binding aging.", + b->logical_port, b->ip); + return; + } + } +} + +static void +init_aging_mac_binding(uint32_t dp_key, uint32_t port_key, const char *ip_s) +{ + char *name = xasprintf("%u-%u-%s", dp_key, port_key, ip_s); + struct aging_mac_binding *amb = + shash_find_data(&aging_mac_bindings, name); + if (amb) { + amb->announce_time = time_msec(); + amb->arp_send_count = 0; + } + free(name); +} + +static bool +get_lport_addresses(const struct sbrec_port_binding *pb, + struct lport_addresses *laddrs) +{ + for (int i = 0; i < pb->n_mac; i++) { + if (extract_lsp_addresses(pb->mac[i], laddrs) + && laddrs->n_ipv4_addrs) { + return true; + } + } + return false; +} + +/* refresh mac bindings cache from ovn sb */ +static void +refresh_aging_mac_bindings(struct controller_ctx *ctx, + struct hmap *local_datapaths, + struct simap *localnet_ofports, + struct sset *local_l3gw_ports, + const struct sbrec_chassis *chassis, + const struct chassis_index *chassis_index, + struct sset *active_tunnels) +{ + struct sset mac_binding_keys = SSET_INITIALIZER(&mac_binding_keys); + + const struct sbrec_mac_binding *mb; + SBREC_MAC_BINDING_FOR_EACH (mb, ctx->ovnsb_idl) { + const struct sbrec_port_binding *pb + = lport_lookup_by_name(ctx->ovnsb_idl, mb->logical_port); + if (!pb) { + continue; + } + + const char *peer_port = smap_get_def(&pb->options, "peer", ""); + if (!sset_contains(local_l3gw_ports, peer_port)) { + continue; + } + + if (!strcmp(pb->type, "patch")) { + char *redirect = xasprintf("cr-%s", mb->logical_port); + if (!pinctrl_is_chassis_resident(ctx, chassis, chassis_index, + active_tunnels, redirect)) { + free(redirect); + continue; + } + free(redirect); + } + + struct lport_addresses laddrs; + if (!get_lport_addresses(pb, &laddrs)) { + continue; + } + + const struct sbrec_port_binding *peer_pb + = lport_lookup_by_name(ctx->ovnsb_idl, peer_port); + if (!peer_pb) { + continue; + } + + /* Find the localnet ofport to send this ARP. */ + struct local_datapath *ld + = get_local_datapath(local_datapaths, + peer_pb->datapath->tunnel_key); + if (!ld || !ld->localnet_port) { + continue; + } + ofp_port_t ofport = + u16_to_ofp(simap_get(localnet_ofports, + ld->localnet_port->logical_port)); + int tag = ld->localnet_port->n_tag ? *ld->localnet_port->tag : -1; + + char *name = xasprintf("%"PRId64"-%"PRId64"-%s", + pb->datapath->tunnel_key, pb->tunnel_key, + mb->ip); + struct aging_mac_binding *amb = + shash_find_data(&aging_mac_bindings, name); + if (amb) { + amb->ofport = ofport; + amb->tag = tag; + amb->ipv4 = laddrs.ipv4_addrs[0].addr; + amb->ea = laddrs.ea; + } else { + insert_aging_mac_bindings(name, pb->datapath->tunnel_key, + pb->tunnel_key, mb->ip, laddrs.ea, + laddrs.ipv4_addrs[0].addr, ofport, tag); + } + sset_add(&mac_binding_keys, name); + free(name); + } + + struct shash_node *iter, *next; + SHASH_FOR_EACH_SAFE (iter, next, &aging_mac_bindings) { + if (!sset_contains(&mac_binding_keys, iter->name)) { + struct aging_mac_binding *amb = iter->data; + shash_delete(&aging_mac_bindings, iter); + free(amb); + } + } + + sset_destroy(&mac_binding_keys); +} + +static void +aging_mac_bindings_run(struct controller_ctx *ctx, + const struct ovsrec_bridge *br_int, + const struct sbrec_chassis *chassis, + const struct chassis_index *chassis_index, + struct hmap *local_datapaths, + struct sset *active_tunnels) +{ + struct sset localnet_vifs = SSET_INITIALIZER(&localnet_vifs); + struct sset local_l3gw_ports = SSET_INITIALIZER(&local_l3gw_ports); + struct simap localnet_ofports = SIMAP_INITIALIZER(&localnet_ofports); + + get_localnet_vifs_l3gwports(ctx, br_int, chassis, local_datapaths, + &localnet_vifs, &localnet_ofports, + &local_l3gw_ports); + + refresh_aging_mac_bindings(ctx, local_datapaths, &localnet_ofports, + &local_l3gw_ports, chassis, chassis_index, + active_tunnels); + + long long int current_time = time_msec(); + next_wait_time = LLONG_MAX; + + struct shash_node *iter; + SHASH_FOR_EACH (iter, &aging_mac_bindings) { + struct aging_mac_binding *amb = iter->data; + + /* mac_binding aging time reachable, aging mac_binding. */ + if (current_time >= amb->announce_time + base_reachable_time) { + const struct sbrec_port_binding *pb + = lport_lookup_by_key(ctx->ovnsb_idl, amb->dp_key, + amb->port_key); + remove_mac_bindings(ctx, pb->logical_port, amb->ip_s); + continue; + } + + /* send arp request in 1/2 2/3 3/4 base_reachable_time, 3 times. */ + long long int time = (amb->arp_send_count + 1) * base_reachable_time / + (amb->arp_send_count + 2); + if (2 >= amb->arp_send_count && + current_time >= amb->announce_time + time) { + send_arp_request(amb->ea, amb->ipv4, + inet_addr(amb->ip_s), amb->ofport, amb->tag); + amb->arp_send_count++; + } + + long long int amb_next_wait_time; + if (2 < amb->arp_send_count) { + amb_next_wait_time = amb->announce_time + base_reachable_time; + } else { + time = (amb->arp_send_count + 1) * base_reachable_time / + (amb->arp_send_count + 2); + amb_next_wait_time = amb->announce_time + time; + } + if (next_wait_time > amb_next_wait_time) { + next_wait_time = amb_next_wait_time; + } + } + + sset_destroy(&localnet_vifs); + sset_destroy(&local_l3gw_ports); + simap_destroy(&localnet_ofports); +} + +static void +list_aging_mac_binding_cache(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, + void *arg OVS_UNUSED) +{ + struct ds ds = DS_EMPTY_INITIALIZER; + + ds_put_format(&ds, "aging mac binding time is %lld s.\n", + base_reachable_time / 1000); + struct shash_node *iter; + SHASH_FOR_EACH (iter, &aging_mac_bindings) { + struct aging_mac_binding *amb = iter->data; + long long int age = (time_msec() - amb->announce_time) / 1000; + ds_put_format(&ds, "dp:%d port:%d ip:%s age:%lld s send arp" + " count:%d\n.", + amb->dp_key, amb->port_key, + amb->ip_s, age, amb->arp_send_count); + } + + unixctl_command_reply(conn, ds_cstr(&ds)); + ds_destroy(&ds); +} + +static void +set_aging_mac_binding_time(struct unixctl_conn *conn, + int argc OVS_UNUSED, + const char *argv[], void *arg OVS_UNUSED) +{ + base_reachable_time = atoll(argv[1]); + unixctl_command_reply(conn, "OK."); +} + static void reload_metadata(struct ofpbuf *ofpacts, const struct match *md) { diff --git a/tests/ovn.at b/tests/ovn.at index 5f985f345..60896ea68 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -9428,3 +9428,73 @@ done OVN_CLEANUP([hv1], [hv2], [hv3]) AT_CLEANUP + +AT_SETUP([ovn -- MAC_Binding aging]) +ovn_start + +net_add n1 + +sim_add hv1 +as hv1 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.1 +ovs-vsctl set Open_vSwitch . external-ids:ovn-bridge-mappings=physnet1:br-phys + +ovn-nbctl lr-add lr0 +ovn-nbctl ls-add ls0 + +# Create logical router port lrp0 and peer lsp0 +ovn-nbctl lrp-add lr0 lrp0 f0:00:00:00:00:02 192.168.0.2/24 \ + -- set Logical_Router_Port lrp0 options:redirect-chassis="hv1" +ovn-nbctl lsp-add ls0 lsp0 \ + -- lsp-set-addresses lsp0 router \ + -- lsp-set-type lsp0 router \ + -- lsp-set-options lsp0 router-port=lrp0 nat-addresses=router + +# Create localnet port in ls0 +ovn-nbctl lsp-add ls0 ln0 \ + -- lsp-set-addresses ln0 unknown \ + -- lsp-set-type ln0 localnet \ + -- lsp-set-options ln0 network_name=physnet1 + +ovs-appctl -t ovn-controller set-mac-binding-aging-time 4000 + +# Create Mac_Binding in lrp0 +dp_uuid=`ovn-sbctl find datapath external_ids:name=lr0 | grep uuid | cut -f2 -d ":" | cut -f2 -d " "` +ovn-sbctl create MAC_Binding ip=192.168.0.3 datapath=$dp_uuid logical_port=lrp0 mac="f0\:00\:00\:00\:00\:03" + +AT_CHECK([ovn-sbctl find MAC_Binding | grep 192.168.0.3 | wc -l ], [0], [1 +]) + +# Wait for mac_binding aging. +OVS_WAIT_UNTIL([grep -c "MAC_Binding aging" hv1/ovn-controller.log]) + +$PYTHON "$top_srcdir/utilities/ovs-pcap.in" hv1/br-phys-tx.pcap > packets +echo "fffffffffffff0000000000208060001080006040001f00000000002c0a80002000000000000c0a80003" > expout +echo "fffffffffffff0000000000208060001080006040001f00000000002c0a80002000000000000c0a80003" >> expout +echo "fffffffffffff0000000000208060001080006040001f00000000002c0a80002000000000000c0a80003" >> expout +AT_CHECK([cat packets], [0], [expout]) + +# Check the mac binding is aged. +AT_CHECK([ovn-sbctl find MAC_Binding | grep 192.168.0.3 | wc -l ], [0], [0 +]) + +# Create Mac_Binding(192.168.0.1) in lrp0 +ovn-sbctl create MAC_Binding ip=192.168.0.1 datapath=$dp_uuid logical_port=lrp0 mac="f0\:00\:00\:00\:00\:01" + +AT_CHECK([ovn-sbctl find MAC_Binding | grep 192.168.0.1 | wc -l ], [0], [1 +]) + +mac=`ovs-vsctl get Interface br-phys mac_in_use | sed s/\"//g | sed s/\://g` + +# Wait for packet to be received. +package=f00000000002${mac}08060001080006040002${mac}c0a80001f00000000002c0a80002 +echo $package > expected +OVN_CHECK_PACKETS([hv1/br-phys-rx.pcap], [expected]) + +# Check the mac binding is not aged. +AT_CHECK([ovn-sbctl find MAC_Binding | grep 192.168.0.1 | wc -l ], [0], [1 +]) + +OVN_CLEANUP([hv1]) +AT_CLEANUP \ No newline at end of file