From patchwork Thu Oct 22 04:45:30 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Zhou X-Patchwork-Id: 534202 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from archives.nicira.com (unknown [IPv6:2600:3c00::f03c:91ff:fe6e:bdf7]) by ozlabs.org (Postfix) with ESMTP id 36018140157 for ; Thu, 22 Oct 2015 15:46:16 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=nicira_com.20150623.gappssmtp.com header.i=@nicira_com.20150623.gappssmtp.com header.b=z2xbIGm1; dkim-atps=neutral Received: from archives.nicira.com (localhost [127.0.0.1]) by archives.nicira.com (Postfix) with ESMTP id 26FA210769; Wed, 21 Oct 2015 21:45:57 -0700 (PDT) X-Original-To: dev@openvswitch.org Delivered-To: dev@openvswitch.org Received: from mx1e4.cudamail.com (mx1.cudamail.com [69.90.118.67]) by archives.nicira.com (Postfix) with ESMTPS id 4DACE106C2 for ; Wed, 21 Oct 2015 21:45:55 -0700 (PDT) Received: from bar2.cudamail.com (unknown [192.168.21.12]) by mx1e4.cudamail.com (Postfix) with ESMTPS id CC9541E01DC for ; Wed, 21 Oct 2015 22:45:54 -0600 (MDT) X-ASG-Debug-ID: 1445489154-03dc533d53895e0001-byXFYA Received: from mx1-pf1.cudamail.com ([192.168.24.1]) by bar2.cudamail.com with ESMTP id npwLRrCCqDwv4iEz (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 21 Oct 2015 22:45:54 -0600 (MDT) X-Barracuda-Envelope-From: azhou@nicira.com X-Barracuda-RBL-Trusted-Forwarder: 192.168.24.1 Received: from unknown (HELO mail-wi0-f170.google.com) (209.85.212.170) by mx1-pf1.cudamail.com with ESMTPS (RC4-SHA encrypted); 22 Oct 2015 04:45:53 -0000 Received-SPF: unknown (mx1-pf1.cudamail.com: Multiple SPF records returned) X-Barracuda-Apparent-Source-IP: 209.85.212.170 X-Barracuda-RBL-IP: 209.85.212.170 Received: by wicfv8 with SMTP id fv8so102722079wic.0 for ; Wed, 21 Oct 2015 21:45:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nicira_com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=74mBjztMFIZU9Ieug+bawZpn8NzB7JVAnq+pkroGPMY=; b=z2xbIGm1ifdMD27a8x4IGEJqh6TlmnlX89Ysgq7fqc5i4UwgGoQtsChe0l+L1DSK3L T6oS6wis71VePCLNL+XoDTZUasNH7YjE8W9+xz9Q+NclofWkvqrT68aGkwe1aNhsXlJD 5wRpXNRrqxbiz+sN78JGgBlzvQlN8ScvxuF01OL0xwh3tupC59/oOiu1l2Qyc4PZ7UNv 5/SPH/hcYwVOKnkfK5UtNlTtI8K0qzpTMm2LCVl/B1etp35kQBdVVQIyUcNG4ZghwQ7E 0LLQM4ke5p6TmI4Ut8NeRNJjg/wltJHgJ+NfQZ4yL9seTCB+GHwS0Pmi+u0/U265vcNe G81g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=74mBjztMFIZU9Ieug+bawZpn8NzB7JVAnq+pkroGPMY=; b=YxWf/kVF3BWH3S4LMEDS6r6h5yREixu6S+BXUI/hvUdXYVtH/efdOxQfrmUaSRrT9w eifx9mNYJQHJNRPbdQwAcK61UwZcAexa22ktU4pcuvR9kaMiSPt412q9KxF1Qd6Byeot uxfdlBYJhEDAYsmkYdYYrhv0XAWGvy7C/bVx6+hzUB5D6HvC8QA/OPaWYADad86DE27i NlRJCP3jWjtf8ShPUv62/tyKY8w4piZ+xeiHLmEp0mT+VKG8l5y71p8paWCw8LQRYdcq Eg3mFoU95n91ZwJZabe0sGNrb7QdOV8haWT3FmF5K8c8K6E5la+K7uzBscacAJSjkNg4 OCDQ== X-Gm-Message-State: ALoCoQmfkAiPkSDPuxj+rLBngXvkwcMWj/85WbtWMUTpJkmsaug6AzDBLMMJUKhLHaecssMjARx/ X-Received: by 10.180.88.227 with SMTP id bj3mr35396576wib.80.1445489152025; Wed, 21 Oct 2015 21:45:52 -0700 (PDT) Received: from ubuntu.localdomain ([208.91.1.34]) by smtp.gmail.com with ESMTPSA id s16sm26389112wik.16.2015.10.21.21.45.50 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 21 Oct 2015 21:45:51 -0700 (PDT) X-CudaMail-Envelope-Sender: azhou@nicira.com From: Andy Zhou To: dev@openvswitch.org X-CudaMail-Whitelist-To: dev@openvswitch.org X-CudaMail-MID: CM-E1-1020120086 X-CudaMail-DTE: 102115 X-CudaMail-Originating-IP: 209.85.212.170 Date: Wed, 21 Oct 2015 21:45:30 -0700 X-ASG-Orig-Subj: [##CM-E1-1020120086##][mointor2 8/9] lib: add monitor2 support in ovsdb-idl. Message-Id: <1445489131-21483-8-git-send-email-azhou@nicira.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1445489131-21483-1-git-send-email-azhou@nicira.com> References: <1445489131-21483-1-git-send-email-azhou@nicira.com> X-Barracuda-Connect: UNKNOWN[192.168.24.1] X-Barracuda-Start-Time: 1445489154 X-Barracuda-Encrypted: DHE-RSA-AES256-SHA X-Barracuda-URL: https://web.cudamail.com:443/cgi-mod/mark.cgi X-ASG-Whitelist: Header =?UTF-8?B?eFwtY3VkYW1haWxcLXdoaXRlbGlzdFwtdG8=?= X-Virus-Scanned: by bsmtpd at cudamail.com X-Barracuda-BRTS-Status: 1 Subject: [ovs-dev] [mointor2 8/9] lib: add monitor2 support in ovsdb-idl. X-BeenThere: dev@openvswitch.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: dev-bounces@openvswitch.org Sender: "dev" Add support for monitor2. When idl starts to run, monitor2 will be attempted first. In case the server is an older version that does not recognize monitor2. IDL will then fall back to use "monitor" method. Signed-off-by: Andy Zhou --- lib/ovsdb-idl.c | 319 ++++++++++++++++++++++++++++++++++++++++++++++++++--- tests/ovsdb-idl.at | 10 +- 2 files changed, 306 insertions(+), 23 deletions(-) diff --git a/lib/ovsdb-idl.c b/lib/ovsdb-idl.c index 00b900d..1a30ad4 100644 --- a/lib/ovsdb-idl.c +++ b/lib/ovsdb-idl.c @@ -78,7 +78,9 @@ struct ovsdb_idl_arc { enum ovsdb_idl_state { IDL_S_SCHEMA_REQUESTED, IDL_S_MONITOR_REQUESTED, - IDL_S_MONITORING + IDL_S_MONITORING, + IDL_S_MONITOR2_REQUESTED, + IDL_S_MONITORING2 }; struct ovsdb_idl { @@ -93,6 +95,7 @@ struct ovsdb_idl { unsigned int state_seqno; enum ovsdb_idl_state state; struct json *request_id; + struct json *schema; /* Database locking. */ char *lock_name; /* Name of lock we need, NULL if none. */ @@ -138,18 +141,27 @@ static struct vlog_rate_limit semantic_rl = VLOG_RATE_LIMIT_INIT(1, 5); static void ovsdb_idl_clear(struct ovsdb_idl *); static void ovsdb_idl_send_schema_request(struct ovsdb_idl *); -static void ovsdb_idl_send_monitor_request(struct ovsdb_idl *, - const struct json *schema_json); +static void ovsdb_idl_send_monitor_request(struct ovsdb_idl *); +static void ovsdb_idl_send_monitor2_request(struct ovsdb_idl *); static void ovsdb_idl_parse_update(struct ovsdb_idl *, const struct json *); +static void ovsdb_idl_parse_update2(struct ovsdb_idl *, const struct json *); static struct ovsdb_error *ovsdb_idl_parse_update__(struct ovsdb_idl *, const struct json *); +static struct ovsdb_error *ovsdb_idl_parse_update2__(struct ovsdb_idl *, + const struct json *); static bool ovsdb_idl_process_update(struct ovsdb_idl_table *, const struct uuid *, const struct json *old, const struct json *new); +static bool ovsdb_idl_process_update2(struct ovsdb_idl_table *, + const struct uuid *, + const char *operation, + const struct json *row); static void ovsdb_idl_insert_row(struct ovsdb_idl_row *, const struct json *); static void ovsdb_idl_delete_row(struct ovsdb_idl_row *); static bool ovsdb_idl_modify_row(struct ovsdb_idl_row *, const struct json *); +static bool ovsdb_idl_modify_row_by_diff(struct ovsdb_idl_row *, + const struct json *); static bool ovsdb_idl_row_is_orphan(const struct ovsdb_idl_row *); static struct ovsdb_idl_row *ovsdb_idl_row_create__( @@ -232,6 +244,7 @@ ovsdb_idl_create(const char *remote, const struct ovsdb_idl_class *class, idl->state_seqno = UINT_MAX; idl->request_id = NULL; + idl->schema = NULL; hmap_init(&idl->outstanding_txns); @@ -260,6 +273,7 @@ ovsdb_idl_destroy(struct ovsdb_idl *idl) json_destroy(idl->request_id); free(idl->lock_name); json_destroy(idl->lock_request_id); + json_destroy(idl->schema); hmap_destroy(&idl->outstanding_txns); free(idl); } @@ -335,12 +349,12 @@ ovsdb_idl_run(struct ovsdb_idl *idl) } if (msg->type == JSONRPC_NOTIFY - && !strcmp(msg->method, "update") + && !strcmp(msg->method, "update2") && msg->params->type == JSON_ARRAY && msg->params->u.array.n == 2 && msg->params->u.array.elems[0]->type == JSON_NULL) { /* Database contents changed. */ - ovsdb_idl_parse_update(idl, msg->params->u.array.elems[1]); + ovsdb_idl_parse_update2(idl, msg->params->u.array.elems[1]); } else if (msg->type == JSONRPC_REPLY && idl->request_id && json_equal(idl->request_id, msg->id)) { @@ -349,24 +363,44 @@ ovsdb_idl_run(struct ovsdb_idl *idl) /* Reply to our "get_schema" request. */ json_destroy(idl->request_id); idl->request_id = NULL; - ovsdb_idl_send_monitor_request(idl, msg->result); - idl->state = IDL_S_MONITOR_REQUESTED; + idl->schema = json_clone(msg->result); + ovsdb_idl_send_monitor2_request(idl); + idl->state = IDL_S_MONITOR2_REQUESTED; break; case IDL_S_MONITOR_REQUESTED: - /* Reply to our "monitor" request. */ + case IDL_S_MONITOR2_REQUESTED: + /* Reply to our "monitor" or "monitor2" request. */ idl->change_seqno++; json_destroy(idl->request_id); idl->request_id = NULL; - idl->state = IDL_S_MONITORING; ovsdb_idl_clear(idl); - ovsdb_idl_parse_update(idl, msg->result); + if (idl->state == IDL_S_MONITOR_REQUESTED) { + idl->state = IDL_S_MONITORING; + ovsdb_idl_parse_update(idl, msg->result); + } else { /* IDL_S_MONITOR2_REQUESTED. */ + idl->state = IDL_S_MONITORING2; + ovsdb_idl_parse_update2(idl, msg->result); + } + + /* Schema is not useful after monitor request is accepted + * by the server. */ + json_destroy(idl->schema); + idl->schema = NULL; break; case IDL_S_MONITORING: + case IDL_S_MONITORING2: default: OVS_NOT_REACHED(); } + } else if (msg->type == JSONRPC_NOTIFY + && !strcmp(msg->method, "update") + && msg->params->type == JSON_ARRAY + && msg->params->u.array.n == 2 + && msg->params->u.array.elems[0]->type == JSON_NULL) { + /* Database contents changed. */ + ovsdb_idl_parse_update(idl, msg->params->u.array.elems[1]); } else if (msg->type == JSONRPC_REPLY && idl->lock_request_id && json_equal(idl->lock_request_id, msg->id)) { @@ -380,6 +414,16 @@ ovsdb_idl_run(struct ovsdb_idl *idl) && !strcmp(msg->method, "stolen")) { /* Someone else stole our lock. */ ovsdb_idl_parse_lock_notify(idl, msg->params, false); + } else if (msg->type == JSONRPC_ERROR + && idl->state == IDL_S_MONITOR2_REQUESTED) { + if (msg->error && !strcmp(json_string(msg->error), + "unknown method")) { + /* Fall back to using "monitor" method. */ + json_destroy(idl->request_id); + idl->request_id = NULL; + ovsdb_idl_send_monitor_request(idl); + idl->state = IDL_S_MONITOR_REQUESTED; + } } else if ((msg->type == JSONRPC_ERROR || msg->type == JSONRPC_REPLY) && ovsdb_idl_txn_process_reply(idl, msg)) { @@ -690,14 +734,15 @@ parse_schema(const struct json *schema_json) } static void -ovsdb_idl_send_monitor_request(struct ovsdb_idl *idl, - const struct json *schema_json) +ovsdb_idl_send_monitor_request__(struct ovsdb_idl *idl, + const char *method) { - struct shash *schema = parse_schema(schema_json); + struct shash *schema; struct json *monitor_requests; struct jsonrpc_msg *msg; size_t i; + schema = parse_schema(idl->schema); monitor_requests = json_object_create(); for (i = 0; i < idl->class->n_tables; i++) { const struct ovsdb_idl_table *table = &idl->tables[i]; @@ -747,7 +792,7 @@ ovsdb_idl_send_monitor_request(struct ovsdb_idl *idl, json_destroy(idl->request_id); msg = jsonrpc_create_request( - "monitor", + method, json_array_create_3(json_string_create(idl->class->database), json_null_create(), monitor_requests), &idl->request_id); @@ -755,16 +800,124 @@ ovsdb_idl_send_monitor_request(struct ovsdb_idl *idl, } static void -ovsdb_idl_parse_update(struct ovsdb_idl *idl, const struct json *table_updates) +ovsdb_idl_send_monitor_request(struct ovsdb_idl *idl) +{ + ovsdb_idl_send_monitor_request__(idl, "monitor"); +} + +static void +log_parse_update_error(struct ovsdb_error *error) { - struct ovsdb_error *error = ovsdb_idl_parse_update__(idl, table_updates); - if (error) { if (!VLOG_DROP_WARN(&syntax_rl)) { char *s = ovsdb_error_to_string(error); VLOG_WARN_RL(&syntax_rl, "%s", s); free(s); } ovsdb_error_destroy(error); +} + +static void +ovsdb_idl_send_monitor2_request(struct ovsdb_idl *idl) +{ + ovsdb_idl_send_monitor_request__(idl, "monitor2"); +} + +static void +ovsdb_idl_parse_update2(struct ovsdb_idl *idl, + const struct json *table_updates) +{ + struct ovsdb_error *error = ovsdb_idl_parse_update2__(idl, table_updates); + if (error) { + log_parse_update_error(error); + } +} + +static struct ovsdb_error * +ovsdb_idl_parse_update2__(struct ovsdb_idl *idl, + const struct json *table_updates) +{ + const struct shash_node *tables_node; + + if (table_updates->type != JSON_OBJECT) { + return ovsdb_syntax_error(table_updates, NULL, + " is not an object"); + } + + SHASH_FOR_EACH (tables_node, json_object(table_updates)) { + const struct json *table_update2 = tables_node->data; + const struct shash_node *table_node; + struct ovsdb_idl_table *table; + + table = shash_find_data(&idl->table_by_name, tables_node->name); + if (!table) { + return ovsdb_syntax_error( + table_updates, NULL, + " includes unknown table \"%s\"", + tables_node->name); + } + + if (table_update2->type != JSON_OBJECT) { + return ovsdb_syntax_error(table_update2, NULL, + " for table \"%s\" is " + "not an object", table->class->name); + } + SHASH_FOR_EACH (table_node, json_object(table_update2)) { + const struct json *row_update2 = table_node->data; + struct uuid uuid; + + if (!uuid_from_string(&uuid, table_node->name)) { + return ovsdb_syntax_error(table_update2, NULL, + " for table \"%s\" " + "contains bad UUID " + "\"%s\" as member name", + table->class->name, + table_node->name); + } + if (row_update2->type != JSON_OBJECT) { + return ovsdb_syntax_error(row_update2, NULL, + " for table \"%s\" " + "contains for %s that " + "is not an object", + table->class->name, + table_node->name); + } + + const char *ops[] = {"modify", "insert", "delete", "initial"}; + const char *operation; + const struct json *row; + int i; + + for (i = 0; i < ARRAY_SIZE(ops); i++) { + operation = ops[i]; + row = shash_find_data(json_object(row_update2), operation); + + if (row) { + if (ovsdb_idl_process_update2(table, &uuid, operation, + row)) { + idl->change_seqno++; + } + break; + } + } + + /* row_update2 should contain one of the objects */ + if (i == ARRAY_SIZE(ops)) { + return ovsdb_syntax_error(row_update2, NULL, + " includes unknown " + "object"); + } + } + } + + return NULL; +} + +static void +ovsdb_idl_parse_update(struct ovsdb_idl *idl, const struct json *table_updates) +{ + struct ovsdb_error *error = ovsdb_idl_parse_update__(idl, table_updates); + if (error) { + log_parse_update_error(error); } } @@ -916,6 +1069,120 @@ ovsdb_idl_process_update(struct ovsdb_idl_table *table, /* Returns true if a column with mode OVSDB_IDL_MODE_RW changed, false * otherwise. */ static bool +ovsdb_idl_process_update2(struct ovsdb_idl_table *table, + const struct uuid *uuid, + const char *operation, + const struct json *json_row) +{ + struct ovsdb_idl_row *row; + + row = ovsdb_idl_get_row(table, uuid); + if (!strcmp(operation, "delete")) { + /* Delete row. */ + if (row && !ovsdb_idl_row_is_orphan(row)) { + ovsdb_idl_delete_row(row); + } else { + VLOG_WARN_RL(&semantic_rl, "cannot delete missing row "UUID_FMT" " + "from table %s", + UUID_ARGS(uuid), table->class->name); + return false; + } + } else if (!strcmp(operation, "insert") || !strcmp(operation, "initial")) { + /* Insert row. */ + if (!row) { + ovsdb_idl_insert_row(ovsdb_idl_row_create(table, uuid), json_row); + } else if (ovsdb_idl_row_is_orphan(row)) { + ovsdb_idl_insert_row(row, json_row); + } else { + VLOG_WARN_RL(&semantic_rl, "cannot add existing row "UUID_FMT" to " + "table %s", UUID_ARGS(uuid), table->class->name); + ovsdb_idl_delete_row(row); + ovsdb_idl_insert_row(row, json_row); + } + } else if (!strcmp(operation, "modify")) { + /* Modify row. */ + if (row) { + if (!ovsdb_idl_row_is_orphan(row)) { + return ovsdb_idl_modify_row_by_diff(row, json_row); + } else { + VLOG_WARN_RL(&semantic_rl, "cannot modify missing but " + "referenced row "UUID_FMT" in table %s", + UUID_ARGS(uuid), table->class->name); + return false; + } + } else { + VLOG_WARN_RL(&semantic_rl, "cannot modify missing row "UUID_FMT" " + "in table %s", UUID_ARGS(uuid), table->class->name); + return false; + } + } else { + VLOG_WARN_RL(&semantic_rl, "unknown operation %s to " + "table %s", operation, table->class->name); + return false; + } + + return true; +} + +/* Returns true if a column with mode OVSDB_IDL_MODE_RW changed, false + * otherwise. */ +static bool +ovsdb_idl_row_apply_diff(struct ovsdb_idl_row *row, + const struct json *diff_json) +{ + struct ovsdb_idl_table *table = row->table; + struct shash_node *node; + bool changed = false; + + SHASH_FOR_EACH (node, json_object(diff_json)) { + const char *column_name = node->name; + const struct ovsdb_idl_column *column; + struct ovsdb_datum diff; + struct ovsdb_error *error; + + column = shash_find_data(&table->columns, column_name); + if (!column) { + VLOG_WARN_RL(&syntax_rl, "unknown column %s updating row "UUID_FMT, + column_name, UUID_ARGS(&row->uuid)); + continue; + } + + error = ovsdb_transient_datum_from_json(&diff, &column->type, + node->data); + if (!error) { + unsigned int column_idx = column - table->class->columns; + struct ovsdb_datum *old = &row->old[column_idx]; + struct ovsdb_datum *new; + + new = ovsdb_datum_apply_diff(old, &diff, &column->type); + if (!new) { + VLOG_WARN_RL(&syntax_rl, "update2 failed to modify column " + "%s row "UUID_FMT, column_name, + UUID_ARGS(&row->uuid)); + } else { + ovsdb_datum_swap(old, new); + ovsdb_datum_destroy(new, &column->type); + free(new); + if (table->modes[column_idx] & OVSDB_IDL_ALERT) { + changed = true; + } + } + ovsdb_datum_destroy(&diff, &column->type); + } else { + char *s = ovsdb_error_to_string(error); + VLOG_WARN_RL(&syntax_rl, "error parsing column %s in row "UUID_FMT + " in table %s: %s", column_name, + UUID_ARGS(&row->uuid), table->class->name, s); + free(s); + ovsdb_error_destroy(error); + } + } + return changed; +} + +/* Returns true if a column with mode OVSDB_IDL_MODE_RW changed, false + * otherwise. */ +static bool ovsdb_idl_row_update(struct ovsdb_idl_row *row, const struct json *row_json) { struct ovsdb_idl_table *table = row->table; @@ -1182,6 +1449,20 @@ ovsdb_idl_modify_row(struct ovsdb_idl_row *row, const struct json *row_json) } static bool +ovsdb_idl_modify_row_by_diff(struct ovsdb_idl_row *row, + const struct json *diff_json) +{ + bool changed; + + ovsdb_idl_row_unparse(row); + ovsdb_idl_row_clear_arcs(row, true); + changed = ovsdb_idl_row_apply_diff(row, diff_json); + ovsdb_idl_row_parse(row); + + return changed; +} + +static bool may_add_arc(const struct ovsdb_idl_row *src, const struct ovsdb_idl_row *dst) { const struct ovsdb_idl_arc *arc; @@ -2559,11 +2840,13 @@ ovsdb_idl_is_lock_contended(const struct ovsdb_idl *idl) return idl->is_lock_contended; } + static void ovsdb_idl_update_has_lock(struct ovsdb_idl *idl, bool new_has_lock) { if (new_has_lock && !idl->has_lock) { - if (idl->state == IDL_S_MONITORING) { + if (idl->state == IDL_S_MONITORING || + idl->state == IDL_S_MONITORING2) { idl->change_seqno++; } else { /* We're setting up a session, so don't signal that the database diff --git a/tests/ovsdb-idl.at b/tests/ovsdb-idl.at index d3d2aeb..5ede75f 100644 --- a/tests/ovsdb-idl.at +++ b/tests/ovsdb-idl.at @@ -588,13 +588,13 @@ test-ovsdb|ovsdb_idl|link1 table in idltest database lacks l2 column (database n # Check that ovsdb-idl sent on "monitor" request and that it didn't # mention that table or column, and (for paranoia) that it did mention another # table and column. -AT_CHECK([grep -c '"monitor"' stderr], [0], [1 +AT_CHECK([grep -c '"monitor\|monitor2"' stderr], [0], [1 ]) -AT_CHECK([grep '"monitor"' stderr | grep link2], [1]) -AT_CHECK([grep '"monitor"' stderr | grep l2], [1]) -AT_CHECK([grep '"monitor"' stderr | grep -c '"link1"'], [0], [1 +AT_CHECK([grep '"monitor\|monitor2"' stderr | grep link2], [1]) +AT_CHECK([grep '"monitor\|monitor2"' stderr | grep l2], [1]) +AT_CHECK([grep '"monitor\|monitor2"' stderr | grep -c '"link1"'], [0], [1 ]) -AT_CHECK([grep '"monitor"' stderr | grep -c '"ua"'], [0], [1 +AT_CHECK([grep '"monitor\|monitor2"' stderr | grep -c '"ua"'], [0], [1 ]) OVSDB_SERVER_SHUTDOWN AT_CLEANUP