From patchwork Wed Aug 25 03:12:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1520493 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4GvWJr3ZkBz9sWl for ; Wed, 25 Aug 2021 13:13:32 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id CD2A640373; Wed, 25 Aug 2021 03:13:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ApjlhSebfgGx; Wed, 25 Aug 2021 03:13:21 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 0D50640239; Wed, 25 Aug 2021 03:13:20 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id CF75CC001A; Wed, 25 Aug 2021 03:13:19 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 12B19C000E for ; Wed, 25 Aug 2021 03:13:18 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id C21B5402AB for ; Wed, 25 Aug 2021 03:13:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Wx2yEsZ07KrC for ; Wed, 25 Aug 2021 03:13:12 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by smtp2.osuosl.org (Postfix) with ESMTPS id 1D4954034B for ; Wed, 25 Aug 2021 03:13:11 +0000 (UTC) Received: (Authenticated sender: numans@ovn.org) by relay3-d.mail.gandi.net (Postfix) with ESMTPSA id 5D6F460002; Wed, 25 Aug 2021 03:13:07 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Tue, 24 Aug 2021 23:12:57 -0400 Message-Id: <20210825031257.3163318-1-numans@ovn.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210825031234.3163243-1-numans@ovn.org> References: <20210825031234.3163243-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH v4 1/2] ovsdb-idl : Add APIs to query if a table and a column is present or not. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This patch adds 2 new APIs in the ovsdb-idl client library - ovsdb_idl_has_table() and ovsdb_idl_has_column_in_table() to query if a table and a column is present in the IDL or not. This is required for scenarios where the server schema is old and missing a table or column and the client (built with a new schema version) does a transaction with the missing table or column. This results in a continuous loop of transaction failures. OVN would require the API - ovsdb_idl_has_table() to address this issue when an old ovsdb-server is used (OVS 2.11) which has the 'datapath' table missing. A recent commit in OVN creates a 'datapath' row in the local ovsdb-server. ovsdb_idl_has_column_in_table() would be useful to have. Related issue: https://bugzilla.redhat.com/show_bug.cgi?id=1992705 Signed-off-by: Numan Siddique --- lib/ovsdb-idl-provider.h | 4 +++ lib/ovsdb-idl.c | 36 ++++++++++++++++++++ lib/ovsdb-idl.h | 3 ++ tests/ovsdb-idl.at | 38 +++++++++++++++++++++ tests/test-ovsdb.c | 73 ++++++++++++++++++++++++++++++++++++++++ 5 files changed, 154 insertions(+) diff --git a/lib/ovsdb-idl-provider.h b/lib/ovsdb-idl-provider.h index 0f38f9b34..0f122e23c 100644 --- a/lib/ovsdb-idl-provider.h +++ b/lib/ovsdb-idl-provider.h @@ -24,6 +24,7 @@ #include "ovsdb-set-op.h" #include "ovsdb-types.h" #include "openvswitch/shash.h" +#include "sset.h" #include "uuid.h" #ifdef __cplusplus @@ -117,9 +118,12 @@ struct ovsdb_idl_table { bool need_table; /* Monitor table even if no columns are selected * for replication. */ struct shash columns; /* Contains "const struct ovsdb_idl_column *"s. */ + struct sset schema_columns; /* Column names from schema. */ struct hmap rows; /* Contains "struct ovsdb_idl_row"s. */ struct ovsdb_idl *idl; /* Containing IDL instance. */ unsigned int change_seqno[OVSDB_IDL_CHANGE_MAX]; + bool in_server_schema; /* Indicates if this table is in the server schema + * or not. */ struct ovs_list indexes; /* Contains "struct ovsdb_idl_index"s */ struct ovs_list track_list; /* Tracked rows (ovsdb_idl_row.track_node). */ }; diff --git a/lib/ovsdb-idl.c b/lib/ovsdb-idl.c index 2198c69c6..b2dfff46c 100644 --- a/lib/ovsdb-idl.c +++ b/lib/ovsdb-idl.c @@ -287,6 +287,8 @@ ovsdb_idl_create_unconnected(const struct ovsdb_idl_class *class, = table->change_seqno[OVSDB_IDL_CHANGE_MODIFY] = table->change_seqno[OVSDB_IDL_CHANGE_DELETE] = 0; table->idl = idl; + table->in_server_schema = true; /* Assume it's in server schema. */ + sset_init(&table->schema_columns); } return idl; @@ -337,6 +339,7 @@ ovsdb_idl_destroy(struct ovsdb_idl *idl) struct ovsdb_idl_table *table = &idl->tables[i]; ovsdb_idl_destroy_indexes(table); shash_destroy(&table->columns); + sset_destroy(&table->schema_columns); hmap_destroy(&table->rows); free(table->modes); } @@ -718,6 +721,7 @@ ovsdb_idl_compose_monitor_request(const struct json *schema_json, void *idl_) struct json *columns = table->need_table ? json_array_create_empty() : NULL; + sset_clear(&table->schema_columns); for (size_t j = 0; j < tc->n_columns; j++) { const struct ovsdb_idl_column *column = &tc->columns[j]; bool idl_has_column = (table_schema && @@ -741,6 +745,7 @@ ovsdb_idl_compose_monitor_request(const struct json *schema_json, void *idl_) } json_array_add(columns, json_string_create(column->name)); } + sset_add(&table->schema_columns, column->name); } if (columns) { @@ -749,7 +754,12 @@ ovsdb_idl_compose_monitor_request(const struct json *schema_json, void *idl_) "(database needs upgrade?)", idl->class_->database, table->class_->name); json_destroy(columns); + /* Set 'table->in_server_schema' to false so that this can be + * excluded from transactions. */ + table->in_server_schema = false; continue; + } else if (schema && table_schema) { + table->in_server_schema = true; } monitor_request = json_object_create(); @@ -4256,3 +4266,29 @@ ovsdb_idl_loop_commit_and_wait(struct ovsdb_idl_loop *loop) return retval; } + +static struct ovsdb_idl_table* +ovsdb_idl_get_table(struct ovsdb_idl *idl, const char *table_name) +{ + struct ovsdb_idl_table *table = shash_find_data(&idl->table_by_name, + table_name); + return table && table->in_server_schema ? table : NULL; +} + +bool +ovsdb_idl_has_table(struct ovsdb_idl *idl, const char *table_name) +{ + return ovsdb_idl_get_table(idl, table_name) ? true: false; +} + +bool +ovsdb_idl_has_column_in_table(struct ovsdb_idl *idl, const char *table_name, + const char *column_name) +{ + struct ovsdb_idl_table *table = ovsdb_idl_get_table(idl, table_name); + if (table && sset_find(&table->schema_columns, column_name)) { + return true; + } + + return false; +} diff --git a/lib/ovsdb-idl.h b/lib/ovsdb-idl.h index d93483245..48425b39a 100644 --- a/lib/ovsdb-idl.h +++ b/lib/ovsdb-idl.h @@ -474,6 +474,9 @@ void ovsdb_idl_cursor_next_eq(struct ovsdb_idl_cursor *); struct ovsdb_idl_row *ovsdb_idl_cursor_data(struct ovsdb_idl_cursor *); +bool ovsdb_idl_has_table(struct ovsdb_idl *, const char *table_name); +bool ovsdb_idl_has_column_in_table(struct ovsdb_idl *, const char *table_name, + const char *column_name); #ifdef __cplusplus } #endif diff --git a/tests/ovsdb-idl.at b/tests/ovsdb-idl.at index 1386f1377..b11fabe70 100644 --- a/tests/ovsdb-idl.at +++ b/tests/ovsdb-idl.at @@ -2372,3 +2372,41 @@ OVSDB_CHECK_CLUSTER_IDL([simple idl, initially empty, force reconnect], [], [], reconnect.*waiting .* seconds before reconnect) + +AT_SETUP([idl table and column presence check]) +AT_KEYWORDS([ovsdb server idl table column check]) +AT_CHECK([ovsdb_start_idltest "" "$abs_srcdir/idltest2.ovsschema"]) + +ovsdb-tool create db2 $abs_srcdir/idltest.ovsschema +ovsdb-server -vconsole:warn --log-file=ovsdb-server2.log --detach --no-chdir --pidfile=ovsdb-server2.pid --remote=punix:socket2 db2 +on_exit 'kill `cat ovsdb-server2.pid`' + +# In this test, test-ovsdb first connects to the server with schema +# idltest2.ovsschema and outputs the presence of tables and columns. +# And then it connectes to the server with the schema idltest.ovsschema +# and does the same. +AT_CHECK([test-ovsdb -vconsole:off -t10 idl-table-column-check unix:socket unix:socket2 \ +simple link1 link2 simple5 foo simple5:irefmap simple5:foo link1:l2], +[0], [dnl +table simple is present +table link1 is present +table link2 is not present +table simple5 is not present +table foo is not present +table simple5 is not present +table simple5 is not present +column l2 in table link1 is not present +--- remote 1 done --- +table simple is present +table link1 is present +table link2 is present +table simple5 is present +table foo is not present +column irefmap in table simple5 is present +column foo in table simple5 is not present +column l2 in table link1 is present +--- remote 2 done --- +]) + +OVSDB_SERVER_SHUTDOWN +AT_CLEANUP diff --git a/tests/test-ovsdb.c b/tests/test-ovsdb.c index daa55dab7..462d2e57e 100644 --- a/tests/test-ovsdb.c +++ b/tests/test-ovsdb.c @@ -3268,6 +3268,77 @@ do_idl_compound_index(struct ovs_cmdl_context *ctx) printf("%03d: done\n", step); } +static void +do_idl_table_column_check(struct ovs_cmdl_context *ctx) +{ + struct jsonrpc *rpc; + struct ovsdb_idl *idl; + unsigned int seqno = 0; + int error; + int i; + + if (ctx->argc < 3) { + exit(1); + } + + idl = ovsdb_idl_create(ctx->argv[1], &idltest_idl_class, true, true); + ovsdb_idl_set_leader_only(idl, false); + struct stream *stream; + + error = stream_open_block(jsonrpc_stream_open(ctx->argv[1], &stream, + DSCP_DEFAULT), -1, &stream); + if (error) { + ovs_fatal(error, "failed to connect to \"%s\"", ctx->argv[1]); + } + rpc = jsonrpc_open(stream); + + for (int r = 1; r <= 2; r++) { + ovsdb_idl_set_remote(idl, ctx->argv[r], true); + ovsdb_idl_force_reconnect(idl); + + /* Wait for update. */ + for (;;) { + ovsdb_idl_run(idl); + ovsdb_idl_check_consistency(idl); + if (ovsdb_idl_get_seqno(idl) != seqno) { + break; + } + jsonrpc_run(rpc); + + ovsdb_idl_wait(idl); + jsonrpc_wait(rpc); + poll_block(); + } + + seqno = ovsdb_idl_get_seqno(idl); + + for (i = 3; i < ctx->argc; i++) { + char *save_ptr2 = NULL; + char *arg = xstrdup(ctx->argv[i]); + char *table_name = strtok_r(arg, ":", &save_ptr2); + char *column_name = strtok_r(NULL, ":", &save_ptr2); + + bool table_present = ovsdb_idl_has_table(idl, table_name); + if (!table_present || !column_name) { + printf("table %s %s present\n", table_name, + table_present ? "is" : "is not"); + } + if (table_present && column_name) { + printf("column %s in table %s %s present\n", column_name, + table_name, + ovsdb_idl_has_column_in_table(idl, table_name, + column_name) ? + "is" : "is not"); + } + free(arg); + } + printf("--- remote %d done ---\n", r); + } + + jsonrpc_close(rpc); + ovsdb_idl_destroy(idl); +} + static struct ovs_cmdl_command all_commands[] = { { "log-io", NULL, 2, INT_MAX, do_log_io, OVS_RO }, { "default-atoms", NULL, 0, 0, do_default_atoms, OVS_RO }, @@ -3306,6 +3377,8 @@ static struct ovs_cmdl_command all_commands[] = { do_idl_partial_update_map_column, OVS_RO }, { "idl-partial-update-set-column", NULL, 1, INT_MAX, do_idl_partial_update_set_column, OVS_RO }, + { "idl-table-column-check", NULL, 1, INT_MAX, + do_idl_table_column_check, OVS_RO }, { "help", NULL, 0, INT_MAX, do_help, OVS_RO }, { NULL, NULL, 0, 0, NULL, OVS_RO }, };