From patchwork Thu Dec 14 01:04:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Maximets X-Patchwork-Id: 1875927 X-Patchwork-Delegate: i.maximets@samsung.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SrDhL3hgSz1ySd for ; Thu, 14 Dec 2023 12:06:42 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 906614370A; Thu, 14 Dec 2023 01:06:40 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 906614370A X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uUUjU27UK3pg; Thu, 14 Dec 2023 01:06:33 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 2E5E043748; Thu, 14 Dec 2023 01:06:32 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 2E5E043748 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 216EFC0DDB; Thu, 14 Dec 2023 01:06:31 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 30F73C0DCF for ; Thu, 14 Dec 2023 01:06:30 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 8D04283367 for ; Thu, 14 Dec 2023 01:05:19 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 8D04283367 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7DPTxwrwaYzB for ; Thu, 14 Dec 2023 01:05:16 +0000 (UTC) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by smtp1.osuosl.org (Postfix) with ESMTPS id 20F678316A for ; Thu, 14 Dec 2023 01:05:15 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 20F678316A Received: by mail.gandi.net (Postfix) with ESMTPSA id BDDEE240002; Thu, 14 Dec 2023 01:05:13 +0000 (UTC) From: Ilya Maximets To: ovs-dev@openvswitch.org Date: Thu, 14 Dec 2023 02:04:14 +0100 Message-ID: <20231214010431.1664005-13-i.maximets@ovn.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20231214010431.1664005-1-i.maximets@ovn.org> References: <20231214010431.1664005-1-i.maximets@ovn.org> MIME-Version: 1.0 X-GND-Sasl: i.maximets@ovn.org Cc: Vladislav Odintsov , Dumitru Ceara , Ilya Maximets Subject: [ovs-dev] [PATCH 12/22] ovsdb-server: Database config isolation. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add a new structure 'db_config' that holds the user-provided configuration of the database. And attach this configuration to each of the databases on the server. Each database has a service model: standalone, clustered, relay or active-backup. Relays and A-B databases have a source, each source has its own set of JSON-RPC session options. A-B also have an indicator of it being active or backup and an optional list of tables to exclude from replication. All of that should be stored per database in the temporary configuration file that is used in order to restore the config after the OVSDB crash. For that, the save/load functions are also updates. This change is written in generic way assuming all the databases can have different configuration including service model. The only user-visible change here is a slight modification of the ovsdb-server/sync-status appctl, since it now needs to skip databases that are not active-backup and also should report active-backup databases that are currently active, i.e. not added to the replication module. If the service model is not defined in the configuration, it is assumed to be standalone or clustered, and determined from the storage type while opening the database. Since the database 'source' connections can't use 'role' or 'read-only' options, a new flag added to corresponding JSON parsing functions to skip these fields. Signed-off-by: Ilya Maximets --- ovsdb/jsonrpc-server.c | 31 +- ovsdb/jsonrpc-server.h | 5 +- ovsdb/ovsdb-server.c | 775 +++++++++++++++++++++++++++++++---------- ovsdb/replication.c | 1 - tests/ovsdb-server.at | 8 +- 5 files changed, 623 insertions(+), 197 deletions(-) diff --git a/ovsdb/jsonrpc-server.c b/ovsdb/jsonrpc-server.c index 299afbb1d..81e9e48a0 100644 --- a/ovsdb/jsonrpc-server.c +++ b/ovsdb/jsonrpc-server.c @@ -231,7 +231,8 @@ ovsdb_jsonrpc_options_clone(const struct ovsdb_jsonrpc_options *options) } struct json * -ovsdb_jsonrpc_options_to_json(const struct ovsdb_jsonrpc_options *options) +ovsdb_jsonrpc_options_to_json(const struct ovsdb_jsonrpc_options *options, + bool jsonrpc_session_only) { struct json *json = json_object_create(); @@ -239,9 +240,15 @@ ovsdb_jsonrpc_options_to_json(const struct ovsdb_jsonrpc_options *options) json_integer_create(options->max_backoff)); json_object_put(json, "inactivity-probe", json_integer_create(options->probe_interval)); + json_object_put(json, "dscp", json_integer_create(options->dscp)); + + if (jsonrpc_session_only) { + /* Caller is not interested in OVSDB-specific options. */ + return json; + } + json_object_put(json, "read-only", json_boolean_create(options->read_only)); - json_object_put(json, "dscp", json_integer_create(options->dscp)); if (options->role) { json_object_put(json, "role", json_string_create(options->role)); } @@ -251,7 +258,8 @@ ovsdb_jsonrpc_options_to_json(const struct ovsdb_jsonrpc_options *options) void ovsdb_jsonrpc_options_update_from_json(struct ovsdb_jsonrpc_options *options, - const struct json *json) + const struct json *json, + bool jsonrpc_session_only) { const struct json *max_backoff, *probe_interval, *read_only, *dscp, *role; struct ovsdb_parser parser; @@ -271,22 +279,29 @@ ovsdb_jsonrpc_options_update_from_json(struct ovsdb_jsonrpc_options *options, options->probe_interval = json_integer(probe_interval); } + dscp = ovsdb_parser_member(&parser, "dscp", OP_INTEGER | OP_OPTIONAL); + if (dscp) { + options->dscp = json_integer(dscp); + } + + if (jsonrpc_session_only) { + /* Caller is not interested to OVSDB-specific options. */ + goto exit; + } + read_only = ovsdb_parser_member(&parser, "read-only", OP_BOOLEAN | OP_OPTIONAL); if (read_only) { options->read_only = json_boolean(read_only); } - dscp = ovsdb_parser_member(&parser, "dscp", OP_INTEGER | OP_OPTIONAL); - if (dscp) { - options->dscp = json_integer(dscp); - } - role = ovsdb_parser_member(&parser, "role", OP_STRING | OP_OPTIONAL); if (role) { + free(options->role); options->role = nullable_xstrdup(json_string(role)); } +exit: error = ovsdb_parser_finish(&parser); if (error) { char *s = ovsdb_error_to_string_free(error); diff --git a/ovsdb/jsonrpc-server.h b/ovsdb/jsonrpc-server.h index 39366ad70..0d8f87da7 100644 --- a/ovsdb/jsonrpc-server.h +++ b/ovsdb/jsonrpc-server.h @@ -45,10 +45,11 @@ struct ovsdb_jsonrpc_options *ovsdb_jsonrpc_options_clone( const struct ovsdb_jsonrpc_options *); struct json *ovsdb_jsonrpc_options_to_json( - const struct ovsdb_jsonrpc_options *) + const struct ovsdb_jsonrpc_options *, bool jsonrpc_session_only) OVS_WARN_UNUSED_RESULT; void ovsdb_jsonrpc_options_update_from_json(struct ovsdb_jsonrpc_options *, - const struct json *); + const struct json *, + bool jsonrpc_session_only); void ovsdb_jsonrpc_server_set_remotes(struct ovsdb_jsonrpc_server *, const struct shash *); diff --git a/ovsdb/ovsdb-server.c b/ovsdb/ovsdb-server.c index 9a3b0add1..d7a823220 100644 --- a/ovsdb/ovsdb-server.c +++ b/ovsdb/ovsdb-server.c @@ -42,6 +42,7 @@ #include "ovsdb-data.h" #include "ovsdb-types.h" #include "ovsdb-error.h" +#include "ovsdb-parser.h" #include "openvswitch/poll-loop.h" #include "process.h" #include "replication.h" @@ -65,12 +66,6 @@ VLOG_DEFINE_THIS_MODULE(ovsdb_server); -struct db { - char *filename; - struct ovsdb *db; - struct uuid row_uuid; -}; - /* SSL configuration. */ static char *private_key_file; static char *certificate_file; @@ -100,16 +95,79 @@ static unixctl_cb_func ovsdb_server_get_sync_exclude_tables; static unixctl_cb_func ovsdb_server_get_sync_status; static unixctl_cb_func ovsdb_server_get_db_storage_status; +#define SERVICE_MODELS \ + SERVICE_MODEL(UNDEFINED, undefined) \ + SERVICE_MODEL(STANDALONE, standalone) \ + SERVICE_MODEL(CLUSTERED, clustered) \ + SERVICE_MODEL(ACTIVE_BACKUP, active-backup) \ + SERVICE_MODEL(RELAY, relay) + +enum service_model { +#define SERVICE_MODEL(ENUM, NAME) SM_##ENUM, + SERVICE_MODELS +#undef SERVICE_MODEL +}; + +static const char * +service_model_to_string(enum service_model model) +{ + switch (model) { +#define SERVICE_MODEL(ENUM, NAME) \ + case SM_##ENUM: return #NAME; + SERVICE_MODELS +#undef SERVICE_MODEL + default: OVS_NOT_REACHED(); + } +} + +static enum service_model +service_model_from_string(const char *model) +{ +#define SERVICE_MODEL(ENUM, NAME) \ + if (!strcmp(model, #NAME)) { \ + return SM_##ENUM; \ + } + SERVICE_MODELS +#undef SERVICE_MODEL + + VLOG_WARN("Unrecognized database service model: '%s'", model); + + return SM_UNDEFINED; +} + +struct db_config { + enum service_model model; + char *source; /* sync-from for backup or relay source. */ + struct ovsdb_jsonrpc_options *options; /* For 'source' connection. */ + + /* Configuration specific to SM_ACTIVE_BACKUP. */ + struct { + char *sync_exclude; /* Tables to exclude. */ + bool backup; /* If true, the database is read-only and receives + * updates from the 'source'. */ + } ab; +}; + +struct db { + struct ovsdb *db; + char *filename; + struct db_config *config; + struct uuid row_uuid; +}; + struct server_config { struct shash *remotes; - struct shash *all_dbs; - FILE *config_tmpfile; + struct shash *all_dbs; /* All the currently serviced databases. + * 'struct db' by a schema name. */ + struct ovsdb_jsonrpc_server *jsonrpc; + + /* Command line + appctl configuration. */ char **sync_from; char **sync_exclude; bool *is_backup; int *replication_probe_interval; int *relay_source_probe_interval; - struct ovsdb_jsonrpc_server *jsonrpc; + FILE *config_tmpfile; }; static unixctl_cb_func ovsdb_server_add_remote; static unixctl_cb_func ovsdb_server_remove_remote; @@ -123,14 +181,15 @@ static unixctl_cb_func ovsdb_server_tlog_list; static void read_db(struct server_config *, struct db *); static struct ovsdb_error *open_db(struct server_config *, - const char *filename) + const char *filename, + const struct db_config *) OVS_WARN_UNUSED_RESULT; static void add_server_db(struct server_config *); static void remove_db(struct server_config *, struct shash_node *db, char *); static void close_db(struct server_config *, struct db *, char *); static void parse_options(int argc, char *argvp[], - struct sset *db_filenames, struct shash *remotes, + struct shash *db_conf, struct shash *remotes, char **unixctl_pathp, char **run_command, char **sync_from, char **sync_exclude, bool *is_backup); @@ -153,29 +212,14 @@ static void update_remote_status(const struct ovsdb_jsonrpc_server *jsonrpc, static void update_server_status(struct shash *all_dbs); static void save_config__(FILE *config_file, const struct shash *remotes, - const struct sset *db_filenames, + const struct shash *db_conf, const char *sync_from, const char *sync_exclude, bool is_backup); static void save_config(struct server_config *); static void load_config(FILE *config_file, struct shash *remotes, - struct sset *db_filenames, char **sync_from, + struct shash *db_conf, char **sync_from, char **sync_exclude, bool *is_backup); -static void -ovsdb_replication_init(const char *sync_from, const char *exclude, - struct shash *all_dbs, const struct uuid *server_uuid, - int probe_interval) -{ - struct shash_node *node; - SHASH_FOR_EACH (node, all_dbs) { - struct db *db = node->data; - if (node->name[0] != '_' && db->db) { - replication_set_db(db->db, sync_from, exclude, - server_uuid, probe_interval); - } - } -} - static void log_and_free_error(struct ovsdb_error *error) { @@ -186,11 +230,52 @@ log_and_free_error(struct ovsdb_error *error) } } +static void +ovsdb_server_replication_remove_db(struct db *db) +{ + replication_remove_db(db->db); + db->config->ab.backup = false; +} + +static void +ovsdb_server_replication_run(struct server_config *config) +{ + struct shash_node *node; + bool all_alive = true; + + replication_run(); + + SHASH_FOR_EACH (node, config->all_dbs) { + struct db *db = node->data; + + if (db->config->model == SM_ACTIVE_BACKUP && db->config->ab.backup + && !replication_is_alive(db->db)) { + ovsdb_server_replication_remove_db(db); + all_alive = false; + } + } + + /* If one connection is broken, switch all databases to active, + * since they are configured via the same command line / appctl. */ + if (!all_alive && *config->is_backup) { + *config->is_backup = false; + + SHASH_FOR_EACH (node, config->all_dbs) { + struct db *db = node->data; + + if (db->config->model == SM_ACTIVE_BACKUP + && db->config->ab.backup) { + ovsdb_server_replication_remove_db(db); + } + } + } +} + static void main_loop(struct server_config *config, struct ovsdb_jsonrpc_server *jsonrpc, struct shash *all_dbs, struct unixctl_server *unixctl, struct shash *remotes, - struct process *run_process, bool *exiting, bool *is_backup) + struct process *run_process, bool *exiting) { char *remotes_error, *ssl_error; struct shash_node *node; @@ -220,7 +305,7 @@ main_loop(struct server_config *config, * the set of remotes that reconfigure_remotes() uses. */ unixctl_server_run(unixctl); - ovsdb_jsonrpc_server_set_read_only(jsonrpc, *is_backup); + ovsdb_jsonrpc_server_set_read_only(jsonrpc, false); report_error_if_changed( reconfigure_remotes(jsonrpc, all_dbs, remotes), @@ -228,23 +313,7 @@ main_loop(struct server_config *config, report_error_if_changed(reconfigure_ssl(all_dbs), &ssl_error); ovsdb_jsonrpc_server_run(jsonrpc); - replication_run(); - if (*is_backup) { - SHASH_FOR_EACH (node, all_dbs) { - struct db *db = node->data; - if (db->db->name[0] != '_' && !replication_is_alive(db->db)) { - *is_backup = false; - break; - } - } - if (!*is_backup) { - SHASH_FOR_EACH (node, all_dbs) { - struct db *db = node->data; - replication_remove_db(db->db); - } - } - } - + ovsdb_server_replication_run(config); ovsdb_relay_run(); SHASH_FOR_EACH_SAFE (node, all_dbs) { @@ -351,6 +420,92 @@ parse_relay_args(const char *arg, char **name, char **remote) return true; } +static void +db_config_destroy(struct db_config *conf) +{ + if (!conf) { + return; + } + + free(conf->source); + if (conf->options) { + free(conf->options->role); + free(conf->options); + } + free(conf->ab.sync_exclude); + free(conf); +} + +static struct db_config * +db_config_clone(const struct db_config *c) +{ + struct db_config *conf = xmemdup(c, sizeof *c); + + conf->source = nullable_xstrdup(c->source); + if (c->options) { + conf->options = ovsdb_jsonrpc_options_clone(c->options); + } + conf->ab.sync_exclude = nullable_xstrdup(c->ab.sync_exclude); + + return conf; +} + +static struct ovsdb_jsonrpc_options * +get_jsonrpc_options(const char *target, enum service_model model) +{ + struct ovsdb_jsonrpc_options *options; + + options = ovsdb_jsonrpc_default_options(target); + if (model == SM_ACTIVE_BACKUP) { + options->probe_interval = REPLICATION_DEFAULT_PROBE_INTERVAL; + } else if (model == SM_RELAY) { + options->probe_interval = RELAY_SOURCE_DEFAULT_PROBE_INTERVAL; + } + + return options; +} + +static void +add_database_config(struct shash *db_conf, const char *opt, + const char *sync_from, const char *sync_exclude, + bool active) +{ + struct db_config *conf = xzalloc(sizeof *conf); + char *filename = NULL; + + if (parse_relay_args(opt, &filename, &conf->source)) { + conf->model = SM_RELAY; + conf->options = get_jsonrpc_options(conf->source, conf->model); + } else if (sync_from) { + conf->model = SM_ACTIVE_BACKUP; + conf->source = xstrdup(sync_from); + conf->options = get_jsonrpc_options(conf->source, conf->model); + conf->ab.sync_exclude = nullable_xstrdup(sync_exclude); + conf->ab.backup = !active; + filename = xstrdup(opt); + } else { + conf->model = SM_UNDEFINED; /* We'll update once the file is open. */ + filename = xstrdup(opt); + } + + conf = shash_replace_nocopy(db_conf, filename, conf); + if (conf) { + VLOG_WARN("Duplicate database configuration: %s", filename); + db_config_destroy(conf); + } +} + +static void +free_database_configs(struct shash *db_conf) +{ + struct shash_node *node; + + SHASH_FOR_EACH (node, db_conf) { + db_config_destroy(node->data); + } + shash_clear(db_conf); +} + int main(int argc, char *argv[]) { @@ -358,11 +513,10 @@ main(int argc, char *argv[]) char *run_command = NULL; struct unixctl_server *unixctl; struct ovsdb_jsonrpc_server *jsonrpc; - struct sset db_filenames; + struct shash db_conf; struct shash remotes; char *sync_from, *sync_exclude; bool is_backup; - const char *db_filename; struct process *run_process; bool exiting; int retval; @@ -381,7 +535,7 @@ main(int argc, char *argv[]) dns_resolve_init(true); bool active = false; - parse_options(argc, argv, &db_filenames, &remotes, &unixctl_path, + parse_options(argc, argv, &db_conf, &remotes, &unixctl_path, &run_command, &sync_from, &sync_exclude, &active); is_backup = sync_from && !active; @@ -400,13 +554,15 @@ main(int argc, char *argv[]) server_config.remotes = &remotes; server_config.config_tmpfile = config_tmpfile; - save_config__(config_tmpfile, &remotes, &db_filenames, sync_from, + save_config__(config_tmpfile, &remotes, &db_conf, sync_from, sync_exclude, is_backup); + free_remotes(&remotes); + free_database_configs(&db_conf); daemonize_start(false, false); /* Load the saved config. */ - load_config(config_tmpfile, &remotes, &db_filenames, &sync_from, + load_config(config_tmpfile, &remotes, &db_conf, &sync_from, &sync_exclude, &is_backup); /* Start ovsdb jsonrpc server. When running as a backup server, @@ -425,13 +581,16 @@ main(int argc, char *argv[]) perf_counters_init(); - SSET_FOR_EACH (db_filename, &db_filenames) { - struct ovsdb_error *error = open_db(&server_config, db_filename); + SHASH_FOR_EACH (node, &db_conf) { + struct ovsdb_error *error = open_db(&server_config, + node->name, node->data); if (error) { char *s = ovsdb_error_to_string_free(error); ovs_fatal(0, "%s", s); } + db_config_destroy(node->data); } + shash_clear(&db_conf); add_server_db(&server_config); char *error = reconfigure_remotes(jsonrpc, &all_dbs, &remotes); @@ -538,15 +697,8 @@ main(int argc, char *argv[]) unixctl_command_register("ovsdb-server/disable-monitor-cond", "", 0, 0, ovsdb_server_disable_monitor_cond, jsonrpc); - if (is_backup) { - const struct uuid *server_uuid; - server_uuid = ovsdb_jsonrpc_server_get_uuid(jsonrpc); - ovsdb_replication_init(sync_from, sync_exclude, &all_dbs, server_uuid, - replication_probe_interval); - } - main_loop(&server_config, jsonrpc, &all_dbs, unixctl, &remotes, - run_process, &exiting, &is_backup); + run_process, &exiting); SHASH_FOR_EACH_SAFE (node, &all_dbs) { struct db *db = node->data; @@ -556,7 +708,8 @@ main(int argc, char *argv[]) ovsdb_jsonrpc_server_destroy(jsonrpc); shash_destroy(&all_dbs); free_remotes(&remotes); - sset_destroy(&db_filenames); + free_database_configs(&db_conf); + shash_destroy(&db_conf); free(sync_from); free(sync_exclude); unixctl_server_destroy(unixctl); @@ -580,7 +733,7 @@ main(int argc, char *argv[]) * * "False negatives" are possible. */ static bool -is_already_open(struct server_config *config OVS_UNUSED, +is_already_open(struct server_config *server_config OVS_UNUSED, const char *filename OVS_UNUSED) { #ifndef _WIN32 @@ -589,11 +742,12 @@ is_already_open(struct server_config *config OVS_UNUSED, if (!stat(filename, &s)) { struct shash_node *node; - SHASH_FOR_EACH (node, config->all_dbs) { + SHASH_FOR_EACH (node, server_config->all_dbs) { struct db *db = node->data; struct stat s2; - if (!stat(db->filename, &s2) + if (db->config->model != SM_RELAY + && !stat(db->filename, &s2) && s.st_dev == s2.st_dev && s.st_ino == s2.st_ino) { return true; @@ -606,16 +760,19 @@ is_already_open(struct server_config *config OVS_UNUSED, } static void -close_db(struct server_config *config, struct db *db, char *comment) +close_db(struct server_config *server_config, struct db *db, char *comment) { if (db) { - ovsdb_jsonrpc_server_remove_db(config->jsonrpc, db->db, comment); - if (db->db->is_relay) { + ovsdb_jsonrpc_server_remove_db(server_config->jsonrpc, + db->db, comment); + if (db->config->model == SM_RELAY) { ovsdb_relay_del_db(db->db); } - if (*config->is_backup) { - replication_remove_db(db->db); + if (db->config->model == SM_ACTIVE_BACKUP + && db->config->ab.backup) { + ovsdb_server_replication_remove_db(db); } + db_config_destroy(db->config); ovsdb_destroy(db->db); free(db->filename); free(db); @@ -768,20 +925,17 @@ add_db(struct server_config *config, struct db *db) } static struct ovsdb_error * OVS_WARN_UNUSED_RESULT -open_db(struct server_config *config, const char *filename) +open_db(struct server_config *server_config, + const char *filename, const struct db_config *conf) { struct ovsdb_storage *storage; - char *relay_remotes = NULL; struct ovsdb_error *error; - bool is_relay; - char *name; - is_relay = parse_relay_args(filename, &name, &relay_remotes); - if (!is_relay) { + if (conf->model != SM_RELAY) { /* If we know that the file is already open, return a good error * message. Otherwise, if the file is open, we'll fail later on with * a harder to interpret file locking error. */ - if (is_already_open(config, filename)) { + if (is_already_open(server_config, filename)) { return ovsdb_error(NULL, "%s: already open", filename); } @@ -789,59 +943,78 @@ open_db(struct server_config *config, const char *filename) if (error) { return error; } - name = xstrdup(filename); } else { - storage = ovsdb_storage_create_unbacked(name); + storage = ovsdb_storage_create_unbacked(filename); + } + + enum service_model model = conf->model; + if (model == SM_UNDEFINED || model == SM_STANDALONE + || model == SM_CLUSTERED) { + /* Check the actual service model from the storage. */ + model = ovsdb_storage_is_clustered(storage) + ? SM_CLUSTERED : SM_STANDALONE; + } + if (conf->model != SM_UNDEFINED && conf->model != model) { + ovsdb_storage_close(storage); + return ovsdb_error(NULL, "%s: database is %s and not %s", + filename, service_model_to_string(model), + service_model_to_string(conf->model)); } struct ovsdb_schema *schema; - if (is_relay || ovsdb_storage_is_clustered(storage)) { + if (model == SM_RELAY || model == SM_CLUSTERED) { schema = NULL; } else { struct json *txn_json; error = ovsdb_storage_read(storage, &schema, &txn_json, NULL); if (error) { ovsdb_storage_close(storage); - free(name); return error; } ovs_assert(schema && !txn_json); } struct db *db = xzalloc(sizeof *db); - db->filename = name; + db->filename = xstrdup(filename); + db->config = db_config_clone(conf); + db->config->model = model; db->db = ovsdb_create(schema, storage); - ovsdb_jsonrpc_server_add_db(config->jsonrpc, db->db); + ovsdb_jsonrpc_server_add_db(server_config->jsonrpc, db->db); /* Enable txn history for clustered and relay modes. It is not enabled for * other modes for now, since txn id is available for clustered and relay * modes only. */ - ovsdb_txn_history_init(db->db, - is_relay || ovsdb_storage_is_clustered(storage)); + ovsdb_txn_history_init(db->db, model == SM_RELAY || model == SM_CLUSTERED); - read_db(config, db); + read_db(server_config, db); error = (db->db->name[0] == '_' ? ovsdb_error(NULL, "%s: names beginning with \"_\" are reserved", db->db->name) - : shash_find(config->all_dbs, db->db->name) + : shash_find(server_config->all_dbs, db->db->name) ? ovsdb_error(NULL, "%s: duplicate database name", db->db->name) : NULL); if (error) { char *error_s = ovsdb_error_to_string(error); - close_db(config, db, + close_db(server_config, db, xasprintf("cannot complete opening %s database (%s)", db->db->name, error_s)); free(error_s); return error; } - add_db(config, db); + add_db(server_config, db); + + if (model == SM_RELAY) { + ovsdb_relay_add_db(db->db, conf->source, update_schema, server_config, + conf->options->probe_interval); + } + if (model == SM_ACTIVE_BACKUP && conf->ab.backup) { + const struct uuid *server_uuid; - if (is_relay) { - ovsdb_relay_add_db(db->db, relay_remotes, update_schema, config, - *config->relay_source_probe_interval); - free(relay_remotes); + server_uuid = ovsdb_jsonrpc_server_get_uuid(server_config->jsonrpc); + replication_set_db(db->db, conf->source, conf->ab.sync_exclude, + server_uuid, conf->options->probe_interval); } return NULL; } @@ -865,6 +1038,8 @@ add_server_db(struct server_config *config) /* We don't need txn_history for server_db. */ db->filename = xstrdup(""); + db->config = xzalloc(sizeof *db->config); + db->config->model = SM_UNDEFINED; db->db = ovsdb_create(schema, ovsdb_storage_create_unbacked(NULL)); db->db->read_only = true; @@ -1457,11 +1632,20 @@ ovsdb_server_set_active_ovsdb_server(struct unixctl_conn *conn, void *config_) { struct server_config *config = config_; + struct shash_node *node; - if (*config->sync_from) { - free(*config->sync_from); - } + free(*config->sync_from); *config->sync_from = xstrdup(argv[1]); + + SHASH_FOR_EACH (node, config->all_dbs) { + struct db *db = node->data; + + if (db->config->model == SM_ACTIVE_BACKUP) { + free(db->config->source); + db->config->source = xstrdup(argv[1]); + } + } + save_config(config); unixctl_command_reply(conn, NULL); @@ -1485,20 +1669,39 @@ ovsdb_server_connect_active_ovsdb_server(struct unixctl_conn *conn, void *config_) { struct server_config *config = config_; + struct shash_node *node; char *msg = NULL; - if ( !*config->sync_from) { + if (!*config->sync_from) { msg = "Unable to connect: active server is not specified.\n"; } else { const struct uuid *server_uuid; server_uuid = ovsdb_jsonrpc_server_get_uuid(config->jsonrpc); - ovsdb_replication_init(*config->sync_from, *config->sync_exclude, - config->all_dbs, server_uuid, - *config->replication_probe_interval); - if (!*config->is_backup) { - *config->is_backup = true; - save_config(config); + + SHASH_FOR_EACH (node, config->all_dbs) { + struct db *db = node->data; + struct db_config *conf = db->config; + + /* This command also converts standalone databases to AB. */ + if (conf->model == SM_STANDALONE) { + conf->model = SM_ACTIVE_BACKUP; + conf->source = xstrdup(*config->sync_from); + conf->options = ovsdb_jsonrpc_default_options(conf->source); + conf->options->probe_interval = + *config->replication_probe_interval; + conf->ab.sync_exclude = + nullable_xstrdup(*config->sync_exclude); + conf->ab.backup = false; + } + + if (conf->model == SM_ACTIVE_BACKUP && !conf->ab.backup) { + replication_set_db(db->db, conf->source, conf->ab.sync_exclude, + server_uuid, conf->options->probe_interval); + conf->ab.backup = true; + } } + *config->is_backup = true; + save_config(config); } unixctl_command_reply(conn, msg); } @@ -1514,7 +1717,11 @@ ovsdb_server_disconnect_active_ovsdb_server(struct unixctl_conn *conn, SHASH_FOR_EACH (node, config->all_dbs) { struct db *db = node->data; - replication_remove_db(db->db); + struct db_config *conf = db->config; + + if (conf->model == SM_ACTIVE_BACKUP && conf->ab.backup) { + ovsdb_server_replication_remove_db(db); + } } *config->is_backup = false; save_config(config); @@ -1528,23 +1735,35 @@ ovsdb_server_set_active_ovsdb_server_probe_interval(struct unixctl_conn *conn, void *config_) { struct server_config *config = config_; - + struct shash_node *node; int probe_interval; - if (str_to_int(argv[1], 10, &probe_interval)) { - *config->replication_probe_interval = probe_interval; - save_config(config); - if (*config->is_backup) { - const struct uuid *server_uuid; - server_uuid = ovsdb_jsonrpc_server_get_uuid(config->jsonrpc); - ovsdb_replication_init(*config->sync_from, *config->sync_exclude, - config->all_dbs, server_uuid, - *config->replication_probe_interval); - } - unixctl_command_reply(conn, NULL); - } else { - unixctl_command_reply( + + if (!str_to_int(argv[1], 10, &probe_interval)) { + unixctl_command_reply_error( conn, "Invalid probe interval, integer value expected"); + return; + } + + const struct uuid *server_uuid; + server_uuid = ovsdb_jsonrpc_server_get_uuid(config->jsonrpc); + + *config->replication_probe_interval = probe_interval; + + SHASH_FOR_EACH (node, config->all_dbs) { + struct db *db = node->data; + struct db_config *conf = db->config; + + if (conf->model == SM_ACTIVE_BACKUP) { + conf->options->probe_interval = probe_interval; + if (conf->ab.backup) { + replication_set_db(db->db, conf->source, conf->ab.sync_exclude, + server_uuid, conf->options->probe_interval); + } + } } + + save_config(config); + unixctl_command_reply(conn, NULL); } static void @@ -1554,17 +1773,30 @@ ovsdb_server_set_relay_source_interval(struct unixctl_conn *conn, void *config_) { struct server_config *config = config_; + struct shash_node *node; int probe_interval; - if (str_to_int(argv[1], 10, &probe_interval)) { - *config->relay_source_probe_interval = probe_interval; - save_config(config); - ovsdb_relay_set_probe_interval(probe_interval); - unixctl_command_reply(conn, NULL); - } else { + if (!str_to_int(argv[1], 10, &probe_interval)) { unixctl_command_reply_error( conn, "Invalid probe interval, integer value expected"); + return; + } + + *config->relay_source_probe_interval = probe_interval; + + SHASH_FOR_EACH (node, config->all_dbs) { + struct db *db = node->data; + struct db_config *conf = db->config; + + if (conf->model == SM_RELAY) { + conf->options->probe_interval = probe_interval; + } } + + ovsdb_relay_set_probe_interval(probe_interval); + save_config(config); + + unixctl_command_reply(conn, NULL); } static void @@ -1574,20 +1806,36 @@ ovsdb_server_set_sync_exclude_tables(struct unixctl_conn *conn, void *config_) { struct server_config *config = config_; + struct shash_node *node; char *err = parse_excluded_tables(argv[1]); - if (!err) { - free(*config->sync_exclude); - *config->sync_exclude = xstrdup(argv[1]); - save_config(config); - if (*config->is_backup) { - const struct uuid *server_uuid; - server_uuid = ovsdb_jsonrpc_server_get_uuid(config->jsonrpc); - ovsdb_replication_init(*config->sync_from, *config->sync_exclude, - config->all_dbs, server_uuid, - *config->replication_probe_interval); + if (err) { + goto exit; + } + + const struct uuid *server_uuid; + server_uuid = ovsdb_jsonrpc_server_get_uuid(config->jsonrpc); + + free(*config->sync_exclude); + *config->sync_exclude = xstrdup(argv[1]); + + SHASH_FOR_EACH (node, config->all_dbs) { + struct db *db = node->data; + struct db_config *conf = db->config; + + if (conf->model == SM_ACTIVE_BACKUP) { + free(conf->ab.sync_exclude); + conf->ab.sync_exclude = xstrdup(argv[1]); + if (conf->ab.backup) { + replication_set_db(db->db, conf->source, conf->ab.sync_exclude, + server_uuid, conf->options->probe_interval); + } } } + + save_config(config); + +exit: unixctl_command_reply(conn, err); free(err); } @@ -1832,22 +2080,26 @@ ovsdb_server_add_database(struct unixctl_conn *conn, int argc OVS_UNUSED, { struct server_config *config = config_; const char *filename = argv[1]; + const struct shash_node *node; + struct shash db_conf; + + shash_init(&db_conf); + add_database_config(&db_conf, filename, *config->sync_from, + *config->sync_exclude, !config->is_backup); + ovs_assert(shash_count(&db_conf) == 1); + node = shash_first(&db_conf); - char *error = ovsdb_error_to_string_free(open_db(config, filename)); + char *error = ovsdb_error_to_string_free(open_db(config, + node->name, node->data)); if (!error) { save_config(config); - if (*config->is_backup) { - const struct uuid *server_uuid; - server_uuid = ovsdb_jsonrpc_server_get_uuid(config->jsonrpc); - ovsdb_replication_init(*config->sync_from, *config->sync_exclude, - config->all_dbs, server_uuid, - *config->replication_probe_interval); - } unixctl_command_reply(conn, NULL); } else { unixctl_command_reply_error(conn, error); free(error); } + db_config_destroy(node->data); + shash_destroy(&db_conf); } static void @@ -1994,23 +2246,34 @@ ovsdb_server_get_sync_status(struct unixctl_conn *conn, int argc OVS_UNUSED, const char *argv[] OVS_UNUSED, void *config_) { struct server_config *config = config_; - bool is_backup = *config->is_backup; struct ds ds = DS_EMPTY_INITIALIZER; + bool any_backup = false; - ds_put_format(&ds, "state: %s\n", is_backup ? "backup" : "active"); + const struct shash_node **db_nodes = shash_sort(config->all_dbs); - if (is_backup) { - const struct shash_node **db_nodes = shash_sort(config->all_dbs); + for (size_t i = 0; i < shash_count(config->all_dbs); i++) { + const struct db *db = db_nodes[i]->data; - for (size_t i = 0; i < shash_count(config->all_dbs); i++) { - const struct db *db = db_nodes[i]->data; + if (db->config->model != SM_ACTIVE_BACKUP) { + continue; + } - if (db->db && db->db->name[0] != '_') { - ds_put_and_free_cstr(&ds, replication_status(db->db)); - ds_put_char(&ds, '\n'); - } + any_backup = true; + + ds_put_format(&ds, "database: %s\n", db->db->name); + ds_put_format(&ds, "state: %s\n", + db->config->ab.backup ? "backup" : "active"); + if (db->config->ab.backup) { + ds_put_and_free_cstr(&ds, replication_status(db->db)); } - free(db_nodes); + if (i + 1 < shash_count(config->all_dbs)) { + ds_put_char(&ds, '\n'); + } + } + free(db_nodes); + + if (!any_backup) { + ds_put_cstr(&ds, "state: active\n"); } unixctl_command_reply(conn, ds_cstr(&ds)); @@ -2054,7 +2317,7 @@ ovsdb_server_get_db_storage_status(struct unixctl_conn *conn, static void parse_options(int argc, char *argv[], - struct sset *db_filenames, struct shash *remotes, + struct shash *db_conf, struct shash *remotes, char **unixctl_pathp, char **run_command, char **sync_from, char **sync_exclude, bool *active) { @@ -2104,7 +2367,7 @@ parse_options(int argc, char *argv[], *sync_from = NULL; *sync_exclude = NULL; - sset_init(db_filenames); + shash_init(db_conf); shash_init(remotes); for (;;) { int c; @@ -2210,10 +2473,15 @@ parse_options(int argc, char *argv[], argv += optind; if (argc > 0) { for (int i = 0; i < argc; i++) { - sset_add(db_filenames, argv[i]); + add_database_config(db_conf, argv[i], *sync_from, *sync_exclude, + *active); } } else if (add_default_db) { - sset_add_and_free(db_filenames, xasprintf("%s/conf.db", ovs_dbdir())); + char *filename = xasprintf("%s/conf.db", ovs_dbdir()); + + add_database_config(db_conf, filename, *sync_from, *sync_exclude, + *active); + free(filename); } } @@ -2264,16 +2532,63 @@ remotes_to_json(const struct shash *remotes) json = json_object_create(); SHASH_FOR_EACH (node, remotes) { json_object_put(json, node->name, - ovsdb_jsonrpc_options_to_json(node->data)); + ovsdb_jsonrpc_options_to_json(node->data, false)); + } + return json; +} + +static struct json * +db_config_to_json(const struct db_config *conf) +{ + struct json *json; + + json = json_object_create(); + + if (conf->model != SM_UNDEFINED) { + json_object_put(json, "service-model", + json_string_create( + service_model_to_string(conf->model))); + } + + if (conf->source) { + struct json *source = json_object_create(); + + json_object_put(source, conf->source, + ovsdb_jsonrpc_options_to_json(conf->options, true)); + json_object_put(json, "source", source); + } + + if (conf->model == SM_ACTIVE_BACKUP) { + if (conf->ab.sync_exclude) { + struct sset set = SSET_INITIALIZER(&set); + + sset_from_delimited_string(&set, conf->ab.sync_exclude, " ,"); + json_object_put(json, "exclude-tables", sset_to_json(&set)); + sset_destroy(&set); + } + json_object_put(json, "backup", json_boolean_create(conf->ab.backup)); + } + return json; +} + +static struct json * +databases_to_json(const struct shash *db_conf) +{ + const struct shash_node *node; + struct json *json; + + json = json_object_create(); + SHASH_FOR_EACH (node, db_conf) { + json_object_put(json, node->name, db_config_to_json(node->data)); } return json; } /* Truncates and replaces the contents of 'config_file' by a representation of - * 'remotes' and 'db_filenames'. */ + * 'remotes', 'db_conf' and a few global replication paramaters. */ static void save_config__(FILE *config_file, const struct shash *remotes, - const struct sset *db_filenames, const char *sync_from, + const struct shash *db_conf, const char *sync_from, const char *sync_exclude, bool is_backup) { struct json *obj; @@ -2286,7 +2601,8 @@ save_config__(FILE *config_file, const struct shash *remotes, obj = json_object_create(); json_object_put(obj, "remotes", remotes_to_json(remotes)); - json_object_put(obj, "db_filenames", sset_to_json(db_filenames)); + json_object_put(obj, "databases", databases_to_json(db_conf)); + if (sync_from) { json_object_put(obj, "sync_from", json_string_create(sync_from)); } @@ -2312,56 +2628,147 @@ save_config__(FILE *config_file, const struct shash *remotes, static void save_config(struct server_config *config) { - struct sset db_filenames; struct shash_node *node; + struct shash db_conf; - sset_init(&db_filenames); + shash_init(&db_conf); SHASH_FOR_EACH (node, config->all_dbs) { struct db *db = node->data; + if (node->name[0] != '_') { - sset_add(&db_filenames, db->filename); + shash_add(&db_conf, db->filename, db->config); } } - save_config__(config->config_tmpfile, config->remotes, &db_filenames, + save_config__(config->config_tmpfile, config->remotes, &db_conf, *config->sync_from, *config->sync_exclude, *config->is_backup); - sset_destroy(&db_filenames); + shash_destroy(&db_conf); } static void -sset_from_json(struct sset *sset, const struct json *array) +remotes_from_json(struct shash *remotes, const struct json *json) { - size_t i; + struct ovsdb_jsonrpc_options *options; + const struct shash_node *node; + const struct shash *object; - sset_clear(sset); + free_remotes(remotes); - ovs_assert(array); - ovs_assert(array->type == JSON_ARRAY); - for (i = 0; i < array->array.n; i++) { - const struct json *elem = array->array.elems[i]; - sset_add(sset, json_string(elem)); + ovs_assert(json); + ovs_assert(json->type == JSON_OBJECT); + + object = json_object(json); + SHASH_FOR_EACH (node, object) { + options = ovsdb_jsonrpc_default_options(node->name); + ovsdb_jsonrpc_options_update_from_json(options, node->data, false); + shash_add(remotes, node->name, options); } } +static struct db_config * +db_config_from_json(const char *name, const struct json *json) +{ + const struct json *model, *source, *sync_exclude, *backup; + struct db_config *conf = xzalloc(sizeof *conf); + struct ovsdb_parser parser; + struct ovsdb_error *error; + + ovsdb_parser_init(&parser, json, "database %s", name); + + model = ovsdb_parser_member(&parser, "service-model", + OP_STRING | OP_OPTIONAL); + conf->model = model ? service_model_from_string(json_string(model)) + : SM_UNDEFINED; + + if (conf->model == SM_ACTIVE_BACKUP) { + backup = ovsdb_parser_member(&parser, "backup", OP_BOOLEAN); + conf->ab.backup = backup ? json_boolean(backup) : false; + + sync_exclude = ovsdb_parser_member(&parser, "exclude-tables", + OP_ARRAY | OP_OPTIONAL); + if (sync_exclude) { + const struct json_array *exclude = json_array(sync_exclude); + struct sset set = SSET_INITIALIZER(&set); + + for (size_t i = 0; i < exclude->n; i++) { + if (exclude->elems[i]->type != JSON_STRING) { + ovsdb_parser_raise_error(&parser, + "'exclude-tables' must contain strings"); + break; + } + sset_add(&set, json_string(exclude->elems[i])); + } + conf->ab.sync_exclude = sset_join(&set, ",", ""); + sset_destroy(&set); + } + } + + if (conf->model == SM_ACTIVE_BACKUP || conf->model == SM_RELAY) { + enum ovsdb_parser_types type = OP_OBJECT; + + if (conf->model == SM_ACTIVE_BACKUP && !conf->ab.backup) { + /* Active database doesn't have to have a source. */ + type |= OP_OPTIONAL; + } + source = ovsdb_parser_member(&parser, "source", type); + + if (source && shash_count(json_object(source)) != 1) { + ovsdb_parser_raise_error(&parser, + "'source' should be an object with exactly one element"); + } else if (source) { + const struct shash_node *node = shash_first(json_object(source)); + const struct json *options; + + ovs_assert(node); + conf->source = xstrdup(node->name); + options = node->data; + + conf->options = get_jsonrpc_options(conf->source, conf->model); + + if (options->type == JSON_OBJECT) { + ovsdb_jsonrpc_options_update_from_json(conf->options, + options, true); + } else if (options->type != JSON_NULL) { + ovsdb_parser_raise_error(&parser, + "JSON-RPC options is not a JSON object or null"); + } + } + } + + error = ovsdb_parser_finish(&parser); + if (error) { + char *s = ovsdb_error_to_string_free(error); + + VLOG_WARN("%s", s); + free(s); + db_config_destroy(conf); + return NULL; + } + + return conf; +} + + static void -remotes_from_json(struct shash *remotes, const struct json *json) +databases_from_json(struct shash *db_conf, const struct json *json) { - struct ovsdb_jsonrpc_options *options; const struct shash_node *node; const struct shash *object; - free_remotes(remotes); + free_database_configs(db_conf); ovs_assert(json); ovs_assert(json->type == JSON_OBJECT); object = json_object(json); SHASH_FOR_EACH (node, object) { - options = ovsdb_jsonrpc_default_options(node->name); - ovsdb_jsonrpc_options_update_from_json(options, node->data); - shash_add(remotes, node->name, options); + struct db_config *conf = db_config_from_json(node->name, node->data); + + if (conf) { + shash_add(db_conf, node->name, conf); + } } } @@ -2369,7 +2776,7 @@ remotes_from_json(struct shash *remotes, const struct json *json) * 'config_file', which must have been previously written by save_config(). */ static void load_config(FILE *config_file, struct shash *remotes, - struct sset *db_filenames, char **sync_from, + struct shash *db_conf, char **sync_from, char **sync_exclude, bool *is_backup) { struct json *json; @@ -2384,8 +2791,8 @@ load_config(FILE *config_file, struct shash *remotes, ovs_assert(json->type == JSON_OBJECT); remotes_from_json(remotes, shash_find_data(json_object(json), "remotes")); - sset_from_json(db_filenames, - shash_find_data(json_object(json), "db_filenames")); + databases_from_json(db_conf, + shash_find_data(json_object(json), "databases")); struct json *string; string = shash_find_data(json_object(json), "sync_from"); diff --git a/ovsdb/replication.c b/ovsdb/replication.c index 3c59d4039..3a062b078 100644 --- a/ovsdb/replication.c +++ b/ovsdb/replication.c @@ -795,7 +795,6 @@ replication_status(const struct ovsdb *db) bool alive = rdb->session && jsonrpc_session_is_alive(rdb->session); struct ds ds = DS_EMPTY_INITIALIZER; - ds_put_format(&ds, "database: %s\n", db->name); if (alive) { switch (rdb->state) { case RPL_S_INIT: diff --git a/tests/ovsdb-server.at b/tests/ovsdb-server.at index 6eb758e22..45aa80cd6 100644 --- a/tests/ovsdb-server.at +++ b/tests/ovsdb-server.at @@ -1988,7 +1988,9 @@ OVS_WAIT_UNTIL([ovs-appctl -t "`pwd`"/unixctl2 ovsdb-server/sync-status |grep re dnl Switch the 'db1' to active AT_CHECK([ovs-appctl -t "`pwd`"/unixctl ovsdb-server/disconnect-active-ovsdb-server]) -AT_CHECK([ovs-appctl -t "`pwd`"/unixctl ovsdb-server/sync-status], [0], [state: active +AT_CHECK([ovs-appctl -t "`pwd`"/unixctl ovsdb-server/sync-status], [0], [dnl +database: mydb +state: active ]) dnl Issue a transaction to 'db1' @@ -2007,7 +2009,9 @@ AT_CHECK([ovs-appctl -t "`pwd`"/unixctl ovsdb-server/connect-active-ovsdb-server dnl Verify the change happend OVS_WAIT_UNTIL([ovs-appctl -t "`pwd`"/unixctl ovsdb-server/sync-status |grep replicating]) -AT_CHECK([ovs-appctl -t "`pwd`"/unixctl2 ovsdb-server/sync-status], [0], [state: active +AT_CHECK([ovs-appctl -t "`pwd`"/unixctl2 ovsdb-server/sync-status], [0], [dnl +database: mydb +state: active ]) dnl Issue an transaction to 'db2' which is now active.