From patchwork Wed Jul 13 18:27:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Van Haaren, Harry" X-Patchwork-Id: 1656118 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=gEVqpTli; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LjmMw6k2Sz9sB4 for ; Thu, 14 Jul 2022 04:28:28 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id C90D8415A8; Wed, 13 Jul 2022 18:28:26 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org C90D8415A8 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=gEVqpTli X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S5PjZuCrssEB; Wed, 13 Jul 2022 18:28:25 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 7161E40A67; Wed, 13 Jul 2022 18:28:24 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 7161E40A67 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2C23CC0035; Wed, 13 Jul 2022 18:28:24 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2978CC002D for ; Wed, 13 Jul 2022 18:28:22 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 96EB860B9B for ; Wed, 13 Jul 2022 18:28:20 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 96EB860B9B Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=gEVqpTli X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p3WlCbwGiB1G for ; Wed, 13 Jul 2022 18:28:19 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 4717A613AE Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by smtp3.osuosl.org (Postfix) with ESMTPS id 4717A613AE for ; Wed, 13 Jul 2022 18:28:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657736899; x=1689272899; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=DxfDy1dASN4/9U76IYG+G9eMhIZRds5Wd4FwvtR7gEY=; b=gEVqpTlinLZzWEzfIbJLf4bGLByO0k+BUzsoeX4r7W0Y6KXYMYIUWLId 2EE5By2tE4rMChVGVeh9JI4Tt+sS2vBDHH9KoCbVLRGcIDhCZjgfXbjFO O0knH07ONnlvFjasmohdVxe/7JqL2h/6JPogbq96EuXRruoHp9/FFAz+7 ieIhVwugHmHNoDruMN+IVHxDSq81teq8Wd3QGehOWeL1nLK0CdkmspTOT QUPuM0MMRY5dfE9E/ZPfpxxMummow9sJpp5PS3Gfde3ByH0Xd1GrAnBp9 qmnFps4rGugoukb3uTrZgBZthnR3iEDytMIpEKytvxyTlAUFN1NuWSS28 g==; X-IronPort-AV: E=McAfee;i="6400,9594,10407"; a="284067966" X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="284067966" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jul 2022 11:28:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="545962407" Received: from silpixa00401454.ir.intel.com ([10.55.128.122]) by orsmga003.jf.intel.com with ESMTP; 13 Jul 2022 11:28:16 -0700 From: Harry van Haaren To: dev@openvswitch.org Date: Wed, 13 Jul 2022 18:27:58 +0000 Message-Id: <20220713182807.3416578-2-harry.van.haaren@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220713182807.3416578-1-harry.van.haaren@intel.com> References: <20220712174456.2682549-1-harry.van.haaren@intel.com> <20220713182807.3416578-1-harry.van.haaren@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org, kumar.amber@intel.com Subject: [ovs-dev] [PATCH v10 01/10] odp-execute: Add function pointers to odp-execute for different action implementations. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Emma Finn This commit introduces the initial infrastructure required to allow different implementations for OvS actions. The patch introduces action function pointers which allows user to switch between different action implementations available. This will allow for more performance and flexibility so the user can choose the action implementation to best suite their use case. Signed-off-by: Emma Finn Acked-by: Harry van Haaren Tested-by: Kumar Amber Acked-by: Sunil Pai G --- v10: - switch from < __OVS_ATTR_MAX back to <= OVS_ATTR_MAX (Eelco) v9: - rebase conflicts on NEWS - As Actions is initialized in vswitchd/bridge.c now, logs are output on startup, which have been added to the tests/ofproto-macros. --- lib/automake.mk | 2 + lib/odp-execute-private.c | 92 +++++++++++++++++++++++++++++++++++++++ lib/odp-execute-private.h | 76 ++++++++++++++++++++++++++++++++ lib/odp-execute.c | 51 +++++++++++++++++++++- lib/odp-execute.h | 7 +++ tests/ofproto-macros.at | 1 + vswitchd/bridge.c | 3 ++ 7 files changed, 231 insertions(+), 1 deletion(-) create mode 100644 lib/odp-execute-private.c create mode 100644 lib/odp-execute-private.h diff --git a/lib/automake.mk b/lib/automake.mk index 1d00cfa20..23ba4fab0 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -216,6 +216,8 @@ lib_libopenvswitch_la_SOURCES = \ lib/object-collection.h \ lib/odp-execute.c \ lib/odp-execute.h \ + lib/odp-execute-private.c \ + lib/odp-execute-private.h \ lib/odp-util.c \ lib/odp-util.h \ lib/ofp-actions.c \ diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c new file mode 100644 index 000000000..2c30ed05b --- /dev/null +++ b/lib/odp-execute-private.c @@ -0,0 +1,92 @@ +/* + * Copyright (c) 2022 Intel. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include +#include +#include + +#include "dpdk.h" +#include "dp-packet.h" +#include "odp-execute-private.h" +#include "odp-netlink.h" +#include "odp-util.h" +#include "openvswitch/vlog.h" + +VLOG_DEFINE_THIS_MODULE(odp_execute_impl); +static int active_action_impl_index; + +static struct odp_execute_action_impl action_impls[] = { + [ACTION_IMPL_SCALAR] = { + .available = false, + .name = "scalar", + .init_func = NULL, + }, +}; + +static void +action_impl_copy_funcs(struct odp_execute_action_impl *dest, + const struct odp_execute_action_impl *src) +{ + for (int i = 0; i < __OVS_ACTION_ATTR_MAX; i++) { + atomic_store_relaxed(&dest->funcs[i], src->funcs[i]); + } +} + +struct odp_execute_action_impl * +odp_execute_action_set(const char *name) +{ + for (int i = 0; i < ACTION_IMPL_MAX; i++) { + /* String compare, and set ptrs atomically. */ + if (!strcmp(action_impls[i].name, name)) { + active_action_impl_index = i; + + VLOG_INFO("Action implementation set to %s", name); + return &action_impls[i]; + } + } + return NULL; +} + +void +odp_execute_action_init(void) +{ + /* Each impl's function array is initialized to reflect the scalar + * implementation. This simplifies adding optimized implementations, + * as the autovalidator can always compare all actions. + * + * Below will check if impl is available and copies the scalar functions + * to all other implementations. + */ + for (int i = 0; i < ACTION_IMPL_MAX; i++) { + bool avail = true; + + if (i != ACTION_IMPL_SCALAR) { + action_impl_copy_funcs(&action_impls[i], + &action_impls[ACTION_IMPL_SCALAR]); + } + + if (action_impls[i].init_func) { + /* Return zero is success, non-zero means error. */ + avail = (action_impls[i].init_func(&action_impls[i]) == 0); + } + + action_impls[i].available = avail; + + VLOG_INFO("Action implementation %s (available: %s)", + action_impls[i].name, avail ? "Yes" : "No"); + } +} diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h new file mode 100644 index 000000000..24126cdca --- /dev/null +++ b/lib/odp-execute-private.h @@ -0,0 +1,76 @@ +/* + * Copyright (c) 2022 Intel. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef ODP_EXTRACT_PRIVATE +#define ODP_EXTRACT_PRIVATE 1 + +#include "dp-packet.h" +#include "odp-execute.h" +#include "odp-netlink.h" +#include "ovs-atomic.h" + +/* Forward declaration for typedef. */ +struct odp_execute_action_impl; + +/* Typedef for an initialization function that can initialize each + * implementation, checking requirements such as CPU ISA. + */ +typedef int (*odp_execute_action_init_func) + (struct odp_execute_action_impl *self); + +/* Structure represents an implementation of the odp actions. */ +struct odp_execute_action_impl { + /* When set, the CPU ISA required for this implementation is available + * and the implementation can be used. + */ + bool available; + + /* Name of the implementation. */ + const char *name; + + /* Function is used to detect if this CPU has the ISA required + * to run the optimized action implementation and if available, initializes + * the implementation for use. + */ + odp_execute_action_init_func init_func; + + /* An array of callback functions, one for each action. */ + ATOMIC(odp_execute_action_cb) funcs[__OVS_ACTION_ATTR_MAX]; +}; + +/* Order of Actions implementations. */ +enum odp_execute_action_impl_idx { + ACTION_IMPL_SCALAR, + /* See ACTION_IMPL_BEGIN below, for "first to-be-validated" impl. + * Do not change the autovalidator position in this list without updating + * the define below. + */ + + ACTION_IMPL_MAX, +}; + +/* Index to start verifying implementations from. */ +BUILD_ASSERT_DECL(ACTION_IMPL_SCALAR == 0); + +/* Odp execute init handles setting up the state of the actions functions at + * initialization time. It cannot return errors, as it must always succeed in + * initializing the scalar/generic codepath. + */ +void odp_execute_action_init(void); + +struct odp_execute_action_impl * odp_execute_action_set(const char *name); + +#endif /* ODP_EXTRACT_PRIVATE */ diff --git a/lib/odp-execute.c b/lib/odp-execute.c index 7da56793d..7f998add6 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -17,6 +17,7 @@ #include #include "odp-execute.h" +#include "odp-execute-private.h" #include #include #include @@ -833,6 +834,38 @@ requires_datapath_assistance(const struct nlattr *a) return false; } +/* The active function pointers on the datapath. ISA optimized implementations + * are enabled by plugging them into this static arary, which is consulted when + * applying actions on the datapath. + */ +static struct odp_execute_action_impl *actions_active_impl; + +static int +odp_actions_impl_set(const char *name) +{ + struct odp_execute_action_impl *active; + active = odp_execute_action_set(name); + if (!active) { + VLOG_ERR("Failed setting action implementation to %s", name); + return 1; + } + + actions_active_impl = active; + return 0; + +} + +void +odp_execute_init(void) +{ + static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; + if (ovsthread_once_start(&once)) { + odp_execute_action_init(); + odp_actions_impl_set("scalar"); + ovsthread_once_done(&once); + } +} + /* Executes all of the 'actions_len' bytes of datapath actions in 'actions' on * the packets in 'batch'. If 'steal' is true, possibly modifies and * definitely free the packets in 'batch', otherwise leaves 'batch' unchanged. @@ -857,6 +890,7 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, NL_ATTR_FOR_EACH_UNSAFE (a, left, actions, actions_len) { int type = nl_attr_type(a); + enum ovs_action_attr attr_type = (enum ovs_action_attr) type; bool last_action = (left <= NLA_ALIGN(a->nla_len)); if (requires_datapath_assistance(a)) { @@ -879,8 +913,20 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, continue; } - switch ((enum ovs_action_attr) type) { + /* If type is set in the active actions implementation, call the + * function-pointer and continue to the next action. + */ + if (actions_active_impl->funcs[attr_type] && + attr_type <= OVS_ACTION_ATTR_MAX) { + actions_active_impl->funcs[attr_type](batch, a); + continue; + } + /* If the action was not handled by the active function pointers above, + * process them by switching on the type below. + */ + + switch (attr_type) { case OVS_ACTION_ATTR_HASH: { const struct ovs_action_hash *hash_act = nl_attr_get(a); @@ -1094,6 +1140,9 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, case __OVS_ACTION_ATTR_MAX: OVS_NOT_REACHED(); } + + /* Do not add any generic processing here, as it won't be executed when + * an ISA-specific action implementation exists. */ } dp_packet_delete_batch(batch, steal); diff --git a/lib/odp-execute.h b/lib/odp-execute.h index a3578a575..0921ee924 100644 --- a/lib/odp-execute.h +++ b/lib/odp-execute.h @@ -28,6 +28,13 @@ struct dp_packet; struct pkt_metadata; struct dp_packet_batch; + +/* Called once at initialization time. */ +void odp_execute_init(void); + +typedef void (*odp_execute_action_cb)(struct dp_packet_batch *batch, + const struct nlattr *action); + typedef void (*odp_execute_cb)(void *dp, struct dp_packet_batch *batch, const struct nlattr *action, bool should_steal); diff --git a/tests/ofproto-macros.at b/tests/ofproto-macros.at index 84f07c108..676d55aa9 100644 --- a/tests/ofproto-macros.at +++ b/tests/ofproto-macros.at @@ -182,6 +182,7 @@ m4_define([_OVS_VSWITCHD_START], on_exit "kill_ovs_vswitchd `cat ovs-vswitchd.pid`" AT_CHECK([[sed < stderr ' /ovs_numa|INFO|Discovered /d +/odp_execute_impl|INFO|Action implementation /d /vlog|INFO|opened log file/d /vswitchd|INFO|ovs-vswitchd (Open vSwitch)/d /reconnect|INFO|/d diff --git a/vswitchd/bridge.c b/vswitchd/bridge.c index e328d8ead..b704410d3 100644 --- a/vswitchd/bridge.c +++ b/vswitchd/bridge.c @@ -40,6 +40,7 @@ #include "netdev.h" #include "netdev-offload.h" #include "nx-match.h" +#include "odp-execute.h" #include "ofproto/bond.h" #include "ofproto/ofproto.h" #include "openvswitch/dynamic-string.h" @@ -530,6 +531,8 @@ bridge_init(const char *remote) stp_init(); lldp_init(); rstp_init(); + odp_execute_init(); + ifaces_changed = seq_create(); last_ifaces_changed = seq_read(ifaces_changed); ifnotifier = if_notifier_create(if_change_cb, NULL); From patchwork Wed Jul 13 18:27:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Van Haaren, Harry" X-Patchwork-Id: 1656120 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=hwLHr/tr; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LjmN13vCMz9sB4 for ; Thu, 14 Jul 2022 04:28:33 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 3C20960BB4; Wed, 13 Jul 2022 18:28:31 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 3C20960BB4 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=hwLHr/tr X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PJc-e2wuRZeL; Wed, 13 Jul 2022 18:28:30 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 3467A613BD; Wed, 13 Jul 2022 18:28:29 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 3467A613BD Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 801EFC007F; Wed, 13 Jul 2022 18:28:28 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9B664C007D for ; Wed, 13 Jul 2022 18:28:24 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 114996136A for ; Wed, 13 Jul 2022 18:28:22 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 114996136A X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EVq_HHeasI5E for ; Wed, 13 Jul 2022 18:28:21 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 5113D60AF2 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by smtp3.osuosl.org (Postfix) with ESMTPS id 5113D60AF2 for ; Wed, 13 Jul 2022 18:28:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657736901; x=1689272901; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=i5uWkMtZw1pBDASs/skY1wJ51onmf8uXAw/sHF4Hr6E=; b=hwLHr/tr+JlqhKsFuR+gQzExpSXFMpjUtlbaViCYtuDpZBXQ3iYznulP htENYQWmZrqU8f1P6UwL7D+JE7sJBP3QhFhwS6rnB8BYHbOSgxuK60vU7 XCMpVavi0Awevhm+trZtBZWeef4FnGSmtQNJAH1v0S7WOve71zqyoK/Uv 0Bc7SpypUt7l8IQg9hK+c0FKmhKLRk8k0UljJDf91DPFeRueNB7tAiprP 2X5orZw2C2dMoLwG8SQvvRYyqGdjJ/k/0lR5353EafIr+Aw5rViDv9Bz1 p49MIvbySw3HDZozmOcABE2fAV9g9OK5+QqOS6tYGsYpUiBIX1BTl7a0x g==; X-IronPort-AV: E=McAfee;i="6400,9594,10407"; a="284067973" X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="284067973" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jul 2022 11:28:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="545962422" Received: from silpixa00401454.ir.intel.com ([10.55.128.122]) by orsmga003.jf.intel.com with ESMTP; 13 Jul 2022 11:28:19 -0700 From: Harry van Haaren To: dev@openvswitch.org Date: Wed, 13 Jul 2022 18:27:59 +0000 Message-Id: <20220713182807.3416578-3-harry.van.haaren@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220713182807.3416578-1-harry.van.haaren@intel.com> References: <20220712174456.2682549-1-harry.van.haaren@intel.com> <20220713182807.3416578-1-harry.van.haaren@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org, kumar.amber@intel.com Subject: [ovs-dev] [PATCH v10 02/10] odp-execute: Add function pointer for pop_vlan action. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Emma Finn This commit removes the pop_vlan action from the large switch and creates a separate function for batched processing. A function pointer is also added to call the new batched function for the pop_vlan action. Signed-off-by: Emma Finn Acked-by: Harry van Haaren Tested-by: Kumar Amber Acked-by: Sunil Pai G Acked-by: Eelco Chaudron --- lib/odp-execute-private.c | 16 +++++++++++++++- lib/odp-execute-private.h | 5 +++++ lib/odp-execute.c | 32 ++++++++++++++++++++++++++------ 3 files changed, 46 insertions(+), 7 deletions(-) diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 2c30ed05b..c1d153c6e 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -33,7 +33,7 @@ static struct odp_execute_action_impl action_impls[] = { [ACTION_IMPL_SCALAR] = { .available = false, .name = "scalar", - .init_func = NULL, + .init_func = odp_action_scalar_init, }, }; @@ -88,5 +88,19 @@ odp_execute_action_init(void) VLOG_INFO("Action implementation %s (available: %s)", action_impls[i].name, avail ? "Yes" : "No"); + + /* The following is a run-time check to make sure a scalar + * implementation exists for the given ISA implementation. This is to + * make sure the autovalidator works as expected. */ + if (avail && i != ACTION_IMPL_SCALAR) { + for (int j = 0; j < __OVS_ACTION_ATTR_MAX; j++) { + /* No ovs_assert(), as it can be compiled out. */ + if (action_impls[ACTION_IMPL_SCALAR].funcs[j] == NULL + && action_impls[i].funcs[j] != NULL) { + ovs_assert_failure(OVS_SOURCE_LOCATOR, __func__, + "Missing scalar action function!"); + } + } + } } } diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index 24126cdca..ae06fbc09 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -71,6 +71,11 @@ BUILD_ASSERT_DECL(ACTION_IMPL_SCALAR == 0); */ void odp_execute_action_init(void); +/* Init functions for the action implementations. Initializes the function + * pointers for optimized action types. + */ +int odp_action_scalar_init(struct odp_execute_action_impl *self); + struct odp_execute_action_impl * odp_execute_action_set(const char *name); #endif /* ODP_EXTRACT_PRIVATE */ diff --git a/lib/odp-execute.c b/lib/odp-execute.c index 7f998add6..368876f27 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -834,6 +834,30 @@ requires_datapath_assistance(const struct nlattr *a) return false; } +static void +action_pop_vlan(struct dp_packet_batch *batch, + const struct nlattr *a OVS_UNUSED) +{ + struct dp_packet *packet; + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + eth_pop_vlan(packet); + } +} + +/* Implementation of the scalar actions impl init function. Build up the + * array of func ptrs here. + */ +int +odp_action_scalar_init(struct odp_execute_action_impl *self) +{ + /* Set function pointers for actions that can be applied directly, these + * are identified by OVS_ACTION_ATTR_*. */ + self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_pop_vlan; + + return 0; +} + /* The active function pointers on the datapath. ISA optimized implementations * are enabled by plugging them into this static arary, which is consulted when * applying actions on the datapath. @@ -982,12 +1006,6 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, break; } - case OVS_ACTION_ATTR_POP_VLAN: - DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { - eth_pop_vlan(packet); - } - break; - case OVS_ACTION_ATTR_PUSH_MPLS: { const struct ovs_action_push_mpls *mpls = nl_attr_get(a); @@ -1138,6 +1156,8 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, case OVS_ACTION_ATTR_CT: case OVS_ACTION_ATTR_UNSPEC: case __OVS_ACTION_ATTR_MAX: + /* The following actions are handled by the scalar implementation. */ + case OVS_ACTION_ATTR_POP_VLAN: OVS_NOT_REACHED(); } From patchwork Wed Jul 13 18:28:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Van Haaren, Harry" X-Patchwork-Id: 1656121 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=e6D2fAJO; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LjmN40z2rz9sB4 for ; Thu, 14 Jul 2022 04:28:35 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id C68EE8462C; Wed, 13 Jul 2022 18:28:33 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org C68EE8462C Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=e6D2fAJO X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SY9FqRxCGmWZ; Wed, 13 Jul 2022 18:28:32 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 9EDBE8415A; Wed, 13 Jul 2022 18:28:31 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 9EDBE8415A Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3EA75C0082; Wed, 13 Jul 2022 18:28:30 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id B2914C0035 for ; Wed, 13 Jul 2022 18:28:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 752B960B9B for ; Wed, 13 Jul 2022 18:28:24 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 752B960B9B Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=e6D2fAJO X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hqrYTw_-Fl7W for ; Wed, 13 Jul 2022 18:28:23 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 95FC260AF2 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by smtp3.osuosl.org (Postfix) with ESMTPS id 95FC260AF2 for ; Wed, 13 Jul 2022 18:28:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657736903; x=1689272903; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6NDvNZ/arce8d0KJJGNI03hMyEdtHx+WH4D1QwT8bAA=; b=e6D2fAJOwUrwsDWyanCifnxn1UlRPFBcIU2EAEVGLPNwn/cnJQ/e8kZH QfT7gecaQ50Wyq2goQg5XWQnpUKxQUQyYBKJ1hxmEGFWcvYVpNei/+Kw1 OT8N4lBiZbSGq9iARsI65zE5bgpbcwRRUytiw3nlILJRLsAvID+xhJpg6 dy6W6rE99obDfoKDELadrr/nNk07nB2Hly7AMEbhJYCnQC+kQiRvGuqx3 CQHBzfq7eTY9vzDR06pFTMyIkF4BPsoATZp2EP2YMO5AbhoSThF2oI6rl enjldblqdUqlh+j2DMoq4+PatCCWi6eI7ouAszkVtLFKuvv0j8R2HUUqA Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10407"; a="284067979" X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="284067979" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jul 2022 11:28:23 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="545962432" Received: from silpixa00401454.ir.intel.com ([10.55.128.122]) by orsmga003.jf.intel.com with ESMTP; 13 Jul 2022 11:28:21 -0700 From: Harry van Haaren To: dev@openvswitch.org Date: Wed, 13 Jul 2022 18:28:00 +0000 Message-Id: <20220713182807.3416578-4-harry.van.haaren@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220713182807.3416578-1-harry.van.haaren@intel.com> References: <20220712174456.2682549-1-harry.van.haaren@intel.com> <20220713182807.3416578-1-harry.van.haaren@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org, kumar.amber@intel.com Subject: [ovs-dev] [PATCH v10 03/10] odp-execute: Add auto validation function for actions. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Emma Finn This commit introduced the auto-validation function which allows users to compare the batch of packets obtained from different action implementations against the linear action implementation. The autovalidator function can be triggered at runtime using the following command: $ ovs-appctl odp-execute/action-impl-set autovalidator Signed-off-by: Emma Finn Acked-by: Harry van Haaren Tested-by: Kumar Amber Acked-by: Sunil Pai G Acked-by: Eelco Chaudron --- NEWS | 2 + lib/dp-packet.c | 24 ++++++++++ lib/dp-packet.h | 4 ++ lib/odp-execute-private.c | 99 +++++++++++++++++++++++++++++++++++++++ lib/odp-execute-private.h | 6 +++ 5 files changed, 135 insertions(+) diff --git a/NEWS b/NEWS index 6001aeb1d..1ef1175d0 100644 --- a/NEWS +++ b/NEWS @@ -50,6 +50,8 @@ Post-v2.17.0 * 'dpif-netdev/subtable-lookup-prio-get' appctl command renamed to 'dpif-netdev/subtable-lookup-info-get' to better reflect its purpose. The old variant is kept for backward compatibility. + * Add actions auto-validator function to compare different actions + implementations against default implementation. - Linux datapath: * Add offloading meter tc police. * Add support for offloading the check_pkt_len action. diff --git a/lib/dp-packet.c b/lib/dp-packet.c index 35c72542a..4538d2a61 100644 --- a/lib/dp-packet.c +++ b/lib/dp-packet.c @@ -506,3 +506,27 @@ dp_packet_resize_l2(struct dp_packet *b, int increment) dp_packet_adjust_layer_offset(&b->l2_5_ofs, increment); return dp_packet_data(b); } + +bool +dp_packet_compare_offsets(struct dp_packet *b1, struct dp_packet *b2, + struct ds *err_str) +{ + if ((b1->l2_pad_size != b2->l2_pad_size) || + (b1->l2_5_ofs != b2->l2_5_ofs) || + (b1->l3_ofs != b2->l3_ofs) || + (b1->l4_ofs != b2->l4_ofs)) { + if (err_str) { + ds_put_format(err_str, "Packet offset comparison failed\n"); + ds_put_format(err_str, "Buffer 1 offsets: l2_pad_size %u," + " l2_5_ofs : %u l3_ofs %u, l4_ofs %u\n", + b1->l2_pad_size, b1->l2_5_ofs, + b1->l3_ofs, b1->l4_ofs); + ds_put_format(err_str, "Buffer 2 offsets: l2_pad_size %u," + " l2_5_ofs : %u l3_ofs %u, l4_ofs %u\n", + b2->l2_pad_size, b2->l2_5_ofs, + b2->l3_ofs, b2->l4_ofs); + } + return false; + } + return true; +} diff --git a/lib/dp-packet.h b/lib/dp-packet.h index eea5a9215..55eeaab2c 100644 --- a/lib/dp-packet.h +++ b/lib/dp-packet.h @@ -236,6 +236,10 @@ void *dp_packet_steal_data(struct dp_packet *); static inline bool dp_packet_equal(const struct dp_packet *, const struct dp_packet *); +bool dp_packet_compare_offsets(struct dp_packet *good, + struct dp_packet *test, + struct ds *err_str); + /* Frees memory that 'b' points to, as well as 'b' itself. */ static inline void diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index c1d153c6e..780d6d289 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -30,6 +30,12 @@ VLOG_DEFINE_THIS_MODULE(odp_execute_impl); static int active_action_impl_index; static struct odp_execute_action_impl action_impls[] = { + [ACTION_IMPL_AUTOVALIDATOR] = { + .available = false, + .name = "autovalidator", + .init_func = action_autoval_init, + }, + [ACTION_IMPL_SCALAR] = { .available = false, .name = "scalar", @@ -104,3 +110,96 @@ odp_execute_action_init(void) } } } + +/* Init sequence required to be scalar first to pick up the default scalar +* implementations, allowing over-riding of the optimized functions later. +*/ +BUILD_ASSERT_DECL(ACTION_IMPL_SCALAR == 0); +BUILD_ASSERT_DECL(ACTION_IMPL_AUTOVALIDATOR == 1); + +/* Loop over packets, and validate each one for the given action. */ +static void +action_autoval_generic(struct dp_packet_batch *batch, const struct nlattr *a) +{ + struct odp_execute_action_impl *scalar = &action_impls[ACTION_IMPL_SCALAR]; + enum ovs_action_attr attr_type = nl_attr_type(a); + struct dp_packet_batch original_batch; + bool failed = false; + + dp_packet_batch_clone(&original_batch, batch); + + scalar->funcs[attr_type](batch, a); + + for (int impl = ACTION_IMPL_BEGIN; impl < ACTION_IMPL_MAX; impl++) { + /* Clone original batch and execute implementation under test. */ + struct dp_packet_batch test_batch; + + dp_packet_batch_clone(&test_batch, &original_batch); + action_impls[impl].funcs[attr_type](&test_batch, a); + + /* Loop over implementations, checking each one. */ + for (int pidx = 0; pidx < original_batch.count; pidx++) { + struct dp_packet *good_pkt = batch->packets[pidx]; + struct dp_packet *test_pkt = test_batch.packets[pidx]; + + struct ds log_msg = DS_EMPTY_INITIALIZER; + + /* Compare packet length and payload contents. */ + bool eq = dp_packet_equal(good_pkt, test_pkt); + + if (!eq) { + ds_put_format(&log_msg, "Packet: %d\nAction : ", pidx); + format_odp_actions(&log_msg, a, a->nla_len, NULL); + ds_put_format(&log_msg, "\nGood hex:\n"); + ds_put_hex_dump(&log_msg, dp_packet_data(good_pkt), + dp_packet_size(good_pkt), 0, false); + ds_put_format(&log_msg, "Test hex:\n"); + ds_put_hex_dump(&log_msg, dp_packet_data(test_pkt), + dp_packet_size(test_pkt), 0, false); + + failed = true; + } + + /* Compare offsets and RSS */ + if (!dp_packet_compare_offsets(good_pkt, test_pkt, &log_msg)) { + failed = true; + } + + if (dp_packet_rss_valid(good_pkt)) { + uint32_t good_hash = dp_packet_get_rss_hash(good_pkt); + uint32_t test_hash = dp_packet_get_rss_hash(test_pkt); + + if (good_hash != test_hash) { + ds_put_format(&log_msg, + "Autovalidation rss hash failed\n"); + ds_put_format(&log_msg, "Good RSS hash : %u\n", good_hash); + ds_put_format(&log_msg, "Test RSS hash : %u\n", test_hash); + + failed = true; + } + } + + if (failed) { + VLOG_ERR("Autovalidation of %s failed. Details:\n%s", + action_impls[impl].name, ds_cstr(&log_msg)); + ds_destroy(&log_msg); + failed = false; + } + } + dp_packet_delete_batch(&test_batch, true); + } + dp_packet_delete_batch(&original_batch, true); +} + +int +action_autoval_init(struct odp_execute_action_impl *self) +{ + /* Set function pointers for actions that can be applied directly, these + * are identified by OVS_ACTION_ATTR_*. */ + for (int i = 0; i < __OVS_ACTION_ATTR_MAX; i++) { + if (action_impls[ACTION_IMPL_SCALAR].funcs[i]) { + self->funcs[i] = action_autoval_generic; + } + } + return 0; +} diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index ae06fbc09..074a8d67e 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -54,6 +54,7 @@ struct odp_execute_action_impl { /* Order of Actions implementations. */ enum odp_execute_action_impl_idx { ACTION_IMPL_SCALAR, + ACTION_IMPL_AUTOVALIDATOR, /* See ACTION_IMPL_BEGIN below, for "first to-be-validated" impl. * Do not change the autovalidator position in this list without updating * the define below. @@ -64,6 +65,9 @@ enum odp_execute_action_impl_idx { /* Index to start verifying implementations from. */ BUILD_ASSERT_DECL(ACTION_IMPL_SCALAR == 0); +BUILD_ASSERT_DECL(ACTION_IMPL_AUTOVALIDATOR == 1); + +#define ACTION_IMPL_BEGIN (ACTION_IMPL_AUTOVALIDATOR + 1) /* Odp execute init handles setting up the state of the actions functions at * initialization time. It cannot return errors, as it must always succeed in @@ -78,4 +82,6 @@ int odp_action_scalar_init(struct odp_execute_action_impl *self); struct odp_execute_action_impl * odp_execute_action_set(const char *name); +int action_autoval_init(struct odp_execute_action_impl *self); + #endif /* ODP_EXTRACT_PRIVATE */ From patchwork Wed Jul 13 18:28:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Van Haaren, Harry" X-Patchwork-Id: 1656122 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=c55u+IdB; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LjmN939pzz9sB4 for ; Thu, 14 Jul 2022 04:28:41 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id DF187424A2; Wed, 13 Jul 2022 18:28:37 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org DF187424A2 Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=c55u+IdB X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EK4_ZdBCj0FG; Wed, 13 Jul 2022 18:28:34 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 003D141B90; Wed, 13 Jul 2022 18:28:32 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 003D141B90 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 20DAFC0085; Wed, 13 Jul 2022 18:28:32 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id D0EEEC0035 for ; Wed, 13 Jul 2022 18:28:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 9DF45613BD for ; Wed, 13 Jul 2022 18:28:26 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 9DF45613BD Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=c55u+IdB X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oIXhm4QuzqC1 for ; Wed, 13 Jul 2022 18:28:25 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org AF42F60BB4 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by smtp3.osuosl.org (Postfix) with ESMTPS id AF42F60BB4 for ; Wed, 13 Jul 2022 18:28:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657736905; x=1689272905; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Xx8yDYGl7zt/CCCFCERWcpbm6QjGZlDZWs0X/deP4kk=; b=c55u+IdBIxyz1A6jM2kLFVHyg9ulyrSX8QYxFblzUX+Ydw8uczrwgHzv WlHv34dzT5FGDMxVgvHgsRYG10phz1SeVYITSc0k0+dXHv5QMcGwsyFl3 YeHyfLETdlDkQ37MzJGCk4gxAejqBL69jTUNq9wk6/pAesMjZg78QuNEa 1mfId/SnD38l4g5ePF4DDltL952Wk/VMYBa4MIw9xLn8dqxwkUnFWL6ys mQnAMY6ITO3z2AvIXDuaxaxxqoACH84G9XKFBxpih8II9u9b+6x1BKVc+ carGNpgIMvYaKXHwW3Ey+odTwrFC6C0I+oSRQekQI2vsw04OK/WkHnzLV Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10407"; a="284067986" X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="284067986" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jul 2022 11:28:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="545962441" Received: from silpixa00401454.ir.intel.com ([10.55.128.122]) by orsmga003.jf.intel.com with ESMTP; 13 Jul 2022 11:28:23 -0700 From: Harry van Haaren To: dev@openvswitch.org Date: Wed, 13 Jul 2022 18:28:01 +0000 Message-Id: <20220713182807.3416578-5-harry.van.haaren@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220713182807.3416578-1-harry.van.haaren@intel.com> References: <20220712174456.2682549-1-harry.van.haaren@intel.com> <20220713182807.3416578-1-harry.van.haaren@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org, kumar.amber@intel.com Subject: [ovs-dev] [PATCH v10 04/10] odp-execute: Add command to switch action implementation. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Emma Finn This commit adds a new command to allow the user to switch the active action implementation at runtime. Usage: $ ovs-appctl odp-execute/action-impl-set scalar This commit also adds a new command to retrieve the list of available action implementations. This can be used by to check what implementations of actions are available and what implementation is active during runtime. Usage: $ ovs-appctl odp-execute/action-impl-show Added separate test-case for ovs-actions show/set commands: PMD - ovs-actions configuration Signed-off-by: Emma Finn Signed-off-by: Kumar Amber Signed-off-by: Sunil Pai G Co-authored-by: Kumar Amber Co-authored-by: Sunil Pai G Acked-by: Harry van Haaren Tested-by: Kumar Amber Acked-by: Sunil Pai G --- v9: rebase conflict on NEWS --- NEWS | 2 ++ lib/automake.mk | 1 + lib/odp-execute-private.c | 12 ++++++++++ lib/odp-execute-private.h | 2 ++ lib/odp-execute-unixctl.man | 10 +++++++++ lib/odp-execute.c | 44 +++++++++++++++++++++++++++++++++++++ tests/pmd.at | 39 ++++++++++++++++++++++++++++++++ vswitchd/ovs-vswitchd.8.in | 1 + 8 files changed, 111 insertions(+) create mode 100644 lib/odp-execute-unixctl.man diff --git a/NEWS b/NEWS index 1ef1175d0..d02733936 100644 --- a/NEWS +++ b/NEWS @@ -52,6 +52,8 @@ Post-v2.17.0 The old variant is kept for backward compatibility. * Add actions auto-validator function to compare different actions implementations against default implementation. + * Add command line option to switch between different actions + implementations available at run time. - Linux datapath: * Add offloading meter tc police. * Add support for offloading the check_pkt_len action. diff --git a/lib/automake.mk b/lib/automake.mk index 23ba4fab0..5c3b05f6b 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -584,6 +584,7 @@ MAN_FRAGMENTS += \ lib/netdev-dpdk-unixctl.man \ lib/dpif-netdev-unixctl.man \ lib/dpif-netlink-unixctl.man \ + lib/odp-execute-unixctl.man \ lib/ofp-version.man \ lib/ovs.tmac \ lib/ovs-replay.man \ diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 780d6d289..38be22ec9 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -67,6 +67,18 @@ odp_execute_action_set(const char *name) return NULL; } +void +odp_execute_action_get_info(struct ds *string) +{ + ds_put_cstr(string, "Available Actions implementations:\n"); + for (int i = 0; i < ACTION_IMPL_MAX; i++) { + ds_put_format(string, " %s (available: %s, active: %s)\n", + action_impls[i].name, + action_impls[i].available ? "Yes" : "No", + i == active_action_impl_index ? "Yes" : "No"); + } +} + void odp_execute_action_init(void) { diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index 074a8d67e..d6eebbf37 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -84,4 +84,6 @@ struct odp_execute_action_impl * odp_execute_action_set(const char *name); int action_autoval_init(struct odp_execute_action_impl *self); +void odp_execute_action_get_info(struct ds *name); + #endif /* ODP_EXTRACT_PRIVATE */ diff --git a/lib/odp-execute-unixctl.man b/lib/odp-execute-unixctl.man new file mode 100644 index 000000000..82d51e1d3 --- /dev/null +++ b/lib/odp-execute-unixctl.man @@ -0,0 +1,10 @@ +.SS "ODP-EXECUTE COMMANDS" +These commands manage the "odp-execute" component. + +.IP "\fBodp-execute/action-impl-show\fR +Lists the actions implementations that are available and highlights the +currently enabled one. +. +.IP "\fBodp-execute/action-impl-set\fR \fIaction_impl\fR" +Sets the action implementation to any available implementation. By default +"scalar" is used. diff --git a/lib/odp-execute.c b/lib/odp-execute.c index 368876f27..d5be190e0 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -39,6 +39,7 @@ #include "csum.h" #include "conntrack.h" #include "openvswitch/vlog.h" +#include "unixctl.h" VLOG_DEFINE_THIS_MODULE(odp_execute); COVERAGE_DEFINE(datapath_drop_sample_error); @@ -879,6 +880,48 @@ odp_actions_impl_set(const char *name) } +static void +action_impl_set(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *aux OVS_UNUSED) +{ + struct ds reply = DS_EMPTY_INITIALIZER; + + int err = odp_actions_impl_set(argv[1]); + if (err) { + ds_put_format(&reply, + "Error: unknown action implementation, %s, specified!\n", + argv[1]); + unixctl_command_reply_error(conn, ds_cstr(&reply)); + } else { + ds_put_format(&reply, "Action implementation set to %s.\n", argv[1]); + unixctl_command_reply(conn, ds_cstr(&reply)); + } + + ds_destroy(&reply); +} + +static void +action_impl_show(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *aux OVS_UNUSED) +{ + struct ds reply = DS_EMPTY_INITIALIZER; + + odp_execute_action_get_info(&reply); + unixctl_command_reply(conn, ds_cstr(&reply)); + ds_destroy(&reply); +} + +static void +odp_execute_unixctl_init(void) +{ + unixctl_command_register("odp-execute/action-impl-set", "name", + 1, 1, action_impl_set, + NULL); + unixctl_command_register("odp-execute/action-impl-show", "", + 0, 0, action_impl_show, + NULL); +} + void odp_execute_init(void) { @@ -886,6 +929,7 @@ odp_execute_init(void) if (ovsthread_once_start(&once)) { odp_execute_action_init(); odp_actions_impl_set("scalar"); + odp_execute_unixctl_init(); ovsthread_once_done(&once); } } diff --git a/tests/pmd.at b/tests/pmd.at index 4342c50e0..4ca926581 100644 --- a/tests/pmd.at +++ b/tests/pmd.at @@ -1192,3 +1192,42 @@ ovs-appctl: ovs-vswitchd: server returned an error OVS_VSWITCHD_STOP AT_CLEANUP + +AT_SETUP([PMD - ovs-actions configuration]) +OVS_VSWITCHD_START([], [], [], [--dummy-numa 0,0]) +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dummy-pmd]) + +AT_CHECK([ovs-vsctl show], [], [stdout]) + +dnl Set the scalar first, so we always have the scalar impl as Active. +AT_CHECK([ovs-appctl odp-execute/action-impl-set scalar], [0], [dnl +Action implementation set to scalar. +]) +AT_CHECK([ovs-appctl odp-execute/action-impl-show | grep "scalar"], [], [dnl + scalar (available: Yes, active: Yes) +]) + +AT_CHECK([ovs-appctl odp-execute/action-impl-show | grep "autovalidator"], [], [dnl + autovalidator (available: Yes, active: No) +]) + +dnl Set the autovalidator impl to active. +AT_CHECK([ovs-appctl odp-execute/action-impl-set autovalidator], [0], [dnl +Action implementation set to autovalidator. +]) + +AT_CHECK([ovs-appctl odp-execute/action-impl-show | grep "scalar"], [], [dnl + scalar (available: Yes, active: No) +]) + +AT_CHECK([ovs-appctl odp-execute/action-impl-show | grep "autovalidator"], [], [dnl + autovalidator (available: Yes, active: Yes) +]) + +AT_CHECK([ovs-appctl odp-execute/action-impl-set invalid_implementation], [2], [], [dnl +Error: unknown action implementation, invalid_implementation, specified! +ovs-appctl: ovs-vswitchd: server returned an error +]) + +OVS_VSWITCHD_STOP(["/Failed setting action implementation to invalid_implementation/d"]) +AT_CLEANUP diff --git a/vswitchd/ovs-vswitchd.8.in b/vswitchd/ovs-vswitchd.8.in index 1a32402be..9569265fc 100644 --- a/vswitchd/ovs-vswitchd.8.in +++ b/vswitchd/ovs-vswitchd.8.in @@ -282,6 +282,7 @@ type). .so lib/dpif-netdev-unixctl.man .so lib/dpif-netlink-unixctl.man .so lib/netdev-dpdk-unixctl.man +.so lib/odp-execute-unixctl.man .so ofproto/ofproto-dpif-unixctl.man .so ofproto/ofproto-unixctl.man .so lib/vlog-unixctl.man From patchwork Wed Jul 13 18:28:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Van Haaren, Harry" X-Patchwork-Id: 1656123 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=kDXk7/mo; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LjmNF2mxzz9sB4 for ; Thu, 14 Jul 2022 04:28:45 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 726FB424DA; Wed, 13 Jul 2022 18:28:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 726FB424DA Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=kDXk7/mo X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W5xspYeXSWPO; Wed, 13 Jul 2022 18:28:41 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id EDA7142499; Wed, 13 Jul 2022 18:28:38 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org EDA7142499 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id A8841C0035; Wed, 13 Jul 2022 18:28:38 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1D06FC0071 for ; Wed, 13 Jul 2022 18:28:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id E182B60B9B for ; Wed, 13 Jul 2022 18:28:28 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org E182B60B9B Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=kDXk7/mo X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 94YdyXan9GYL for ; Wed, 13 Jul 2022 18:28:28 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org BF6F8613C6 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by smtp3.osuosl.org (Postfix) with ESMTPS id BF6F8613C6 for ; Wed, 13 Jul 2022 18:28:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657736907; x=1689272907; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=e27M6UxlybkfA5Zr5y9uzJzIt9e9m33n36+UhtR3QBA=; b=kDXk7/moS+RXO8qUqo4NHtUfhQNGEiFs0vykYGyU9xfOIVtE22AQ/tFh 8OlWKaC2ffYCTHZk/rk31/zkTg7yh1vTP+rDAZWDSgb5rdX/wKqLCy52N J/tj8tPwH5UWIW2mYLrJizBAMuzfHx1nk4U3lyRU+FDbzAXV+p91hrgZh krKlxaNi6dQLdAkaBr7S3J8XJk82HLUFzme6URlqg3oYmXxm0jv9tqhwO voh6mv4ASHJu5xVXZ5bN0S5st6S4W6CiJNvnyOAWGTs708cI964Eic1MG 0/sxEi78rQbG8iSOWBdeukAHO+BsQ2zKHRJlUduRYJrU6ugFiAas2p8CK A==; X-IronPort-AV: E=McAfee;i="6400,9594,10407"; a="284067991" X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="284067991" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jul 2022 11:28:27 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="545962446" Received: from silpixa00401454.ir.intel.com ([10.55.128.122]) by orsmga003.jf.intel.com with ESMTP; 13 Jul 2022 11:28:25 -0700 From: Harry van Haaren To: dev@openvswitch.org Date: Wed, 13 Jul 2022 18:28:02 +0000 Message-Id: <20220713182807.3416578-6-harry.van.haaren@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220713182807.3416578-1-harry.van.haaren@intel.com> References: <20220712174456.2682549-1-harry.van.haaren@intel.com> <20220713182807.3416578-1-harry.van.haaren@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org, kumar.amber@intel.com Subject: [ovs-dev] [PATCH v10 05/10] acinclude: Add configure option to enable actions autovalidator at build time. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Kumar Amber This commit adds a new command to allow the user to enable the actions autovalidator by default at build time thus allowing for running unit test by default. $ ./configure --enable-actions-default-autovalidator Signed-off-by: Kumar Amber Acked-by: Harry van Haaren Acked-by: Sunil Pai G --- v9: - rebase conflict on NEWS - fixup missing "dnl" in comment introduced by previous line-wrapping --- NEWS | 2 ++ acinclude.m4 | 20 ++++++++++++++++++++ configure.ac | 1 + lib/odp-execute.c | 4 ++++ 4 files changed, 27 insertions(+) diff --git a/NEWS b/NEWS index d02733936..bb03457f6 100644 --- a/NEWS +++ b/NEWS @@ -54,6 +54,8 @@ Post-v2.17.0 implementations against default implementation. * Add command line option to switch between different actions implementations available at run time. + * Add build time configure command to enable auto-validator as default + actions implementation at build time. - Linux datapath: * Add offloading meter tc police. * Add support for offloading the check_pkt_len action. diff --git a/acinclude.m4 b/acinclude.m4 index d15f11a4e..84fad425c 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -14,6 +14,26 @@ # See the License for the specific language governing permissions and # limitations under the License. +dnl Set OVS Actions Autovalidator as the default action implementation +dnl at compile time This enables automatically running all unit tests +dnl with all actions implementations. +AC_DEFUN([OVS_CHECK_ACTIONS_AUTOVALIDATOR], [ + AC_ARG_ENABLE([actions-default-autovalidator], + [AC_HELP_STRING([--enable-actions-default-autovalidator], + [Enable actions autovalidator as default + ovs actions implementation.])], + [autovalidator=yes],[autovalidator=no]) + AC_MSG_CHECKING([whether actions Autovalidator is default implementation]) + if test "$autovalidator" != yes; then + AC_MSG_RESULT([no]) + else + AC_DEFINE([ACTIONS_AUTOVALIDATOR_DEFAULT], [1], + [Autovalidator for actions is a default implementation.]) + AC_MSG_RESULT([yes]) + fi +]) + + dnl Set OVS MFEX Autovalidator as default miniflow extract at compile time? dnl This enables automatically running all unit tests with all MFEX dnl implementations. diff --git a/configure.ac b/configure.ac index 6f8679d7c..6c51e48ce 100644 --- a/configure.ac +++ b/configure.ac @@ -184,6 +184,7 @@ OVS_CONDITIONAL_CC_OPTION([-Wno-unused-parameter], [HAVE_WNO_UNUSED_PARAMETER]) OVS_ENABLE_WERROR_TOP OVS_ENABLE_SPARSE OVS_CTAGS_IDENTIFIERS +OVS_CHECK_ACTIONS_AUTOVALIDATOR OVS_CHECK_DPCLS_AUTOVALIDATOR OVS_CHECK_DPIF_AVX512_DEFAULT OVS_CHECK_MFEX_AUTOVALIDATOR diff --git a/lib/odp-execute.c b/lib/odp-execute.c index d5be190e0..f112f3b48 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -928,7 +928,11 @@ odp_execute_init(void) static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; if (ovsthread_once_start(&once)) { odp_execute_action_init(); +#ifdef ACTIONS_AUTOVALIDATOR_DEFAULT + odp_actions_impl_set("autovalidator"); +#else odp_actions_impl_set("scalar"); +#endif odp_execute_unixctl_init(); ovsthread_once_done(&once); } From patchwork Wed Jul 13 18:28:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Van Haaren, Harry" X-Patchwork-Id: 1656124 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=Qgx0WIJ3; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LjmNK06H8z9sB4 for ; Thu, 14 Jul 2022 04:28:48 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id E92A6424C6; Wed, 13 Jul 2022 18:28:46 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org E92A6424C6 Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=Qgx0WIJ3 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ySfmliD266ct; Wed, 13 Jul 2022 18:28:45 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 0E37A4249F; Wed, 13 Jul 2022 18:28:42 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 0E37A4249F Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D8C65C007F; Wed, 13 Jul 2022 18:28:39 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 10986C0088 for ; Wed, 13 Jul 2022 18:28:34 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 59D22613E1 for ; Wed, 13 Jul 2022 18:28:31 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 59D22613E1 Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=Qgx0WIJ3 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hGcpci1cKEbb for ; Wed, 13 Jul 2022 18:28:30 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 0B8AF613CE Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by smtp3.osuosl.org (Postfix) with ESMTPS id 0B8AF613CE for ; Wed, 13 Jul 2022 18:28:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657736910; x=1689272910; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cYI/7DHmAJtk1jMzEpolbQz+CjHnYnG/EPXaRxql5hU=; b=Qgx0WIJ3eeC2lVWcmYKFo4TbiU9wGfMm2NbMpP+U5utBkmrSSO9LM6Gu VaN+MvSbMJLPquJ6uNDLqhIIV31A3UwhpqYG38AHJ3GtVSjxqG39mtxc3 ZUuqx4gAfohCY/H65y4Kk3gDUMVHnOILliSNBxWM6Bgp9r68CClFEUIdD K0bDI/c1R1cukrVquqT8YOwAUFnnW/TnvwCbjpYKLnGR/IDFctrBBTOOO PId9QD5JdfNWjvs4JYVat6I3Axb+cHLwABCiGTPWc6Qu/fIY7aoR5kGjY y840asiBp/ufRKBQ9J8kUPkytsPWcAFERnlD67g8LdhIJrkfdc0PKXS1u A==; X-IronPort-AV: E=McAfee;i="6400,9594,10407"; a="284067996" X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="284067996" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jul 2022 11:28:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="545962456" Received: from silpixa00401454.ir.intel.com ([10.55.128.122]) by orsmga003.jf.intel.com with ESMTP; 13 Jul 2022 11:28:27 -0700 From: Harry van Haaren To: dev@openvswitch.org Date: Wed, 13 Jul 2022 18:28:03 +0000 Message-Id: <20220713182807.3416578-7-harry.van.haaren@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220713182807.3416578-1-harry.van.haaren@intel.com> References: <20220712174456.2682549-1-harry.van.haaren@intel.com> <20220713182807.3416578-1-harry.van.haaren@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org, kumar.amber@intel.com Subject: [ovs-dev] [PATCH v10 06/10] odp-execute: Add ISA implementation of actions. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Emma Finn This commit adds the AVX512 implementation of the action functionality. Usage: $ ovs-appctl odp-execute/action-impl-set avx512 Signed-off-by: Emma Finn Signed-off-by: Harry van Haaren Tested-by: Kumar Amber Acked-by: Sunil Pai G --- v10: - Move location and rework documentation (Ilya) - Improve ISA checks to fix CI build issue v9: rebase conflict on NEWS --- Documentation/topics/dpdk/bridge.rst | 30 ++++++++++++++++++++++++++++ Documentation/topics/testing.rst | 24 ++++++++++++++-------- NEWS | 1 + acinclude.m4 | 1 + lib/cpu.c | 1 + lib/cpu.h | 1 + lib/odp-execute-private.c | 8 ++++++++ lib/odp-execute-private.h | 12 +++++++++++ m4/openvswitch.m4 | 29 +++++++++++++++++++++++++++ 9 files changed, 99 insertions(+), 8 deletions(-) diff --git a/Documentation/topics/dpdk/bridge.rst b/Documentation/topics/dpdk/bridge.rst index 1f626c7c2..354f1ced1 100644 --- a/Documentation/topics/dpdk/bridge.rst +++ b/Documentation/topics/dpdk/bridge.rst @@ -321,3 +321,33 @@ following command:: ``scalar`` can be selected on core ``3`` by the following command:: $ ovs-appctl dpif-netdev/miniflow-parser-set -pmd 3 scalar + + +Actions Implementations (Experimental) +-------------------------------------- + +Actions describe what processing or modification should be performed on a +packet when it matches a given flow. Similar to the datapath interface, +DPCLS and MFEX (see above), the implementation of these actions can be +accelerated using SIMD instructions, resulting in improved performance. + +OVS provides multiple implementations of the actions, however some +implementations requiring a CPU capable of executing the required SIMD +instructions. + +Available implementations can be listed with the following command:: + + $ ovs-appctl odp-execute/action-impl-show + Available Actions implementations: + scalar (available: Yes, active: Yes) + autovalidator (available: Yes, active: No) + avx512 (available: Yes, active: No) + +By default, ``scalar`` is used. Implementations can be selected by +name:: + + $ ovs-appctl odp-execute/action-impl-set avx512 + Action implementation set to avx512. + + $ ovs-appctl odp-execute/action-impl-set scalar + Action implementation set to scalar. diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst index c15d5b38f..a6c747b18 100644 --- a/Documentation/topics/testing.rst +++ b/Documentation/topics/testing.rst @@ -361,12 +361,12 @@ testsuite. Userspace datapath: Testing and Validation of CPU-specific Optimizations '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' -As multiple versions of the datapath classifier and packet parsing functions -can co-exist, each with different CPU ISA optimizations, it is important to -validate that they all give the exact same results. To easily test all the -implementations, an ``autovalidator`` implementation of them exists. This -implementation runs all other available implementations, and verifies that the -results are identical. +As multiple versions of the datapath classifier, packet parsing functions and +actions can co-exist, each with different CPU ISA optimizations, it is +important to validate that they all give the exact same results. To easily +test all the implementations, an ``autovalidator`` implementation of them +exists. This implementation runs all other available implementations, and +verifies that the results are identical. Running the OVS unit tests with the autovalidator enabled ensures all implementations provide the same results. Note that the performance of the @@ -382,18 +382,26 @@ To set the autovalidator for the packet parser, use this command:: $ ovs-appctl dpif-netdev/miniflow-parser-set autovalidator +To set the autovalidator for actions, use this command:: + + $ ovs-appctl odp-execute/action-impl-set autovalidator + To run the OVS unit test suite with the autovalidator as the default implementation, it is required to recompile OVS. During the recompilation, the default priority of the `autovalidator` implementation is set to the -maximum priority, ensuring every test will be run with every implementation:: +maximum priority, ensuring every test will be run with every implementation. +Priority is only related to mfex autovalidator and not the actions +autovalidator.:: - $ ./configure --enable-autovalidator --enable-mfex-default-autovalidator + $ ./configure --enable-autovalidator --enable-mfex-default-autovalidator \ + --enable-actions-default-autovalidator The following line should be seen in the configuration log when the above options are used:: checking whether DPCLS Autovalidator is default implementation... yes checking whether MFEX Autovalidator is default implementation... yes + checking whether actions Autovalidator is default implementation... yes Compile OVS in debug mode to have `ovs_assert` statements error out if there is a mis-match in the datapath classifier lookup or packet parser diff --git a/NEWS b/NEWS index bb03457f6..94f9d7f66 100644 --- a/NEWS +++ b/NEWS @@ -56,6 +56,7 @@ Post-v2.17.0 implementations available at run time. * Add build time configure command to enable auto-validator as default actions implementation at build time. + * Add AVX512 implementation of actions. - Linux datapath: * Add offloading meter tc police. * Add support for offloading the check_pkt_len action. diff --git a/acinclude.m4 b/acinclude.m4 index 84fad425c..555e50956 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -96,6 +96,7 @@ dnl dnl Checks if compiler and binutils supports various AVX512 ISA. AC_DEFUN([OVS_CHECK_AVX512], [ OVS_CHECK_BINUTILS_AVX512 + OVS_CHECK_GCC_AVX512VL OVS_CONDITIONAL_CC_OPTION_DEFINE([-mavx512f], [HAVE_AVX512F]) OVS_CONDITIONAL_CC_OPTION_DEFINE([-mavx512bw], [HAVE_AVX512BW]) OVS_CONDITIONAL_CC_OPTION_DEFINE([-mavx512vl], [HAVE_AVX512VL]) diff --git a/lib/cpu.c b/lib/cpu.c index 2df003c51..0292f715e 100644 --- a/lib/cpu.c +++ b/lib/cpu.c @@ -53,6 +53,7 @@ X86_ISA(X86_EXT_FEATURES_LEAF, EBX, 16, OVS_CPU_ISA_X86_AVX512F) X86_ISA(X86_EXT_FEATURES_LEAF, EBX, 30, OVS_CPU_ISA_X86_AVX512BW) X86_ISA(X86_EXT_FEATURES_LEAF, ECX, 1, OVS_CPU_ISA_X86_AVX512VBMI) X86_ISA(X86_EXT_FEATURES_LEAF, ECX, 14, OVS_CPU_ISA_X86_VPOPCNTDQ) +X86_ISA(X86_EXT_FEATURES_LEAF, EBX, 31, OVS_CPU_ISA_X86_AVX512VL) #endif bool diff --git a/lib/cpu.h b/lib/cpu.h index 92897bb71..3215229bc 100644 --- a/lib/cpu.h +++ b/lib/cpu.h @@ -25,6 +25,7 @@ enum ovs_cpu_isa { OVS_CPU_ISA_X86_AVX512F, OVS_CPU_ISA_X86_AVX512BW, OVS_CPU_ISA_X86_AVX512VBMI, + OVS_CPU_ISA_X86_AVX512VL, OVS_CPU_ISA_X86_VPOPCNTDQ, OVS_CPU_ISA_X86_LAST = OVS_CPU_ISA_X86_VPOPCNTDQ, }; diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 38be22ec9..3591da2e5 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -41,6 +41,14 @@ static struct odp_execute_action_impl action_impls[] = { .name = "scalar", .init_func = odp_action_scalar_init, }, + +#if ACTION_IMPL_AVX512_CHECK + [ACTION_IMPL_AVX512] = { + .available = false, + .name = "avx512", + .init_func = NULL, + }, +#endif }; static void diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index d6eebbf37..1943eb600 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -22,6 +22,15 @@ #include "odp-netlink.h" #include "ovs-atomic.h" +/* Combine all required ISA and Linker checks into a single #define + * for readability and simplicity where the checks are needed. Note + * that it is always #defined, so code must use the #if preprocesor + * directive (not #ifdef). + */ +#define ACTION_IMPL_AVX512_CHECK (__x86_64__ && HAVE_AVX512F \ + && HAVE_LD_AVX512_GOOD && __SSE4_2__ && HAVE_AVX512BW && HAVE_AVX512VL \ + && HAVE_GCC_AVX512VL_GOOD) + /* Forward declaration for typedef. */ struct odp_execute_action_impl; @@ -59,6 +68,9 @@ enum odp_execute_action_impl_idx { * Do not change the autovalidator position in this list without updating * the define below. */ +#if ACTION_IMPL_AVX512_CHECK + ACTION_IMPL_AVX512, +#endif ACTION_IMPL_MAX, }; diff --git a/m4/openvswitch.m4 b/m4/openvswitch.m4 index 4c3bace6e..fe51455b4 100644 --- a/m4/openvswitch.m4 +++ b/m4/openvswitch.m4 @@ -421,6 +421,35 @@ AC_DEFUN([OVS_CHECK_SPHINX], AC_ARG_VAR([SPHINXBUILD]) AM_CONDITIONAL([HAVE_SPHINX], [test "$SPHINXBUILD" != none])]) + +dnl Checks for compiler correctly emitting AVX512-VL vpermd instruction. +dnl GCC5 says it exports AVX512-VL, but it doesn't implement "vpermd" instruction +dnl resulting in compilation failures. To workaround this "reported vs actual" +dnl mismatch, we compile a small snippet, and conditionally enable AVX512-VL. +AC_DEFUN([OVS_CHECK_GCC_AVX512VL], [ + AC_MSG_CHECKING([whether compiler correctly emits AVX512-VL]) + AC_COMPILE_IFELSE( + [AC_LANG_PROGRAM([#include + static void __attribute__((__target__("avx512vl"))) + check_permutexvar(void) + { + __m256i v_swap32a = _mm256_setr_epi32(0x0, 0x4, 0xF, + 0xF, 0xF, 0xF, + 0xF, 0xF); + v_swap32a = _mm256_permutexvar_epi32(v_swap32a, + v_swap32a); + }],[])], + [AC_MSG_RESULT([yes]) + ovs_cv_gcc_avx512vl_good=yes], + [AC_MSG_RESULT([no]) + ovs_cv_gcc_avx512vl_good=no]) + if test "$ovs_cv_gcc_avx512vl_good" = yes; then + AC_DEFINE([HAVE_GCC_AVX512VL_GOOD], [1], + [Define to 1 if gcc implements the vpermd instruction.]) + fi + AM_CONDITIONAL([HAVE_GCC_AVX512VL_GOOD], + [test "$ovs_cv_gcc_avx512vl_good" = yes])]) + dnl Checks for binutils/assembler known issue with AVX512. dnl Due to backports, we probe assembling a reproducer instead of checking dnl binutils version string. More details, including ASM dumps and debug here: From patchwork Wed Jul 13 18:28:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Van Haaren, Harry" X-Patchwork-Id: 1656125 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=OqLrncc7; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LjmNS41rrz9sB4 for ; Thu, 14 Jul 2022 04:28:56 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 3B70C8466F; Wed, 13 Jul 2022 18:28:54 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 3B70C8466F Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=OqLrncc7 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7F6Bn_kEESJ2; Wed, 13 Jul 2022 18:28:51 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 36C32840B4; Wed, 13 Jul 2022 18:28:45 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 36C32840B4 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1A478C0085; Wed, 13 Jul 2022 18:28:41 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1C725C0035 for ; Wed, 13 Jul 2022 18:28:37 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 20A64613CE for ; Wed, 13 Jul 2022 18:28:34 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 20A64613CE Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=OqLrncc7 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wgxe05DF45EG for ; Wed, 13 Jul 2022 18:28:32 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 751EB613F8 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by smtp3.osuosl.org (Postfix) with ESMTPS id 751EB613F8 for ; Wed, 13 Jul 2022 18:28:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657736912; x=1689272912; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xHgAYdYXLe7LmKA+uKZ3npQSmPlfQRkls7FvGz6hcII=; b=OqLrncc7P5HZykJPb+dfRVejzTs75Viwq17e/jznmpkf0eYOLiT6KFrb vqjWSinw9Uu/eQx62HswvkZxZFYwZg9ZECJHbDLt4XmQ5rKgt0bBvfbBP FB2BzJ6KeVPx1ysWrwYhiNuE56lCoqBvo1nX8mhnU2zHBrpdHB/vSFqXZ eSXOdfzwM89LIe9nRuFcZAp6x35dR7QrtI2Zbk+16J7jBYHneFETqqJH6 bKLND2iU+qz2LXUooyyYx52hko+75qoJh2E7F0V6gCPYZ5PQ4aZrDWT4J 3OHtHk9JO8qcvL4NvXYyqAhK6AQf1CCICYlUGJrr1tM9ysWh3MrHeu5SH A==; X-IronPort-AV: E=McAfee;i="6400,9594,10407"; a="284068004" X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="284068004" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jul 2022 11:28:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="545962461" Received: from silpixa00401454.ir.intel.com ([10.55.128.122]) by orsmga003.jf.intel.com with ESMTP; 13 Jul 2022 11:28:29 -0700 From: Harry van Haaren To: dev@openvswitch.org Date: Wed, 13 Jul 2022 18:28:04 +0000 Message-Id: <20220713182807.3416578-8-harry.van.haaren@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220713182807.3416578-1-harry.van.haaren@intel.com> References: <20220712174456.2682549-1-harry.van.haaren@intel.com> <20220713182807.3416578-1-harry.van.haaren@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org, kumar.amber@intel.com Subject: [ovs-dev] [PATCH v10 07/10] odp-execute: Add ISA implementation of pop_vlan action. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit adds the AVX512 implementation of the pop_vlan action. Signed-off-by: Emma Finn Tested-by: Kumar Amber Acked-by: Sunil Pai G --- v10: - Improved ISA checks to fix CI build --- lib/automake.mk | 4 + lib/odp-execute-avx512.c | 182 ++++++++++++++++++++++++++++++++++++++ lib/odp-execute-private.c | 33 ++++++- lib/odp-execute-private.h | 2 + 4 files changed, 220 insertions(+), 1 deletion(-) create mode 100644 lib/odp-execute-avx512.c diff --git a/lib/automake.mk b/lib/automake.mk index 5c3b05f6b..a76de6dbf 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -45,6 +45,10 @@ lib_libopenvswitchavx512_la_CFLAGS += \ lib_libopenvswitchavx512_la_SOURCES += \ lib/dpif-netdev-extract-avx512.c \ lib/dpif-netdev-lookup-avx512-gather.c +if HAVE_GCC_AVX512VL_GOOD +lib_libopenvswitchavx512_la_SOURCES += \ + lib/odp-execute-avx512.c +endif # HAVE_GCC_AVX512VL_GOOD endif # HAVE_AVX512VL endif # HAVE_AVX512BW lib_libopenvswitchavx512_la_LDFLAGS = \ diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c new file mode 100644 index 000000000..fd10f7f5c --- /dev/null +++ b/lib/odp-execute-avx512.c @@ -0,0 +1,182 @@ +/* + * Copyright (c) 2022 Intel. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifdef __x86_64__ +/* Sparse cannot handle the AVX512 instructions. */ +#if !defined(__CHECKER__) + +#include +#include + +#include "dp-packet.h" +#include "immintrin.h" +#include "odp-execute-private.h" +#include "odp-netlink.h" +#include "openvswitch/vlog.h" + +VLOG_DEFINE_THIS_MODULE(odp_execute_avx512); + +/* The below three build asserts make sure that l2_5_ofs, l3_ofs, and l4_ofs + * fields remain in the same order and offset to l2_padd_size. This is needed + * as the avx512_dp_packet_resize_l2() function will manipulate those fields at + * a fixed memory index based on the l2_padd_size offset. */ +BUILD_ASSERT_DECL(offsetof(struct dp_packet, l2_pad_size) + + MEMBER_SIZEOF(struct dp_packet, l2_pad_size) == + offsetof(struct dp_packet, l2_5_ofs)); + +BUILD_ASSERT_DECL(offsetof(struct dp_packet, l2_5_ofs) + + MEMBER_SIZEOF(struct dp_packet, l2_5_ofs) == + offsetof(struct dp_packet, l3_ofs)); + +BUILD_ASSERT_DECL(offsetof(struct dp_packet, l3_ofs) + + MEMBER_SIZEOF(struct dp_packet, l3_ofs) == + offsetof(struct dp_packet, l4_ofs)); + +/* The below build assert makes sure it's safe to read/write 128-bits starting + * at the l2_pad_size location. */ +BUILD_ASSERT_DECL(sizeof(struct dp_packet) - + offsetof(struct dp_packet, l2_pad_size) >= sizeof(__m128i)); + +static inline void ALWAYS_INLINE +avx512_dp_packet_resize_l2(struct dp_packet *b, int resize_by_bytes) +{ + /* Update packet size/data pointers, same as the scalar implementation. */ + if (resize_by_bytes >= 0) { + dp_packet_push_uninit(b, resize_by_bytes); + } else { + dp_packet_pull(b, -resize_by_bytes); + } + + /* The next step is to update the l2_5_ofs, l3_ofs and l4_ofs fields which + * the scalar implementation does with the dp_packet_adjust_layer_offset() + * function. */ + + /* Set the v_zero register to all zero's. */ + const __m128i v_zeros = _mm_setzero_si128(); + + /* Set the v_u16_max register to all one's. */ + const __m128i v_u16_max = _mm_cmpeq_epi16(v_zeros, v_zeros); + + /* Each lane represents 16 bits in a 12-bit register. In this case the + * first three 16-bit values, which will map to the l2_5_ofs, l3_ofs and + * l4_ofs fields. */ + const uint8_t k_lanes = 0b1110; + + /* Set all 16-bit words in the 128-bits v_offset register to the value we + * need to add/substract from the l2_5_ofs, l3_ofs, and l4_ofs fields. */ + __m128i v_offset = _mm_set1_epi16(abs(resize_by_bytes)); + + /* Load 128 bits from the dp_packet structure starting at the l2_pad_size + * offset. */ + void *adjust_ptr = &b->l2_pad_size; + __m128i v_adjust_src = _mm_loadu_si128(adjust_ptr); + + /* Here is the tricky part, we only need to update the value of the three + * fields if they are not UINT16_MAX. The following function will return + * a mask of lanes (read fields) that are not UINT16_MAX. It will do this + * by comparing only the lanes we requested, k_lanes, and if they match + * v_u16_max, the bit will be set. */ + __mmask8 k_cmp = _mm_mask_cmpneq_epu16_mask(k_lanes, v_adjust_src, + v_u16_max); + + /* Based on the bytes adjust (positive, or negative) it will do the actual + * add or subtraction. These functions will only operate on the lanes + * (fields) requested based on k_cmp, i.e: + * k_cmp = [l2_5_ofs, l3_ofs, l4_ofs] + * for field in kcmp + * v_adjust_src[field] = v_adjust_src[field] + v_offset + */ + __m128i v_adjust_wip; + + if (resize_by_bytes >= 0) { + v_adjust_wip = _mm_mask_add_epi16(v_adjust_src, k_cmp, + v_adjust_src, v_offset); + } else { + v_adjust_wip = _mm_mask_sub_epi16(v_adjust_src, k_cmp, + v_adjust_src, v_offset); + } + + /* Here we write back the full 128-bits. */ + _mm_storeu_si128(adjust_ptr, v_adjust_wip); +} + +/* This function performs the same operation on each packet in the batch as + * the scalar eth_pop_vlan() function. */ +static void +action_avx512_pop_vlan(struct dp_packet_batch *batch, + const struct nlattr *a OVS_UNUSED) +{ + struct dp_packet *packet; + + /* Set the v_zero register to all zero's. */ + const __m128i v_zeros = _mm_setzero_si128(); + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + struct vlan_eth_header *veh = dp_packet_eth(packet); + + if (veh && dp_packet_size(packet) >= sizeof *veh && + eth_type_vlan(veh->veth_type)) { + + /* Load the first 128-bits of l2 header into the v_ether register. + * This result in the veth_dst/src and veth_type/tci of the + * vlan_eth_header structure to be loaded. */ + __m128i v_ether = _mm_loadu_si128((void *) veh); + + /* This creates a 256-bit value containing the first four fields + * of the vlan_eth_header plus 128 zero-bit. The result will be the + * lowest 128-bits after the right shift, hence we shift the data + * 128(zero)-bits minus the VLAN_HEADER_LEN, so we are left with + * only the veth_dst and veth_src fields. */ + __m128i v_realign = _mm_alignr_epi8(v_ether, v_zeros, + sizeof(__m128i) - + VLAN_HEADER_LEN); + + /* Write back the modified ethernet header. */ + _mm_storeu_si128((void *) veh, v_realign); + + /* As we removed the VLAN_HEADER we now need to adjust all the + * offsets. */ + avx512_dp_packet_resize_l2(packet, -VLAN_HEADER_LEN); + } + } +} + +int +action_avx512_init(struct odp_execute_action_impl *self OVS_UNUSED) +{ + /* Set function pointers for actions that can be applied directly, these + * are identified by OVS_ACTION_ATTR_*. */ + self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_avx512_pop_vlan; + return 0; +} + +#endif /* Sparse */ + +#else /* __x86_64__ */ + +#include +#include "odp-execute-private.h" +/* Function itself is required to be called, even in e.g. 32-bit builds. + * This dummy init function ensures 32-bit builds succeed too. + */ + +int +action_avx512_init(struct odp_execute_action_impl *self OVS_UNUSED) +{ + return 0; +} + +#endif diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 3591da2e5..2fabf6c62 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -19,6 +19,7 @@ #include #include +#include "cpu.h" #include "dpdk.h" #include "dp-packet.h" #include "odp-execute-private.h" @@ -29,6 +30,36 @@ VLOG_DEFINE_THIS_MODULE(odp_execute_impl); static int active_action_impl_index; +#if ACTION_IMPL_AVX512_CHECK +/* Probe functions to check ISA requirements. */ +static bool +action_avx512_isa_probe(void) +{ + static enum ovs_cpu_isa isa_required[] = { + OVS_CPU_ISA_X86_AVX512F, + OVS_CPU_ISA_X86_AVX512BW, + OVS_CPU_ISA_X86_BMI2, + OVS_CPU_ISA_X86_AVX512VL, + }; + for (int i = 0; i < ARRAY_SIZE(isa_required); i++) { + if (!cpu_has_isa(isa_required[i])) { + return false; + } + } + return true; +} +static int +action_avx512_probe(struct odp_execute_action_impl *self) +{ + if (!action_avx512_isa_probe()) { + return -ENOTSUP; + } else { + action_avx512_init(self); + } + return 0; +} +#endif + static struct odp_execute_action_impl action_impls[] = { [ACTION_IMPL_AUTOVALIDATOR] = { .available = false, @@ -46,7 +77,7 @@ static struct odp_execute_action_impl action_impls[] = { [ACTION_IMPL_AVX512] = { .available = false, .name = "avx512", - .init_func = NULL, + .init_func = action_avx512_probe, }, #endif }; diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index 1943eb600..f66e6e6d1 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -96,6 +96,8 @@ struct odp_execute_action_impl * odp_execute_action_set(const char *name); int action_autoval_init(struct odp_execute_action_impl *self); +int action_avx512_init(struct odp_execute_action_impl *self); + void odp_execute_action_get_info(struct ds *name); #endif /* ODP_EXTRACT_PRIVATE */ From patchwork Wed Jul 13 18:28:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Van Haaren, Harry" X-Patchwork-Id: 1656126 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=PB/Jai51; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LjmNW6D1Xz9sB4 for ; Thu, 14 Jul 2022 04:28:59 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 70FC484735; Wed, 13 Jul 2022 18:28:57 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 70FC484735 Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=PB/Jai51 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HjXN-YHs_8GN; Wed, 13 Jul 2022 18:28:54 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id C9483846F5; Wed, 13 Jul 2022 18:28:47 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org C9483846F5 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 55109C0071; Wed, 13 Jul 2022 18:28:47 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0DAF1C0088 for ; Wed, 13 Jul 2022 18:28:43 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 2D93A6140C for ; Wed, 13 Jul 2022 18:28:35 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 2D93A6140C Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=PB/Jai51 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KZSu_7O6dbm0 for ; Wed, 13 Jul 2022 18:28:34 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 45AB861405 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by smtp3.osuosl.org (Postfix) with ESMTPS id 45AB861405 for ; Wed, 13 Jul 2022 18:28:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657736914; x=1689272914; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=C0q30RSeMEFgp502BJfdf8oi/XFbeBlnFHD11e9sa+U=; b=PB/Jai51Mb6bVX4v+4hNOhBThgRcCwSj1ErH8hjuIqRW/TO9yN+lBMxg 9pGYBRXS1OnWEjLR+foKsNGVjDVrcuotEAS25yP+IoUbHqkvLpaSaByeh otpiODOZd+rcyaLpRbZHk+xFE7A35gVkTPtku1siJw/7PyDaZ2IP+Ncpv VEH8T9wyZ+7X4KCYPfQ5EI/nK0noueqUXdM2rRBpOt+kf4WEYSD/9gFly fuUyvuByTHXmYOhDZrodrdd8Ej+3R4Kbsa1T0otJcz2WMAqZ4KauO8fYn P8H1mVnP6Cu0DsN4FXMA1g6GpvXq5ivQ7ufRslMT9nPt8I/HkCzi4SSen A==; X-IronPort-AV: E=McAfee;i="6400,9594,10407"; a="284068010" X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="284068010" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jul 2022 11:28:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="545962469" Received: from silpixa00401454.ir.intel.com ([10.55.128.122]) by orsmga003.jf.intel.com with ESMTP; 13 Jul 2022 11:28:32 -0700 From: Harry van Haaren To: dev@openvswitch.org Date: Wed, 13 Jul 2022 18:28:05 +0000 Message-Id: <20220713182807.3416578-9-harry.van.haaren@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220713182807.3416578-1-harry.van.haaren@intel.com> References: <20220712174456.2682549-1-harry.van.haaren@intel.com> <20220713182807.3416578-1-harry.van.haaren@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org, kumar.amber@intel.com Subject: [ovs-dev] [PATCH v10 08/10] odp-execute: Add ISA implementation of push_vlan action. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Emma Finn This commit adds the AVX512 implementation of the push_vlan action. Signed-off-by: Emma Finn Tested-by: Kumar Amber Acked-by: Sunil Pai G Acked-by: Eelco Chaudron --- lib/odp-execute-avx512.c | 55 ++++++++++++++++++++++++++++++++++++++++ lib/odp-execute.c | 22 +++++++++------- 2 files changed, 68 insertions(+), 9 deletions(-) diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c index fd10f7f5c..3449acff7 100644 --- a/lib/odp-execute-avx512.c +++ b/lib/odp-execute-avx512.c @@ -154,12 +154,67 @@ action_avx512_pop_vlan(struct dp_packet_batch *batch, } } +/* This function performs the same operation on each packet in the batch as + * the scalar eth_push_vlan() function. */ +static void +action_avx512_push_vlan(struct dp_packet_batch *batch, const struct nlattr *a) +{ + struct dp_packet *packet; + const struct ovs_action_push_vlan *vlan = nl_attr_get(a); + ovs_be16 tpid, tci; + + /* This shuffle mask is used below, and each position tells where to + * move the bytes to. So here, the fourth byte in v_ether is moved to + * byte location 0 in v_shift. The fifth is moved to 1, etc., etc. + * The 0xFF is special it tells to fill that position with 0. + */ + static const uint8_t vlan_push_shuffle_mask[16] = { + 4, 5, 6, 7, 8, 9, 10, 11, + 12, 13, 14, 15, 0xFF, 0xFF, 0xFF, 0xFF + }; + + /* Load the shuffle mask in v_index. */ + __m128i v_index = _mm_loadu_si128((void *) vlan_push_shuffle_mask); + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + tpid = vlan->vlan_tpid; + tci = vlan->vlan_tci; + + /* As we are about to insert the VLAN_HEADER we now need to adjust all + * the offsets. */ + avx512_dp_packet_resize_l2(packet, VLAN_HEADER_LEN); + + char *pkt_data = (char *) dp_packet_data(packet); + + /* Build up the VLAN TCI/TPID in a single uint32_t. */ + const uint32_t tci_proc = tci & htons(~VLAN_CFI); + const uint32_t tpid_tci = (tci_proc << 16) | tpid; + + /* Load the first 128-bits of the packet into the v_ether register. + * Note that this includes the 4 unused bytes (VLAN_HEADER_LEN). */ + __m128i v_ether = _mm_loadu_si128((void *) pkt_data); + + /* Move(shuffle) the veth_dst and veth_src data to create room for + * the vlan header. */ + __m128i v_shift = _mm_shuffle_epi8(v_ether, v_index); + + /* Copy(insert) the 32-bit VLAN header, tpid_tci, at the 3rd 32-bit + * word offset, i.e. ofssetof(vlan_eth_header, veth_type) */ + __m128i v_vlan_hdr = _mm_insert_epi32(v_shift, tpid_tci, 3); + + /* Write back the modified ethernet header. */ + _mm_storeu_si128((void *) pkt_data, v_vlan_hdr); + } +} + int action_avx512_init(struct odp_execute_action_impl *self OVS_UNUSED) { /* Set function pointers for actions that can be applied directly, these * are identified by OVS_ACTION_ATTR_*. */ self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_avx512_pop_vlan; + self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_avx512_push_vlan; + return 0; } diff --git a/lib/odp-execute.c b/lib/odp-execute.c index f112f3b48..0c5837640 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -846,6 +846,17 @@ action_pop_vlan(struct dp_packet_batch *batch, } } +static void +action_push_vlan(struct dp_packet_batch *batch, const struct nlattr *a) +{ + struct dp_packet *packet; + const struct ovs_action_push_vlan *vlan = nl_attr_get(a); + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + eth_push_vlan(packet, vlan->vlan_tpid, vlan->vlan_tci); + } +} + /* Implementation of the scalar actions impl init function. Build up the * array of func ptrs here. */ @@ -855,6 +866,7 @@ odp_action_scalar_init(struct odp_execute_action_impl *self) /* Set function pointers for actions that can be applied directly, these * are identified by OVS_ACTION_ATTR_*. */ self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_pop_vlan; + self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_push_vlan; return 0; } @@ -1045,15 +1057,6 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, break; } - case OVS_ACTION_ATTR_PUSH_VLAN: { - const struct ovs_action_push_vlan *vlan = nl_attr_get(a); - - DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { - eth_push_vlan(packet, vlan->vlan_tpid, vlan->vlan_tci); - } - break; - } - case OVS_ACTION_ATTR_PUSH_MPLS: { const struct ovs_action_push_mpls *mpls = nl_attr_get(a); @@ -1206,6 +1209,7 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, case __OVS_ACTION_ATTR_MAX: /* The following actions are handled by the scalar implementation. */ case OVS_ACTION_ATTR_POP_VLAN: + case OVS_ACTION_ATTR_PUSH_VLAN: OVS_NOT_REACHED(); } From patchwork Wed Jul 13 18:28:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Van Haaren, Harry" X-Patchwork-Id: 1656128 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=V7JAMIC6; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LjmNv5FjNz9sB4 for ; Thu, 14 Jul 2022 04:29:19 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 15A69416B8; Wed, 13 Jul 2022 18:29:17 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 15A69416B8 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=V7JAMIC6 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qoR-UskmAb4o; Wed, 13 Jul 2022 18:29:13 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 9BA20416B7; Wed, 13 Jul 2022 18:29:05 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 9BA20416B7 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5AE86C0035; Wed, 13 Jul 2022 18:29:05 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2AC86C007D for ; Wed, 13 Jul 2022 18:29:04 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 77D59613DF for ; Wed, 13 Jul 2022 18:28:41 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 77D59613DF Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=V7JAMIC6 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0hmsQRDRUo4R for ; Wed, 13 Jul 2022 18:28:36 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 6EDE361400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by smtp3.osuosl.org (Postfix) with ESMTPS id 6EDE361400 for ; Wed, 13 Jul 2022 18:28:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657736916; x=1689272916; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ox4iHjAOjjARPYDOKTbzFB3JLH9JpKJwWZxXS5XIm7A=; b=V7JAMIC6YGkPqF7I0E9XJfa9BBu4gFF4M8cALXiDJVv4niKAJaqXYS8+ KIXUcjv4SfjlcCto6+h2wcC5BMhYv/I8nwsig2lRN0Zo1lNehJ4R4LxJd bc8o22tKgeGyWZgU1z7y7M3bgvy8ZE8bcx3edlIqKdi9RA2yzMjm5sObV lq1xzXGANRI4Jfvo+/a/JZ3JRQqzK0zhyZCB1PlFLCht9MGCGqVxqZPIx sv+fnVGIRocHMXG5gXiAT3u9DLR1kg6Cn8AHvRjBNh20dK3eSu7dB6+F2 c+BwYYixwQtPQGONVqrKd4zX4cdqQaaakhYaarVl44PkrIMtkPP4ZASZz w==; X-IronPort-AV: E=McAfee;i="6400,9594,10407"; a="284068019" X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="284068019" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jul 2022 11:28:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="545962481" Received: from silpixa00401454.ir.intel.com ([10.55.128.122]) by orsmga003.jf.intel.com with ESMTP; 13 Jul 2022 11:28:34 -0700 From: Harry van Haaren To: dev@openvswitch.org Date: Wed, 13 Jul 2022 18:28:06 +0000 Message-Id: <20220713182807.3416578-10-harry.van.haaren@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220713182807.3416578-1-harry.van.haaren@intel.com> References: <20220712174456.2682549-1-harry.van.haaren@intel.com> <20220713182807.3416578-1-harry.van.haaren@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org, kumar.amber@intel.com Subject: [ovs-dev] [PATCH v10 09/10] odp-execute: Add ISA implementation of set_masked ETH X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Emma Finn This commit includes infrastructure changes for enabling set_masked_X actions and also adds support for the AVX512 implementation of the eth_set_addrs action. Signed-off-by: Emma Finn Tested-by: Kumar Amber Acked-by: Sunil Pai G --- lib/odp-execute-avx512.c | 90 +++++++++++++++++++++++++++++++++++++++ lib/odp-execute-private.c | 14 ++++++ lib/odp-execute-private.h | 3 ++ lib/odp-execute.c | 49 +++++++++++---------- lib/odp-execute.h | 3 ++ 5 files changed, 137 insertions(+), 22 deletions(-) diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c index 3449acff7..8ecdaecf6 100644 --- a/lib/odp-execute-avx512.c +++ b/lib/odp-execute-avx512.c @@ -23,6 +23,7 @@ #include "dp-packet.h" #include "immintrin.h" +#include "odp-execute.h" #include "odp-execute-private.h" #include "odp-netlink.h" #include "openvswitch/vlog.h" @@ -50,6 +51,16 @@ BUILD_ASSERT_DECL(offsetof(struct dp_packet, l3_ofs) + BUILD_ASSERT_DECL(sizeof(struct dp_packet) - offsetof(struct dp_packet, l2_pad_size) >= sizeof(__m128i)); +/* The below build assert makes sure the order of the fields needed by + * the set masked functions shuffle operations do not change. This should not + * happen as these are defined under the Linux uapi. */ +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ethernet, eth_src) + + MEMBER_SIZEOF(struct ovs_key_ethernet, eth_src) == + offsetof(struct ovs_key_ethernet, eth_dst)); + +/* Array of callback functions, one for each masked operation. */ +odp_execute_action_cb impl_set_masked_funcs[__OVS_KEY_ATTR_MAX]; + static inline void ALWAYS_INLINE avx512_dp_packet_resize_l2(struct dp_packet *b, int resize_by_bytes) { @@ -207,6 +218,80 @@ action_avx512_push_vlan(struct dp_packet_batch *batch, const struct nlattr *a) } } +/* This function performs the same operation on each packet in the batch as + * the scalar odp_eth_set_addrs() function. */ +static void +action_avx512_eth_set_addrs(struct dp_packet_batch *batch, + const struct nlattr *a) +{ + const struct ovs_key_ethernet *key, *mask; + struct dp_packet *packet; + + a = nl_attr_get(a); + key = nl_attr_get(a); + mask = odp_get_key_mask(a, struct ovs_key_ethernet); + + /* Read the content of the key(src) and mask in the respective registers. + * We only load the src and dest addresses, which is only 96-bits and not + * 128-bits. */ + __m128i v_src = _mm_maskz_loadu_epi32(0x7,(void *) key); + __m128i v_mask = _mm_maskz_loadu_epi32(0x7, (void *) mask); + + + /* These shuffle masks are used below, and each position tells where to + * move the bytes to. So here, the fourth sixth byte in + * ovs_key_ethernet is moved to byte location 0 in v_src/v_mask. + * The seventh is moved to 1, etc., etc. + * This swap is needed to move the src and dest MAC addresses in the + * same order as in the ethernet packet. */ + static const uint8_t eth_shuffle[16] = { + 6, 7, 8, 9, 10, 11, 0, 1, + 2, 3, 4, 5, 0xFF, 0xFF, 0xFF, 0xFF + }; + + /* Load the shuffle mask in v_shuf. */ + __m128i v_shuf = _mm_loadu_si128((void *) eth_shuffle); + + /* Swap the key/mask src and dest addresses to the ethernet order. */ + v_src = _mm_shuffle_epi8(v_src, v_shuf); + v_mask = _mm_shuffle_epi8(v_mask, v_shuf); + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + + struct eth_header *eh = dp_packet_eth(packet); + + if (!eh) { + continue; + } + + /* Load the first 128-bits of the packet into the v_ether register. */ + __m128i v_dst = _mm_loadu_si128((void *) eh); + + /* AND the v_mask to the packet data (v_dst). */ + __m128i dst_masked = _mm_andnot_si128(v_mask, v_dst); + + /* OR the new addresses (v_src) with the masked packet addresses + * (dst_masked). */ + __m128i res = _mm_or_si128(v_src, dst_masked); + + /* Write back the modified ethernet addresses. */ + _mm_storeu_si128((void *) eh, res); + } +} + +static void +action_avx512_set_masked(struct dp_packet_batch *batch, const struct nlattr *a) +{ + const struct nlattr *mask = nl_attr_get(a); + enum ovs_key_attr attr_type = nl_attr_type(mask); + + if (attr_type <= OVS_KEY_ATTR_MAX && impl_set_masked_funcs[attr_type]) { + impl_set_masked_funcs[attr_type](batch, a); + } else { + odp_execute_scalar_action(batch, a); + } +} + int action_avx512_init(struct odp_execute_action_impl *self OVS_UNUSED) { @@ -214,6 +299,11 @@ action_avx512_init(struct odp_execute_action_impl *self OVS_UNUSED) * are identified by OVS_ACTION_ATTR_*. */ self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_avx512_pop_vlan; self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_avx512_push_vlan; + self->funcs[OVS_ACTION_ATTR_SET_MASKED] = action_avx512_set_masked; + + /* Set function pointers for the individual operations supported by the + * SET_MASKED action. */ + impl_set_masked_funcs[OVS_KEY_ATTR_ETHERNET] = action_avx512_eth_set_addrs; return 0; } diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 2fabf6c62..ec42d3d17 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -22,6 +22,7 @@ #include "cpu.h" #include "dpdk.h" #include "dp-packet.h" +#include "odp-execute.h" #include "odp-execute-private.h" #include "odp-netlink.h" #include "odp-util.h" @@ -242,6 +243,19 @@ action_autoval_generic(struct dp_packet_batch *batch, const struct nlattr *a) dp_packet_delete_batch(&original_batch, true); } +void +odp_execute_scalar_action(struct dp_packet_batch *batch, + const struct nlattr *action) +{ + enum ovs_action_attr type = nl_attr_type(action); + + if (action_impls[ACTION_IMPL_SCALAR].funcs[type] && + type <= OVS_ACTION_ATTR_MAX) { + + action_impls[ACTION_IMPL_SCALAR].funcs[type](batch, action); + } +} + int action_autoval_init(struct odp_execute_action_impl *self) { diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index f66e6e6d1..b3707783f 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -100,4 +100,7 @@ int action_avx512_init(struct odp_execute_action_impl *self); void odp_execute_action_get_info(struct ds *name); +void odp_execute_scalar_action(struct dp_packet_batch *batch, + const struct nlattr *action); + #endif /* ODP_EXTRACT_PRIVATE */ diff --git a/lib/odp-execute.c b/lib/odp-execute.c index 0c5837640..dafb198bb 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -562,8 +562,6 @@ odp_execute_set_action(struct dp_packet *packet, const struct nlattr *a) } } -#define get_mask(a, type) ((const type *)(const void *)(a + 1) + 1) - static void odp_execute_masked_set_action(struct dp_packet *packet, const struct nlattr *a) @@ -575,17 +573,17 @@ odp_execute_masked_set_action(struct dp_packet *packet, switch (type) { case OVS_KEY_ATTR_PRIORITY: md->skb_priority = nl_attr_get_u32(a) - | (md->skb_priority & ~*get_mask(a, uint32_t)); + | (md->skb_priority & ~*odp_get_key_mask(a, uint32_t)); break; case OVS_KEY_ATTR_SKB_MARK: md->pkt_mark = nl_attr_get_u32(a) - | (md->pkt_mark & ~*get_mask(a, uint32_t)); + | (md->pkt_mark & ~*odp_get_key_mask(a, uint32_t)); break; case OVS_KEY_ATTR_ETHERNET: odp_eth_set_addrs(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_ethernet)); + odp_get_key_mask(a, struct ovs_key_ethernet)); break; case OVS_KEY_ATTR_NSH: { @@ -595,27 +593,27 @@ odp_execute_masked_set_action(struct dp_packet *packet, case OVS_KEY_ATTR_IPV4: odp_set_ipv4(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_ipv4)); + odp_get_key_mask(a, struct ovs_key_ipv4)); break; case OVS_KEY_ATTR_IPV6: odp_set_ipv6(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_ipv6)); + odp_get_key_mask(a, struct ovs_key_ipv6)); break; case OVS_KEY_ATTR_TCP: odp_set_tcp(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_tcp)); + odp_get_key_mask(a, struct ovs_key_tcp)); break; case OVS_KEY_ATTR_UDP: odp_set_udp(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_udp)); + odp_get_key_mask(a, struct ovs_key_udp)); break; case OVS_KEY_ATTR_SCTP: odp_set_sctp(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_sctp)); + odp_get_key_mask(a, struct ovs_key_sctp)); break; case OVS_KEY_ATTR_MPLS: @@ -623,33 +621,33 @@ odp_execute_masked_set_action(struct dp_packet *packet, if (mh) { put_16aligned_be32(&mh->mpls_lse, nl_attr_get_be32(a) | (get_16aligned_be32(&mh->mpls_lse) - & ~*get_mask(a, ovs_be32))); + & ~*odp_get_key_mask(a, ovs_be32))); } break; case OVS_KEY_ATTR_ARP: set_arp(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_arp)); + odp_get_key_mask(a, struct ovs_key_arp)); break; case OVS_KEY_ATTR_ND: odp_set_nd(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_nd)); + odp_get_key_mask(a, struct ovs_key_nd)); break; case OVS_KEY_ATTR_ND_EXTENSIONS: odp_set_nd_ext(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_nd_extensions)); + odp_get_key_mask(a, struct ovs_key_nd_extensions)); break; case OVS_KEY_ATTR_DP_HASH: md->dp_hash = nl_attr_get_u32(a) - | (md->dp_hash & ~*get_mask(a, uint32_t)); + | (md->dp_hash & ~*odp_get_key_mask(a, uint32_t)); break; case OVS_KEY_ATTR_RECIRC_ID: md->recirc_id = nl_attr_get_u32(a) - | (md->recirc_id & ~*get_mask(a, uint32_t)); + | (md->recirc_id & ~*odp_get_key_mask(a, uint32_t)); break; case OVS_KEY_ATTR_TUNNEL: /* Masked data not supported for tunnel. */ @@ -857,6 +855,17 @@ action_push_vlan(struct dp_packet_batch *batch, const struct nlattr *a) } } +static void +action_set_masked(struct dp_packet_batch *batch, const struct nlattr *a) +{ + const struct nlattr *key = nl_attr_get(a); + struct dp_packet *packet; + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + odp_execute_masked_set_action(packet, key); + } +} + /* Implementation of the scalar actions impl init function. Build up the * array of func ptrs here. */ @@ -867,6 +876,7 @@ odp_action_scalar_init(struct odp_execute_action_impl *self) * are identified by OVS_ACTION_ATTR_*. */ self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_pop_vlan; self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_push_vlan; + self->funcs[OVS_ACTION_ATTR_SET_MASKED] = action_set_masked; return 0; } @@ -1078,12 +1088,6 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, } break; - case OVS_ACTION_ATTR_SET_MASKED: - DP_PACKET_BATCH_FOR_EACH(i, packet, batch) { - odp_execute_masked_set_action(packet, nl_attr_get(a)); - } - break; - case OVS_ACTION_ATTR_SAMPLE: DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { odp_execute_sample(dp, packet, steal && last_action, a, @@ -1210,6 +1214,7 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, /* The following actions are handled by the scalar implementation. */ case OVS_ACTION_ATTR_POP_VLAN: case OVS_ACTION_ATTR_PUSH_VLAN: + case OVS_ACTION_ATTR_SET_MASKED: OVS_NOT_REACHED(); } diff --git a/lib/odp-execute.h b/lib/odp-execute.h index 0921ee924..2ba1ec5d2 100644 --- a/lib/odp-execute.h +++ b/lib/odp-execute.h @@ -46,4 +46,7 @@ void odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, const struct nlattr *actions, size_t actions_len, odp_execute_cb dp_execute_action); + +#define odp_get_key_mask(a, type) ((const type *)(const void *)(a + 1) + 1) + #endif From patchwork Wed Jul 13 18:28:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Van Haaren, Harry" X-Patchwork-Id: 1656127 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=G/aR+dVn; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LjmNd66Pzz9sGD for ; Thu, 14 Jul 2022 04:29:05 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id EDB1841646; Wed, 13 Jul 2022 18:29:03 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org EDB1841646 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=G/aR+dVn X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WchVP2TDLlAP; Wed, 13 Jul 2022 18:29:01 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 21864416C0; Wed, 13 Jul 2022 18:28:55 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 21864416C0 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B318AC0035; Wed, 13 Jul 2022 18:28:54 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1F14BC0035 for ; Wed, 13 Jul 2022 18:28:53 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 6B40F613E1 for ; Wed, 13 Jul 2022 18:28:40 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 6B40F613E1 Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=G/aR+dVn X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uM5_40PATeZs for ; Wed, 13 Jul 2022 18:28:39 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org AF81F613DE Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by smtp3.osuosl.org (Postfix) with ESMTPS id AF81F613DE for ; Wed, 13 Jul 2022 18:28:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657736918; x=1689272918; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=NSe/91uVhPqVbrnCDjYHwC+ehYqyTTksG7pFReFtBXA=; b=G/aR+dVnydsqUBi2l3EISDw94qEsoLauIiZwgxKO3rzuDsgTYbUYq8bD uwLUj1cWoK3oXqTD8zqnCGtv5Fz6xY9YMWyxRAm6UG6ZhitnuAc7WlWF0 OQRRZJRE1XhQeMB/Cn88qYPYgQO4nEqEMwZE6g6KtiPWJEId3xiNU8la3 tQ8MzAsxz9z3VNckNFnUNaQ5YbKX6QF3qMxKcbKVUZNdUe0NAw+2yOqfX wxaCof1tZJvaCboQCfb5gkhLcLHvlV9NK5PZr23MarhFKaVWEHHBcQDBh w5xdjXOy3DdmkB+/nNJtTvcaXILvFGOAQaDpRRyET8GjTM4O/ZzGsqTLQ g==; X-IronPort-AV: E=McAfee;i="6400,9594,10407"; a="284068026" X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="284068026" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jul 2022 11:28:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,267,1650956400"; d="scan'208";a="545962487" Received: from silpixa00401454.ir.intel.com ([10.55.128.122]) by orsmga003.jf.intel.com with ESMTP; 13 Jul 2022 11:28:36 -0700 From: Harry van Haaren To: dev@openvswitch.org Date: Wed, 13 Jul 2022 18:28:07 +0000 Message-Id: <20220713182807.3416578-11-harry.van.haaren@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220713182807.3416578-1-harry.van.haaren@intel.com> References: <20220712174456.2682549-1-harry.van.haaren@intel.com> <20220713182807.3416578-1-harry.van.haaren@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org, kumar.amber@intel.com Subject: [ovs-dev] [PATCH v10 10/10] odp-execute: Add ISA implementation of set_masked IPv4 action X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Emma Finn This commit adds support for the AVX512 implementation of the ipv4_set_addrs action as well as an AVX512 implementation of updating the checksums. Signed-off-by: Emma Finn Tested-by: Kumar Amber Acked-by: Sunil Pai G --- lib/odp-execute-avx512.c | 208 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 208 insertions(+) diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c index 8ecdaecf6..a0c97f312 100644 --- a/lib/odp-execute-avx512.c +++ b/lib/odp-execute-avx512.c @@ -21,6 +21,7 @@ #include #include +#include "csum.h" #include "dp-packet.h" #include "immintrin.h" #include "odp-execute.h" @@ -58,6 +59,22 @@ BUILD_ASSERT_DECL(offsetof(struct ovs_key_ethernet, eth_src) + MEMBER_SIZEOF(struct ovs_key_ethernet, eth_src) == offsetof(struct ovs_key_ethernet, eth_dst)); +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ipv4, ipv4_src) + + MEMBER_SIZEOF(struct ovs_key_ipv4, ipv4_src) == + offsetof(struct ovs_key_ipv4, ipv4_dst)); + +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ipv4, ipv4_dst) + + MEMBER_SIZEOF(struct ovs_key_ipv4, ipv4_dst) == + offsetof(struct ovs_key_ipv4, ipv4_proto)); + +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ipv4, ipv4_proto) + + MEMBER_SIZEOF(struct ovs_key_ipv4, ipv4_proto) == + offsetof(struct ovs_key_ipv4, ipv4_tos)); + +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ipv4, ipv4_tos) + + MEMBER_SIZEOF(struct ovs_key_ipv4, ipv4_tos) == + offsetof(struct ovs_key_ipv4, ipv4_ttl)); + /* Array of callback functions, one for each masked operation. */ odp_execute_action_cb impl_set_masked_funcs[__OVS_KEY_ATTR_MAX]; @@ -279,6 +296,196 @@ action_avx512_eth_set_addrs(struct dp_packet_batch *batch, } } +static inline uint16_t ALWAYS_INLINE +avx512_get_delta(__m256i old_header, __m256i res) +{ + __m256i v_zeros = _mm256_setzero_si256(); + uint16_t delta; + + /* These two shuffle masks, v_swap16a and v_swap16b, are to shuffle the + * old and new header to add padding after each 16-bit value for the + * following carry over addition. */ + __m256i v_swap16a = _mm256_setr_epi16(0x0100, 0xFFFF, 0x0302, 0xFFFF, + 0x0504, 0xFFFF, 0x0706, 0xFFFF, + 0x0100, 0xFFFF, 0x0302, 0xFFFF, + 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF); + __m256i v_swap16b = _mm256_setr_epi16(0x0908, 0xFFFF, 0x0B0A, 0xFFFF, + 0x0D0C, 0xFFFF, 0x0F0E, 0xFFFF, + 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, + 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF); + __m256i v_shuf_old1 = _mm256_shuffle_epi8(old_header, v_swap16a); + __m256i v_shuf_old2 = _mm256_shuffle_epi8(old_header, v_swap16b); + __m256i v_shuf_new1 = _mm256_shuffle_epi8(res, v_swap16a); + __m256i v_shuf_new2 = _mm256_shuffle_epi8(res, v_swap16b); + + /* Add each part of the old and new headers together. */ + __m256i v_delta1 = _mm256_add_epi32(v_shuf_old1, v_shuf_new1); + __m256i v_delta2 = _mm256_add_epi32(v_shuf_old2, v_shuf_new2); + + /* Add old and new header. */ + __m256i v_delta = _mm256_add_epi32(v_delta1, v_delta2); + + /* Perform horizontal add to go from 8x32-bits to 2x32-bits. */ + v_delta = _mm256_hadd_epi32(v_delta, v_zeros); + v_delta = _mm256_hadd_epi32(v_delta, v_zeros); + + /* Shuffle 32-bit value from 3rd lane into first lane for final + * horizontal add. */ + __m256i v_swap32a = _mm256_setr_epi32(0x0, 0x4, 0xF, 0xF, + 0xF, 0xF, 0xF, 0xF); + v_delta = _mm256_permutexvar_epi32(v_swap32a, v_delta); + + v_delta = _mm256_hadd_epi32(v_delta, v_zeros); + v_delta = _mm256_hadd_epi16(v_delta, v_zeros); + + /* Extract delta value. */ + delta = _mm256_extract_epi16(v_delta, 0); + + return delta; +} + +static inline uint16_t ALWAYS_INLINE +avx512_l4_update_csum(__m256i old_header, __m256i res) +{ + __m256i v_zeros = _mm256_setzero_si256(); + uint16_t delta; + + /* Set the v_ones register to all one's. */ + __m256i v_ones = _mm256_cmpeq_epi16(v_zeros, v_zeros); + + /* Combine the old and new header, i.e. adding in the new IP addresses + * in the old header (oh). This is done by using the 0x03C 16-bit mask, + * picking 16-bit word 7 till 10. */ + __m256i v_blend_new = _mm256_mask_blend_epi16(0x03C0, old_header, res); + + /* Invert the old_header register. */ + old_header =_mm256_andnot_si256(old_header, v_ones); + + /* Calculate the delta between the old and new header. */ + delta = avx512_get_delta(old_header, v_blend_new); + + return delta; + +} + +static inline uint16_t ALWAYS_INLINE +avx512_ipv4_update_csum(__m256i res, __m256i old_header) +{ + __m256i v_zeros = _mm256_setzero_si256(); + uint16_t delta; + + /* Set the v_ones register to all one's. */ + __m256i v_ones = _mm256_cmpeq_epi16(v_zeros, v_zeros); + + /* Invert the old_header register. */ + old_header =_mm256_andnot_si256(old_header, v_ones); + + /* Calculate the delta between the old and new header. */ + delta = avx512_get_delta(old_header, res); + + return delta; +} + +/* This function performs the same operation on each packet in the batch as + * the scalar odp_set_ipv4() function. */ +static void +action_avx512_ipv4_set_addrs(struct dp_packet_batch *batch, + const struct nlattr *a) +{ + const struct ovs_key_ipv4 *key, *mask; + struct dp_packet *packet; + a = nl_attr_get(a); + key = nl_attr_get(a); + mask = odp_get_key_mask(a, struct ovs_key_ipv4); + + /* Read the content of the key(src) and mask in the respective registers. + * We only load the size of the actual structure, which is only 96-bits. */ + __m256i v_key = _mm256_maskz_loadu_epi32(0x7, (void *) key); + __m256i v_mask = _mm256_maskz_loadu_epi32(0x7, (void *) mask); + + /* This two shuffle masks, v_shuf32, v_shuffle, are to shuffle key and + * mask to match the ip_header structure layout. */ + static const uint8_t ip_shuffle_mask[32] = { + 0xFF, 0x05, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0x06, 0xFF, 0xFF, 0xFF, 0x00, 0x01, 0x02, 0x03, + 0x00, 0x01, 0x02, 0x03, 0xFF, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; + + __m256i v_shuf32 = _mm256_setr_epi32(0x0, 0x2, 0xF, 0xF, + 0x1, 0xF, 0xF, 0xF); + + __m256i v_shuffle = _mm256_loadu_si256((void *) ip_shuffle_mask); + + /* Two shuffles are required for key and mask to match the layout of + * the ip_header struct. The _shuffle_epi8 only works within 128-bit + * lanes, so a permute is required to move src and dst into the correct + * lanes. And then a shuffle is used to move the fields into the right + * order. + */ + __m256i v_key_shuf = _mm256_permutexvar_epi32(v_shuf32, v_key); + v_key_shuf = _mm256_shuffle_epi8(v_key_shuf, v_shuffle); + + __m256i v_mask_shuf = _mm256_permutexvar_epi32(v_shuf32, v_mask); + v_mask_shuf = _mm256_shuffle_epi8(v_mask_shuf, v_shuffle); + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + struct ip_header *nh = dp_packet_l3(packet); + ovs_be16 old_csum = ~nh->ip_csum; + + /* Load the 20 bytes of the IPv4 header. Without options, which is the + * most common case it's 20 bytes, but can be up to 60 bytes. */ + __m256i v_packet = _mm256_maskz_loadu_epi32(0x1F, (void *) nh); + + /* AND the v_pkt_mask to the packet data (v_packet). */ + __m256i v_pkt_masked = _mm256_andnot_si256(v_mask_shuf, v_packet); + + /* OR the new addresses (v_key_shuf) with the masked packet addresses + * (v_pkt_masked). */ + __m256i v_res = _mm256_or_si256(v_key_shuf, v_pkt_masked); + + /* Update the IP checksum based on updated IP values. */ + uint16_t delta = avx512_ipv4_update_csum(v_res, v_packet); + uint32_t new_csum = old_csum + delta; + delta = csum_finish(new_csum); + + /* Insert new checksum. */ + v_res = _mm256_insert_epi16(v_res, delta, 5); + + /* If ip_src or ip_dst has been modified, L4 checksum needs to + * be updated too. */ + if (mask->ipv4_src || mask->ipv4_dst) { + + uint16_t delta_checksum = avx512_l4_update_csum(v_packet, v_res); + + if (nh->ip_proto == IPPROTO_UDP) { + /* New UDP checksum. */ + struct udp_header *uh = dp_packet_l4(packet); + if (uh->udp_csum) { + uint16_t old_udp_checksum = ~uh->udp_csum; + uint32_t udp_checksum = old_udp_checksum + delta_checksum; + udp_checksum = csum_finish(udp_checksum); + + if (!udp_checksum) { + udp_checksum = htons(0xffff); + } + /* Insert new udp checksum. */ + uh->udp_csum = udp_checksum; + } + } else if (nh->ip_proto == IPPROTO_TCP) { + /* New TCP checksum. */ + struct tcp_header *th = dp_packet_l4(packet); + uint16_t old_tcp_checksum = ~th->tcp_csum; + uint32_t tcp_checksum = old_tcp_checksum + delta_checksum; + tcp_checksum = csum_finish(tcp_checksum); + + th->tcp_csum = tcp_checksum; + } + } + /* Write back the modified IPv4 addresses. */ + _mm256_mask_storeu_epi32((void *) nh, 0x1F, v_res); + } +} + static void action_avx512_set_masked(struct dp_packet_batch *batch, const struct nlattr *a) { @@ -304,6 +511,7 @@ action_avx512_init(struct odp_execute_action_impl *self OVS_UNUSED) /* Set function pointers for the individual operations supported by the * SET_MASKED action. */ impl_set_masked_funcs[OVS_KEY_ATTR_ETHERNET] = action_avx512_eth_set_addrs; + impl_set_masked_funcs[OVS_KEY_ATTR_IPV4] = action_avx512_ipv4_set_addrs; return 0; }