From patchwork Mon Jan 30 16:42:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Horman X-Patchwork-Id: 1734285 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=corigine.onmicrosoft.com header.i=@corigine.onmicrosoft.com header.a=rsa-sha256 header.s=selector2-corigine-onmicrosoft-com header.b=uaAoCOc8; dkim-atps=neutral Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4P5DX36tZmz1yhq for ; Tue, 31 Jan 2023 03:43:31 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id CBB8541856; Mon, 30 Jan 2023 16:43:29 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org CBB8541856 Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=corigine.onmicrosoft.com header.i=@corigine.onmicrosoft.com header.a=rsa-sha256 header.s=selector2-corigine-onmicrosoft-com header.b=uaAoCOc8 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hIdFkPdJn9kJ; Mon, 30 Jan 2023 16:43:27 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id E39E741848; Mon, 30 Jan 2023 16:43:25 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org E39E741848 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D64AAC007F; Mon, 30 Jan 2023 16:43:24 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 49447C0071 for ; Mon, 30 Jan 2023 16:43:22 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 0DAF641831 for ; Mon, 30 Jan 2023 16:43:22 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 0DAF641831 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HFUFL1q_q69h for ; Mon, 30 Jan 2023 16:43:20 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 6BCE341817 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2124.outbound.protection.outlook.com [40.107.92.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 6BCE341817 for ; Mon, 30 Jan 2023 16:43:20 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TdA7mpRo0pjeasQP9b14zfeCnUcUiHylnepd5Q0hqN8cD5iEP6QN/i7600/eYGJOETCBeA9LVn9ynd7QobYQrtr1XLokqP6TAWTqbXi996Fyyw0lOZ8TQ9ZoDPvL6c5wVFFE6RONd58yp4pXwrjgLiiI89eY5C1OuVsc2fhwFPY9AkIfZwgoK6w3xrUxNXazYbaDjfmvpN0erUWXfd7xPYtC1AZRApoOE1qrSzNWy5h0ba/48q0sbgikgr3gz+wz+PwGivyIe2ul7Ns5iWhG6gE3M0gBoitBcvtWVRqL4sptoTj1GzwWzv9czm+hlPTIOOS9tEKc28hA2Um3hKWMdA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QsoZYpP1p+Dvaa2NOe/65jscmCyV5S7EJWy2ifdw/mI=; b=KVf/FpOdm07+WOl1akWUTcBzPz+90nBI5R0oonB7jfQEGs+JWVGWOy071bN6Eo7QN5JpAsHqV55DIuCj5boUwWW3IfKMjUgmFCPkmPRJL/FvJbP7FW5XA7YgljBOzjESs/Rxvc6NjTD5rxagUfXSiBYlCTEZVFktquoX+lsgJ5cP85DFsUIgsiaICAhwCDdZfCNUm6ZmQoIrDWifhOfiX+HYk5V99voR6Tkf46l5rLbIE1cXN4E7PLMr8QHPYAHoXIpOtRoGp1BdLj2CRA1W1QTYpNVUwbM1UteVktP/Z5yIDJem/zoaA18+CjV09U6cTytI46iwlaWDR9AewWYVgw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QsoZYpP1p+Dvaa2NOe/65jscmCyV5S7EJWy2ifdw/mI=; b=uaAoCOc8o3z4aGC0iFYjjeeTxZZQoNS0b4lHdU/FHXeU8FkqeS5AwjPeh5GVUH3YPXFtaCpJOHdoYVOWBFu0Ls2tvsBkj1IeRfcEuIjuFdKfOg8EFDYAcuJIEv+BcL0/fEMavGEJ2rmFAw8SXMxF4puGnsltnfh3TH/hV0GbdAE= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) by DM6PR13MB3866.namprd13.prod.outlook.com (2603:10b6:5:241::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6043.36; Mon, 30 Jan 2023 16:43:17 +0000 Received: from PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::eb5c:910f:3730:fd65]) by PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::eb5c:910f:3730:fd65%6]) with mapi id 15.20.6043.036; Mon, 30 Jan 2023 16:43:17 +0000 From: Simon Horman To: dev@openvswitch.org Date: Mon, 30 Jan 2023 17:42:56 +0100 Message-Id: <20230130164256.764534-4-simon.horman@corigine.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230130164256.764534-1-simon.horman@corigine.com> References: <20230130164256.764534-1-simon.horman@corigine.com> X-ClientProxiedBy: AM4PR05CA0014.eurprd05.prod.outlook.com (2603:10a6:205::27) To PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PH0PR13MB4842:EE_|DM6PR13MB3866:EE_ X-MS-Office365-Filtering-Correlation-Id: 49a13915-6608-4f27-ebc1-08db02e11682 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: PnUNGPWm9J8Qkch3n8bFOXff5f2IA8JJmufL1YQN+qceJXdtuJKasGy3K/VyslsdWW/tZRvqQmJtv6w/ybZPtyCn3jrDeDNFpHapGxr9nlU0B3ya5pfX16L1pBrXxkGBw4tEfbmM61Y1Rik2K57eY86zRQHjeBZZwUbKRpgLWvjxKfXHG2SSuLwXJyErYQ/atfL/NeJd55fUETGOJXEXp8+065zNY+CEebbpgPnD8XGhAZhgqWEpMryciOCepSVvbRpu4r3DyTeDL7jw4EpIqz7XDDhmCGaOWkIx0lt1UMEpyZ7IPFbqQ3KMxVgj4nt9IVfCaIP81q8he1C17kbXTlLbav127q/FBh/NNlNG/5GLo58S5haV7hT8ZfUxl3BjSWRItWG32AtBkUFTv3JsZskZWoI/03qFtKC15FYpEpLMs1Z5AGDyrIfE6srb7JscsxwcMBbxS2gpe82FOc2NDN/BCPaV07q9i+KZd8a5MS2nJqi7UUHR/byxhqzapI9WZU+mDVVsScoOMojKkvNQLsj84w/i40/dqbm1JXMirDVmy3H5MuLRanAGluRQW3XePCidP5cOeoD1elSHzu3BbiEFTMD8PSg0Gl3cQh98J/uJs2I0EDVhv+3i254WvubfvDCSjDgHuQ9beBqv9afr/dbZgwJkn9I+6ACnHKcJorjheoRXAT5sfpWUrN5UhV1T X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR13MB4842.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230025)(4636009)(366004)(346002)(136003)(376002)(396003)(39830400003)(451199018)(186003)(478600001)(2616005)(6512007)(6486002)(2906002)(4326008)(54906003)(6916009)(107886003)(83380400001)(6666004)(6506007)(1076003)(52116002)(8676002)(66556008)(66476007)(66946007)(316002)(41300700001)(44832011)(5660300002)(8936002)(38100700002)(36756003)(86362001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: bXBVEsG/HTIPdUAqyJP6l+Y2qie5pEkqho7PxOPQlQEaeq9+UvZKmbwEmYQpy2pTMxA9XplGUgZfQElVLEARAdpJ7/tRkJC1Ei9Z+e68cppWcTzkf1j+pHhYq6hFWnfAHq5FWvf6dcdfiEH8tTi9YdpDtAUig1HNS8MlksOh0qb3oFPGXyZZvlPP0DyQOBR4mG14DXsy8xfNGUfqbb3Y4cHtnQoPiwgBQ/19jHGhe23eLdwK0cXQLZH7c/ngwWqqFmHrvYtD4f+UJk5BWiwcRpf4A5C//SQyxWOaugeLnTNz0L4QZX5EZahW1VZs02VqrnFe7kSZTm6HpY0PHitVFy6SsDmU3iFdhrRRT/h5QuS+i3C7oEIzTc5OoeWFzZgmok8LjKU4DEeWItJN7PXnjB3LhGueKecQhGrkVMQmDh+DtRAFjBjsY11qBgGDMCV5mRa+Z1OQv6FhOQPouqU7y9ujkv/Iwx7sLdxp0Xovx49e0yTYr8SQw0WG7p+AGUjlMNlY5b4nHF5iSP9RA/It2HEfdoFsFZVMyfZTPEjRJ4hZC9u36OMco+MwmD8L9KChDHrCXPPu6ynqHedPn3tZFvrDgqY/ExYgXBCuS5nvOQHVs1tjVXX8WO4R01jXUr1DhmgYVC9o+fgMmxNocCP5VyrD8boKrDFua4HXBfU9AfGD6HILQ0gLderlrtrMHeWqrbvQc70t6l6+164vXcP2CqJT+nAgkMGiC6i0xcdBOx+iazA1GzbMb6FRkdVaAETC46cTW3a2qzyn+5rsrKBMr5zHULa0wpaIOluT8M0MZurVhZqALrJZ4GZdvSltDLce2Bu63NtvZv1iuTgS/zmO0X5YYFrbJGxlg2K5kYqyl9ZEno6z5rHPeuDCdppQS0LimQtd6YH6BrF3MlZYXTMl2BIfNFJ6qtA2++6vef5+qB7i/JUteAxMJhpE2rLHukA7ji8kz7gZNsfK4bF3fnIgx1+AyDjuZGvgvj8FmsnM83BdH42pY32YODpCb5xdVCPkiAFQ38NZ1Pel/+Asb8BT2oWG6WzmlbY7TC42cxZitUe6H9WV6o+YIKNG5C5WjW446Fiehyt821eJySTAYrw+AUQnvu7CTpHWGMWEb3j4AEtyVwxURU0Qz8D7chwEHFx3JmPojoFd3Od/QzWlxKw/BlGtkv0yzepPTzSVYof5ppVYDXDmgC/W4GAv1jHcx18Xw/RW/TTGLdNZk/gc43K1q9kh+6EXMc+cTPNOMflvF0mN7VnZf0bg+uLFp5Tl/V9Tu8GpwV/Vae8leSajm8OvEuLvSo8ETffdiIU3L795ELLSW1ikzvy2Sr23eM4xHt/+MWBqyBlBEmgjAwwwYzOj+YSqqfpcG2eSy0zYp6iBBzNO8tVMD1MbpcPSnfES4nQUJm3Zo+Vjqdn8BQdCPz3XhOwpg6oe0d9tzAE9SBGOTMe3PnP631/8hzYzveSl1qPOybDVBc+UdYC6yKM4KnUhIiMhQWzhl8g2PuGtZiS8bwnFyS18QWeyJRJQu5I5bcjveU4LozuUnGSAy5Jm00rBcNQE3aW58EZE89f4DFwULlZ2OSf5e4kiiY1EWkqnf5vZtrxDqi7hndre6AA22xn+BGeQgSkUHMzOX5DfONcz0RrVQmpK8uGn76lmWfsGE3KdIA6Oze53kwHLUk4YNBjmhQ== X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 49a13915-6608-4f27-ebc1-08db02e11682 X-MS-Exchange-CrossTenant-AuthSource: PH0PR13MB4842.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2023 16:43:17.4132 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: xqYl8sJMwKRElYzGSZUIAw1e8nweX7HOp46MhUcx+szta0hO3e2i7Y1SGhZyp23WNXaHXCnVPcNfVVUBH7T4e5+bYACSVqbUxFBdKWXgvJg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR13MB3866 Cc: oss-drivers@corigine.com, Jianbo Liu Subject: [ovs-dev] [PATCH v3 3/3] dpif-netlink: add revalidator for offload of meters X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Tianyu Yuan Allow revalidator to continuously delete police in kernel tc datapath until it is deleted. In current implementation, polices in tc datapath will not deleted when they are being used and these remaining polices will be cleared when the openvswitch restarts. This patch supports revalidator to delete remaining polices in tc datapath by storing meter_id deleted unsuccessfully in a hmap and continuously deleting them in revalidator until success. Signed-off-by: Tianyu Yuan Signed-off-by: Simon Horman --- lib/dpif-netdev.c | 1 + lib/dpif-netlink.c | 26 +++++++++++++++++++++++++- lib/dpif-provider.h | 4 ++++ lib/dpif.c | 7 +++++++ lib/dpif.h | 2 ++ lib/netdev-offload-tc.c | 5 ++++- lib/netdev-offload.c | 8 +++++--- ofproto/ofproto-dpif-upcall.c | 6 ++++++ ofproto/ofproto-dpif.c | 10 +++++++++- ofproto/ofproto-dpif.h | 3 +++ 10 files changed, 66 insertions(+), 6 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index c9f7179c3b4c..878fe5933d1a 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -9689,6 +9689,7 @@ const struct dpif_class dpif_netdev_class = { dpif_netdev_meter_set, dpif_netdev_meter_get, dpif_netdev_meter_del, + NULL, dpif_netdev_bond_add, dpif_netdev_bond_del, dpif_netdev_bond_stats_get, diff --git a/lib/dpif-netlink.c b/lib/dpif-netlink.c index 026b0daa8d83..8b95912aea97 100644 --- a/lib/dpif-netlink.c +++ b/lib/dpif-netlink.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include #include @@ -37,6 +38,7 @@ #include "dpif-provider.h" #include "fat-rwlock.h" #include "flow.h" +#include "id-pool.h" #include "netdev-linux.h" #include "netdev-offload.h" #include "netdev-provider.h" @@ -61,6 +63,7 @@ #include "packets.h" #include "random.h" #include "sset.h" +#include "tc.h" #include "timeval.h" #include "unaligned.h" #include "util.h" @@ -4363,12 +4366,32 @@ dpif_netlink_meter_del(struct dpif *dpif, ofproto_meter_id meter_id, err = dpif_netlink_meter_get_stats(dpif, meter_id, stats, max_bands, OVS_METER_CMD_DEL); if (!err && netdev_is_flow_api_enabled()) { - meter_offload_del(meter_id, stats); + return meter_offload_del(meter_id, stats); } return err; } +static void +dpif_netlink_meter_revalidate(struct dpif *dpif_ OVS_UNUSED, + struct hmap *meter_map) +{ + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); + ofproto_meter_id meter_id; + struct id_node *id_node; + + HMAP_FOR_EACH_POP (id_node, node, meter_map) { + meter_id.uint32 = id_node->id; + + if (meter_offload_del(meter_id, NULL) == EPERM) { + id_hmap_add(meter_map, meter_id.uint32); + } else { + VLOG_DBG_RL(&rl, "Delete meter %u in datapath success", + meter_id.uint32); + } + } +} + static bool probe_broken_meters__(struct dpif *dpif) { @@ -4588,6 +4611,7 @@ const struct dpif_class dpif_netlink_class = { dpif_netlink_meter_set, dpif_netlink_meter_get, dpif_netlink_meter_del, + dpif_netlink_meter_revalidate, NULL, /* bond_add */ NULL, /* bond_del */ NULL, /* bond_stats_get */ diff --git a/lib/dpif-provider.h b/lib/dpif-provider.h index 12477a24feee..794fbc21686a 100644 --- a/lib/dpif-provider.h +++ b/lib/dpif-provider.h @@ -631,6 +631,10 @@ struct dpif_class { int (*meter_del)(struct dpif *, ofproto_meter_id meter_id, struct ofputil_meter_stats *, uint16_t n_bands); + /* Checks unneeded meters from 'dpif' and removes them. They may + * be caused by deleting in-use meters. */ + void (*meter_revalidate)(struct dpif *, struct hmap *meter_map); + /* Adds a bond with 'bond_id' and the member-map to 'dpif'. */ int (*bond_add)(struct dpif *dpif, uint32_t bond_id, odp_port_t *member_map); diff --git a/lib/dpif.c b/lib/dpif.c index fe4db83fbfee..c8718239117d 100644 --- a/lib/dpif.c +++ b/lib/dpif.c @@ -2028,6 +2028,13 @@ dpif_meter_del(struct dpif *dpif, ofproto_meter_id meter_id, return error; } +void +dpif_meter_revalidate(struct dpif *dpif, struct hmap *meter_map){ + if (dpif->dpif_class->meter_revalidate) { + dpif->dpif_class->meter_revalidate(dpif, meter_map); + } +} + int dpif_bond_add(struct dpif *dpif, uint32_t bond_id, odp_port_t *member_map) { diff --git a/lib/dpif.h b/lib/dpif.h index 6cb4dae6d8d7..e181e4ee1d6c 100644 --- a/lib/dpif.h +++ b/lib/dpif.h @@ -378,6 +378,7 @@ #include "dpdk.h" #include "dp-packet.h" +#include "id-pool.h" #include "netdev.h" #include "openflow/openflow.h" #include "openvswitch/ofp-meter.h" @@ -905,6 +906,7 @@ int dpif_meter_get(const struct dpif *, ofproto_meter_id meter_id, struct ofputil_meter_stats *, uint16_t n_bands); int dpif_meter_del(struct dpif *, ofproto_meter_id meter_id, struct ofputil_meter_stats *, uint16_t n_bands); +void dpif_meter_revalidate(struct dpif *dpif, struct hmap *meter_map); /* Bonding. */ diff --git a/lib/netdev-offload-tc.c b/lib/netdev-offload-tc.c index 15d1c36aa04e..f87a5e05e202 100644 --- a/lib/netdev-offload-tc.c +++ b/lib/netdev-offload-tc.c @@ -2980,8 +2980,11 @@ meter_tc_del_policer(ofproto_meter_id meter_id, police_index, meter_id.uint32, ovs_strerror(err)); } else { meter_free_police_index(police_index); + /* Do not remove the mapping between meter_id and police_index + * until this meter is deleted successfully in datapath. This + * mapping will be used in meter deletion in revalidator. */ + meter_id_remove(meter_id.uint32); } - meter_id_remove(meter_id.uint32); } return err; diff --git a/lib/netdev-offload.c b/lib/netdev-offload.c index 4592262bd34e..0769f796e5b7 100644 --- a/lib/netdev-offload.c +++ b/lib/netdev-offload.c @@ -241,19 +241,21 @@ int meter_offload_del(ofproto_meter_id meter_id, struct ofputil_meter_stats *stats) { struct netdev_registered_flow_api *rfa; + int ret = 0; CMAP_FOR_EACH (rfa, cmap_node, &netdev_flow_apis) { if (rfa->flow_api->meter_del) { - int ret = rfa->flow_api->meter_del(meter_id, stats); - if (ret) { + int err = rfa->flow_api->meter_del(meter_id, stats); + if (err) { VLOG_DBG_RL(&rl, "Failed deleting meter %u for flow api %s, " "error %d", meter_id.uint32, rfa->flow_api->type, ret); + ret = err; } } } - return 0; + return ret; } int diff --git a/ofproto/ofproto-dpif-upcall.c b/ofproto/ofproto-dpif-upcall.c index 31ac02d116fc..b802c9c7f13a 100644 --- a/ofproto/ofproto-dpif-upcall.c +++ b/ofproto/ofproto-dpif-upcall.c @@ -2828,6 +2828,12 @@ revalidate(struct revalidator *revalidator) } ovsrcu_quiesce(); } + /* When fail_meter_hmap is not empty, revalidate to delete meters in this + * hashmap. */ + if (hmap_count(&udpif->backer->fail_meter_hmap)) { + dpif_meter_revalidate(udpif->dpif, &udpif->backer->fail_meter_hmap); + } + dpif_flow_dump_thread_destroy(dump_thread); ofpbuf_uninit(&odp_actions); } diff --git a/ofproto/ofproto-dpif.c b/ofproto/ofproto-dpif.c index f87e27a8cd7f..591af5503765 100644 --- a/ofproto/ofproto-dpif.c +++ b/ofproto/ofproto-dpif.c @@ -835,6 +835,7 @@ open_dpif_backer(const char *type, struct dpif_backer **backerp) dpif_meter_get_features(backer->dpif, &features); if (features.max_meters) { backer->meter_ids = id_pool_create(0, features.max_meters); + hmap_init(&backer->fail_meter_hmap); } else { backer->meter_ids = NULL; } @@ -6790,8 +6791,15 @@ static void free_meter_id(struct free_meter_id_args *args) { struct ofproto_dpif *ofproto = args->ofproto; + int err; + + err = dpif_meter_del(ofproto->backer->dpif, args->meter_id, NULL, 0); + /* Add fail deleted meter to fail_meter_hmap and waiting to be deleted in + * revalidator. */ + if (err == EPERM) { + id_hmap_add(&ofproto->backer->fail_meter_hmap, args->meter_id.uint32); + } - dpif_meter_del(ofproto->backer->dpif, args->meter_id, NULL, 0); id_pool_free_id(ofproto->backer->meter_ids, args->meter_id.uint32); free(args); } diff --git a/ofproto/ofproto-dpif.h b/ofproto/ofproto-dpif.h index d8e0cd37ac5b..befc68d13baa 100644 --- a/ofproto/ofproto-dpif.h +++ b/ofproto/ofproto-dpif.h @@ -61,6 +61,7 @@ struct ofproto_dpif; struct uuid; struct xlate_cache; struct xlate_ctx; +struct id_pool; /* Number of implemented OpenFlow tables. */ enum { N_TABLES = 255 }; @@ -260,6 +261,8 @@ struct dpif_backer { /* Meter. */ struct id_pool *meter_ids; /* Datapath meter allocation. */ + struct hmap fail_meter_hmap; /* Hashmap for meter with failed deletion + * in datapath */ /* Connection tracking. */ struct id_pool *tp_ids; /* Datapath timeout policy id