From patchwork Wed Mar 13 09:03:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1911588 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=djX0zsEY; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Tvl1J42khz1yWy for ; Wed, 13 Mar 2024 20:03:48 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 9929D40721; Wed, 13 Mar 2024 09:03:45 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BmYnJkVFWt9f; Wed, 13 Mar 2024 09:03:44 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org EE1FC405AB Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=djX0zsEY Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id EE1FC405AB; Wed, 13 Mar 2024 09:03:43 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id CA2F1C0072; Wed, 13 Mar 2024 09:03:43 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 031EBC0037 for ; Wed, 13 Mar 2024 09:03:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id DCE66405C4 for ; Wed, 13 Mar 2024 09:03:42 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ifwsys3uz18e for ; Wed, 13 Mar 2024 09:03:41 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp2.osuosl.org 60146404B4 Authentication-Results: smtp2.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 60146404B4 Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=djX0zsEY Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 60146404B4 for ; Wed, 13 Mar 2024 09:03:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710320620; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VoIfRBfg3stZ2+WoHH7dADfh/BUjacdUz+C91XFlG2w=; b=djX0zsEYktqYvpEk5C4XltKFKuy4EGwcMnez0biPLIJD03h/p2ptC/BboOPZPDixbFooZW KT6XRBmwEPz3MQlRXjmtQtbWof67xhaI+4snYbc6/BtytG+aMcr2pyM8HwiyUSW+XkB6Mf 4IejXcYfzFaDfuBJOGSGwBj1IoY67Lc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-592--5k2DjmnPP2nKEyh-MwlLQ-1; Wed, 13 Mar 2024 05:03:38 -0400 X-MC-Unique: -5k2DjmnPP2nKEyh-MwlLQ-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B0C3F84B165 for ; Wed, 13 Mar 2024 09:03:38 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.194.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id ED3C2492BC7; Wed, 13 Mar 2024 09:03:37 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Wed, 13 Mar 2024 10:03:21 +0100 Message-ID: <20240313090334.414226-2-amorenoz@redhat.com> In-Reply-To: <20240313090334.414226-1-amorenoz@redhat.com> References: <20240313090334.414226-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v2 01/12] python: ovs: Add flowviz scheleton. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add a new python package (just the scheleton for now) to hold a flow visualization tool based on the flow parsing library. flowviz dependencies are installed via "extras_require", so a user must run: $ pip install .[flowviz] or $ pip install ovs[flowviz] Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 14 ++++++++--- python/ovs/flowviz/__init__.py | 0 python/ovs/flowviz/main.py | 40 ++++++++++++++++++++++++++++++ python/ovs/flowviz/odp/__init__.py | 0 python/ovs/flowviz/ofp/__init__.py | 0 python/ovs/flowviz/ovs-flowviz | 20 +++++++++++++++ python/setup.py | 11 +++++--- 7 files changed, 79 insertions(+), 6 deletions(-) create mode 100644 python/ovs/flowviz/__init__.py create mode 100644 python/ovs/flowviz/main.py create mode 100644 python/ovs/flowviz/odp/__init__.py create mode 100644 python/ovs/flowviz/ofp/__init__.py create mode 100755 python/ovs/flowviz/ovs-flowviz diff --git a/python/automake.mk b/python/automake.mk index 84cf2eab5..124032c92 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -63,6 +63,14 @@ ovs_pytests = \ python/ovs/tests/test_odp.py \ python/ovs/tests/test_ofp.py +ovs_flowviz = \ + python/ovs/flowviz/__init__.py \ + python/ovs/flowviz/main.py \ + python/ovs/flowviz/odp/__init__.py \ + python/ovs/flowviz/ofp/__init__.py \ + python/ovs/flowviz/ovs-flowviz + + # These python files are used at build time but not runtime, # so they are not installed. EXTRA_DIST += \ @@ -81,10 +89,10 @@ EXTRA_DIST += \ # C extension support. EXTRA_DIST += python/ovs/_json.c -PYFILES = $(ovs_pyfiles) python/ovs/dirs.py $(ovstest_pyfiles) $(ovs_pytests) +PYFILES = $(ovs_pyfiles) python/ovs/dirs.py $(ovstest_pyfiles) $(ovs_pytests) $(ovs_flowviz) EXTRA_DIST += $(PYFILES) -PYCOV_CLEAN_FILES += $(PYFILES:.py=.py,cover) +PYCOV_CLEAN_FILES += $($(filter %.py, PYFILES):.py=.py,cover) python/ovs/flowviz/ovs-flowviz,cover FLAKE8_PYFILES += \ $(filter-out python/ovs/compat/% python/ovs/dirs.py,$(PYFILES)) \ @@ -95,7 +103,7 @@ FLAKE8_PYFILES += \ python/ovs/dirs.py.template \ python/setup.py -nobase_pkgdata_DATA = $(ovs_pyfiles) $(ovstest_pyfiles) +nobase_pkgdata_DATA = $(ovs_pyfiles) $(ovstest_pyfiles) $(ovs_flowviz) ovs-install-data-local: $(MKDIR_P) python/ovs sed \ diff --git a/python/ovs/flowviz/__init__.py b/python/ovs/flowviz/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/python/ovs/flowviz/main.py b/python/ovs/flowviz/main.py new file mode 100644 index 000000000..f5bf142be --- /dev/null +++ b/python/ovs/flowviz/main.py @@ -0,0 +1,40 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import click + + +class Options(dict): + """Options dictionary""" + + +@click.group( + context_settings=dict(help_option_names=["-h", "--help"]), +) +@click.pass_context +def maincli(ctx): + """ + OpenvSwitch flow visualization utility. + + It reads openflow and datapath flows + (such as the output of ovs-ofctl dump-flows or ovs-appctl dpctl/dump-flows) + and prints them in different formats. + """ + + +def main(): + """ + Main Function + """ + maincli() diff --git a/python/ovs/flowviz/odp/__init__.py b/python/ovs/flowviz/odp/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/python/ovs/flowviz/ofp/__init__.py b/python/ovs/flowviz/ofp/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/python/ovs/flowviz/ovs-flowviz b/python/ovs/flowviz/ovs-flowviz new file mode 100755 index 000000000..9d0959812 --- /dev/null +++ b/python/ovs/flowviz/ovs-flowviz @@ -0,0 +1,20 @@ +#!/usr/bin/env python3 +# +# Copyright (c) 2022,2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from ovs.flowviz import main + +if __name__ == '__main__': + main.main() diff --git a/python/setup.py b/python/setup.py index bcf832ce9..4b9c751d2 100644 --- a/python/setup.py +++ b/python/setup.py @@ -80,6 +80,7 @@ else: extra_cflags = os.environ.get('extra_cflags', '').split() extra_libs = os.environ.get('extra_libs', '').split() +flow_extras_require = ['netaddr', 'pyparsing'] setup_args = dict( name='ovs', @@ -89,7 +90,8 @@ setup_args = dict( author='Open vSwitch', author_email='dev@openvswitch.org', packages=['ovs', 'ovs.compat', 'ovs.compat.sortedcontainers', - 'ovs.db', 'ovs.unixctl', 'ovs.flow'], + 'ovs.db', 'ovs.flow', 'ovs.flowviz', 'ovs.flowviz.odp', + 'ovs.flowviz.ofp', 'ovs.unixctl'], keywords=['openvswitch', 'ovs', 'OVSDB'], license='Apache 2.0', classifiers=[ @@ -109,8 +111,11 @@ setup_args = dict( cmdclass={'build_ext': try_build_ext}, install_requires=['sortedcontainers'], extras_require={':sys_platform == "win32"': ['pywin32 >= 1.0'], - 'flow': ['netaddr', 'pyparsing'], - 'dns': ['unbound']}, + 'dns': ['unbound'], + 'flow': flow_extras_require, + 'flowviz': [*flow_extras_require, 'click'], + }, + scripts=["ovs/flowviz/ovs-flowviz"], ) try: From patchwork Wed Mar 13 09:03:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1911590 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=H5IoUzws; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Tvl1N38lXz1yWy for ; Wed, 13 Mar 2024 20:03:52 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 191F660776; Wed, 13 Mar 2024 09:03:49 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id St-orXx7YKxK; Wed, 13 Mar 2024 09:03:47 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 6031260782 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=H5IoUzws Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 6031260782; Wed, 13 Mar 2024 09:03:47 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D926EC0DD0; Wed, 13 Mar 2024 09:03:46 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 07FD3C0072 for ; Wed, 13 Mar 2024 09:03:45 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id D8D5360716 for ; Wed, 13 Mar 2024 09:03:44 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id U4iIUl_ZZhcG for ; Wed, 13 Mar 2024 09:03:43 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org D387D60602 Authentication-Results: smtp3.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org D387D60602 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id D387D60602 for ; Wed, 13 Mar 2024 09:03:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710320621; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sfkJR2/4YLDculi6AT+6joL2n+hBMGqhodK2zykrM3I=; b=H5IoUzws9Ip041kyIlO/xQ1g4OMW/M21wellMXipNRMVUIz+Boz0evFCBJcT9ao5ZyOuT6 9kBGeigt0Qf9e1MnpQ/zrrAyGnNlYBwp2LWWXM7bic26e8xAmdJx0HMV2U6I2n2Ib+u96O cJIbi+Yh54e5G+nJz9ENBEw7N1obckM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-288--rh5Try0OAOgQQUDni660w-1; Wed, 13 Mar 2024 05:03:40 -0400 X-MC-Unique: -rh5Try0OAOgQQUDni660w-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E4C58800267 for ; Wed, 13 Mar 2024 09:03:39 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.194.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id 17A05492BC6; Wed, 13 Mar 2024 09:03:38 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Wed, 13 Mar 2024 10:03:22 +0100 Message-ID: <20240313090334.414226-3-amorenoz@redhat.com> In-Reply-To: <20240313090334.414226-1-amorenoz@redhat.com> References: <20240313090334.414226-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v2 02/12] python: ovs: flowviz: Add file processing infra. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" process.py contains a useful base class that processes files odp.py and ofp.py: contain datapath and openflow subcommand definitions as well as the first formatting option: json. Also, this patch adds basic filtering support. Examples: $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow json $ ovs-ofctl dump-flows br-int > flows.txt && ovs-flowviz -i flows.txt openflow json $ ovs-ofctl appctl dpctl/dump-flows | ovs-flowviz -f 'ct' datapath json $ ovs-ofctl appctl dpctl/dump-flows > flows.txt && ovs-flowviz -i flows.txt -f 'drop' datapath json Signed-off-by: Adrian Moreno Acked-by: Eelco Chaudron --- python/automake.mk | 5 +- python/ovs/flowviz/__init__.py | 2 + python/ovs/flowviz/main.py | 102 +++++++++++++++++- python/ovs/flowviz/odp/cli.py | 42 ++++++++ python/ovs/flowviz/ofp/cli.py | 42 ++++++++ python/ovs/flowviz/process.py | 192 +++++++++++++++++++++++++++++++++ 6 files changed, 383 insertions(+), 2 deletions(-) create mode 100644 python/ovs/flowviz/odp/cli.py create mode 100644 python/ovs/flowviz/ofp/cli.py create mode 100644 python/ovs/flowviz/process.py diff --git a/python/automake.mk b/python/automake.mk index 124032c92..fd5e74081 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -67,8 +67,11 @@ ovs_flowviz = \ python/ovs/flowviz/__init__.py \ python/ovs/flowviz/main.py \ python/ovs/flowviz/odp/__init__.py \ + python/ovs/flowviz/odp/cli.py \ python/ovs/flowviz/ofp/__init__.py \ - python/ovs/flowviz/ovs-flowviz + python/ovs/flowviz/ofp/cli.py \ + python/ovs/flowviz/ovs-flowviz \ + python/ovs/flowviz/process.py # These python files are used at build time but not runtime, diff --git a/python/ovs/flowviz/__init__.py b/python/ovs/flowviz/__init__.py index e69de29bb..898dba522 100644 --- a/python/ovs/flowviz/__init__.py +++ b/python/ovs/flowviz/__init__.py @@ -0,0 +1,2 @@ +import ovs.flowviz.ofp.cli # noqa: F401 +import ovs.flowviz.odp.cli # noqa: F401 diff --git a/python/ovs/flowviz/main.py b/python/ovs/flowviz/main.py index f5bf142be..64b0e8a0a 100644 --- a/python/ovs/flowviz/main.py +++ b/python/ovs/flowviz/main.py @@ -13,17 +13,64 @@ # limitations under the License. import click +import os + +from ovs.flow.filter import OFFilter class Options(dict): """Options dictionary""" +def validate_input(ctx, param, value): + """Validate the "-i" option""" + result = list() + for input_str in value: + parts = input_str.strip().split(",") + if len(parts) == 2: + file_parts = tuple(parts) + elif len(parts) == 1: + file_parts = tuple(["Filename: " + parts[0], parts[0]]) + else: + raise click.BadParameter( + "input filename should have the following format: " + "[alias,]FILENAME" + ) + + if not os.path.isfile(file_parts[1]): + raise click.BadParameter( + "input filename %s does not exist" % file_parts[1] + ) + result.append(file_parts) + return result + + @click.group( context_settings=dict(help_option_names=["-h", "--help"]), ) +@click.option( + "-i", + "--input", + "filename", + help="Read flows from specified filepath. If not provided, flows will be" + " read from stdin. This option can be specified multiple times." + " Format [alias,]FILENAME. Where alias is a name that shall be used to" + " refer to this FILENAME", + multiple=True, + type=click.Path(), + callback=validate_input, +) +@click.option( + "-f", + "--filter", + help="Filter flows that match the filter expression." + "Run 'ovs-flowviz filter' for a detailed description of the filtering " + "syntax", + type=str, + show_default=False, +) @click.pass_context -def maincli(ctx): +def maincli(ctx, filename, filter): """ OpenvSwitch flow visualization utility. @@ -31,6 +78,59 @@ def maincli(ctx): (such as the output of ovs-ofctl dump-flows or ovs-appctl dpctl/dump-flows) and prints them in different formats. """ + ctx.obj = Options() + ctx.obj["filename"] = filename or None + if filter: + try: + ctx.obj["filter"] = OFFilter(filter) + except Exception as e: + raise click.BadParameter("Wrong filter syntax: {}".format(e)) + + +@maincli.command(hidden=True) +@click.pass_context +def filter(ctx): + """ + \b + Filter Syntax + ************* + + [! | not ] {key}[[.subkey]...] [OPERATOR] {value})] [LOGICAL OPERATOR] ... + + \b + Comparison operators are: + = equality + < less than + > more than + ~= masking (valid for IP and Ethernet fields) + + \b + Logical operators are: + !{expr}: NOT + {expr} && {expr}: AND + {expr} || {expr}: OR + + \b + Matches and flow metadata: + To compare against a match or info field, use the field directly, e.g: + priority=100 + n_bytes>10 + Use simple keywords for flags: + tcp and ip_src=192.168.1.1 + \b + Actions: + Actions values might be dictionaries, use subkeys to access individual + values, e.g: + output.port=3 + Use simple keywords for flags + drop + + \b + Examples of valid filters. + nw_addr~=192.168.1.1 && (tcp_dst=80 || tcp_dst=443) + arp=true && !arp_tsa=192.168.1.1 + n_bytes>0 && drop=true""" + click.echo(ctx.command.get_help(ctx)) def main(): diff --git a/python/ovs/flowviz/odp/cli.py b/python/ovs/flowviz/odp/cli.py new file mode 100644 index 000000000..ed2f82065 --- /dev/null +++ b/python/ovs/flowviz/odp/cli.py @@ -0,0 +1,42 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import click + +from ovs.flowviz.main import maincli +from ovs.flowviz.process import ( + DatapathFactory, + JSONProcessor, +) + + +@maincli.group(subcommand_metavar="FORMAT") +@click.pass_obj +def datapath(opts): + """Process Datapath Flows.""" + pass + + +class JSONPrint(DatapathFactory, JSONProcessor): + def __init__(self, opts): + super().__init__(opts) + + +@datapath.command() +@click.pass_obj +def json(opts): + """Print the flows in JSON format.""" + proc = JSONPrint(opts) + proc.process() + print(proc.json_string()) diff --git a/python/ovs/flowviz/ofp/cli.py b/python/ovs/flowviz/ofp/cli.py new file mode 100644 index 000000000..b9a2a8aad --- /dev/null +++ b/python/ovs/flowviz/ofp/cli.py @@ -0,0 +1,42 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import click + +from ovs.flowviz.main import maincli +from ovs.flowviz.process import ( + OpenFlowFactory, + JSONProcessor, +) + + +@maincli.group(subcommand_metavar="FORMAT") +@click.pass_obj +def openflow(opts): + """Process OpenFlow Flows.""" + pass + + +class JSONPrint(OpenFlowFactory, JSONProcessor): + def __init__(self, opts): + super().__init__(opts) + + +@openflow.command() +@click.pass_obj +def json(opts): + """Print the flows in JSON format.""" + proc = JSONPrint(opts) + proc.process() + print(proc.json_string()) diff --git a/python/ovs/flowviz/process.py b/python/ovs/flowviz/process.py new file mode 100644 index 000000000..3e520e431 --- /dev/null +++ b/python/ovs/flowviz/process.py @@ -0,0 +1,192 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import sys +import json +import click + +from ovs.flow.decoders import FlowEncoder +from ovs.flow.odp import ODPFlow +from ovs.flow.ofp import OFPFlow + + +class FileProcessor(object): + """Base class for file-based Flow processing. It is able to create flows + from strings found in a file (or stdin). + + The process of parsing the flows is extendable in many ways by deriving + this class. + + When process() is called, the base class will: + - call self.start_file() for each new file that get's processed + - call self.create_flow() for each flow line + - apply the filter defined in opts if provided (can be optionally + disabled) + - call self.process_flow() for after the flow has been filtered + - call self.stop_file() after the file has been processed entirely + + In the case of stdin, the filename and file alias is 'stdin'. + + Child classes must at least implement create_flow() and process_flow() + functions. + + Args: + opts (dict): Options dictionary + """ + + def __init__(self, opts): + self.opts = opts + + # Methods that must be implemented by derived classes. + def init(self): + """Called before the flow processing begins.""" + pass + + def start_file(self, alias, filename): + """Called before the processing of a file begins. + Args: + alias(str): The alias name of the filename + filename(str): The filename string + """ + pass + + def create_flow(self, line, idx): + """Called for each line in the file. + Args: + line(str): The flow line + idx(int): The line index + + Returns a Flow. + Must be implemented by child classes. + """ + raise NotImplementedError + + def process_flow(self, flow, name): + """Called for built flow (after filtering). + Args: + flow(Flow): The flow created by create_flow + name(str): The name of the file from which the flow comes + """ + raise NotImplementedError + + def stop_file(self, alias, filename): + """Called after the processing of a file ends. + Args: + alias(str): The alias name of the filename + filename(str): The filename string + """ + pass + + def end(self): + """Called after the processing ends.""" + pass + + def process(self, do_filter=True): + idx = 0 + filenames = self.opts.get("filename") + filt = self.opts.get("filter") if do_filter else None + self.init() + if filenames: + for alias, filename in filenames: + try: + with open(filename) as f: + self.start_file(alias, filename) + for line in f: + flow = self.create_flow(line, idx) + idx += 1 + if not flow or (filt and not filt.evaluate(flow)): + continue + self.process_flow(flow, alias) + self.stop_file(alias, filename) + except IOError as e: + raise click.BadParameter( + "Failed to read from file {} ({}): {}".format( + filename, e.errno, e.strerror + ) + ) + else: + data = sys.stdin.read() + self.start_file("stdin", "stdin") + for line in data.split("\n"): + line = line.strip() + if line: + flow = self.create_flow(line, idx) + idx += 1 + if ( + not flow + or not getattr(flow, "_sections", None) + or (filt and not filt.evaluate(flow)) + ): + continue + self.process_flow(flow, "stdin") + self.stop_file("stdin", "stdin") + self.end() + + +class DatapathFactory(): + """A mixin class that creates Datapath flows.""" + + def create_flow(self, line, idx): + # Skip strings commonly found in Datapath flow dumps. + if any(s in line for s in [ + "flow-dump from the main thread", + "flow-dump from pmd on core", + ]): + return None + + return ODPFlow(line, idx) + + +class OpenFlowFactory(): + """A mixin class that creates OpenFlow flows.""" + + def create_flow(self, line, idx): + # Skip strings commonly found in OpenFlow flow dumps. + if " reply " in line: + return None + + return OFPFlow(line, idx) + + +class JSONProcessor(FileProcessor): + """A FileProcessor that prints flows in JSON format.""" + + def __init__(self, opts): + super().__init__(opts) + self.flows = dict() + + def start_file(self, name, filename): + self.flows_list = list() + + def stop_file(self, name, filename): + self.flows[name] = self.flows_list + + def process_flow(self, flow, name): + self.flows_list.append(flow) + + def json_string(self): + if len(self.flows.keys()) > 1: + return json.dumps( + [ + {"name": name, "flows": [flow.dict() for flow in flows]} + for name, flows in self.flows.items() + ], + indent=4, + cls=FlowEncoder, + ) + return json.dumps( + [flow.dict() for flow in self.flows_list], + indent=4, + cls=FlowEncoder, + ) From patchwork Wed Mar 13 09:03:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1911594 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Dr5lWgR1; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Tvl1c4wHhz23qj for ; Wed, 13 Mar 2024 20:04:04 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 54DC441555; Wed, 13 Mar 2024 09:04:02 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id U3ZoXv3pd0md; Wed, 13 Mar 2024 09:03:51 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 98ABD414BE Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Dr5lWgR1 Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 98ABD414BE; Wed, 13 Mar 2024 09:03:50 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 32ABDC0DDB; Wed, 13 Mar 2024 09:03:50 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 307A9C0DD7 for ; Wed, 13 Mar 2024 09:03:47 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id F042240783 for ; Wed, 13 Mar 2024 09:03:46 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RGSdIdoVu3To for ; Wed, 13 Mar 2024 09:03:45 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp4.osuosl.org 75C1E405EE Authentication-Results: smtp4.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 75C1E405EE Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Dr5lWgR1 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 75C1E405EE for ; Wed, 13 Mar 2024 09:03:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710320623; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bK4qPBcTG04/5LSwFTVPVey4Kze7YPaG6NCVxNLcNRc=; b=Dr5lWgR1W7HNcB2VovVnrxAitpsAQVF9GYQcGDXKZqSrQnpn1Jx4ELeMHm5WV8hUIxVBfW vNZfy+LEzDSzLeunAXMsKxr29n+GA7uFLfB+nfdYiZvNf+BAdwW4MnL4KkFFTHbgCtPwXB JcJC/ORbLsryXsWlYrcxUb+ttr+5uLs= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-655-hz9NNaH2P6iTavOzEDZJvg-1; Wed, 13 Mar 2024 05:03:41 -0400 X-MC-Unique: hz9NNaH2P6iTavOzEDZJvg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0EFBE800267 for ; Wed, 13 Mar 2024 09:03:41 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.194.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2EA53492BC6; Wed, 13 Mar 2024 09:03:40 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Wed, 13 Mar 2024 10:03:23 +0100 Message-ID: <20240313090334.414226-4-amorenoz@redhat.com> In-Reply-To: <20240313090334.414226-1-amorenoz@redhat.com> References: <20240313090334.414226-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v2 03/12] python: ovs: flowviz: Add console formatting. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add a flow formatting framework and one implementation for console printing using rich. The flow formatting framework is a simple set of classes that can be used to write different flow formatting implementations. It supports styles to be described by any class, highlighting and config-file based style definition. The first flow formatting implementation is also introduced: the ConsoleFormatter. It uses the an advanced rich-text printing library [1]. The console printing supports: - Heatmap: printing the packet/byte statistics of each flow in a color that represents its relative size: blue (low) -> red (high). - Printing a banner with the file name and alias. - Extensive style definition via config file. This console format is added to both OpenFlow and Datapath flows. Examples: - Highlight drops in datapath flows: $ ovs-flowviz -i flows.txt --highlight "drop" datapath console - Quickly detect where most packets are going using heatmap and paginated output: $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow console -h [1] https://rich.readthedocs.io/en/stable/introduction.html Signed-off-by: Adrian Moreno --- python/automake.mk | 2 + python/ovs/flowviz/console.py | 175 ++++++++++++++++ python/ovs/flowviz/format.py | 371 ++++++++++++++++++++++++++++++++++ python/ovs/flowviz/main.py | 58 +++++- python/ovs/flowviz/odp/cli.py | 25 +++ python/ovs/flowviz/ofp/cli.py | 26 +++ python/ovs/flowviz/process.py | 83 +++++++- python/setup.py | 4 +- 8 files changed, 736 insertions(+), 8 deletions(-) create mode 100644 python/ovs/flowviz/console.py create mode 100644 python/ovs/flowviz/format.py diff --git a/python/automake.mk b/python/automake.mk index fd5e74081..bd53c5405 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -65,6 +65,8 @@ ovs_pytests = \ ovs_flowviz = \ python/ovs/flowviz/__init__.py \ + python/ovs/flowviz/console.py \ + python/ovs/flowviz/format.py \ python/ovs/flowviz/main.py \ python/ovs/flowviz/odp/__init__.py \ python/ovs/flowviz/odp/cli.py \ diff --git a/python/ovs/flowviz/console.py b/python/ovs/flowviz/console.py new file mode 100644 index 000000000..4a3443360 --- /dev/null +++ b/python/ovs/flowviz/console.py @@ -0,0 +1,175 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import colorsys + +from rich.console import Console +from rich.color import Color +from rich.emoji import Emoji +from rich.panel import Panel +from rich.text import Text +from rich.style import Style + +from ovs.flowviz.format import FlowFormatter, FlowBuffer, FlowStyle + + +def file_header(name): + return Panel( + Text( + Emoji.replace(":scroll:") + + " " + + name + + " " + + Emoji.replace(":scroll:"), + style="bold", + justify="center", + ) + ) + + +class ConsoleBuffer(FlowBuffer): + """ConsoleBuffer implements FlowBuffer to provide console-based text + formatting based on rich.Text. + + Append functions accept a rich.Style. + + Args: + rtext(rich.Text): Optional; text instance to reuse + """ + + def __init__(self, rtext): + self._text = rtext or Text() + + @property + def text(self): + return self._text + + def _append(self, string, style): + """Append to internal text.""" + return self._text.append(string, style) + + def append_key(self, kv, style): + """Append a key. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(kv.meta.kstring, style) + + def append_delim(self, kv, style): + """Append a delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(kv.meta.delim, style) + + def append_end_delim(self, kv, style): + """Append an end delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(kv.meta.end_delim, style) + + def append_value(self, kv, style): + """Append a value. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(kv.meta.vstring, style) + + def append_extra(self, extra, style): + """Append extra string. + Args: + kv (KeyValue): the KeyValue instance to append + style (rich.Style): the style to use + """ + return self._append(extra, style) + + +class ConsoleFormatter(FlowFormatter): + """ConsoleFormatter is a FlowFormatter that formats flows into the console + using rich.Console. + + Args: + console (rich.Console): Optional, an existing console to use + max_value_len (int): Optional; max length of the printed values + kwargs (dict): Optional; Extra arguments to be passed down to + rich.console.Console() + """ + + def __init__(self, opts=None, console=None, **kwargs): + super(ConsoleFormatter, self).__init__() + style = self.style_from_opts(opts) + self.console = console or Console(color_system="256", **kwargs) + self.style = style or FlowStyle() + + def style_from_opts(self, opts): + return self._style_from_opts(opts, "console", Style) + + def print_flow(self, flow, highlighted=None): + """Prints a flow to the console. + + Args: + flow (ovs_dbg.OFPFlow): the flow to print + style (dict): Optional; style dictionary to use + highlighted (list): Optional; list of KeyValues to highlight + """ + + buf = ConsoleBuffer(Text()) + self.format_flow(buf, flow, highlighted) + self.console.print(buf.text) + + def format_flow(self, buf, flow, highlighted=None): + """Formats the flow into the provided buffer as a rich.Text. + + Args: + buf (FlowBuffer): the flow buffer to append to + flow (ovs_dbg.OFPFlow): the flow to format + style (FlowStyle): Optional; style object to use + highlighted (list): Optional; list of KeyValues to highlight + """ + return super(ConsoleFormatter, self).format_flow( + buf, flow, self.style, highlighted + ) + + +def heat_pallete(min_value, max_value): + """Generates a color pallete based on the 5-color heat pallete so that + for each value between min and max a color is returned that represents it's + relative size. + Args: + min_value (int): minimum value + max_value (int) maximum value + """ + h_min = 0 # red + h_max = 220 / 360 # blue + + def heat(value): + if max_value == min_value: + r, g, b = colorsys.hsv_to_rgb(h_max / 2, 1.0, 1.0) + else: + normalized = (int(value) - min_value) / (max_value - min_value) + hue = ((1 - normalized) + h_min) * (h_max - h_min) + r, g, b = colorsys.hsv_to_rgb(hue, 1.0, 1.0) + return Style(color=Color.from_rgb(r * 255, g * 255, b * 255)) + + return heat + + +def default_highlight(): + """Generates a default style for highlights.""" + return Style(underline=True) diff --git a/python/ovs/flowviz/format.py b/python/ovs/flowviz/format.py new file mode 100644 index 000000000..70af2fa26 --- /dev/null +++ b/python/ovs/flowviz/format.py @@ -0,0 +1,371 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Flow formatting framework. + +This file defines a simple flow formatting framework. It's comprised of 3 +classes: FlowStyle, FlowFormatter and FlowBuffer. + +The FlowStyle arranges opaque style objects in a dictionary that can be queried +to determine what style a particular key-value should be formatted with. +That way, a particular implementation can represent its style using their own +object. + +The FlowBuffer is an abstract class and must be derived by particular +implementations. It should know how to append parts of a flow using a style. +Only here the type of the style is relevant. + +When asked to format a flow, the FlowFormatter will determine which style +the flow must be formatted with and call FlowBuffer functions with each part +of the flow and their corresponding style. +""" + + +class FlowStyle: + """A FlowStyle determines the KVStyle to use for each key value in a flow. + + Styles are internally represented by a dictionary. + In order to determine the style for a "key", the following items in the + dictionary are fetched: + - key.highlighted.{key} (if key is found in hightlighted) + - key.highlighted (if key is found in hightlighted) + - key.{key} + - key + - default + + In order to determine the style for a "value", the following items in the + dictionary are fetched: + - value.highlighted.{key} (if key is found in hightlighted) + - value.highlighted.type{value.__class__.__name__} + - value.highlighted + (if key is found in hightlighted) + - value.{key} + - value.type.{value.__class__.__name__} + - value + - default + + The actual type of the style object stored for each item above is opaque + to this class and it depends on the particular FlowFormatter child class + that will handle them. Even callables can be stored, if so they will be + called with the value of the field that is to be formatted and the return + object will be used as style. + + Additionally, the following style items can be defined: + - delim: for delimiters + - delim.highlighted: for delimiters of highlighted key-values + """ + + def __init__(self, initial=None): + self._styles = initial if initial is not None else dict() + + def __len__(self): + return len(self._styles) + + def set_flag_style(self, kvstyle): + self._styles["flag"] = kvstyle + + def set_delim_style(self, kvstyle, highlighted=False): + if highlighted: + self._styles["delim.highlighted"] = kvstyle + else: + self._styles["delim"] = kvstyle + + def set_default_key_style(self, kvstyle, highlighted=False): + if highlighted: + self._styles["key.highlighted"] = kvstyle + else: + self._styles["key"] = kvstyle + + def set_default_value_style(self, kvstyle, highlighted=False): + if highlighted: + self._styles["value.highlighted"] = kvstyle + else: + self._styles["value"] = kvstyle + + def set_key_style(self, key, kvstyle, highlighted=False): + if highlighted: + self._styles["key.highlighted.{}".format(key)] = kvstyle + else: + self._styles["key.{}".format(key)] = kvstyle + + def set_value_style(self, key, kvstyle, highlighted=None): + if highlighted: + self._styles["value.highlighted.{}".format(key)] = kvstyle + else: + self._styles["value.{}".format(key)] = kvstyle + + def set_value_type_style(self, name, kvstyle, highlighted=None): + if highlighted: + self._styles["value.highlighted.type.{}".format(name)] = kvstyle + else: + self._styles["value.type.{}".format(name)] = kvstyle + + def get(self, key): + return self._styles.get(key) + + def get_delim_style(self, highlighted=False): + delim_style_lookup = ["delim.highlighted"] if highlighted else [] + delim_style_lookup.extend(["delim", "default"]) + return next( + ( + self._styles.get(s) + for s in delim_style_lookup + if self._styles.get(s) + ), + None, + ) + + def get_flag_style(self): + return self._styles.get("flag") or self._styles.get("default") + + def get_key_style(self, kv, highlighted=False): + key = kv.key + + key_style_lookup = ( + ["key.highlighted.%s" % key, "key.highlighted"] + if highlighted + else [] + ) + key_style_lookup.extend(["key.%s" % key, "key", "default"]) + + style = next( + ( + self._styles.get(s) + for s in key_style_lookup + if self._styles.get(s) + ), + None, + ) + if callable(style): + return style(kv.meta.kstring) + return style + + def get_value_style(self, kv, highlighted=False): + key = kv.key + value_type = kv.value.__class__.__name__.lower() + value_style_lookup = ( + [ + "value.highlighted.%s" % key, + "value.highlighted.type.%s" % value_type, + "value.highlighted", + ] + if highlighted + else [] + ) + value_style_lookup.extend( + [ + "value.%s" % key, + "value.type.%s" % value_type, + "value", + "default", + ] + ) + + style = next( + ( + self._styles.get(s) + for s in value_style_lookup + if self._styles.get(s) + ), + None, + ) + if callable(style): + return style(kv.meta.vstring) + return style + + +class FlowFormatter: + """FlowFormatter is a base class for Flow Formatters.""" + + def __init__(self): + self._highlighted = list() + + def _style_from_opts(self, opts, opts_key, style_constructor): + """Create style object from options. + + Args: + opts (dict): Options dictionary + opts_key (str): The options style key to extract + (e.g: console or html) + style_constructor(callable): A callable that creates a derived + style object + """ + if not opts or not opts.get("style"): + return None + + section_name = ".".join(["styles", opts.get("style")]) + if section_name not in opts.get("config").sections(): + return None + + config = opts.get("config")[section_name] + style = {} + for key in config: + (_, console, style_full_key) = key.partition(opts_key + ".") + if not console: + continue + + (style_key, _, prop) = style_full_key.rpartition(".") + if not prop or not style_key: + raise Exception("malformed style config: {}".format(key)) + + if not style.get(style_key): + style[style_key] = {} + style[style_key][prop] = config[key] + + return FlowStyle({k: style_constructor(**v) for k, v in style.items()}) + + def format_flow(self, buf, flow, style_obj=None, highlighted=None): + """Formats the flow into the provided buffer. + + Args: + buf (FlowBuffer): the flow buffer to append to + flow (ovs_dbg.OFPFlow): the flow to format + style_obj (FlowStyle): Optional; style to use + highlighted (list): Optional; list of KeyValues to highlight + """ + last_printed_pos = 0 + + if style_obj: + style_obj = style_obj or FlowStyle() + for section in sorted(flow.sections, key=lambda x: x.pos): + buf.append_extra( + flow.orig[last_printed_pos : section.pos], + style=style_obj.get("default"), + ) + self.format_kv_list( + buf, section.data, section.string, style_obj, highlighted + ) + last_printed_pos = section.pos + len(section.string) + else: + # Don't pay the cost of formatting each section one by one. + buf.append_extra(flow.orig.strip(), None) + + def format_kv_list(self, buf, kv_list, full_str, style_obj, highlighted): + """Format a KeyValue List. + + Args: + buf (FlowBuffer): a FlowBuffer to append formatted KeyValues to + kv_list (list[KeyValue]: the KeyValue list to format + full_str (str): the full string containing all k-v + style_obj (FlowStyle): a FlowStyle object to use + highlighted (list): Optional; list of KeyValues to highlight + """ + for i, kv in enumerate(kv_list): + written = self.format_kv( + buf, kv, style_obj=style_obj, highlighted=highlighted + ) + + end = ( + kv_list[i + 1].meta.kpos + if i < (len(kv_list) - 1) + else len(full_str) + ) + + buf.append_extra( + full_str[(kv.meta.kpos + written) : end].rstrip("\n\r"), + style=style_obj.get("default"), + ) + + def format_kv(self, buf, kv, style_obj, highlighted=None): + """Format a KeyValue + + A formatted keyvalue has the following parts: + {key}{delim}{value}[{delim}] + + Args: + buf (FlowBuffer): buffer to append the KeyValue to + kv (KeyValue): The KeyValue to print + style_obj (FlowStyle): The style object to use + highlighted (list): Optional; list of KeyValues to highlight + + Returns the number of printed characters. + """ + ret = 0 + key = kv.meta.kstring + is_highlighted = ( + key in [k.key for k in highlighted] if highlighted else False + ) + + key_style = style_obj.get_key_style(kv, is_highlighted) + buf.append_key(kv, key_style) # format value + ret += len(key) + + if not kv.meta.vstring: + return ret + + if kv.meta.delim not in ("\n", "\t", "\r", ""): + buf.append_delim(kv, style_obj.get_delim_style(is_highlighted)) + ret += len(kv.meta.delim) + + value_style = style_obj.get_value_style(kv, is_highlighted) + buf.append_value(kv, value_style) # format value + ret += len(kv.meta.vstring) + + if kv.meta.end_delim: + buf.append_end_delim(kv, style_obj.get_delim_style(is_highlighted)) + ret += len(kv.meta.end_delim) + + return ret + + +class FlowBuffer: + """A FlowBuffer is a base class for format buffers. + + Childs must implement the following methods: + append_key(self, kv, style) + append_value(self, kv, style) + append_delim(self, delim, style) + append_end_delim(self, delim, style) + append_extra(self, extra, style) + """ + + def append_key(self, kv, style): + """Append a key. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError + + def append_delim(self, kv, style): + """Append a delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError + + def append_end_delim(self, kv, style): + """Append an end delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError + + def append_value(self, kv, style): + """Append a value. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError + + def append_extra(self, extra, style): + """Append extra string. + Args: + kv (KeyValue): the KeyValue instance to append + style (Any): the style to use + """ + raise NotImplementedError diff --git a/python/ovs/flowviz/main.py b/python/ovs/flowviz/main.py index 64b0e8a0a..723c71fa7 100644 --- a/python/ovs/flowviz/main.py +++ b/python/ovs/flowviz/main.py @@ -12,10 +12,30 @@ # See the License for the specific language governing permissions and # limitations under the License. +import configparser import click import os from ovs.flow.filter import OFFilter +from ovs.dirs import PKGDATADIR + +_default_config_file = "ovs-flowviz.conf" +_default_config_path = next( + ( + p + for p in [ + os.path.join( + os.getenv("HOME"), ".config", "ovs", _default_config_file + ), + os.path.join(PKGDATADIR, _default_config_file), + os.path.abspath( + os.path.join(os.path.dirname(__file__), _default_config_file) + ), + ] + if os.path.exists(p) + ), + "", +) class Options(dict): @@ -48,6 +68,20 @@ def validate_input(ctx, param, value): @click.group( context_settings=dict(help_option_names=["-h", "--help"]), ) +@click.option( + "-c", + "--config", + help="Use config file", + type=click.Path(), + default=_default_config_path, + show_default=True, +) +@click.option( + "--style", + help="Select style (defined in config file)", + default=None, + show_default=True, +) @click.option( "-i", "--input", @@ -69,8 +103,17 @@ def validate_input(ctx, param, value): type=str, show_default=False, ) +@click.option( + "-l", + "--highlight", + help="Highlight flows that match the filter expression." + "Run 'ovs-flowviz filter' for a detailed description of the filtering " + "syntax", + type=str, + show_default=False, +) @click.pass_context -def maincli(ctx, filename, filter): +def maincli(ctx, config, style, filename, filter, highlight): """ OpenvSwitch flow visualization utility. @@ -86,6 +129,19 @@ def maincli(ctx, filename, filter): except Exception as e: raise click.BadParameter("Wrong filter syntax: {}".format(e)) + if highlight: + try: + ctx.obj["highlight"] = OFFilter(highlight) + except Exception as e: + raise click.BadParameter("Wrong filter syntax: {}".format(e)) + + config_file = config or _default_config_path + parser = configparser.ConfigParser() + parser.read(config_file) + + ctx.obj["config"] = parser + ctx.obj["style"] = style + @maincli.command(hidden=True) @click.pass_context diff --git a/python/ovs/flowviz/odp/cli.py b/python/ovs/flowviz/odp/cli.py index ed2f82065..a1cba0135 100644 --- a/python/ovs/flowviz/odp/cli.py +++ b/python/ovs/flowviz/odp/cli.py @@ -16,6 +16,7 @@ import click from ovs.flowviz.main import maincli from ovs.flowviz.process import ( + ConsoleProcessor, DatapathFactory, JSONProcessor, ) @@ -40,3 +41,27 @@ def json(opts): proc = JSONPrint(opts) proc.process() print(proc.json_string()) + + +class DPConsoleProcessor(DatapathFactory, ConsoleProcessor): + def __init__(self, opts, heat_map): + super().__init__(opts, heat_map) + + +@datapath.command() +@click.option( + "-h", + "--heat-map", + is_flag=True, + default=False, + show_default=True, + help="Create heat-map with packet and byte counters", +) +@click.pass_obj +def console(opts, heat_map): + """Print the flows in the console with some style.""" + proc = DPConsoleProcessor( + opts, heat_map=["packets", "bytes"] if heat_map else [] + ) + proc.process() + proc.print() diff --git a/python/ovs/flowviz/ofp/cli.py b/python/ovs/flowviz/ofp/cli.py index b9a2a8aad..a399dbd82 100644 --- a/python/ovs/flowviz/ofp/cli.py +++ b/python/ovs/flowviz/ofp/cli.py @@ -16,6 +16,7 @@ import click from ovs.flowviz.main import maincli from ovs.flowviz.process import ( + ConsoleProcessor, OpenFlowFactory, JSONProcessor, ) @@ -40,3 +41,28 @@ def json(opts): proc = JSONPrint(opts) proc.process() print(proc.json_string()) + + +class OFConsoleProcessor(OpenFlowFactory, ConsoleProcessor): + def __init__(self, opts, heat_map): + super().__init__(opts, heat_map) + + +@openflow.command() +@click.option( + "-h", + "--heat-map", + is_flag=True, + default=False, + show_default=True, + help="Create heat-map with packet and byte counters", +) +@click.pass_obj +def console(opts, heat_map): + """Print the flows in the console with some style.""" + proc = OFConsoleProcessor( + opts, + heat_map=["n_packets", "n_bytes"] if heat_map else [], + ) + proc.process() + proc.print() diff --git a/python/ovs/flowviz/process.py b/python/ovs/flowviz/process.py index 3e520e431..349da8017 100644 --- a/python/ovs/flowviz/process.py +++ b/python/ovs/flowviz/process.py @@ -20,6 +20,13 @@ from ovs.flow.decoders import FlowEncoder from ovs.flow.odp import ODPFlow from ovs.flow.ofp import OFPFlow +from ovs.flowviz.console import ( + ConsoleFormatter, + default_highlight, + heat_pallete, + file_header, +) + class FileProcessor(object): """Base class for file-based Flow processing. It is able to create flows @@ -134,21 +141,24 @@ class FileProcessor(object): self.end() -class DatapathFactory(): +class DatapathFactory: """A mixin class that creates Datapath flows.""" def create_flow(self, line, idx): # Skip strings commonly found in Datapath flow dumps. - if any(s in line for s in [ - "flow-dump from the main thread", - "flow-dump from pmd on core", - ]): + if any( + s in line + for s in [ + "flow-dump from the main thread", + "flow-dump from pmd on core", + ] + ): return None return ODPFlow(line, idx) -class OpenFlowFactory(): +class OpenFlowFactory: """A mixin class that creates OpenFlow flows.""" def create_flow(self, line, idx): @@ -190,3 +200,64 @@ class JSONProcessor(FileProcessor): indent=4, cls=FlowEncoder, ) + + +class ConsoleProcessor(FileProcessor): + """A generic Console Processor that prints flows into the console""" + + def __init__(self, opts, heat_map=[]): + super().__init__(opts) + self.heat_map = heat_map + self.console = ConsoleFormatter(opts) + if len(self.console.style) == 0 and self.opts.get("highlight"): + # Add some style to highlights or else they won't be seen. + self.console.style.set_default_value_style( + default_highlight(), True + ) + self.console.style.set_default_key_style(default_highlight(), True) + + self.flows = dict() # Dictionary of flow-lists, one per file. + self.min_max = dict() # Used for heat-map. calculation + + def start_file(self, name, filename): + self.flows_list = list() + if len(self.heat_map) > 0: + self.min = [-1] * len(self.heat_map) + self.max = [0] * len(self.heat_map) + + def stop_file(self, name, filename): + self.flows[name] = self.flows_list + if len(self.heat_map) > 0: + self.min_max[name] = (self.min, self.max) + + def process_flow(self, flow, name): + # Running calculation of min and max values for all the fields that + # take place in the heatmap. + for i, field in enumerate(self.heat_map): + val = flow.info.get(field) + if self.min[i] == -1 or val < self.min[i]: + self.min[i] = val + if val > self.max[i]: + self.max[i] = val + + self.flows_list.append(flow) + + def print(self): + for name, flows in self.flows.items(): + self.console.console.print("\n") + self.console.console.print(file_header(name)) + + if len(self.heat_map) > 0 and len(self.flows) > 0: + for i, field in enumerate(self.heat_map): + (min_val, max_val) = self.min_max[name][i] + self.console.style.set_value_style( + field, heat_pallete(min_val, max_val) + ) + + for flow in flows: + high = None + if self.opts.get("highlight"): + result = self.opts.get("highlight").evaluate(flow) + if result: + high = result.kv + self.console.print_flow(flow, high) diff --git a/python/setup.py b/python/setup.py index 4b9c751d2..76f9fc820 100644 --- a/python/setup.py +++ b/python/setup.py @@ -113,9 +113,11 @@ setup_args = dict( extras_require={':sys_platform == "win32"': ['pywin32 >= 1.0'], 'dns': ['unbound'], 'flow': flow_extras_require, - 'flowviz': [*flow_extras_require, 'click'], + 'flowviz': + [*flow_extras_require, 'click', 'rich'], }, scripts=["ovs/flowviz/ovs-flowviz"], + include_package_data=True, ) try: From patchwork Wed Mar 13 09:03:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1911591 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=cYAD9hXN; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Tvl1Q3dyVz1yWy for ; Wed, 13 Mar 2024 20:03:54 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 2DEA981F56; Wed, 13 Mar 2024 09:03:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HSF4AGGoJCDz; Wed, 13 Mar 2024 09:03:48 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 3E8E881EC1 Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=cYAD9hXN Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 3E8E881EC1; Wed, 13 Mar 2024 09:03:48 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id CEE36C0DD7; Wed, 13 Mar 2024 09:03:47 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 40D88C0072 for ; Wed, 13 Mar 2024 09:03:46 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 3A2B860602 for ; Wed, 13 Mar 2024 09:03:46 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MnfK3nPbAOKe for ; Wed, 13 Mar 2024 09:03:45 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org DECBC60762 Authentication-Results: smtp3.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org DECBC60762 Authentication-Results: smtp3.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=cYAD9hXN Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id DECBC60762 for ; Wed, 13 Mar 2024 09:03:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710320623; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gS29DfrhsaELhFtHr5HifcrlCqyzQSpuibJWO36inJY=; b=cYAD9hXNJ4BecsoeI6zDPjj8X23Tmj8gZtqLajWKLByXA0zMVjLDpgE77YtTeOFjrAs3ih vuoAdgzVIqEiptmODR4Jwv04AO6e3CUWqci6q7bIBA4LUkg2+d1DjRFeTX6b9+frBxzP1x yTMExS0zDjD/3I6KqWGjMEL7YVtJVj8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-624-IKfhw6X4PxCtN5Jn0IZVIg-1; Wed, 13 Mar 2024 05:03:42 -0400 X-MC-Unique: IKfhw6X4PxCtN5Jn0IZVIg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 321FC84B166 for ; Wed, 13 Mar 2024 09:03:42 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.194.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6E4C3492BC6; Wed, 13 Mar 2024 09:03:41 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Wed, 13 Mar 2024 10:03:24 +0100 Message-ID: <20240313090334.414226-5-amorenoz@redhat.com> In-Reply-To: <20240313090334.414226-1-amorenoz@redhat.com> References: <20240313090334.414226-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v2 04/12] python: ovs: flowviz: Add default config file. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" It has two basic styles defined: "dark" and "light" intended for dark and light terminals. Examples: $ ovs-flowviz -i /tmp/dpflows --style=dark datapath console $ ovs-flowviz -i /tmp/ofpflows --style=light openflow console Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 5 +- python/ovs/flowviz/ovs-flowviz.conf | 94 +++++++++++++++++++++++++++++ python/setup.py | 1 + 3 files changed, 99 insertions(+), 1 deletion(-) create mode 100644 python/ovs/flowviz/ovs-flowviz.conf diff --git a/python/automake.mk b/python/automake.mk index bd53c5405..23212e4b5 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -89,7 +89,8 @@ EXTRA_DIST += \ python/ovs/compat/sortedcontainers/LICENSE \ python/README.rst \ python/setup.py \ - python/test_requirements.txt + python/test_requirements.txt \ + python/ovs/flowviz/ovs-flowviz.conf # C extension support. EXTRA_DIST += python/ovs/_json.c @@ -109,6 +110,8 @@ FLAKE8_PYFILES += \ python/setup.py nobase_pkgdata_DATA = $(ovs_pyfiles) $(ovstest_pyfiles) $(ovs_flowviz) +nobase_pkgdata_DATA += python/ovs/flowviz/ovs-flowviz.conf + ovs-install-data-local: $(MKDIR_P) python/ovs sed \ diff --git a/python/ovs/flowviz/ovs-flowviz.conf b/python/ovs/flowviz/ovs-flowviz.conf new file mode 100644 index 000000000..3acd0a29e --- /dev/null +++ b/python/ovs/flowviz/ovs-flowviz.conf @@ -0,0 +1,94 @@ +# Create any number of styles.{style_name} sections with a defined style. +# +# Syntax: +# +# [FORMAT].[PORTION].[SELECTOR].[ELEMENT] = [VALUE] +# +# * FORMAT: console +# * PORTION: The portion of the flow that the style applies to +# - key: Selects how to print the key of a KeyValue pair +# - key: Selects how to print the value of a KeyValue pair +# - flag: Selects how to print the a flag +# - delim: Selects how to print the delimiters around key and values +# +# * SELECTOR: +# - highlighted: to apply when the key is highlighted +# - type.{TYPE}: to apply when the value matches a type +# (special types such as IPAddress or EthMask can be used) +# (only aplicable to 'value') +# - {key_name}: to apply when the key matches the key_name +# +# Console Styles +# ============== +# * ELEMENT: +# - color: defines the color in hex or a color rich starndard ones [1] +# - underline: if set to "true", the selected portion will be underlined +# +#[1] https://rich.readthedocs.io/en/stable/appendix/colors.html#standard-colors + +[styles.dark] + +# defaults for key-values +console.key.color = #5D86BA +console.value.color= #B0C4DE +console.delim.color= #B0C4DE +console.default.color= #FFFFFF + +# defaults for special types +console.value.type.IPAddress.color = #008700 +console.value.type.IPMask.color = #008700 +console.value.type.EthMask.color = #008700 + +# dim some long arguments +console.value.ct.color = grey66 +console.value.ufid.color = grey66 +console.value.clone.color = grey66 +console.value.controller.color = grey66 + +# highlight flags +console.flag.color = #875fff + +# show drop and recirculations +console.key.drop.color = red +console.key.resubmit.color = #00d700 +console.key.output.color = #00d700 +console.value.output.color = #00d700 + +# highlights +console.key.highlighted.color = red +console.key.highlighted.underline = true +console.value.highlighted.underline = true +console.delim.highlighted.underline = true + + +[styles.light] +# If a color is omitted, the default terminal color will be used +# highlight keys +console.key.color = blue + +# special types +console.value.type.IPAddress.color = #008700 +console.value.type.IPMask.color = #008700 +console.value.type.EthMask.color = #008700 + +# dim long arguments +console.value.ct.color = bright_black +console.value.ufid.color = #870000 +console.value.clone.color = bright_black +console.value.controller.color = bright_black + +# highlight flags +console.flag.color = #00005F + +# show drop and recirculations +console.key.drop.color = red +console.key.resubmit.color = #00d700 +console.key.output.color = #005f00 +console.value.output.color = #00d700 + +# highlights +console.key.highlighted.color = #f20905 +console.value.highlighted.color = #f20905 +console.key.highlighted.underline = true +console.value.highlighted.underline = true +console.delim.highlighted.underline = true diff --git a/python/setup.py b/python/setup.py index 76f9fc820..c734f68f3 100644 --- a/python/setup.py +++ b/python/setup.py @@ -117,6 +117,7 @@ setup_args = dict( [*flow_extras_require, 'click', 'rich'], }, scripts=["ovs/flowviz/ovs-flowviz"], + data_files=["ovs/flowviz/ovs-flowviz.conf"], include_package_data=True, ) From patchwork Wed Mar 13 09:03:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1911592 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=YwucBiEh; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Tvl1W1VRQz1yWy for ; Wed, 13 Mar 2024 20:03:59 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 2CD8B81A47; Wed, 13 Mar 2024 09:03:57 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id b9faf7b9ylsp; Wed, 13 Mar 2024 09:03:54 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 3DE0481F74 Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=YwucBiEh Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 3DE0481F74; Wed, 13 Mar 2024 09:03:52 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9CF62C007C; Wed, 13 Mar 2024 09:03:51 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 70BB1C0DD8 for ; Wed, 13 Mar 2024 09:03:50 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 4173840BD7 for ; Wed, 13 Mar 2024 09:03:50 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BQkdQT_PxPFu for ; Wed, 13 Mar 2024 09:03:47 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp2.osuosl.org 5886540AAE Authentication-Results: smtp2.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 5886540AAE Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=YwucBiEh Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 5886540AAE for ; Wed, 13 Mar 2024 09:03:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710320625; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UUeJjWP0t76iiMi/6PhE9393PXFWFtNwfPRuPKjr+Fc=; b=YwucBiEhpmAUe52CixxGMYYUdGBtyBVAJKmFbzlmJ+Ic9GTzT7+PZ1RB1EuuAGWjMQ91eV tMeMhoQ+Ar/ae6KW0m+153kLWzCJiZIzHrCkvYtK2FTq2BBPBLlid6sjHFLpijpbMC1K8X 0M0ozhxGLbZloLzcsJokSj0QngfoRsw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-553-A1Rzy_CsNwqtWqzBTlTKPQ-1; Wed, 13 Mar 2024 05:03:43 -0400 X-MC-Unique: A1Rzy_CsNwqtWqzBTlTKPQ-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7ECD4800269 for ; Wed, 13 Mar 2024 09:03:43 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.194.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id 741F9492BC7; Wed, 13 Mar 2024 09:03:42 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Wed, 13 Mar 2024 10:03:25 +0100 Message-ID: <20240313090334.414226-6-amorenoz@redhat.com> In-Reply-To: <20240313090334.414226-1-amorenoz@redhat.com> References: <20240313090334.414226-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v2 05/12] python: ovs: flowviz: Add html formatting. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add a HTML Formatter and use it to print OpenFlow flows in an HTML list with table links. Examples $ ovs-flowviz -i offlows.txt --highlight "drop" openflow html > /tmp/flows.html $ ovs-flowviz -i offlows.txt --filter "n_packets > 0" openflow html > /tmp/flows.html Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 3 +- python/ovs/flowviz/html_format.py | 136 ++++++++++++++++++++++++++++ python/ovs/flowviz/ofp/cli.py | 10 ++ python/ovs/flowviz/ofp/html.py | 80 ++++++++++++++++ python/ovs/flowviz/ovs-flowviz.conf | 16 +++- 5 files changed, 243 insertions(+), 2 deletions(-) create mode 100644 python/ovs/flowviz/html_format.py create mode 100644 python/ovs/flowviz/ofp/html.py diff --git a/python/automake.mk b/python/automake.mk index 23212e4b5..0487494d0 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -67,15 +67,16 @@ ovs_flowviz = \ python/ovs/flowviz/__init__.py \ python/ovs/flowviz/console.py \ python/ovs/flowviz/format.py \ + python/ovs/flowviz/html_format.py \ python/ovs/flowviz/main.py \ python/ovs/flowviz/odp/__init__.py \ python/ovs/flowviz/odp/cli.py \ python/ovs/flowviz/ofp/__init__.py \ python/ovs/flowviz/ofp/cli.py \ + python/ovs/flowviz/ofp/html.py \ python/ovs/flowviz/ovs-flowviz \ python/ovs/flowviz/process.py - # These python files are used at build time but not runtime, # so they are not installed. EXTRA_DIST += \ diff --git a/python/ovs/flowviz/html_format.py b/python/ovs/flowviz/html_format.py new file mode 100644 index 000000000..ebfa65c34 --- /dev/null +++ b/python/ovs/flowviz/html_format.py @@ -0,0 +1,136 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from ovs.flowviz.format import FlowFormatter, FlowBuffer, FlowStyle + + +class HTMLStyle: + """HTMLStyle defines a style for html-formatted flows. + + Args: + color(str): Optional; a string representing the CSS color to use + anchor_gen(callable): Optional; a callable to be used to generate the + href + """ + + def __init__(self, color=None, anchor_gen=None): + self.color = color + self.anchor_gen = anchor_gen + + +class HTMLBuffer(FlowBuffer): + """HTMLBuffer implementes FlowBuffer to provide html-based flow formatting. + + Each flow gets formatted as: +
...
+ """ + + def __init__(self): + self._text = "" + + @property + def text(self): + return self._text + + def _append(self, string, color, href): + """Append a key a string""" + style = ' style="color:{}"'.format(color) if color else "" + self._text += "".format(style) + if href: + self._text += "".format(href) + self._text += string + if href: + self._text += "" + self._text += "" + + def append_key(self, kv, style): + """Append a key. + Args: + kv (KeyValue): the KeyValue instance to append + style (HTMLStyle): the style to use + """ + href = style.anchor_gen(kv) if (style and style.anchor_gen) else "" + return self._append( + kv.meta.kstring, style.color if style else "", href + ) + + def append_delim(self, kv, style): + """Append a delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (HTMLStyle): the style to use + """ + href = style.anchor_gen(kv) if (style and style.anchor_gen) else "" + return self._append(kv.meta.delim, style.color if style else "", href) + + def append_end_delim(self, kv, style): + """Append an end delimiter. + Args: + kv (KeyValue): the KeyValue instance to append + style (HTMLStyle): the style to use + """ + href = style.anchor_gen(kv) if (style and style.anchor_gen) else "" + return self._append( + kv.meta.end_delim, style.color if style else "", href + ) + + def append_value(self, kv, style): + """Append a value. + Args: + kv (KeyValue): the KeyValue instance to append + style (HTMLStyle): the style to use + """ + href = style.anchor_gen(kv) if (style and style.anchor_gen) else "" + return self._append( + kv.meta.vstring, style.color if style else "", href + ) + + def append_extra(self, extra, style): + """Append extra string. + Args: + kv (KeyValue): the KeyValue instance to append + style (HTMLStyle): the style to use + """ + return self._append(extra, style.color if style else "", "") + + +class HTMLFormatter(FlowFormatter): + """Formts a flow in HTML Format.""" + + default_style_obj = FlowStyle( + { + "value.resubmit": HTMLStyle( + anchor_gen=lambda x: "#table_{}".format(x.value["table"]) + ), + "default": HTMLStyle(), + } + ) + + def __init__(self, opts=None): + super(HTMLFormatter, self).__init__() + self.style = ( + self._style_from_opts(opts, "html", HTMLStyle) or FlowStyle() + ) + + def format_flow(self, buf, flow, highlighted=None): + """Formats the flow into the provided buffer as a html object. + + Args: + buf (FlowBuffer): the flow buffer to append to + flow (ovs_dbg.OFPFlow): the flow to format + highlighted (list): Optional; list of KeyValues to highlight + """ + return super(HTMLFormatter, self).format_flow( + buf, flow, self.style, highlighted + ) diff --git a/python/ovs/flowviz/ofp/cli.py b/python/ovs/flowviz/ofp/cli.py index a399dbd82..2cd8e1c89 100644 --- a/python/ovs/flowviz/ofp/cli.py +++ b/python/ovs/flowviz/ofp/cli.py @@ -15,6 +15,7 @@ import click from ovs.flowviz.main import maincli +from ovs.flowviz.ofp.html import HTMLProcessor from ovs.flowviz.process import ( ConsoleProcessor, OpenFlowFactory, @@ -66,3 +67,12 @@ def console(opts, heat_map): ) proc.process() proc.print() + + +@openflow.command() +@click.pass_obj +def html(opts): + """Print the flows in an linked HTML list arranged by tables.""" + processor = HTMLProcessor(opts) + processor.process() + print(processor.html()) diff --git a/python/ovs/flowviz/ofp/html.py b/python/ovs/flowviz/ofp/html.py new file mode 100644 index 000000000..a66f5fe8e --- /dev/null +++ b/python/ovs/flowviz/ofp/html.py @@ -0,0 +1,80 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from ovs.flowviz.html_format import HTMLBuffer, HTMLFormatter, HTMLStyle +from ovs.flowviz.process import ( + OpenFlowFactory, + FileProcessor, +) + + +class HTMLProcessor(OpenFlowFactory, FileProcessor): + """File processor that prints Openflow tables in HTML.""" + + def __init__(self, opts): + super().__init__(opts) + self.data = dict() + + def start_file(self, name, filename): + self.tables = dict() + + def stop_file(self, name, filename): + self.data[name] = self.tables + + def process_flow(self, flow, name): + table = flow.info.get("table") or 0 + if not self.tables.get(table): + self.tables[table] = list() + self.tables[table].append(flow) + + def html(self): + html_obj = "" + for name, tables in self.data.items(): + name = name.replace(" ", "_") + html_obj += "

{}

".format(name) + html_obj += "
" + for table, flows in tables.items(): + formatter = HTMLFormatter(self.opts) + + def anchor(x): + return "#table_%s_%s" % (name, x.value["table"]) + + formatter.style.set_value_style( + "resubmit", + HTMLStyle( + formatter.style.get("value.resubmit"), + anchor_gen=anchor, + ), + ) + html_obj += ( + "

Table {table}

".format( + name=name, table=table + ) + ) + html_obj += "
    ".format(table) + for flow in flows: + html_obj += "
  • ".format(flow.id) + highlighted = None + if self.opts.get("highlight"): + result = self.opts.get("highlight").evaluate(flow) + if result: + highlighted = result.kv + buf = HTMLBuffer() + formatter.format_flow(buf, flow, highlighted) + html_obj += buf.text + html_obj += "
    • " + html_obj += "
" + html_obj += "
" + + return html_obj diff --git a/python/ovs/flowviz/ovs-flowviz.conf b/python/ovs/flowviz/ovs-flowviz.conf index 3acd0a29e..165c453ec 100644 --- a/python/ovs/flowviz/ovs-flowviz.conf +++ b/python/ovs/flowviz/ovs-flowviz.conf @@ -4,7 +4,7 @@ # # [FORMAT].[PORTION].[SELECTOR].[ELEMENT] = [VALUE] # -# * FORMAT: console +# * FORMAT: console or html # * PORTION: The portion of the flow that the style applies to # - key: Selects how to print the key of a KeyValue pair # - key: Selects how to print the value of a KeyValue pair @@ -25,6 +25,11 @@ # - underline: if set to "true", the selected portion will be underlined # #[1] https://rich.readthedocs.io/en/stable/appendix/colors.html#standard-colors +# +# HTML Styles +# ============== +# * ELEMENT: +# - color: defines the color in hex format [styles.dark] @@ -92,3 +97,12 @@ console.value.highlighted.color = #f20905 console.key.highlighted.underline = true console.value.highlighted.underline = true console.delim.highlighted.underline = true + +# html +html.key.color = #00005f +html.value.color = #870000 +html.key.resubmit.color = #00d700 +html.key.output.color = #005f00 +html.value.output.color = #00d700 +html.key.highlighted.color = #FF00FF +html.value.highlighted.color = #FF00FF From patchwork Wed Mar 13 09:03:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1911595 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=SmmgcOmx; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Tvl1m6SPTz1yWy for ; Wed, 13 Mar 2024 20:04:12 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 48B55415AC; Wed, 13 Mar 2024 09:04:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZsHuQuWHMfdY; Wed, 13 Mar 2024 09:04:04 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 51FAA414FA Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=SmmgcOmx Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 51FAA414FA; Wed, 13 Mar 2024 09:04:00 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id F0885C0072; Wed, 13 Mar 2024 09:03:59 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 26F11C0037 for ; Wed, 13 Mar 2024 09:03:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 80B7381F78 for ; Wed, 13 Mar 2024 09:03:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 09vQCVuqCfbj for ; Wed, 13 Mar 2024 09:03:50 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp1.osuosl.org B2A1881A47 Authentication-Results: smtp1.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org B2A1881A47 Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=SmmgcOmx Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id B2A1881A47 for ; Wed, 13 Mar 2024 09:03:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710320628; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hbsrznNyBJty/BOfoqm0+TKyw6qRAMZ+8SvKgSOXKhc=; b=SmmgcOmxOdZjisFo6UacGqGE7wDgTYCyqqdbbnkLpiMJ1f1JiwnaezNYkPIRk9+uujEsl/ up9/iX/w5kAPELCToKuzoqow7TB+K2BugdO6D42clKtZIdX92rhLTEPdcH9+4OUBVwA7tD xyXDjS2fjr+vAPrveZvdjGVaaz6MxG0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-482-AwuhFpVWO9up_Hy2CnFzFQ-1; Wed, 13 Mar 2024 05:03:44 -0400 X-MC-Unique: AwuhFpVWO9up_Hy2CnFzFQ-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6D4BA85CBB4 for ; Wed, 13 Mar 2024 09:03:44 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.194.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id BCC31492BC6; Wed, 13 Mar 2024 09:03:43 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Wed, 13 Mar 2024 10:03:26 +0100 Message-ID: <20240313090334.414226-7-amorenoz@redhat.com> In-Reply-To: <20240313090334.414226-1-amorenoz@redhat.com> References: <20240313090334.414226-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v2 06/12] python: ovs: flowviz: Add datapath tree format. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Datapath flows can be arranged into a "tree"-like structure based on recirculation ids, e.g: recirc(0),eth(...),ipv4(...) actions=ct,recirc(0x42) \-> recirc(42),ct_state(0/0),eth(...),ipv4(...) actions=1 \-> recirc(42),ct_state(1/0),eth(...),ipv4(...) actions=userspace(...) This patch adds support for building such logical datapath trees in a format-agnostic way and adds support for console-based formatting supporting: - head-maps formatting of statistics - hash-based pallete of recirculation ids: each recirculation id is assigned a unique color to easily follow the sequence of related actions. - full-tree filtering: if a user specifies a filter, an entire subtree is filtered out if none of its branches satisfy it. Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 1 + python/ovs/flowviz/console.py | 21 +++ python/ovs/flowviz/odp/cli.py | 19 ++- python/ovs/flowviz/odp/tree.py | 291 +++++++++++++++++++++++++++++++++ 4 files changed, 331 insertions(+), 1 deletion(-) create mode 100644 python/ovs/flowviz/odp/tree.py diff --git a/python/automake.mk b/python/automake.mk index 0487494d0..b3fef9bed 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -71,6 +71,7 @@ ovs_flowviz = \ python/ovs/flowviz/main.py \ python/ovs/flowviz/odp/__init__.py \ python/ovs/flowviz/odp/cli.py \ + python/ovs/flowviz/odp/tree.py \ python/ovs/flowviz/ofp/__init__.py \ python/ovs/flowviz/ofp/cli.py \ python/ovs/flowviz/ofp/html.py \ diff --git a/python/ovs/flowviz/console.py b/python/ovs/flowviz/console.py index 4a3443360..93cd9b0b1 100644 --- a/python/ovs/flowviz/console.py +++ b/python/ovs/flowviz/console.py @@ -13,6 +13,8 @@ # limitations under the License. import colorsys +import itertools +import zlib from rich.console import Console from rich.color import Color @@ -170,6 +172,25 @@ def heat_pallete(min_value, max_value): return heat +def hash_pallete(hue, saturation, value): + """Generates a color pallete with the cartesian product + of the hsv values provided and returns a callable that assigns a color for + each value hash + """ + HSV_tuples = itertools.product(hue, saturation, value) + RGB_tuples = map(lambda x: colorsys.hsv_to_rgb(*x), HSV_tuples) + styles = [ + Style(color=Color.from_rgb(r * 255, g * 255, b * 255)) + for r, g, b in RGB_tuples + ] + + def get_style(string): + hash_val = zlib.crc32(bytes(str(string), "utf-8")) + return styles[hash_val % len(styles)] + + return get_style + + def default_highlight(): """Generates a default style for highlights.""" return Style(underline=True) diff --git a/python/ovs/flowviz/odp/cli.py b/python/ovs/flowviz/odp/cli.py index a1cba0135..615bac55b 100644 --- a/python/ovs/flowviz/odp/cli.py +++ b/python/ovs/flowviz/odp/cli.py @@ -13,8 +13,8 @@ # limitations under the License. import click - from ovs.flowviz.main import maincli +from ovs.flowviz.odp.tree import ConsoleTreeProcessor from ovs.flowviz.process import ( ConsoleProcessor, DatapathFactory, @@ -65,3 +65,20 @@ def console(opts, heat_map): ) proc.process() proc.print() + + +@datapath.command() +@click.option( + "-h", + "--heat-map", + is_flag=True, + default=False, + show_default=True, + help="Create heat-map with packet and byte counters", +) +@click.pass_obj +def tree(opts, heat_map): + """Print the flows in a tree based on the 'recirc_id'.""" + processor = ConsoleTreeProcessor(opts) + processor.process() + processor.print(heat_map) diff --git a/python/ovs/flowviz/odp/tree.py b/python/ovs/flowviz/odp/tree.py new file mode 100644 index 000000000..d249e7d6d --- /dev/null +++ b/python/ovs/flowviz/odp/tree.py @@ -0,0 +1,291 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from rich.style import Style +from rich.text import Text +from rich.tree import Tree + +from ovs.flowviz.console import ( + ConsoleFormatter, + ConsoleBuffer, + hash_pallete, + heat_pallete, + file_header, +) +from ovs.flowviz.process import ( + DatapathFactory, + FileProcessor, +) + + +class TreeElem: + """Element in the tree. + Args: + children (list[TreeElem]): Optional, list of children + is_root (bool): Optional; whether this is the root elemen + """ + + def __init__(self, children=None, is_root=False): + self.children = children or list() + self.is_root = is_root + + def append(self, child): + self.children.append(child) + + +class FlowElem(TreeElem): + """An element that contains a flow. + Args: + flow (Flow): The flow that this element contains + children (list[TreeElem]): Optional, list of children + is_root (bool): Optional; whether this is the root elemen + """ + + def __init__(self, flow, children=None, is_root=False): + self.flow = flow + super(FlowElem, self).__init__(children, is_root) + + def evaluate_any(self, filter): + """Evaluate the filter on the element and all its children. + Args: + filter(OFFilter): the filter to evaluate + + Returns: + True if ANY of the flows (including self and children) evaluates + true + """ + if filter.evaluate(self.flow): + return True + + return any([child.evaluate_any(filter) for child in self.children]) + + +class FlowTree: + """A Flow tree is a a class that processes datapath flows into a tree based + on recirculation ids. + + Args: + flows (list[ODPFlow]): Optional, initial list of flows + root (TreeElem): Optional, root of the tree. + """ + + def __init__(self, flows=None, root=TreeElem(is_root=True)): + self._flows = {} + self.root = root + if flows: + for flow in flows: + self.add(flow) + + def add(self, flow): + """Add a flow""" + rid = flow.match.get("recirc_id") or 0 + if not self._flows.get(rid): + self._flows[rid] = list() + self._flows[rid].append(flow) + + def build(self): + """Build the flow tree.""" + self._build(self.root, 0) + + def traverse(self, callback): + """Traverses the tree calling callback on each element. + + callback: callable that accepts two TreeElem, the current one being + traversed and its parent + func callback(elem, parent): + ... + Note that "parent" can be None if it's the first element. + """ + self._traverse(self.root, None, callback) + + def _traverse(self, elem, parent, callback): + callback(elem, parent) + + for child in elem.children: + self._traverse(child, elem, callback) + + def _build(self, parent, recirc): + """Build the subtree starting at a specific recirc_id. + Recursive function. + + Args: + parent (TreeElem): parent of the (sub)tree + recirc(int): the recirc_id subtree to build + """ + flows = self._flows.get(recirc) + if not flows: + return + for flow in sorted( + flows, key=lambda x: x.info.get("packets") or 0, reverse=True + ): + next_recircs = self._get_next_recirc(flow) + + elem = self._new_elem(flow, parent) + parent.append(elem) + + for next_recirc in next_recircs: + self._build(elem, next_recirc) + + def _get_next_recirc(self, flow): + """Get the next recirc_ids from a Flow. + + The recirc_id is obtained from actions such as recirc, but also + complex actions such as check_pkt_len and sample + Args: + flow (ODPFlow): flow to get the recirc_id from. + Returns: + set of next recirculation ids. + """ + + # Helper function to find a recirc in a dictionary of actions. + def find_in_list(actions_list): + recircs = [] + for item in actions_list: + (action, value) = next(iter(item.items())) + if action == "recirc": + recircs.append(value) + elif action == "check_pkt_len": + recircs.extend(find_in_list(value.get("gt"))) + recircs.extend(find_in_list(value.get("le"))) + elif action == "clone": + recircs.extend(find_in_list(value)) + elif action == "sample": + recircs.extend(find_in_list(value.get("actions"))) + return recircs + + recircs = [] + recircs.extend(find_in_list(flow.actions)) + + return set(recircs) + + def _new_elem(self, flow, _): + """Creates a new TreeElem. + + Default implementation is to create a FlowElem. Derived classes can + override this method to return any derived TreeElem + """ + return FlowElem(flow) + + def filter(self, filter): + """Removes the first level subtrees if none of its sub-elements match + the filter. + + Args: + filter(OFFilter): filter to apply + """ + to_remove = list() + for l0 in self.root.children: + passes = l0.evaluate_any(filter) + if not passes: + to_remove.append(l0) + for elem in to_remove: + self.root.children.remove(elem) + + +class ConsoleTreeProcessor(DatapathFactory, FileProcessor): + def __init__(self, opts): + super().__init__(opts) + self.data = dict() + self.ofconsole = ConsoleFormatter(self.opts) + + # Generate a color pallete for cookies. + recirc_style_gen = hash_pallete( + hue=[x / 50 for x in range(0, 50)], saturation=[0.7], value=[0.8] + ) + + style = self.ofconsole.style + style.set_default_value_style(Style(color="grey66")) + style.set_key_style("output", Style(color="green")) + style.set_value_style("output", Style(color="green")) + style.set_value_style("recirc", recirc_style_gen) + style.set_value_style("recirc_id", recirc_style_gen) + + def start_file(self, name, filename): + self.tree = ConsoleTree(self.ofconsole, self.opts) + + def process_flow(self, flow, name): + self.tree.add(flow) + + def process(self): + super().process(False) + + def stop_file(self, name, filename): + self.data[name] = self.tree + + def print(self, heat_map): + for name, tree in self.data.items(): + self.ofconsole.console.print("\n") + self.ofconsole.console.print(file_header(name)) + tree.build() + if self.opts.get("filter"): + tree.filter(self.opts.get("filter")) + tree.print(heat_map) + + +class ConsoleTree(FlowTree): + """ConsoleTree is a FlowTree that prints the tree in the console. + + Args: + console (ConsoleFormatter): console to use for printing + opts (dict): Options dictionary + """ + + class ConsoleElem(FlowElem): + def __init__(self, flow=None, is_root=False): + self.tree = None + super(ConsoleTree.ConsoleElem, self).__init__( + flow, is_root=is_root + ) + + def __init__(self, console, opts): + self.console = console + self.opts = opts + super(ConsoleTree, self).__init__(root=self.ConsoleElem(is_root=True)) + + def _new_elem(self, flow, _): + """Override _new_elem to provide ConsoleElems""" + return self.ConsoleElem(flow) + + def _append_to_tree(self, elem, parent): + """Callback to be used for FlowTree._build + Appends the flow to the rich.Tree + """ + if elem.is_root: + elem.tree = Tree("Datapath Flows (logical)") + return + + buf = ConsoleBuffer(Text()) + highlighted = None + if self.opts.get("highlight"): + result = self.opts.get("highlight").evaluate(elem.flow) + if result: + highlighted = result.kv + self.console.format_flow(buf, elem.flow, highlighted) + elem.tree = parent.tree.add(buf.text) + + def print(self, heat=False): + """Print the Flow Tree. + Args: + heat (bool): Optional; whether heat-map style shall be applied + """ + if heat: + for field in ["packets", "bytes"]: + values = [] + for flow_list in self._flows.values(): + values.extend([f.info.get(field) or 0 for f in flow_list]) + self.console.style.set_value_style( + field, heat_pallete(min(values), max(values)) + ) + self.traverse(self._append_to_tree) + self.console.console.print(self.root.tree) From patchwork Wed Mar 13 09:03:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1911596 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=hWRlRcNp; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Tvl1r0HD6z1yWy for ; Wed, 13 Mar 2024 20:04:16 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 2285C60891; Wed, 13 Mar 2024 09:04:14 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5IldtRIJKmSX; Wed, 13 Mar 2024 09:04:05 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 426B360AC8 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=hWRlRcNp Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 426B360AC8; Wed, 13 Mar 2024 09:04:02 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id CCF03C0DD7; Wed, 13 Mar 2024 09:04:01 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 22262C0DD2 for ; Wed, 13 Mar 2024 09:03:59 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 5CDB881F8B for ; Wed, 13 Mar 2024 09:03:55 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GQr0AjHr774Q for ; Wed, 13 Mar 2024 09:03:50 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp1.osuosl.org 9810F81F1D Authentication-Results: smtp1.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 9810F81F1D Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=hWRlRcNp Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 9810F81F1D for ; Wed, 13 Mar 2024 09:03:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710320628; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1VGvc77fhbk+ymiXHsfoiGKw1+5rb/X9c7mY9Qvi8ME=; b=hWRlRcNpe2mc2u4KdZdrKI5xj2KXdb0F26hglGb7PTRau5sIEXJgQ0ji/+bU4Ymi3Px/pM Sf2jQyZ+01haYe3Uk7v4nKYwcQO4r79dbjCJ2hWm/cqySPJAVPGt828nIQKSAa3nL0DDIU TbgR6GbS7ntmsl3YGGKj4fg76oiK4yo= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-620-xGBafu92N2qxP3R7UstWvg-1; Wed, 13 Mar 2024 05:03:45 -0400 X-MC-Unique: xGBafu92N2qxP3R7UstWvg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 90B2F3C02B4E for ; Wed, 13 Mar 2024 09:03:45 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.194.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id AB711492BC6; Wed, 13 Mar 2024 09:03:44 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Wed, 13 Mar 2024 10:03:27 +0100 Message-ID: <20240313090334.414226-8-amorenoz@redhat.com> In-Reply-To: <20240313090334.414226-1-amorenoz@redhat.com> References: <20240313090334.414226-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v2 07/12] python: ovs: flowviz: Add OpenFlow logical view. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This view is interesting for debugging the logical pipeline. It arranges the flows in "logical" groups (not to be confused with OVN's Logical_Flows). A logical group of flows is a set of flows that: - Have the same table number and priority - Match on the same fields (regardless of the value they match against) - Have the same actions, regardless of the arguments for those actions, except for output and recirc, for which arguments do care. Optionally, the cookie can also be force to be unique for the logical group. By doing so, we can extend the information we show by querying an external OVN database and running "ovn-detrace" on each cookie. The result is a compact list of flow groups with interlieved OVN information. Furthermore, if connected to an OVN database, we can apply an OVN regexp filter. Examples: $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow logic $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow logic -s -h $ export OVN_NB_DB=... $ export OVN_SB_DB=... $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow logic -d $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow logic -d --ovn-filter="acl.*icmp4" Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 4 +- python/ovs/flowviz/ofp/cli.py | 113 ++++++++++++ python/ovs/flowviz/ofp/logic.py | 303 ++++++++++++++++++++++++++++++++ 3 files changed, 418 insertions(+), 2 deletions(-) create mode 100644 python/ovs/flowviz/ofp/logic.py diff --git a/python/automake.mk b/python/automake.mk index b3fef9bed..449daf023 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -74,12 +74,12 @@ ovs_flowviz = \ python/ovs/flowviz/odp/tree.py \ python/ovs/flowviz/ofp/__init__.py \ python/ovs/flowviz/ofp/cli.py \ + python/ovs/flowviz/ofp/logic.py \ python/ovs/flowviz/ofp/html.py \ python/ovs/flowviz/ovs-flowviz \ python/ovs/flowviz/process.py -# These python files are used at build time but not runtime, -# so they are not installed. +# These python files are used at build time but not runtime, so they are not installed. EXTRA_DIST += \ python/ovs_build_helpers/__init__.py \ python/ovs_build_helpers/extract_ofp_fields.py \ diff --git a/python/ovs/flowviz/ofp/cli.py b/python/ovs/flowviz/ofp/cli.py index 2cd8e1c89..51428ede0 100644 --- a/python/ovs/flowviz/ofp/cli.py +++ b/python/ovs/flowviz/ofp/cli.py @@ -12,10 +12,13 @@ # See the License for the specific language governing permissions and # limitations under the License. +import os + import click from ovs.flowviz.main import maincli from ovs.flowviz.ofp.html import HTMLProcessor +from ovs.flowviz.ofp.logic import LogicFlowProcessor from ovs.flowviz.process import ( ConsoleProcessor, OpenFlowFactory, @@ -69,6 +72,116 @@ def console(opts, heat_map): proc.print() +def ovn_detrace_callback(ctx, param, value): + """click callback to add detrace information to config object and + set general ovn-detrace flag to True + """ + ctx.obj[param.name] = value + if value != param.default: + ctx.obj["ovn_detrace_flag"] = True + return value + + +@openflow.command() +@click.option( + "-d", + "--ovn-detrace", + "ovn_detrace_flag", + is_flag=True, + show_default=True, + help="Use ovn-detrace to extract cookie information (implies '-c')", +) +@click.option( + "--ovn-detrace-path", + default="/usr/bin", + type=click.Path(), + help="Use an alternative path to where ovn_detrace.py is located. " + "Instead of using this option you can just set PYTHONPATH accordingly.", + show_default=True, + callback=ovn_detrace_callback, +) +@click.option( + "--ovnnb-db", + default=os.getenv("OVN_NB_DB") or "unix:/var/run/ovn/ovnnb_db.sock", + help="Specify the OVN NB database string (implies -d). " + "If the OVN_NB_DB environment variable is set, it's used as default. " + "Otherwise, the default is unix:/var/run/ovn/ovnnb_db.sock", + callback=ovn_detrace_callback, +) +@click.option( + "--ovnsb-db", + default=os.getenv("OVN_SB_DB") or "unix:/var/run/ovn/ovnsb_db.sock", + help="Specify the OVN NB database string (implies -d). " + "If the OVN_NB_DB environment variable is set, it's used as default. " + "Otherwise, the default is unix:/var/run/ovn/ovnnb_db.sock", + callback=ovn_detrace_callback, +) +@click.option( + "-o", + "--ovn-filter", + help="Specify a filter to be run on ovn-detrace information (implied -d). " + "Format: python regular expression " + "(see https://docs.python.org/3/library/re.html)", + callback=ovn_detrace_callback, +) +@click.option( + "-s", + "--show-flows", + is_flag=True, + default=False, + show_default=True, + help="Show the full flows under each logical flow", +) +@click.option( + "-c", + "--cookie", + "cookie_flag", + is_flag=True, + default=False, + show_default=True, + help="Consider the cookie in the logical flow", +) +@click.option( + "-h", + "--heat-map", + is_flag=True, + default=False, + show_default=True, + help="Create heat-map with packet and byte counters (when -s is used)", +) +@click.pass_obj +def logic( + opts, + ovn_detrace_flag, + ovn_detrace_path, + ovnnb_db, + ovnsb_db, + ovn_filter, + show_flows, + cookie_flag, + heat_map, +): + """ + Print the logical structure of the flows. + + First, sorts the flows based on tables and priorities. + Then, deduplicates logically equivalent flows: these a flows that match + on the same set of fields (regardless of the values they match against), + have the same priority, and actions (regardless of action arguments, + except in the case of output and recirculate). + Optionally, the cookie can also be considered to be part of the logical + flow. + """ + if ovn_detrace_flag: + opts["ovn_detrace_flag"] = True + if opts.get("ovn_detrace_flag"): + cookie_flag = True + + processor = LogicFlowProcessor(opts, cookie_flag, heat_map) + processor.process() + processor.print(show_flows) + + @openflow.command() @click.pass_obj def html(opts): diff --git a/python/ovs/flowviz/ofp/logic.py b/python/ovs/flowviz/ofp/logic.py new file mode 100644 index 000000000..db2124374 --- /dev/null +++ b/python/ovs/flowviz/ofp/logic.py @@ -0,0 +1,303 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import sys +import io +import re + +from rich.tree import Tree +from rich.text import Text + +from ovs.flowviz.process import FileProcessor, OpenFlowFactory +from ovs.flowviz.console import ( + ConsoleFormatter, + ConsoleBuffer, + hash_pallete, + file_header, + heat_pallete, +) + + +class LFlow: + """A Logical Flow represents the scheleton of a flow. + + Two logical flows have the same logical representation if they match + against same fields (regardless of the matching value) and have the same + set of actions (regardless of the actions' arguments, except for those + in the exact_actions list). + + Attributes: + flow (OFPFlow): The flow + exact_actions(list): Optional; list of action keys that are + considered unique if the value is also the same. + match_cookie (bool): Optional; if cookies are part of the logical + flow + """ + + def __init__(self, flow, exact_actions=[], match_cookie=False): + self.cookie = flow.info.get("cookie") or 0 if match_cookie else None + self.priority = flow.match.get("priority") or 0 + self.match_keys = tuple([kv.key for kv in flow.match_kv]) + + self.action_keys = tuple( + [ + kv.key + for kv in flow.actions_kv + if kv.key not in exact_actions + ] + ) + self.match_action_kvs = [ + kv for kv in flow.actions_kv if kv.key in exact_actions + ] + + def __eq__(self, other): + return ( + (self.cookie == other.cookie if self.cookie else True) + and self.priority == other.priority + and self.action_keys == other.action_keys + and self.equal_match_action_kvs(other) + and self.match_keys == other.match_keys + ) + + def equal_match_action_kvs(self, other): + """ Compares the logical flow's match action key-values with the + others. + + Args: + other (LFlow): The other LFlow to compare against + + Returns true if both LFlow have the same action k-v. + """ + if len(other.match_action_kvs) != len(self.match_action_kvs): + return False + + for kv in self.match_action_kvs: + found = False + for other_kv in other.match_action_kvs: + if self.match_kv(kv, other_kv): + found = True + break + if not found: + return False + return True + + def match_kv(self, one, other): + """Compares a KeyValue. + Args: + one, other (KeyValue): The objects to compare + + Returns true if both KeyValue objects have the same key and value + """ + return one.key == other.key and one.value == other.value + + def __hash__(self): + hash_data = [ + self.cookie, + self.priority, + self.action_keys, + tuple((kv.key, str(kv.value)) for kv in self.match_action_kvs), + self.match_keys, + ] + if self.cookie: + hash_data.append(self.cookie) + return tuple(hash_data).__hash__() + + def format(self, buf, formatter): + """Format the Logical Flow into a Buffer.""" + if self.cookie: + buf.append_extra( + "cookie={} ".format(hex(self.cookie)).ljust(18), + style=cookie_style_gen(str(self.cookie)), + ) + + buf.append_extra( + "priority={} ".format(self.priority), style="steel_blue" + ) + buf.append_extra(",".join(self.match_keys), style="steel_blue") + buf.append_extra(" ---> ", style="bold magenta") + buf.append_extra(",".join(self.action_keys), style="steel_blue") + + if len(self.match_action_kvs) > 0: + buf.append_extra(" ", style=None) + + for kv in self.match_action_kvs: + formatter.format_kv(buf, kv, formatter.style) + buf.append_extra(",", style=None) + + +class LogicFlowProcessor(OpenFlowFactory, FileProcessor): + def __init__(self, opts, match_cookie, heat_map): + super().__init__(opts) + self.data = dict() + self.min_max = dict() + self.match_cookie = match_cookie + self.heat_map = ["n_packets", "n_bytes"] if heat_map else [] + self.ovn_detrace = ( + OVNDetrace(opts) if opts.get("ovn_detrace_flag") else None + ) + + def start_file(self, name, filename): + if len(self.heat_map) > 0: + self.min = [-1] * len(self.heat_map) + self.max = [0] * len(self.heat_map) + self.tables = dict() + + def stop_file(self, name, filename): + if len(self.heat_map) > 0: + self.min_max[name] = (self.min, self.max) + self.data[name] = self.tables + + def process_flow(self, flow, name): + """Sort the flows by table and logical flow.""" + # Running calculation of min and max values for all the fields that + # take place in the heatmap. + for i, field in enumerate(self.heat_map): + val = flow.info.get(field) + if self.min[i] == -1 or val < self.min[i]: + self.min[i] = val + if val > self.max[i]: + self.max[i] = val + + table = flow.info.get("table") or 0 + if not self.tables.get(table): + self.tables[table] = dict() + + # Group flows by logical hash + lflow = LFlow( + flow, + exact_actions=["output", "resubmit", "drop"], + match_cookie=self.match_cookie, + ) + + if not self.tables[table].get(lflow): + self.tables[table][lflow] = list() + + self.tables[table][lflow].append(flow) + + def print(self, show_flows): + formatter = ConsoleFormatter(opts=self.opts) + console = formatter.console + for name, tables in self.data.items(): + console.print("\n") + console.print(file_header(name)) + tree = Tree("Ofproto Flows (logical)") + + for table_num in sorted(tables.keys()): + table = tables[table_num] + table_tree = tree.add("** TABLE {} **".format(table_num)) + + if len(self.heat_map) > 0 and len(table.values()) > 0: + for i, field in enumerate(self.heat_map): + (min_val, max_val) = self.min_max[name][i] + formatter.style.set_value_style( + field, heat_pallete(min_val, max_val) + ) + + for lflow in sorted( + table.keys(), + key=(lambda x: x.priority), + reverse=True, + ): + flows = table[lflow] + ovn_info = None + if self.ovn_detrace: + ovn_info = self.ovn_detrace.get_ovn_info(lflow.cookie) + if self.opts.get("ovn_filter"): + ovn_regexp = re.compile( + self.opts.get("ovn_filter") + ) + if not ovn_regexp.search(ovn_info): + continue + + buf = ConsoleBuffer(Text()) + + lflow.format(buf, formatter) + buf.append_extra( + " ( x {} )".format(len(flows)), + style="dark_olive_green3", + ) + lflow_tree = table_tree.add(buf.text) + + if ovn_info: + ovn = lflow_tree.add("OVN Info") + for part in ovn_info.split("\n"): + if part.strip(): + ovn.add(part.strip()) + + if show_flows: + for flow in flows: + buf = ConsoleBuffer(Text()) + highlighted = None + if self.opts.get("highlight"): + result = self.opts.get("highlight").evaluate( + flow + ) + if result: + highlighted = result.kv + formatter.format_flow(buf, flow, highlighted) + lflow_tree.add(buf.text) + + console.print(tree) + + +class OVNDetrace(object): + def __init__(self, opts): + if not opts.get("ovn_detrace_flag"): + raise Exception("Cannot initialize OVN Detrace connection") + + if opts.get("ovn_detrace_path"): + sys.path.append(opts.get("ovn_detrace_path")) + + import ovn_detrace + + class FakePrinter(ovn_detrace.Printer): + def __init__(self): + self.buff = io.StringIO() + + def print_p(self, msg): + print(" * ", msg, file=self.buff) + + def print_h(self, msg): + print(" * ", msg, file=self.buff) + + def clear(self): + self.buff = io.StringIO() + + self.ovn_detrace = ovn_detrace + self.ovnnb_conn = ovn_detrace.OVSDB( + opts.get("ovnnb_db"), "OVN_Northbound" + ) + self.ovnsb_conn = ovn_detrace.OVSDB( + opts.get("ovnsb_db"), "OVN_Southbound" + ) + self.ovn_printer = FakePrinter() + self.cookie_handlers = ovn_detrace.get_cookie_handlers( + self.ovnnb_conn, self.ovnsb_conn, self.ovn_printer + ) + + def get_ovn_info(self, cookie): + self.ovn_printer.clear() + self.ovn_detrace.print_record_from_cookie( + self.ovnsb_conn, self.cookie_handlers, "{:x}".format(cookie) + ) + return self.ovn_printer.buff.getvalue() + + +# Try to make it easy to spot same cookies by printing them in different +# colors +cookie_style_gen = hash_pallete( + hue=[x / 10 for x in range(0, 10)], + saturation=[0.5], + value=[0.5 + x / 10 * (0.85 - 0.5) for x in range(0, 10)], +) From patchwork Wed Mar 13 09:03:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1911593 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=JOWP1u9I; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Tvl1c4Xy5z1yWy for ; Wed, 13 Mar 2024 20:04:04 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id C4FB160A94; Wed, 13 Mar 2024 09:04:02 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IuwbodppdRj7; Wed, 13 Mar 2024 09:04:00 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org F3ED360A98 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=JOWP1u9I Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id F3ED360A98; Wed, 13 Mar 2024 09:03:56 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 985FBC0072; Wed, 13 Mar 2024 09:03:56 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 48D38C0037 for ; Wed, 13 Mar 2024 09:03:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 6CCCC414C0 for ; Wed, 13 Mar 2024 09:03:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id y0T_GPJmNLX9 for ; Wed, 13 Mar 2024 09:03:50 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp2.osuosl.org 62429405F0 Authentication-Results: smtp2.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 62429405F0 Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=JOWP1u9I Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 62429405F0 for ; Wed, 13 Mar 2024 09:03:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710320628; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JrNSoJLYaNRW9yMokZQzbmO5eSQzo8HcTAVQfPcN8mw=; b=JOWP1u9ITNEooR4pKULADxKDoqe3/ezn6BxWrmBueehLc1RTezYmW8Be+0cGmU12iXRJgg pdeZyXTLFZUlwshnkpP7NT0bz/v1h5SF2OsAEV9ykrUNXJ1Fh/FhDPhAT+MWFm2EIjgnSZ yPCv/WeMMAw0+P50u5uc6AWtTtyvlsc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-296-gEcOSp6YNberc155EmIwtg-1; Wed, 13 Mar 2024 05:03:46 -0400 X-MC-Unique: gEcOSp6YNberc155EmIwtg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 81B09101A586 for ; Wed, 13 Mar 2024 09:03:46 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.194.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id D0BE0492BC7; Wed, 13 Mar 2024 09:03:45 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Wed, 13 Mar 2024 10:03:28 +0100 Message-ID: <20240313090334.414226-9-amorenoz@redhat.com> In-Reply-To: <20240313090334.414226-1-amorenoz@redhat.com> References: <20240313090334.414226-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v2 08/12] python: ovs: flowviz: Add Openflow cookie format. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" When anaylizing OVN issues, it might be useful to see what OpenFlow flows were generated from each logical flow. In order to make it simpler to visualize this, add a cookie format that simply sorts the flows first by cookie, then by table. Example: $ export OVN_NB_DB=... $ export OVN_SB_DB=... $ ovs-vsctl dump-flows br-int | ovs-flowviz openflow cookie --ovn-filter="acl.*icmp4" $ ovs-vsctl dump-flows br-int | ovs-flowviz openflow cookie --ovn-detrace Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/ovs/flowviz/ofp/cli.py | 57 +++++++++++++++++++++++++++++- python/ovs/flowviz/ofp/logic.py | 61 +++++++++++++++++++++++++++++++++ 2 files changed, 117 insertions(+), 1 deletion(-) diff --git a/python/ovs/flowviz/ofp/cli.py b/python/ovs/flowviz/ofp/cli.py index 51428ede0..690e7618e 100644 --- a/python/ovs/flowviz/ofp/cli.py +++ b/python/ovs/flowviz/ofp/cli.py @@ -18,7 +18,7 @@ import click from ovs.flowviz.main import maincli from ovs.flowviz.ofp.html import HTMLProcessor -from ovs.flowviz.ofp.logic import LogicFlowProcessor +from ovs.flowviz.ofp.logic import CookieProcessor, LogicFlowProcessor from ovs.flowviz.process import ( ConsoleProcessor, OpenFlowFactory, @@ -182,6 +182,61 @@ def logic( processor.print(show_flows) +@openflow.command() +@click.option( + "-d", + "--ovn-detrace", + "ovn_detrace_flag", + is_flag=True, + show_default=True, + help="Use ovn-detrace to extract cookie information", +) +@click.option( + "--ovn-detrace-path", + default="/usr/bin", + type=click.Path(), + help="Use an alternative path to where ovn_detrace.py is located. " + "Instead of using this option you can just set PYTHONPATH accordingly", + show_default=True, + callback=ovn_detrace_callback, +) +@click.option( + "--ovnnb-db", + default=os.getenv("OVN_NB_DB") or "unix:/var/run/ovn/ovnnb_db.sock", + help="Specify the OVN NB database string (implies -d). " + "If the OVN_NB_DB environment variable is set, it's used as default. " + "Otherwise, the default is unix:/var/run/ovn/ovnnb_db.sock", + callback=ovn_detrace_callback, +) +@click.option( + "--ovnsb-db", + default=os.getenv("OVN_SB_DB") or "unix:/var/run/ovn/ovnsb_db.sock", + help="Specify the OVN NB database string (implies -d). " + "If the OVN_NB_DB environment variable is set, it's used as default. " + "Otherwise, the default is unix:/var/run/ovn/ovnnb_db.sock", + callback=ovn_detrace_callback, +) +@click.option( + "-o", + "--ovn-filter", + help="Specify a filter to be run on ovn-detrace information (implied -d). " + "Format: python regular expression " + "(see https://docs.python.org/3/library/re.html)", + callback=ovn_detrace_callback, +) +@click.pass_obj +def cookie( + opts, ovn_detrace_flag, ovn_detrace_path, ovnnb_db, ovnsb_db, ovn_filter +): + """Print the flow tables sorted by cookie.""" + if ovn_detrace_flag: + opts["ovn_detrace_flag"] = True + + processor = CookieProcessor(opts) + processor.process() + processor.print() + + @openflow.command() @click.pass_obj def html(opts): diff --git a/python/ovs/flowviz/ofp/logic.py b/python/ovs/flowviz/ofp/logic.py index db2124374..9d244d137 100644 --- a/python/ovs/flowviz/ofp/logic.py +++ b/python/ovs/flowviz/ofp/logic.py @@ -301,3 +301,64 @@ cookie_style_gen = hash_pallete( saturation=[0.5], value=[0.5 + x / 10 * (0.85 - 0.5) for x in range(0, 10)], ) + + +class CookieProcessor(OpenFlowFactory, FileProcessor): + """Processor that sorts flows into cookies and tables.""" + + def __init__(self, opts): + super().__init__(opts) + self.data = dict() + self.ovn_detrace = ( + OVNDetrace(opts) if opts.get("ovn_detrace_flag") else None + ) + + def start_file(self, name, filename): + self.cookies = dict() + + def stop_file(self, name, filename): + self.data[name] = self.cookies + + def process_flow(self, flow, name): + """Sort the flows by table and logical flow.""" + cookie = flow.info.get("cookie") or 0 + if not self.cookies.get(cookie): + self.cookies[cookie] = dict() + + table = flow.info.get("table") or 0 + if not self.cookies[cookie].get(table): + self.cookies[cookie][table] = list() + self.cookies[cookie][table].append(flow) + + def print(self): + ofconsole = ConsoleFormatter(opts=self.opts) + console = ofconsole.console + for name, cookies in self.data.items(): + console.print("\n") + console.print(file_header(name)) + tree = Tree("Ofproto Cookie Tree") + + for cookie, tables in cookies.items(): + ovn_info = None + if self.ovn_detrace: + ovn_info = self.ovn_detrace.get_ovn_info(cookie) + if self.opts.get("ovn_filter"): + ovn_regexp = re.compile(self.opts.get("ovn_filter")) + if not ovn_regexp.search(ovn_info): + continue + + cookie_tree = tree.add("** Cookie {} **".format(hex(cookie))) + if ovn_info: + ovn = cookie_tree.add("OVN Info") + for part in ovn_info.split("\n"): + if part.strip(): + ovn.add(part.strip()) + + tables_tree = cookie_tree.add("Tables") + for table, flows in tables.items(): + table_tree = tables_tree.add("* Table {} * ".format(table)) + for flow in flows: + buf = ConsoleBuffer(Text()) + ofconsole.format_flow(buf, flow) + table_tree.add(buf.text) + console.print(tree) From patchwork Wed Mar 13 09:03:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1911599 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ThxMhSvJ; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Tvl235rfjz1yWy for ; Wed, 13 Mar 2024 20:04:27 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id BD07141524; Wed, 13 Mar 2024 09:04:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BuQ1zk8f0M81; Wed, 13 Mar 2024 09:04:21 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 1E4B441575 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ThxMhSvJ Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 1E4B441575; Wed, 13 Mar 2024 09:04:05 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 00F30C0072; Wed, 13 Mar 2024 09:04:03 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1ACE8C0DDB for ; Wed, 13 Mar 2024 09:04:02 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 2B6724152B for ; Wed, 13 Mar 2024 09:03:58 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id r3tqiN8C20O6 for ; Wed, 13 Mar 2024 09:03:54 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp2.osuosl.org 216FC414EE Authentication-Results: smtp2.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 216FC414EE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 216FC414EE for ; Wed, 13 Mar 2024 09:03:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710320629; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Plv0gryON0flNuHPpXWOqiaowCxx3QSihM+SP4OuDaQ=; b=ThxMhSvJtz9zryw3tRvkB/By62VlEHT7MCRHLxA06P6erc7RSXlmmZy2i4XiOMaTlztclG rKsYfkF3CM9wK1anu0gbJ2IIShNdlJO4oi28sCkEAAnw0R51UUUmdZT/2pe7G9d7uJZ4l7 WHpaXCpR198l2RMy99XReDTKEvN4x5c= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-316-l57aPL0vO12wuzB1AKjazQ-1; Wed, 13 Mar 2024 05:03:47 -0400 X-MC-Unique: l57aPL0vO12wuzB1AKjazQ-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7C64F101A586 for ; Wed, 13 Mar 2024 09:03:47 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.194.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id C2BE8492BC6; Wed, 13 Mar 2024 09:03:46 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Wed, 13 Mar 2024 10:03:29 +0100 Message-ID: <20240313090334.414226-10-amorenoz@redhat.com> In-Reply-To: <20240313090334.414226-1-amorenoz@redhat.com> References: <20240313090334.414226-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v2 09/12] python: ovs: flowviz: Add datapath html format. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Using the existing FlowTree and HTMLFormatter, create an HTML tree visualization that also supports collapsing and expanding entire flow trees and subtrees. Examples: $ ovs-appcl dpctl/dump-flows | ovs-flowviz --highlight drop datapath html > /tmp/flows.html $ ovs-appcl dpctl/dump-flows | ovs-flowviz -f "output.port=3" datapath html > /tmp/flows.html Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 1 + python/ovs/flowviz/odp/cli.py | 10 ++ python/ovs/flowviz/odp/html.py | 259 +++++++++++++++++++++++++++++++++ 3 files changed, 270 insertions(+) create mode 100644 python/ovs/flowviz/odp/html.py diff --git a/python/automake.mk b/python/automake.mk index 449daf023..44e9e08ab 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -71,6 +71,7 @@ ovs_flowviz = \ python/ovs/flowviz/main.py \ python/ovs/flowviz/odp/__init__.py \ python/ovs/flowviz/odp/cli.py \ + python/ovs/flowviz/odp/html.py \ python/ovs/flowviz/odp/tree.py \ python/ovs/flowviz/ofp/__init__.py \ python/ovs/flowviz/ofp/cli.py \ diff --git a/python/ovs/flowviz/odp/cli.py b/python/ovs/flowviz/odp/cli.py index 615bac55b..dd64fdc65 100644 --- a/python/ovs/flowviz/odp/cli.py +++ b/python/ovs/flowviz/odp/cli.py @@ -14,6 +14,7 @@ import click from ovs.flowviz.main import maincli +from ovs.flowviz.odp.html import HTMLTreeProcessor from ovs.flowviz.odp.tree import ConsoleTreeProcessor from ovs.flowviz.process import ( ConsoleProcessor, @@ -82,3 +83,12 @@ def tree(opts, heat_map): processor = ConsoleTreeProcessor(opts) processor.process() processor.print(heat_map) + + +@datapath.command() +@click.pass_obj +def html(opts): + """Print the flows in an HTML list sorted by recirc_id.""" + processor = HTMLTreeProcessor(opts) + processor.process() + processor.print() diff --git a/python/ovs/flowviz/odp/html.py b/python/ovs/flowviz/odp/html.py new file mode 100644 index 000000000..4aa08dc70 --- /dev/null +++ b/python/ovs/flowviz/odp/html.py @@ -0,0 +1,259 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from ovs.flowviz.html_format import HTMLBuffer, HTMLFormatter +from ovs.flowviz.odp.tree import FlowElem, FlowTree +from ovs.flowviz.process import DatapathFactory, FileProcessor + + +class HTMLTreeProcessor(DatapathFactory, FileProcessor): + def __init__(self, opts): + super().__init__(opts) + self.data = dict() + + def start_file(self, name, filename): + self.tree = HTMLTree(name, self.opts) + + def process_flow(self, flow, name): + self.tree.add(flow) + + def process(self): + super().process(False) + + def stop_file(self, name, filename): + self.data[name] = self.tree + + def print(self): + html_obj = "" + for name, tree in self.data.items(): + html_obj += "
" + html_obj += "

{}

".format(name) + tree.build() + if self.opts.get("filter"): + tree.filter(self.opts.get("filter")) + html_obj += tree.render() + html_obj += "
" + print(html_obj) + + +class HTMLTree(FlowTree): + """HTMLTree is a Flowtree that prints the tree in html format. + + Args: + opts(dict): Options dictionary + flows(dict[int, list[DPFlow]): Optional; initial flows + """ + + html_header = """ + + + + """ # noqa: E501 + + class HTMLTreeElem(FlowElem): + """An element within the HTML Tree. + + It is composed of a flow and its subflows that can be added by calling + append() + """ + + def __init__(self, parent_name, flow=None, opts=None): + self._parent_name = parent_name + self._formatter = HTMLFormatter(opts) + self._opts = opts + super(HTMLTree.HTMLTreeElem, self).__init__(flow) + + def render(self, item=0): + """Render the HTML Element. + Args: + item (int): the item id + + Returns: + (html_obj, items) tuple where html_obj is the html string and + items is the number of subitems rendered in total + """ + parent_name = self._parent_name.replace(" ", "_") + html_obj = "
" + if self.flow: + html_text = """ + + + """ # noqa: E501 + html_obj += html_text.format( + item=item, id=self.flow.id, name=parent_name + ) + + html_text = '
' # noqa: E501 + html_obj += html_text.format(id=self.flow.id) + buf = HTMLBuffer() + highlighted = None + if self._opts.get("highlight"): + result = self._opts.get("highlight").evaluate(self.flow) + if result: + highlighted = result.kv + self._formatter.format_flow(buf, self.flow, highlighted) + html_obj += buf.text + html_obj += "
" + if self.children: + html_obj += "
" + html_obj += "
    " + for sf in self.children: + item += 1 + html_obj += "
  • " + (html_elem, items) = sf.render(item) + html_obj += html_elem + item += items + html_obj += "
    • " + html_obj += "
" + html_obj += "
" + html_obj += "
" + return html_obj, item + + def __init__(self, name, opts, flows=None): + self.opts = opts + self.name = name + super(HTMLTree, self).__init__( + flows, self.HTMLTreeElem("", flow=None, opts=self.opts) + ) + + def _new_elem(self, flow, _): + """Override _new_elem to provide HTMLTreeElems.""" + return self.HTMLTreeElem(self.name, flow, self.opts) + + def render(self): + """Render the Tree in HTML. + + Returns: + an html string representing the element + """ + name = self.name.replace(" ", "_") + + html_text = """ +""" # noqa: E501 + html_obj = self.html_header + html_text.format(name=name) + + html_obj += "
".format(name=name) + (html_elem, _) = self.root.render() + html_obj += html_elem + html_obj += "
" + return html_obj From patchwork Wed Mar 13 09:03:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1911598 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=T4JZclZp; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Tvl1y0GxPz1yWy for ; Wed, 13 Mar 2024 20:04:22 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id A7CA64152D; Wed, 13 Mar 2024 09:04:19 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QrhI0IuKMGjw; Wed, 13 Mar 2024 09:04:14 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 7B5B041580 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=T4JZclZp Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 7B5B041580; Wed, 13 Mar 2024 09:04:03 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D513DC0DD7; Wed, 13 Mar 2024 09:04:02 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 26D22C0DD9 for ; Wed, 13 Mar 2024 09:04:01 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 1CCEA60AC0 for ; Wed, 13 Mar 2024 09:04:00 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VV9_UnZLW5f7 for ; Wed, 13 Mar 2024 09:03:54 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org EC0A860A78 Authentication-Results: smtp3.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org EC0A860A78 Authentication-Results: smtp3.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=T4JZclZp Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id EC0A860A78 for ; Wed, 13 Mar 2024 09:03:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710320631; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yZRw74yP2HEK0PKR8Kp3xEXAjxW/pbyTXTV3FfSAIhM=; b=T4JZclZpR/kWFTtlNxOA5KGYM5xCXXN/OGI++1wn84Ol60TWhAqn/Dl7f4cHjfASnGEGYD gD+GaRxRN+JA7usbO1miEtY7utyoyMf+/xGy3cptL4JqIA+ImP3WEiqhrq5+DjDX/foe7J UXPzr1CFOfDCBSNwKmuhc+WbJG8KEZ0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-370-Ezh2eofrMRKn7NKZAXaxXA-1; Wed, 13 Mar 2024 05:03:48 -0400 X-MC-Unique: Ezh2eofrMRKn7NKZAXaxXA-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B524D85A58C for ; Wed, 13 Mar 2024 09:03:48 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.194.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id DC840492BC6; Wed, 13 Mar 2024 09:03:47 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Wed, 13 Mar 2024 10:03:30 +0100 Message-ID: <20240313090334.414226-11-amorenoz@redhat.com> In-Reply-To: <20240313090334.414226-1-amorenoz@redhat.com> References: <20240313090334.414226-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v2 10/12] python: ovs: flowviz: Add datapath graph format. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Graph view leverages the tree format (specially the tree-based filtering) and uses graphviz library to build a visual graph of the datapath in graphviz format. Conntrack zones are shown in random colors to help visualize connection tracking interdependencies. An html flag builds an HTML page with both the html flows and the graph (in svg) that enables navegation. Examples: $ ovs-appctl dpctl/dump-flows -m | ovs-flowviz datapath graph | dot -Tpng -o graph.png $ ovs-appctl dpctl/dump-flows -m | ovs-flowviz datapath graph --html > flows.html Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/automake.mk | 1 + python/ovs/flowviz/odp/cli.py | 22 ++ python/ovs/flowviz/odp/graph.py | 418 ++++++++++++++++++++++++++++++++ python/ovs/flowviz/odp/tree.py | 18 +- python/setup.py | 2 +- 5 files changed, 457 insertions(+), 4 deletions(-) create mode 100644 python/ovs/flowviz/odp/graph.py diff --git a/python/automake.mk b/python/automake.mk index 44e9e08ab..9ef000480 100644 --- a/python/automake.mk +++ b/python/automake.mk @@ -71,6 +71,7 @@ ovs_flowviz = \ python/ovs/flowviz/main.py \ python/ovs/flowviz/odp/__init__.py \ python/ovs/flowviz/odp/cli.py \ + python/ovs/flowviz/odp/graph.py \ python/ovs/flowviz/odp/html.py \ python/ovs/flowviz/odp/tree.py \ python/ovs/flowviz/ofp/__init__.py \ diff --git a/python/ovs/flowviz/odp/cli.py b/python/ovs/flowviz/odp/cli.py index dd64fdc65..94fdb80eb 100644 --- a/python/ovs/flowviz/odp/cli.py +++ b/python/ovs/flowviz/odp/cli.py @@ -14,6 +14,7 @@ import click from ovs.flowviz.main import maincli +from ovs.flowviz.odp.graph import GraphProcessor from ovs.flowviz.odp.html import HTMLTreeProcessor from ovs.flowviz.odp.tree import ConsoleTreeProcessor from ovs.flowviz.process import ( @@ -92,3 +93,24 @@ def html(opts): processor = HTMLTreeProcessor(opts) processor.process() processor.print() + + +@datapath.command() +@click.option( + "-h", + "--html", + is_flag=True, + default=False, + show_default=True, + help="Output an html file containing the graph", +) +@click.pass_obj +def graph(opts, html): + """Print the flows in an graphviz (.dot) format showing the relationship + of recirc_ids.""" + if len(opts.get("filename")) > 1: + raise click.BadParameter("Graph format only supports one input file") + + processor = GraphProcessor(opts) + processor.process() + processor.print(html) diff --git a/python/ovs/flowviz/odp/graph.py b/python/ovs/flowviz/odp/graph.py new file mode 100644 index 000000000..b26551e67 --- /dev/null +++ b/python/ovs/flowviz/odp/graph.py @@ -0,0 +1,418 @@ +# Copyright (c) 2023 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""" Defines a Datapath Graph using graphviz. """ +import colorsys +import graphviz +import random + +from ovs.flowviz.odp.html import HTMLTree +from ovs.flowviz.odp.tree import FlowTree +from ovs.flowviz.process import DatapathFactory, FileProcessor + + +class GraphProcessor(DatapathFactory, FileProcessor): + def __init__(self, opts): + super().__init__(opts) + + def start_file(self, name, filename): + self.tree = FlowTree() + + def process_flow(self, flow, name): + self.tree.add(flow) + + def process(self): + super().process(False) + + def print(self, html): + flows = {} + + # Tree traverse callback + def add_flow(elem, _): + if elem.is_root: + return + rid = elem.flow.match.get("recirc_id") or 0 + if not flows.get(rid): + flows[rid] = set() + flows[rid].add(elem.flow) + + self.tree.build() + if self.opts.get("filter"): + self.tree.filter(self.opts.get("filter")) + self.tree.traverse(add_flow) + + if len(flows) == 0: + return + + dpg = DatapathGraph(flows) + if not html: + print(dpg.source()) + return + + html_obj = "" + html_obj += "

Flow Graph

" + html_obj += "
" + svg = dpg.pipe(format="svg") + html_obj += svg.decode("utf-8") + html_obj += "
" + html_tree = HTMLTree("graph", self.opts, flows) + html_tree.build() + html_obj += html_tree.render() + + print(html_obj) + + +class DatapathGraph: + """A DatapathGraph is a class that renders a set of datapath flows into + graphviz graphs. + + Args: + flows(dict[int, list(Flow)]): Dictionary of lists of flows indexed by + recirc_id + """ + + ct_styles = {} + node_styles = { + "default": { + "style": {}, + "desc": "Default", + }, + "action_and_match": { + "style": {"color": "#ff00ff"}, + "desc": "Flow uses CT as match and action", + }, + "match": { + "style": {"color": "#0000ff"}, + "desc": "Flow uses CT only to match", + }, + "action": { + "style": {"color": "#ff0000"}, + "desc": "Flow uses CT only as action", + }, + } + + def __init__(self, flows): + self._flows = flows + + self._output_nodes = [] + self._graph = graphviz.Digraph( + "DP flows", node_attr={"shape": "rectangle"} + ) + self._graph.attr(compound="true") + self._graph.attr(rankdir="LR") + self._graph.attr(ranksep="3") + + self._populate_graph() + + def source(self): + """Return the graphviz source representation of the graph.""" + return self._graph.source + + def pipe(self, *args, **kwargs): + """Output the graph based on arguments given to graphviz.pipe.""" + return self._graph.pipe(*args, **kwargs) + + @classmethod + def recirc_cluster_name(cls, recirc_id): + """Name of the recirculation cluster.""" + return "cluster_recirc_{}".format(hex(recirc_id)) + + @classmethod + def inport_cluster_name(cls, inport): + """Name of the input port cluster.""" + return "cluster_inport_{}".format(inport) + + @classmethod + def invis_node_name(cls, cluster_name): + """Name of the invisible node.""" + return "invis_{}".format(cluster_name) + + @classmethod + def output_node_name(cls, port): + """Name of the ouput node.""" + return "output_{}".format(port) + + def _flow_node(self, flow, name): + """Returns the dictionary of attributes of a graphviz node that + represents the flow with a given name. + """ + summary = "Line: {} \n".format(flow.id) + summary += "\n".join( + [ + flow.section("info").string, + ",".join(flow.match.keys()), + "actions: " + + ",".join(list(a.keys())[0] for a in flow.actions), + ] + ) + + has_ct_match = flow.match.get("ct_state", "0/0") != "0/0" + has_ct_action = bool( + next( + filter(lambda x: x.key in ["ct", "ct_clear"], flow.actions_kv), + None, + ) + ) + + if has_ct_action: + if has_ct_match: + style = "action_and_match" + else: + style = "action" + elif has_ct_match: + style = "match" + else: + style = "default" + + style = self.node_styles.get(style, {}) + + return { + "name": name, + "label": summary, + "tooltip": flow.orig, + "_attributes": style.get("style", {}), + "fontsize": "10", + "nojustify": "true", + "URL": "#flow_{}".format(flow.id), + } + + def _create_recirc_cluster(self, recirc): + """Process a recirculation id, creating its cluster.""" + cluster_name = self.recirc_cluster_name(recirc) + label = "recirc x0{:0x}".format(recirc) + + cluster = self._graph.subgraph(name=cluster_name, comment=label) + with cluster as sg: + sg.attr(rankdir="TB") + sg.attr(ranksep="0.02") + sg.attr(label=label) + sg.attr(margin="5") + self._add_flows_to_graph(sg, self._flows[recirc]) + + self.processed_recircs.append(recirc) + + def _add_flows_to_graph(self, graph, flows): + # Create an invisible node and an edge to the first flow so that + # it ends up at the top of the cluster. + invis = self.invis_node_name(graph.name) + graph.node(invis) + graph.node( + invis, + color="white", + len="0", + shape="point", + width="0", + height="0", + ) + first = True + for flow in flows: + name = "Flow_{}".format(flow.id) + graph.node(**self._flow_node(flow, name)) + if first: + with graph.subgraph() as c: + c.attr(rank="same") + c.edge(name, invis, style="invis") + first = False + # determine next hop based on actions + self._set_next_node_from_actions(name, flow.actions) + + def set_next_node_from_actions(self, name, actions): + """Determine the next nodes based on action list and add edges to + them. + """ + if not self._set_next_node_from_actions(self, name, actions): + # Add to a generic "End" if no other action was detected + self._graph.edge(name, "end") + + def _set_next_node_from_actions(self, name, actions): + created = False + for action in actions: + key, value = next(iter(action.items())) + if key == "check_pkt_len": + created |= self._set_next_node_from_actions( + name, value.get("gt") + ) + created |= self._set_next_node_from_actions( + name, value.get("le") + ) + elif key == "sample": + created |= self._set_next_node_from_actions( + name, value.get("actions") + ) + elif key == "clone": + created |= self._set_next_node_from_actions( + name, value.get("actions") + ) + else: + created |= self._set_next_node_action(name, key, value) + return created + + def _set_next_node_action(self, name, action_name, action_obj): + """Based on the action object, set the next node.""" + if action_name == "recirc": + # If the targer recirculation cluster has not yet been created, + # do it now. + if action_obj not in self.processed_recircs: + self._create_recirc_cluster(action_obj) + + cname = self.recirc_cluster_name(action_obj) + self._graph.edge( + name, + self.invis_node_name(cname), + lhead=cname, + _attributes={"weight": "20"}, + ) + return True + elif action_name == "output": + port = action_obj.get("port") + if port not in self._output_nodes: + self._output_nodes.append(port) + self._graph.edge( + name, self.output_node_name(port), _attributes={"weight": "1"} + ) + return True + elif action_name in ["drop", "userspace", "controller"]: + if action_name not in self._output_nodes: + self._output_nodes.append(action_name) + self._graph.edge(name, action_name, _attributes={"weight": "1"}) + return True + elif action_name == "ct": + zone = action_obj.get("zone", 0) + node_name = "CT zone {}".format(action_obj.get("zone", "default")) + if zone not in self.ct_styles: + # Pick a random (highly saturated) color. + (r, g, b) = colorsys.hsv_to_rgb(random.random(), 1, 1) + color = "#%02x%02x%02x" % ( + int(r * 255), + int(g * 255), + int(b * 255), + ) + self.ct_styles[zone] = color + self._graph.node(node_name, color=color) + + color = self.ct_styles[zone] + self._graph.edge(name, node_name, style="dashed", color=color) + # test + name = node_name + return True + return False + + def _populate_graph(self): + """Populate the the internal graph.""" + self.processed_recircs = [] + + # Create a subcluster for each input port and one for flows that don't + # have in_port() for which we create a dummy inport. + flows_per_inport = {} + free_flows = [] + + for flow in self._flows.get(0): + port = flow.match.get("in_port") + if port: + if not flows_per_inport.get(port): + flows_per_inport[port] = list() + flows_per_inport[port].append(flow) + else: + free_flows.append(flow) + + # It's rare to find flows without input_port match but let's add them + # nevertheless. + if free_flows: + self._graph.edge( + "start", + self.invis_node_name(self.recirc_cluster_name(0)), + lhead=self.recirc_cluster_name(0), + ) + self._graph.node("no_port", shape="Mdiamond") + + # Recirc_clusters are created recursively when an edge is found to + # them. + # Process recirc(0) which is split by input port. + for inport, flows in flows_per_inport.items(): + # Build a subgraph per input port + cluster_name = self.inport_cluster_name(inport) + label = "recirc 0; input port: {}".format(inport) + + with self._graph.subgraph( + name=cluster_name, comment=label + ) as per_port: + per_port.attr(rankdir="TB") + per_port.attr(ranksep="0.02") + per_port.attr(margin="5") + per_port.attr(label=label) + self._add_flows_to_graph(per_port, flows_per_inport[inport]) + + # Create an input node that points to each input subgraph + # They are all inside an anonymous subgraph so that they can be + # alligned. + with self._graph.subgraph() as s: + s.attr(rank="same") + for inport in flows_per_inport: + # Make an Input node point to each subgraph + node_name = "input_{}".format(inport) + cluster_name = self.inport_cluster_name(inport) + s.node( + node_name, + shape="Mdiamond", + label="input port {}".format(inport), + ) + self._graph.edge( + node_name, + self.invis_node_name(cluster_name), + lhead=cluster_name, + _attributes={"weight": "20"}, + ) + + # Create the output nodes in a subgraph so that they are alligned + with self._graph.subgraph() as s: + for port in self._output_nodes: + s.attr(rank="same") + if port == "drop": + s.node( + "drop", + shape="Msquare", + color="red", + label="DROP", + rank="sink", + ) + elif port == "controller": + s.node( + "controller", + shape="Msquare", + color="blue", + label="CONTROLLER", + rank="sink", + ) + elif port == "userspace": + s.node( + "userspace", + shape="Msquare", + color="blue", + label="CONTROLLER", + rank="sink", + ) + else: + s.node( + self.output_node_name(port), + shape="Msquare", + color="green", + label="Port {}".format(port), + rank="sink", + ) + + # Print style legend + with self._graph.subgraph(name="cluster_legend") as s: + s.attr(label="Legend") + for style in self.node_styles.values(): + s.node(name=style.get("desc"), _attributes=style.get("style")) diff --git a/python/ovs/flowviz/odp/tree.py b/python/ovs/flowviz/odp/tree.py index d249e7d6d..72be1c427 100644 --- a/python/ovs/flowviz/odp/tree.py +++ b/python/ovs/flowviz/odp/tree.py @@ -76,7 +76,8 @@ class FlowTree: on recirculation ids. Args: - flows (list[ODPFlow]): Optional, initial list of flows + flows (list[ODPFlow]): Optional, initial list of flows or dictionary of + flows indexed by recirc_id root (TreeElem): Optional, root of the tree. """ @@ -84,8 +85,15 @@ class FlowTree: self._flows = {} self.root = root if flows: - for flow in flows: - self.add(flow) + if isinstance(flows, dict): + self._flows = flows + elif isinstance(flows, list): + for flow in flows: + self.add(flow) + else: + raise Exception( + "flows in wrong format: {}".format(type(flows)) + ) def add(self, flow): """Add a flow""" @@ -192,6 +200,10 @@ class FlowTree: for elem in to_remove: self.root.children.remove(elem) + def all(self): + """Return all the flows in a dictionary by recirc_id.""" + return self._flows + class ConsoleTreeProcessor(DatapathFactory, FileProcessor): def __init__(self, opts): diff --git a/python/setup.py b/python/setup.py index c734f68f3..018a75eb0 100644 --- a/python/setup.py +++ b/python/setup.py @@ -114,7 +114,7 @@ setup_args = dict( 'dns': ['unbound'], 'flow': flow_extras_require, 'flowviz': - [*flow_extras_require, 'click', 'rich'], + [*flow_extras_require, 'click', 'rich', 'graphviz'], }, scripts=["ovs/flowviz/ovs-flowviz"], data_files=["ovs/flowviz/ovs-flowviz.conf"], From patchwork Wed Mar 13 09:03:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1911597 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=VEdh9DcV; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Tvl1r5nDjz23qj for ; Wed, 13 Mar 2024 20:04:16 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 057D340800; Wed, 13 Mar 2024 09:04:14 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id T-o9vqEef_kI; Wed, 13 Mar 2024 09:04:12 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 0DB704081B Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=VEdh9DcV Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 0DB704081B; Wed, 13 Mar 2024 09:04:10 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B745CC0072; Wed, 13 Mar 2024 09:04:09 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 24855C0DCF for ; Wed, 13 Mar 2024 09:04:08 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 35BE181FAC for ; Wed, 13 Mar 2024 09:04:03 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FzxNE-IwJw5N for ; Wed, 13 Mar 2024 09:03:57 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp1.osuosl.org CD80581DEB Authentication-Results: smtp1.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org CD80581DEB Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=VEdh9DcV Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id CD80581DEB for ; Wed, 13 Mar 2024 09:03:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710320631; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VTt9w875Vozmtxt0Yf3M+jTL+vaVDeX0UQQ5pJGw494=; b=VEdh9DcVGsdwinq/2RXaz5Z0svlxxHgWXrQr3bzU4Y38ZTozfUUG99Xtl5obcLIYeCJFhV KgJYdO4RliUPvMP4NJOTq/YDmBjXpQSmKQnMcbAIFnu/NmKlUSYZepRCA/nZrE5oPH/TjR 5VjCKuvhMjx3rolnxd/ZPMTbaDJZ8lU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-426-Lh-ANwlRMNSt-ZGP5diuqg-1; Wed, 13 Mar 2024 05:03:50 -0400 X-MC-Unique: Lh-ANwlRMNSt-ZGP5diuqg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B94DE8007A4 for ; Wed, 13 Mar 2024 09:03:49 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.194.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id F2C96492BC6; Wed, 13 Mar 2024 09:03:48 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Wed, 13 Mar 2024 10:03:31 +0100 Message-ID: <20240313090334.414226-12-amorenoz@redhat.com> In-Reply-To: <20240313090334.414226-1-amorenoz@redhat.com> References: <20240313090334.414226-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v2 11/12] python: ovs: flowviz: Support html dark style. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" In order to support dark style in html outputs, allow the config file to express the background color and set it in a top style object. Acked-by: Eelco Chaudron Signed-off-by: Adrian Moreno --- python/ovs/flowviz/html_format.py | 4 +++- python/ovs/flowviz/odp/html.py | 30 ++++++++++++++++++++++++----- python/ovs/flowviz/ofp/html.py | 28 ++++++++++++++++++++++----- python/ovs/flowviz/ovs-flowviz.conf | 20 +++++++++++++++++++ 4 files changed, 71 insertions(+), 11 deletions(-) diff --git a/python/ovs/flowviz/html_format.py b/python/ovs/flowviz/html_format.py index ebfa65c34..3f3550da5 100644 --- a/python/ovs/flowviz/html_format.py +++ b/python/ovs/flowviz/html_format.py @@ -48,7 +48,9 @@ class HTMLBuffer(FlowBuffer): style = ' style="color:{}"'.format(color) if color else "" self._text += "".format(style) if href: - self._text += "".format(href) + self._text += " ".format( + href, 'style="color:{}"'.format(color) if color else "" + ) self._text += string if href: self._text += "" diff --git a/python/ovs/flowviz/odp/html.py b/python/ovs/flowviz/odp/html.py index 4aa08dc70..b2855bf40 100644 --- a/python/ovs/flowviz/odp/html.py +++ b/python/ovs/flowviz/odp/html.py @@ -55,10 +55,18 @@ class HTMLTree(FlowTree): flows(dict[int, list[DPFlow]): Optional; initial flows """ + html_body_style = """ + """ + html_header = """ """.format( + bg=bg, fg=fg + ) for name, tables in self.data.items(): name = name.replace(" ", "_") html_obj += "

{}

".format(name) html_obj += "
" for table, flows in tables.items(): - formatter = HTMLFormatter(self.opts) def anchor(x): return "#table_%s_%s" % (name, x.value["table"]) - formatter.style.set_value_style( + self.formatter.style.set_value_style( "resubmit", HTMLStyle( - formatter.style.get("value.resubmit"), + self.formatter.style.get("value.resubmit").color, anchor_gen=anchor, ), ) @@ -71,7 +89,7 @@ class HTMLProcessor(OpenFlowFactory, FileProcessor): if result: highlighted = result.kv buf = HTMLBuffer() - formatter.format_flow(buf, flow, highlighted) + self.formatter.format_flow(buf, flow, highlighted) html_obj += buf.text html_obj += "" html_obj += "" diff --git a/python/ovs/flowviz/ovs-flowviz.conf b/python/ovs/flowviz/ovs-flowviz.conf index 165c453ec..82b5b47d2 100644 --- a/python/ovs/flowviz/ovs-flowviz.conf +++ b/python/ovs/flowviz/ovs-flowviz.conf @@ -28,6 +28,8 @@ # # HTML Styles # ============== +# * PORTION: An extra portion is supported: "backgroud" which defines the +# background color of the page. # * ELEMENT: # - color: defines the color in hex format @@ -65,6 +67,23 @@ console.key.highlighted.underline = true console.value.highlighted.underline = true console.delim.highlighted.underline = true +# html +html.background.color = #23282e +html.default.color = white +html.key.color = #5D86BA +html.value.color = #B0C4DE +html.delim.color = #B0C4DE + +html.key.resubmit.color = #005f00 +html.key.recirc.color = #005f00 +html.value.resubmit.color = #005f00 +html.value.recirc.color = #005f00 +html.key.output.color = #00d700 +html.value.output.color = #00d700 +html.key.highlighted.color = #FF00FF +html.value.highlighted.color = #FF00FF +html.key.drop.color = red + [styles.light] # If a color is omitted, the default terminal color will be used @@ -99,6 +118,7 @@ console.value.highlighted.underline = true console.delim.highlighted.underline = true # html +html.background.color = white html.key.color = #00005f html.value.color = #870000 html.key.resubmit.color = #00d700 From patchwork Wed Mar 13 09:03:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1911600 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=EDNLT51R; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Tvl2N4mkNz1yWy for ; Wed, 13 Mar 2024 20:04:44 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 4F20F60A98; Wed, 13 Mar 2024 09:04:42 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id llGUCdVW85yH; Wed, 13 Mar 2024 09:04:35 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org C976C60C31 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=EDNLT51R Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id C976C60C31; Wed, 13 Mar 2024 09:04:16 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 89B07C0072; Wed, 13 Mar 2024 09:04:16 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 20EC7C0DCF for ; Wed, 13 Mar 2024 09:04:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id C0152407FA for ; Wed, 13 Mar 2024 09:03:59 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id t9c-FsEPhcis for ; Wed, 13 Mar 2024 09:03:56 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp4.osuosl.org 0600D407C8 Authentication-Results: smtp4.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 0600D407C8 Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=EDNLT51R Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 0600D407C8 for ; Wed, 13 Mar 2024 09:03:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710320632; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yBsiK4GoC2i5fBxyT6KAxcqrLgDEQsN6tSiGLu3/Gxk=; b=EDNLT51RiLmVBY81Yzl6CCZDvMtOEDtZDjqK1KkTEQ5FjSwpOJAxq37f0lFpkuF1OCvNAT mmupxVD1Mt46us9+Dx0sF6rT1MXE5W14LYHzv2EvrVEZu5gxOk7CMtyonh5z+EeuCSWsfB qXAqw3mJlFtJcvVswo0uaM3k6XHcvqY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-637-NtzcqB7IPyGcpcIUMF4h3A-1; Wed, 13 Mar 2024 05:03:50 -0400 X-MC-Unique: NtzcqB7IPyGcpcIUMF4h3A-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B5ACA85CBB6 for ; Wed, 13 Mar 2024 09:03:50 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.194.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0319A492BC6; Wed, 13 Mar 2024 09:03:49 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Wed, 13 Mar 2024 10:03:32 +0100 Message-ID: <20240313090334.414226-13-amorenoz@redhat.com> In-Reply-To: <20240313090334.414226-1-amorenoz@redhat.com> References: <20240313090334.414226-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v2 12/12] documentation: Document ovs-flowviz. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add a man page for ovs-flowviz as well as a topic page with some more detailed examples. Signed-off-by: Adrian Moreno --- Documentation/automake.mk | 4 +- Documentation/conf.py | 2 + Documentation/ref/index.rst | 1 + Documentation/ref/ovs-flowviz.8.rst | 531 ++++++++++++++++++++ Documentation/topics/flow-visualization.rst | 271 ++++++++++ Documentation/topics/index.rst | 1 + 6 files changed, 809 insertions(+), 1 deletion(-) create mode 100644 Documentation/ref/ovs-flowviz.8.rst create mode 100644 Documentation/topics/flow-visualization.rst diff --git a/Documentation/automake.mk b/Documentation/automake.mk index 47d2e336a..539870aa2 100644 --- a/Documentation/automake.mk +++ b/Documentation/automake.mk @@ -45,7 +45,7 @@ DOC_SOURCE = \ Documentation/topics/fuzzing/ovs-fuzzing-infrastructure.rst \ Documentation/topics/fuzzing/ovs-fuzzers.rst \ Documentation/topics/fuzzing/security-analysis-of-ovs-fuzzers.rst \ - Documentation/topics/testing.rst \ + Documentation/topics/flow-visualization.rst \ Documentation/topics/integration.rst \ Documentation/topics/language-bindings.rst \ Documentation/topics/networking-namespaces.rst \ @@ -55,6 +55,7 @@ DOC_SOURCE = \ Documentation/topics/ovsdb-replication.rst \ Documentation/topics/porting.rst \ Documentation/topics/record-replay.rst \ + Documentation/topics/testing.rst \ Documentation/topics/tracing.rst \ Documentation/topics/usdt-probes.rst \ Documentation/topics/userspace-checksum-offloading.rst \ @@ -162,6 +163,7 @@ RST_MANPAGES = \ ovs-actions.7.rst \ ovs-appctl.8.rst \ ovs-ctl.8.rst \ + ovs-flowviz.8.rst \ ovs-l3ping.8.rst \ ovs-parse-backtrace.8.rst \ ovs-pki.8.rst \ diff --git a/Documentation/conf.py b/Documentation/conf.py index 085ca2cd6..e41cf6031 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -120,6 +120,8 @@ _man_pages = [ u'utility for configuring running Open vSwitch daemons'), ('ovs-ctl.8', u'OVS startup helper script'), + ('ovs-flowviz.8', + u'utility for visualizing OpenFlow and datapath flows'), ('ovs-l3ping.8', u'check network deployment for L3 tunneling problems'), ('ovs-parse-backtrace.8', diff --git a/Documentation/ref/index.rst b/Documentation/ref/index.rst index 03ada932f..7f2fe6177 100644 --- a/Documentation/ref/index.rst +++ b/Documentation/ref/index.rst @@ -42,6 +42,7 @@ time: ovs-actions.7 ovs-appctl.8 ovs-ctl.8 + ovs-flowviz.8 ovs-l3ping.8 ovs-pki.8 ovs-sim.1 diff --git a/Documentation/ref/ovs-flowviz.8.rst b/Documentation/ref/ovs-flowviz.8.rst new file mode 100644 index 000000000..da1135918 --- /dev/null +++ b/Documentation/ref/ovs-flowviz.8.rst @@ -0,0 +1,531 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +=========== +ovs-flowviz +=========== + +Synopsis +======== + +``ovs-flowviz`` +[``[-i | --input] <[alias,]file>``] +[``[-c | --config] ``] +[``[-f | --filter] ``] +[``[-h | --highlight] ``] +[``--style