From patchwork Tue May 7 14:30:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Adri=C3=A1n_Moreno?= X-Patchwork-Id: 1932498 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Ut/TO6b9; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VYggf4vhmz1xnT for ; Wed, 8 May 2024 00:31:10 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 986FE60A4D; Tue, 7 May 2024 14:31:05 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id m07wDR29xJkC; Tue, 7 May 2024 14:31:03 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 69D2260747 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Ut/TO6b9 Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 69D2260747; Tue, 7 May 2024 14:31:03 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4A63CC0077; Tue, 7 May 2024 14:31:03 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 00747C0037 for ; Tue, 7 May 2024 14:31:02 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id D70EF6078F for ; Tue, 7 May 2024 14:31:01 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 9yswRvtA1YXw for ; Tue, 7 May 2024 14:31:01 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amorenoz@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org CFD1A60747 Authentication-Results: smtp3.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org CFD1A60747 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id CFD1A60747 for ; Tue, 7 May 2024 14:31:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1715092259; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=aMuc24CwzUUgFzfvLwt+QhO3BQEb1f+ZUdlKBVx8T/M=; b=Ut/TO6b9YpXP8joRS8t52ij/BMGPxpKO/h+hlffiarKL6v0Q8PzdY92ttqnoqKV9Jq3P8y IoCsp5MmP2g+u3TPitWu6BY9fMMIxQIUfzK3waBqJ60vQL027pikTVY5ZK8d4cUpsNMDLm 4XrzyqE+/JvhUSVEtS+ZDKKuzw0tpTE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-190-ZJ6WQ75yPwGYIaghs3C2tg-1; Tue, 07 May 2024 10:30:56 -0400 X-MC-Unique: ZJ6WQ75yPwGYIaghs3C2tg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 515E4857A81 for ; Tue, 7 May 2024 14:30:56 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.88]) by smtp.corp.redhat.com (Postfix) with ESMTP id 79D1C3C27; Tue, 7 May 2024 14:30:55 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Tue, 7 May 2024 16:30:26 +0200 Message-ID: <20240507143044.3056232-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v4 00/12] Add flow visualization utility. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The goal of this utility is to read both datapath and Openflow flows (using the flow library already available) and print them in different formats and styles to make it easier to understand them and troubleshoot issues. The formats are quite opinionated and so are the colors chosen so I'm eager to hear what is the impression caused to the community. Here is a summary of the formats and features supported: - Openflow - Console: Prints flows back to the console allowing filtering and extensive formatting. - Cookie: Arranges flows based on cookie (instead of table) to ease visualization of OVN-generated flows. - HTML: Prints an HTML file that shows the flows arranged by tables. resubmit actions have a hyperlinke to the target table to ease navegation. - Logic: Many times OVN generates many "logically-equivalent" flows. These are flows that have the same structure: match on the same values and have the same actions. The only thing that changes is the actual value they match against and maybe the arguments of the actions. This format collapses these flows so you can visualize the logical pipeline easier. - JSON: JSON format. More OpenFlow features: - OVN metadata: Both the "cookie" and the "logic" format allow to connect to a running OVN NB/SB databases and enrich the flows with OVN context based on the flow's cookie. - Datapath: - Console: Prints flows back to the console allowing filtering and extensive formatting. - Tree: Datapath flows use recirculation to further execute additional actions. This format builds a tree of flows following the recirculation identifiers and prints it in the console. - HTML: Prints the datapath flow tree in HTML. It includes some minimal JS to support expanding and collapsing of entire branches. - Graph: Following the "tree" format, this one prints the tree in graphviz format. - JSON: JSON format. Additional datapath features: - Many datapath formats are based on the tree flow hierarchy. An interesting feature of this structure is that filtering is done at the branch level. This means that when a flow satisfies the filter, the entire sub-tree leading to that flow is shown. Additional common features: - Styling: Styles for both console and HTML formats can be defined using a configuration file. - Heat maps: Some formats support heat-maps. A color pallete ranging from blue (cold) to red (hot) is used to print the number of packets and bytes of the flows. That way, the flows that are handling more traffic are much easier to spot --- V3 -> V4: - Add manpage to rpm package - Fix Eelco's comments V2 -> V3: - Fix grammar thanks to Eelco's review V1 -> V2: - Fix typos and nits on documentation - Split documentation in two: ovs-flowviz.8 man page and a topic article with more detailed examples. RFC -> V1: - Addressed Eelco's comments - Added a documentation page - Added support for dark style HTML pages - Patch 3. Small fix in the way a style is looked up. Use the key in the KV instead of the metadata string. This helps with "free" values such as "output". Adrian Moreno (12): python: ovs: Add flowviz scheleton. python: ovs: flowviz: Add file processing infra. python: ovs: flowviz: Add console formatting. python: ovs: flowviz: Add default config file. python: ovs: flowviz: Add html formatting. python: ovs: flowviz: Add datapath tree format. python: ovs: flowviz: Add OpenFlow logical view. python: ovs: flowviz: Add Openflow cookie format. python: ovs: flowviz: Add datapath html format. python: ovs: flowviz: Add datapath graph format. python: ovs: flowviz: Support html dark style. documentation: Document ovs-flowviz. Documentation/automake.mk | 4 +- Documentation/conf.py | 2 + Documentation/ref/index.rst | 1 + Documentation/ref/ovs-flowviz.8.rst | 531 ++++++++++++++++++++ Documentation/topics/flow-visualization.rst | 271 ++++++++++ Documentation/topics/index.rst | 1 + python/automake.mk | 32 +- python/ovs/flowviz/__init__.py | 2 + python/ovs/flowviz/console.py | 196 ++++++++ python/ovs/flowviz/format.py | 371 ++++++++++++++ python/ovs/flowviz/html_format.py | 138 +++++ python/ovs/flowviz/main.py | 196 ++++++++ python/ovs/flowviz/odp/__init__.py | 0 python/ovs/flowviz/odp/cli.py | 116 +++++ python/ovs/flowviz/odp/graph.py | 418 +++++++++++++++ python/ovs/flowviz/odp/html.py | 279 ++++++++++ python/ovs/flowviz/odp/tree.py | 303 +++++++++++ python/ovs/flowviz/ofp/__init__.py | 0 python/ovs/flowviz/ofp/cli.py | 246 +++++++++ python/ovs/flowviz/ofp/html.py | 98 ++++ python/ovs/flowviz/ofp/logic.py | 364 ++++++++++++++ python/ovs/flowviz/ovs-flowviz | 20 + python/ovs/flowviz/ovs-flowviz.conf | 128 +++++ python/ovs/flowviz/process.py | 263 ++++++++++ python/setup.py | 14 +- rhel/openvswitch-fedora.spec.in | 1 + rhel/openvswitch.spec.in | 1 + 27 files changed, 3986 insertions(+), 10 deletions(-) create mode 100644 Documentation/ref/ovs-flowviz.8.rst create mode 100644 Documentation/topics/flow-visualization.rst create mode 100644 python/ovs/flowviz/__init__.py create mode 100644 python/ovs/flowviz/console.py create mode 100644 python/ovs/flowviz/format.py create mode 100644 python/ovs/flowviz/html_format.py create mode 100644 python/ovs/flowviz/main.py create mode 100644 python/ovs/flowviz/odp/__init__.py create mode 100644 python/ovs/flowviz/odp/cli.py create mode 100644 python/ovs/flowviz/odp/graph.py create mode 100644 python/ovs/flowviz/odp/html.py create mode 100644 python/ovs/flowviz/odp/tree.py create mode 100644 python/ovs/flowviz/ofp/__init__.py create mode 100644 python/ovs/flowviz/ofp/cli.py create mode 100644 python/ovs/flowviz/ofp/html.py create mode 100644 python/ovs/flowviz/ofp/logic.py create mode 100755 python/ovs/flowviz/ovs-flowviz create mode 100644 python/ovs/flowviz/ovs-flowviz.conf create mode 100644 python/ovs/flowviz/process.py