[ovs-dev,v2,03/12] python: ovs: flowviz: Add console formatting.

Add a flow formatting framework and one implementation for console
printing using rich.

The flow formatting framework is a simple set of classes that can be
used to write different flow formatting implementations. It supports
styles to be described by any class, highlighting and config-file based
style definition.

The first flow formatting implementation is also introduced: the
ConsoleFormatter. It uses the an advanced rich-text printing library
[1].

The console printing supports:
- Heatmap: printing the packet/byte statistics of each flow in a color
  that represents its relative size: blue (low) -> red (high).
- Printing a banner with the file name and alias.
- Extensive style definition via config file.

This console format is added to both OpenFlow and Datapath flows.

Examples:
- Highlight drops in datapath flows:
$ ovs-flowviz -i flows.txt --highlight "drop" datapath console
- Quickly detect where most packets are going using heatmap and
  paginated output:
$ ovs-ofctl dump-flows br-int | ovs-flowviz openflow console -h

[1] https://rich.readthedocs.io/en/stable/introduction.html

Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
---
 python/automake.mk            |   2 +
 python/ovs/flowviz/console.py | 175 ++++++++++++++++
 python/ovs/flowviz/format.py  | 371 ++++++++++++++++++++++++++++++++++
 python/ovs/flowviz/main.py    |  58 +++++-
 python/ovs/flowviz/odp/cli.py |  25 +++
 python/ovs/flowviz/ofp/cli.py |  26 +++
 python/ovs/flowviz/process.py |  83 +++++++-
 python/setup.py               |   4 +-
 8 files changed, 736 insertions(+), 8 deletions(-)
 create mode 100644 python/ovs/flowviz/console.py
 create mode 100644 python/ovs/flowviz/format.py

On 13 Mar 2024, at 10:03, Adrian Moreno wrote:

> Add a flow formatting framework and one implementation for console
> printing using rich.
>
> The flow formatting framework is a simple set of classes that can be
> used to write different flow formatting implementations. It supports
> styles to be described by any class, highlighting and config-file based
> style definition.
>
> The first flow formatting implementation is also introduced: the
> ConsoleFormatter. It uses the an advanced rich-text printing library
> [1].
>
> The console printing supports:
> - Heatmap: printing the packet/byte statistics of each flow in a color
>   that represents its relative size: blue (low) -> red (high).
> - Printing a banner with the file name and alias.
> - Extensive style definition via config file.
>
> This console format is added to both OpenFlow and Datapath flows.
>
> Examples:
> - Highlight drops in datapath flows:
> $ ovs-flowviz -i flows.txt --highlight "drop" datapath console
> - Quickly detect where most packets are going using heatmap and
>   paginated output:
> $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow console -h
>
> [1] https://rich.readthedocs.io/en/stable/introduction.html
>
> Signed-off-by: Adrian Moreno <amorenoz@redhat.com>

Thanks for making these changes, one small nit. Guess your cursor was at a different place than you thought it was :)

If this is the only change in your next rev, add my ‘Acked-by: Eelco Chaudron <echaudro@redhat.com>’.

//Eelco

> ---
>  python/automake.mk            |   2 +
>  python/ovs/flowviz/console.py | 175 ++++++++++++++++
>  python/ovs/flowviz/format.py  | 371 ++++++++++++++++++++++++++++++++++
>  python/ovs/flowviz/main.py    |  58 +++++-
>  python/ovs/flowviz/odp/cli.py |  25 +++
>  python/ovs/flowviz/ofp/cli.py |  26 +++
>  python/ovs/flowviz/process.py |  83 +++++++-
>  python/setup.py               |   4 +-
>  8 files changed, 736 insertions(+), 8 deletions(-)
>  create mode 100644 python/ovs/flowviz/console.py
>  create mode 100644 python/ovs/flowviz/format.py
>
> diff --git a/python/automake.mk b/python/automake.mk
> index fd5e74081..bd53c5405 100644
> --- a/python/automake.mk
> +++ b/python/automake.mk
> @@ -65,6 +65,8 @@ ovs_pytests = \
>
>  ovs_flowviz = \
>  	python/ovs/flowviz/__init__.py \
> +	python/ovs/flowviz/console.py \
> +	python/ovs/flowviz/format.py \
>  	python/ovs/flowviz/main.py \
>  	python/ovs/flowviz/odp/__init__.py \
>  	python/ovs/flowviz/odp/cli.py \
> diff --git a/python/ovs/flowviz/console.py b/python/ovs/flowviz/console.py
> new file mode 100644
> index 000000000..4a3443360
> --- /dev/null
> +++ b/python/ovs/flowviz/console.py
> @@ -0,0 +1,175 @@
> +# Copyright (c) 2023 Red Hat, Inc.
> +#
> +# Licensed under the Apache License, Version 2.0 (the "License");
> +# you may not use this file except in compliance with the License.
> +# You may obtain a copy of the License at:
> +#
> +#     http://www.apache.org/licenses/LICENSE-2.0
> +#
> +# Unless required by applicable law or agreed to in writing, software
> +# distributed under the License is distributed on an "AS IS" BASIS,
> +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> +# See the License for the specific language governing permissions and
> +# limitations under the License.
> +
> +import colorsys
> +
> +from rich.console import Console
> +from rich.color import Color
> +from rich.emoji import Emoji
> +from rich.panel import Panel
> +from rich.text import Text
> +from rich.style import Style
> +
> +from ovs.flowviz.format import FlowFormatter, FlowBuffer, FlowStyle
> +
> +
> +def file_header(name):
> +    return Panel(
> +        Text(
> +            Emoji.replace(":scroll:")
> +            + " "
> +            + name
> +            + " "
> +            + Emoji.replace(":scroll:"),
> +            style="bold",
> +            justify="center",
> +        )
> +    )
> +
> +
> +class ConsoleBuffer(FlowBuffer):
> +    """ConsoleBuffer implements FlowBuffer to provide console-based text
> +    formatting based on rich.Text.
> +
> +    Append functions accept a rich.Style.
> +
> +    Args:
> +        rtext(rich.Text): Optional; text instance to reuse
> +    """
> +
> +    def __init__(self, rtext):
> +        self._text = rtext or Text()
> +
> +    @property
> +    def text(self):
> +        return self._text
> +
> +    def _append(self, string, style):
> +        """Append to internal text."""
> +        return self._text.append(string, style)
> +
> +    def append_key(self, kv, style):
> +        """Append a key.
> +        Args:
> +            kv (KeyValue): the KeyValue instance to append
> +            style (rich.Style): the style to use
> +        """
> +        return self._append(kv.meta.kstring, style)
> +
> +    def append_delim(self, kv, style):
> +        """Append a delimiter.
> +        Args:
> +            kv (KeyValue): the KeyValue instance to append
> +            style (rich.Style): the style to use
> +        """
> +        return self._append(kv.meta.delim, style)
> +
> +    def append_end_delim(self, kv, style):
> +        """Append an end delimiter.
> +        Args:
> +            kv (KeyValue): the KeyValue instance to append
> +            style (rich.Style): the style to use
> +        """
> +        return self._append(kv.meta.end_delim, style)
> +
> +    def append_value(self, kv, style):
> +        """Append a value.
> +        Args:
> +            kv (KeyValue): the KeyValue instance to append
> +            style (rich.Style): the style to use
> +        """
> +        return self._append(kv.meta.vstring, style)
> +
> +    def append_extra(self, extra, style):
> +        """Append extra string.
> +        Args:
> +            kv (KeyValue): the KeyValue instance to append
> +            style (rich.Style): the style to use
> +        """
> +        return self._append(extra, style)
> +
> +
> +class ConsoleFormatter(FlowFormatter):
> +    """ConsoleFormatter is a FlowFormatter that formats flows into the console
> +    using rich.Console.
> +
> +    Args:
> +        console (rich.Console): Optional, an existing console to use
> +        max_value_len (int): Optional; max length of the printed values
> +        kwargs (dict): Optional; Extra arguments to be passed down to
> +            rich.console.Console()
> +    """
> +
> +    def __init__(self, opts=None, console=None, **kwargs):
> +        super(ConsoleFormatter, self).__init__()
> +        style = self.style_from_opts(opts)
> +        self.console = console or Console(color_system="256", **kwargs)
> +        self.style = style or FlowStyle()
> +
> +    def style_from_opts(self, opts):
> +        return self._style_from_opts(opts, "console", Style)
> +
> +    def print_flow(self, flow, highlighted=None):
> +        """Prints a flow to the console.
> +
> +        Args:
> +            flow (ovs_dbg.OFPFlow): the flow to print
> +            style (dict): Optional; style dictionary to use
> +            highlighted (list): Optional; list of KeyValues to highlight
> +        """
> +
> +        buf = ConsoleBuffer(Text())
> +        self.format_flow(buf, flow, highlighted)
> +        self.console.print(buf.text)
> +
> +    def format_flow(self, buf, flow, highlighted=None):
> +        """Formats the flow into the provided buffer as a rich.Text.
> +
> +        Args:
> +            buf (FlowBuffer): the flow buffer to append to
> +            flow (ovs_dbg.OFPFlow): the flow to format
> +            style (FlowStyle): Optional; style object to use
> +            highlighted (list): Optional; list of KeyValues to highlight
> +        """
> +        return super(ConsoleFormatter, self).format_flow(
> +            buf, flow, self.style, highlighted
> +        )
> +
> +
> +def heat_pallete(min_value, max_value):
> +    """Generates a color pallete based on the 5-color heat pallete so that
> +    for each value between min and max a color is returned that represents it's
> +    relative size.
> +    Args:
> +        min_value (int): minimum value
> +        max_value (int) maximum value
> +    """
> +    h_min = 0  # red
> +    h_max = 220 / 360  # blue
> +
> +    def heat(value):
> +        if max_value == min_value:
> +            r, g, b = colorsys.hsv_to_rgb(h_max / 2, 1.0, 1.0)
> +        else:
> +            normalized = (int(value) - min_value) / (max_value - min_value)
> +            hue = ((1 - normalized) + h_min) * (h_max - h_min)
> +            r, g, b = colorsys.hsv_to_rgb(hue, 1.0, 1.0)
> +        return Style(color=Color.from_rgb(r * 255, g * 255, b * 255))
> +
> +    return heat
> +
> +
> +def default_highlight():
> +    """Generates a default style for highlights."""
> +    return Style(underline=True)
> diff --git a/python/ovs/flowviz/format.py b/python/ovs/flowviz/format.py
> new file mode 100644
> index 000000000..70af2fa26
> --- /dev/null
> +++ b/python/ovs/flowviz/format.py
> @@ -0,0 +1,371 @@
> +# Copyright (c) 2023 Red Hat, Inc.
> +#
> +# Licensed under the Apache License, Version 2.0 (the "License");
> +# you may not use this file except in compliance with the License.
> +# You may obtain a copy of the License at:
> +#
> +#     http://www.apache.org/licenses/LICENSE-2.0
> +#
> +# Unless required by applicable law or agreed to in writing, software
> +# distributed under the License is distributed on an "AS IS" BASIS,
> +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> +# See the License for the specific language governing permissions and
> +# limitations under the License.
> +
> +"""Flow formatting framework.
> +
> +This file defines a simple flow formatting framework. It's comprised of 3
> +classes: FlowStyle, FlowFormatter and FlowBuffer.
> +
> +The FlowStyle arranges opaque style objects in a dictionary that can be queried
> +to determine what style a particular key-value should be formatted with.
> +That way, a particular implementation can represent its style using their own
> +object.
> +
> +The FlowBuffer is an abstract class and must be derived by particular
> +implementations. It should know how to append parts of a flow using a style.
> +Only here the type of the style is relevant.
> +
> +When asked to format a flow, the FlowFormatter will determine which style
> +the flow must be formatted with and call FlowBuffer functions with each part
> +of the flow and their corresponding style.
> +"""
> +
> +
> +class FlowStyle:
> +    """A FlowStyle determines the KVStyle to use for each key value in a flow.
> +
> +    Styles are internally represented by a dictionary.
> +    In order to determine the style for a "key", the following items in the
> +    dictionary are fetched:
> +        - key.highlighted.{key} (if key is found in hightlighted)
> +        - key.highlighted (if key is found in hightlighted)
> +        - key.{key}
> +        - key
> +        - default
> +
> +    In order to determine the style for a "value", the following items in the
> +    dictionary are fetched:
> +        - value.highlighted.{key} (if key is found in hightlighted)
> +        - value.highlighted.type{value.__class__.__name__}
> +        - value.highlighted
> +        (if key is found in hightlighted)
> +        - value.{key}
> +        - value.type.{value.__class__.__name__}
> +        - value
> +        - default
> +
> +    The actual type of the style object stored for each item above is opaque
> +    to this class and it depends on the particular FlowFormatter child class
> +    that will handle them. Even callables can be stored, if so they will be
> +    called with the value of the field that is to be formatted and the return
> +    object will be used as style.
> +
> +    Additionally, the following style items can be defined:
> +        - delim: for delimiters
> +        - delim.highlighted: for delimiters of highlighted key-values
> +    """
> +
> +    def __init__(self, initial=None):
> +        self._styles = initial if initial is not None else dict()
> +
> +    def __len__(self):
> +        return len(self._styles)
> +
> +    def set_flag_style(self, kvstyle):
> +        self._styles["flag"] = kvstyle
> +
> +    def set_delim_style(self, kvstyle, highlighted=False):
> +        if highlighted:
> +            self._styles["delim.highlighted"] = kvstyle
> +        else:
> +            self._styles["delim"] = kvstyle
> +
> +    def set_default_key_style(self, kvstyle, highlighted=False):
> +        if highlighted:
> +            self._styles["key.highlighted"] = kvstyle
> +        else:
> +            self._styles["key"] = kvstyle
> +
> +    def set_default_value_style(self, kvstyle, highlighted=False):
> +        if highlighted:
> +            self._styles["value.highlighted"] = kvstyle
> +        else:
> +            self._styles["value"] = kvstyle
> +
> +    def set_key_style(self, key, kvstyle, highlighted=False):
> +        if highlighted:
> +            self._styles["key.highlighted.{}".format(key)] = kvstyle
> +        else:
> +            self._styles["key.{}".format(key)] = kvstyle
> +
> +    def set_value_style(self, key, kvstyle, highlighted=None):
> +        if highlighted:
> +            self._styles["value.highlighted.{}".format(key)] = kvstyle
> +        else:
> +            self._styles["value.{}".format(key)] = kvstyle
> +
> +    def set_value_type_style(self, name, kvstyle, highlighted=None):
> +        if highlighted:
> +            self._styles["value.highlighted.type.{}".format(name)] = kvstyle
> +        else:
> +            self._styles["value.type.{}".format(name)] = kvstyle
> +
> +    def get(self, key):
> +        return self._styles.get(key)
> +
> +    def get_delim_style(self, highlighted=False):
> +        delim_style_lookup = ["delim.highlighted"] if highlighted else []
> +        delim_style_lookup.extend(["delim", "default"])
> +        return next(
> +            (
> +                self._styles.get(s)
> +                for s in delim_style_lookup
> +                if self._styles.get(s)
> +            ),
> +            None,
> +        )
> +
> +    def get_flag_style(self):
> +        return self._styles.get("flag") or self._styles.get("default")
> +
> +    def get_key_style(self, kv, highlighted=False):
> +        key = kv.key
> +
> +        key_style_lookup = (
> +            ["key.highlighted.%s" % key, "key.highlighted"]
> +            if highlighted
> +            else []
> +        )
> +        key_style_lookup.extend(["key.%s" % key, "key", "default"])
> +
> +        style = next(
> +            (
> +                self._styles.get(s)
> +                for s in key_style_lookup
> +                if self._styles.get(s)
> +            ),
> +            None,
> +        )
> +        if callable(style):
> +            return style(kv.meta.kstring)
> +        return style
> +
> +    def get_value_style(self, kv, highlighted=False):
> +        key = kv.key
> +        value_type = kv.value.__class__.__name__.lower()
> +        value_style_lookup = (
> +            [
> +                "value.highlighted.%s" % key,
> +                "value.highlighted.type.%s" % value_type,
> +                "value.highlighted",
> +            ]
> +            if highlighted
> +            else []
> +        )
> +        value_style_lookup.extend(
> +            [
> +                "value.%s" % key,
> +                "value.type.%s" % value_type,
> +                "value",
> +                "default",
> +            ]
> +        )
> +
> +        style = next(
> +            (
> +                self._styles.get(s)
> +                for s in value_style_lookup
> +                if self._styles.get(s)
> +            ),
> +            None,
> +        )
> +        if callable(style):
> +            return style(kv.meta.vstring)
> +        return style
> +
> +
> +class FlowFormatter:
> +    """FlowFormatter is a base class for Flow Formatters."""
> +
> +    def __init__(self):
> +        self._highlighted = list()
> +
> +    def _style_from_opts(self, opts, opts_key, style_constructor):
> +        """Create style object from options.
> +
> +        Args:
> +            opts (dict): Options dictionary
> +            opts_key (str): The options style key to extract
> +                (e.g: console or html)
> +            style_constructor(callable): A callable that creates a derived
> +                style object
> +        """
> +        if not opts or not opts.get("style"):
> +            return None
> +
> +        section_name = ".".join(["styles", opts.get("style")])
> +        if section_name not in opts.get("config").sections():
> +            return None
> +
> +        config = opts.get("config")[section_name]
> +        style = {}
> +        for key in config:
> +            (_, console, style_full_key) = key.partition(opts_key + ".")
> +            if not console:
> +                continue
> +
> +            (style_key, _, prop) = style_full_key.rpartition(".")
> +            if not prop or not style_key:
> +                raise Exception("malformed style config: {}".format(key))
> +
> +            if not style.get(style_key):
> +                style[style_key] = {}
> +            style[style_key][prop] = config[key]
> +
> +        return FlowStyle({k: style_constructor(**v) for k, v in style.items()})
> +
> +    def format_flow(self, buf, flow, style_obj=None, highlighted=None):
> +        """Formats the flow into the provided buffer.
> +
> +        Args:
> +            buf (FlowBuffer): the flow buffer to append to
> +            flow (ovs_dbg.OFPFlow): the flow to format
> +            style_obj (FlowStyle): Optional; style to use
> +            highlighted (list): Optional; list of KeyValues to highlight
> +        """
> +        last_printed_pos = 0
> +
> +        if style_obj:
> +            style_obj = style_obj or FlowStyle()
> +            for section in sorted(flow.sections, key=lambda x: x.pos):
> +                buf.append_extra(
> +                    flow.orig[last_printed_pos : section.pos],
> +                    style=style_obj.get("default"),
> +                )
> +                self.format_kv_list(
> +                    buf, section.data, section.string, style_obj, highlighted
> +                )
> +                last_printed_pos = section.pos + len(section.string)
> +        else:
> +            # Don't pay the cost of formatting each section one by one.
> +            buf.append_extra(flow.orig.strip(), None)
> +
> +    def format_kv_list(self, buf, kv_list, full_str, style_obj, highlighted):
> +        """Format a KeyValue List.
> +
> +        Args:
> +            buf (FlowBuffer): a FlowBuffer to append formatted KeyValues to
> +            kv_list (list[KeyValue]: the KeyValue list to format
> +            full_str (str): the full string containing all k-v
> +            style_obj (FlowStyle): a FlowStyle object to use
> +            highlighted (list): Optional; list of KeyValues to highlight
> +        """
> +        for i, kv in enumerate(kv_list):
> +            written = self.format_kv(
> +                buf, kv, style_obj=style_obj, highlighted=highlighted
> +            )
> +
> +            end = (
> +                kv_list[i + 1].meta.kpos
> +                if i < (len(kv_list) - 1)
> +                else len(full_str)
> +            )
> +
> +            buf.append_extra(
> +                full_str[(kv.meta.kpos + written) : end].rstrip("\n\r"),
> +                style=style_obj.get("default"),
> +            )
> +
> +    def format_kv(self, buf, kv, style_obj, highlighted=None):
> +        """Format a KeyValue
> +
> +        A formatted keyvalue has the following parts:
> +            {key}{delim}{value}[{delim}]
> +
> +        Args:
> +            buf (FlowBuffer): buffer to append the KeyValue to
> +            kv (KeyValue): The KeyValue to print
> +            style_obj (FlowStyle): The style object to use
> +            highlighted (list): Optional; list of KeyValues to highlight
> +
> +        Returns the number of printed characters.
> +        """
> +        ret = 0
> +        key = kv.meta.kstring
> +        is_highlighted = (
> +            key in [k.key for k in highlighted] if highlighted else False
> +        )
> +
> +        key_style = style_obj.get_key_style(kv, is_highlighted)
> +        buf.append_key(kv, key_style)  # format value
> +        ret += len(key)
> +
> +        if not kv.meta.vstring:
> +            return ret
> +
> +        if kv.meta.delim not in ("\n", "\t", "\r", ""):
> +            buf.append_delim(kv, style_obj.get_delim_style(is_highlighted))
> +            ret += len(kv.meta.delim)
> +
> +        value_style = style_obj.get_value_style(kv, is_highlighted)
> +        buf.append_value(kv, value_style)  # format value
> +        ret += len(kv.meta.vstring)
> +
> +        if kv.meta.end_delim:
> +            buf.append_end_delim(kv, style_obj.get_delim_style(is_highlighted))
> +            ret += len(kv.meta.end_delim)
> +
> +        return ret
> +
> +
> +class FlowBuffer:
> +    """A FlowBuffer is a base class for format buffers.
> +
> +    Childs must implement the following methods:
> +        append_key(self, kv, style)
> +        append_value(self, kv, style)
> +        append_delim(self, delim, style)
> +        append_end_delim(self, delim, style)
> +        append_extra(self, extra, style)
> +    """
> +
> +    def append_key(self, kv, style):
> +        """Append a key.
> +        Args:
> +            kv (KeyValue): the KeyValue instance to append
> +            style (Any): the style to use
> +        """
> +        raise NotImplementedError
> +
> +    def append_delim(self, kv, style):
> +        """Append a delimiter.
> +        Args:
> +            kv (KeyValue): the KeyValue instance to append
> +            style (Any): the style to use
> +        """
> +        raise NotImplementedError
> +
> +    def append_end_delim(self, kv, style):
> +        """Append an end delimiter.
> +        Args:
> +            kv (KeyValue): the KeyValue instance to append
> +            style (Any): the style to use
> +        """
> +        raise NotImplementedError
> +
> +    def append_value(self, kv, style):
> +        """Append a value.
> +        Args:
> +            kv (KeyValue): the KeyValue instance to append
> +            style (Any): the style to use
> +        """
> +        raise NotImplementedError
> +
> +    def append_extra(self, extra, style):
> +        """Append extra string.
> +        Args:
> +            kv (KeyValue): the KeyValue instance to append
> +            style (Any): the style to use
> +        """
> +        raise NotImplementedError
> diff --git a/python/ovs/flowviz/main.py b/python/ovs/flowviz/main.py
> index 64b0e8a0a..723c71fa7 100644
> --- a/python/ovs/flowviz/main.py
> +++ b/python/ovs/flowviz/main.py
> @@ -12,10 +12,30 @@
>  # See the License for the specific language governing permissions and
>  # limitations under the License.
>
> +import configparser
>  import click
>  import os
>
>  from ovs.flow.filter import OFFilter
> +from ovs.dirs import PKGDATADIR
> +
> +_default_config_file = "ovs-flowviz.conf"
> +_default_config_path = next(
> +    (
> +        p
> +        for p in [
> +            os.path.join(
> +                os.getenv("HOME"), ".config", "ovs", _default_config_file
> +            ),
> +            os.path.join(PKGDATADIR, _default_config_file),
> +            os.path.abspath(
> +                os.path.join(os.path.dirname(__file__), _default_config_file)
> +            ),
> +        ]
> +        if os.path.exists(p)
> +    ),
> +    "",
> +)
>
>
>  class Options(dict):
> @@ -48,6 +68,20 @@ def validate_input(ctx, param, value):
>  @click.group(
>      context_settings=dict(help_option_names=["-h", "--help"]),
>  )
> +@click.option(
> +    "-c",
> +    "--config",
> +    help="Use config file",
> +    type=click.Path(),
> +    default=_default_config_path,
> +    show_default=True,
> +)
> +@click.option(
> +    "--style",
> +    help="Select style (defined in config file)",
> +    default=None,
> +    show_default=True,
> +)
>  @click.option(
>      "-i",
>      "--input",
> @@ -69,8 +103,17 @@ def validate_input(ctx, param, value):
>      type=str,
>      show_default=False,
>  )
> +@click.option(
> +    "-l",
> +    "--highlight",
> +    help="Highlight flows that match the filter expression."
> +    "Run 'ovs-flowviz filter' for a detailed description of the filtering "
> +    "syntax",
> +    type=str,
> +    show_default=False,
> +)
>  @click.pass_context
> -def maincli(ctx, filename, filter):
> +def maincli(ctx, config, style, filename, filter, highlight):
>      """
>      OpenvSwitch flow visualization utility.
>
> @@ -86,6 +129,19 @@ def maincli(ctx, filename, filter):
>          except Exception as e:
>              raise click.BadParameter("Wrong filter syntax: {}".format(e))
>
> +    if highlight:
> +        try:
> +            ctx.obj["highlight"] = OFFilter(highlight)
> +        except Exception as e:
> +            raise click.BadParameter("Wrong filter syntax: {}".format(e))
> +
> +    config_file = config or _default_config_path
> +    parser = configparser.ConfigParser()
> +    parser.read(config_file)
> +
> +    ctx.obj["config"] = parser
> +    ctx.obj["style"] = style
> +
>
>  @maincli.command(hidden=True)
>  @click.pass_context
> diff --git a/python/ovs/flowviz/odp/cli.py b/python/ovs/flowviz/odp/cli.py
> index ed2f82065..a1cba0135 100644
> --- a/python/ovs/flowviz/odp/cli.py
> +++ b/python/ovs/flowviz/odp/cli.py
> @@ -16,6 +16,7 @@ import click
>
>  from ovs.flowviz.main import maincli
>  from ovs.flowviz.process import (
> +    ConsoleProcessor,
>      DatapathFactory,
>      JSONProcessor,
>  )
> @@ -40,3 +41,27 @@ def json(opts):
>      proc = JSONPrint(opts)
>      proc.process()
>      print(proc.json_string())
> +
> +
> +class DPConsoleProcessor(DatapathFactory, ConsoleProcessor):
> +    def __init__(self, opts, heat_map):
> +        super().__init__(opts, heat_map)
> +
> +
> +@datapath.command()
> +@click.option(
> +    "-h",
> +    "--heat-map",
> +    is_flag=True,
> +    default=False,
> +    show_default=True,
> +    help="Create heat-map with packet and byte counters",
> +)
> +@click.pass_obj
> +def console(opts, heat_map):
> +    """Print the flows in the console with some style."""
> +    proc = DPConsoleProcessor(
> +        opts, heat_map=["packets", "bytes"] if heat_map else []
> +    )
> +    proc.process()
> +    proc.print()
> diff --git a/python/ovs/flowviz/ofp/cli.py b/python/ovs/flowviz/ofp/cli.py
> index b9a2a8aad..a399dbd82 100644
> --- a/python/ovs/flowviz/ofp/cli.py
> +++ b/python/ovs/flowviz/ofp/cli.py
> @@ -16,6 +16,7 @@ import click
>
>  from ovs.flowviz.main import maincli
>  from ovs.flowviz.process import (
> +    ConsoleProcessor,
>      OpenFlowFactory,
>      JSONProcessor,
>  )
> @@ -40,3 +41,28 @@ def json(opts):
>      proc = JSONPrint(opts)
>      proc.process()
>      print(proc.json_string())
> +
> +
> +class OFConsoleProcessor(OpenFlowFactory, ConsoleProcessor):
> +    def __init__(self, opts, heat_map):
> +        super().__init__(opts, heat_map)
> +
> +
> +@openflow.command()
> +@click.option(
> +    "-h",
> +    "--heat-map",
> +    is_flag=True,
> +    default=False,
> +    show_default=True,
> +    help="Create heat-map with packet and byte counters",
> +)
> +@click.pass_obj
> +def console(opts, heat_map):
> +    """Print the flows in the console with some style."""
> +    proc = OFConsoleProcessor(
> +        opts,
> +        heat_map=["n_packets", "n_bytes"] if heat_map else [],
> +    )
> +    proc.process()
> +    proc.print()
> diff --git a/python/ovs/flowviz/process.py b/python/ovs/flowviz/process.py
> index 3e520e431..349da8017 100644
> --- a/python/ovs/flowviz/process.py
> +++ b/python/ovs/flowviz/process.py
> @@ -20,6 +20,13 @@ from ovs.flow.decoders import FlowEncoder
>  from ovs.flow.odp import ODPFlow
>  from ovs.flow.ofp import OFPFlow
>
> +from ovs.flowviz.console import (
> +    ConsoleFormatter,
> +    default_highlight,
> +    heat_pallete,
> +    file_header,
> +)
> +
>
>  class FileProcessor(object):
>      """Base class for file-based Flow processing. It is able to create flows
> @@ -134,21 +141,24 @@ class FileProcessor(object):
>          self.end()
>
>
> -class DatapathFactory():
> +class DatapathFactory:
>      """A mixin class that creates Datapath flows."""
>
>      def create_flow(self, line, idx):
>          # Skip strings commonly found in Datapath flow dumps.
> -        if any(s in line for s in [
> -            "flow-dump from the main thread",
> -            "flow-dump from pmd on core",
> -        ]):
> +        if any(
> +            s in line
> +            for s in [
> +                "flow-dump from the main thread",
> +                "flow-dump from pmd on core",
> +            ]
> +        ):
>              return None
>
>          return ODPFlow(line, idx)
>
>
> -class OpenFlowFactory():
> +class OpenFlowFactory:
>      """A mixin class that creates OpenFlow flows."""
>
>      def create_flow(self, line, idx):
> @@ -190,3 +200,64 @@ class JSONProcessor(FileProcessor):
>              indent=4,
>              cls=FlowEncoder,
>          )
> +
> +
> +class ConsoleProcessor(FileProcessor):
> +    """A generic Console Processor that prints flows into the console"""
> +
> +    def __init__(self, opts, heat_map=[]):
> +        super().__init__(opts)
> +        self.heat_map = heat_map
> +        self.console = ConsoleFormatter(opts)
> +        if len(self.console.style) == 0 and self.opts.get("highlight"):
> +            # Add some style to highlights or else they won't be seen.
> +            self.console.style.set_default_value_style(
> +                default_highlight(), True
> +            )
> +            self.console.style.set_default_key_style(default_highlight(), True)
> +
> +        self.flows = dict()  # Dictionary of flow-lists, one per file.
> +        self.min_max = dict()  # Used for heat-map. calculation

Guess the dot was dotted at the wrong place ;)

> +
> +    def start_file(self, name, filename):
> +        self.flows_list = list()
> +        if len(self.heat_map) > 0:
> +            self.min = [-1] * len(self.heat_map)
> +            self.max = [0] * len(self.heat_map)
> +
> +    def stop_file(self, name, filename):
> +        self.flows[name] = self.flows_list
> +        if len(self.heat_map) > 0:
> +            self.min_max[name] = (self.min, self.max)
> +
> +    def process_flow(self, flow, name):
> +        # Running calculation of min and max values for all the fields that
> +        # take place in the heatmap.
> +        for i, field in enumerate(self.heat_map):
> +            val = flow.info.get(field)
> +            if self.min[i] == -1 or val < self.min[i]:
> +                self.min[i] = val
> +            if val > self.max[i]:
> +                self.max[i] = val
> +
> +        self.flows_list.append(flow)
> +
> +    def print(self):
> +        for name, flows in self.flows.items():
> +            self.console.console.print("\n")
> +            self.console.console.print(file_header(name))
> +
> +            if len(self.heat_map) > 0 and len(self.flows) > 0:
> +                for i, field in enumerate(self.heat_map):
> +                    (min_val, max_val) = self.min_max[name][i]
> +                    self.console.style.set_value_style(
> +                        field, heat_pallete(min_val, max_val)
> +                    )
> +
> +            for flow in flows:
> +                high = None
> +                if self.opts.get("highlight"):
> +                    result = self.opts.get("highlight").evaluate(flow)
> +                    if result:
> +                        high = result.kv
> +                self.console.print_flow(flow, high)
> diff --git a/python/setup.py b/python/setup.py
> index 4b9c751d2..76f9fc820 100644
> --- a/python/setup.py
> +++ b/python/setup.py
> @@ -113,9 +113,11 @@ setup_args = dict(
>      extras_require={':sys_platform == "win32"': ['pywin32 >= 1.0'],
>                      'dns': ['unbound'],
>                      'flow': flow_extras_require,
> -                    'flowviz': [*flow_extras_require, 'click'],
> +                    'flowviz':
> +                        [*flow_extras_require, 'click', 'rich'],
>                      },
>      scripts=["ovs/flowviz/ovs-flowviz"],
> +    include_package_data=True,
>  )
>
>  try:
> -- 
> 2.44.0
>
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev

On 3/15/24 12:29, Eelco Chaudron wrote:
> 
> 
> On 13 Mar 2024, at 10:03, Adrian Moreno wrote:
> 
>> Add a flow formatting framework and one implementation for console
>> printing using rich.
>>
>> The flow formatting framework is a simple set of classes that can be
>> used to write different flow formatting implementations. It supports
>> styles to be described by any class, highlighting and config-file based
>> style definition.
>>
>> The first flow formatting implementation is also introduced: the
>> ConsoleFormatter. It uses the an advanced rich-text printing library
>> [1].
>>
>> The console printing supports:
>> - Heatmap: printing the packet/byte statistics of each flow in a color
>>    that represents its relative size: blue (low) -> red (high).
>> - Printing a banner with the file name and alias.
>> - Extensive style definition via config file.
>>
>> This console format is added to both OpenFlow and Datapath flows.
>>
>> Examples:
>> - Highlight drops in datapath flows:
>> $ ovs-flowviz -i flows.txt --highlight "drop" datapath console
>> - Quickly detect where most packets are going using heatmap and
>>    paginated output:
>> $ ovs-ofctl dump-flows br-int | ovs-flowviz openflow console -h
>>
>> [1] https://rich.readthedocs.io/en/stable/introduction.html
>>
>> Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
> 
> Thanks for making these changes, one small nit. Guess your cursor was at a different place than you thought it was :)
> 
> If this is the only change in your next rev, add my ‘Acked-by: Eelco Chaudron <echaudro@redhat.com>’.
> 
> //Eelco
> 
>> ---
>>   python/automake.mk            |   2 +
>>   python/ovs/flowviz/console.py | 175 ++++++++++++++++
>>   python/ovs/flowviz/format.py  | 371 ++++++++++++++++++++++++++++++++++
>>   python/ovs/flowviz/main.py    |  58 +++++-
>>   python/ovs/flowviz/odp/cli.py |  25 +++
>>   python/ovs/flowviz/ofp/cli.py |  26 +++
>>   python/ovs/flowviz/process.py |  83 +++++++-
>>   python/setup.py               |   4 +-
>>   8 files changed, 736 insertions(+), 8 deletions(-)
>>   create mode 100644 python/ovs/flowviz/console.py
>>   create mode 100644 python/ovs/flowviz/format.py
>>
>> diff --git a/python/automake.mk b/python/automake.mk
>> index fd5e74081..bd53c5405 100644
>> --- a/python/automake.mk
>> +++ b/python/automake.mk
>> @@ -65,6 +65,8 @@ ovs_pytests = \
>>
>>   ovs_flowviz = \
>>   	python/ovs/flowviz/__init__.py \
>> +	python/ovs/flowviz/console.py \
>> +	python/ovs/flowviz/format.py \
>>   	python/ovs/flowviz/main.py \
>>   	python/ovs/flowviz/odp/__init__.py \
>>   	python/ovs/flowviz/odp/cli.py \
>> diff --git a/python/ovs/flowviz/console.py b/python/ovs/flowviz/console.py
>> new file mode 100644
>> index 000000000..4a3443360
>> --- /dev/null
>> +++ b/python/ovs/flowviz/console.py
>> @@ -0,0 +1,175 @@
>> +# Copyright (c) 2023 Red Hat, Inc.
>> +#
>> +# Licensed under the Apache License, Version 2.0 (the "License");
>> +# you may not use this file except in compliance with the License.
>> +# You may obtain a copy of the License at:
>> +#
>> +#     http://www.apache.org/licenses/LICENSE-2.0
>> +#
>> +# Unless required by applicable law or agreed to in writing, software
>> +# distributed under the License is distributed on an "AS IS" BASIS,
>> +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>> +# See the License for the specific language governing permissions and
>> +# limitations under the License.
>> +
>> +import colorsys
>> +
>> +from rich.console import Console
>> +from rich.color import Color
>> +from rich.emoji import Emoji
>> +from rich.panel import Panel
>> +from rich.text import Text
>> +from rich.style import Style
>> +
>> +from ovs.flowviz.format import FlowFormatter, FlowBuffer, FlowStyle
>> +
>> +
>> +def file_header(name):
>> +    return Panel(
>> +        Text(
>> +            Emoji.replace(":scroll:")
>> +            + " "
>> +            + name
>> +            + " "
>> +            + Emoji.replace(":scroll:"),
>> +            style="bold",
>> +            justify="center",
>> +        )
>> +    )
>> +
>> +
>> +class ConsoleBuffer(FlowBuffer):
>> +    """ConsoleBuffer implements FlowBuffer to provide console-based text
>> +    formatting based on rich.Text.
>> +
>> +    Append functions accept a rich.Style.
>> +
>> +    Args:
>> +        rtext(rich.Text): Optional; text instance to reuse
>> +    """
>> +
>> +    def __init__(self, rtext):
>> +        self._text = rtext or Text()
>> +
>> +    @property
>> +    def text(self):
>> +        return self._text
>> +
>> +    def _append(self, string, style):
>> +        """Append to internal text."""
>> +        return self._text.append(string, style)
>> +
>> +    def append_key(self, kv, style):
>> +        """Append a key.
>> +        Args:
>> +            kv (KeyValue): the KeyValue instance to append
>> +            style (rich.Style): the style to use
>> +        """
>> +        return self._append(kv.meta.kstring, style)
>> +
>> +    def append_delim(self, kv, style):
>> +        """Append a delimiter.
>> +        Args:
>> +            kv (KeyValue): the KeyValue instance to append
>> +            style (rich.Style): the style to use
>> +        """
>> +        return self._append(kv.meta.delim, style)
>> +
>> +    def append_end_delim(self, kv, style):
>> +        """Append an end delimiter.
>> +        Args:
>> +            kv (KeyValue): the KeyValue instance to append
>> +            style (rich.Style): the style to use
>> +        """
>> +        return self._append(kv.meta.end_delim, style)
>> +
>> +    def append_value(self, kv, style):
>> +        """Append a value.
>> +        Args:
>> +            kv (KeyValue): the KeyValue instance to append
>> +            style (rich.Style): the style to use
>> +        """
>> +        return self._append(kv.meta.vstring, style)
>> +
>> +    def append_extra(self, extra, style):
>> +        """Append extra string.
>> +        Args:
>> +            kv (KeyValue): the KeyValue instance to append
>> +            style (rich.Style): the style to use
>> +        """
>> +        return self._append(extra, style)
>> +
>> +
>> +class ConsoleFormatter(FlowFormatter):
>> +    """ConsoleFormatter is a FlowFormatter that formats flows into the console
>> +    using rich.Console.
>> +
>> +    Args:
>> +        console (rich.Console): Optional, an existing console to use
>> +        max_value_len (int): Optional; max length of the printed values
>> +        kwargs (dict): Optional; Extra arguments to be passed down to
>> +            rich.console.Console()
>> +    """
>> +
>> +    def __init__(self, opts=None, console=None, **kwargs):
>> +        super(ConsoleFormatter, self).__init__()
>> +        style = self.style_from_opts(opts)
>> +        self.console = console or Console(color_system="256", **kwargs)
>> +        self.style = style or FlowStyle()
>> +
>> +    def style_from_opts(self, opts):
>> +        return self._style_from_opts(opts, "console", Style)
>> +
>> +    def print_flow(self, flow, highlighted=None):
>> +        """Prints a flow to the console.
>> +
>> +        Args:
>> +            flow (ovs_dbg.OFPFlow): the flow to print
>> +            style (dict): Optional; style dictionary to use
>> +            highlighted (list): Optional; list of KeyValues to highlight
>> +        """
>> +
>> +        buf = ConsoleBuffer(Text())
>> +        self.format_flow(buf, flow, highlighted)
>> +        self.console.print(buf.text)
>> +
>> +    def format_flow(self, buf, flow, highlighted=None):
>> +        """Formats the flow into the provided buffer as a rich.Text.
>> +
>> +        Args:
>> +            buf (FlowBuffer): the flow buffer to append to
>> +            flow (ovs_dbg.OFPFlow): the flow to format
>> +            style (FlowStyle): Optional; style object to use
>> +            highlighted (list): Optional; list of KeyValues to highlight
>> +        """
>> +        return super(ConsoleFormatter, self).format_flow(
>> +            buf, flow, self.style, highlighted
>> +        )
>> +
>> +
>> +def heat_pallete(min_value, max_value):
>> +    """Generates a color pallete based on the 5-color heat pallete so that
>> +    for each value between min and max a color is returned that represents it's
>> +    relative size.
>> +    Args:
>> +        min_value (int): minimum value
>> +        max_value (int) maximum value
>> +    """
>> +    h_min = 0  # red
>> +    h_max = 220 / 360  # blue
>> +
>> +    def heat(value):
>> +        if max_value == min_value:
>> +            r, g, b = colorsys.hsv_to_rgb(h_max / 2, 1.0, 1.0)
>> +        else:
>> +            normalized = (int(value) - min_value) / (max_value - min_value)
>> +            hue = ((1 - normalized) + h_min) * (h_max - h_min)
>> +            r, g, b = colorsys.hsv_to_rgb(hue, 1.0, 1.0)
>> +        return Style(color=Color.from_rgb(r * 255, g * 255, b * 255))
>> +
>> +    return heat
>> +
>> +
>> +def default_highlight():
>> +    """Generates a default style for highlights."""
>> +    return Style(underline=True)
>> diff --git a/python/ovs/flowviz/format.py b/python/ovs/flowviz/format.py
>> new file mode 100644
>> index 000000000..70af2fa26
>> --- /dev/null
>> +++ b/python/ovs/flowviz/format.py
>> @@ -0,0 +1,371 @@
>> +# Copyright (c) 2023 Red Hat, Inc.
>> +#
>> +# Licensed under the Apache License, Version 2.0 (the "License");
>> +# you may not use this file except in compliance with the License.
>> +# You may obtain a copy of the License at:
>> +#
>> +#     http://www.apache.org/licenses/LICENSE-2.0
>> +#
>> +# Unless required by applicable law or agreed to in writing, software
>> +# distributed under the License is distributed on an "AS IS" BASIS,
>> +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>> +# See the License for the specific language governing permissions and
>> +# limitations under the License.
>> +
>> +"""Flow formatting framework.
>> +
>> +This file defines a simple flow formatting framework. It's comprised of 3
>> +classes: FlowStyle, FlowFormatter and FlowBuffer.
>> +
>> +The FlowStyle arranges opaque style objects in a dictionary that can be queried
>> +to determine what style a particular key-value should be formatted with.
>> +That way, a particular implementation can represent its style using their own
>> +object.
>> +
>> +The FlowBuffer is an abstract class and must be derived by particular
>> +implementations. It should know how to append parts of a flow using a style.
>> +Only here the type of the style is relevant.
>> +
>> +When asked to format a flow, the FlowFormatter will determine which style
>> +the flow must be formatted with and call FlowBuffer functions with each part
>> +of the flow and their corresponding style.
>> +"""
>> +
>> +
>> +class FlowStyle:
>> +    """A FlowStyle determines the KVStyle to use for each key value in a flow.
>> +
>> +    Styles are internally represented by a dictionary.
>> +    In order to determine the style for a "key", the following items in the
>> +    dictionary are fetched:
>> +        - key.highlighted.{key} (if key is found in hightlighted)
>> +        - key.highlighted (if key is found in hightlighted)
>> +        - key.{key}
>> +        - key
>> +        - default
>> +
>> +    In order to determine the style for a "value", the following items in the
>> +    dictionary are fetched:
>> +        - value.highlighted.{key} (if key is found in hightlighted)
>> +        - value.highlighted.type{value.__class__.__name__}
>> +        - value.highlighted
>> +        (if key is found in hightlighted)
>> +        - value.{key}
>> +        - value.type.{value.__class__.__name__}
>> +        - value
>> +        - default
>> +
>> +    The actual type of the style object stored for each item above is opaque
>> +    to this class and it depends on the particular FlowFormatter child class
>> +    that will handle them. Even callables can be stored, if so they will be
>> +    called with the value of the field that is to be formatted and the return
>> +    object will be used as style.
>> +
>> +    Additionally, the following style items can be defined:
>> +        - delim: for delimiters
>> +        - delim.highlighted: for delimiters of highlighted key-values
>> +    """
>> +
>> +    def __init__(self, initial=None):
>> +        self._styles = initial if initial is not None else dict()
>> +
>> +    def __len__(self):
>> +        return len(self._styles)
>> +
>> +    def set_flag_style(self, kvstyle):
>> +        self._styles["flag"] = kvstyle
>> +
>> +    def set_delim_style(self, kvstyle, highlighted=False):
>> +        if highlighted:
>> +            self._styles["delim.highlighted"] = kvstyle
>> +        else:
>> +            self._styles["delim"] = kvstyle
>> +
>> +    def set_default_key_style(self, kvstyle, highlighted=False):
>> +        if highlighted:
>> +            self._styles["key.highlighted"] = kvstyle
>> +        else:
>> +            self._styles["key"] = kvstyle
>> +
>> +    def set_default_value_style(self, kvstyle, highlighted=False):
>> +        if highlighted:
>> +            self._styles["value.highlighted"] = kvstyle
>> +        else:
>> +            self._styles["value"] = kvstyle
>> +
>> +    def set_key_style(self, key, kvstyle, highlighted=False):
>> +        if highlighted:
>> +            self._styles["key.highlighted.{}".format(key)] = kvstyle
>> +        else:
>> +            self._styles["key.{}".format(key)] = kvstyle
>> +
>> +    def set_value_style(self, key, kvstyle, highlighted=None):
>> +        if highlighted:
>> +            self._styles["value.highlighted.{}".format(key)] = kvstyle
>> +        else:
>> +            self._styles["value.{}".format(key)] = kvstyle
>> +
>> +    def set_value_type_style(self, name, kvstyle, highlighted=None):
>> +        if highlighted:
>> +            self._styles["value.highlighted.type.{}".format(name)] = kvstyle
>> +        else:
>> +            self._styles["value.type.{}".format(name)] = kvstyle
>> +
>> +    def get(self, key):
>> +        return self._styles.get(key)
>> +
>> +    def get_delim_style(self, highlighted=False):
>> +        delim_style_lookup = ["delim.highlighted"] if highlighted else []
>> +        delim_style_lookup.extend(["delim", "default"])
>> +        return next(
>> +            (
>> +                self._styles.get(s)
>> +                for s in delim_style_lookup
>> +                if self._styles.get(s)
>> +            ),
>> +            None,
>> +        )
>> +
>> +    def get_flag_style(self):
>> +        return self._styles.get("flag") or self._styles.get("default")
>> +
>> +    def get_key_style(self, kv, highlighted=False):
>> +        key = kv.key
>> +
>> +        key_style_lookup = (
>> +            ["key.highlighted.%s" % key, "key.highlighted"]
>> +            if highlighted
>> +            else []
>> +        )
>> +        key_style_lookup.extend(["key.%s" % key, "key", "default"])
>> +
>> +        style = next(
>> +            (
>> +                self._styles.get(s)
>> +                for s in key_style_lookup
>> +                if self._styles.get(s)
>> +            ),
>> +            None,
>> +        )
>> +        if callable(style):
>> +            return style(kv.meta.kstring)
>> +        return style
>> +
>> +    def get_value_style(self, kv, highlighted=False):
>> +        key = kv.key
>> +        value_type = kv.value.__class__.__name__.lower()
>> +        value_style_lookup = (
>> +            [
>> +                "value.highlighted.%s" % key,
>> +                "value.highlighted.type.%s" % value_type,
>> +                "value.highlighted",
>> +            ]
>> +            if highlighted
>> +            else []
>> +        )
>> +        value_style_lookup.extend(
>> +            [
>> +                "value.%s" % key,
>> +                "value.type.%s" % value_type,
>> +                "value",
>> +                "default",
>> +            ]
>> +        )
>> +
>> +        style = next(
>> +            (
>> +                self._styles.get(s)
>> +                for s in value_style_lookup
>> +                if self._styles.get(s)
>> +            ),
>> +            None,
>> +        )
>> +        if callable(style):
>> +            return style(kv.meta.vstring)
>> +        return style
>> +
>> +
>> +class FlowFormatter:
>> +    """FlowFormatter is a base class for Flow Formatters."""
>> +
>> +    def __init__(self):
>> +        self._highlighted = list()
>> +
>> +    def _style_from_opts(self, opts, opts_key, style_constructor):
>> +        """Create style object from options.
>> +
>> +        Args:
>> +            opts (dict): Options dictionary
>> +            opts_key (str): The options style key to extract
>> +                (e.g: console or html)
>> +            style_constructor(callable): A callable that creates a derived
>> +                style object
>> +        """
>> +        if not opts or not opts.get("style"):
>> +            return None
>> +
>> +        section_name = ".".join(["styles", opts.get("style")])
>> +        if section_name not in opts.get("config").sections():
>> +            return None
>> +
>> +        config = opts.get("config")[section_name]
>> +        style = {}
>> +        for key in config:
>> +            (_, console, style_full_key) = key.partition(opts_key + ".")
>> +            if not console:
>> +                continue
>> +
>> +            (style_key, _, prop) = style_full_key.rpartition(".")
>> +            if not prop or not style_key:
>> +                raise Exception("malformed style config: {}".format(key))
>> +
>> +            if not style.get(style_key):
>> +                style[style_key] = {}
>> +            style[style_key][prop] = config[key]
>> +
>> +        return FlowStyle({k: style_constructor(**v) for k, v in style.items()})
>> +
>> +    def format_flow(self, buf, flow, style_obj=None, highlighted=None):
>> +        """Formats the flow into the provided buffer.
>> +
>> +        Args:
>> +            buf (FlowBuffer): the flow buffer to append to
>> +            flow (ovs_dbg.OFPFlow): the flow to format
>> +            style_obj (FlowStyle): Optional; style to use
>> +            highlighted (list): Optional; list of KeyValues to highlight
>> +        """
>> +        last_printed_pos = 0
>> +
>> +        if style_obj:
>> +            style_obj = style_obj or FlowStyle()
>> +            for section in sorted(flow.sections, key=lambda x: x.pos):
>> +                buf.append_extra(
>> +                    flow.orig[last_printed_pos : section.pos],
>> +                    style=style_obj.get("default"),
>> +                )
>> +                self.format_kv_list(
>> +                    buf, section.data, section.string, style_obj, highlighted
>> +                )
>> +                last_printed_pos = section.pos + len(section.string)
>> +        else:
>> +            # Don't pay the cost of formatting each section one by one.
>> +            buf.append_extra(flow.orig.strip(), None)
>> +
>> +    def format_kv_list(self, buf, kv_list, full_str, style_obj, highlighted):
>> +        """Format a KeyValue List.
>> +
>> +        Args:
>> +            buf (FlowBuffer): a FlowBuffer to append formatted KeyValues to
>> +            kv_list (list[KeyValue]: the KeyValue list to format
>> +            full_str (str): the full string containing all k-v
>> +            style_obj (FlowStyle): a FlowStyle object to use
>> +            highlighted (list): Optional; list of KeyValues to highlight
>> +        """
>> +        for i, kv in enumerate(kv_list):
>> +            written = self.format_kv(
>> +                buf, kv, style_obj=style_obj, highlighted=highlighted
>> +            )
>> +
>> +            end = (
>> +                kv_list[i + 1].meta.kpos
>> +                if i < (len(kv_list) - 1)
>> +                else len(full_str)
>> +            )
>> +
>> +            buf.append_extra(
>> +                full_str[(kv.meta.kpos + written) : end].rstrip("\n\r"),
>> +                style=style_obj.get("default"),
>> +            )
>> +
>> +    def format_kv(self, buf, kv, style_obj, highlighted=None):
>> +        """Format a KeyValue
>> +
>> +        A formatted keyvalue has the following parts:
>> +            {key}{delim}{value}[{delim}]
>> +
>> +        Args:
>> +            buf (FlowBuffer): buffer to append the KeyValue to
>> +            kv (KeyValue): The KeyValue to print
>> +            style_obj (FlowStyle): The style object to use
>> +            highlighted (list): Optional; list of KeyValues to highlight
>> +
>> +        Returns the number of printed characters.
>> +        """
>> +        ret = 0
>> +        key = kv.meta.kstring
>> +        is_highlighted = (
>> +            key in [k.key for k in highlighted] if highlighted else False
>> +        )
>> +
>> +        key_style = style_obj.get_key_style(kv, is_highlighted)
>> +        buf.append_key(kv, key_style)  # format value
>> +        ret += len(key)
>> +
>> +        if not kv.meta.vstring:
>> +            return ret
>> +
>> +        if kv.meta.delim not in ("\n", "\t", "\r", ""):
>> +            buf.append_delim(kv, style_obj.get_delim_style(is_highlighted))
>> +            ret += len(kv.meta.delim)
>> +
>> +        value_style = style_obj.get_value_style(kv, is_highlighted)
>> +        buf.append_value(kv, value_style)  # format value
>> +        ret += len(kv.meta.vstring)
>> +
>> +        if kv.meta.end_delim:
>> +            buf.append_end_delim(kv, style_obj.get_delim_style(is_highlighted))
>> +            ret += len(kv.meta.end_delim)
>> +
>> +        return ret
>> +
>> +
>> +class FlowBuffer:
>> +    """A FlowBuffer is a base class for format buffers.
>> +
>> +    Childs must implement the following methods:
>> +        append_key(self, kv, style)
>> +        append_value(self, kv, style)
>> +        append_delim(self, delim, style)
>> +        append_end_delim(self, delim, style)
>> +        append_extra(self, extra, style)
>> +    """
>> +
>> +    def append_key(self, kv, style):
>> +        """Append a key.
>> +        Args:
>> +            kv (KeyValue): the KeyValue instance to append
>> +            style (Any): the style to use
>> +        """
>> +        raise NotImplementedError
>> +
>> +    def append_delim(self, kv, style):
>> +        """Append a delimiter.
>> +        Args:
>> +            kv (KeyValue): the KeyValue instance to append
>> +            style (Any): the style to use
>> +        """
>> +        raise NotImplementedError
>> +
>> +    def append_end_delim(self, kv, style):
>> +        """Append an end delimiter.
>> +        Args:
>> +            kv (KeyValue): the KeyValue instance to append
>> +            style (Any): the style to use
>> +        """
>> +        raise NotImplementedError
>> +
>> +    def append_value(self, kv, style):
>> +        """Append a value.
>> +        Args:
>> +            kv (KeyValue): the KeyValue instance to append
>> +            style (Any): the style to use
>> +        """
>> +        raise NotImplementedError
>> +
>> +    def append_extra(self, extra, style):
>> +        """Append extra string.
>> +        Args:
>> +            kv (KeyValue): the KeyValue instance to append
>> +            style (Any): the style to use
>> +        """
>> +        raise NotImplementedError
>> diff --git a/python/ovs/flowviz/main.py b/python/ovs/flowviz/main.py
>> index 64b0e8a0a..723c71fa7 100644
>> --- a/python/ovs/flowviz/main.py
>> +++ b/python/ovs/flowviz/main.py
>> @@ -12,10 +12,30 @@
>>   # See the License for the specific language governing permissions and
>>   # limitations under the License.
>>
>> +import configparser
>>   import click
>>   import os
>>
>>   from ovs.flow.filter import OFFilter
>> +from ovs.dirs import PKGDATADIR
>> +
>> +_default_config_file = "ovs-flowviz.conf"
>> +_default_config_path = next(
>> +    (
>> +        p
>> +        for p in [
>> +            os.path.join(
>> +                os.getenv("HOME"), ".config", "ovs", _default_config_file
>> +            ),
>> +            os.path.join(PKGDATADIR, _default_config_file),
>> +            os.path.abspath(
>> +                os.path.join(os.path.dirname(__file__), _default_config_file)
>> +            ),
>> +        ]
>> +        if os.path.exists(p)
>> +    ),
>> +    "",
>> +)
>>
>>
>>   class Options(dict):
>> @@ -48,6 +68,20 @@ def validate_input(ctx, param, value):
>>   @click.group(
>>       context_settings=dict(help_option_names=["-h", "--help"]),
>>   )
>> +@click.option(
>> +    "-c",
>> +    "--config",
>> +    help="Use config file",
>> +    type=click.Path(),
>> +    default=_default_config_path,
>> +    show_default=True,
>> +)
>> +@click.option(
>> +    "--style",
>> +    help="Select style (defined in config file)",
>> +    default=None,
>> +    show_default=True,
>> +)
>>   @click.option(
>>       "-i",
>>       "--input",
>> @@ -69,8 +103,17 @@ def validate_input(ctx, param, value):
>>       type=str,
>>       show_default=False,
>>   )
>> +@click.option(
>> +    "-l",
>> +    "--highlight",
>> +    help="Highlight flows that match the filter expression."
>> +    "Run 'ovs-flowviz filter' for a detailed description of the filtering "
>> +    "syntax",
>> +    type=str,
>> +    show_default=False,
>> +)
>>   @click.pass_context
>> -def maincli(ctx, filename, filter):
>> +def maincli(ctx, config, style, filename, filter, highlight):
>>       """
>>       OpenvSwitch flow visualization utility.
>>
>> @@ -86,6 +129,19 @@ def maincli(ctx, filename, filter):
>>           except Exception as e:
>>               raise click.BadParameter("Wrong filter syntax: {}".format(e))
>>
>> +    if highlight:
>> +        try:
>> +            ctx.obj["highlight"] = OFFilter(highlight)
>> +        except Exception as e:
>> +            raise click.BadParameter("Wrong filter syntax: {}".format(e))
>> +
>> +    config_file = config or _default_config_path
>> +    parser = configparser.ConfigParser()
>> +    parser.read(config_file)
>> +
>> +    ctx.obj["config"] = parser
>> +    ctx.obj["style"] = style
>> +
>>
>>   @maincli.command(hidden=True)
>>   @click.pass_context
>> diff --git a/python/ovs/flowviz/odp/cli.py b/python/ovs/flowviz/odp/cli.py
>> index ed2f82065..a1cba0135 100644
>> --- a/python/ovs/flowviz/odp/cli.py
>> +++ b/python/ovs/flowviz/odp/cli.py
>> @@ -16,6 +16,7 @@ import click
>>
>>   from ovs.flowviz.main import maincli
>>   from ovs.flowviz.process import (
>> +    ConsoleProcessor,
>>       DatapathFactory,
>>       JSONProcessor,
>>   )
>> @@ -40,3 +41,27 @@ def json(opts):
>>       proc = JSONPrint(opts)
>>       proc.process()
>>       print(proc.json_string())
>> +
>> +
>> +class DPConsoleProcessor(DatapathFactory, ConsoleProcessor):
>> +    def __init__(self, opts, heat_map):
>> +        super().__init__(opts, heat_map)
>> +
>> +
>> +@datapath.command()
>> +@click.option(
>> +    "-h",
>> +    "--heat-map",
>> +    is_flag=True,
>> +    default=False,
>> +    show_default=True,
>> +    help="Create heat-map with packet and byte counters",
>> +)
>> +@click.pass_obj
>> +def console(opts, heat_map):
>> +    """Print the flows in the console with some style."""
>> +    proc = DPConsoleProcessor(
>> +        opts, heat_map=["packets", "bytes"] if heat_map else []
>> +    )
>> +    proc.process()
>> +    proc.print()
>> diff --git a/python/ovs/flowviz/ofp/cli.py b/python/ovs/flowviz/ofp/cli.py
>> index b9a2a8aad..a399dbd82 100644
>> --- a/python/ovs/flowviz/ofp/cli.py
>> +++ b/python/ovs/flowviz/ofp/cli.py
>> @@ -16,6 +16,7 @@ import click
>>
>>   from ovs.flowviz.main import maincli
>>   from ovs.flowviz.process import (
>> +    ConsoleProcessor,
>>       OpenFlowFactory,
>>       JSONProcessor,
>>   )
>> @@ -40,3 +41,28 @@ def json(opts):
>>       proc = JSONPrint(opts)
>>       proc.process()
>>       print(proc.json_string())
>> +
>> +
>> +class OFConsoleProcessor(OpenFlowFactory, ConsoleProcessor):
>> +    def __init__(self, opts, heat_map):
>> +        super().__init__(opts, heat_map)
>> +
>> +
>> +@openflow.command()
>> +@click.option(
>> +    "-h",
>> +    "--heat-map",
>> +    is_flag=True,
>> +    default=False,
>> +    show_default=True,
>> +    help="Create heat-map with packet and byte counters",
>> +)
>> +@click.pass_obj
>> +def console(opts, heat_map):
>> +    """Print the flows in the console with some style."""
>> +    proc = OFConsoleProcessor(
>> +        opts,
>> +        heat_map=["n_packets", "n_bytes"] if heat_map else [],
>> +    )
>> +    proc.process()
>> +    proc.print()
>> diff --git a/python/ovs/flowviz/process.py b/python/ovs/flowviz/process.py
>> index 3e520e431..349da8017 100644
>> --- a/python/ovs/flowviz/process.py
>> +++ b/python/ovs/flowviz/process.py
>> @@ -20,6 +20,13 @@ from ovs.flow.decoders import FlowEncoder
>>   from ovs.flow.odp import ODPFlow
>>   from ovs.flow.ofp import OFPFlow
>>
>> +from ovs.flowviz.console import (
>> +    ConsoleFormatter,
>> +    default_highlight,
>> +    heat_pallete,
>> +    file_header,
>> +)
>> +
>>
>>   class FileProcessor(object):
>>       """Base class for file-based Flow processing. It is able to create flows
>> @@ -134,21 +141,24 @@ class FileProcessor(object):
>>           self.end()
>>
>>
>> -class DatapathFactory():
>> +class DatapathFactory:
>>       """A mixin class that creates Datapath flows."""
>>
>>       def create_flow(self, line, idx):
>>           # Skip strings commonly found in Datapath flow dumps.
>> -        if any(s in line for s in [
>> -            "flow-dump from the main thread",
>> -            "flow-dump from pmd on core",
>> -        ]):
>> +        if any(
>> +            s in line
>> +            for s in [
>> +                "flow-dump from the main thread",
>> +                "flow-dump from pmd on core",
>> +            ]
>> +        ):
>>               return None
>>
>>           return ODPFlow(line, idx)
>>
>>
>> -class OpenFlowFactory():
>> +class OpenFlowFactory:
>>       """A mixin class that creates OpenFlow flows."""
>>
>>       def create_flow(self, line, idx):
>> @@ -190,3 +200,64 @@ class JSONProcessor(FileProcessor):
>>               indent=4,
>>               cls=FlowEncoder,
>>           )
>> +
>> +
>> +class ConsoleProcessor(FileProcessor):
>> +    """A generic Console Processor that prints flows into the console"""
>> +
>> +    def __init__(self, opts, heat_map=[]):
>> +        super().__init__(opts)
>> +        self.heat_map = heat_map
>> +        self.console = ConsoleFormatter(opts)
>> +        if len(self.console.style) == 0 and self.opts.get("highlight"):
>> +            # Add some style to highlights or else they won't be seen.
>> +            self.console.style.set_default_value_style(
>> +                default_highlight(), True
>> +            )
>> +            self.console.style.set_default_key_style(default_highlight(), True)
>> +
>> +        self.flows = dict()  # Dictionary of flow-lists, one per file.
>> +        self.min_max = dict()  # Used for heat-map. calculation
> 
> Guess the dot was dotted at the wrong place ;)
> 

Oh my goodness! I need glasses!

>> +
>> +    def start_file(self, name, filename):
>> +        self.flows_list = list()
>> +        if len(self.heat_map) > 0:
>> +            self.min = [-1] * len(self.heat_map)
>> +            self.max = [0] * len(self.heat_map)
>> +
>> +    def stop_file(self, name, filename):
>> +        self.flows[name] = self.flows_list
>> +        if len(self.heat_map) > 0:
>> +            self.min_max[name] = (self.min, self.max)
>> +
>> +    def process_flow(self, flow, name):
>> +        # Running calculation of min and max values for all the fields that
>> +        # take place in the heatmap.
>> +        for i, field in enumerate(self.heat_map):
>> +            val = flow.info.get(field)
>> +            if self.min[i] == -1 or val < self.min[i]:
>> +                self.min[i] = val
>> +            if val > self.max[i]:
>> +                self.max[i] = val
>> +
>> +        self.flows_list.append(flow)
>> +
>> +    def print(self):
>> +        for name, flows in self.flows.items():
>> +            self.console.console.print("\n")
>> +            self.console.console.print(file_header(name))
>> +
>> +            if len(self.heat_map) > 0 and len(self.flows) > 0:
>> +                for i, field in enumerate(self.heat_map):
>> +                    (min_val, max_val) = self.min_max[name][i]
>> +                    self.console.style.set_value_style(
>> +                        field, heat_pallete(min_val, max_val)
>> +                    )
>> +
>> +            for flow in flows:
>> +                high = None
>> +                if self.opts.get("highlight"):
>> +                    result = self.opts.get("highlight").evaluate(flow)
>> +                    if result:
>> +                        high = result.kv
>> +                self.console.print_flow(flow, high)
>> diff --git a/python/setup.py b/python/setup.py
>> index 4b9c751d2..76f9fc820 100644
>> --- a/python/setup.py
>> +++ b/python/setup.py
>> @@ -113,9 +113,11 @@ setup_args = dict(
>>       extras_require={':sys_platform == "win32"': ['pywin32 >= 1.0'],
>>                       'dns': ['unbound'],
>>                       'flow': flow_extras_require,
>> -                    'flowviz': [*flow_extras_require, 'click'],
>> +                    'flowviz':
>> +                        [*flow_extras_require, 'click', 'rich'],
>>                       },
>>       scripts=["ovs/flowviz/ovs-flowviz"],
>> +    include_package_data=True,
>>   )
>>
>>   try:
>> -- 
>> 2.44.0
>>
>> _______________________________________________
>> dev mailing list
>> dev@openvswitch.org
>> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>