From patchwork Sat Mar 16 14:34:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marcel Apfelbaum X-Patchwork-Id: 1057376 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="NLnn+cLI"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44M4vy4CLGz9s00 for ; Sun, 17 Mar 2019 01:44:14 +1100 (AEDT) Received: from localhost ([127.0.0.1]:42676 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h5AXk-0008LX-Df for incoming@patchwork.ozlabs.org; Sat, 16 Mar 2019 10:44:12 -0400 Received: from eggs.gnu.org ([209.51.188.92]:49759) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h5AOx-0001IK-QO for qemu-devel@nongnu.org; Sat, 16 Mar 2019 10:35:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1h5AOv-0000s6-3j for qemu-devel@nongnu.org; Sat, 16 Mar 2019 10:35:07 -0400 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]:39369) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1h5AOu-0000qi-Lp for qemu-devel@nongnu.org; Sat, 16 Mar 2019 10:35:05 -0400 Received: by mail-wr1-x441.google.com with SMTP id p8so12458781wrq.6 for ; Sat, 16 Mar 2019 07:35:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=gyWG2fNJOhzuupqPY5E/v2SdFww67Gm55iW4ibJFxNI=; b=NLnn+cLIUChFgR2AkcVLQERN0ugRYEsEr1b7Eic3omol5Dvsw9WqzoRbp91Hv5YUtp g1rfLZZNk77rjsbrJADsYT5QjdvQqULVy658RdtusWYp0H47kUyUEYiurdKKQt0P9SoP sRlCDS3vP741UTXwYym5H8RkRFQ4jzR27mw+l9xjFnQjgNwhKgVMAIl068cTMx17bC7a 5/rgzFwKghrHAy1B51VbYykvigxl+XdSfwOl6KlvKQ1pzqLG25+Yj/P6m3yZrhSFED/m cyQ1B0yeDCgkW/sVog1uLUPJpPrJuvIQpudwRPgVLLRzIUEIqXGzhaIIAptFfaQatgBt N0rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=gyWG2fNJOhzuupqPY5E/v2SdFww67Gm55iW4ibJFxNI=; b=e7jWFBSHo4E130MSOmAcmqfm7McDM+aQZBomB/Pxe08itI3HtMp8QVhQGSUbdVSXW3 HisgxWmV2ZE5IYJPsk0Sb/LwvKZVxc563MUfMfmrLNQ2FHB0VRvjLmDpS8cIlEzaNdSK JwA+HcC8lP5o8w1QtS5RFF3WfMrlB1cBSUt/7e9d10thrsus6CX7Z674Kx2E03qXXNa1 XXnQ6pV2pR30gRrUV5irDOV6HzWGWPp3Fs9AfA1o+mSVWML8BNh2tSkxlVztYXz2KjGD pd6/ke+SLqNDNIKRx7k6ekN3gl9OCcO8KnAkQOxkrIL9f6vztnsaRqJRMrntGBE6fOel i4gg== X-Gm-Message-State: APjAAAV3AWquqGJvXh9KwG2P0SWSM1TGCuJQcSfGF6yEW4x5BqscckKJ 1hMOtChVJTJbA1QscANu+iOI42gi X-Google-Smtp-Source: APXvYqw7DrGCLMlHnEtPxqPWz6qsakZO1nCVoq/WakBLc4PNSN6bwVSo7Y06v7KrQgaHc9afp6NC3Q== X-Received: by 2002:adf:bc89:: with SMTP id g9mr3985985wrh.102.1552746876823; Sat, 16 Mar 2019 07:34:36 -0700 (PDT) Received: from localhost.localdomain ([176.228.155.165]) by smtp.gmail.com with ESMTPSA id l8sm7518895wrv.45.2019.03.16.07.34.35 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 16 Mar 2019 07:34:36 -0700 (PDT) From: Marcel Apfelbaum To: qemu-devel@nongnu.org, peter.maydell@linaro.org Date: Sat, 16 Mar 2019 16:34:09 +0200 Message-Id: <20190316143421.8194-7-marcel.apfelbaum@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190316143421.8194-1-marcel.apfelbaum@gmail.com> References: <20190316143421.8194-1-marcel.apfelbaum@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::441 Subject: [Qemu-devel] [PATCH PULL 06/18] {hmp, hw/pvrdma}: Expose device internals via monitor interface X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: philmd@redhat.com, kamalheib1@gmail.com, yuval.shaia@oracle.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Yuval Shaia Allow interrogating device internals through HMP interface. The exposed indicators can be used for troubleshooting by developers or sysadmin. There is no need to expose these attributes to a management system (e.x. libvirt) because (1) most of them are not "device-management' related info and (2) there is no guarantee the interface is stable. Signed-off-by: Yuval Shaia Acked-by: Dr. David Alan Gilbert Acked-by: Markus Armbruster Message-Id: <1552300155-25216-6-git-send-email-yuval.shaia@oracle.com> Reviewed-by: Marcel Apfelbaum Signed-off-by: Marcel Apfelbaum --- hmp-commands-info.hx | 14 +++++++++++ hmp.c | 27 ++++++++++++++++++++ hmp.h | 1 + hw/rdma/Makefile.objs | 2 +- hw/rdma/rdma.c | 30 ++++++++++++++++++++++ hw/rdma/rdma_rm.c | 53 +++++++++++++++++++++++++++++++++++++++ hw/rdma/rdma_rm.h | 1 + hw/rdma/vmw/pvrdma_main.c | 26 +++++++++++++++++++ include/hw/rdma/rdma.h | 40 +++++++++++++++++++++++++++++ 9 files changed, 193 insertions(+), 1 deletion(-) create mode 100644 hw/rdma/rdma.c create mode 100644 include/hw/rdma/rdma.h diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx index cbee8b944d..c59444c461 100644 --- a/hmp-commands-info.hx +++ b/hmp-commands-info.hx @@ -202,6 +202,20 @@ STEXI @item info pic @findex info pic Show PIC state. +ETEXI + + { + .name = "rdma", + .args_type = "", + .params = "", + .help = "show RDMA state", + .cmd = hmp_info_rdma, + }, + +STEXI +@item info rdma +@findex info rdma +Show RDMA state. ETEXI { diff --git a/hmp.c b/hmp.c index 4a702d5b97..fa1e59a2fc 100644 --- a/hmp.c +++ b/hmp.c @@ -51,6 +51,7 @@ #include "qemu/error-report.h" #include "exec/ramlist.h" #include "hw/intc/intc.h" +#include "hw/rdma/rdma.h" #include "migration/snapshot.h" #include "migration/misc.h" @@ -1013,6 +1014,32 @@ void hmp_info_pic(Monitor *mon, const QDict *qdict) hmp_info_pic_foreach, mon); } +static int hmp_info_rdma_foreach(Object *obj, void *opaque) +{ + RdmaProvider *rdma; + RdmaProviderClass *k; + Monitor *mon = opaque; + + if (object_dynamic_cast(obj, INTERFACE_RDMA_PROVIDER)) { + rdma = RDMA_PROVIDER(obj); + k = RDMA_PROVIDER_GET_CLASS(obj); + if (k->print_statistics) { + k->print_statistics(mon, rdma); + } else { + monitor_printf(mon, "RDMA statistics not available for %s.\n", + object_get_typename(obj)); + } + } + + return 0; +} + +void hmp_info_rdma(Monitor *mon, const QDict *qdict) +{ + object_child_foreach_recursive(object_get_root(), + hmp_info_rdma_foreach, mon); +} + void hmp_info_pci(Monitor *mon, const QDict *qdict) { PciInfoList *info_list, *info; diff --git a/hmp.h b/hmp.h index e0f32f04d3..43617f2646 100644 --- a/hmp.h +++ b/hmp.h @@ -36,6 +36,7 @@ void hmp_info_spice(Monitor *mon, const QDict *qdict); void hmp_info_balloon(Monitor *mon, const QDict *qdict); void hmp_info_irq(Monitor *mon, const QDict *qdict); void hmp_info_pic(Monitor *mon, const QDict *qdict); +void hmp_info_rdma(Monitor *mon, const QDict *qdict); void hmp_info_pci(Monitor *mon, const QDict *qdict); void hmp_info_block_jobs(Monitor *mon, const QDict *qdict); void hmp_info_tpm(Monitor *mon, const QDict *qdict); diff --git a/hw/rdma/Makefile.objs b/hw/rdma/Makefile.objs index bd36cbf51c..c354e60e5b 100644 --- a/hw/rdma/Makefile.objs +++ b/hw/rdma/Makefile.objs @@ -1,5 +1,5 @@ ifeq ($(CONFIG_PVRDMA),y) -obj-$(CONFIG_PCI) += rdma_utils.o rdma_backend.o rdma_rm.o +obj-$(CONFIG_PCI) += rdma_utils.o rdma_backend.o rdma_rm.o rdma.o obj-$(CONFIG_PCI) += vmw/pvrdma_dev_ring.o vmw/pvrdma_cmd.o \ vmw/pvrdma_qp_ops.o vmw/pvrdma_main.o endif diff --git a/hw/rdma/rdma.c b/hw/rdma/rdma.c new file mode 100644 index 0000000000..7bec0d0d2c --- /dev/null +++ b/hw/rdma/rdma.c @@ -0,0 +1,30 @@ +/* + * RDMA device interface + * + * Copyright (C) 2018 Oracle + * Copyright (C) 2018 Red Hat Inc + * + * Authors: + * Yuval Shaia + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#include "qemu/osdep.h" +#include "hw/rdma/rdma.h" +#include "qemu/module.h" + +static const TypeInfo rdma_hmp_info = { + .name = INTERFACE_RDMA_PROVIDER, + .parent = TYPE_INTERFACE, + .class_size = sizeof(RdmaProviderClass), +}; + +static void rdma_register_types(void) +{ + type_register_static(&rdma_hmp_info); +} + +type_init(rdma_register_types) diff --git a/hw/rdma/rdma_rm.c b/hw/rdma/rdma_rm.c index 35fa1ab44e..b50e192b49 100644 --- a/hw/rdma/rdma_rm.c +++ b/hw/rdma/rdma_rm.c @@ -16,6 +16,7 @@ #include "qemu/osdep.h" #include "qapi/error.h" #include "cpu.h" +#include "monitor/monitor.h" #include "trace.h" #include "rdma_utils.h" @@ -26,6 +27,58 @@ #define PG_DIR_SZ { TARGET_PAGE_SIZE / sizeof(__u64) } #define PG_TBL_SZ { TARGET_PAGE_SIZE / sizeof(__u64) } +void rdma_dump_device_counters(Monitor *mon, RdmaDeviceResources *dev_res) +{ + monitor_printf(mon, "\ttx : %" PRId64 "\n", + dev_res->stats.tx); + monitor_printf(mon, "\ttx_len : %" PRId64 "\n", + dev_res->stats.tx_len); + monitor_printf(mon, "\ttx_err : %" PRId64 "\n", + dev_res->stats.tx_err); + monitor_printf(mon, "\trx_bufs : %" PRId64 "\n", + dev_res->stats.rx_bufs); + monitor_printf(mon, "\trx_bufs_len : %" PRId64 "\n", + dev_res->stats.rx_bufs_len); + monitor_printf(mon, "\trx_bufs_err : %" PRId64 "\n", + dev_res->stats.rx_bufs_err); + monitor_printf(mon, "\tcomps : %" PRId64 "\n", + dev_res->stats.completions); + monitor_printf(mon, "\tmissing_comps : %" PRId32 "\n", + dev_res->stats.missing_cqe); + monitor_printf(mon, "\tpoll_cq (bk) : %" PRId64 "\n", + dev_res->stats.poll_cq_from_bk); + monitor_printf(mon, "\tpoll_cq_ppoll_to : %" PRId64 "\n", + dev_res->stats.poll_cq_ppoll_to); + monitor_printf(mon, "\tpoll_cq (fe) : %" PRId64 "\n", + dev_res->stats.poll_cq_from_guest); + monitor_printf(mon, "\tpoll_cq_empty : %" PRId64 "\n", + dev_res->stats.poll_cq_from_guest_empty); + monitor_printf(mon, "\tmad_tx : %" PRId64 "\n", + dev_res->stats.mad_tx); + monitor_printf(mon, "\tmad_tx_err : %" PRId64 "\n", + dev_res->stats.mad_tx_err); + monitor_printf(mon, "\tmad_rx : %" PRId64 "\n", + dev_res->stats.mad_rx); + monitor_printf(mon, "\tmad_rx_err : %" PRId64 "\n", + dev_res->stats.mad_rx_err); + monitor_printf(mon, "\tmad_rx_bufs : %" PRId64 "\n", + dev_res->stats.mad_rx_bufs); + monitor_printf(mon, "\tmad_rx_bufs_err : %" PRId64 "\n", + dev_res->stats.mad_rx_bufs_err); + monitor_printf(mon, "\tPDs : %" PRId32 "\n", + dev_res->pd_tbl.used); + monitor_printf(mon, "\tMRs : %" PRId32 "\n", + dev_res->mr_tbl.used); + monitor_printf(mon, "\tUCs : %" PRId32 "\n", + dev_res->uc_tbl.used); + monitor_printf(mon, "\tQPs : %" PRId32 "\n", + dev_res->qp_tbl.used); + monitor_printf(mon, "\tCQs : %" PRId32 "\n", + dev_res->cq_tbl.used); + monitor_printf(mon, "\tCEQ_CTXs : %" PRId32 "\n", + dev_res->cqe_ctx_tbl.used); +} + static inline void res_tbl_init(const char *name, RdmaRmResTbl *tbl, uint32_t tbl_sz, uint32_t res_sz) { diff --git a/hw/rdma/rdma_rm.h b/hw/rdma/rdma_rm.h index f9b2ec5076..4f03f9b8c5 100644 --- a/hw/rdma/rdma_rm.h +++ b/hw/rdma/rdma_rm.h @@ -81,5 +81,6 @@ static inline union ibv_gid *rdma_rm_get_gid(RdmaDeviceResources *dev_res, { return &dev_res->port.gid_tbl[sgid_idx].gid; } +void rdma_dump_device_counters(Monitor *mon, RdmaDeviceResources *dev_res); #endif diff --git a/hw/rdma/vmw/pvrdma_main.c b/hw/rdma/vmw/pvrdma_main.c index dd35646324..729a2df5a0 100644 --- a/hw/rdma/vmw/pvrdma_main.c +++ b/hw/rdma/vmw/pvrdma_main.c @@ -25,6 +25,8 @@ #include "cpu.h" #include "trace.h" #include "sysemu/sysemu.h" +#include "monitor/monitor.h" +#include "hw/rdma/rdma.h" #include "../rdma_rm.h" #include "../rdma_backend.h" @@ -55,6 +57,26 @@ static Property pvrdma_dev_properties[] = { DEFINE_PROP_END_OF_LIST(), }; +static void pvrdma_print_statistics(Monitor *mon, RdmaProvider *obj) +{ + PVRDMADev *dev = PVRDMA_DEV(obj); + PCIDevice *pdev = PCI_DEVICE(dev); + + monitor_printf(mon, "%s, %x.%x\n", pdev->name, PCI_SLOT(pdev->devfn), + PCI_FUNC(pdev->devfn)); + monitor_printf(mon, "\tcommands : %" PRId64 "\n", + dev->stats.commands); + monitor_printf(mon, "\tregs_reads : %" PRId64 "\n", + dev->stats.regs_reads); + monitor_printf(mon, "\tregs_writes : %" PRId64 "\n", + dev->stats.regs_writes); + monitor_printf(mon, "\tuar_writes : %" PRId64 "\n", + dev->stats.uar_writes); + monitor_printf(mon, "\tinterrupts : %" PRId64 "\n", + dev->stats.interrupts); + rdma_dump_device_counters(mon, &dev->rdma_dev_res); +} + static void free_dev_ring(PCIDevice *pci_dev, PvrdmaRing *ring, void *ring_state) { @@ -639,6 +661,7 @@ static void pvrdma_class_init(ObjectClass *klass, void *data) { DeviceClass *dc = DEVICE_CLASS(klass); PCIDeviceClass *k = PCI_DEVICE_CLASS(klass); + RdmaProviderClass *ir = INTERFACE_RDMA_PROVIDER_CLASS(klass); k->realize = pvrdma_realize; k->exit = pvrdma_exit; @@ -650,6 +673,8 @@ static void pvrdma_class_init(ObjectClass *klass, void *data) dc->desc = "RDMA Device"; dc->props = pvrdma_dev_properties; set_bit(DEVICE_CATEGORY_NETWORK, dc->categories); + + ir->print_statistics = pvrdma_print_statistics; } static const TypeInfo pvrdma_info = { @@ -659,6 +684,7 @@ static const TypeInfo pvrdma_info = { .class_init = pvrdma_class_init, .interfaces = (InterfaceInfo[]) { { INTERFACE_CONVENTIONAL_PCI_DEVICE }, + { INTERFACE_RDMA_PROVIDER }, { } } }; diff --git a/include/hw/rdma/rdma.h b/include/hw/rdma/rdma.h new file mode 100644 index 0000000000..68290fb58c --- /dev/null +++ b/include/hw/rdma/rdma.h @@ -0,0 +1,40 @@ +/* + * RDMA device interface + * + * Copyright (C) 2019 Oracle + * Copyright (C) 2019 Red Hat Inc + * + * Authors: + * Yuval Shaia + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#ifndef RDMA_H +#define RDMA_H + +#include "qom/object.h" + +#define INTERFACE_RDMA_PROVIDER "rdma" + +#define INTERFACE_RDMA_PROVIDER_CLASS(klass) \ + OBJECT_CLASS_CHECK(RdmaProviderClass, (klass), \ + INTERFACE_RDMA_PROVIDER) +#define RDMA_PROVIDER_GET_CLASS(obj) \ + OBJECT_GET_CLASS(RdmaProviderClass, (obj), \ + INTERFACE_RDMA_PROVIDER) +#define RDMA_PROVIDER(obj) \ + INTERFACE_CHECK(RdmaProvider, (obj), \ + INTERFACE_RDMA_PROVIDER) + +typedef struct RdmaProvider RdmaProvider; + +typedef struct RdmaProviderClass { + InterfaceClass parent; + + void (*print_statistics)(Monitor *mon, RdmaProvider *obj); +} RdmaProviderClass; + +#endif