From patchwork Thu Mar 29 16:10:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Wise X-Patchwork-Id: 893023 X-Patchwork-Delegate: dsahern@gmail.com Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=opengridcomputing.com Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40BytV1BvMz9s2R for ; Fri, 30 Mar 2018 08:43:46 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751398AbeC2Vno (ORCPT ); Thu, 29 Mar 2018 17:43:44 -0400 Received: from opengridcomputing.com ([70.118.0.34]:50404 "EHLO smtp.opengridcomputing.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752246AbeC2Vnm (ORCPT ); Thu, 29 Mar 2018 17:43:42 -0400 Received: by smtp.opengridcomputing.com (Postfix, from userid 503) id 970032BC23; Thu, 29 Mar 2018 16:43:41 -0500 (CDT) Message-Id: <8b0ef68809c44e8d92d47a7bf95a69f2d9743407.1522359537.git.swise@opengridcomputing.com> In-Reply-To: References: From: Steve Wise Date: Thu, 29 Mar 2018 09:10:30 -0700 Subject: [PATCH v4 iproute2-next 1/7] rdma: update rdma_netlink.h To: dsahern@gmail.com Cc: leon@kernel.org, stephen@networkplumber.org, netdev@vger.kernel.org, linux-rdma@vger.kernel.org Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Steve Wise Pull in the latest rdma_netlink.h which has support for the rdma nldev resource tracking objects being added with this patch series. Signed-off-by: Steve Wise Reviewed-by: Leon Romanovsky --- rdma/include/uapi/rdma/rdma_netlink.h | 38 +++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/rdma/include/uapi/rdma/rdma_netlink.h b/rdma/include/uapi/rdma/rdma_netlink.h index dbac3b8..9446a72 100644 --- a/rdma/include/uapi/rdma/rdma_netlink.h +++ b/rdma/include/uapi/rdma/rdma_netlink.h @@ -238,6 +238,14 @@ enum rdma_nldev_command { RDMA_NLDEV_CMD_RES_QP_GET, /* can dump */ + RDMA_NLDEV_CMD_RES_CM_ID_GET, /* can dump */ + + RDMA_NLDEV_CMD_RES_CQ_GET, /* can dump */ + + RDMA_NLDEV_CMD_RES_MR_GET, /* can dump */ + + RDMA_NLDEV_CMD_RES_PD_GET, /* can dump */ + RDMA_NLDEV_NUM_OPS }; @@ -350,6 +358,36 @@ enum rdma_nldev_attr { */ RDMA_NLDEV_ATTR_RES_KERN_NAME, /* string */ + RDMA_NLDEV_ATTR_RES_CM_ID, /* nested table */ + RDMA_NLDEV_ATTR_RES_CM_ID_ENTRY, /* nested table */ + /* + * rdma_cm_id port space. + */ + RDMA_NLDEV_ATTR_RES_PS, /* u32 */ + /* + * Source and destination socket addresses + */ + RDMA_NLDEV_ATTR_RES_SRC_ADDR, /* __kernel_sockaddr_storage */ + RDMA_NLDEV_ATTR_RES_DST_ADDR, /* __kernel_sockaddr_storage */ + + RDMA_NLDEV_ATTR_RES_CQ, /* nested table */ + RDMA_NLDEV_ATTR_RES_CQ_ENTRY, /* nested table */ + RDMA_NLDEV_ATTR_RES_CQE, /* u32 */ + RDMA_NLDEV_ATTR_RES_USECNT, /* u64 */ + RDMA_NLDEV_ATTR_RES_POLL_CTX, /* u8 */ + + RDMA_NLDEV_ATTR_RES_MR, /* nested table */ + RDMA_NLDEV_ATTR_RES_MR_ENTRY, /* nested table */ + RDMA_NLDEV_ATTR_RES_RKEY, /* u32 */ + RDMA_NLDEV_ATTR_RES_LKEY, /* u32 */ + RDMA_NLDEV_ATTR_RES_IOVA, /* u64 */ + RDMA_NLDEV_ATTR_RES_MRLEN, /* u64 */ + + RDMA_NLDEV_ATTR_RES_PD, /* nested table */ + RDMA_NLDEV_ATTR_RES_PD_ENTRY, /* nested table */ + RDMA_NLDEV_ATTR_RES_LOCAL_DMA_LKEY, /* u32 */ + RDMA_NLDEV_ATTR_RES_UNSAFE_GLOBAL_RKEY, /* u32 */ + RDMA_NLDEV_ATTR_MAX }; #endif /* _RDMA_NETLINK_H */ From patchwork Thu Mar 29 16:10:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Wise X-Patchwork-Id: 893025 X-Patchwork-Delegate: dsahern@gmail.com Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=opengridcomputing.com Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40BytY5tcVz9s1s for ; Fri, 30 Mar 2018 08:43:49 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752315AbeC2Vns (ORCPT ); Thu, 29 Mar 2018 17:43:48 -0400 Received: from opengridcomputing.com ([70.118.0.34]:50409 "EHLO smtp.opengridcomputing.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752278AbeC2Vnr (ORCPT ); Thu, 29 Mar 2018 17:43:47 -0400 Received: by smtp.opengridcomputing.com (Postfix, from userid 503) id A2F372BBEC; Thu, 29 Mar 2018 16:43:46 -0500 (CDT) Message-Id: <8fe917e01a80f115f795c92b1287afd0163e5089.1522359537.git.swise@opengridcomputing.com> In-Reply-To: References: From: Steve Wise Date: Thu, 29 Mar 2018 09:10:32 -0700 Subject: [PATCH v4 iproute2-next 2/7] rdma: add UAPI rdma_user_cm.h To: dsahern@gmail.com Cc: leon@kernel.org, stephen@networkplumber.org, netdev@vger.kernel.org, linux-rdma@vger.kernel.org Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This allows parsing rdma_cm_id UAPI values. Signed-off-by: Steve Wise --- rdma/include/uapi/rdma/rdma_user_cm.h | 324 ++++++++++++++++++++++++++++++++++ 1 file changed, 324 insertions(+) create mode 100644 rdma/include/uapi/rdma/rdma_user_cm.h diff --git a/rdma/include/uapi/rdma/rdma_user_cm.h b/rdma/include/uapi/rdma/rdma_user_cm.h new file mode 100644 index 0000000..da099af --- /dev/null +++ b/rdma/include/uapi/rdma/rdma_user_cm.h @@ -0,0 +1,324 @@ +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) */ +/* + * Copyright (c) 2005-2006 Intel Corporation. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +#ifndef _RDMA_USER_CM_H +#define _RDMA_USER_CM_H + +#include +#include +#include +#include +#include + +#define RDMA_USER_CM_ABI_VERSION 4 + +#define RDMA_MAX_PRIVATE_DATA 256 + +enum { + RDMA_USER_CM_CMD_CREATE_ID, + RDMA_USER_CM_CMD_DESTROY_ID, + RDMA_USER_CM_CMD_BIND_IP, + RDMA_USER_CM_CMD_RESOLVE_IP, + RDMA_USER_CM_CMD_RESOLVE_ROUTE, + RDMA_USER_CM_CMD_QUERY_ROUTE, + RDMA_USER_CM_CMD_CONNECT, + RDMA_USER_CM_CMD_LISTEN, + RDMA_USER_CM_CMD_ACCEPT, + RDMA_USER_CM_CMD_REJECT, + RDMA_USER_CM_CMD_DISCONNECT, + RDMA_USER_CM_CMD_INIT_QP_ATTR, + RDMA_USER_CM_CMD_GET_EVENT, + RDMA_USER_CM_CMD_GET_OPTION, + RDMA_USER_CM_CMD_SET_OPTION, + RDMA_USER_CM_CMD_NOTIFY, + RDMA_USER_CM_CMD_JOIN_IP_MCAST, + RDMA_USER_CM_CMD_LEAVE_MCAST, + RDMA_USER_CM_CMD_MIGRATE_ID, + RDMA_USER_CM_CMD_QUERY, + RDMA_USER_CM_CMD_BIND, + RDMA_USER_CM_CMD_RESOLVE_ADDR, + RDMA_USER_CM_CMD_JOIN_MCAST +}; + +/* See IBTA Annex A11, servies ID bytes 4 & 5 */ +enum rdma_ucm_port_space { + RDMA_PS_IPOIB = 0x0002, + RDMA_PS_IB = 0x013F, + RDMA_PS_TCP = 0x0106, + RDMA_PS_UDP = 0x0111, +}; + +/* + * command ABI structures. + */ +struct rdma_ucm_cmd_hdr { + __u32 cmd; + __u16 in; + __u16 out; +}; + +struct rdma_ucm_create_id { + __aligned_u64 uid; + __aligned_u64 response; + __u16 ps; /* use enum rdma_ucm_port_space */ + __u8 qp_type; + __u8 reserved[5]; +}; + +struct rdma_ucm_create_id_resp { + __u32 id; +}; + +struct rdma_ucm_destroy_id { + __aligned_u64 response; + __u32 id; + __u32 reserved; +}; + +struct rdma_ucm_destroy_id_resp { + __u32 events_reported; +}; + +struct rdma_ucm_bind_ip { + __aligned_u64 response; + struct sockaddr_in6 addr; + __u32 id; +}; + +struct rdma_ucm_bind { + __u32 id; + __u16 addr_size; + __u16 reserved; + struct __kernel_sockaddr_storage addr; +}; + +struct rdma_ucm_resolve_ip { + struct sockaddr_in6 src_addr; + struct sockaddr_in6 dst_addr; + __u32 id; + __u32 timeout_ms; +}; + +struct rdma_ucm_resolve_addr { + __u32 id; + __u32 timeout_ms; + __u16 src_size; + __u16 dst_size; + __u32 reserved; + struct __kernel_sockaddr_storage src_addr; + struct __kernel_sockaddr_storage dst_addr; +}; + +struct rdma_ucm_resolve_route { + __u32 id; + __u32 timeout_ms; +}; + +enum { + RDMA_USER_CM_QUERY_ADDR, + RDMA_USER_CM_QUERY_PATH, + RDMA_USER_CM_QUERY_GID +}; + +struct rdma_ucm_query { + __aligned_u64 response; + __u32 id; + __u32 option; +}; + +struct rdma_ucm_query_route_resp { + __aligned_u64 node_guid; + struct ib_user_path_rec ib_route[2]; + struct sockaddr_in6 src_addr; + struct sockaddr_in6 dst_addr; + __u32 num_paths; + __u8 port_num; + __u8 reserved[3]; +}; + +struct rdma_ucm_query_addr_resp { + __aligned_u64 node_guid; + __u8 port_num; + __u8 reserved; + __u16 pkey; + __u16 src_size; + __u16 dst_size; + struct __kernel_sockaddr_storage src_addr; + struct __kernel_sockaddr_storage dst_addr; +}; + +struct rdma_ucm_query_path_resp { + __u32 num_paths; + __u32 reserved; + struct ib_path_rec_data path_data[0]; +}; + +struct rdma_ucm_conn_param { + __u32 qp_num; + __u32 qkey; + __u8 private_data[RDMA_MAX_PRIVATE_DATA]; + __u8 private_data_len; + __u8 srq; + __u8 responder_resources; + __u8 initiator_depth; + __u8 flow_control; + __u8 retry_count; + __u8 rnr_retry_count; + __u8 valid; +}; + +struct rdma_ucm_ud_param { + __u32 qp_num; + __u32 qkey; + struct ib_uverbs_ah_attr ah_attr; + __u8 private_data[RDMA_MAX_PRIVATE_DATA]; + __u8 private_data_len; + __u8 reserved[7]; +}; + +struct rdma_ucm_connect { + struct rdma_ucm_conn_param conn_param; + __u32 id; + __u32 reserved; +}; + +struct rdma_ucm_listen { + __u32 id; + __u32 backlog; +}; + +struct rdma_ucm_accept { + __aligned_u64 uid; + struct rdma_ucm_conn_param conn_param; + __u32 id; + __u32 reserved; +}; + +struct rdma_ucm_reject { + __u32 id; + __u8 private_data_len; + __u8 reserved[3]; + __u8 private_data[RDMA_MAX_PRIVATE_DATA]; +}; + +struct rdma_ucm_disconnect { + __u32 id; +}; + +struct rdma_ucm_init_qp_attr { + __aligned_u64 response; + __u32 id; + __u32 qp_state; +}; + +struct rdma_ucm_notify { + __u32 id; + __u32 event; +}; + +struct rdma_ucm_join_ip_mcast { + __aligned_u64 response; /* rdma_ucm_create_id_resp */ + __aligned_u64 uid; + struct sockaddr_in6 addr; + __u32 id; +}; + +/* Multicast join flags */ +enum { + RDMA_MC_JOIN_FLAG_FULLMEMBER, + RDMA_MC_JOIN_FLAG_SENDONLY_FULLMEMBER, + RDMA_MC_JOIN_FLAG_RESERVED, +}; + +struct rdma_ucm_join_mcast { + __aligned_u64 response; /* rdma_ucma_create_id_resp */ + __aligned_u64 uid; + __u32 id; + __u16 addr_size; + __u16 join_flags; + struct __kernel_sockaddr_storage addr; +}; + +struct rdma_ucm_get_event { + __aligned_u64 response; +}; + +struct rdma_ucm_event_resp { + __aligned_u64 uid; + __u32 id; + __u32 event; + __u32 status; + /* + * NOTE: This union is not aligned to 8 bytes so none of the union + * members may contain a u64 or anything with higher alignment than 4. + */ + union { + struct rdma_ucm_conn_param conn; + struct rdma_ucm_ud_param ud; + } param; + __u32 reserved; +}; + +/* Option levels */ +enum { + RDMA_OPTION_ID = 0, + RDMA_OPTION_IB = 1 +}; + +/* Option details */ +enum { + RDMA_OPTION_ID_TOS = 0, + RDMA_OPTION_ID_REUSEADDR = 1, + RDMA_OPTION_ID_AFONLY = 2, + RDMA_OPTION_IB_PATH = 1 +}; + +struct rdma_ucm_set_option { + __aligned_u64 optval; + __u32 id; + __u32 level; + __u32 optname; + __u32 optlen; +}; + +struct rdma_ucm_migrate_id { + __aligned_u64 response; + __u32 id; + __u32 fd; +}; + +struct rdma_ucm_migrate_resp { + __u32 events_reported; +}; + +#endif /* _RDMA_USER_CM_H */ From patchwork Thu Mar 29 16:10:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Wise X-Patchwork-Id: 893026 X-Patchwork-Delegate: dsahern@gmail.com Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=opengridcomputing.com Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40Bytg0yYRz9s2R for ; Fri, 30 Mar 2018 08:43:55 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752367AbeC2Vnx (ORCPT ); Thu, 29 Mar 2018 17:43:53 -0400 Received: from opengridcomputing.com ([70.118.0.34]:50413 "EHLO smtp.opengridcomputing.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752326AbeC2Vnw (ORCPT ); Thu, 29 Mar 2018 17:43:52 -0400 Received: by smtp.opengridcomputing.com (Postfix, from userid 503) id ACC702BC23; Thu, 29 Mar 2018 16:43:51 -0500 (CDT) Message-Id: <0459489667e983bb3a6f9ecaf25daff6a329f2da.1522359537.git.swise@opengridcomputing.com> In-Reply-To: References: From: Steve Wise Date: Thu, 29 Mar 2018 09:10:35 -0700 Subject: [PATCH v4 iproute2-next 3/7] rdma: initialize the rd struct To: dsahern@gmail.com Cc: leon@kernel.org, stephen@networkplumber.org, netdev@vger.kernel.org, linux-rdma@vger.kernel.org Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Initialize the rd struct so port_idx is 0 unless set otherwise. Otherwise, strict_port queries end up passing an uninitialized PORT nlattr. Signed-off-by: Steve Wise Reviewed-by: Leon Romanovsky --- rdma/rdma.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rdma/rdma.c b/rdma/rdma.c index ab2c960..b43e538 100644 --- a/rdma/rdma.c +++ b/rdma/rdma.c @@ -135,7 +135,7 @@ int main(int argc, char **argv) bool json_output = false; bool force = false; char *filename; - struct rd rd; + struct rd rd = {}; int opt; int err; From patchwork Thu Mar 29 16:10:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Wise X-Patchwork-Id: 893027 X-Patchwork-Delegate: dsahern@gmail.com Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=opengridcomputing.com Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40Bytn54Gqz9s1s for ; Fri, 30 Mar 2018 08:44:01 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752484AbeC2Vn7 (ORCPT ); Thu, 29 Mar 2018 17:43:59 -0400 Received: from opengridcomputing.com ([70.118.0.34]:50430 "EHLO smtp.opengridcomputing.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752394AbeC2Vn5 (ORCPT ); Thu, 29 Mar 2018 17:43:57 -0400 Received: by smtp.opengridcomputing.com (Postfix, from userid 503) id B92032BC23; Thu, 29 Mar 2018 16:43:56 -0500 (CDT) Message-Id: <7b9c8884ee332781f637931aea808653ebafa82a.1522359537.git.swise@opengridcomputing.com> In-Reply-To: References: From: Steve Wise Date: Thu, 29 Mar 2018 09:10:37 -0700 Subject: [PATCH v4 iproute2-next 4/7] rdma: Add CM_ID resource tracking information To: dsahern@gmail.com Cc: leon@kernel.org, stephen@networkplumber.org, netdev@vger.kernel.org, linux-rdma@vger.kernel.org Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Sample output: # rdma resource 2: cxgb4_0: pd 5 cq 2 qp 2 cm_id 3 mr 7 3: mlx4_0: pd 7 cq 3 qp 3 cm_id 3 mr 7 # rdma resource show cm_id link cxgb4_0/- lqpn 0 qp-type RC state LISTEN ps TCP pid 30485 comm rping src-addr 0.0.0.0:7174 link cxgb4_0/2 lqpn 1048 qp-type RC state CONNECT ps TCP pid 30503 comm rping src-addr 172.16.2.1:7174 dst-addr 172.16.2.1:38246 link cxgb4_0/2 lqpn 1040 qp-type RC state CONNECT ps TCP pid 30498 comm rping src-addr 172.16.2.1:38246 dst-addr 172.16.2.1:7174 link mlx4_0/- lqpn 0 qp-type RC state LISTEN ps TCP pid 30485 comm rping src-addr 0.0.0.0:7174 link mlx4_0/1 lqpn 539 qp-type RC state CONNECT ps TCP pid 30494 comm rping src-addr 172.16.99.1:7174 dst-addr 172.16.99.1:43670 link mlx4_0/1 lqpn 538 qp-type RC state CONNECT ps TCP pid 30492 comm rping src-addr 172.16.99.1:43670 dst-addr 172.16.99.1:7174 # rdma resource show cm_id dst-port 7174 link cxgb4_0/2 lqpn 1040 qp-type RC state CONNECT ps TCP pid 30498 comm rping src-addr 172.16.2.1:38246 dst-addr 172.16.2.1:7174 link mlx4_0/1 lqpn 538 qp-type RC state CONNECT ps TCP pid 30492 comm rping src-addr 172.16.99.1:43670 dst-addr 172.16.99.1:7174 Signed-off-by: Steve Wise Reviewed-by: Leon Romanovsky --- rdma/rdma.h | 2 + rdma/res.c | 262 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- rdma/utils.c | 5 ++ 3 files changed, 268 insertions(+), 1 deletion(-) diff --git a/rdma/rdma.h b/rdma/rdma.h index 5809f70..1908fc4 100644 --- a/rdma/rdma.h +++ b/rdma/rdma.h @@ -17,7 +17,9 @@ #include #include #include +#include #include +#include #include "list.h" #include "utils.h" diff --git a/rdma/res.c b/rdma/res.c index 62f5c54..5506cf3 100644 --- a/rdma/res.c +++ b/rdma/res.c @@ -16,9 +16,11 @@ static int res_help(struct rd *rd) { pr_out("Usage: %s resource\n", rd->filename); pr_out(" resource show [DEV]\n"); - pr_out(" resource show [qp]\n"); + pr_out(" resource show [qp|cm_id]\n"); pr_out(" resource show qp link [DEV/PORT]\n"); pr_out(" resource show qp link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n"); + pr_out(" resource show cm_id link [DEV/PORT]\n"); + pr_out(" resource show cm_id link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n"); return 0; } @@ -433,6 +435,245 @@ static int res_qp_parse_cb(const struct nlmsghdr *nlh, void *data) return MNL_CB_OK; } +static void print_qp_type(struct rd *rd, uint32_t val) +{ + if (rd->json_output) + jsonw_string_field(rd->jw, "qp-type", + qp_types_to_str(val)); + else + pr_out("qp-type %s ", qp_types_to_str(val)); +} + +static const char *cm_id_state_to_str(uint8_t idx) +{ + static const char * const cm_id_states_str[] = { + "IDLE", "ADDR_QUERY", "ADDR_RESOLVED", "ROUTE_QUERY", + "ROUTE_RESOLVED", "CONNECT", "DISCONNECT", "ADDR_BOUND", + "LISTEN", "DEVICE_REMOVAL", "DESTROYING" }; + + if (idx < ARRAY_SIZE(cm_id_states_str)) + return cm_id_states_str[idx]; + return "UNKNOWN"; +} + +static const char *cm_id_ps_to_str(uint32_t ps) +{ + switch (ps) { + case RDMA_PS_IPOIB: + return "IPoIB"; + case RDMA_PS_IB: + return "IPoIB"; + case RDMA_PS_TCP: + return "TCP"; + case RDMA_PS_UDP: + return "UDP"; + default: + return "---"; + } +} + +static void print_cm_id_state(struct rd *rd, uint8_t state) +{ + if (rd->json_output) { + jsonw_string_field(rd->jw, "state", cm_id_state_to_str(state)); + return; + } + pr_out("state %s ", cm_id_state_to_str(state)); +} + +static void print_ps(struct rd *rd, uint32_t ps) +{ + if (rd->json_output) { + jsonw_string_field(rd->jw, "ps", cm_id_ps_to_str(ps)); + return; + } + pr_out("ps %s ", cm_id_ps_to_str(ps)); +} + +static void print_ipaddr(struct rd *rd, const char *key, char *addrstr, + uint16_t port) +{ + if (rd->json_output) { + int name_size = INET6_ADDRSTRLEN+strlen(":65535"); + char json_name[name_size]; + + snprintf(json_name, name_size, "%s:%u", addrstr, port); + jsonw_string_field(rd->jw, key, json_name); + return; + } + pr_out("%s %s:%u ", key, addrstr, port); +} + +static int ss_ntop(struct nlattr *nla_line, char *addr_str, uint16_t *port) +{ + struct __kernel_sockaddr_storage *addr; + + addr = (struct __kernel_sockaddr_storage *) + mnl_attr_get_payload(nla_line); + switch (addr->ss_family) { + case AF_INET: { + struct sockaddr_in *sin = (struct sockaddr_in *)addr; + + if (!inet_ntop(AF_INET, (const void *)&sin->sin_addr, addr_str, + INET6_ADDRSTRLEN)) + return -EINVAL; + *port = ntohs(sin->sin_port); + break; + } + case AF_INET6: { + struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)addr; + + if (!inet_ntop(AF_INET6, (const void *)&sin6->sin6_addr, + addr_str, INET6_ADDRSTRLEN)) + return -EINVAL; + *port = ntohs(sin6->sin6_port); + break; + } + default: + return -EINVAL; + } + return 0; +} + +static int res_cm_id_parse_cb(const struct nlmsghdr *nlh, void *data) +{ + struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {}; + struct nlattr *nla_table, *nla_entry; + struct rd *rd = data; + const char *name; + int idx; + + mnl_attr_parse(nlh, 0, rd_attr_cb, tb); + if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] || + !tb[RDMA_NLDEV_ATTR_DEV_NAME] || + !tb[RDMA_NLDEV_ATTR_RES_CM_ID]) + return MNL_CB_ERROR; + + name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]); + idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]); + nla_table = tb[RDMA_NLDEV_ATTR_RES_CM_ID]; + mnl_attr_for_each_nested(nla_entry, nla_table) { + struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {}; + char src_addr_str[INET6_ADDRSTRLEN]; + char dst_addr_str[INET6_ADDRSTRLEN]; + uint16_t src_port, dst_port; + uint32_t port = 0, pid = 0; + uint8_t type = 0, state; + uint32_t lqpn = 0, ps; + char *comm = NULL; + int err; + + err = mnl_attr_parse_nested(nla_entry, rd_attr_cb, nla_line); + if (err != MNL_CB_OK) + return -EINVAL; + + if (!nla_line[RDMA_NLDEV_ATTR_RES_STATE] || + !nla_line[RDMA_NLDEV_ATTR_RES_PS] || + (!nla_line[RDMA_NLDEV_ATTR_RES_PID] && + !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) { + return MNL_CB_ERROR; + } + + if (nla_line[RDMA_NLDEV_ATTR_PORT_INDEX]) + port = mnl_attr_get_u32( + nla_line[RDMA_NLDEV_ATTR_PORT_INDEX]); + + if (port && port != rd->port_idx) + continue; + + if (nla_line[RDMA_NLDEV_ATTR_RES_LQPN]) { + lqpn = mnl_attr_get_u32( + nla_line[RDMA_NLDEV_ATTR_RES_LQPN]); + if (rd_check_is_filtered(rd, "lqpn", lqpn)) + continue; + } + if (nla_line[RDMA_NLDEV_ATTR_RES_TYPE]) { + type = mnl_attr_get_u8( + nla_line[RDMA_NLDEV_ATTR_RES_TYPE]); + if (rd_check_is_string_filtered(rd, "qp-type", + qp_types_to_str(type))) + continue; + } + + ps = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PS]); + if (rd_check_is_string_filtered(rd, "ps", cm_id_ps_to_str(ps))) + continue; + + state = mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_STATE]); + if (rd_check_is_string_filtered(rd, "state", + cm_id_state_to_str(state))) + continue; + + if (nla_line[RDMA_NLDEV_ATTR_RES_SRC_ADDR]) { + if (ss_ntop(nla_line[RDMA_NLDEV_ATTR_RES_SRC_ADDR], + src_addr_str, &src_port)) + continue; + if (rd_check_is_string_filtered(rd, "src-addr", + src_addr_str)) + continue; + } + + if (nla_line[RDMA_NLDEV_ATTR_RES_DST_ADDR]) { + if (ss_ntop(nla_line[RDMA_NLDEV_ATTR_RES_DST_ADDR], + dst_addr_str, &dst_port)) + continue; + if (rd_check_is_string_filtered(rd, "dst-addr", + dst_addr_str)) + continue; + } + + if (rd_check_is_filtered(rd, "src-port", src_port)) + continue; + + if (rd_check_is_filtered(rd, "dst-port", dst_port)) + continue; + + if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) { + pid = mnl_attr_get_u32( + nla_line[RDMA_NLDEV_ATTR_RES_PID]); + comm = get_task_name(pid); + } + + if (rd_check_is_filtered(rd, "pid", pid)) { + free(comm); + continue; + } + + if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]) { + /* discard const from mnl_attr_get_str */ + comm = (char *)mnl_attr_get_str( + nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]); + } + + if (rd->json_output) + jsonw_start_array(rd->jw); + + print_link(rd, idx, name, port, nla_line); + if (nla_line[RDMA_NLDEV_ATTR_RES_LQPN]) + print_lqpn(rd, lqpn); + if (nla_line[RDMA_NLDEV_ATTR_RES_TYPE]) + print_qp_type(rd, type); + print_cm_id_state(rd, state); + print_ps(rd, ps); + print_pid(rd, pid); + print_comm(rd, comm, nla_line); + + if (nla_line[RDMA_NLDEV_ATTR_RES_SRC_ADDR]) + print_ipaddr(rd, "src-addr", src_addr_str, src_port); + if (nla_line[RDMA_NLDEV_ATTR_RES_DST_ADDR]) + print_ipaddr(rd, "dst-addr", dst_addr_str, dst_port); + + if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) + free(comm); + + if (rd->json_output) + jsonw_end_array(rd->jw); + else + pr_out("\n"); + } + return MNL_CB_OK; +} + RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true); static const struct @@ -457,11 +698,30 @@ filters qp_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link", RES_FUNC(res_qp, RDMA_NLDEV_CMD_RES_QP_GET, qp_valid_filters, false); +static const +struct filters cm_id_valid_filters[MAX_NUMBER_OF_FILTERS] = { + { .name = "link", .is_number = false }, + { .name = "lqpn", .is_number = true }, + { .name = "qp-type", .is_number = false }, + { .name = "state", .is_number = false }, + { .name = "ps", .is_number = false }, + { .name = "dev-type", .is_number = false }, + { .name = "transport-type", .is_number = false }, + { .name = "pid", .is_number = true }, + { .name = "src-addr", .is_number = false }, + { .name = "src-port", .is_number = true }, + { .name = "dst-addr", .is_number = false }, + { .name = "dst-port", .is_number = true } +}; + +RES_FUNC(res_cm_id, RDMA_NLDEV_CMD_RES_CM_ID_GET, cm_id_valid_filters, false); + static int res_show(struct rd *rd) { const struct rd_cmd cmds[] = { { NULL, res_no_args }, { "qp", res_qp }, + { "cm_id", res_cm_id }, { 0 } }; diff --git a/rdma/utils.c b/rdma/utils.c index f946016..ec81737 100644 --- a/rdma/utils.c +++ b/rdma/utils.c @@ -375,6 +375,11 @@ static const enum mnl_attr_data_type nldev_policy[RDMA_NLDEV_ATTR_MAX] = { [RDMA_NLDEV_ATTR_RES_STATE] = MNL_TYPE_U8, [RDMA_NLDEV_ATTR_RES_PID] = MNL_TYPE_U32, [RDMA_NLDEV_ATTR_RES_KERN_NAME] = MNL_TYPE_NUL_STRING, + [RDMA_NLDEV_ATTR_RES_CM_ID] = MNL_TYPE_NESTED, + [RDMA_NLDEV_ATTR_RES_CM_ID_ENTRY] = MNL_TYPE_NESTED, + [RDMA_NLDEV_ATTR_RES_PS] = MNL_TYPE_U32, + [RDMA_NLDEV_ATTR_RES_SRC_ADDR] = MNL_TYPE_UNSPEC, + [RDMA_NLDEV_ATTR_RES_DST_ADDR] = MNL_TYPE_UNSPEC, }; int rd_attr_cb(const struct nlattr *attr, void *data) From patchwork Thu Mar 29 16:10:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Wise X-Patchwork-Id: 893028 X-Patchwork-Delegate: dsahern@gmail.com Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=opengridcomputing.com Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40Bytt6pQTz9s1s for ; Fri, 30 Mar 2018 08:44:06 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752500AbeC2VoE (ORCPT ); Thu, 29 Mar 2018 17:44:04 -0400 Received: from opengridcomputing.com ([70.118.0.34]:50439 "EHLO smtp.opengridcomputing.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752486AbeC2VoD (ORCPT ); Thu, 29 Mar 2018 17:44:03 -0400 Received: by smtp.opengridcomputing.com (Postfix, from userid 503) id C322A2BC23; Thu, 29 Mar 2018 16:44:01 -0500 (CDT) Message-Id: In-Reply-To: References: From: Steve Wise Date: Thu, 29 Mar 2018 09:10:39 -0700 Subject: [PATCH v4 iproute2-next 5/7] rdma: Add CQ resource tracking information To: dsahern@gmail.com Cc: leon@kernel.org, stephen@networkplumber.org, netdev@vger.kernel.org, linux-rdma@vger.kernel.org Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Sample output: # rdma resource show cq dev cxgb4_0 cqe 46 users 2 pid 30503 comm rping dev cxgb4_0 cqe 46 users 2 pid 30498 comm rping dev mlx4_0 cqe 63 users 2 pid 30494 comm rping dev mlx4_0 cqe 63 users 2 pid 30489 comm rping dev mlx4_0 cqe 1023 users 2 poll_ctx WORKQUEUE pid 0 comm [ib_core] # rdma resource show cq pid 30489 dev mlx4_0 cqe 63 users 2 pid 30489 comm rping Signed-off-by: Steve Wise Reviewed-by: Leon Romanovsky --- rdma/res.c | 149 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ rdma/utils.c | 5 ++ 2 files changed, 154 insertions(+) diff --git a/rdma/res.c b/rdma/res.c index 5506cf3..bb5f3dd 100644 --- a/rdma/res.c +++ b/rdma/res.c @@ -21,6 +21,8 @@ static int res_help(struct rd *rd) pr_out(" resource show qp link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n"); pr_out(" resource show cm_id link [DEV/PORT]\n"); pr_out(" resource show cm_id link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n"); + pr_out(" resource show cq link [DEV/PORT]\n"); + pr_out(" resource show cq link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n"); return 0; } @@ -265,6 +267,16 @@ static void print_comm(struct rd *rd, const char *str, pr_out("comm %s ", tmp); } +static void print_dev(struct rd *rd, uint32_t idx, const char *name) +{ + if (rd->json_output) { + jsonw_uint_field(rd->jw, "ifindex", idx); + jsonw_string_field(rd->jw, "ifname", name); + } else { + pr_out("dev %s ", name); + } +} + static void print_link(struct rd *rd, uint32_t idx, const char *name, uint32_t port, struct nlattr **nla_line) { @@ -674,6 +686,132 @@ static int res_cm_id_parse_cb(const struct nlmsghdr *nlh, void *data) return MNL_CB_OK; } +static void print_cqe(struct rd *rd, uint32_t val) +{ + if (rd->json_output) + jsonw_uint_field(rd->jw, "cqe", val); + else + pr_out("cqe %u ", val); +} + +static void print_users(struct rd *rd, uint64_t val) +{ + if (rd->json_output) + jsonw_uint_field(rd->jw, "users", val); + else + pr_out("users %" PRIu64 " ", val); +} + +static const char *poll_ctx_to_str(uint8_t idx) +{ + static const char * const cm_id_states_str[] = { + "DIRECT", "SOFTIRQ", "WORKQUEUE"}; + + if (idx < ARRAY_SIZE(cm_id_states_str)) + return cm_id_states_str[idx]; + return "UNKNOWN"; +} + +static void print_poll_ctx(struct rd *rd, uint8_t poll_ctx) +{ + if (rd->json_output) { + jsonw_string_field(rd->jw, "poll-ctx", + poll_ctx_to_str(poll_ctx)); + return; + } + pr_out("poll-ctx %s ", poll_ctx_to_str(poll_ctx)); +} + +static int res_cq_parse_cb(const struct nlmsghdr *nlh, void *data) +{ + struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {}; + struct nlattr *nla_table, *nla_entry; + struct rd *rd = data; + const char *name; + uint32_t idx; + + mnl_attr_parse(nlh, 0, rd_attr_cb, tb); + if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] || + !tb[RDMA_NLDEV_ATTR_DEV_NAME] || + !tb[RDMA_NLDEV_ATTR_RES_CQ]) + return MNL_CB_ERROR; + + name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]); + idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]); + nla_table = tb[RDMA_NLDEV_ATTR_RES_CQ]; + + mnl_attr_for_each_nested(nla_entry, nla_table) { + struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {}; + char *comm = NULL; + uint32_t pid = 0; + uint8_t poll_ctx = 0; + uint64_t users; + uint32_t cqe; + int err; + + err = mnl_attr_parse_nested(nla_entry, rd_attr_cb, nla_line); + if (err != MNL_CB_OK) + return MNL_CB_ERROR; + + if (!nla_line[RDMA_NLDEV_ATTR_RES_CQE] || + !nla_line[RDMA_NLDEV_ATTR_RES_USECNT] || + (!nla_line[RDMA_NLDEV_ATTR_RES_PID] && + !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) { + return MNL_CB_ERROR; + } + + cqe = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_CQE]); + + users = mnl_attr_get_u64(nla_line[RDMA_NLDEV_ATTR_RES_USECNT]); + if (rd_check_is_filtered(rd, "users", users)) + continue; + + if (nla_line[RDMA_NLDEV_ATTR_RES_POLL_CTX]) { + poll_ctx = mnl_attr_get_u8( + nla_line[RDMA_NLDEV_ATTR_RES_POLL_CTX]); + if (rd_check_is_string_filtered(rd, "poll-ctx", + poll_ctx_to_str(poll_ctx))) + continue; + } + + if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) { + pid = mnl_attr_get_u32( + nla_line[RDMA_NLDEV_ATTR_RES_PID]); + comm = get_task_name(pid); + } + + if (rd_check_is_filtered(rd, "pid", pid)) { + free(comm); + continue; + } + + if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]) + /* discard const from mnl_attr_get_str */ + comm = (char *)mnl_attr_get_str( + nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]); + + if (rd->json_output) + jsonw_start_array(rd->jw); + + print_dev(rd, idx, name); + print_cqe(rd, cqe); + print_users(rd, users); + if (nla_line[RDMA_NLDEV_ATTR_RES_POLL_CTX]) + print_poll_ctx(rd, poll_ctx); + print_pid(rd, pid); + print_comm(rd, comm, nla_line); + + if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) + free(comm); + + if (rd->json_output) + jsonw_end_array(rd->jw); + else + pr_out("\n"); + } + return MNL_CB_OK; +} + RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true); static const struct @@ -716,12 +854,23 @@ struct filters cm_id_valid_filters[MAX_NUMBER_OF_FILTERS] = { RES_FUNC(res_cm_id, RDMA_NLDEV_CMD_RES_CM_ID_GET, cm_id_valid_filters, false); +static const +struct filters cq_valid_filters[MAX_NUMBER_OF_FILTERS] = { + { .name = "dev", .is_number = false }, + { .name = "users", .is_number = true }, + { .name = "poll-ctx", .is_number = false }, + { .name = "pid", .is_number = true } +}; + +RES_FUNC(res_cq, RDMA_NLDEV_CMD_RES_CQ_GET, cq_valid_filters, true); + static int res_show(struct rd *rd) { const struct rd_cmd cmds[] = { { NULL, res_no_args }, { "qp", res_qp }, { "cm_id", res_cm_id }, + { "cq", res_cq }, { 0 } }; diff --git a/rdma/utils.c b/rdma/utils.c index ec81737..5e79b62 100644 --- a/rdma/utils.c +++ b/rdma/utils.c @@ -380,6 +380,11 @@ static const enum mnl_attr_data_type nldev_policy[RDMA_NLDEV_ATTR_MAX] = { [RDMA_NLDEV_ATTR_RES_PS] = MNL_TYPE_U32, [RDMA_NLDEV_ATTR_RES_SRC_ADDR] = MNL_TYPE_UNSPEC, [RDMA_NLDEV_ATTR_RES_DST_ADDR] = MNL_TYPE_UNSPEC, + [RDMA_NLDEV_ATTR_RES_CQ] = MNL_TYPE_NESTED, + [RDMA_NLDEV_ATTR_RES_CQ_ENTRY] = MNL_TYPE_NESTED, + [RDMA_NLDEV_ATTR_RES_CQE] = MNL_TYPE_U32, + [RDMA_NLDEV_ATTR_RES_USECNT] = MNL_TYPE_U64, + [RDMA_NLDEV_ATTR_RES_POLL_CTX] = MNL_TYPE_U8, }; int rd_attr_cb(const struct nlattr *attr, void *data) From patchwork Thu Mar 29 16:10:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Wise X-Patchwork-Id: 893029 X-Patchwork-Delegate: dsahern@gmail.com Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=opengridcomputing.com Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40Bytz2gT1z9s1s for ; Fri, 30 Mar 2018 08:44:11 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752530AbeC2VoI (ORCPT ); Thu, 29 Mar 2018 17:44:08 -0400 Received: from opengridcomputing.com ([70.118.0.34]:50448 "EHLO smtp.opengridcomputing.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752116AbeC2VoH (ORCPT ); Thu, 29 Mar 2018 17:44:07 -0400 Received: by smtp.opengridcomputing.com (Postfix, from userid 503) id CD6302BC4C; Thu, 29 Mar 2018 16:44:06 -0500 (CDT) Message-Id: In-Reply-To: References: From: Steve Wise Date: Thu, 29 Mar 2018 09:10:41 -0700 Subject: [PATCH v4 iproute2-next 6/7] rdma: Add MR resource tracking information To: dsahern@gmail.com Cc: leon@kernel.org, stephen@networkplumber.org, netdev@vger.kernel.org, linux-rdma@vger.kernel.org Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Sample output: Without CAP_NET_ADMIN: $ rdma resource show mr mrlen 65536 dev mlx4_0 mrlen 65536 pid 0 comm [nvme_rdma] dev cxgb4_0 mrlen 65536 pid 0 comm [nvme_rdma] With CAP_NET_ADMIN: # rdma resource show mr mrlen 65536 dev mlx4_0 rkey 0x12702 lkey 0x12702 iova 0x85724a000 mrlen 65536 pid 0 comm [nvme_rdma] dev cxgb4_0 rkey 0x68fe4e9 lkey 0x68fe4e9 iova 0x835b91000 mrlen 65536 pid 0 comm [nvme_rdma] Signed-off-by: Steve Wise Reviewed-by: Leon Romanovsky --- include/json_writer.h | 2 + lib/json_writer.c | 11 +++++ rdma/res.c | 127 ++++++++++++++++++++++++++++++++++++++++++++++++++ rdma/utils.c | 6 +++ 4 files changed, 146 insertions(+) diff --git a/include/json_writer.h b/include/json_writer.h index 45459fa..4b4dec2 100644 --- a/include/json_writer.h +++ b/include/json_writer.h @@ -35,6 +35,7 @@ void jsonw_bool(json_writer_t *self, bool value); void jsonw_float(json_writer_t *self, double number); void jsonw_float_fmt(json_writer_t *self, const char *fmt, double num); void jsonw_uint(json_writer_t *self, uint64_t number); +void jsonw_xint(json_writer_t *self, uint64_t number); void jsonw_hu(json_writer_t *self, unsigned short number); void jsonw_int(json_writer_t *self, int64_t number); void jsonw_null(json_writer_t *self); @@ -45,6 +46,7 @@ void jsonw_string_field(json_writer_t *self, const char *prop, const char *val); void jsonw_bool_field(json_writer_t *self, const char *prop, bool value); void jsonw_float_field(json_writer_t *self, const char *prop, double num); void jsonw_uint_field(json_writer_t *self, const char *prop, uint64_t num); +void jsonw_xint_field(json_writer_t *self, const char *prop, uint64_t num); void jsonw_hu_field(json_writer_t *self, const char *prop, unsigned short num); void jsonw_int_field(json_writer_t *self, const char *prop, int64_t num); void jsonw_null_field(json_writer_t *self, const char *prop); diff --git a/lib/json_writer.c b/lib/json_writer.c index 68401ae..0ad0421 100644 --- a/lib/json_writer.c +++ b/lib/json_writer.c @@ -225,6 +225,11 @@ void jsonw_uint(json_writer_t *self, uint64_t num) jsonw_printf(self, "%"PRIu64, num); } +void jsonw_xint(json_writer_t *self, uint64_t num) +{ + jsonw_printf(self, "%"PRIx64, num); +} + void jsonw_lluint(json_writer_t *self, unsigned long long int num) { jsonw_printf(self, "%llu", num); @@ -269,6 +274,12 @@ void jsonw_uint_field(json_writer_t *self, const char *prop, uint64_t num) jsonw_uint(self, num); } +void jsonw_xint_field(json_writer_t *self, const char *prop, uint64_t num) +{ + jsonw_name(self, prop); + jsonw_xint(self, num); +} + void jsonw_hu_field(json_writer_t *self, const char *prop, unsigned short num) { jsonw_name(self, prop); diff --git a/rdma/res.c b/rdma/res.c index bb5f3dd..9c1f736 100644 --- a/rdma/res.c +++ b/rdma/res.c @@ -812,6 +812,121 @@ static int res_cq_parse_cb(const struct nlmsghdr *nlh, void *data) return MNL_CB_OK; } +static void print_key(struct rd *rd, const char *name, uint32_t val) +{ + if (rd->json_output) + jsonw_xint_field(rd->jw, name, val); + else + pr_out("%s 0x%x ", name, val); +} + +static void print_iova(struct rd *rd, uint64_t val) +{ + if (rd->json_output) + jsonw_xint_field(rd->jw, "iova", val); + else + pr_out("iova 0x%" PRIx64 " ", val); +} + +static void print_mrlen(struct rd *rd, uint64_t val) +{ + if (rd->json_output) + jsonw_uint_field(rd->jw, "mrlen", val); + else + pr_out("mrlen %" PRIu64 " ", val); +} + +static int res_mr_parse_cb(const struct nlmsghdr *nlh, void *data) +{ + struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {}; + struct nlattr *nla_table, *nla_entry; + struct rd *rd = data; + const char *name; + uint32_t idx; + + mnl_attr_parse(nlh, 0, rd_attr_cb, tb); + if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] || + !tb[RDMA_NLDEV_ATTR_DEV_NAME] || + !tb[RDMA_NLDEV_ATTR_RES_MR]) + return MNL_CB_ERROR; + + name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]); + idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]); + nla_table = tb[RDMA_NLDEV_ATTR_RES_MR]; + + mnl_attr_for_each_nested(nla_entry, nla_table) { + struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {}; + uint32_t rkey = 0, lkey = 0; + uint64_t iova = 0, mrlen; + char *comm = NULL; + uint32_t pid = 0; + int err; + + err = mnl_attr_parse_nested(nla_entry, rd_attr_cb, nla_line); + if (err != MNL_CB_OK) + return MNL_CB_ERROR; + + if (!nla_line[RDMA_NLDEV_ATTR_RES_MRLEN] || + (!nla_line[RDMA_NLDEV_ATTR_RES_PID] && + !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) { + return MNL_CB_ERROR; + } + + if (nla_line[RDMA_NLDEV_ATTR_RES_RKEY]) + rkey = mnl_attr_get_u32( + nla_line[RDMA_NLDEV_ATTR_RES_RKEY]); + if (nla_line[RDMA_NLDEV_ATTR_RES_LKEY]) + lkey = mnl_attr_get_u32( + nla_line[RDMA_NLDEV_ATTR_RES_LKEY]); + if (nla_line[RDMA_NLDEV_ATTR_RES_IOVA]) + iova = mnl_attr_get_u64( + nla_line[RDMA_NLDEV_ATTR_RES_IOVA]); + + mrlen = mnl_attr_get_u64(nla_line[RDMA_NLDEV_ATTR_RES_MRLEN]); + if (rd_check_is_filtered(rd, "mrlen", mrlen)) + continue; + + if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) { + pid = mnl_attr_get_u32( + nla_line[RDMA_NLDEV_ATTR_RES_PID]); + comm = get_task_name(pid); + } + + if (rd_check_is_filtered(rd, "pid", pid)) { + free(comm); + continue; + } + + if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]) + /* discard const from mnl_attr_get_str */ + comm = (char *)mnl_attr_get_str( + nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]); + + if (rd->json_output) + jsonw_start_array(rd->jw); + + print_dev(rd, idx, name); + if (nla_line[RDMA_NLDEV_ATTR_RES_RKEY]) + print_key(rd, "rkey", rkey); + if (nla_line[RDMA_NLDEV_ATTR_RES_LKEY]) + print_key(rd, "lkey", lkey); + if (nla_line[RDMA_NLDEV_ATTR_RES_IOVA]) + print_iova(rd, iova); + print_mrlen(rd, mrlen); + print_pid(rd, pid); + print_comm(rd, comm, nla_line); + + if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) + free(comm); + + if (rd->json_output) + jsonw_end_array(rd->jw); + else + pr_out("\n"); + } + return MNL_CB_OK; +} + RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true); static const struct @@ -864,6 +979,17 @@ struct filters cq_valid_filters[MAX_NUMBER_OF_FILTERS] = { RES_FUNC(res_cq, RDMA_NLDEV_CMD_RES_CQ_GET, cq_valid_filters, true); +static const +struct filters mr_valid_filters[MAX_NUMBER_OF_FILTERS] = { + { .name = "dev", .is_number = false }, + { .name = "rkey", .is_number = true }, + { .name = "lkey", .is_number = true }, + { .name = "mrlen", .is_number = true }, + { .name = "pid", .is_number = true } +}; + +RES_FUNC(res_mr, RDMA_NLDEV_CMD_RES_MR_GET, mr_valid_filters, true); + static int res_show(struct rd *rd) { const struct rd_cmd cmds[] = { @@ -871,6 +997,7 @@ static int res_show(struct rd *rd) { "qp", res_qp }, { "cm_id", res_cm_id }, { "cq", res_cq }, + { "mr", res_mr }, { 0 } }; diff --git a/rdma/utils.c b/rdma/utils.c index 5e79b62..a2e08e9 100644 --- a/rdma/utils.c +++ b/rdma/utils.c @@ -385,6 +385,12 @@ static const enum mnl_attr_data_type nldev_policy[RDMA_NLDEV_ATTR_MAX] = { [RDMA_NLDEV_ATTR_RES_CQE] = MNL_TYPE_U32, [RDMA_NLDEV_ATTR_RES_USECNT] = MNL_TYPE_U64, [RDMA_NLDEV_ATTR_RES_POLL_CTX] = MNL_TYPE_U8, + [RDMA_NLDEV_ATTR_RES_MR] = MNL_TYPE_NESTED, + [RDMA_NLDEV_ATTR_RES_MR_ENTRY] = MNL_TYPE_NESTED, + [RDMA_NLDEV_ATTR_RES_RKEY] = MNL_TYPE_U32, + [RDMA_NLDEV_ATTR_RES_LKEY] = MNL_TYPE_U32, + [RDMA_NLDEV_ATTR_RES_IOVA] = MNL_TYPE_U64, + [RDMA_NLDEV_ATTR_RES_MRLEN] = MNL_TYPE_U64, }; int rd_attr_cb(const struct nlattr *attr, void *data) From patchwork Thu Mar 29 16:10:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Wise X-Patchwork-Id: 893030 X-Patchwork-Delegate: dsahern@gmail.com Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=opengridcomputing.com Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40Byv36Cspz9s1s for ; Fri, 30 Mar 2018 08:44:15 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752545AbeC2VoN (ORCPT ); Thu, 29 Mar 2018 17:44:13 -0400 Received: from opengridcomputing.com ([70.118.0.34]:50456 "EHLO smtp.opengridcomputing.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752116AbeC2VoM (ORCPT ); Thu, 29 Mar 2018 17:44:12 -0400 Received: by smtp.opengridcomputing.com (Postfix, from userid 503) id D76D92BC23; Thu, 29 Mar 2018 16:44:11 -0500 (CDT) Message-Id: <9745441dbbcbbd3e0b4ac12c2930c752cf35c342.1522359537.git.swise@opengridcomputing.com> In-Reply-To: References: From: Steve Wise Date: Thu, 29 Mar 2018 09:10:44 -0700 Subject: [PATCH v4 iproute2-next 7/7] rdma: Add PD resource tracking information To: dsahern@gmail.com Cc: leon@kernel.org, stephen@networkplumber.org, netdev@vger.kernel.org, linux-rdma@vger.kernel.org Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Sample output: Without CAP_NET_ADMIN capability: dev mlx4_0 users 0 pid 0 comm [ib_srpt] dev mlx4_0 users 0 pid 0 comm [ib_srp] dev mlx4_0 users 1 pid 0 comm [ib_core] dev cxgb4_0 users 0 pid 0 comm [ib_srp] With CAP_NET_ADMIN capability: dev mlx4_0 local_dma_lkey 0x8000 users 0 pid 0 comm [ib_srpt] dev mlx4_0 local_dma_lkey 0x8000 users 0 pid 0 comm [ib_srp] dev mlx4_0 local_dma_lkey 0x8000 users 1 pid 0 comm [ib_core] dev cxgb4_0 local_dma_lkey 0x0 users 0 pid 0 comm [ib_srp] Signed-off-by: Steve Wise Reviewed-by: Leon Romanovsky --- rdma/res.c | 95 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) diff --git a/rdma/res.c b/rdma/res.c index 9c1f736..1a0aab6 100644 --- a/rdma/res.c +++ b/rdma/res.c @@ -927,6 +927,91 @@ static int res_mr_parse_cb(const struct nlmsghdr *nlh, void *data) return MNL_CB_OK; } +static int res_pd_parse_cb(const struct nlmsghdr *nlh, void *data) +{ + struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {}; + struct nlattr *nla_table, *nla_entry; + struct rd *rd = data; + const char *name; + uint32_t idx; + + mnl_attr_parse(nlh, 0, rd_attr_cb, tb); + if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] || + !tb[RDMA_NLDEV_ATTR_DEV_NAME] || + !tb[RDMA_NLDEV_ATTR_RES_PD]) + return MNL_CB_ERROR; + + name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]); + idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]); + nla_table = tb[RDMA_NLDEV_ATTR_RES_PD]; + + mnl_attr_for_each_nested(nla_entry, nla_table) { + uint32_t local_dma_lkey = 0, unsafe_global_rkey = 0; + struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {}; + char *comm = NULL; + uint32_t pid = 0; + uint64_t users; + int err; + + err = mnl_attr_parse_nested(nla_entry, rd_attr_cb, nla_line); + if (err != MNL_CB_OK) + return MNL_CB_ERROR; + + if (!nla_line[RDMA_NLDEV_ATTR_RES_USECNT] || + (!nla_line[RDMA_NLDEV_ATTR_RES_PID] && + !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) { + return MNL_CB_ERROR; + } + + if (nla_line[RDMA_NLDEV_ATTR_RES_LOCAL_DMA_LKEY]) + local_dma_lkey = mnl_attr_get_u32( + nla_line[RDMA_NLDEV_ATTR_RES_LOCAL_DMA_LKEY]); + + users = mnl_attr_get_u64(nla_line[RDMA_NLDEV_ATTR_RES_USECNT]); + if (rd_check_is_filtered(rd, "users", users)) + continue; + + if (nla_line[RDMA_NLDEV_ATTR_RES_UNSAFE_GLOBAL_RKEY]) + unsafe_global_rkey = mnl_attr_get_u32( + nla_line[RDMA_NLDEV_ATTR_RES_UNSAFE_GLOBAL_RKEY]); + + if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) { + pid = mnl_attr_get_u32( + nla_line[RDMA_NLDEV_ATTR_RES_PID]); + comm = get_task_name(pid); + } + + if (rd_check_is_filtered(rd, "pid", pid)) + continue; + + if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]) + /* discard const from mnl_attr_get_str */ + comm = (char *)mnl_attr_get_str( + nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]); + + if (rd->json_output) + jsonw_start_array(rd->jw); + + print_dev(rd, idx, name); + if (nla_line[RDMA_NLDEV_ATTR_RES_LOCAL_DMA_LKEY]) + print_key(rd, "local_dma_lkey", local_dma_lkey); + print_users(rd, users); + if (nla_line[RDMA_NLDEV_ATTR_RES_UNSAFE_GLOBAL_RKEY]) + print_key(rd, "unsafe_global_rkey", unsafe_global_rkey); + print_pid(rd, pid); + print_comm(rd, comm, nla_line); + + if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) + free(comm); + + if (rd->json_output) + jsonw_end_array(rd->jw); + else + pr_out("\n"); + } + return MNL_CB_OK; +} + RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true); static const struct @@ -990,6 +1075,15 @@ struct filters mr_valid_filters[MAX_NUMBER_OF_FILTERS] = { RES_FUNC(res_mr, RDMA_NLDEV_CMD_RES_MR_GET, mr_valid_filters, true); +static const +struct filters pd_valid_filters[MAX_NUMBER_OF_FILTERS] = { + { .name = "dev", .is_number = false }, + { .name = "users", .is_number = true }, + { .name = "pid", .is_number = true } +}; + +RES_FUNC(res_pd, RDMA_NLDEV_CMD_RES_PD_GET, pd_valid_filters, true); + static int res_show(struct rd *rd) { const struct rd_cmd cmds[] = { @@ -998,6 +1092,7 @@ static int res_show(struct rd *rd) { "cm_id", res_cm_id }, { "cq", res_cq }, { "mr", res_mr }, + { "pd", res_pd }, { 0 } };