From patchwork Tue Aug 6 07:48:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 1142645 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.b="WvJIqCmB"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 462mw31bF2z9sDB for ; Tue, 6 Aug 2019 17:48:19 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731918AbfHFHsS (ORCPT ); Tue, 6 Aug 2019 03:48:18 -0400 Received: from mail.kernel.org ([198.145.29.99]:44256 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728886AbfHFHsR (ORCPT ); Tue, 6 Aug 2019 03:48:17 -0400 Received: from localhost (unknown [193.47.165.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 75CCD206A2; Tue, 6 Aug 2019 07:48:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1565077697; bh=13CP357uqJDYp2yGgrBzeBL4lf24AxhuZ0UF4+J5Oew=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WvJIqCmBI8McAtvN9M/zBIloMYeAmM7+kCdqZvc4unXllEYZg3DTJdC8oVUz6XtTz VCj9G6kY4Suh6eK7N5ZxItbn460WyRd6Ju6Pm30KUjcJz8UT3NVbKEXIcxr6zo0s/e XiN5QzL29cldTZUr94H7xO9VExI1hCY3ocyHzKVM= From: Leon Romanovsky To: Doug Ledford , Jason Gunthorpe Cc: Leon Romanovsky , RDMA mailing list , Michael Guralnik , Moni Shoua , Saeed Mahameed , linux-netdev Subject: [PATCH mlx5-next v2 1/4] net/mlx5: Set ODP capabilities for DC transport Date: Tue, 6 Aug 2019 10:48:04 +0300 Message-Id: <20190806074807.9111-2-leon@kernel.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190806074807.9111-1-leon@kernel.org> References: <20190806074807.9111-1-leon@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Michael Guralnik Query ODP capabilities for DC transport from FW. Signed-off-by: Michael Guralnik Signed-off-by: Leon Romanovsky --- drivers/net/ethernet/mellanox/mlx5/core/main.c | 6 ++++++ include/linux/mlx5/mlx5_ifc.h | 4 +++- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c index b15b27a497fc..3995fc6d4d34 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -495,6 +495,12 @@ static int handle_hca_cap_odp(struct mlx5_core_dev *dev) ODP_CAP_SET_MAX(dev, xrc_odp_caps.write); ODP_CAP_SET_MAX(dev, xrc_odp_caps.read); ODP_CAP_SET_MAX(dev, xrc_odp_caps.atomic); + ODP_CAP_SET_MAX(dev, dc_odp_caps.srq_receive); + ODP_CAP_SET_MAX(dev, dc_odp_caps.send); + ODP_CAP_SET_MAX(dev, dc_odp_caps.receive); + ODP_CAP_SET_MAX(dev, dc_odp_caps.write); + ODP_CAP_SET_MAX(dev, dc_odp_caps.read); + ODP_CAP_SET_MAX(dev, dc_odp_caps.atomic); if (do_set) err = set_caps(dev, set_ctx, set_sz, diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h index ec571fd7fcf8..b96d5c37f70f 100644 --- a/include/linux/mlx5/mlx5_ifc.h +++ b/include/linux/mlx5/mlx5_ifc.h @@ -944,7 +944,9 @@ struct mlx5_ifc_odp_cap_bits { struct mlx5_ifc_odp_per_transport_service_cap_bits xrc_odp_caps; - u8 reserved_at_100[0x700]; + struct mlx5_ifc_odp_per_transport_service_cap_bits dc_odp_caps; + + u8 reserved_at_120[0x6E0]; }; struct mlx5_ifc_calc_op { From patchwork Tue Aug 6 07:48:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 1142646 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.b="w4Tk9XAx"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 462mw81W8jz9sDB for ; Tue, 6 Aug 2019 17:48:24 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731979AbfHFHsX (ORCPT ); Tue, 6 Aug 2019 03:48:23 -0400 Received: from mail.kernel.org ([198.145.29.99]:44280 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726834AbfHFHsW (ORCPT ); Tue, 6 Aug 2019 03:48:22 -0400 Received: from localhost (unknown [193.47.165.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 1AAF1206A2; Tue, 6 Aug 2019 07:48:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1565077701; bh=mr1mypDDs4Ad4WE/fiyJ3l8cfS/rbaQiAQKIFkYEtPY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=w4Tk9XAxosRkUXOL8iRDmr5sClXAi2xjqmS/YbUml6Y02RmhtSxaGtskzCxLWX00K TmUOfjClzO+6etzfCYePBh1NrGe109BjryQ8Wqz8dMps/Xsx7xAw1o2rw+SY1j9T5f 127HZd+NMfnzMO3Lxer/SgFvg4iumPq4jYK/+mpQ= From: Leon Romanovsky To: Doug Ledford , Jason Gunthorpe Cc: Leon Romanovsky , RDMA mailing list , Michael Guralnik , Moni Shoua , Saeed Mahameed , linux-netdev Subject: [PATCH rdma-next v2 2/4] IB/mlx5: Query ODP capabilities for DC Date: Tue, 6 Aug 2019 10:48:05 +0300 Message-Id: <20190806074807.9111-3-leon@kernel.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190806074807.9111-1-leon@kernel.org> References: <20190806074807.9111-1-leon@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Michael Guralnik Cache current ODP capabilities for DC in mlx5_ib device. Signed-off-by: Michael Guralnik Reviewed-by: Moni Shoua Signed-off-by: Leon Romanovsky --- drivers/infiniband/hw/mlx5/mlx5_ib.h | 1 + drivers/infiniband/hw/mlx5/odp.c | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+) diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index cb41a7e6255a..f99c71b3c876 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -967,6 +967,7 @@ struct mlx5_ib_dev { struct mutex slow_path_mutex; int fill_delay; struct ib_odp_caps odp_caps; + uint32_t dc_odp_caps; u64 odp_max_size; struct mlx5_ib_pf_eq odp_pf_eq; diff --git a/drivers/infiniband/hw/mlx5/odp.c b/drivers/infiniband/hw/mlx5/odp.c index b0c5de39d186..5e87a5e25574 100644 --- a/drivers/infiniband/hw/mlx5/odp.c +++ b/drivers/infiniband/hw/mlx5/odp.c @@ -353,6 +353,24 @@ void mlx5_ib_internal_fill_odp_caps(struct mlx5_ib_dev *dev) if (MLX5_CAP_ODP(dev->mdev, xrc_odp_caps.srq_receive)) caps->per_transport_caps.xrc_odp_caps |= IB_ODP_SUPPORT_SRQ_RECV; + if (MLX5_CAP_ODP(dev->mdev, dc_odp_caps.send)) + dev->dc_odp_caps |= IB_ODP_SUPPORT_SEND; + + if (MLX5_CAP_ODP(dev->mdev, dc_odp_caps.receive)) + dev->dc_odp_caps |= IB_ODP_SUPPORT_RECV; + + if (MLX5_CAP_ODP(dev->mdev, dc_odp_caps.write)) + dev->dc_odp_caps |= IB_ODP_SUPPORT_WRITE; + + if (MLX5_CAP_ODP(dev->mdev, dc_odp_caps.read)) + dev->dc_odp_caps |= IB_ODP_SUPPORT_READ; + + if (MLX5_CAP_ODP(dev->mdev, dc_odp_caps.atomic)) + dev->dc_odp_caps |= IB_ODP_SUPPORT_ATOMIC; + + if (MLX5_CAP_ODP(dev->mdev, dc_odp_caps.srq_receive)) + dev->dc_odp_caps |= IB_ODP_SUPPORT_SRQ_RECV; + if (MLX5_CAP_GEN(dev->mdev, fixed_buffer_size) && MLX5_CAP_GEN(dev->mdev, null_mkey) && MLX5_CAP_GEN(dev->mdev, umr_extended_translation_offset)) From patchwork Tue Aug 6 07:48:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 1142648 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.b="t2+M/ref"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 462mwJ4BD0z9sDB for ; Tue, 6 Aug 2019 17:48:32 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732105AbfHFHsb (ORCPT ); Tue, 6 Aug 2019 03:48:31 -0400 Received: from mail.kernel.org ([198.145.29.99]:44322 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726834AbfHFHsb (ORCPT ); Tue, 6 Aug 2019 03:48:31 -0400 Received: from localhost (unknown [193.47.165.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 3476D206A2; Tue, 6 Aug 2019 07:48:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1565077710; bh=wSob9UeSfITrkANzxhKX2TkJljgV5+yGRG2sdNOgVF4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=t2+M/refQjGV8zVlt1I08psa7pIK7bdEl2CFIETOMrvI/Lrih/rlEGw6BUfuPupRp /p1ka1upC/InCm48vAm9E+hBw6EUT3e9f+lPKKMrz36apfZC+uv4QGKnGlq6PX2CcD iuXXE+dPl03cKF02FsT5WJg9yVOHUga2ov3OftLI= From: Leon Romanovsky To: Doug Ledford , Jason Gunthorpe Cc: Leon Romanovsky , RDMA mailing list , Michael Guralnik , Moni Shoua , Saeed Mahameed , linux-netdev Subject: [PATCH rdma-next v2 3/4] IB/mlx5: Expose ODP for DC capabilities to user Date: Tue, 6 Aug 2019 10:48:06 +0300 Message-Id: <20190806074807.9111-4-leon@kernel.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190806074807.9111-1-leon@kernel.org> References: <20190806074807.9111-1-leon@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Michael Guralnik Return ODP capabilities for DC to user in alloc_context. Signed-off-by: Michael Guralnik Reviewed-by: Moni Shoua Signed-off-by: Leon Romanovsky --- drivers/infiniband/hw/mlx5/main.c | 6 ++++++ include/uapi/rdma/mlx5-abi.h | 3 +++ 2 files changed, 9 insertions(+) diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index 4a3d700cd783..a53e0dc7c17f 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -1954,6 +1954,12 @@ static int mlx5_ib_alloc_ucontext(struct ib_ucontext *uctx, resp.response_length += sizeof(resp.dump_fill_mkey); } + if (field_avail(typeof(resp), dc_odp_caps, udata->outlen)) { + resp.dc_odp_caps = dev->dc_odp_caps; + resp.comp_mask |= MLX5_IB_ALLOC_UCONTEXT_RESP_MASK_DC_ODP_CAPS; + resp.response_length += sizeof(resp.dc_odp_caps); + } + err = ib_copy_to_udata(udata, &resp, resp.response_length); if (err) goto out_mdev; diff --git a/include/uapi/rdma/mlx5-abi.h b/include/uapi/rdma/mlx5-abi.h index 624f5b53eb1f..7cab806d7fa7 100644 --- a/include/uapi/rdma/mlx5-abi.h +++ b/include/uapi/rdma/mlx5-abi.h @@ -98,6 +98,7 @@ struct mlx5_ib_alloc_ucontext_req_v2 { enum mlx5_ib_alloc_ucontext_resp_mask { MLX5_IB_ALLOC_UCONTEXT_RESP_MASK_CORE_CLOCK_OFFSET = 1UL << 0, MLX5_IB_ALLOC_UCONTEXT_RESP_MASK_DUMP_FILL_MKEY = 1UL << 1, + MLX5_IB_ALLOC_UCONTEXT_RESP_MASK_DC_ODP_CAPS = 1UL << 2, }; enum mlx5_user_cmds_supp_uhw { @@ -147,6 +148,8 @@ struct mlx5_ib_alloc_ucontext_resp { __u32 num_uars_per_page; __u32 num_dyn_bfregs; __u32 dump_fill_mkey; + __u32 dc_odp_caps; + __u32 reserved; }; struct mlx5_ib_alloc_pd_resp { From patchwork Tue Aug 6 07:48:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 1142647 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.b="nAdIS12K"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 462mwD5P3yz9sDB for ; Tue, 6 Aug 2019 17:48:28 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732069AbfHFHs1 (ORCPT ); Tue, 6 Aug 2019 03:48:27 -0400 Received: from mail.kernel.org ([198.145.29.99]:44302 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726834AbfHFHs1 (ORCPT ); Tue, 6 Aug 2019 03:48:27 -0400 Received: from localhost (unknown [193.47.165.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id ED005206A2; Tue, 6 Aug 2019 07:48:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1565077706; bh=cdIzpma59+8ccw4pf9IkQYTsoLRo13CbMbJywDBJas4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nAdIS12KdWnt06TO+QaSvyTtNLOs5uObHbqvyUQglHtWdrUO0A0q5+Y1zbikE5EWO jhrizk7XIlY/lLqj6LCakdX5Lo16p8dJwM5j7CG6dz90yc4pyM67J2y2wXbVNFKGeP KBDgl58GkPaPQ4Rhhls31I3izlq4Ih/BYLqOSs+A= From: Leon Romanovsky To: Doug Ledford , Jason Gunthorpe Cc: Leon Romanovsky , RDMA mailing list , Michael Guralnik , Moni Shoua , Saeed Mahameed , linux-netdev Subject: [PATCH rdma-next v2 4/4] IB/mlx5: Add page fault handler for DC initiator WQE Date: Tue, 6 Aug 2019 10:48:07 +0300 Message-Id: <20190806074807.9111-5-leon@kernel.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190806074807.9111-1-leon@kernel.org> References: <20190806074807.9111-1-leon@kernel.org> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Michael Guralnik Parsing DC initiator WQEs upon page fault requires skipping an address vector segment, as in UD WQEs. Signed-off-by: Michael Guralnik Reviewed-by: Moni Shoua Signed-off-by: Leon Romanovsky --- drivers/infiniband/hw/mlx5/odp.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/mlx5/odp.c b/drivers/infiniband/hw/mlx5/odp.c index 5e87a5e25574..6f1de5edbe8e 100644 --- a/drivers/infiniband/hw/mlx5/odp.c +++ b/drivers/infiniband/hw/mlx5/odp.c @@ -1065,6 +1065,12 @@ static int mlx5_ib_mr_initiator_pfault_handler( case IB_QPT_UD: transport_caps = dev->odp_caps.per_transport_caps.ud_odp_caps; break; + case IB_QPT_DRIVER: + if (qp->qp_sub_type == MLX5_IB_QPT_DCI) { + transport_caps = dev->dc_odp_caps; + break; + } + /* fall through */ default: mlx5_ib_err(dev, "ODP fault on QP of an unsupported transport 0x%x\n", qp->ibqp.qp_type); @@ -1078,7 +1084,8 @@ static int mlx5_ib_mr_initiator_pfault_handler( return -EFAULT; } - if (qp->ibqp.qp_type == IB_QPT_UD) { + if (qp->ibqp.qp_type == IB_QPT_UD || + qp->qp_sub_type == MLX5_IB_QPT_DCI) { av = *wqe; if (av->dqp_dct & cpu_to_be32(MLX5_EXTENDED_UD_AV)) *wqe += sizeof(struct mlx5_av);