From patchwork Tue Sep 13 22:19:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tom Talpey X-Patchwork-Id: 1677555 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-cifs-owner@vger.kernel.org; receiver=) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4MRyZW4YW5z1yhR for ; Wed, 14 Sep 2022 08:20:03 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229698AbiIMWUB (ORCPT ); Tue, 13 Sep 2022 18:20:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50374 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229693AbiIMWT6 (ORCPT ); Tue, 13 Sep 2022 18:19:58 -0400 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2062.outbound.protection.outlook.com [40.107.94.62]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 129EF67462 for ; Tue, 13 Sep 2022 15:19:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=O8pt8irI2UGBh22mJpvb83U7ub1KAqyoMylzVzBQTefjrr3ZbkTlU/yvDKBEox79pizrpjumlTFUIp8Rnp3pk2MutZyuLLyCShTIecEsK4bsjVObuP80aeeYSI4+rg5IygjxGXfIOXVdAHW4b31ItYInAfrA4a1KUBUtwKcbYOgRUE5ztZbKQFzjH9Qlg8WPL/lNqtHyO/UB9xctoAj8PhrBJ5YxeG8EtvwRjBkvt4OR8+FIIg5WtX7F682akttrcbpJegLgcFvI7pue4TK/vaSJUHrI8trGcmftJH+zVJjHekpXhHHcMs5w6igQof471K0g1zUBKpZdxloP5JnfhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1awBvjJcIY9fyo8Cq/++72dqXQ1Aa9ecuKIxN8HY8Zk=; b=U8/cfOGiHqQACT0i/jr5GwljOEd3dMkeVN9eX5Fgop5TqCxf4zuh+DSwd7m0xZGHDelCYeLaJcMk6iucD0IvTdstFqtQzbrjEY57LgPFgKJ9WpRTdqLL/01c6TLUaU27Sr4IAuAUlV/fMOHPVAApTsSxmxf0onNtxBy9llsCQn93E2GU3C2UkB3qaJ0VMcntRKo8xsYSN6b0C0AFymOsRUdlUi1rgVe1cl5roPv7PpTQdhKrHHX+KibmNWIFaTzeR6hmsQT7HHaSUluy1G7sLSRxjdVUQqgfTPA59qURpTlDsGAAbh09yLP2+SEV/1XEvCtX4EMedhiabSq+T/m4AA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=talpey.com; dmarc=pass action=none header.from=talpey.com; dkim=pass header.d=talpey.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=talpey.com; Received: from SN6PR01MB4445.prod.exchangelabs.com (2603:10b6:805:e2::33) by SN6PR01MB4573.prod.exchangelabs.com (2603:10b6:805:eb::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.16; Tue, 13 Sep 2022 22:19:53 +0000 Received: from SN6PR01MB4445.prod.exchangelabs.com ([fe80::6566:9fdc:aac:8b51]) by SN6PR01MB4445.prod.exchangelabs.com ([fe80::6566:9fdc:aac:8b51%2]) with mapi id 15.20.5612.022; Tue, 13 Sep 2022 22:19:53 +0000 From: Tom Talpey To: smfrench@gmail.com, linux-cifs@vger.kernel.org Cc: linkinjeon@kernel.org, senozhatsky@chromium.org, bmt@zurich.ibm.com, Tom Talpey Subject: [RFC PATCH 0/6] Reduce SMBDirect RDMA SGE counts and sizes Date: Tue, 13 Sep 2022 22:19:31 +0000 Message-Id: X-Mailer: git-send-email 2.34.1 X-ClientProxiedBy: BYAPR08CA0047.namprd08.prod.outlook.com (2603:10b6:a03:117::24) To SN6PR01MB4445.prod.exchangelabs.com (2603:10b6:805:e2::33) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN6PR01MB4445:EE_|SN6PR01MB4573:EE_ X-MS-Office365-Filtering-Correlation-Id: 00a921ba-954e-4a53-73d7-08da95d614f9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: EgyOO6kzpApD2+uOx1Unu/BysszHQ0TrS4Af/OVIsAcnhZNQsySmRRJMK/gJoFh9mIKuPLBdHwPvrjVMK7Z5LRwqYAQzh8tOEClXA6ZJWcR0qgCPbv18fcI8ae17EOku/OmIGfi9NJNaIHpHK990t/P3aOpgn6Oi4qrViwgC+GxnmnzARpTymFYI3cokxag3JtSM+ZJCFsMNFEYGAbr/B2ri3bEl1c3DkGRJEzKJ96vyOwnQhKzOCbDVI+C4YW6YRnVqNdDh4czYx+ejHn6Qr7kEZ5IkEb0/HKNQcL3tWcUhnT/XczkkfMfrJDeQDuHiE7myAhwYR29LOc0VqxjcJe58Bsh+j2nM82qMWZy+luCjR8MVf1vq+6iz2bE9p3mh+go2i26S4MFF97NbovrYcxOExwhxJrTW1VwnoO5Dna2sGXM6R/zQJsRbgDN1hYzxeUuUppB7ncESr15YHYoCOMXbL0EzBeB9D9sV7AtxPitKWnipLzcO1zKKI/fNmHbTpS6g82KNDKHU4wNBZsZK8lqsPhHlS545kigCDIASF0Ommk7L6jQGs70MNgUlGRvpv2HU/uE/SxncJ/z0EKI320wqyAwfJwAbuRmpjIPMSASaQ89825W9Iw1qAd7iBt5f9u04FJ5HD+lIdwXw66PvSZfsZAyAFwsyQdES1QUAjdN1Qti4dh98jSvfMpYkrwIuwaZYdzAy45kxvvjEcc4/SlpmDL0Y+jBQisdhWBdwuHgQ2VfDgkRVzBuZOgs3OrPq X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR01MB4445.prod.exchangelabs.com;PTR:;CAT:NONE;SFS:(13230022)(346002)(136003)(366004)(376002)(396003)(39830400003)(451199015)(38350700002)(107886003)(6486002)(8676002)(8936002)(6666004)(41300700001)(66556008)(6512007)(66476007)(2616005)(186003)(2906002)(52116002)(478600001)(86362001)(6506007)(5660300002)(36756003)(26005)(4326008)(38100700002)(66946007)(316002)(83380400001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: aXBAv33jOD3m9gkq3fmm8lIyWBg3/Bv/+nxZRa3e3XCzvdjXAKCrkh2njEm1wXGGyM7xr1JwXXYjFfhVQqG6NjOueyrzF3e8kQZ5XXUW3zEaM01Z7zdE1NYD9zoHirZql7vje8Z0VLubvW9+PGk35+ke4BBKUyrGT37iyBVAE89+RRTYHw61pIN/H42EvbD/keHUK/A5rEQxtgZyMU1PtXvOriD3guST9OPiDl0dQbpxUwxWNDMPHmVNPhoRo2sy4lEb8F8Eyj1U8oGXYX69zek4qWicUyS30F6VEbe/+0uE1RdPmCPsrFpqX8yotmJJMSBCDN/KV8m3uuJERp738+y7YmuhYzZKKJTdgzvNqbRDFJS9lK+3C/RMOw0odRm9lxGzdxV4BXuRMaSkkpYXkk+gDN1cy6KOYw0IpsY2ElTDOTIeKkyvjJ7rc+68ZBkAjvpNRZHxPi9Ygu6wf9ChUEngqKIn4eVUvkuoIG8N4uJ1dGHgZ++nNT0U6mH9xy6k858RMVTj/RqDsT5A24YwwmDAAf1AxQLcgbNlkYZ/zvdB+9Xcp9U9L//zg2ggpv+u31OetiMV2mZOd2CBWXkuWcuNLMYwVDaQQ/Fi7DmZKEkg8Q4d3ARjyOT9/2amHHWYkKb+itHAFFImF6MEoGerKqi2hdzvxLX17Ty29jmvVEz9r4/yDCJaAvgdpyCP4RUPLXe/+RIEflR71/DsKvrSPbE6FYs8B+KMZSeI+MGbG5copSGfO3WJ0jNBxDkjWshL8E2oKlMvmw1Gc2WmRRvhLGQ9oSTMJBMbvarsschIMU+Q60udxc7fM/2rFtTU87C/sitWL11G2kTWQUVRFt9Z+Rz6pdYNiZRhVlZSQA//CxL7gY3cpskSqc3iLc5xxKHA5VORJNvWrAc4rA3Or5iy9nv3jbgsapY8zseyaOsiElZdxaA22+kKzfpnC4NNDnbgSoHScAi/0Lew4XM3pavIMsTFPuaWY5YUowb/Uw24HWcciG5Mi/WPBm0ZHwqiRVbjYmiNLJfQw57KHUvXdgkS830aD+WjNFXlX8mJ8X4m1cfkB2wYil3DdR3mhhTpuYS2wkcPG9Ar3uN7SzGNS+b6qNuB/jhTOo6w1b3H+UR8mwJDFbQRn2xYfqcaHkdE+/XJpjc30RbYJjvywumq8RX24DOj2Mj/i+LFP0F1FM9m1KHe4KGoR8zMc4Y+5r1h4ElsscHINNEFoIBN0Aj9fdKIKcCvlvV7m37frf8JbrFyRNiB6fMdSPOU6zCUIZ+uUiw+CvQypgLHo5fEbLTGMBpOlhF0Oef6lVk+ebHM2fU5xJIEp7jm5OJqjVdirOCjhDNwkjx4048b30+u+hEw0m3LP4ZBWD+SzG0kyY9HJgT2tzfcl7hSmJepW3awekirIZenCWmqyNEZ1SoI3/XYMH6/F6MUfeyWUbaM+bmEhvEecYt+r/u6BSeWacnPodh2FBYa/wdkvmdXBlkbGTN3XzqfnzK6jDw8PWAMpAWl0V765JiEgQThTy6PAa3qvjs4vnxeiaCVyBJLKrZgDhn1IF9NcVn6Lohvu6gozwlgu31S4JJ66BoKbSE4R6jBxKMmtcmr X-OriginatorOrg: talpey.com X-MS-Exchange-CrossTenant-Network-Message-Id: 00a921ba-954e-4a53-73d7-08da95d614f9 X-MS-Exchange-CrossTenant-AuthSource: SN6PR01MB4445.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2022 22:19:53.5497 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 2b2dcae7-2555-4add-bc80-48756da031d5 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: L+RR4KIHhakAbaogLqvv/CF5OFHNS+zcKecVtYo3my1lqqD22rL5rbJMjbpgi6On X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR01MB4573 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-cifs@vger.kernel.org Allocate fewer SGEs and standard packet sizes in both kernel SMBDirect implementations. The current maximum values (16 SGEs and 8192 bytes) cause failures on the SoftiWARP provider, and are suboptimal on others. Reduce these to 6 and 1364. Additionally, recode smbd_send() to work with as few as 2 SGEs, and for debug sanity, reformat client-side logging to more clearly show addresses, lengths and flags in the appropriate base. Tested over SoftiWARP and SoftRoCE with shell, Connectathon basic and general, and to be tested with XFStests here at SMB3 IOLab this week. Tom Talpey (6): Decrease the number of SMB3 smbdirect client SGEs Decrease the number of SMB3 smbdirect server SGEs Reduce client smbdirect max receive segment size Reduce server smbdirect max send/receive segment sizes Handle variable number of SGEs in client smbdirect send. Fix formatting of client smbdirect RDMA logging fs/cifs/smbdirect.c | 227 ++++++++++++++++---------------------- fs/cifs/smbdirect.h | 14 ++- fs/ksmbd/transport_rdma.c | 6 +- 3 files changed, 109 insertions(+), 138 deletions(-)