From patchwork Sat Jun 23 23:52:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timothy Pearson X-Patchwork-Id: 933817 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41Ctpq4bDJz9s2R for ; Sun, 24 Jun 2018 10:44:03 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=raptorengineering.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=raptorengineering.com header.i=@raptorengineering.com header.b="O3QZV0Zv"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 41Ctpq33FTzF16f for ; Sun, 24 Jun 2018 10:44:03 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=raptorengineering.com Authentication-Results: lists.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=raptorengineering.com header.i=@raptorengineering.com header.b="O3QZV0Zv"; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=raptorengineering.com (client-ip=192.119.205.245; helo=mail.rptsys.com; envelope-from=tpearson@raptorengineering.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=raptorengineering.com Authentication-Results: lists.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=raptorengineering.com header.i=@raptorengineering.com header.b="O3QZV0Zv"; dkim-atps=neutral X-Greylist: delayed 369 seconds by postgrey-1.36 at bilbo; Sun, 24 Jun 2018 09:58:23 AEST Received: from mail.rptsys.com (mail.rptsys.com [192.119.205.245]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 41Csp71VDdzF0Pd for ; Sun, 24 Jun 2018 09:58:23 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by mail.rptsys.com (Postfix) with ESMTP id 424726428A4; Sat, 23 Jun 2018 18:52:05 -0500 (CDT) Received: from mail.rptsys.com ([127.0.0.1]) by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id u4aOjOCy60Lr; Sat, 23 Jun 2018 18:52:04 -0500 (CDT) Received: from localhost (localhost [127.0.0.1]) by mail.rptsys.com (Postfix) with ESMTP id 2AF496428A6; Sat, 23 Jun 2018 18:52:04 -0500 (CDT) DKIM-Filter: OpenDKIM Filter v2.9.2 mail.rptsys.com 2AF496428A6 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raptorengineering.com; s=B8E824E6-0BE2-11E6-931D-288C65937AAD; t=1529797924; bh=xZvM731Z7OJtjO4Xpamz+FeF7hdqomY7K/4tRjXZobg=; h=Date:From:To:Message-ID:Subject:MIME-Version:Content-Type: Content-Transfer-Encoding; b=O3QZV0ZvB2G6j2gLrTXfvVop8/yks/Re0CEqpkoJOQKX9lw+Edb4BTFfIPqLvYM5U okdhultAttt+5K8TaJZ4hqkkem5F/h8efoPJqsU7vFfSLnih9TrkSf1AJ4qQBsy8/z QRw3mcAh4XPlRoo4FcdTJA77DwHtDZxGBRLJnOvw= X-Virus-Scanned: amavisd-new at rptsys.com Received: from mail.rptsys.com ([127.0.0.1]) by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id q9KYdptwNI7H; Sat, 23 Jun 2018 18:52:03 -0500 (CDT) Received: from vali.starlink.edu (netrouter.starlink.edu [192.168.3.1]) by mail.rptsys.com (Postfix) with ESMTP id B152B6428A1; Sat, 23 Jun 2018 18:52:03 -0500 (CDT) Date: Sat, 23 Jun 2018 18:52:02 -0500 (CDT) From: Timothy Pearson To: linuxppc-dev@lists.ozlabs.org Message-ID: <1564865529.2569245.1529797922226.JavaMail.zimbra@raptorengineeringinc.com> Subject: [PATCH 0/7] Add initial version of "cognitive DMA" MIME-Version: 1.0 X-Mailer: Zimbra 8.5.0_GA_3042 (ZimbraWebClient - FF3.6 (Linux)/8.5.0_GA_3042) Thread-Topic: Add initial version of "cognitive DMA" Thread-Index: 8vEntm7NNNTH8w9EVizwAaXV3546Fw== X-Mailman-Approved-At: Sun, 24 Jun 2018 10:42:17 +1000 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paul Mackerras Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" POWER9 (PHB4) requires all peripherals using DMA to be either restricted to 32-bit windows or capable of accessing the entire 64 bits of memory space. Some devices, such as most GPUs, can only address up to a certain number of bits (approximately 40, in many cases), while at the same time it is highly desireable to use a larger DMA space than the fallback 32 bits. This series adds something called "cognitive DMA", which is a form of dynamic TCE allocation. This allows the peripheral to DMA to host addresses mapped in 1G (PHB4) or 256M (PHB3) chunks, and is transparent to the peripheral and its driver stack. This series has been tested on a Talos II server with a Radeon WX4100 and a wide range of OpenGL applications. While there is still work, notably involving what happens if a peripheral attempts to DMA close to a TCE window boundary, this series greatly improves functionality for AMD GPUs on POWER9 devices over the existing 32-bit DMA support. Russell Currey (4): powerpc/powernv/pci: Track largest available TCE order per PHB powerpc/powernv: DMA operations for discontiguous allocation powerpc/powernv/pci: Track DMA and TCE tables in debugfs powerpc/powernv/pci: Safety fixes for pseudobypass TCE allocation Timothy Pearson (3): powerpc/powernv/pci: Export pnv_pci_ioda2_tce_invalidate_pe powerpc/powernv/pci: Invalidate TCE cache after DMA map setup powerpc/powernv/pci: Don't use the lower 4G TCEs in pseudo-DMA mode arch/powerpc/include/asm/dma-mapping.h | 1 + arch/powerpc/platforms/powernv/Makefile | 2 +- arch/powerpc/platforms/powernv/pci-dma.c | 320 ++++++++++++++++++++++ arch/powerpc/platforms/powernv/pci-ioda.c | 169 ++++++++---- arch/powerpc/platforms/powernv/pci.h | 11 + 5 files changed, 452 insertions(+), 51 deletions(-) create mode 100644 arch/powerpc/platforms/powernv/pci-dma.c