From patchwork Fri Nov 23 05:52:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexey Kardashevskiy X-Patchwork-Id: 1002120 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ozlabs.ru Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 431QTR6d5kz9sCV for ; Fri, 23 Nov 2018 16:53:15 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729581AbeKWQf4 (ORCPT ); Fri, 23 Nov 2018 11:35:56 -0500 Received: from 107-173-13-209-host.colocrossing.com ([107.173.13.209]:34106 "EHLO ozlabs.ru" rhost-flags-OK-FAIL-OK-OK) by vger.kernel.org with ESMTP id S1729050AbeKWQf4 (ORCPT ); Fri, 23 Nov 2018 11:35:56 -0500 Received: from fstn1-p1.ozlabs.ibm.com (localhost [IPv6:::1]) by ozlabs.ru (Postfix) with ESMTP id E51BCAE8001D; Fri, 23 Nov 2018 00:53:09 -0500 (EST) From: Alexey Kardashevskiy To: linuxppc-dev@lists.ozlabs.org Cc: Alexey Kardashevskiy , David Gibson , kvm-ppc@vger.kernel.org, Alistair Popple , Reza Arbab , Sam Bobroff , Piotr Jaroszynski , =?utf-8?q?Leonardo_Augusto_Guimar=C3=A3es_?= =?utf-8?q?Garcia?= , Jose Ricardo Ziviani , Daniel Henrique Barboza , Andrew Donnellan , Alex Williamson , Balbir Singh , Russell Currey , Oliver O'Halloran Subject: [PATCH kernel v4 00/19] powerpc/powernv/npu, vfio: NVIDIA V100 + P9 passthrough Date: Fri, 23 Nov 2018 16:52:45 +1100 Message-Id: <20181123055304.25116-1-aik@ozlabs.ru> X-Mailer: git-send-email 2.17.1 Sender: kvm-ppc-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org This is for passing through NVIDIA V100 GPUs on POWER9 systems. 19/19 has the details of hardware setup. This implements support for NVIDIA V100 GPU with coherent memory and NPU/ATS support available in the POWER9 CPU. The aim is to support unmodified vendor driver in the guest. This is pushed to (both guest and host kernels): https://github.com/aik/linux/tree/nv2-stage7 Matching qemu is pushed to github: https://github.com/aik/qemu/tree/nv2-stage6 Skiboot bits are here: https://github.com/aik/skiboot/tree/nv2-stage4 The individual patches have changelogs. Some were dropped as not required or quite useless. Please comment. Thanks. Alexey Kardashevskiy (19): powerpc/ioda/npu: Call skiboot's hot reset hook when disabling NPU2 powerpc/mm/iommu/vfio_spapr_tce: Change mm_iommu_get to reference a region powerpc/vfio/iommu/kvm: Do not pin device memory powerpc/powernv: Move npu struct from pnv_phb to pci_controller powerpc/powernv/npu: Move OPAL calls away from context manipulation powerpc/pseries/iommu: Use memory@ nodes in max RAM address calculation powerpc/pseries/npu: Enable platform support powerpc/pseries: Remove IOMMU API support for non-LPAR systems powerpc/powernv/pseries: Rework device adding to IOMMU groups powerpc/iommu_api: Move IOMMU groups setup to a single place powerpc/powernv: Reference iommu_table while it is linked to a group powerpc/powernv: Add purge cache OPAL call powerpc/powernv/npu: Move single TVE handling to NPU PE powerpc/powernv/npu: Convert NPU IOMMU helpers to iommu_table_group_ops powerpc/powernv/npu: Add compound IOMMU groups powerpc/powernv/npu: Add release_ownership hook vfio_pci: Allow mapping extra regions vfio_pci: Allow regions to add own capabilities vfio_pci: Add NVIDIA GV100GL [Tesla V100 SXM2] subdriver drivers/vfio/pci/Makefile | 1 + arch/powerpc/include/asm/iommu.h | 17 +- arch/powerpc/include/asm/mmu_context.h | 9 +- arch/powerpc/include/asm/opal-api.h | 3 +- arch/powerpc/include/asm/opal.h | 1 + arch/powerpc/include/asm/pci-bridge.h | 1 + arch/powerpc/include/asm/pci.h | 4 + arch/powerpc/platforms/powernv/pci.h | 30 +- drivers/vfio/pci/trace.h | 102 ++++ drivers/vfio/pci/vfio_pci_private.h | 8 + include/uapi/linux/vfio.h | 27 + arch/powerpc/kernel/iommu.c | 67 +-- arch/powerpc/kvm/book3s_64_vio.c | 18 +- arch/powerpc/mm/mmu_context_iommu.c | 103 +++- arch/powerpc/platforms/powernv/npu-dma.c | 529 +++++++++++++++--- arch/powerpc/platforms/powernv/opal.c | 1 + arch/powerpc/platforms/powernv/pci-ioda-tce.c | 3 +- arch/powerpc/platforms/powernv/pci-ioda.c | 231 ++++---- arch/powerpc/platforms/powernv/pci.c | 43 +- arch/powerpc/platforms/pseries/iommu.c | 89 ++- arch/powerpc/platforms/pseries/pci.c | 22 + drivers/vfio/pci/vfio_pci.c | 52 +- drivers/vfio/pci/vfio_pci_nvlink2.c | 448 +++++++++++++++ drivers/vfio/vfio_iommu_spapr_tce.c | 65 ++- .../powerpc/platforms/powernv/opal-wrappers.S | 1 + drivers/vfio/pci/Kconfig | 6 + 26 files changed, 1492 insertions(+), 389 deletions(-) create mode 100644 drivers/vfio/pci/trace.h create mode 100644 drivers/vfio/pci/vfio_pci_nvlink2.c