From patchwork Wed Sep 24 08:47:27 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Blaschka X-Patchwork-Id: 392797 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 7678A14009B for ; Wed, 24 Sep 2014 18:48:22 +1000 (EST) Received: from localhost ([::1]:58268 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XWiFI-0004kr-Mu for incoming@patchwork.ozlabs.org; Wed, 24 Sep 2014 04:48:20 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39535) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XWiEl-0004HB-MG for qemu-devel@nongnu.org; Wed, 24 Sep 2014 04:47:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XWiEc-0001x0-HS for qemu-devel@nongnu.org; Wed, 24 Sep 2014 04:47:47 -0400 Received: from e06smtp16.uk.ibm.com ([195.75.94.112]:38408) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XWiEc-0001wU-8T for qemu-devel@nongnu.org; Wed, 24 Sep 2014 04:47:38 -0400 Received: from /spool/local by e06smtp16.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 24 Sep 2014 09:47:31 +0100 Received: from d06dlp03.portsmouth.uk.ibm.com (9.149.20.15) by e06smtp16.uk.ibm.com (192.168.101.146) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 24 Sep 2014 09:47:28 +0100 Received: from b06cxnps4074.portsmouth.uk.ibm.com (d06relay11.portsmouth.uk.ibm.com [9.149.109.196]) by d06dlp03.portsmouth.uk.ibm.com (Postfix) with ESMTP id B33101B0806B for ; Wed, 24 Sep 2014 09:48:37 +0100 (BST) Received: from d06av06.portsmouth.uk.ibm.com (d06av06.portsmouth.uk.ibm.com [9.149.37.217]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id s8O8lR1i37028030 for ; Wed, 24 Sep 2014 08:47:27 GMT Received: from d06av06.portsmouth.uk.ibm.com (localhost [127.0.0.1]) by d06av06.portsmouth.uk.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id s8O3jdre021047 for ; Tue, 23 Sep 2014 23:45:39 -0400 Received: from tuxmaker.boeblingen.de.ibm.com (tuxmaker.boeblingen.de.ibm.com [9.152.85.9]) by d06av06.portsmouth.uk.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id s8O3jdSq021035; Tue, 23 Sep 2014 23:45:39 -0400 Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 24631) id 3C4091224439; Wed, 24 Sep 2014 10:47:27 +0200 (CEST) Date: Wed, 24 Sep 2014 10:47:27 +0200 From: Frank Blaschka To: Alex Williamson Message-ID: <20140924084727.GA17378@tuxmaker.boeblingen.de.ibm.com> References: <20140919115429.557279920@de.ibm.com> <1411418851.1199.137.camel@ul30vt.home> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <1411418851.1199.137.camel@ul30vt.home> User-Agent: Mutt/1.5.17 (2007-11-01) X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 14092408-3548-0000-0000-000001992EBB X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 195.75.94.112 Cc: linux-s390@vger.kernel.org, frank.blaschka@de.ibm.com, kvm@vger.kernel.org, agraf@suse.de, qemu-devel@nongnu.org, pbonzini@redhat.com Subject: Re: [Qemu-devel] [RFC patch 0/6] vfio based pci pass-through for qemu/KVM on s390 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org On Mon, Sep 22, 2014 at 02:47:31PM -0600, Alex Williamson wrote: > On Fri, 2014-09-19 at 13:54 +0200, frank.blaschka@de.ibm.com wrote: > > This set of patches implements a vfio based solution for pci > > pass-through on the s390 platform. The kernel stuff is pretty > > much straight forward, but qemu needs more work. > > > > Most interesting patch is: > > vfio: make vfio run on s390 platform > > > > I hope Alex & Alex can give me some guidance how to do the changes > > in an appropriate way. After creating a separate iommmu address space > > for each attached PCI device I can successfully run the vfio type1 > > iommu. So If we could extend type1 not registering all guest memory > > (see patch) I think we do not need a special vfio iommu for s390 > > for the moment. > > > > The patches implement the base pass-through support. s390 specific > > virtualization functions are currently not included. This would > > be a second step after the base support is done. > > > > kernel patches apply to linux-kvm-next > > > > KVM: s390: Enable PCI instructions > > iommu: add iommu for s390 platform > > vfio: make vfio build on s390 > > > > qemu patches apply to qemu-master > > > > s390: Add PCI bus support > > s390: implement pci instruction > > vfio: make vfio run on s390 platform > > > > Thx for feedback and review comments > > Sending patches as attachments makes it difficult to comment inline. > Sorry, don't understand this. I sent every patch as separate email so you can comment directly on the patch. What do you prefer? > 2/6 > - careful of the namespace as you're changing functions from static and > exporting them > - doesn't seem like functions need to be exported, just non-static to > call from s390-iommu.c > Ok, will change this. > 6/6 > - We shouldn't need to globally disable mmap, each VFIO region reports > whether it supports mmap and vfio-pci on s390 should indicate mmap is > not supported on the platform. Yes, this is even better to let the kernel announce a BAR can not be mmap'ed. Checking the kernel code I realized the BARs are valid for mmap'ing but the s390 platform does simply not allow this. So I feal we have to introduce a platform switch in kernel. How about this ... > - INTx should be done the same way, the interrupt index for INTx should > report 0 count. The current code likely doesn't handle this, but it > should be easy to fix. The current code is fine. Problem is the card reports an interrupt index (PCI_INTERRUPT_PIN) but again the platform does not support INTx at all. So we need a platform switch as well. > - s390_msix_notify() vs msix_notify() should be abstracted somewhere Platform does not have have an apic so there is nothing we could emulate in qemu to make the existing msix_notify() work. > else. How would an emulated PCI device with MSI-X support work? > - same for add_msi_route Same here, we have to setup an adapter route due to the fact MSIX notifications are delivered as adapter/thin IRQs on the platform. Any suggestion or idea how a better abstraction could look like? With all the platform constraints I was not able to find a suitable emulated device. Remember s390: - does not support IO BARs - does not support INTx only MSIX - in reality currently there is only a PCI network card available - platform does not support fancy I/O like usb or audio :-) So we don't even have kernel (host and guest) support for this kind of devices. > - We can probably come up with a better way to determine which address > space to connect to the memory listener. Any suggestion or idea for that? > > Looks like a reasonable first pass, good re-use of vfio code. Thanks, > > Alex > Thx, Frank > --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -377,9 +377,11 @@ static long vfio_pci_ioctl(void *device_ info.flags = VFIO_REGION_INFO_FLAG_READ | VFIO_REGION_INFO_FLAG_WRITE; +#ifndef CONFIG_S390 if (pci_resource_flags(pdev, info.index) & IORESOURCE_MEM && info.size >= PAGE_SIZE) info.flags |= VFIO_REGION_INFO_FLAG_MMAP; +#endif break; case VFIO_PCI_ROM_REGION_INDEX: {