From patchwork Wed May 29 16:16:56 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maik Broemme X-Patchwork-Id: 247309 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id E58FE2C0354 for ; Thu, 30 May 2013 02:17:37 +1000 (EST) Received: from localhost ([::1]:41545 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Uhj4B-0002QJ-Ia for incoming@patchwork.ozlabs.org; Wed, 29 May 2013 12:17:35 -0400 Received: from eggs.gnu.org ([208.118.235.92]:33159) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Uhj3k-0002PH-Hq for qemu-devel@nongnu.org; Wed, 29 May 2013 12:17:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Uhj3c-0002HL-Ei for qemu-devel@nongnu.org; Wed, 29 May 2013 12:17:08 -0400 Received: from relay.swsoft.eu ([109.70.220.8]:57362) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Uhj3c-0002Gq-2c for qemu-devel@nongnu.org; Wed, 29 May 2013 12:17:00 -0400 Received: from mail.swsoft.eu ([109.70.220.2]) by relay.swsoft.eu with esmtps (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1Uhj3Z-0006eL-Gu; Wed, 29 May 2013 18:16:57 +0200 Received: from parallels.com (78.34.77.17) by mail.swsoft.eu (10.67.2.30) with Microsoft SMTP Server (TLS) id 14.2.342.3; Wed, 29 May 2013 18:16:57 +0200 Date: Wed, 29 May 2013 18:16:56 +0200 From: Maik Broemme To: Alex Williamson Message-ID: <20130529161656.GM28436@parallels.com> References: <1368478534.14279.201.camel@ori.omang.mine.nu> <1368483819.5520.101.camel@ul30vt.home> <1368977758.14279.329.camel@ori.omang.mine.nu> <20130519192640.GB27523@parallels.com> <1369019831.5520.275.camel@ul30vt.home> <20130520110555.GH3391@parallels.com> <20130528014002.GB1241@parallels.com> <20130528184543.GH1241@parallels.com> <1369780099.2646.340.camel@ul30vt.home> <20130529152720.GL28436@parallels.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20130529152720.GL28436@parallels.com> X-Operating-System: Linux bart.theraso.int 3.9.4-200.fc18.x86_64 Priority: normal Organization: Parallels X-PGP-Key-FingerPrint: 109D 0AC6 86CF 06BD 4890 17B0 8FB9 9971 4EEB 31F1 User-Agent: Mutt/1.5.21 (2010-09-15) X-Originating-IP: [78.34.77.17] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 109.70.220.8 Cc: Knut Omang , kvm , qemu-devel Subject: Re: [Qemu-devel] VFIO VGA test branches X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Hi, Maik Broemme wrote: > > > > > > Here is the DEBUG_VFIO output: > > > > > > vfio: vfio_initfn(0000:04:00.0) group 14 > > > vfio: region_add 0 - afffffff [0x7f8698000000] > > > vfio: SKIPPING region_add fec00000 - fec00fff > > > vfio: SKIPPING region_add fed00000 - fed003ff > > > vfio: SKIPPING region_add fee00000 - feefffff > > > vfio: region_add fffe0000 - ffffffff [0x7f88aa400000] > > > vfio: region_add 100000000 - 24fffffff [0x7f8748000000] > > > vfio: Device 0000:04:00.0 flags: 3, regions: 9, irgs: 4 > > > vfio: Device 0000:04:00.0 region 0: > > > vfio: size: 0x10000000, offset: 0x0, flags: 0x7 > > > vfio: Device 0000:04:00.0 region 1: > > > vfio: size: 0x0, offset: 0x10000000000, flags: 0x0 > > > vfio: Device 0000:04:00.0 region 2: > > > vfio: size: 0x40000, offset: 0x20000000000, flags: 0x7 > > > vfio: Device 0000:04:00.0 region 3: > > > vfio: size: 0x0, offset: 0x30000000000, flags: 0x0 > > > vfio: Device 0000:04:00.0 region 4: > > > vfio: size: 0x100, offset: 0x40000000000, flags: 0x3 > > > vfio: Device 0000:04:00.0 region 5: > > > vfio: size: 0x0, offset: 0x50000000000, flags: 0x0 > > > vfio: Device 0000:04:00.0 ROM: > > > vfio: size: 0x20000, offset: 0x60000000000, flags: 0x1 > > > vfio: Device 0000:04:00.0 config: > > > vfio: size: 0x1000, offset: 0x70000000000, flags: 0x3 > > > vfio: vfio_load_rom(0000:04:00.0) > > > vfio: Enabled ATI/AMD BAR2 0x4000 quirk for device 0000:04:00.0 > > > vfio: Enabled ATI/AMD BAR4 window quirk for device 0000:04:00.0 > > > vfio: Enabled ATI/AMD quirk 0x3c3 BAR4 for device 0000:04:00.0 > > > vfio: 0000:04:00.0 PCI MSI CAP @0xa0 > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x3d, len=0x1) 1 > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x3d, len=0x1) 1 > > > vfio: vfio_enable_intx_kvm(0000:04:00.0) KVM INTx accel enabled > > > vfio: vfio_enable_intx(0000:04:00.0) > > > vfio: vfio_initfn(0000:04:00.1) group 14 > > > vfio: Device 0000:04:00.1 flags: 3, regions: 9, irgs: 4 > > > vfio: Device 0000:04:00.1 region 0: > > > vfio: size: 0x4000, offset: 0x0, flags: 0x7 > > > vfio: Device 0000:04:00.1 region 1: > > > vfio: size: 0x0, offset: 0x10000000000, flags: 0x0 > > > vfio: Device 0000:04:00.1 region 2: > > > vfio: size: 0x0, offset: 0x20000000000, flags: 0x0 > > > vfio: Device 0000:04:00.1 region 3: > > > vfio: size: 0x0, offset: 0x30000000000, flags: 0x0 > > > vfio: Device 0000:04:00.1 region 4: > > > vfio: size: 0x0, offset: 0x40000000000, flags: 0x0 > > > vfio: Device 0000:04:00.1 region 5: > > > vfio: size: 0x0, offset: 0x50000000000, flags: 0x0 > > > vfio: Device 0000:04:00.1 ROM: > > > vfio: size: 0x0, offset: 0x60000000000, flags: 0x0 > > > vfio: Device 0000:04:00.1 config: > > > vfio: size: 0x1000, offset: 0x70000000000, flags: 0x3 > > > vfio: 0000:04:00.1 PCI MSI CAP @0xa0 > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x3d, len=0x1) 2 > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x3d, len=0x1) 2 > > > vfio: vfio_enable_intx_kvm(0000:04:00.1) KVM INTx accel enabled > > > vfio: vfio_enable_intx(0000:04:00.1) > > > vfio: region_del 0 - afffffff > > > vfio: region_add 0 - bffff [0x7f8698000000] > > > vfio: region_add c0000 - dffff [0x7f88aa200000] > > > vfio: region_add e0000 - fffff [0x7f88aa400000] > > > vfio: region_add 100000 - afffffff [0x7f8698100000] > > > vfio: vfio_pci_reset(0000:04:00.0) > > > vfio: vfio_disable_intx_kvm(0000:04:00.0) KVM INTx accel disabled > > > vfio: vfio_disable_intx(0000:04:00.0) > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x54, len=0x2) 0 > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x4, len=0x2) 3 > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x4, 0x0, len=0x2) > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x3d, len=0x1) 1 > > > vfio: vfio_enable_intx_kvm(0000:04:00.0) KVM INTx accel enabled > > > vfio: vfio_enable_intx(0000:04:00.0) > > > vfio: vfio_pci_reset(0000:04:00.1) > > > vfio: vfio_disable_intx_kvm(0000:04:00.1) KVM INTx accel disabled > > > vfio: vfio_disable_intx(0000:04:00.1) > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x54, len=0x2) 0 > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x4, len=0x2) 6 > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x4, 0x0, len=0x2) > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x3d, len=0x1) 2 > > > vfio: vfio_enable_intx_kvm(0000:04:00.1) KVM INTx accel enabled > > > vfio: vfio_enable_intx(0000:04:00.1) > > > vfio: region_del 0 - bffff > > > vfio: region_del c0000 - dffff > > > vfio: region_add 0 - c7fff [0x7f8698000000] > > > vfio: region_add c8000 - dffff [0x7f88aa208000] > > > vfio: region_del 0 - c7fff > > > vfio: region_del c8000 - dffff > > > vfio: region_add 0 - cffff [0x7f8698000000] > > > vfio: region_add d0000 - dffff [0x7f88aa210000] > > > vfio: region_del 0 - cffff > > > vfio: region_del d0000 - dffff > > > vfio: region_add 0 - d7fff [0x7f8698000000] > > > vfio: region_add d8000 - dffff [0x7f88aa218000] > > > vfio: region_del 0 - d7fff > > > vfio: region_del d8000 - dffff > > > vfio: region_add 0 - dffff [0x7f8698000000] > > > vfio: region_del 0 - dffff > > > vfio: region_del e0000 - fffff > > > vfio: region_add 0 - e7fff [0x7f8698000000] > > > vfio: region_add e8000 - fffff [0x7f88aa408000] > > > vfio: region_del 0 - e7fff > > > vfio: region_del e8000 - fffff > > > vfio: region_add 0 - effff [0x7f8698000000] > > > vfio: region_add f0000 - fffff [0x7f88aa410000] > > > vfio: region_del 0 - effff > > > vfio: region_del f0000 - fffff > > > vfio: region_del 100000 - afffffff > > > vfio: region_add 0 - afffffff [0x7f8698000000] > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x0, len=0x2) 1002 > > > vfio: vfio_pci_read_config(0000:04:00.0, @0xa, len=0x2) 300 > > > vfio: vfio_pci_read_config(0000:04:00.0, @0xe, len=0x1) 80 > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x0, len=0x2) 1002 > > > vfio: vfio_pci_read_config(0000:04:00.1, @0xa, len=0x2) 403 > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x0, len=0x2) 1002 > > > vfio: vfio_pci_read_config(0000:04:00.0, @0xa, len=0x2) 300 > > > vfio: vfio_pci_read_config(0000:04:00.0, @0xe, len=0x1) 80 > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x0, len=0x2) 1002 > > > vfio: vfio_pci_read_config(0000:04:00.1, @0xa, len=0x2) 403 > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x0, len=0x2) 1002 > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x0, len=0x4) 68181002 > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x8, len=0x4) 3000000 > > > vfio: vfio_pci_read_config(0000:04:00.0, @0xe, len=0x1) 80 > > > vfio: vfio_pci_read_config(0000:04:00.0, @0xe, len=0x1) 80 > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x0, len=0x2) 1002 > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x0, len=0x4) aab01002 > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x8, len=0x4) 4030000 > > > vfio: vfio_pci_read_config(0000:04:00.1, @0xe, len=0x1) 80 > > > vfio: SKIPPING region_add b0000000 - bfffffff > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x10, len=0x4) c000000c > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x10, 0xffffffff, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x10, len=0x4) f000000c > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x10, 0xc000000c, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x14, len=0x4) 0 > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x14, 0xffffffff, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x14, len=0x4) ffffffff > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x14, 0x0, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x18, len=0x4) fde80004 > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x18, 0xffffffff, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x18, len=0x4) fffc0004 > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x18, 0xfde80004, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x1c, len=0x4) 0 > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x1c, 0xffffffff, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x1c, len=0x4) ffffffff > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x1c, 0x0, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x20, len=0x4) ce01 > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x20, 0xffffffff, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x20, len=0x4) ffffff01 > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x20, 0xce01, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x24, len=0x4) 0 > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x24, 0xffffffff, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x24, len=0x4) 0 > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x24, 0x0, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x30, len=0x4) 0 > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x30, 0xfffff800, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x30, len=0x4) fffe0000 > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x30, 0x0, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x10, len=0x4) fdefc004 > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x10, 0xffffffff, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x10, len=0x4) ffffc004 > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x10, 0xfdefc004, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x14, len=0x4) 0 > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x14, 0xffffffff, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x14, len=0x4) ffffffff > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x14, 0x0, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x18, len=0x4) 0 > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x18, 0xffffffff, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x18, len=0x4) 0 > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x18, 0x0, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x1c, len=0x4) 0 > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x1c, 0xffffffff, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x1c, len=0x4) 0 > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x1c, 0x0, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x20, len=0x4) 0 > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x20, 0xffffffff, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x20, len=0x4) 0 > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x20, 0x0, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x24, len=0x4) 0 > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x24, 0xffffffff, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x24, len=0x4) 0 > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x24, 0x0, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x30, len=0x4) 0 > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x30, 0xfffff800, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x30, len=0x4) 0 > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x30, 0x0, len=0x4) > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x20, 0xc000, len=0x4) > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x18, 0xfea00000, len=0x4) > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x1c, 0x0, len=0x4) > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x30, 0xfea40000, len=0x4) > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x10, 0xfea60000, len=0x4) > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x14, 0x0, len=0x4) > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x10, 0xe0000000, len=0x4) > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x14, 0x0, len=0x4) > > > vfio: SKIPPING region_add feb40000 - feb4002f > > > vfio: SKIPPING region_add feb40800 - feb40807 > > > vfio: SKIPPING region_add feb41000 - feb4101f > > > vfio: SKIPPING region_add feb41800 - feb41807 > > > vfio: vfio_update_irq(0000:04:00.1) IRQ moved 20 -> 10 > > > vfio: vfio_disable_intx_kvm(0000:04:00.1) KVM INTx accel disabled > > > vfio: vfio_enable_intx_kvm(0000:04:00.1) KVM INTx accel enabled > > > vfio: vfio_update_irq(0000:04:00.0) IRQ moved 23 -> 11 > > > vfio: vfio_disable_intx_kvm(0000:04:00.0) KVM INTx accel disabled > > > vfio: vfio_enable_intx_kvm(0000:04:00.0) KVM INTx accel enabled > > > vfio: SKIPPING region_add feb42000 - feb42fff > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x3d, len=0x1) 1 > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x3c, 0xb, len=0x1) > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x4, len=0x2) 0 > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x4, 0x103, len=0x2) > > > vfio: region_add e0000000 - efffffff [0x7f8688000000] > > > vfio: region_add fea00000 - fea03fff [0x7f88aa7b8000] > > > vfio: SKIPPING region_add fea04000 - fea04fff > > > vfio: region_add fea05000 - fea3ffff [0x7f88aa7bd000] > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x3d, len=0x1) 2 > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x3c, 0xa, len=0x1) > > > vfio: vfio_pci_read_config(0000:04:00.1, @0x4, len=0x2) 0 > > > vfio: vfio_pci_write_config(0000:04:00.1, @0x4, 0x103, len=0x2) > > > vfio: region_add fea60000 - fea63fff [0x7f88bc710000] > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x4, len=0x2) 103 > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x4, len=0x2) 103 > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x4, 0x103, len=0x2) > > > vfio: region_del 0 - afffffff > > > vfio: region_add 0 - 9ffff [0x7f8698000000] > > > vfio: SKIPPING region_add a0000 - bffff > > > vfio: region_add c0000 - afffffff [0x7f86980c0000] > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x4, len=0x2) 103 > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x30, len=0x4) fea40000 > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x30, 0xfffffffe, len=0x4) > > > vfio: vfio_pci_read_config(0000:04:00.0, @0x30, len=0x4) fffe0000 > > > > Here the option ROM was sized > > > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x30, 0xfea40001, len=0x4) > > > > Then enabled > > > > > vfio: region_add fea40000 - fea5ffff [0x7f88a9e00000] > > > > Adding this memory region > > > > > vfio: vfio_pci_write_config(0000:04:00.0, @0x30, 0xfea40000, len=0x4) > > > vfio: region_del fea40000 - fea5ffff > > > > Then disabled, removing the memory region. Presumably between the > > enable and disable the contents were read and copied to 0xc0000, which > > is where the VGA BIOS is shadowed. > > > > > Here is the strace output from this failure: > > > > > > 1110 ioctl(14, KVM_RUN, 0) = 0 > > > 1110 pread(20, > > > 1099 <... poll resumed> ) = 1 ([{fd=0, revents=POLLIN}]) > > > 1099 futex(0x7ff73ca62fa0, FUTEX_WAIT_PRIVATE, 2, NULL > > > 1109 <... futex resumed> ) = -1 ETIMEDOUT (Connection timed out) > > > 1109 madvise(0x7ff72fe17000, 8368128, MADV_DONTNEED) = 0 > > > 1109 _exit(0) = ? > > > 1109 +++ exited with 0 +++ > > > > > > From reading the source 'hw/misc/vfio.c' it looks like the following > > > in 'vfio_vga_read' never finished: > > > > > > if (pread(vga->fd, &buf, size, offset) != size) { > > > error_report("%s(,0x%"HWADDR_PRIx", %d) failed: %m", > > > __func__, region->offset + addr, size); > > > return (uint64_t)-1; > > > } > > > > I agree. Every VGA access requires us to lock the VGA resources on the > > device, so if we can't get the lock, we stop making progress. I took a > > look at Xorg last night and it seems like it should be taking and > > releasing the VGA arbiter lock in a way that would be compatible with > > our use. That's in the xserver, not the actual display hardware driver, > > and it wraps access functions in the arbiter support, so should be > > transparent to the drivers. So for nouveau, it seems like it should > > work. For nvidia, we don't really know, it could be locking the device > > from the kernel module. > > > > You could instrument vga_get, vga_tryget, and vga_put to figure out > > what's happening. It might be enough to look at /dev/vga_arbiter at > > each step in the sequence to reproduce (sudo head > > --lines=1 /dev/vga_arbiter). Thanks, > > > > I've played a bit more with it and there are some differences in > behavior with 'nouveau' and 'nvidia' driver. As soon as I load the > binary driver I see the following: > > [ 18.628676] [drm] Initialized drm 1.1.0 20060810 > [ 18.668038] nvidia: module license 'NVIDIA' taints kernel. > [ 18.668107] Disabling lock debugging due to kernel taint > [ 18.676638] vgaarb: device changed decodes: PCI:0000:01:00.0,olddecodes=io+mem,decodes=none:owns=io+mem > [ 18.676722] vgaarb: transferring owner from PCI:0000:01:00.0 to PCI:0000:04:00.0 > [ 18.677007] [drm] Initialized nvidia-drm 0.0.0 20130102 for 0000:01:00.0 on minor 0 > [ 18.677090] NVRM: loading NVIDIA UNIX x86_64 Kernel Module 319.23 Thu May 16 19:36:02 PDT 2013 > > root@homer:~# head --lines=1 /dev/vga_arbiter > count:2,PCI:0000:01:00.0,decodes=io+mem,owns=io+mem,locks=none(0:0) > root@homer:~# modprobe nvidia > root@homer:~# head --lines=1 /dev/vga_arbiter > count:1,PCI:0000:01:00.0,decodes=none,owns=none,locks=io+mem(1:1) > root@homer:~# /usr/local/bin/qemu-system-x86_64 \ > -L /usr/local/share/qemu \ > -L /usr/local/share/qemu \ > -M q35 -enable-kvm -cpu host -smp cores=4,threads=1,sockets=1 \ > -m 8192 -rtc base=localtime -k de -nodefaults -vga none \ > -drive file=/home/mbroemme/.kvm/maggie.img,id=drive0,if=none,cache=none,aio=threads \ > -device virtio-blk-pci,drive=drive0,ioeventfd=on \ > -device ioh3420,id=pcie0,multifunction=on \ > -device vfio-pci,host=04:00.0,addr=0.0,bus=pcie0,multifunction=on,x-vga=on \ > -device vfio-pci,host=04:00.1,addr=0.1,bus=pcie0 -monitor stdio -nographic > root@homer:~# head --lines=1 /dev/vga_arbiter > count:1,PCI:0000:01:00.0,decodes=none,owns=none,locks=io+mem(1:1) > > It looks like nvidia binary driver locks some resources. :( Whereas > with nouveau driver the VGA arbitration starts first if I start the > VM with VGA passthrough: > > [ 178.187706] vfio-pci 0000:04:00.0: enabling device (0000 -> 0003) > [ 178.209599] vfio_ecap_init: 0000:04:00.0 hiding ecap 0x19@0x270 > [ 178.209631] vfio_ecap_init: 0000:04:00.0 hiding ecap 0x1b@0x2d0 > [ 181.198191] vgaarb: device changed decodes: PCI:0000:01:00.0,olddecodes=io+mem,decodes=none:owns=io+mem > [ 181.198208] vgaarb: transferring owner from PCI:0000:01:00.0 to PCI:0000:04:00.0 > > root@homer:~# head --lines=1 /dev/vga_arbiter > count:2,PCI:0000:01:00.0,decodes=io+mem,owns=io+mem,locks=none(0:0) > root@homer:~# /usr/local/bin/qemu-system-x86_64 \ > -L /usr/local/share/qemu \ > -L /usr/local/share/qemu \ > -M q35 -enable-kvm -cpu host -smp cores=4,threads=1,sockets=1 \ > -m 8192 -rtc base=localtime -k de -nodefaults -vga none \ > -drive file=/home/mbroemme/.kvm/maggie.img,id=drive0,if=none,cache=none,aio=threads \ > -device virtio-blk-pci,drive=drive0,ioeventfd=on \ > -device ioh3420,id=pcie0,multifunction=on \ > -device vfio-pci,host=04:00.0,addr=0.0,bus=pcie0,multifunction=on,x-vga=on \ > -device vfio-pci,host=04:00.1,addr=0.1,bus=pcie0 -monitor stdio -nographic > root@homer:~# head --lines=1 /dev/vga_arbiter > count:1,PCI:0000:01:00.0,decodes=none,owns=none,locks=none(0:0) > Alex you pointed me in the right direction, many thanks! I got it working with 'nvidia' binary driver now, but I had to patch it. I've attached it here just for reference if others want to try the same and I don't know if it is the proper way. After some testing (Xorg, DRI, VDPAU, switching between X and text-console) it looks like a stable workaround for me now until NVIDIA will fix their driver. --Maik diff -Nur NVIDIA-Linux-x86_64-319.23/kernel/nv-linux.h NVIDIA-Linux-x86_64-319.23-vfio-vgaarb-fix/kernel/nv-linux.h --- NVIDIA-Linux-x86_64-319.23/kernel/nv-linux.h 2013-05-17 04:00:02.000000000 +0200 +++ NVIDIA-Linux-x86_64-319.23-vfio-vgaarb-fix/kernel/nv-linux.h 2013-05-29 18:09:42.382925622 +0200 @@ -151,9 +151,9 @@ #error "struct file_operations compile test likely failed!" #endif -#if defined(CONFIG_VGA_ARB) -#include -#endif +//#if defined(CONFIG_VGA_ARB) +//#include +//#endif #if defined(NV_VM_INSERT_PAGE_PRESENT) #include diff -Nur NVIDIA-Linux-x86_64-319.23/kernel/nv.c NVIDIA-Linux-x86_64-319.23-vfio-vgaarb-fix/kernel/nv.c --- NVIDIA-Linux-x86_64-319.23/kernel/nv.c 2013-05-17 04:00:02.000000000 +0200 +++ NVIDIA-Linux-x86_64-319.23-vfio-vgaarb-fix/kernel/nv.c 2013-05-29 18:10:01.494277314 +0200 @@ -2914,12 +2914,12 @@ pci_set_master(dev); -#if defined(CONFIG_VGA_ARB) -#if defined(VGA_DEFAULT_DEVICE) - vga_tryget(VGA_DEFAULT_DEVICE, VGA_RSRC_LEGACY_MASK); -#endif - vga_set_legacy_decoding(dev, VGA_RSRC_NONE); -#endif +//#if defined(CONFIG_VGA_ARB) +//#if defined(VGA_DEFAULT_DEVICE) +// vga_tryget(VGA_DEFAULT_DEVICE, VGA_RSRC_LEGACY_MASK); +//#endif +// vga_set_legacy_decoding(dev, VGA_RSRC_NONE); +//#endif if (NV_IS_GVI_DEVICE(nv)) {